Senior C++ Software Engineer, Data Engines

Overview

Job Description

Responsibilities

• - Gain a deep understanding of Apache Arrow, the compute primitives in Theseus, query parsing, optimizers, and foundational components
• - Learn the architecture of the data processing engine, including component interactions and compute graph data flows
• - Master memory management mechanics, including memory spilling from GPU to Host and Disk
• - Embrace Voltron Data''s software development culture
• - Profile single-node and distributed query executions, analyze telemetry, and solve distributed engine issues
• - Study distributed relational algebra algorithms to identify areas for improvement
• - Work with the team on bug fixes, code optimizations, and refactoring
• - Build new relational algebra components to expand SQL/DataFrame functionality
• - Implement improvements in resource management, task scheduling, and fault tolerance
• - Propose and implement core architectural enhancements for the engine
• - Work on complex projects such as language-agnostic user-defined functions, multi-query concurrency, and multi-tenancy
• - Integrate the engine with other company-developed components to deliver enterprise-grade solutions

Required Skills

• - Expertise with data processing engines or frameworks
• - Experience in distributed High-Performance Computing (HPC) systems
• - Hands-on knowledge of GPU and CPU resource management, including memory and thread pools
• - Several years of experience developing in C++
• - Experience developing for multiple hardware types (CPU, GPU)
• - Familiarity with UCX or libfabric
• - Experience working with open-source projects such as Apache Arrow, Flight, Rapids, Velox, and Data Fusion
• - Deep understanding of distributed and multi-threaded systems
• - Proficiency in SQL and non-SQL systems with their computational abstractions

Benefits

• - Work from Anywhere - Payroll and Benefits in 150+ Countries
• - Unlimited PTO
• - Medical, Dental, and Vision
• - Retirement [USA Only]
• - Home Office Budget
• - Continuing Education Budget