Member of Technical Staff - ML Systems

Job Description:

• Build CUDA kernels and compute primitives for training and serving graph neural networks (GNNs)
• Evolve Monad, the sampler and distributed-training library, including neighbor sampling and training performance
• Specify binary data formats (Lance, Arrow, CSR/CSC), and own materializations and feature backfills for training and evaluation
• Define data contracts and consumption requirements with teams building customer and proprietary datasets
• Build and operate experiment tracking, checkpoints, and evaluation infrastructure with reproducibility by default
• Own the model registry, lineage, versioning, and compatibility across models, embeddings, and downstream models
• Define and run release gates for governed production, batch, and on-premise model releases
• Enable auditing of the data, code, configuration, and evidence that produced each release
• Improve time-to-experiment, time-to-governed-release, GPU training throughput, release records, lineage, and reproducibility

Requirements:

• Strong systems engineering skills and production-quality Python
• Experience with distributed training (e.g., Ray, PyTorch distributed) and multi-node GPU workloads
• Experience with columnar data formats and large-scale data materialization
• Familiarity with ML lifecycle tooling: experiment tracking, model registries, evaluation, and reproducibility
• A product mindset: treat an internal platform as a product with real users
• CUDA kernel development or GPU performance optimization (nice to have)
• Graph neural networks or graph sampling at scale (nice to have)
• Lance, Arrow, or other columnar/indexed storage formats (nice to have)
• Multi-cloud GPU compute (e.g., SkyPilot) (nice to have)
• Model governance or audit requirements in financial services or other regulated environments (nice to have)

Benefits: