Senior Data Engineer - ML Platform

A dedicated data engineer on the ML team, a greenfield mandate. Look for someone who has fed ML training pipelines, not just analytics dashboards.

The role

You sit on the ML team beside Machine Learning Engineers and own the data

infrastructure for model training, inference and monitoring. Today strong software engineers do the data work without lineage, pipeline observability or quality guarantees.

This hire changes that.

The data is structured claims and billing data plus unstructured clinical notes: terabytes, billions of rows, in a medallion lakehouse on GCP.

What you'll own

Data lake architecture: lay out a partly built bronze/silver/gold structure properly for

long-term ML use.

ML training pipelines: extract from siloed stores, transform, and feed structured and unstructured data into training.

Feature stores: build and own both offline and online stores, and know the trade-offs between them.

Observability and data quality: monitoring, alerting, validation and lineage from day one.

Kafka integration: work with Java teams to consume application events into the pipeline reliably.

Quality frameworks: automated checks and schema validation the ML team can trust.

Engineering design: lead architecture decisions and document trade-offs. As much a software role as a data role.

Requirements

5+ years of software development, 3+ focused on data engineering for data science or ML teams.

Built ML training pipelines and lake or lakehouse architecture, not just analytics pipelines.

Production-grade Python and SQL that other engineers can maintain.

Strong Spark, including optimization at terabyte scale (partitioning, broadcast joins, predicate pushdown).

Snowflake, Databricks, Delta Lake or equivalent.

Feature store experience (offline vs online, freshness, training-serving skew); ideally Feast or Tecton.

MLOps basics: lineage, versioning, quality monitoring, leakage prevention.

Kafka or another event-driven source in an ingestion pipeline.

Airflow or equivalent set up from scratch at production scale.

CI/CD, monitoring and alerting as standard practice.

Nice to have: unstructured text pipelines (clinical notes, documents)

· healthcare data such as HL7, FHIR or claims

· RAG or vector search in production · DVC or LakeFS

· GCP (Vertex AI, BigQuery, Cloud Composer, Pub/Sub)

· open-source or published work.