Senior Data Engineer

Responsibilities:

• Own the complete medallion architecture Bronze Silver Gold pipeline: land raw events (object storage), clean, conform and dedupe them (Silver), and model curated aggregates (Gold).
• Design the multi-tenant data model: data partitioned and isolated per tenant and per facility, with row-level tenancy and ensure that isolation holds across every layer.
• Author and operate Airflow DAGs for both incremental ingestion and historical backfill.
• Build and support the streaming ingestion path: API polling Kafka consumer Bronze and pair with the junior engineer who owns it.
• Build real-time / stream processing on top of the ingestion path: low-latency transformations, event-time windowing and aggregations over Kafka streams to power near-real-time analytics.
• Make pragmatic design decisions and document the patterns clearly for the team.

Requirements:

• 6+ years in data engineering, with senior-level ownership of production pipelines.
• Strong Python for data work.
• Hands-on Airflow with DAG authoring, dependencies, and backfills.
• Solid PostgreSQL: schema/data modelling and query performance.
• Experience with medallion / layered ELT patterns (raw, cleaned, curated).
• Comfortable with S3 / object storage and partitioning strategies.
• Working knowledge of Kafka consumers.
• Hands-on stream processing with Apache Flink (or similar) event-time semantics, windowing, and stateful streaming for real-time pipelines.
• Experience with a data lake/lakehouse (e. g. Delta Lake, Apache Iceberg) or a cloud data warehouse (e. g. Snowflake, BigQuery, Redshift).
• A real data-quality and reconciliation mindset you validate, you don't assume.
• Very good at problem solving.
• Tech stack: Python, Apache Airflow, Apache Flink, Apache Kafka, PostgreSQL, S3 / object storage, data lake/lakehouse (Delta Lake, Apache Iceberg), cloud data warehouse (Snowflake / BigQuery / Redshift), medallion / ELT architecture.

Nice to have:

• Multi-tenant data modelling and tenant-isolation experience.
• dbt or similar transformation tooling.
• Awareness of data-governance / GDPR-readiness.
• Experience mentoring junior engineers.