Data Engineer

Responsibilities:

• Own the complete medallion architecture Bronze, Silver, Gold pipeline: land raw events (object storage), clean, conform and dedupe them (Silver), and model curated aggregates (Gold).
• Design the multi-tenant data model: data partitioned and isolated per tenant and per facility, with row-level tenancy and ensure that isolation holds across every layer.
• Author and operate Airflow DAGs for both incremental ingestion and historical backfill.
• Build and support the streaming ingestion path: API polling, Kafka consumer Bronze and pair with the junior engineer who owns it.
• Build real-time / stream processing on top of the ingestion path: low-latency transformations, event-time windowing and aggregations over Kafka streams to power near-real-time analytics.
• Establish data-quality and reconciliation practices, including Bronze/Silver/Gold row-count and integrity checks.
• Make pragmatic design decisions and document the patterns clearly for the team.

Requirements:

• 3+ years in data engineering, with senior-level ownership of production pipelines.
• Strong Python for data work.
• Hands-on Airflow with DAG authoring, dependencies, backfills.
• Solid PostgreSQL: schema/data modelling and query performance.
• Experience with medallion / layered ELT patterns (raw, cleaned, curated).
• Comfortable with S3 / object storage and partitioning strategies.
• Working knowledge of Kafka consumers.
• Hands-on stream processing with Apache Flink (or similar) event-time semantics, windowing, and stateful streaming for real-time pipelines.
• Experience with a data lake/lakehouse (e. g., Delta Lake, Apache Iceberg) or a cloud data warehouse (e. g., Snowflake, BigQuery, Redshift).
• A real data-quality and reconciliation mindset: you validate; you don't assume.
• Very good at problem-solving.