Senior Data Engineer (Platform)

We are looking for a Senior Data Engineer to build and evolve the data platform powering our global workforce management ecosystem. You will design, implement, and maintain scalable data pipelines that consolidate data from multiple operational systems, transform it into trusted analytical datasets, and make it available for reporting, product analytics, and business intelligence.

You should be comfortable working with modern cloud-native data architectures on AWS, building reliable ETL/ELT pipelines, and designing data models optimized for analytical workloads. This role requires a strong engineering mindset, balancing performance, scalability, data quality, and operational excellence while collaborating closely with software engineers, product teams, analysts, and data scientists.

What You Will Own

• Design, build, and maintain scalable batch and streaming data pipelines using AWS-native services and distributed processing frameworks
• Develop ETL/ELT workflows to ingest, consolidate, sanitize, enrich, and transform data from multiple internal and external systems
• Build and optimize AWS Data Lake solutions using Amazon S3, AWS Glue, Amazon Redshift, and Amazon Kinesis Firehose
• Design and implement distributed data processing jobs using Apache Spark, AWS Glue, Databricks, or equivalent technologies
• Develop orchestration workflows using Apache Airflow (MWAA), AWS Step Functions, or similar workflow orchestration platforms
• Design analytical data models including star schemas, snowflake schemas, dimensional models, and optimized reporting datasets
• Optimize Redshift performance through distribution strategies, sort keys, partitioning, workload tuning, and query optimization
• Build resilient pipelines supporting retries, idempotency, checkpointing, incremental processing, and partial failure recovery
• Implement automated data quality validation, schema evolution, lineage tracking, and governance controls
• Develop infrastructure and deployment automation using Infrastructure as Code and CI/CD pipelines
• Monitor, troubleshoot, and continuously improve the reliability, scalability, and performance of the data platform
• Collaborate with analysts, software engineers, data scientists, and product managers to translate business requirements into scalable data solutions
• Participate in architecture discussions and contribute technical documentation, standards, and best practices

What We Are Looking For

• 5+ years of professional experience building production data pipelines and cloud-based data platforms
• Strong experience with AWS data services including Amazon Redshift, AWS Glue, Amazon S3, and Amazon Kinesis Firehose
• Strong Python programming skills for ETL development, automation, event processing, and scripting
• Advanced SQL expertise including query optimization, window functions, analytical queries, versioned migrations, rollback strategies, and warehouse tuning
• Experience designing scalable ETL/ELT pipelines for both batch and streaming workloads
• Experience with distributed compute and storage using Apache Spark, AWS Glue, Databricks, or similar distributed processing frameworks
• Strong understanding of data warehousing concepts including dimensional modeling, star schemas, snowflake schemas, partitioning strategies, and analytical data structures
• Experience designing end-to-end data architectures including ingestion, transformation, orchestration, and consumption layers
• Experience implementing workflow orchestration using Apache Airflow (MWAA), AWS Step Functions, or equivalent orchestration tools
• Understanding of data governance, metadata management, security best practices, IAM, encryption, and regulatory compliance considerations
• Experience with Git-based collaborative development workflows, CI/CD pipelines, Infrastructure as Code, deployment approvals, versioned migrations, and safe rollback strategies
• Experience monitoring and maintaining production data infrastructure, ensuring high availability, observability, data quality, and operational reliability
• Strong communication skills with the ability to explain technical concepts to business stakeholders and collaborate effectively across engineering, analytics, and product teams

Nice to Have

• Experience with Apache Iceberg, Delta Lake, Apache Hudi, or modern open table formats
• Experience with dbt or SQL-based transformation frameworks
• Familiarity with Kafka, Amazon MSK, or other streaming platforms
• Experience with Lakehouse architectures and modern analytical data platforms
• Knowledge of Terraform or AWS CloudFormation
• Experience with containerized data workloads using Docker and ECS/EKS
• Experience implementing DataOps practices and automated testing for data pipelines
• Familiarity with BI platforms such as Tableau, Power BI, Looker, or QuickSight
• Experience implementing data catalogs, lineage, and governance solutions
• Exposure to machine learning feature pipelines or data science infrastructure

Tech Stack

Layer

Technology

Programming

Python, SQL, PySpark

Data Processing

Apache Spark, AWS Glue, Databricks

Data Storage

Amazon S3, Amazon Redshift, Parquet

Streaming

Amazon Kinesis Firehose, EventBridge

Orchestration

Apache Airflow (MWAA), AWS Step Functions

Data Modeling

Star Schema, Snowflake Schema, Dimensional Modeling

Infrastructure

AWS, IAM, CloudWatch

IaC/CI

Git, GitHub Actions, Terraform, CloudFormation

Observability

CloudWatch, Datadog (or equivalent observability platforms)

Governance

Data Catalog, Metadata Management, Data Lineage