Senior Data Engineer
Role Overview
We are seeking a skilled Data Engineer to design, build, and maintain scalable data platforms and pipelines across AWS and Azure. The role will support analytics, AI/ML, and business intelligence use cases using modern data technologies including Databricks, Snowflake, SageMaker, and Amazon Bedrock. The ideal candidate has strong cloud data engineering experience and collaborates closely with data scientists, analytics teams, and business stakeholders.
Key Responsibilities
Data Platform & Pipeline Development
Design, develop, and optimize scalable, secure, and reliable data pipelines using batch and streaming patterns.
Build and maintain ETL/ELT workflows ingesting data from structured and unstructured sources.
Implement data transformations and orchestration using Databricks (Spark), SQL, and Python.
Develop and maintain data models optimized for analytics and reporting.
Cloud & Data Architecture
Implement data solutions on AWS and Azure leveraging native services.
Design and manage cloud storage solutions (e.g., S3, ADLS Gen2).
Build cloud-agnostic or hybrid architectures supporting multi-cloud strategies when required.
Optimize performance, scalability, reliability, and cost across platforms.
Analytics & Data Warehousing
Design and support Snowflake data warehouse solutions, including schema design, performance tuning, and cost management.
Enable self-service analytics and BI use cases through curated, high-quality datasets.
Partner with reporting and analytics teams to ensure data availability and accuracy.
AI / ML & Advanced Analytics Enablement
Support ML workflows by preparing and engineering features for SageMaker and Amazon Bedrock use cases.
Collaborate with data scientists to operationalize ML models (MLOps integration).
Enable GenAI and advanced analytics workloads through secure, governed data access.
Data Quality, Security & Governance
Implement data quality checks, monitoring, and validation frameworks.
Ensure compliance with security, privacy, and regulatory requirements.
Apply data governance standards including lineage, metadata management, and access controls.
DevOps & Automation
Implement CI/CD pipelines for data solutions using infrastructure-as-code (Terraform, ARM, CloudFormation).
Automate deployments, monitoring, logging, and alerting.
Participate in on-call or support rotations as needed.
Collaboration & Stakeholder Engagement
Work closely with product owners, business stakeholders, architects, and analytics teams.
Translate business requirements into scalable technical solutions.
Contribute to documentation, standards, and best practices across the data engineering team.
Required Skills & Experience
Technical Skills
Strong experience with Python and SQL.
Hands-on experience with Databricks (Apache Spark).
Proven experience building solutions on AWS and/or Azure.
Experience with Snowflake data warehouse.
Experience supporting ML or AI workloads using SageMaker and Amazon Bedrock (or equivalent services).
Knowledge of data integration tools, APIs, and message/streaming platforms.
Understanding of data modeling principles and analytics use cases.
Familiarity with DevOps concepts, CI/CD, and IaC.
Soft Skills
Strong problem-solving and analytical skills.
Ability to communicate technical concepts to non-technical audiences.
Collaborative mindset with experience working in agile teams.
Attention to detail with a focus on data accuracy and reliability.
Education Requirements
Bachelor’s degree in Computer Science, Engineering, Information Systems, Data Science, or a related field.
Master’s degree is a plus but not required.
Experience Requirements
5+ years of experience in data engineering or related roles.
3+ years of hands-on experience working with cloud data platforms (AWS and/or Azure).
Experience supporting enterprise-scale data, analytics, and AI solutions.
Preferred Qualifications
Cloud certifications (AWS, Azure, Databricks, Snowflake).
Experience in healthcare, life sciences, finance, or other regulated industries.
Exposure to real-time or streaming data architectures.
Experience with data governance, metadata tools, and privacy frameworks.