Data Engineer

Job description:

As a Data Engineer, you are developing and operating reliable data pipelines that enable timely access to high-quality data assets.

You are part of a dynamic team of technical experts, working hand-in-hand with subject matter experts, data product managers, semantic experts and data scientists to create data products that help Bayer to accelerate time-to-market, drive innovation and enhance efficiency across the whole pharma value chain.

YOUR TASKS AND RESPONSIBILITIES:

• Apply cloud and big-data engineering skills to build, test, and maintain data pipelines and data-product components using tools such as Spark, dbt, Airflow, or equivalent cloud-native services.
• Execute ingestion, transformation, and loading jobs that reliably move data from operational systems, IoT devices, or clinical sources into data platforms at agreed SLAs.
• Troubleshoot pipeline failures, data-quality anomalies, and performance bottlenecks, diagnosing root causes and implementing fixes without service disruption.
• Validate that implemented pipelines meet data contracts, schema standards, and access-control policies defined by data architects and governance teams.
• Optimize pipeline performance and cloud infrastructure costs by profiling job runtimes, partitioning strategies, and resource configurations.
• Document pipeline designs, data lineage, and operational runbooks in shared repositories accessible to data consumers and support teams.
• Coach less-experienced colleagues on data-engineering patterns, testing practices, and cloud-platform tooling used in the assigned domain.

WHO YOU ARE:

• Required -
• Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field, 6+ years of experience
• Proficiency in SQL, Python, PySpark, Spark, or another programming language, such as Java, Scala or R
• Proven experience building highly scalable, performant, resilient, and cost-optimized data pipelines
• Experience with modern data processing frameworks, such as Snowflake, Databricks, or AWS SageMaker Unified Studio
• Hands-on experience with core AWS services, including Amazon S3, IAM, Lake Formation, AWS Glue, Lambda, and related services required to build, operate, and secure end-to-end data solutions
• Strong understanding of relational databases and modern lakehouse table formats (Apache Iceberg, Delta Lake)
• Expertise in design, build, maintain and document conceptual, logical, and physical data models
• Sound understanding of data governance principles, access control, data cataloging, lineage, encryption, auditability, and compliance requirements. Ability to implement secure cloud environments and role-based access controls using IAM and Lake Formation.
• Preferred -
• Familiarity with the concept of Retrieval-Augmented Generation (RAG) data pipelines
• Familiarity with data architecture patterns, such as data hub, mesh, or fabric
• Experience with RESTful APIs and data integration
• Familiarity with the Data Product concept and hands-on implications
• Relevant certifications (e.g., GCP Certified, AWS Certified, Azure Certified)
• Demonstrated experience with Terraform for infrastructure automation, together with Git, GitHub, GitHub Actions, and CI/CD practices
• Experience with GxP and computerized system validation

Skills

AWS, AWS Glue, Amazon S3, IAM, Java, Lake Formation, Lambda, PySpark, Python, SQL, Scala, Spark