Data Engineer

The Data Engineer is a hands-on technical lead responsible for designing, building, and supporting BioAgilytix's Enterprise Data Platform. This role develops scalable, validated, production-ready data pipelines, enterprise data models, and certified data products supporting laboratory operations, scientific analytics, sponsor reporting, regulatory compliance, and AI initiatives.

Working under the direction of the Data Management Lead, and collaborating with cross-functional stakeholders, this role implements modern data engineering practices including Snowflake, dbt, dimensional modeling, semantic modeling, CI/CD automation, DataOps and enterprise data governance standards.

Essential Responsibilities

• Design, develop, and support scalable enterprise data platforms that enable trusted, governed, and high-quality data products for analytics, scientific operations, sponsor reporting, regulatory compliance, and AI initiatives.
• Build and maintain production-grade ELT pipelines that integrate data from laboratory information systems (LIMS), ERP, CRM, APIs, sponsor systems, cloud applications, and other enterprise data sources.
• Develop modular, reusable data transformation frameworks using modern ELT practices, including automated testing, documentation, lineage, version control, and deployment automation.
• Develop and implement dimensional data models, semantic models and governed datasets based on the established enterprise business definitions across the organization.
• Develop validated, traceable, and auditable data pipelines supporting regulated laboratory operations, sponsor deliverables, and enterprise reporting while ensuring data integrity, reproducibility, lineage, and compliance with GxP, GLP, HIPAA, 21 CFR Part 11, and enterprise data governance standards.
• Implement automated data validation, reconciliation, data quality controls, audit logging, monitoring, observability, and operational alerting to ensure reliable and trusted enterprise data.
• Optimize the performance, scalability, security, governance, and operational efficiency of the enterprise data platform.
• Provide operational support for the enterprise data platform through production monitoring, incident resolution, root cause analysis, and continuous reliability improvements.
• Support sponsor-facing data delivery by building automated data harmonization, transformation, validation, lineage, and regulatory reporting processes.
• Develop certified, governed, and AI-ready data products that support enterprise analytics, machine learning, semantic search, and generative AI initiatives.
• Collaborate with Laboratory Operations, Quality teams, Information Technology, and business stakeholders to deliver scalable, reusable, and governed enterprise data solutions.

Additional Responsibilities

• Other duties as needed

Minimum Preferred Qualifications: Education/Experience

• Bachelor’s degree in computer science, Information Systems, Engineering, Mathematics, Data Science, or related field; Master's preferred.
• 5+ years of experience in Data Engineering, Data Management, Software Engineering, Business Intelligence, or related technical disciplines, preferably within life sciences, biotechnology, pharmaceuticals, CROs, healthcare, or other regulated industries.
• 3+ years of hands-on experience designing, developing, and supporting enterprise-scale data engineering solutions in production environments.
• 3+ years of hands-on experience architecting, developing, and administering enterprise solutions using Snowflake as a primary cloud data platform, including performance optimization, security, governance, workload management, and operational support.

Minimum Preferred Qualifications: Skills

• Strong hands-on experience with dbt Cloud or dbt Core for modular data transformation, automated testing, documentation, lineage, and deployment.
• Demonstrated expertise in enterprise dimensional data modeling, including star schemas, conformed dimensions, slowly changing dimensions, snapshot fact tables, and analytical data warehouse design.
• Experience designing semantic models, enterprise business vocabularies, ontology-driven data products, or knowledge graph concepts that enable consistent business definitions across enterprise analytics.
• Strong proficiency in SQL and Python for enterprise data engineering, automation, data transformation, and performance optimization.
• Experience developing enterprise data integration solutions using ETL/ELT platforms such as Talend, Fivetran, or equivalent technologies.
• Experience integrating enterprise applications using REST APIs, GraphQL APIs, file-based interfaces, Change Data Capture (CDC), and event-driven messaging platforms.
• Experience with a cloud platform including AWS (S3, Lambda, ECS, Glue) and/or Azure.
• Experience designing, developing, validating, and maintaining certified enterprise data products with documented business definitions, transformation logic, lineage, ownership, and lifecycle management.
• Experience implementing least-privilege security, role-based access control (RBAC), data masking, row-level security, encryption, secrets management, and secure data sharing.
• Experience supporting enterprise production data platforms, including incident management, root cause analysis, operational monitoring, performance tuning, release management, and platform reliability engineering.
• Experience working with Laboratory Information Management Systems, bioanalytical data, sponsor deliverables, and regulated laboratory environments is strongly preferred.
• Expert proficiency in SQL and Python.
• Snowflake architecture including Snowpark, Dynamic Tables, Streams, Tasks, data sharing, security, governance, workload management, and performance tuning.
• dbt Cloud/Core including models, snapshots, macros, tests, semantic models, documentation, lineage, and deployment automation.
• Enterprise ETL/ELT frameworks including Fivetran/Talend, APIs, CDC, and event-driven integrations.
• Enterprise data architecture, metadata-driven architecture, medallion architecture, and modern cloud data platform design patterns.
• Git, GitHub Actions, CI/CD, Infrastructure-as-Code, and DataOps practices.
• Automated testing, observability, reconciliation, data quality, lineage, and operational monitoring.
• Power BI, Sigma, Tableau, and semantic reporting platforms.
• Ability to independently deliver assigned complex data engineering solutions within established architecture, priorities, procedures, and technical standards.
• Ability to translate complex scientific, laboratory, and business requirements into scalable enterprise data models, semantic models, and certified data products based on established architectural and business standards.
• Strong analytical, troubleshooting, and optimization skills. Applies comprehensive data engineering knowledge and advanced analytical techniques to investigate complex issues, identify root causes, evaluate available information, and recommend appropriate solutions.
• Ability to develop validated, traceable, and auditable data solutions in accordance with established compliance, validation, quality, and scientific data-integrity requirements.
• Collaborate effectively across Scientific Operations, Quality Engineering, Quality Assurance, and IT, with the ability to communicate clearly with stakeholders at all levels of the organization.
• Demonstrated ability to work independently on complex data engineering assignments, use professional judgment to adapt established approaches, and escalate decisions affecting enterprise architecture, governance strategy, security policy, compliance strategy, or platform direction.
• Excellent communication and documentation skills.
• Able to navigate a fast-paced, evolving data landscape, demonstrating resilience and flexibility in the face of new challenges.
• Excellent written and spoken English language skills
• Excellent interpersonal and negotiating skills
• Strong presentation skills
• Excellent computer skills

Preferred Credentials

• Master’s degree

Supervisory Responsibility:

• No supervisory responsibilities

Supervision Received

• Reports to the Data Management Lead and works independently on complex engineering initiatives Infrequent supervision and instructions
• Frequently exercises discretionary authority

Physical Demands

• Ability to work in an upright and/or stationary position for up to eight (8) hours per day
• Repetitive hand movement of both hands with the ability to make fast, simple, repeated movements of the fingers, hands, and wrists to operate office equipment
• Occasional mobility needed
• Occasional crouching, stooping, with frequent bending and twisting of upper body and neck
• Light to moderate lifting and carrying (or otherwise moving) objects, including luggage and laptop computer, with a maximum lift of 20 pounds
• Ability to access and use a variety of computer software
• Ability to communicate information and ideas so others will understand, with the ability to listen to and understand information and ideas presented through spoken words and sentences
• Frequently interacts with others to obtain or relate information to diverse groups
• Works independently with little guidance or reliance on oral or written instructions and plans work schedules to meet goals; requires multiple periods of intense concentration
• Performs a wide range of variable tasks as dictated by variable demands and changing conditions with little predictability as to the occurrence
• Ability to perform under stress and multi-task
• Regular and consistent attendance

Position Type and Expected Hours of Work

• This is a full-time position
• Some flexibility in hours is allowed, but the employee must be available during the “core” work hours as published in the BioAgilytix Employee Handbook
• Occasional weekend, holiday, and evening work needed