Senior Data Engineer
First seen by Alion on Sep 2, 2026.
We are looking for an experienced and driven Senior Data Engineer with 6+ years of hands-on experience designing, building, and scaling robust data pipelines.
The ideal candidate will combine deep technical expertise in PySpark, SQL, and Python with strong business acumen, acting as a crucial bridge between complex data infrastructure and our business stakeholders.
Experience with platforms like Dataiku (DataIQ) is highly preferred to support rapid pipeline development and cross-functional collaboration.
Key Responsibilities :
- Pipeline Architecture & Development : Design, build, and optimize scalable ETL/ELT data pipelines using Python, PySpark, and modern orchestration tools.
- Dataiku (DataIQ) Integration : Leverage visual and code-based recipes within Dataiku to rapidly prototype, deploy, and monitor end-to-end data workflows and machine learning pipelines.
- Database Optimization : Write, tune, and maintain highly complex SQL queries and data models to ensure high-performance analytical reporting.
- Stakeholder Collaboration : Partner directly with business units, product managers, and leadership to understand data requirements, translate them into technical specifications, and present insights clearly.
- Data Governance & Quality : Ensure data integrity, reliability, and security across all pipelines through robust testing, monitoring, and best-practice implementation.
- Mentorship : Guide junior and mid-level engineers on best practices for coding, optimization, and scalable data architecture.
Required Qualifications & Skills :
- Experience : 6+ years of professional experience in data engineering, data warehousing, or software engineering focused on data systems.
- Big Data Technologies : Advanced proficiency in PySpark and distributed computing frameworks.
- Database Expertise : Exceptional SQL skills, including performance tuning, complex window functions, and schema design for modern cloud data warehouses.
- Platform Preference : Hands-on experience building pipelines using code or Dataiku (DataIQ).
- Programming : Strong software development fundamentals in Python.
- Communication : Exceptional verbal and written communication skills with a proven track record of interacting with non-technical business stakeholders, gathering requirements, and presenting technical concepts clearly.
Preferred Qualifications :
- Experience with cloud platforms (AWS, Azure, or GCP).
- Familiarity with workflow orchestrators like Apache Airflow or dbt.
- Prior exposure to agile development environments and cross-functional teams.
Skills
Data Engineering, PySpark, SQL, Python, Data Pipeline, Dataiku, ETL, Big Data, AWS, Data Governance