Confirmed on the employer's own hiring board on Nov 20, 2026. First seen by Alion on Nov 20, 2026.
Key Responsibilities:**
* Design, develop, and maintain robust and scalable ETL/ELT data pipelines using Azure Data Factory, Azure Databricks (PySpark, Scala, SQL), and other Azure data services.
* Build and optimize data models within data warehousing solutions (e.g., Azure Synapse Analytics) and data lakes (e.g., Azure Data Lake Storage Gen2).
* Build and deploy Apache Airflow pipelines on a Red Hat Linux environment using Kubernetes for container orchestration.
* Implement data quality checks, data validation, and error handling mechanisms to ensure data integrity and reliability.
* Manage and administer Azure data platform components, ensuring performance, security, and cost-effectiveness.
* Collaborate with Data Scientists, LLM Engineers, and Product Analysts to understand their data requirements and provide them with accessible, clean, and well-structured data.
* Develop and implement solutions for data ingestion from various sources (APIs, databases, streaming sources like Azure Event Hubs or Kafka).
* Optimize data storage and processing for performance and efficiency, leveraging partitioning, indexing, and appropriate file formats (e.g., Parquet, Delta Lake).
* Implement data governance and security best practices in compliance with organizational policies.
* Monitor data pipeline performance, troubleshoot issues, and implement proactive measures to ensure uptime and reliability.
* Automate data engineering workflows and processes using CI/CD practices and infrastructure-as-code (e.g., ARM templates, Bicep, Terraform).
* Stay current with Azure data services, open-source big data technologies, and industry best practices.
**Qualifications:**
* Bachelor's degree in Computer Science, Engineering, Information Technology, or a related technical field.
* 3-5+ years of hands-on experience in a Data Engineering role.
* Strong proficiency in SQL and at least one programming language like Python or Scala.
* Extensive experience with Azure data services, including Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage, and Azure Blob Storage.
* Hands-on experience with Apache Spark (PySpark or Scala) for large-scale data processing.
* Solid understanding of data warehousing concepts, data modeling techniques (dimensional, relational), and ETL/ELT design patterns.
* Experience with data pipeline orchestration and scheduling tools.
* Familiarity with version control systems (e.g., Git) and CI/CD pipelines.
* Knowledge of data security and data governance principles.
* Experience with big data technologies and file formats (e.g., Parquet, Avro, ORC, Delta Lake).
* Familiarity with streaming data technologies (e.g., Azure Event Hubs, Kafka, Spark Streaming) is a plus.
* Azure certifications (e.g., DP-203: Data Engineering on Microsoft Azure) are highly desirable.
* Strong problem-solving skills and attention to detail.

