Data Engineer
First seen by Alion on Aug 31, 2026.
About the Role:
We are looking for an experienced Data Engineer with strong Azure Data Engineering and AI capabilities to design, develop, and deploy scalable, production-ready data and AI solutions. The ideal candidate will have strong hands-on expertise in Azure Databricks, Python, and Azure Data Factory, along with practical experience in Azure AI Foundry, LLMs, Prompt Engineering, and AI Agent development.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines and data processing solutions using Azure Databricks, Python, and Azure Data Factory (ADF).
- Build and deploy AI applications and AI agents using Azure AI Foundry.
- Work with LLMs and Prompt Engineering to develop and integrate AI-powered solutions into data and business workflows.
- Implement scalable data architectures using Medallion Architecture and ensure reliable data ingestion, transformation, and processing.
- Develop and manage data solutions using Azure SQL Database, Databricks Apps, and related Azure services.
- Implement CI/CD pipelines and GitHub-based development and deployment practices.
- Ensure data and AI solutions meet requirements for scalability, security, governance, performance, and production readiness.
- Collaborate with Data Engineering, AI/ML, Architecture, and other technology teams to deliver enterprise data and AI solutions.
- Troubleshoot data pipelines, applications, and production issues and contribute to continuous improvement.
Required Skills & Experience:
- 5 - 9 years of overall experience in Data Engineering, Cloud Data Engineering, or related roles.
- 5+ years of hands-on experience with Azure Data Factory (ADF), Python, and Azure Databricks.
- Strong hands-on experience with Azure Databricks for data engineering and pipeline development.
- Strong programming and data-processing skills in Python.
- Strong experience with Azure Data Factory and enterprise data pipeline development.
- Hands-on experience with Azure AI Foundry and AI application development.
- Practical experience with LLMs, Prompt Engineering, and AI Agents.
- Experience with Azure SQL Database and Databricks Apps.
- Strong understanding of Medallion Architecture and modern data engineering practices.
- Experience with CI/CD and GitHub.
- Good understanding of data security, governance, scalability, and production deployment.
- Strong analytical, troubleshooting, communication, and problem-solving skills.
Good to Have:
- Experience with Databricks Mosaic AI.
- Knowledge of newer Databricks AI capabilities and AI/ML services.
- Experience integrating AI capabilities with enterprise data platforms.
Skills
Data Engineering, Azure Foundry, Artificial Intelligence, LLM, AI Integration, SQL, Azure Data Factory, Azure Databricks, Python, Data Pipeline, Medallion Architecture