Senior Data Engineer - Databricks
First seen by Alion on Nov 4, 2026.
Required Qualifications :
- Minimum 7 years of professional experience in data engineering, data integration, ETL/ELT development, or related areas.
- Strong hands-on experience with Databricks, including Apache Spark, PySpark, SQL, Delta Lake, and job orchestration.
- Strong proficiency in Python, PySpark, and advanced SQL.
- Experience building production-grade data pipelines using medallion architecture.
- Good working knowledge of AWS, particularly Amazon S3, IAM, and cloud-based data integration and processing.
- Experience integrating data from enterprise databases and SaaS applications, with working knowledge of PostgreSQL, Couchbase, Salesforce, and Oracle NetSuite.
- Understanding of JDBC, REST APIs, CDC, incremental ingestion, and batch processing.
- Experience working with structured, semi-structured, and JSON-based data.
- Strong knowledge of data modeling, data transformation, and data quality frameworks.
- Experience with Databricks Unity Catalog and data governance fundamentals.
- Experience with Git, CI/CD, automated testing, and production deployment practices.
- Strong troubleshooting, performance tuning, and problem-tuning capabilities.
- Ability to independently design, develop, test, and deploy enterprise data pipelines.
Skills
Data Engineering, Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, AWS, ETL, Data Modeling