Confirmed on the employer's own hiring board on Nov 20, 2026. First seen by Alion on Nov 18, 2026. LTM scores B on the Alion truth index.
Experience 11to15 years
RoleTechnical Lead
Role Summary
We are looking for an experienced Databricks Technical Lead with 11 to15 years of overall experience and strong expertise in designing and delivering enterprisescale data engineering solutions The candidate should have handson experience with Databricks Lakehouse architecture Delta Lake PySpark Unity Catalog and cloudnative data services with the ability to build secure scalable and productionready data platforms
Primary Skills
Databricks
PySpark Spark
Delta Lake and Lakehouse Architecture
Bronze Silver and Gold data architecture
Databricks Workflows
Unity Catalog
Databricks Serverless Compute
SQL and Data Modeling
Cloud Data Engineering preferably AzureAWS
Data Pipeline Development and Optimization
Data Quality Monitoring and Observability
CICD and DevOps for Data Engineering
Key Responsibilities
Design develop test deploy and support enterprisescale data products data pipelines and integration solutions using Databricks and cloudnative technologies
Build and maintain Bronze Silver and Gold data layers within a Lakehouse architecture using Delta Lake PySpark Databricks Workflows Unity Catalog and serverless compute
Develop scalable data transformation validation and enrichment pipelines supporting analytics reporting operational and AIML workloads
Implement robust data quality monitoring reconciliation and observability controls to ensure trusted and reliable data products
Design and maintain enterprise data models business rules and reusable data engineering frameworks aligned with architectural standards
Work closely with solutiondata architects product owners business stakeholders and engineering teams to translate requirements into scalable technical solutions
Conduct and participate in technical design reviews code reviews unitintegration testing and CICD processes
Troubleshoot complex pipeline platform and performance issues and optimize Spark workloads for performance reliability and cost
Implement data governance security lineage and rolebased access controls using Unity Catalog and cloudnative security capabilities
Automate deployment testing dataquality validation monitoring and operationalsupport activities
Create and maintain technical designs data mappings operational procedures engineering standards and support documentation
Mentor engineers and provide technical leadership around Databricks Spark data engineering and software engineering best practices
Continuously evaluate newer Databricks and cloud capabilities to improve scalability automation performance and maintainability
Secondary Skills
Cloud Microsoft Azure or AWS
Azure ADLS Gen2 Azure Data Factory Key Vault Entra ID
AWS S3 IAM Glue and related data services
Programming Python SQL
DevOps Azure DevOps GitHub Actions Jenkins
Infrastructure as Code Terraform
Data Modeling Dimensional modeling StarSnowflake schemas
Streaming Apache Kafka Spark Structured Streaming
Orchestration Databricks Workflows ADF or Airflow
Data Governance Data lineage RBAC data classification and metadata management
Engineering Git automated testing CICD and AgileScrum
Exposure to MLflow Databricks SQL and AIML workloads would be advantageous
Qualifying Questions
1 Databricks Architecture
Have you designed and implemented an enterprise Databricks Lakehouse solution using Bronze Silver and Gold layers Please briefly explain how you used Delta Lake PySpark and Unity Catalog in the solution
Qualify if Candidate demonstrates handson ownership of Medallion architecture Delta tables transformation pipelines and Unity Catalog rather than only theoretical knowledge
2 Performance Production Troubleshooting
Describe a production DatabricksSpark pipeline performance issue you resolved What was the root cause and what specific optimization techniques did you use
Qualify if Candidate can discuss practical techniques such as partitioning shuffle optimization skew handling Spark execution plans join strategies caching file sizingcompaction Photon or clusterserverless optimization
3 Governance CICD
How have you implemented enterprise data governance and automated deployment in Databricks using Unity Catalog and CICD
Qualify if Candidate demonstrates experience with catalogsschemas RBAC or privileges lineage service principalsmanaged identities environment promotion Gitbased version control and automated deployment

