Databricks Architect - Artificial Intelligence
First seen by Alion on Jul 28, 2026.
Job Description :
Technical Skills :
- Deep expertise in Databricks platform (Unity Catalog, Delta Lake, MLflow, Workflows).
- Strong proficiency in PySpark, Scala, or SQL for large-scale data processing.
- Experience designing and building medallion architecture (Bronze/Silver/Gold).
- Hands-on with cloud platforms, preferably Azure.
- Familiarity with CI/CD pipelines and DevOps practices for data engineering.
Architecture & Design :
- Ability to design and implement modern data platforms using Databricks Lakehouse architecture end-to-end.
- Define data architecture, data models, and governance frameworks tailored for enterprise analytics platforms.
- Experience migrating legacy data warehouses (Teradata, Snowflake, etc.) to Databricks.
- Ability to design scalable, cost-optimized data lakehouse solutions.
- Can produce architecture diagrams, technical documentation, and lead design reviews.
Data Modeling & Legacy-to-Cloud Transition :
- Strong data modeling background with hands-on experience in legacy systems such as SQL Server and Oracle.
- Proven track record of transitioning from traditional relational/on-prem data models to modern cloud-native architectures.
- Ability to re-platform legacy schemas, stored procedures, and ETL pipelines into Databricks lakehouse patterns.
- Experience translating dimensional models (star/snowflake schemas) into lakehouse-optimized Delta Lake designs.
- Comfortable bridging the gap between legacy DBA/data modeler roles and modern cloud architecture responsibilities.
Data Governance, Security & Compliance :
- Strong understanding of metadata management, data lineage, data quality, and governance frameworks.
- Hands-on with Databricks Unity Catalog for centralized governance, access control, and lineage tracking.
- Experience enforcing data security and compliance standards, particularly within Financial Services (SOX, PCI-DSS, GDPR).
- Ability to implement role-based access control (RBAC), column/row-level security, and data masking.
- Familiarity with data quality frameworks and tooling integrated within the Databricks ecosystem.
AI & Machine Learning Features (Databricks-Native) :
- Hands-on experience with Databricks Mosaic AI for building, training, and deploying ML models.
- Proficiency with MLflow (native to Databricks) for experiment tracking, model registry, and serving.
- Experience with Databricks Feature Store for creating and managing reusable ML features.
- Ability to build GenAI applications using Databricks Vector Search and Mosaic AI Model Serving.
- Familiarity with Databricks AI/BI and the Genie interface for natural language analytics.
Soft Skills & Collaboration :
- Strong communication skills for working across time zones with onshore teams.
- Able to lead technical discussions, produce architecture diagrams, and document decisions.
- Experience mentoring junior engineers.
Engagement Expectations :
- Willingness to participate in design reviews, sprint planning, and stakeholder calls.
- Deliverable-driven with ability to work independently.
Good to have :
- Good understanding of Agent bricks and Databricks Generative AI capabilities.
Education :
- UG: B.Tech / B.E. in Any Specialization.
Key Skills :
- Skills highlighted with are preferred keyskills: Architect, Azure, Databricks, Artificial Intelligence, SQL, Unity Catalog.
Skills
Azure Databricks, Artificial Intelligence, Python, SQL, ETL, PySpark, Scala, Data Catalogue