Data Architect - Databricks

Job Description :

We are looking for an experienced Data Architect to design and implement scalable, secure, and high-performance data architectures across enterprise environments. The role will focus on data platform architecture, data integration, data modeling, governance, and data quality, while enabling modern data platforms that are ready to support AI/ML and advanced analytics use cases.

The ideal candidate should have strong hands-on expertise in Azure Data Lake, Databricks, PySpark, SQL, Python, enterprise data integration, and data modeling, with experience in banking, financial services, or wealth management domains preferred.

Key Responsibilities :

Data Architecture & Platform Design :

- Design and implement scalable, high-performance enterprise data architectures.

- Define data platform strategies, reference architectures, technology standards, and design patterns.

- Design modern data platforms using Azure Data Lake, Databricks, and cloud-native technologies.

- Define architectures that support analytics, reporting, AI/ML, and advanced data science use cases.

- Ensure data platforms are scalable, secure, reliable, and optimized for performance and cost.

Data Integration :

- Lead enterprise-wide data integration initiatives across multiple applications, systems, and data sources.

- Design batch and near-real-time data ingestion and processing architectures.

- Develop and review data pipelines using Databricks, PySpark, Python, SQL, and related technologies.

- Define integration patterns for structured and unstructured data.

- Work closely with application, cloud, engineering, and analytics teams to establish seamless data flows.

Data Modeling :

- Design conceptual, logical, and physical data models based on business and analytical requirements.

- Develop and review 3NF, dimensional, and Data Vault modeling approaches.

- Define enterprise data structures, data domains, entities, relationships, and transformation rules.

- Ensure data models are optimized for analytical workloads and downstream AI/ML use cases.

- Maintain consistency and standardization of data models across platforms.

Data Governance & Quality :

- Lead data governance, data quality, metadata management, and data cataloguing initiatives.

- Define data ownership, stewardship, classification, lineage, and quality standards.

- Establish data quality rules, validation frameworks, monitoring, and remediation processes.

- Work with governance teams to ensure compliance with enterprise policies and regulatory requirements.

- Promote consistent data definitions and trusted enterprise data assets.

AI/ML-Ready Data Platforms :

- Design data architectures capable of supporting AI/ML, Generative AI, advanced analytics, and data science workloads.

- Ensure availability of high-quality, governed, and accessible data for AI/ML models.

- Collaborate with data scientists and ML engineers to define data requirements and feature/data pipelines.

- Evaluate emerging data technologies and recommend solutions that enable AI-first transformation.

Banking & Financial Services :

- Work with business and technology stakeholders to understand complex banking, finance, and wealth management data requirements.

- Design data solutions supporting financial products, customer data, transactions, portfolios, and reporting.

- Ensure appropriate security, governance, privacy, and compliance controls for sensitive financial data.

Required Technical Skills :

- Strong experience in Data Architecture and Enterprise Data Platforms.

- Hands-on experience with Databricks and Azure Data Lake.

- Strong proficiency in PySpark, Python, and SQL.

- Strong understanding of enterprise data integration and data pipeline architecture.

- Expertise in data modeling, including :

1. 3NF

2. Dimensional Modeling

3. Data Vault

- Strong understanding of data governance, data quality, metadata, lineage, and cataloguing.

- Experience designing scalable cloud-based data platforms.

- Strong understanding of distributed data processing and large-scale data environments.

- Ability to design data platforms for analytics and AI/ML workloads.

Preferred Skills :

- Azure Data Factory / Azure Synapse / Microsoft Fabric

- Databricks Lakehouse

- Delta Lake

- Data Vault 2.0

- Data Cataloguing & Metadata Management

- Data Quality Frameworks

- Enterprise Data Governance

- Data Lineage

- AI/ML Data Architecture

- Banking & Financial Services

- Wealth Management

- Cloud Security & Data Privacy

Skills

Data Engineering, Azure Databricks, Azure, Python, SQL, Data Architect, DataLake, Data Catalogue, PySpark