Databricks Architect

The Databricks Architect will design, build, and govern scalable data platforms on the Databricks Lakehouse for enterprise healthcare and banking workloads. This is a hybrid role that blends hands-on architecture leadership with strategic guidance on data engineering, analytics, and regulatory-compliant data management. The architect will set standards for ingestion, governance, security, and performance while mentoring engineering teams and partnering with business, data science, and compliance stakeholders.

A fast-growing global computer software company delivering advanced data and AI solutions to enterprise clients. The organisation is known for its engineering-led culture, strong focus on modern data platforms, and commitment to building regulated, high-trust systems for its customers.

Responsibilities

• Architect end-to-end data solutions on the Databricks Lakehouse platform for enterprise healthcare and banking workloads.
• Define standards for data ingestion, transformation, governance, security, and performance optimisation.
• Lead and mentor data engineering teams, reviewing designs and code for quality and scalability.
• Collaborate with business stakeholders, data scientists, and compliance teams to translate requirements into technical blueprints.
• Establish cost management, cluster policies, and monitoring practices for the Databricks environment.

Requirements

• 12-18 years of experience in data engineering, data architecture, or enterprise data platforms.
• Strong expertise in Databricks, Apache Spark, Delta Lake, and Unity Catalog.
• Proven experience delivering solutions in the Healthcare and/or Banking domains.
• Solid knowledge of cloud platforms (Azure, AWS, or GCP) and data security and privacy practices.
• Experience with SQL, Python or Scala, and CI/CD for data pipelines.
• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• Nice to have: Databricks certifications such as Certified Data Engineer Professional or Solutions Architect.
• Nice to have: Familiarity with regulatory frameworks relevant to healthcare or banking data, and experience with streaming technologies and MLOps workflows.

Skills

Databricks, Apache Spark, Delta Lake, Unity Catalog, SQL, Python, Scala, CI/CD for data pipelines, Cloud platforms (Azure, AWS, or GCP), Data security and privacy practices, Databricks certifications (Certified Data Engineer Professional or Solutions Architect), Regulatory frameworks for healthcare or banking data, Streaming technologies, MLOps workflows