Platform Engineer (Databricks Administrator)

This senior role is responsible for designing, implementing and evolving enterprise-scale Databricks and Azure data and analytics platforms that are secure, resilient, scalable and aligned to business and regulatory requirements. The successful candidate will provide technical leadership across workspace design, cluster strategy, storage architecture, security controls and CI/CD integration. The role also involves ongoing platform stewardship, operational excellence and knowledge transfer to internal engineering teams.

A leading global technology and analytics firm in the computer software industry, serving enterprise clients worldwide with data-driven solutions. The organisation operates at significant scale and is known for a collaborative, innovation-led culture that values engineering excellence and long-term platform sustainability.

Responsibilities

• Provide senior Databricks and Azure platform expertise to design, implement and evolve enterprise-scale data and analytics platforms.
• Lead technical architecture for the Databricks platform, including workspace design, cluster strategy, data storage architecture and environment management.
• Ensure platform resilience, performance and operational excellence through proactive monitoring, capacity planning, cost optimisation and performance tuning across development, test and production environments.
• Lead incident management and root cause analysis to resolve complex technical issues and improve platform reliability.
• Lead implementation of platform governance and security controls, including access management, Entra ID integration, RBAC, network isolation and private connectivity.
• Ensure auditability, data protection and compliance with enterprise architecture and regulatory standards.
• Integrate CI/CD pipelines and platform engineering best practices to support current and future business demand.
• Establish operational standards and provide ongoing technical stewardship alongside internal engineering teams.
• Drive knowledge transfer and long-term platform sustainability.

Requirements

• 8+ years of experience in data platform engineering or administration, with deep hands-on expertise in Azure Databricks.
• Strong experience with ADLS Gen2, Azure networking and cloud security patterns.
• Solid understanding of identity management, including Entra ID and RBAC.
• Proven experience with CI/CD integration and platform engineering practices.
• Demonstrated ability in capacity planning, cost optimisation and performance tuning.
• Experience leading platform architecture decisions and governing security controls in enterprise environments.
• Strong troubleshooting, incident management and root cause analysis skills.
• Excellent communication skills with the ability to mentor and collaborate with engineering teams.

Skills

Azure Databricks, ADLS Gen2, Azure networking, Identity management, Cloud security patterns, Entra ID, RBAC, CI/CD integration, Platform engineering, Capacity planning, Cost optimisation, Performance tuning