Senior Technical Lead

Job Summary

Job Summary

We are seeking an experienced Databricks Platform Administrator with 7+ years of overall IT experience and strong expertise in administering, securing, monitoring, and optimizing Databricks environments. The ideal candidate will be responsible for managing Databricks workspaces, clusters, security, governance, performance, and integrations across cloud platforms such as Azure, AWS, or GCP.

Enterprise references indicate strong demand for skills around Databricks Lakehouse, Unity Catalog, governance, monitoring, and cloud platform administration

Key Responsibilities

Key Responsibilities

• Administer and manage Databricks workspaces across Azure/AWS/GCP.
• Configure and maintain cluster policies, job scheduling, and resource management.
• Implement and manage Unity Catalog for data governance and access control.
• Setup RBAC, SSO, SCIM provisioning, and identity integrations with Azure AD/Entra ID or equivalent.
• Monitor platform usage, performance, costs, and system health.
• Perform platform upgrades, patching, and capacity planning.
• Configure networking, VNet injection, Private Link, firewall rules, and secure connectivity.
• Manage secrets, key vault integrations, and data security controls.
• Support CI/CD deployments using Azure DevOps, GitHub Actions, or Jenkins.
• Troubleshoot platform, notebook, cluster, and job execution issues.
• Create operational dashboards, alerts, and monitoring solutions.
• Ensure compliance with organizational security and governance standards

Skill Requirements

Required Skills

• 7+ years of IT experience with 3+ years in Databricks Administration.
• Strong experience with:
  • Databricks Workspace Administration
  • Cluster Management
  • Unity Catalog
  • Databricks SQL
  • Delta Lake
  • Job & Workflow Management
  • MLflow Administration
• Experience with cloud platforms:
  • Azure (Preferred)
  • AWS
  • GCP
• Good understanding of:
  • Python
  • PySpark
  • SQL
  • REST APIs
• Experience implementing platform security, governance, and monitoring.
• Knowledge of CI/CD tools such as Jenkins, GitHub, Azure DevOps.
• Understanding of Infrastructure as Code (Terraform preferred).

Other Requirements

Preferred Skills

• Databricks Certified Data Engineer Associate/Professional.
• Experience with FinOps and cloud cost optimization.
• Exposure to MLOps and AI platform administration.
• Knowledge of data governance, data lineage, and compliance frameworks.
• Experience supporting enterprise-scale analytics platform