Databricks Platform Engineer/Administrator

Role Overview :

We are looking for an experienced Databricks Platform Engineer to manage, maintain, secure, and optimize enterprise-scale Databricks environments hosted on AWS. The ideal candidate will have strong expertise in Databricks administration, Unity Catalog, AWS infrastructure, Terraform automation, platform operations, and cost optimization.

Key Responsibilities :

- Administer Databricks workspaces, clusters, compute policies, jobs, and access controls.

- Manage Unity Catalog, data governance, permissions, and workspace security.

- Monitor platform health, utilization, performance, and capacity.

- Troubleshoot production incidents, perform root cause analysis, and restore services.

- Automate infrastructure provisioning and platform operations using Terraform, Databricks APIs, CLI, Python, and Shell scripting.

- Manage AWS services, networking, IAM, security controls, and monitoring.

- Analyze usage and implement cost optimization and FinOps practices.

- Support deployments, environment promotion, CI/CD integration, and release management.

- Maintain operational runbooks, SOPs, and technical documentation.

- Collaborate with Cloud, Security, Data Engineering, and Architecture teams to ensure platform reliability and compliance.

Mandatory Skills :

- Databricks Administration : Account and workspace administration, cluster management, compute policies, Unity Catalog, access governance.

- AWS Cloud : IAM, S3, VPC, KMS, DynamoDB, MSK, PrivateLink, CloudWatch.

- Infrastructure Automation : Terraform, IaC, Databricks REST APIs, CLI, Python, Shell scripting.

- Platform Operations : Production support, incident management, monitoring, alerting, troubleshooting, capacity planning.

- Cost Optimization : Usage analytics, cost monitoring, resource optimization, FinOps.

- Data Platform : SQL, ETL processes, data pipelines, workflow scheduling.

Candidate Requirements :

- 5+ years of experience in cloud platform engineering, administration, or data platform support.

- Minimum 3+ years of hands-on Databricks Administration experience in production environments.

- Strong experience managing Databricks on AWS, including Unity Catalog and platform governance.

- Proven expertise in Terraform and Databricks API/CLI automation.

- Strong troubleshooting, communication, documentation, and incident management skills.

- Bachelor's degree in Computer Science, IT, Engineering, or a related field.

Mandatory Certifications : Databricks Certification (Administrator/Data Engineer) and AWS Certification (Associate level or above).

Skills

Databricks, AWS, AWS Infrastructure, IAC Terraform, IT Infrastructure Automation, Data Governance, Data Catalogue, Python, Production Support