DevOps + Databricks Engineer

DevOps + Databricks Engineer

Experience: 6+ Years

Location: Chennai & Pune

Work Mode: Hybrid - 3 days working from office

Budget: Up to 19 LPA

Payroll: Haparz

Notice Period: Immediate / Short Notice preferred

Mandatory: Strong DevOps + Databricks experience

Role Overview:

We are looking for an experienced DevOps + Databricks Engineer to build and manage secure, scalable, and production-ready cloud environments. The role will focus on AWS infrastructure, infrastructure automation, CI/CD, Databricks platform enablement, security, monitoring, and reliable deployment of data and AI workloads.

Key Responsibilities:

- Establish and manage development, testing, staging, and production-ready cloud environments on AWS.

- Build infrastructure-as-code solutions and automate environment provisioning using Terraform, CloudFormation, or equivalent tools.

- Design and maintain CI/CD pipelines for Python services, data pipelines, analytical workloads, and Databricks deployments.

- Implement deployment gates, release controls, environment promotion, rollback, and recovery procedures.

- Manage Databricks workspace enablement, Unity Catalog, access management, secrets, networking, and platform monitoring.

- Configure and maintain AWS infrastructure required for PRODIGY and related enterprise services.

- Implement IAM, enterprise authentication, secrets management, and appropriate access controls.

- Establish centralized logging, metrics, alerting, audit trails, and operational dashboards.

- Implement security scanning, dependency management, secrets protection, and compliance controls.

- Support performance optimization, availability, backup, disaster recovery, and production-readiness activities.

- Collaborate with developers, data engineers, ML engineers, and QA teams to improve build, testing, and deployment automation.

- Troubleshoot infrastructure, deployment, Databricks, networking, and production issues and drive them to resolution.

Candidate Profile:

- 6+ years of experience in DevOps, Cloud Engineering, Platform Engineering, or a closely related role.

- Strong hands-on experience with AWS and production cloud infrastructure.

- Strong hands-on experience with Databricks administration and deployment automation.

- Practical experience with Unity Catalog, Databricks access management, secrets, and workspace configuration.

- Strong Infrastructure-as-Code experience using Terraform, CloudFormation, or equivalent.

- Experience building and managing CI/CD pipelines using GitHub Actions, Azure DevOps, Jenkins, or similar platforms.

- Good knowledge of Docker and container orchestration.

- Strong understanding of cloud networking, IAM, monitoring, logging, security, and secrets management.

- Understanding of Python applications, data engineering pipelines, and ML/AI deployment workflows.

- Experience working with enterprise platforms where security, compliance, availability, and operational controls are important.

- Familiarity with Azure AD / Microsoft Entra ID and cross-cloud integrations is an advantage.

Skills

DevOps, Databricks, AWS, CI/CD, Terraform, Cloud Infrastructure, AWS Infrastructure, IT Infrastructure Automation, Production Support, Data Infrastructure