Lead Network Engineer

Job Description - Databricks Platform Administrator

Position

Databricks Platform Administrator

Location

Offshore / Remote

Experience

8+ Years (with 4+ years of hands-on Databricks administration experience)

Role Overview

We are seeking an experienced Databricks Platform Administrator to lead the setup, configuration, administration, and operational management of enterprise Databricks environments supporting large-scale Data Warehouse Transformation and Lakehouse modernization initiatives. The ideal candidate will be responsible for ensuring platform reliability, security, governance, performance optimization, and seamless integration with cloud and enterprise services.

Key Responsibilities

• Configure, administer, and maintain Databricks workspaces across Azure and/or AWS environments.
• Manage clusters, compute policies, job scheduling, workspace configurations, and platform upgrades.
• Implement and manage security controls including SSO, RBAC, IAM roles, service principals, network security, and data access governance.
• Configure and administer Unity Catalog, data access controls, lineage, and governance policies.
• Support CI/CD integration for Databricks notebooks, jobs, and workflows using Azure DevOps, GitHub, or similar tools.
• Monitor platform health, performance, capacity utilization, and costs; recommend optimization opportunities.
• Establish operational standards, platform monitoring, alerting, logging, and incident management processes.
• Collaborate with Data Engineers, Architects, DevOps teams, and business stakeholders to support platform adoption and onboarding.
• Troubleshoot platform, connectivity, authentication, and performance issues.
• Implement and maintain disaster recovery, backup, and environment management processes.
• Document platform architecture, standards, operational procedures, and best practices.

Required Skills

• Strong hands-on experience administering Databricks in Azure and/or AWS environments.
• Experience with Unity Catalog, Delta Lake, Databricks Workflows, Jobs, and Workspace Administration.
• Strong understanding of Spark ecosystem, Data Lakehouse architecture, and enterprise data platforms.
• Experience with cloud security, IAM, networking, VPNs, Private Endpoints/PrivateLink, and encryption.
• Knowledge of Azure AD, AWS IAM, SSO, RBAC, and enterprise identity management.
• Experience with Infrastructure as Code (Terraform preferred).
• Familiarity with Git, Azure DevOps, GitHub Actions, and CI/CD pipelines.
• Experience with monitoring and logging tools such as Azure Monitor, Datadog, Splunk, Grafana, or similar.
• Strong troubleshooting and operational support capabilities.

Preferred Qualifications

• Databricks Certified Professional or Associate Certification.
• Experience supporting enterprise-scale Data Warehouse or Lakehouse modernization programs.
• Knowledge of ADF, Kafka, Snowflake, MLflow, Power BI, and related data ecosystem tools.
• Experience working in regulated enterprise environments with governance and compliance requirements.

Key Attributes

• Self-starter capable of working independently with minimal supervision.
• Strong stakeholder communication and collaboration skills.
• Ability to operate across multiple projects and environments simultaneously.
• Focus on operational excellence, governance, security, and platform stability.