{"id":4309723,"url":"https://alion.io/job/newvision-software-consultancy-azure-databricks-platform-engineer","title":"Azure Databricks Platform Engineer","company":{"id":4489094,"name":"NewVision Software & Consultancy","domain":"newvision-software.com","url":"https://alion.io/company/newvision-software-and-consultancy","size_band":"501-1000","is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Career site","truth_index":null},"role":"DevOps","role_family":"DevOps","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Pune, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Platform Engineering","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Terraform","optional":false},{"name":"Bicep","optional":true}],"status":"live","first_seen_at":"2026-11-06T00:00:00Z","employer_posted_date":"2026-11-14","last_verified_at":"2026-11-14T11:21:27Z","board_verified":true,"closed_at":null,"days_open":8,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":8},"description":"Senior Azure Databricks Platform Engineer\nAbout the role:\nWe are looking for a Senior Azure Databricks Platform Engineer with strong hands-on experience in both Databricks development and platform operations. The candidate should be able to manage and support all the activities related to Databricks platform in conjunction with Azure cloud single handedly. Workspace creation, maintenance, deployment using DEVOPS CICD is integral for the role.\nResponsibilities\nDesign, develop, and optimize data pipelines using Azure Databricks, Python, PySpark, Spark SQL, and Delta Lake.\nDevelop and maintain Databricks notebooks, workflows, and scheduled jobs.\nConfigure and manage Databricks clusters, cluster policies, autoscaling, runtime versions, and libraries.\nSupport Databricks workspace configuration, infrastructure management, and platform operations.\nImplement Unity Catalog access controls, data governance, and data lineage.\nSupport Azure security, Key Vault integration, and secrets management.\nImplement CI/CD pipelines and automate Databricks deployments.\nMonitor and troubleshoot Databricks jobs, clusters, connectivity, and production issues.\nOptimize Spark performance, compute resources, and platform costs.\nCollaborate with Data Engineering, DevOps, and Infrastructure teams.\nMust have Requirements\n5+ years of experience in Platform Engineering on Azure data platforms, including at least 3 years with Azure Databricks platform management.\nThe ideal candidate should have a good end-to-end understanding of Databricks, from developing Python/PySpark solutions to configuring clusters, managing jobs, and supporting production environments. Workspace management, cluster management, Unity catalog implementation.\nStrong Python fundamentals, including data structures, functions, error handling, and problem-solving.\nStrong hands-on experience with Azure Databricks, PySpark, Spark SQL, SQL, ADLS Gen2, and Delta Lake.\nPractical experience developing Databricks notebooks, workflows, and job scheduling.\nHands-on experience with Databricks cluster management, autoscaling, runtime configuration, and cluster policies.\nExperience with Unity Catalog, access management, and data lineage implementation and monitoring.\nGood understanding of Databricks infrastructure, workspace configuration, and Azure cloud services.\nExperience with CI/CD pipelines, deployment automation, and DevOps practices.\nKnowledge of Azure networking, security, Key Vault, and secrets management.\nExperience troubleshooting production Databricks jobs, clusters, and platform issues.\nExperience with Terraform for provisioning, configuring, and managing Azure Databricks infrastructure, including workspaces, clusters, Unity Catalog, and related Azure resources.\nGood understanding of Spark architecture and performance optimization.\nStrong communication and problem-solving skills.\nNice to Have\nExperience with Bicep, ARM Templates, or other Infrastructure as Code tools.\nExperience with Databricks Asset Bundles (DABs).\nExperience with Azure Monitor, Log Analytics, and cost optimization.\nExperience provisioning Databricks workspaces from scratch.\nExperience in banking, insurance, or other regulated industries.\nAzure or Databricks certifications.","description_format":"text","description_chars":3234,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-11-14T11:21:27Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":8,"expected_fill_days":41,"reasons":["conf:2","win:early","comp:brand"],"computed_at":"2026-11-14T13:32:30Z"},"pay":null,"html_url":"https://alion.io/job/newvision-software-consultancy-azure-databricks-platform-engineer","json_url":"https://alion.io/job/newvision-software-consultancy-azure-databricks-platform-engineer.json","meta":{"generated_at":"2026-11-14T13:32:30Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":13848,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-15T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4489094},"rest":"https://alion.io/mcp/rest/get_company?id=4489094"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fnewvision-software-consultancy-azure-databricks-platform-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fnewvision-software-consultancy-azure-databricks-platform-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fnewvision-software-consultancy-azure-databricks-platform-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/newvision-software-consultancy-azure-databricks-platform-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fnewvision-software-consultancy-azure-databricks-platform-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4309723}},{"tool":"search_jobs","arguments":{"company":"newvision-software.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4309723},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}