{"id":3311221,"url":"https://alion.io/job/unitedhealth-group-data-engineering-lead-python-pyspark-azure-databricks","title":"Data Engineering Lead - Python, Pyspark, Azure Databricks","company":{"id":1043512,"name":"UnitedHealth Group","domain":"unitedhealthgroup.com","url":"https://alion.io/company/unitedhealthgrp","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Radancy","truth_index":{"grade":"B","score":75,"open_postings":130,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-11-03T05:54:40Z"}},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Gurgaon, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":28000,"max_usd":60000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":20},"experience_years_min":6,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"Azure DevOps","optional":false},{"name":"Azure SQL Database","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Git","optional":false},{"name":"GitHub","optional":false},{"name":"Power BI","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Apache Kafka","optional":true},{"name":"FHIR","optional":true},{"name":"HIPAA","optional":true},{"name":"HL7","optional":true}],"status":"live","first_seen_at":"2026-10-27T03:27:51Z","employer_posted_date":"2026-10-26","last_verified_at":"2026-11-04T01:59:21Z","board_verified":true,"closed_at":null,"days_open":8,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":8},"description":"Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to startCaring. Connecting. Growing together.\nPrimary Responsibilities:\nDesign, build and maintain scalable batch and streaming data pipelines on Azure and Databricks using the Medallion Architecture (Bronze, Silver, Gold) on Delta Lake\nIngest and process structured, semi-structured and unstructured data in formats such as CSV, JSON, XML, Parquet, Avro, ORC, Excel and Delta, using Azure Data Factory, Auto Loader and Lakeflow Spark Declarative Pipelines\nDevelop complex transformations in PySpark and Spark SQL, including CDC, SCD Type 1/2, deduplication and handling of nested or schema-evolving data\nDesign data models and build the right consumption objects for each use case: managed/external tables, views, temporary views, materialized views and streaming tables\nImplement data governance, lineage and security using Unity Catalog, including row/column-level security and PHI/PII protection\nBuild data quality, validation and reconciliation frameworks to keep data accurate and reliable\nTune Spark jobs and Delta tables for performance and cost (partitioning, liquid clustering, Z-ordering, OPTIMIZE/VACUUM, compute sizing)\nDrive CI/CD and DataOps practices using Azure DevOps or GitHub, Declarative Automation Bundles and automated testing\nTroubleshoot production issues, do root cause analysis and put preventive fixes in place\nWork with architects, analysts, data scientists and business stakeholders, and mentor junior engineers\nUse AI-assisted engineering tools to improve development, testing and documentation\nComply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements\nRequired Qualifications:\nB.E./B.Tech./MCA or equivalent degree in Computer Science, IT or Engineering\n6+ years of data engineering experience, including 3+ years hands-on with Azure Databricks\nHands-on experience with Delta Lake, Lakehouse architecture and Unity Catalog\nExperience working with file formats such as CSV, JSON, XML, Parquet, Avro and Delta, including schema inference, schema evolution and nested data handling\nExperience with Azure services: ADF, ADLS Gen2, Key Vault, Azure SQL , Power Bi\nSolid knowledge of data modeling, ETL/ELT design, Git, CI/CD and Agile delivery\nGood understanding of views, temporary/global temporary views, materialized views, streaming tables and when to use each\nSolid proficiency in Python, PySpark and SQL, with a solid understanding of Spark internals and performance tuning\nProven solid problem-solving and communication skills\nPreferred Qualifications:\nDatabricks Data Engineer Associate/Professional or Microsoft DP-203/DP-700 certification\nExperience with Structured Streaming, Kafka/Event Hubs, Lakeflow Jobs and Databricks SQL\nInfrastructure as Code experience\nExposure to healthcare data (claims, prior authorization, HL7/FHIR, EDI 837/278) and HIPAA compliance\nExperience with Power BI or AI/BI dashboards\nExperience mentoring or leading a team, or driving cloud migration and modernization\nAt UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.","description_format":"text","description_chars":4281,"description_truncated":false,"requirements":{"experience_years_min":6,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":true},"security_clearance":false,"languages":[]},"benefits":["Flexible schedule"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Hospitals & Health Systems","Pharmacies","Health Insurance & Benefits"],"lifecycle":[{"event":"open","at":"2026-10-27T03:27:51Z"}],"visa":[],"liveness":{"score":63,"band":"ok","label":"Likely open","p_open":1,"p_active":0.632,"p_room":1,"age_days":7,"expected_fill_days":26,"reasons":["conf:4","stale_co","velocity","win:early","comp:brand"],"computed_at":"2026-11-03T05:54:40Z"},"pay":null,"html_url":"https://alion.io/job/unitedhealth-group-data-engineering-lead-python-pyspark-azure-databricks","json_url":"https://alion.io/job/unitedhealth-group-data-engineering-lead-python-pyspark-azure-databricks.json","meta":{"generated_at":"2026-11-04T06:22:37Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":9942,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-05T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1043512},"rest":"https://alion.io/mcp/rest/get_company?id=1043512"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Funitedhealth-group-data-engineering-lead-python-pyspark-azure-databricks"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Funitedhealth-group-data-engineering-lead-python-pyspark-azure-databricks"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Funitedhealth-group-data-engineering-lead-python-pyspark-azure-databricks"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/unitedhealth-group-data-engineering-lead-python-pyspark-azure-databricks\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Funitedhealth-group-data-engineering-lead-python-pyspark-azure-databricks"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3311221}},{"tool":"search_jobs","arguments":{"company":"unitedhealthgroup.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3311221},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}