{"id":2771316,"url":"https://alion.io/job/lancesoft-india-pvt-ltd-data-engineering-lead-azure-databricks","title":"Data Engineering Lead - Azure Databricks","company":{"id":3800169,"name":"Lancesoft India Pvt Ltd","domain":"lancesoft.in","url":"https://alion.io/company/lancesoft-india-pvt-ltd","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":23000,"max_usd":67000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":14},"experience_years_min":9,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Azure","optional":false},{"name":"Databricks","optional":false},{"name":"ETL/ELT","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-08-19T10:55:58Z","employer_posted_date":null,"last_verified_at":"2026-08-19T10:55:58Z","board_verified":false,"closed_at":null,"days_open":66,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":66},"description":"Role Overview:\n\nWe are looking for an experienced Data Engineering Lead who can combine strong technical leadership with hands-on engineering expertise. The role will be responsible for driving the design, development, and maintenance of scalable data pipelines and platforms while remaining actively involved in technical implementation. The ideal candidate should have strong expertise in Databricks, Spark, PySpark, Python, SQL, and Data Architecture, along with the ability to make technical decisions, establish engineering standards, mentor team members, and solve complex production challenges.\n\nKey Responsibilities:\n\nTechnical Leadership:\n\n- Provide technical leadership for the design, development, and maintenance of scalable data pipelines and data platforms.\n\n- Act as a technical SME for Databricks, Spark, PySpark, Python, SQL, and Data Architecture.\n\n- Drive technical design decisions and establish scalable and effective engineering patterns.\n\n- Conduct code reviews and design reviews to maintain technical quality and consistency.\n\n- Define and enforce coding standards, development guidelines, and engineering best practices.\n\n- Mentor and guide Data Engineers and Senior Data Engineers.\n\n- Identify technical risks, dependencies, bottlenecks, and opportunities for improvement.\n\n- Monitor and optimize data infrastructure performance.\n\n- Lead troubleshooting and resolution of complex technical and production issues.\n\n- Ensure appropriate testing, deployment, release, and rollback processes are followed.\n\n- Collaborate with architecture and product teams to translate business requirements into effective technical solutions.\n\nIndividual Contributor Responsibilities:\n\n- The Data Engineering Lead will remain approximately 60% hands-on / individual contributor, balancing technical leadership with direct engineering contribution.\n\n- Participate directly in architecture and technical design.\n\n- Contribute to coding and development activities.\n\n- Perform debugging and troubleshoot complex technical issues.\n\n- Drive performance optimization of data engineering solutions.\n\n- Participate in code and design reviews.\n\n- Contribute to production support and issue resolution.\n\n- Remain actively involved in hands-on delivery while providing technical direction to the team.\n\nTechnical Skills & Requirements:\n\n- Strong expertise in Databricks.\n\n- Strong hands-on experience with Apache Spark and PySpark.\n\n- Strong proficiency in Python and SQL.\n\n- Strong understanding of Data Architecture.\n\n- Experience designing and developing scalable data pipelines and platforms.\n\n- Strong technical design and problem-solving capabilities.\n\n- Experience conducting code reviews and design reviews.\n\n- Ability to establish and enforce engineering standards and best practices.\n\n- Experience mentoring and guiding Data Engineers and Senior Data Engineers.\n\n- Strong understanding of production support, troubleshooting, and performance optimization.\n\n- Ability to balance technical leadership with hands-on engineering.\n\nRole Expectations:\n\n- This role is suited for a technical leader who prefers to remain closely involved in engineering rather than moving completely into people management.\n\n- The successful candidate will provide technical direction, support the development team, make architecture and design decisions, and continue contributing directly to coding, debugging, optimization, reviews, and production support.\n\nSkills\nData Engineering, Azure Databricks, Apache Spark, Python, SQL, ETL, PySpark, Data Pipeline","description_format":"text","description_chars":3540,"description_truncated":false,"requirements":{"experience_years_min":9,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Outsourcing & Dedicated Teams","IT Staffing & Staff Augmentation"],"lifecycle":[{"event":"open","at":"2026-10-17T04:49:54Z"}],"visa":[],"liveness":{"score":6,"band":"cold","label":"Long shot","p_open":0.4,"p_active":0.524,"p_room":0.28,"age_days":65,"expected_fill_days":27,"reasons":["seen:65","velocity","win:tail","crowd:"],"computed_at":"2026-10-24T06:02:35Z"},"pay":null,"html_url":"https://alion.io/job/lancesoft-india-pvt-ltd-data-engineering-lead-azure-databricks","json_url":"https://alion.io/job/lancesoft-india-pvt-ltd-data-engineering-lead-azure-databricks.json","meta":{"generated_at":"2026-10-24T20:07:40Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":29460,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-25T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3800169},"rest":"https://alion.io/mcp/rest/get_company?id=3800169"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Flancesoft-india-pvt-ltd-data-engineering-lead-azure-databricks"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Flancesoft-india-pvt-ltd-data-engineering-lead-azure-databricks"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Flancesoft-india-pvt-ltd-data-engineering-lead-azure-databricks"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/lancesoft-india-pvt-ltd-data-engineering-lead-azure-databricks\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Flancesoft-india-pvt-ltd-data-engineering-lead-azure-databricks"}]}