{"id":2815987,"url":"https://alion.io/job/persistent-systems-lead-databricks-data-engineer","title":"Lead Databricks Data Engineer","company":{"id":59479,"name":"Persistent Systems","domain":"persistent.com","url":"https://alion.io/company/persistent-systems","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Zwayam","truth_index":{"grade":"A","score":90,"open_postings":89,"ghost_share":0,"stale_share":0.382,"repost_share":0,"time_to_fill_p50_days":47,"computed_at":"2026-10-22T06:10:10Z"}},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":null,"work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":30000,"max_usd":66000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":20},"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Apache Kafka","optional":false},{"name":"AWS","optional":false},{"name":"AWS Glue","optional":false},{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"Azure DevOps","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Dimensional Modeling","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"Git","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false}],"status":"live","first_seen_at":"2026-10-07T06:02:27Z","employer_posted_date":"2026-10-21","last_verified_at":"2026-10-22T04:48:42Z","board_verified":true,"closed_at":null,"days_open":15,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":15},"description":"About Persistent\nWe are an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what?s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many industry-leading organizations across the world, including 20 Fortune 50 companies and 4 of the 5 top banks in both the US and India, and numerous innovators across the healthcare ecosystem.\nOur disruptor?s mindset, commitment to client success, and agility to thrive in the dynamic environment have enabled us to sustain our growth momentum. Persistent has been recognized across top industry platforms for innovation, leadership, and inclusion. We reported $1,654.4M FY26 revenue with 17.4% Y-o-Y growth. We have delivered 24 sequential quarters of growth with $436.0M in Q4 FY26 revenue, up 3.2% Q-o-Q and 16.2% Y-o-Y growth. Our 27,500+ global team members, located in 18 countries, have been instrumental in helping the market leaders transform their industries. We have been recognized as the Fastest Growing IT Services Brand Globally in the 2026 Brand Finance IT Services 25 Report. We named a Leader in the Everest Group Private Equity (PE) Services PEAK Matrix� Assessment 2026 and Software Product Engineering PEAK Matrix� Assessment 2026.\nAbout Position:\nWe are looking for a skilled Lead Databricks Data Engineer with strong expertise in Data Engineering, Big Data technologies, and the Databricks Lakehouse Platform. The ideal candidate will be responsible for designing, developing, and supporting scalable data pipelines, enterprise analytics solutions, and cloud-native data platforms. The role requires deep hands-on experience in Databricks, PySpark, SQL, ETL/ELT frameworks, and cloud technologies. You will work closely with architects, data scientists, business stakeholders, and engineering teams to deliver high-performance, scalable, and reliable data solutions that enable enterprise-wide analytics and data-driven decision-making.\nRole: Lead Databricks Data Engineer\nLocation: Bengaluru, Pune\nExperience: 8 to 12 Years\nJob Type: Full Time Employment\nWhat You'll Do:\nDesign, develop, and maintain scalable ETL/ELT pipelines using Databricks.\nBuild and optimize large-scale data processing workflows using PySpark and Spark SQL.\nDevelop data ingestion frameworks for structured, semi-structured, and unstructured data sources.\nImplement batch and real-time data processing solutions on the Databricks Lakehouse Platform.\nDesign and support Delta Lake implementations and Lakehouse architectures.\nOptimize Databricks workloads for performance, scalability, reliability, and cost efficiency.\nDevelop data transformation, validation, cleansing, and reconciliation frameworks.\nSupport enterprise data modeling and data warehousing initiatives.\nImplement monitoring, alerting, and troubleshooting mechanisms for data pipelines.\nIntegrate Databricks with cloud services, APIs, enterprise applications, and third-party systems.\nDevelop cloud-native data solutions across Azure, AWS, or Google Cloud environments.\nSupport CI/CD implementation and deployment automation for data engineering workloads.\nCollaborate with Architects, Business Analysts, Data Scientists, and Product Owners to deliver business-driven data solutions.\nParticipate in solution design discussions, architecture reviews, and code review processes.\nSupport production deployments, incident resolution, and performance tuning activities.\nCreate technical documentation, design specifications, and operational runbooks.\nMentor junior engineers and promote data engineering best practices across teams.\nExpertise You'll Bring:\n8?12 years of experience in Data Engineering, Big Data, and Analytics platform development.\nStrong hands-on expertise with Databricks Lakehouse Platform and enterprise-scale data engineering solutions.\nAdvanced proficiency in PySpark, Spark SQL, Python, and SQL.\nExtensive experience designing and developing scalable ETL/ELT pipelines and data integration frameworks.\nDeep understanding of Delta Lake architecture, transaction management, and performance optimization techniques.\nStrong knowledge of Data Warehousing concepts, dimensional modeling, and enterprise analytics platforms.\nExperience working with structured, semi-structured, and unstructured data at scale.\nHands-on experience building batch and real-time data processing solutions.\nStrong expertise in data ingestion, transformation, validation, cleansing, and reconciliation processes.\nExperience integrating enterprise applications, APIs, cloud services, and diverse data sources.\nFamiliarity with Azure Data Factory, AWS Glue, or equivalent cloud-native data integration services.\nExperience working with Azure, AWS, or Google Cloud Platform environments.\nStrong understanding of Git, source control practices, and collaborative development methodologies.\nExperience implementing and managing CI/CD pipelines for data engineering workloads.\nKnowledge of Kafka, Event Hub, or similar streaming technologies for real-time data processing.\nFamiliarity with Unity Catalog, metadata management, and enterprise data governance frameworks.\nStrong understanding of schema design, data modeling, and query optimization techniques.\nExperience using Azure DevOps and deployment automation frameworks.\nExposure to AI/ML data pipelines, feature engineering, and model data preparation workflows.\nStrong troubleshooting, production support, and root cause analysis capabilities.\nExperience optimizing Databricks workloads for enterprise-scale performance and reliability.\nAbility to independently lead medium-to-large data engineering initiatives.\nExperience mentoring team members and conducting technical reviews.\nExcellent analytical, communication, and stakeholder management skills.\nStrong commitment to code quality, testing, engineering best practices, and continuous improvement.\nAbility to contribute to architecture discussions and strategic data platform initiatives.\nBenefits:\nCompetitive salary and benefits package\nCulture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications\nOpportunity to work with cutting-edge technologies\nEmployee engagement initiatives such as project parties, flexible work hours, and Long Service awards\nAnnual health check-ups\nInsurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents\nValues-Driven, People-Centric & Inclusive Work Environment:\nPersistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.\nWe support hybrid work and flexible hours to fit diverse lifestyles.\nOur office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.\nIf you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment\nLet?s unleash your full potential at Persistent -persistent.com/careers\n?Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.?","description_format":"text","description_chars":7472,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity","Flexible schedule","Growth opportunities","Hybrid work","Insurance coverage"],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Health Care","Information Technology","IT Outsourcing & Dedicated Teams"],"lifecycle":[{"event":"open","at":"2026-10-18T13:09:05Z"},{"event":"close","at":"2026-10-21T06:26:57Z"},{"event":"reopen","at":"2026-10-21T12:26:12Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.858,"p_room":1,"age_days":15,"expected_fill_days":47,"reasons":["conf:1","velocity","win:early","comp:brand"],"computed_at":"2026-10-22T06:10:10Z"},"pay":null,"html_url":"https://alion.io/job/persistent-systems-lead-databricks-data-engineer","json_url":"https://alion.io/job/persistent-systems-lead-databricks-data-engineer.json","meta":{"generated_at":"2026-10-22T06:33:28Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":1563,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-23T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":59479},"rest":"https://alion.io/mcp/rest/get_company?id=59479"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fpersistent-systems-lead-databricks-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fpersistent-systems-lead-databricks-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fpersistent-systems-lead-databricks-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/persistent-systems-lead-databricks-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fpersistent-systems-lead-databricks-data-engineer"}]}