{"id":3796161,"url":"https://alion.io/job/celebal-tech-remote-lead-data-engineer","title":"[Remote] Lead Data Engineer","company":{"id":2810284,"name":"Celebal Tech","domain":"celebaltech.com","url":"https://alion.io/company/celebaltech","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"posting_text","remote_working_hours":null,"hiring_geo_confidence":"explicit","locations":["United States"],"countries":["US"],"hiring_countries":["US"],"hiring_countries_total":1,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Git","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"Terraform","optional":false}],"status":"closed","first_seen_at":"2026-11-05T07:33:30Z","employer_posted_date":null,"last_verified_at":"2026-11-05T15:27:10Z","board_verified":false,"closed_at":"2026-11-05T15:27:10Z","days_open":0,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":0},"description":"Note: The job is a remote job and is open to candidates in USA. Celebal Technologies is seeking a Lead Data Engineer to lead data engineering initiatives and architecture decisions for enterprise-grade solutions on Azure or AWS. The role focuses on designing and optimizing scalable Databricks data pipelines and lakehouse platforms while guiding engineers and collaborating with stakeholders.\n\nResponsibilities\nLead the design, development, and implementation of scalable and high-performance data engineering solutions using Databricks\nDesign and implement end-to-end data pipelines based on Medallion Architecture (Bronze, Silver, and Gold layers) \nDevelop and optimize data ingestion and transformation pipelines using Delta Live Tables (DLT) and Auto Loader \nBuild and manage data lakehouse solutions using Delta Lake and Apache Spark\nDesign, develop, and optimize ETL/ELT workflows using PySpark, Python, and SQL\nWork with Azure or AWS cloud services to build and manage scalable data platforms\nImplement data quality, validation, governance, and monitoring frameworks\nOptimize Spark workloads, cluster configurations, and query performance\nEstablish best practices for data engineering, coding standards, CI/CD, and deployment\nLead technical discussions, mentor data engineers, and provide technical guidance to the team\nCollaborate with architects, business stakeholders, and other technical teams to translate business requirements into technical solutions\n\nSkills\nWe are looking for candidates who are Green Card holders or US Citizens (USC) only\n10+ years of overall experience\nThe ideal candidate should have hands-on experience designing and implementing scalable data pipelines using Databricks, Medallion Architecture, Delta Live Tables (DLT), and Auto Loader\n10+ years of overall experience in Data Engineering, Big Data, and Data Platform development\nStrong hands-on experience with Databricks and its Lakehouse Platform\nExtensive experience implementing Medallion Architecture (Bronze, Silver, Gold)\nStrong experience with Delta Live Tables (DLT)\nHands-on experience with Databricks Auto Loader for incremental and continuous data ingestion\nStrong programming experience in PySpark, Python, and SQL\nExperience with Delta Lake , Apache Spark, and distributed data processing\nStrong cloud experience in either Microsoft Azure or AWS\nExperience designing and implementing scalable ETL/ELT pipelines\nGood understanding of data modeling, data governance, security, and data quality frameworks\nA strong technical leader with excellent hands-on data engineering capabilities\nAbility to independently drive complex data engineering initiatives\nExperience designing scalable, reliable, and cost-effective data solutions\nDatabricks certifications are an added advantage\nExperience with Unity Catalog and data governance\nKnowledge of CI/CD, Git, and DevOps practices\nExperience with Infrastructure as Code (Terraform)\nExperience leading teams and managing end-to-end data engineering projects\nStrong communication, stakeholder management, and problem-solving skills\n\nBenefits\nRemote work (must be open to travel to client locations based on project requirements)\n\nCompany Overview\nCelebal Technologies is a software services company that specializes in data science, generative AI, and digital transformation. It was founded in 2015, and is headquartered in Jaipur, Rajasthan, IND, with a workforce of 1001-5000 employees. Its website is https://celebaltech.com/.","description_format":"text","description_chars":3470,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Blockchain & Crypto","IT Consulting & Digital Transformation","AI Consulting & Integration","Data Engineering & Migration Services"],"lifecycle":[{"event":"open","at":"2026-11-05T09:20:50Z"},{"event":"close","at":"2026-11-05T15:27:10Z"}],"visa":[],"liveness":null,"pay":null,"html_url":"https://alion.io/job/celebal-tech-remote-lead-data-engineer","json_url":"https://alion.io/job/celebal-tech-remote-lead-data-engineer.json","meta":{"generated_at":"2026-11-07T19:40:48Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":30710,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-08T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":2810284},"rest":"https://alion.io/mcp/rest/get_company?id=2810284"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcelebal-tech-remote-lead-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcelebal-tech-remote-lead-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcelebal-tech-remote-lead-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/celebal-tech-remote-lead-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcelebal-tech-remote-lead-data-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3796161}},{"tool":"search_jobs","arguments":{"company":"celebaltech.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3796161},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}