{"id":1279031,"url":"https://alion.io/job/databeat-senior-databricks-engineer","title":"Senior Databricks Engineer","company":{"id":3806738,"name":"DataBeat","domain":"databeat.io","url":"https://alion.io/company/databeat","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Keka","truth_index":{"grade":"B","score":75,"open_postings":7,"ghost_share":0,"stale_share":1,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-09-28T05:45:00Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Hyderabad, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":20000,"max_usd":40000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":51},"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"AWS Lambda","optional":false},{"name":"Azure","optional":false},{"name":"Azure Data Factory","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"Dimensional Modeling","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GitHub","optional":false},{"name":"GitLab","optional":false},{"name":"ISO 27001","optional":false},{"name":"Machine Learning","optional":false},{"name":"MLFlow","optional":false},{"name":"Power BI","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Tableau","optional":false}],"status":"live","first_seen_at":"2026-06-30T13:19:45Z","employer_posted_date":"2026-06-30","last_verified_at":"2026-09-26T07:58:58Z","board_verified":false,"closed_at":null,"days_open":90,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":90},"description":"Job Title: Senior Databricks Engineer\nExperience Required: 8+ Years\nRole Summary\nWe are seeking an experienced Senior Databricks Developer with 8+ years of experience in Data Engineering, Big Data, and Cloud Analytics solutions, including extensive hands-onexpertise in Databricks, Apache Spark, PySpark, SQL, and Delta Lake. The ideal candidate will be responsible for leading end-to-end data engineering projects, engaging directly with clients and business stakeholders, mentoring development teams, and delivering scalable, secure, and high-performance data platforms across Azure and AWS environments.\nThis role requires strong technical leadership, solution design capabilities, stakeholder management skills, and the ability to drive data transformation initiatives from requirements gathering through deployment and production support.\nTechnical Expertise\nDatabricks (PySpark, SQL, Notebooks, Workflows) , Apache Spark , Delta Lake , Python, Azure\nData Factory (ADF) ,Azure Synapse Analytics ,AWS S3, Glue, Lambda , Unity Catalog\nMLflow ,Databricks Job Clusters ,CI/CD Pipelines ,GitLab / GitHub ,Data Governance & Data\nQuality Frameworks, Power BI / Tableau , Data Cataloging ,Lakehouse Architecture ,Medallion\nArchitecture, Performance Tuning & Optimization\nKey Responsibilities\nLead the design, development, and implementation of enterprise-scale data engineering\nsolutions using Databricks, PySpark, SQL, and Delta Lake.\nOwn end-to-end project delivery, including requirement gathering, solution design,\ndevelopment, testing, deployment, and production support.\nCollaborate directly with clients, business stakeholders, architects, and product owners to\nunderstand business requirements and translate them into scalable technical solutions.\nConduct client discussions, solution workshops, effort estimations, technical presentations,\nand architecture reviews.\nDesign and implement scalable Lakehouse architectures and data platforms across Azure and\nAWS cloud environments.\nLead the development of high-performance ETL/ELT pipelines supporting both batch and\nreal-time data processing workloads.\nDrive best practices around coding standards, architecture governance, version control, CI/CD\nimplementation, and operational excellence.\nArchitect and optimize Databricks Workflows, Job Clusters, Delta Tables, and Spark\napplications to maximize performance and minimize infrastructure costs.\nImplement data governance frameworks using Unity Catalog, ensuring proper access controls,\nlineage tracking, metadata management, and compliance standards.\nMentor and guide junior and mid-level data engineers through code reviews, technical\ncoaching, and knowledge-sharing initiatives.\nLead technical teams and coordinate project activities to ensure timely and successful project\ndelivery.\nCollaborate with Data Scientists, BI Teams, and Analytics stakeholders to enable advanced\nanalytics and machine learning use cases.\nDrive automation initiatives through Infrastructure as Code (IaC), DevOps practices, and\nCI/CD pipelines.\nEstablish monitoring, observability, and performance tracking frameworks for data platforms\nand pipelines.\nEnsure security, compliance, data quality, and operational reliability across enterprise data\necosystems.\nPrepare and maintain technical architecture documents, design specifications, implementation\nguides, and operational runbooks.\nRequirements\n8+ years of experience in Data Engineering, Big Data, and Analytics platforms.\nMinimum 5+ years of hands-on experience working with Databricks, Apache Spark, PySpark,\nand SQL.\nStrong expertise in designing and implementing enterprise-scale Data Lake, Lakehouse, and\nData Warehouse solutions.\nDeep understanding of Spark internals, cluster management, performance tuning, partitioning\nstrategies, and resource optimization.\nExtensive experience working with Delta Lake, schema evolution, ACID transactions, and\nlarge-scale distributed data processing.\nHands-on experience integrating Databricks with Azure (ADF, Synapse) and/or AWS (S3,\nGlue, Lambda) services.\nProven experience handling end-to-end project delivery and managing technical engagements\nwith clients and stakeholders.\nExperience leading development teams and mentoring engineers in enterprise environments.\nStrong understanding of data modeling, dimensional modeling, Medallion Architecture, and\nmodern data platform design principles.\nExpertise in implementing CI/CD pipelines, DevOps practices, and automated deployment\nstrategies.\nExperience with Unity Catalog or equivalent governance platforms.\nExcellent communication, stakeholder management, presentation, and client-facing skills.\nAbility to lead technical discussions, architecture reviews, and solution design workshops.\nBachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a\nrelated field.\nInformation Security & ISO 27001 Compliance\nSecurity is at the heart of everything we do. In this role, you will be strictly required to uphold our Information Security Management System (ISMS) policies in alignment with ISO/IEC 27001 standards. Responsibilities include safeguarding sensitive asset data, completing mandatory security awareness training, reporting potential security incidents or vulnerabilities immediately, and ensuring that daily operations comply with our rigorous data protection protocols.","description_format":"text","description_chars":5348,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Marketing Analytics & Attribution","Analytics & BI Consulting"],"lifecycle":[{"event":"open","at":"2026-09-26T02:29:56Z"}],"liveness":{"score":16,"band":"cold","label":"Long shot","p_open":1,"p_active":0.564,"p_room":0.28,"age_days":89,"expected_fill_days":31,"reasons":["conf:45","velocity","win:tail","crowd:"],"computed_at":"2026-09-28T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/databeat-senior-databricks-engineer","json_url":"https://alion.io/job/databeat-senior-databricks-engineer.json","meta":{"generated_at":"2026-09-29T03:39:09Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","usage":{"tier":"crawler","counted_by":"address","units_charged":1,"used_today":3390,"day_limit":5000,"remaining_today":1610,"minute_limit":60,"resets_at":"2026-09-30T00:00:00Z"}}}