{"id":3829838,"url":"https://alion.io/job/aubrant-digital-senior-databricks-engineer","title":"Senior Databricks Engineer","company":{"id":673309,"name":"Aubrant Digital","domain":"aubrant.com","url":"https://alion.io/company/aubrant","size_band":"201-500","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Rippling","truth_index":{"grade":"C","score":69,"open_postings":25,"ghost_share":0.52,"stale_share":0,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-11-07T05:47:08Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Hyderabad, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Databricks","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Azure","optional":true},{"name":"dbt","optional":true},{"name":"Delta Lake","optional":true},{"name":"Great Expectations","optional":true},{"name":"pySpark","optional":true},{"name":"Python","optional":true},{"name":"Spark","optional":true},{"name":"SQL","optional":true}],"status":"live","first_seen_at":"2026-11-05T16:12:27Z","employer_posted_date":"2026-11-05","last_verified_at":"2026-11-07T10:43:08Z","board_verified":true,"closed_at":null,"days_open":1,"trust":{"level":"ok","repost_count":0,"flags":["company_stale"],"days_open":1},"description":"About Aubrant\nAubrant Digital is a leader in multi-shore custom application development. We are passionate about solving our clients’ business problems through consultative teamwork, innovative software, and proven processes. We’ve served more than 50 clients and delivered hundreds of high quality, custom enterprise applications. Our clients value us as integral team members who get the job done on time and on spec, and we are proud of our high client retention rate and under 2% staff turnover. With offices in New Jersey, Boston, Costa Rica, and Eastern Europe, we execute the full software lifecycle, from architecture and design through development, QA and application maintenance & support. Our company culture emphasizes client service, trust-based relationships, and innovation.\nAbout the role\nT he Senior Databricks Engineer will design, build, and optimize scalable data ingestion and transformation pipelines to support enterprise-grade data modernization programs. This role focuses on implementing ELT workflows within modern Data Lakehouse architectures, ensuring high-quality, reliable, secure, and efficient movement of data from diverse source systems into curated analytical and application-ready layers.\nYou will work closely with Data Architects, Data Modelers, Cloud Engineers, and Application teams to develop repeatable processing frameworks, enforce data quality rules, and operationalize Medallion-layered datasets (Bronze/Silver/Gold or equivalents). This role is domain-agnostic but requires the ability to translate complex business and technical requirements into performant data pipelines. Experience with high-volume operational data domains (e.g., logistics, telematics, IoT, ERP, mobility platforms) is a plus.\nKey Qualifications:\nBachelor’s/Master’s in Computer Science, Information Systems, Engineering, or related field (or equivalent experience).\n5-8 years of experience building data ingestion and transformation pipelines.\nStrong proficiency in SQL and PySpark for large-scale data processing.\nHands-on experience with dbt (data build tool) for building, testing, and documenting modular data transformation pipelines - a core requirement for this role.\nHands-on experience with Databricks, Spark jobs, and Delta Lake-based transformations.\nExperience with ELT patterns and Medallion Architectures (Bronze/Silver/Gold).\nPractical knowledge of Airbyte, ADF, or equivalent orchestration tools.\nExperience with data quality frameworks such as Great Expectations or Databricks expectations.\nStrong understanding of schema evolution, partitioning, optimization, and Delta Lake performance tuning.\nFamiliarity with Azure components including ADLS Gen2, Event Hub, Synapse Serverless/Dedicated SQL.\nExperience integrating pipelines with applications and APIs (read/write patterns).\nStrong debugging skills for data integrity, ingestion failures, schema drift, and performance issues.\nExcellent communication, documentation, and cross-team collaboration skills.\nResponsibilities:\nBuild scalable ELT pipelines using Databricks (PySpark/SQL), dbt, and Delta Lake.\nDevelop Bronze, Silver, and Gold transformations aligned with enterprise modeling standards.\nImplement ingestion using tools like Airbyte, ADF, or custom connectors.\nCollaborate with Data Modelers to operationalize business rules, mappings, and canonical definitions.\nIntegrate data pipelines with downstream applications (mobile apps, web apps, APIs).\nImplement data quality and validation rules, error handling, and automated remediation steps.\nManage schema evolution, CDC ingestion, late-arriving data, and incremental refresh patterns.\nOptimize Spark/Delta workloads through partitioning, indexing, caching, Z-ordering, and cluster tuning.\nDevelop reusable pipeline templates and frameworks for cross-domain scalability.\nCollaborate with Cloud Engineers to ensure pipelines meet security, monitoring, and DevOps standards.\nProduce technical documentation, lineage, and metadata for governance teams.\nSupport ML workloads by preparing feature datasets and inference-ready tables.\nParticipate in cross-functional design sessions, reviews, and architectural decisions.\nAbout Aubrant & Our Culture","description_format":"text","description_chars":4194,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":false},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-11-05T19:11:51Z"}],"visa":[],"liveness":{"score":63,"band":"ok","label":"Likely open","p_open":1,"p_active":0.632,"p_room":1,"age_days":1,"expected_fill_days":36,"reasons":["conf:3","stale_co","velocity","win:early"],"computed_at":"2026-11-07T05:47:08Z"},"pay":null,"html_url":"https://alion.io/job/aubrant-digital-senior-databricks-engineer","json_url":"https://alion.io/job/aubrant-digital-senior-databricks-engineer.json","meta":{"generated_at":"2026-11-07T12:07:19Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":13165,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-08T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":673309},"rest":"https://alion.io/mcp/rest/get_company?id=673309"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Faubrant-digital-senior-databricks-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Faubrant-digital-senior-databricks-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Faubrant-digital-senior-databricks-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/aubrant-digital-senior-databricks-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Faubrant-digital-senior-databricks-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3829838}},{"tool":"search_jobs","arguments":{"company":"aubrant.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3829838},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}