{"id":3290384,"url":"https://alion.io/job/cogniify-databricks-data-engineer-india","title":"Databricks Data Engineer - India","company":{"id":4610,"name":"Cogniify","domain":"cogniify.ai","url":"https://alion.io/company/cogniify","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":null,"employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"posting_text","remote_working_hours":{"label":"EST","utc_offset_min":-5,"utc_offset_max":-5},"hiring_geo_confidence":"explicit","locations":[],"countries":[],"hiring_countries":["IN"],"hiring_countries_total":1,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"pySpark","optional":false},{"name":"Scala","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Airflow","optional":true},{"name":"Azure Data Factory","optional":true},{"name":"CI/CD","optional":true},{"name":"Git","optional":true},{"name":"Python","optional":true},{"name":"Terraform","optional":true}],"status":"live","first_seen_at":"2026-09-05T08:01:38Z","employer_posted_date":null,"last_verified_at":"2026-09-05T08:01:38Z","board_verified":false,"closed_at":null,"days_open":56,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":56},"description":"Location: RemoteWork Hours: EST (Eastern Standard Time) aligned\nExperience: 6–9 yearsRole OverviewWe are looking for an experienced Databricks Data Engineer to support, maintain, and enhance existing Databricks-based data applications and pipelines. The role focuses on ensuring reliability, performance, and scalability of production Databricks workloads rather than building net-new platforms from scratch. You will work closely with data, analytics, and engineering teams to keep critical data applications stable, optimized, and aligned with business needs.\nKey Responsibilities\nSupport and maintain existing Databricks applications, notebooks, jobs, and Delta Lake pipelines in production.\n\nMonitor, troubleshoot, and resolve issues related to job failures, performance degradation, data quality, and cluster utilization.\n\nOptimize existing Spark jobs, SQL queries, and Delta tables for cost, performance, and reliability.\n\nManage and improve Databricks workspace configurations, including clusters, job scheduling, access controls, and Unity Catalog (where applicable).\n\nImplement and maintain data quality checks, logging, alerting, and basic observability for Databricks workloads.\n\nCollaborate with stakeholders to understand requirements for enhancements or bug fixes on existing applications.\n\nPerform incremental improvements, refactoring, and technical debt reduction on current Databricks solutions.\n\nEnsure adherence to best practices around security, governance, and cost management within the Databricks environment.\n\nDocument existing pipelines, dependencies, and operational runbooks.\n\nParticipate in on-call or support rotations as needed to maintain production stability (within EST working hours).\n\nRequired Qualifications\n6–9 years of overall experience in data engineering, with strong hands-on experience in Databricks.\n\nSolid proficiency in Apache Spark (PySpark and/or Scala) and SQL.\n\nProven experience supporting and optimizing production Databricks workloads (jobs, notebooks, Delta Lake, workflows).\n\nStrong understanding of Delta Lake concepts (ACID transactions, time travel, optimization techniques such as Z-ordering, vacuum, optimize).\n\nExperience with Databricks Job clusters, Interactive clusters, and performance tuning (partitioning, caching, shuffle optimization, autoscaling).\n\nFamiliarity with data modeling, ETL/ELT patterns, and production data pipeline support.\n\nExperience working with cloud platforms (preferably Azure, AWS, or GCP) in the context of Databricks.\n\nAbility to troubleshoot complex Spark and Databricks issues independently.\n\nStrong communication skills and ability to work effectively in a remote, EST-aligned team.\n\nPreferred Qualifications\nExperience with Unity Catalog, Databricks SQL, or Lakehouse architecture.\n\nKnowledge of CI/CD practices for Databricks (e.g., Databricks Asset Bundles, Git integration, Terraform/ARM templates).\n\nFamiliarity with orchestration tools (Airflow, Azure Data Factory, or Databricks Workflows).\n\nExposure to data quality frameworks, monitoring tools, or cost optimization initiatives on Databricks.\n\nExperience supporting analytics or BI teams consuming Databricks data products.\n\nWork Arrangement\nFully remote\n\nMust be available and productive during EST business hours\n\nCollaborative remote environment with regular syncs and support responsibilities\n\nWhy Join UsWhy Join Us?\nJoin a team of industry veterans from Google, Meta, and top-tier tech companies.\n\nWork on impactful, high-scale projects with leading global clients.\n\nEnjoy a flexible, remote-first culture focused on innovation and excellence.\n\nCompetitive salary, equity options, and continuous learning opportunities.\n\nShape the future of cloud and AI infrastructure at a rapidly growing company.\n\nPerks And Benefits Of Working With Us\nInternet allowance\n\nLaptop\n\nPF\n\nPaid PTO\n\nAnnual Bonus\n\nGraduity\n\nYearly Team building experiences\n\nMentorship and sponsorship opportunities\n\nManager resources and support\n\nLife & accidental insurance for additional protection.\n\nWe are an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or any other protected characteristic.\nOriginally posted on Himalayas","description_format":"text","description_chars":4411,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Accidental insurance","Continuous learning","Dental insurance","Equity"],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["MLOps","Custom Software Development","AI Consulting & Integration"],"lifecycle":[{"event":"open","at":"2026-10-26T19:11:04Z"}],"visa":[],"liveness":{"score":10,"band":"cold","label":"Long shot","p_open":0.4,"p_active":0.538,"p_room":0.45,"age_days":55,"expected_fill_days":35,"reasons":["seen:55","velocity","win:tail"],"computed_at":"2026-10-31T05:58:55Z"},"pay":null,"html_url":"https://alion.io/job/cogniify-databricks-data-engineer-india","json_url":"https://alion.io/job/cogniify-databricks-data-engineer-india.json","meta":{"generated_at":"2026-10-31T22:49:06Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":35591,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-01T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4610},"rest":"https://alion.io/mcp/rest/get_company?id=4610"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fcogniify-databricks-data-engineer-india"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fcogniify-databricks-data-engineer-india"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fcogniify-databricks-data-engineer-india"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/cogniify-databricks-data-engineer-india\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fcogniify-databricks-data-engineer-india"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3290384}},{"tool":"search_jobs","arguments":{"company":"cogniify.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3290384},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}