{"id":4724332,"url":"https://alion.io/job/svaantech-senior-data-engineer-etl-databricks-and-data-platforms","title":"Senior Data Engineer – ETL, Databricks and Data Platforms","company":{"id":4545985,"name":"Svaantech","domain":"svaantech.com","url":"https://alion.io/company/svaantech","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Schema","truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Chennai, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":6,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Amazon Redshift","optional":false},{"name":"Apache Kafka","optional":false},{"name":"ClickHouse","optional":false},{"name":"Dagster","optional":false},{"name":"Databricks","optional":false},{"name":"dbt","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Feature Store","optional":false},{"name":"Git","optional":false},{"name":"Google BigQuery","optional":false},{"name":"Great Expectations","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Python","optional":false},{"name":"Redis","optional":false},{"name":"Shuffle","optional":false},{"name":"Snowflake","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Time Series","optional":false}],"status":"live","first_seen_at":"2026-06-15T11:52:59Z","employer_posted_date":"2026-06-15","last_verified_at":"2026-11-20T22:54:01Z","board_verified":true,"closed_at":null,"days_open":158,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":158},"description":"Role Summary\n\nThe Senior Data Engineer will build the data foundation for a large-scale AI platform. The role includes ingesting large datasets, creating reliable ETL/ELT pipelines, designing feature-ready tables, managing data quality and supporting model training and product analytics. This person should understand that good AI systems depend on disciplined data engineering.\n\nThe role requires hands-on experience with distributed data processing, batch pipelines, schema evolution, performance tuning and production-grade data operations. The right person can work with messy source data and turn it into trusted datasets that engineers, data scientists and product teams can use.\n\nWhat You Will Do\n\nDesign and build scalable data ingestion, transformation and validation pipelines for structured, semi-structured and time-series data.\n\nWork with large datasets using Databricks, Spark, lakehouse patterns, object storage and distributed processing frameworks.\n\nCreate curated tables, feature datasets, training labels, data quality checks and metadata needed for model development and product use cases.\n\nHandle schema changes, late-arriving data, missing values, inconsistent identifiers, timestamp alignment and unit normalization.\n\nOptimize data pipelines for cost, speed, reliability and repeatable backfills.\n\nCollaborate with data scientists and ML engineers to ensure model training data is consistent, auditable and versioned.\n\nBuild monitoring and alerting for pipeline health, data freshness, data quality and downstream impact.\n\nDocument datasets, transformations, assumptions and lineage clearly.\n\nParticipate in architecture decisions around data warehouses, lakehouses, feature stores, orchestration and storage formats.\n\nSupport production incidents related to data quality, pipeline failures or performance bottlenecks.\n\nRequirements and Skills\n\n6+ years of data engineering experience with large-scale production datasets.\n\nStrong SQL and Python experience for data transformation, validation and automation.\n\nHands-on experience with Apache Spark, Databricks or equivalent distributed processing environments.\n\nExperience with ETL/ELT design, batch processing, incremental loads, partitioning, file formats such as Parquet/Delta and object storage.\n\nStrong understanding of data modeling, schema evolution, data quality frameworks, lineage and reproducibility.\n\nExperience with orchestration tools such as Airflow, Dagster, Databricks Workflows or similar systems.\n\nAbility to tune data jobs for performance and cost, including memory, shuffle, partitioning and parallelism considerations.\n\nExperience working with messy operational datasets and turning them into reliable analytical or model-ready tables.\n\nGood software engineering practices, including Git, code review, tests, modular code and clear documentation.\n\nPreferred Background\n\nExperience building feature stores, model training datasets or ML data pipelines.\n\nExperience with ClickHouse, PostgreSQL, Snowflake, BigQuery, Redshift, Kafka, Redis or similar data systems.\n\nExperience with data contracts, Great Expectations, Deequ, dbt or comparable data validation tools.\n\nComfortable working with data scientists, backend engineers and product managers to translate business needs into data structures.\n\nStartup Environment\n\nThis is a startup environment for people who want meaningful responsibility rather than narrowly defined corporate roles. Team members should expect exposure to multiple parts of the business, including product design, engineering decisions, customer problem solving, implementation planning and operational execution. The team will be small, highly technical and organized around talented builders who can work directly with one another without unnecessary layers of hierarchy. We expect people to use modern development tools aggressively, including coding assistants, automation, test tools, model tooling and sufficient token budgets where they improve speed and quality. The working style is flexible, but the expectations are high: clear ownership, written thinking, disciplined execution, frequent communication, clean handoffs and the ability to make progress without waiting for a complete corporate structure.\n\nWhat Success Looks Like\n\nData pipelines are reliable, observable and easy to re-run or backfill.\n\nModeling and product teams trust the curated datasets and understand their definitions.\n\nLarge datasets are processed efficiently without uncontrolled cloud cost or fragile manual steps.","description_format":"text","description_chars":4518,"description_truncated":false,"requirements":{"experience_years_min":6,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Software","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-11-20T22:54:01Z"}],"visa":[],"liveness":{"score":10,"band":"cold","label":"Long shot","p_open":1,"p_active":0.344,"p_room":0.28,"age_days":158,"expected_fill_days":30,"reasons":["conf:1","win:tail","crowd:"],"computed_at":"2026-11-21T00:34:23Z"},"pay":null,"html_url":"https://alion.io/job/svaantech-senior-data-engineer-etl-databricks-and-data-platforms","json_url":"https://alion.io/job/svaantech-senior-data-engineer-etl-databricks-and-data-platforms.json","meta":{"generated_at":"2026-11-21T00:34:23Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":1070,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-22T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4545985},"rest":"https://alion.io/mcp/rest/get_company?id=4545985"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fsvaantech-senior-data-engineer-etl-databricks-and-data-platforms"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fsvaantech-senior-data-engineer-etl-databricks-and-data-platforms"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fsvaantech-senior-data-engineer-etl-databricks-and-data-platforms"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/svaantech-senior-data-engineer-etl-databricks-and-data-platforms\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fsvaantech-senior-data-engineer-etl-databricks-and-data-platforms"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4724332}},{"tool":"search_jobs","arguments":{"company":"svaantech.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4724332},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}