{"id":2231267,"url":"https://alion.io/job/clanx-senior-data-engineer","title":"Senior Data Engineer","company":{"id":673123,"name":"Clanx","domain":"clanx.com","url":"https://alion.io/company/clanx","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Gurgaon, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":21000,"max_usd":43000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":55},"experience_years_min":4,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"Dagster","optional":false},{"name":"ETL/ELT","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Prefect","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"SLI/SLO/SLA","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Time Series Forecasting","optional":false},{"name":"Amazon Redshift","optional":true},{"name":"Apache Kafka","optional":true},{"name":"CI/CD","optional":true},{"name":"RabbitMQ","optional":true},{"name":"Snowflake","optional":true},{"name":"Terraform","optional":true}],"status":"live","first_seen_at":"2026-10-10T17:35:06Z","employer_posted_date":null,"last_verified_at":"2026-10-10T17:35:06Z","board_verified":false,"closed_at":null,"days_open":4,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":4},"description":"Responsibilities\nWhat You'll Own\nLakehouse Pipelines & Ingestion (35%)\nDesign and own batch ETL/ELT and CDC pipelines that bring sensor and operational data into the lakehouse, orchestrated in Dagster\nBuild for reliability: idempotent, incremental, backfill-safe pipelines with sane retry and failure handling, that still produce correct output when a worker is killed mid-run or a message is delivered twice\nOnboard new client data sources: schema and tag mapping, time-series normalization, resampling, gap handling at scale\n2. Modeling & Serving (25%)\nModel raw data into well-structured, documented tables that downstream ML and analytics can trust\nBuild and maintain the datasets behind ML feature pipelines and the lakehouse layer powering self-serve analytics\nWrite performant Spark/PySpark and SQL; optimize partitioning, storage formats, and query cost\n3. Data Quality & Reliability (10%)\nOwn data quality: validation, freshness/SLA monitoring, and observability so bad data is caught before it reaches consumers\nMake the data layer debuggable: lineage, tests, and alerting that tell you what broke and where\nReason about failure modes across the whole path (queue, worker, orchestrator, database, object store) and design so that a partial failure leaves the system in a state you can recover from\n4. Relational & Operational Data (30%)\nContribute to the schema, indexing, and query performance of the relational database the product runs on\nDesign tables and constraints so that correctness is enforced at the database layer, and diagnose slow queries from their plans\nOwn retention and the boundary between the operational database and the lakehouse: what stays, what moves, and how it gets there\n\nWhat Success Looks Like\nFirst 30 days: Productive in the codebase and orchestration layer. First pipeline change merged.\nFirst 90 days: Independently shipping and owning pipelines. Onboarded at least one new data source end-to-end.\nFirst 6 months: Owning a lakehouse data domain, its ingestion, models, and quality, that ML and analytics teams rely on you to drive.\n\nRequirement\nMust-Have\n4+ years building production data pipelines: real systems with real consumers, not just one-off scripts\nStrong data engineering fundamentals: data modeling, batch vs. streaming, idempotency, incremental processing, partitioning.\nExpert SQL and strong Python: query optimization, window functions, clean production-quality code\nRelational database depth: you've designed schemas for a production PostgreSQL (or equivalent) system and understand normalization and when to break it, indexing strategies, transactions and isolation levels, locking, and how to read a query plan and fix the query\nDistributed systems fundamentals: at-least-once delivery and idempotent consumers, partitioning and its effect on ordering, consistency and durability trade-offs, retries, timeouts, and backpressure. You can explain what happens to in-flight work when a worker or a database node dies\nHands-on with a distributed processing engine (Spark/PySpark or equivalent) on non-trivial data volumes\nExperience with an orchestrator (Dagster, Airflow, Prefect, or equivalent) and a cloud platform (AWS/Azure)\nData-quality mindset: you build validation and monitoring into pipelines, not after something breaks\n\nStrong Signals (Nice-to-Have)\nTime-series or high-frequency sensor data at scale\nTimescaleDB or another time-series database (hypertables, continuous aggregates, compression, retention policies)\nWarehouse/lakehouse modeling (Delta/Iceberg/Snowflake/Redshift or equivalent) and file-format/partition tuning (Parquet)\nCDC / database-replication pipelines\nMessage brokers or task queues in production (Kafka, RabbitMQ, or equivalent)\nBuilding data for ML: feature pipelines, training datasets, serving consistency\ndbt or similar transformation/modeling frameworks\nDocker, Terraform/IaC, CI/CD for data\nIoT, energy, or industrial sector experience. Not required, but it compresses your ramp\n\nJob Details\nGurugram - Hybrid (2–3 days on-site)\n\nInterview Process\nTechnical Round (Python & SQL)\nSystem Design Round\nCulture Fit Round","description_format":"text","description_chars":4114,"description_truncated":false,"requirements":{"experience_years_min":4,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-10T20:09:30Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":3,"expected_fill_days":23,"reasons":["seen:3","win:early"],"computed_at":"2026-10-14T06:34:16Z"},"pay":null,"html_url":"https://alion.io/job/clanx-senior-data-engineer","json_url":"https://alion.io/job/clanx-senior-data-engineer.json","meta":{"generated_at":"2026-10-14T18:22:02Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":23854,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-15T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":673123},"rest":"https://alion.io/mcp/rest/get_company?id=673123"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fclanx-senior-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fclanx-senior-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fclanx-senior-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/clanx-senior-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fclanx-senior-data-engineer"}]}