{"id":2902329,"url":"https://alion.io/job/saleostech-senior-data-engineer","title":"Senior Data Engineer","company":{"id":4451127,"name":"Saleostech","domain":"saleostech.com","url":"https://alion.io/company/saleostech","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":null,"work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":["India"],"countries":["IN"],"hiring_countries":["IN"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":21000,"max_usd":43000,"period":"year","method":"role_seniority_country_cell","sample_n":77},"experience_years_min":6,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"AWS Step Functions","optional":false},{"name":"CI/CD","optional":false},{"name":"Dagster","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Apache Hudi","optional":true},{"name":"Apache Iceberg","optional":true},{"name":"Azure","optional":true},{"name":"dbt","optional":true},{"name":"Delta Lake","optional":true},{"name":"Machine Learning","optional":true}],"status":"live","first_seen_at":"2026-10-20T06:41:15Z","employer_posted_date":null,"last_verified_at":"2026-10-20T06:41:15Z","board_verified":false,"closed_at":null,"days_open":4,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":4},"description":"We are hiring a Senior Data Engineer for our privacy-preserving data platform. This is a highly technical, hands-on role on a team that prioritizes engineering discipline, a deep understanding of distributed systems, and verifiable production experience over theoretical knowledge. You will not just be writing PySpark syntax; you will be architecting resilient pipelines, proactively optimizing AWS cloud costs, and building secure platforms that handle highly sensitive Personally Identifiable Information (PII) at scale.\n\nResponsibilities: \nData Extraction and Curation: Build robust extractions from transactional systems and our Athena data lake; land, transform, and write curated datasets in Parquet (and modern table formats).\nPrivacy and Security: Implement complex de-identification transforms, dynamic data masking, and tokenization. Write automated tests that assert on the handling class of every single column, ensuring strict PII compliance.\nPipeline Orchestration: Own orchestration, scheduling, retries, backfills, alerting, and data-quality checks using Apache Airflow (or similar tools like Dagster/Step Functions).\nCost and Performance Optimization: Actively monitor and reduce AWS billing by tuning Spark jobs, optimizing joins (e. g., broadcast vs. shuffle), handling data skewness, shutting down idle clusters, and choosing optimal storage structures (partitioning vs. bucketing).\nProduction Reliability: Support what you build in production. Handle complex upstream schema drift, write idempotent pipelines, and troubleshoot 3 AM pipeline failures efficiently. Establish SLAs and drive Root Cause Analyses (RCAs) for pipeline outages.\nInfrastructure and Documentation: Contribute to CI/CD and infrastructure-as-code alongside our platform team. Populate the catalog and lineage, and maintain pristine written technical standards for every dataset produced.\nMentorship and Architecture: Own systems end-to-end. Architect solutions from scratch, write author-written technical standards, mentor junior engineers, and push back on designs you identify as flawed, inefficient, or insecure.\n\nRequirements: \nCore Stack: Deep, hands-on production experience with Python, SQL, and distributed processing via Apache Spark. You must understand how Spark works under the hood (memory management, Catalyst Optimizer, OOM prevention).\nAWS Ecosystem: Proven track record of building, operating, and maintaining batch pipelines in production on AWS (S3, Glue, EMR, Athena).\nEngineering Discipline: Real software engineering rigor: version control, thorough code review, automated testing, and a battle-tested understanding of idempotency and backfills.\nEvidence-Based Metrics: If you claim performance improvements or cost reductions on your CV, you must be able to explain the exact technical mechanisms used to achieve them.\nCommunication: Absolute comfort working from written technical standards and documentation rather than relying solely on verbal conversations.\n\nNice to Have: \nExperience with modern open-source data formats in production (Apache Iceberg, Delta Lake, or Hudi).\nChange Data Capture (CDC) tooling such as Debezium or AWS DMS.\nAdvanced exposure to data masking, tokenization, or AWS security tooling (Macie, GuardDuty, Security Hub).\nExperience with dbt or a comparable transformation framework.\nExperience building datasets specifically designed to feed machine learning model training and understanding how ML consumers differ from standard BI/reporting consumers.\nSkills\nAWS, Azure, apache spark, pyspark, python, spark, sql","description_format":"text","description_chars":3559,"description_truncated":false,"requirements":{"experience_years_min":6,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"India","iso":"IN","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-20T06:41:15Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":3,"expected_fill_days":29,"reasons":["seen:3","win:early"],"computed_at":"2026-10-24T06:02:35Z"},"pay":null,"html_url":"https://alion.io/job/saleostech-senior-data-engineer","json_url":"https://alion.io/job/saleostech-senior-data-engineer.json","meta":{"generated_at":"2026-10-24T12:01:17Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":17241,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-25T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4451127},"rest":"https://alion.io/mcp/rest/get_company?id=4451127"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fsaleostech-senior-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fsaleostech-senior-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fsaleostech-senior-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/saleostech-senior-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fsaleostech-senior-data-engineer"}]}