{"id":3214131,"url":"https://alion.io/job/scaletrix-ai-data-engineer","title":"Data Engineer","company":{"id":4471731,"name":"Scaletrix.AI","domain":"scaletrix.ai","url":"https://alion.io/company/scaletrix-ai","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Gurgaon, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":21000,"max_usd":46000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":75},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"Airflow","optional":false},{"name":"Amazon CloudWatch","optional":false},{"name":"Amazon S3","optional":false},{"name":"Apache Kafka","optional":false},{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"Git","optional":false},{"name":"GitHub","optional":false},{"name":"PostgreSQL","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Azure","optional":true},{"name":"Azure Data Factory","optional":true},{"name":"Talend","optional":true}],"status":"live","first_seen_at":"2026-10-24T17:35:45Z","employer_posted_date":null,"last_verified_at":"2026-10-24T17:35:45Z","board_verified":false,"closed_at":null,"days_open":5,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":5},"description":"JOB DESCRIPTION\nExperience: 5+ Years\nJob Type: Full-Time\nDepartment: Data Engineering\nLocation: Hybrid / Gurgaon\n\nAbout the Role \nWe are looking for an experienced Data Engineer with 5+ years of hands-on experience in designing, developing, and optimizing scalable data pipelines and data processing solutions. The ideal candidate should have strong expertise in Python, PySpark, SQL, Apache Spark, Databricks, Delta Lake, and cloud platforms, along with experience in batch and real-time data processing, data orchestration, and data quality management.\nThe candidate will be responsible for building reliable ETL/ELT pipelines, implementing scalable Lakehouse architectures, optimizing data processing performance, and ensuring data accuracy, consistency, and availability for downstream analytics and business applications.\n\nKey Responsibilities\nDesign, develop, and maintain scalable ETL/ELT pipelines using Python, PySpark, SQL, Apache Spark, and Databricks.\nImplement Lakehouse architectures and Medallion Architecture (Bronze, Silver, and Gold layers) using Delta Lake.\nBuild and maintain batch and real-time data processing workflows using Apache Kafka and Spark Structured Streaming.\nDevelop incremental data loading, merge/upsert operations, and efficient data transformation strategies to minimize processing time and resource utilization.\nOptimize Spark jobs through partitioning, caching, query optimization, and performance tuning techniques.\nDevelop and manage workflow orchestration using Apache Airflow, including DAGs, dependencies, retries, monitoring, and failure notifications.\nImplement data quality frameworks covering schema validation, null checks, duplicate detection, and data consistency.\nWork with cloud data services, particularly AWS S3, AWS RDS, CloudWatch, and other relevant cloud data storage and processing services.\nDesign and optimize relational databases, SQL queries, stored procedures, and database-level business logic.\nMonitor pipeline performance, troubleshoot production issues, and ensure data reliability, scalability, and availability.\nImplement version control, CI/CD practices, and controlled releases across development, testing, and production environments.\nCollaborate with cross-functional teams and stakeholders to understand business requirements and deliver effective data engineering solutions.\nParticipate in code reviews, technical documentation, and mentoring junior team members.\n\nRequired Skills and Qualifications\nMinimum 5+ years of professional experience in Data Engineering or a related role.\nStrong programming skills in Python, PySpark, and SQL.\nHands-on experience with Apache Spark, Databricks, and Delta Lake.\nStrong understanding of ETL/ELT pipeline development, data warehousing, data modeling, and Lakehouse architecture.\nExperience with Apache Kafka and real-time data processing using Spark Structured Streaming.\nHands-on experience with Apache Airflow or similar workflow orchestration tools.\nExperience with AWS data services such as S3, RDS, and CloudWatch.\nStrong understanding of incremental data processing, data quality, and pipeline performance optimization.\nExperience working with relational databases such as PostgreSQL, SQL Server, or equivalent platforms.\nFamiliarity with Git, GitHub, CI/CD pipelines, Agile methodologies, and deployment processes.\nStrong analytical, problem-solving, communication, and collaboration skills.\n\nPreferred Qualifications\nExperience with Azure Data Lake Storage (ADLS) and Azure Data Factory.\nFamiliarity with Talend or other ETL tools.\nAWS or Databricks certifications.\nExperience working with large-scale datasets, production data pipelines, and enterprise data platforms.\nExperience mentoring junior engineers and collaborating with business stakeholders\n\nWhat we are looking for \nWe are looking for a hands-on Data Engineer who can independently design and optimize complex data pipelines, troubleshoot data processing challenges, improve platform performance, and deliver reliable data solutions. The ideal candidate should demonstrate strong technical ownership, a problem-solving mindset, and the ability to work effectively in a collaborative engineering environment.","description_format":"text","description_chars":4197,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-24T20:09:15Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":5,"expected_fill_days":28,"reasons":["seen:5","win:early"],"computed_at":"2026-10-30T05:48:23Z"},"pay":null,"html_url":"https://alion.io/job/scaletrix-ai-data-engineer","json_url":"https://alion.io/job/scaletrix-ai-data-engineer.json","meta":{"generated_at":"2026-10-30T16:02:18Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":3991,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-31T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4471731},"rest":"https://alion.io/mcp/rest/get_company?id=4471731"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fscaletrix-ai-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fscaletrix-ai-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fscaletrix-ai-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/scaletrix-ai-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fscaletrix-ai-data-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3214131}},{"tool":"search_jobs","arguments":{"company":"scaletrix.ai","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3214131},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}