{"id":3218048,"url":"https://alion.io/job/neogencode-technologies-pvt-ltd-data-engineer-aws-pyspark-airflow","title":"Data Engineer – AWS, PySpark & Airflow","company":{"id":3798318,"name":"NeoGenCode Technologies Pvt Ltd","domain":"neogencode.com","url":"https://alion.io/company/neogencode-technologies-pvt-ltd","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Gurgaon, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":600000,"max":1600000,"currency":"INR","period":"year","gross":true,"usd_annual":16886},"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Amazon Redshift","optional":false},{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"AWS Glue","optional":false},{"name":"AWS Lambda","optional":false},{"name":"Dimensional Modeling","optional":false},{"name":"ETL/ELT","optional":false},{"name":"IAM","optional":false},{"name":"PostgreSQL","optional":false},{"name":"pySpark","optional":false},{"name":"Python","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Apache HTTP Server","optional":true},{"name":"Apache Hudi","optional":true},{"name":"Apache Iceberg","optional":true},{"name":"Apache Kafka","optional":true},{"name":"dbt","optional":true},{"name":"Delta Lake","optional":true},{"name":"GraphQL","optional":true}],"status":"live","first_seen_at":"2026-10-24T10:52:46Z","employer_posted_date":null,"last_verified_at":"2026-10-24T10:52:46Z","board_verified":false,"closed_at":null,"days_open":4,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":4},"description":"🚀 Job Title : Data Engineer – AWS, PySpark & Airflow\nExperience : 5+ Years\nIndustry : Health Insurance / BFSI\nLocation : Sector 43, Gurugram, Haryana\nWork Mode : Work From Office (5 Days a Week)\nEmployment Type : Full-Time\n\nRole Overview :\nWe are looking for an experienced Data Engineer with strong hands-on expertise in AWS Data Engineering, Data Lake Architecture, Python, SQL, PySpark, and Apache Airflow.\nThe candidate will be responsible for designing, developing, and maintaining scalable, reliable, and high-performance data pipelines using AWS cloud technologies. The role involves building end-to-end ETL / ELT workflows, implementing data transformation and processing solutions, optimizing pipeline performance, and ensuring the reliability of production data systems.\n\nMandatory Skills :\nAWS Data Engineering | Data Lake Architecture | Amazon S3 | AWS Glue | Amazon EMR | Athena | Redshift | Python | SQL | PySpark | Apache Spark | Apache Airflow | ETL/ELT | PostgreSQL | Data Modelling | Pipeline Optimization | Production Support | Root Cause Analysis\n\nKey Responsibilities :\nDesign, develop, and maintain end-to-end data pipelines for data ingestion, staging, transformation, curation, and consumption.\nBuild and manage AWS-based data lake solutions using Amazon S3, AWS Glue, Amazon EMR, Athena, and Redshift.\nDevelop, maintain, and optimize ETL / ELT pipelines using Python, SQL, Apache Spark, and PySpark.\nImplement and manage workflow orchestration using Apache Airflow, including scheduling, dependency management, retries, monitoring, and failure handling.\nIntegrate data from multiple structured and semi-structured sources.\nDesign and maintain data models, dimensional models, and analytical data marts.\nWork with relational databases, including PostgreSQL and Amazon RDS.\nOptimize Spark jobs, SQL queries, data transformations, and cloud resource utilization.\nImplement data quality checks, validation rules, and error-handling mechanisms.\nMonitor production data pipelines, troubleshoot failures, and perform root cause analysis (RCA).\nEnsure pipeline reliability, scalability, performance, and compliance with service-level agreements (SLAs).\nFollow data security, access management, governance, and sensitive-data protection best practices.\nCollaborate with business, analytics, and application teams to deliver effective data solutions.\nMaintain technical documentation and follow established data engineering standards.\n\nRequired Technical Skills :\nAWS Data Engineering : Strong hands-on experience with AWS-based data engineering and data lake architecture.\nAWS Services : Amazon S3, AWS Glue, Glue Data Catalog, Amazon EMR, Athena, Redshift, AWS Lambda, and IAM.\nProgramming : Strong proficiency in Python and SQL.\nBig Data Processing : Hands-on experience with Apache Spark and PySpark.\nWorkflow Orchestration : Experience developing and managing workflows using Apache Airflow.\nETL/ELT Development : Experience building scalable data ingestion, transformation, and processing pipelines.\nDatabase Technologies : Good understanding of relational databases and PostgreSQL concepts.\nData Modelling : Experience with data modelling, data transformation, and analytical data structures.\nProduction Support : Ability to monitor pipelines, troubleshoot failures, perform RCA, and resolve production issues.\nPerformance Optimization : Experience optimizing SQL queries, Spark jobs, and data processing workflows.\n\nGood-to-Have Skills :\nExperience with AWS Database Migration Service (DMS) and Amazon RDS.\nFamiliarity with dbt for data transformation and modelling.\nKnowledge of Delta Lake, Apache Iceberg, or Apache Hudi.\nExposure to Apache Kafka and real-time data streaming.\nFamiliarity with GraphQL, Hasura, or similar data-serving technologies.\nUnderstanding of data governance and data quality frameworks.\nExperience in the BFSI, healthcare, or health insurance domain.\n\nCandidate Requirements :\n5+ years of relevant experience in Data Engineering.\nProven hands-on experience building and maintaining production-grade data pipelines on AWS.\nStrong understanding of data lake architecture and large-scale data processing.\nPractical experience with Apache Airflow and Spark / PySpark-based transformations.\nStrong programming, analytical, debugging, and problem-solving skills.\nExperience handling production incidents, pipeline failures, and performance bottlenecks.\nAbility to take ownership of deliverables and work effectively with cross-functional teams.\nGood communication, collaboration, and technical documentation skills.\nWillingness to work from the Gurugram office five days a week.\n\nIdeal Candidate :\nThe ideal candidate is a hands-on Data Engineer who can independently build, optimize, and maintain AWS-based data pipelines, work with large datasets using PySpark, orchestrate workflows through Airflow, and resolve production issues in a business-critical environment.\nSkills\nAmazon Web Services (AWS), Data engineering, Data Lake Architecture, Amazon S3, AWS Glue, Amazon EMR, athena, Amazon Redshift, Python, SQL, PySpark, Apache Spark, Apache Airflow, ETL, ELT, PostgreSQL, Data modeling, Pipeline Optimization, Production support, Root cause analysis","description_format":"text","description_chars":5197,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Health insurance"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Consulting & Digital Transformation","Custom Software Development","AI Consulting & Integration","IT Staffing & Staff Augmentation"],"lifecycle":[{"event":"open","at":"2026-10-25T00:22:00Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":3,"expected_fill_days":28,"reasons":["seen:3","velocity","win:early"],"computed_at":"2026-10-28T05:59:10Z"},"pay":{"stated_usd_annual":16886,"is_top_pay":false},"html_url":"https://alion.io/job/neogencode-technologies-pvt-ltd-data-engineer-aws-pyspark-airflow","json_url":"https://alion.io/job/neogencode-technologies-pvt-ltd-data-engineer-aws-pyspark-airflow.json","meta":{"generated_at":"2026-10-29T01:02:48Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":1548,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-30T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3798318},"rest":"https://alion.io/mcp/rest/get_company?id=3798318"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fneogencode-technologies-pvt-ltd-data-engineer-aws-pyspark-airflow"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fneogencode-technologies-pvt-ltd-data-engineer-aws-pyspark-airflow"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fneogencode-technologies-pvt-ltd-data-engineer-aws-pyspark-airflow"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/neogencode-technologies-pvt-ltd-data-engineer-aws-pyspark-airflow\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fneogencode-technologies-pvt-ltd-data-engineer-aws-pyspark-airflow"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3218048}},{"tool":"search_jobs","arguments":{"company":"neogencode.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3218048},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}