{"id":4515669,"url":"https://alion.io/job/jpmorganchase-lead-data-engineer-python-pyspark-aws-ai","title":"Lead Data Engineer - Python, Pyspark, AWS, AI","company":{"id":257,"name":"JPMorganChase","domain":"jpmorganchase.com","url":"https://alion.io/company/jpmorganchase","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Oracle","truth_index":{"grade":"A","score":87,"open_postings":1137,"ghost_share":0.014,"stale_share":0.631,"repost_share":0.06,"time_to_fill_p50_days":15,"computed_at":"2026-11-18T05:48:20Z"}},"role":"Data Science","role_family":"Data Science","seniority":"lead","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Hyderabad, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Amazon CloudWatch","optional":false},{"name":"Amazon EventBridge","optional":false},{"name":"Amazon S3","optional":false},{"name":"AWS","optional":false},{"name":"AWS Step Functions","optional":false},{"name":"CI/CD","optional":false},{"name":"Copilot","optional":false},{"name":"IAM","optional":false},{"name":"LLM","optional":false},{"name":"pySpark","optional":false},{"name":"Snowflake","optional":false},{"name":"Python","optional":true},{"name":"Spark","optional":true}],"status":"live","first_seen_at":"2026-11-17T14:23:24Z","employer_posted_date":"2026-11-17","last_verified_at":"2026-11-18T06:58:55Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"Join us as we embark on a journey of collaboration and innovation, where your unique skills and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference.\nAs a Lead Data Engineer at JPMorganChase within the Consumer and Community Banking, you are an integral part of an agile team that works to enhance, build, and deliver data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. As a core technical contributor, you are responsible for maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm’s business objectives.\nJob responsibilities\nBuild and operate scalable data pipelines on AWS using PySpark, with deep hands-on expertise across AWS data and orchestration services. Will be a technical owner within the Data Delivery Platform, driving end-to-end delivery through strong engineering execution and stakeholder partnership (without people management responsibilities). The core stack includes AWS, PySpark, and Snowflake, with emphasis on AI-assisted engineering (Copilot-style tools and LLM-based automations).\nDelivers data collection, storage, access, and analytics data platform solutions in a secure, stable, and scalable way\nUses enterprise-authorized AI capabilities within the work environment to accelerate data platform and model design analysis and documentation, validating outputs and handling data according to sensitivity and security requirements.\nGenerates data models for the team using firmwide approaches, linear algebra, statistical and geometrical algorithms\nImplements database back-up, recovery, and archiving strategy\nEvaluates and reports on access control processes to determine effectiveness of data asset security with minimal supervision\nApplies reuse-first, AI-assisted practices within delivery and operational routines (e.g., backup/recovery validation and access control review support), ensuring traceability/auditability and alignment to resiliency and security expectations.\nRequired qualifications, capabilities, and skills\nFormal training or certification on data engineering concepts and 5+ years applied experience \n Design & build data pipelines (DPL): Own the architecture and implementation of robust batch and event-driven pipelines using PySpark on AWS (ingestion, transformation, publishing, SLAs).\nAWS services implementation: Hands-on delivery using AWS services such as:\nS3 (data lake design, partitioning, lifecycle)\nAthena (query patterns, optimization, metadata integration)\nStep Functions (workflow orchestration, retries, error handling)\nAirflow (DAG design, scheduling, backfills, dependency management)\nEventBridge (event triggers, routing, integrations)\nRDS (source integrations, extraction patterns, performance considerations)\n\nSupporting services (IAM, CloudWatch, KMS, networking) as needed\n\nSnowflake engineering: Build curated layers and data products in Snowflake (loading patterns, data modeling, performance/cost tuning, access controls).\nOperational excellence: Implement monitoring, alerting, logging, runbooks, and on-call-ready practices; lead root-cause analysis and remediation for pipeline incidents.\nData quality & governance: Embed data validation checks, schema/versioning discipline, metadata/lineage readiness, and secure handling of sensitive data.\nEngineering standards: Drive best practices for code quality, testing, CI/CD, documentation, and reusable pipeline frameworks/patterns.\nAI automations: Apply Copilot/LLM-based techniques to accelerate delivery (e.g., code scaffolding, test generation, documentation/runbook automation, operational assistants) with appropriate controls.\nDemonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows with strong validation habits and awareness of data sensitivity.\nAbility to review and validate AI-assisted outputs (e.g., model/design summaries or operational checklists) before use, escalating when uncertain and following data handling requirements.","description_format":"text","description_chars":4140,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Wealth Management & Financial Advisors","Asset Management & Funds","Investment Banking & M&A Advisory"],"lifecycle":[{"event":"open","at":"2026-11-18T06:54:47Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":15,"reasons":["conf:2","win:early","comp:brand"],"computed_at":"2026-11-18T09:02:14Z"},"pay":null,"html_url":"https://alion.io/job/jpmorganchase-lead-data-engineer-python-pyspark-aws-ai","json_url":"https://alion.io/job/jpmorganchase-lead-data-engineer-python-pyspark-aws-ai.json","meta":{"generated_at":"2026-11-18T09:02:14Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":585,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-19T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":257},"rest":"https://alion.io/mcp/rest/get_company?id=257"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fjpmorganchase-lead-data-engineer-python-pyspark-aws-ai"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fjpmorganchase-lead-data-engineer-python-pyspark-aws-ai"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fjpmorganchase-lead-data-engineer-python-pyspark-aws-ai"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/jpmorganchase-lead-data-engineer-python-pyspark-aws-ai\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fjpmorganchase-lead-data-engineer-python-pyspark-aws-ai"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4515669}},{"tool":"search_jobs","arguments":{"company":"jpmorganchase.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4515669},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}