Lead Software Engineer - Data Engineering | Data Technology
This is your chance to change the path of your career and work at one of the world's leading financial institutions.
As a Lead Software Engineer - Data Engineering at JPMorgan Chase within the Consumer & Community Banking/Data Products team, you serve as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure, stable, and scalable way. You are responsible for carrying out critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives.
Job Responsibilities:
Design, develop, and optimize large-scale ETL (Extract Transform Load) data pipelines.
Build high-quality Python applications using modular code, reusable components, logging, and automated testing.
Develop and maintain distributed data processing solutions using PySpark.
Large scale end-to-end testing design and validation.
Implement and support workflow orchestration using Control-M or Apache Airflow (MWAA).
Develop cloud-native solutions leveraging AWS services, including Glue, Athena, Lambda, and CloudWatch.
Design and manage modern data lake architectures utilizing Iceberg and/or Delta Lake.
Administer and optimize Snowflake environments, including streams, tasks, roles, and warehouses.
Participate in code reviews and champion engineering best practices, testing standards, and CI/CD processes.
Leverage approved AI-assisted development tools while ensuring secure, responsible, and compliant software delivery.
Required qualifications, capabilities, and skills
Formal training or certification on software engineering concepts and 5+ years applied experience with a strong focus on data engineering.
Experience in developing, debugging, and maintaining code in a large corporate environment with one or more modern programming languages - Python (primary) & Java (secondary)
Hands-on experience with PySpark or other distributed data processing frameworks.
Strong expertise in DBT (Data Build Tool) and modern ETL (Extract Transform Load) development practices.
Experience with workflow orchestration platforms such as Control-M or Apache Airflow (MWAA).
Expertise with AWS data services, including Glue, Athena, CloudWatch, and Lambda.
Knowledge of modern open table formats such as Iceberg and/or Delta Lake.
Experience with Snowflake administration and development.
Strong SQL skills and experience with modern database technologies.
Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.
Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.
Preferred qualifications, capabilities, and skills
Experience with Kafka, Flink, or other streaming technologies.
Familiarity with AI/ML technologies including LLMs, prompt engineering, vector search, and responsible AI practices.
Experience using AI-assisted software development tools such as GitHub Copilot, Claude, or similar technologies.
Financial services industry experience and understanding of large-scale enterprise data environments.
Experience mentoring engineers and leading technical delivery initiatives.