Data Engineer – Python, PySpark, SQL & ETL

Job Summary

Synechron is seeking a Data Engineer with 5+ years of experience in building data marts, ETL pipelines, and large-scale data-processing solutions.

The role will focus on designing, developing, optimizing, and supporting scalable data pipelines using Python, PySpark, Spark, SQL, and data-warehousing technologies. The position will cover the end-to-end software development lifecycle, including development, testing, UAT support, production deployment, issue resolution, and post-production support.

The role will contribute to business objectives by delivering reliable and efficient data solutions, improving data quality and accessibility, supporting informed decision-making, and enabling consistent data operations across enterprise environments.

Software Requirements

Required Software Skills

• Python: Strong hands-on experience developing data-processing applications, ETL workflows, automation scripts, and production services.
• PySpark and Spark: Strong experience building distributed data-processing and ETL solutions.
• Pandas: Experience with data preparation, analysis, transformation, validation, and profiling.
• SQL: Strong experience with data extraction, transformation, validation, analysis, and query optimization.
• Oracle SQL: Strong knowledge of Oracle SQL, complex queries, joins, aggregations, subqueries, stored procedures, and performance optimization.
• ETL development: Experience designing, developing, testing, deploying, and supporting ETL and data-pipeline solutions.
• Data warehousing: Strong understanding of data-warehouse concepts, dimensional modelling, data marts, transformations, fact tables, dimensions, and data quality.
• Hadoop, Hive, and MapReduce: Experience working with distributed data-processing and big-data technologies.
• SQL and NoSQL databases: Experience working with structured, semi-structured, and unstructured data.
• Jupyter Notebook: Experience using notebooks for data analysis, development, validation, troubleshooting, and technical exploration.
• CI/CD pipelines: Familiarity with automated build, testing, deployment, and release processes.
• Testing and data validation: Experience validating data transformations, pipeline results, data quality, and application behavior.
• Software engineering practices: Understanding of coding standards, version control, documentation, code reviews, testing, and maintainable development practices.

Preferred Software Skills

• Experience with banking or financial-services data and reporting requirements.
• Exposure to cloud-based data platforms and data-processing services.
• Familiarity with workflow scheduling, orchestration, monitoring, and alerting tools.
• Experience with automated data-quality checks, reconciliation, lineage, and metadata management.
• Exposure to infrastructure automation, containerization, and modern DevOps practices.
• Familiarity with tools supporting data pipeline observability and performance analysis.

Version and Experience Guidance

• Specific software versions are not mandated in the requirements.
• Candidates should have experience with current, production-supported versions of Python, PySpark, Spark, Pandas, Oracle SQL, Hadoop, Hive, MapReduce, and related tools.
• Candidates should be able to assess compatibility, manage dependencies, support upgrades, and follow version-control practices.
• Experience should include production data engineering, pipeline support, and end-to-end SDLC participation rather than only academic or prototype development.

Overall Responsibilities

• Design, develop, maintain, and optimize scalable ETL and data-pipeline solutions using Python and PySpark.
• Build and enhance data marts and data-transformation processes.
• Develop distributed data-processing solutions using Spark, Hadoop, Hive, and MapReduce.
• Work with structured, semi-structured, and unstructured data from multiple sources.
• Develop and optimize Oracle SQL queries for data extraction, transformation, analysis, and validation.
• Analyze and debug data issues using Oracle SQL, PySpark, Pandas, and structured data-analysis techniques.
• Perform end-to-end SDLC activities, including development, testing, UAT support, bug fixing, production deployment, and post-production support.
• Collaborate with cross-functional teams to define data requirements, validate solutions, resolve defects, and improve data quality.
• Implement data validation, reconciliation, completeness, accuracy, and consistency checks.
• Support CI/CD implementation for data pipelines, applications, and deployment workflows.
• Monitor production environments and troubleshoot pipeline failures, data-quality issues, performance problems, and processing delays.
• Create and maintain technical documentation, data mappings, transformation logic, support procedures, and operational runbooks.
• Apply software engineering best practices, coding standards, testing practices, and maintainable design principles.
• Contribute to performance improvements through query optimization, efficient transformations, appropriate partitioning, resource management, and pipeline tuning.
• Identify opportunities to improve automation, reliability, data quality, maintainability, and sustainable use of processing resources.
• Contribute to measurable outcomes, including reliable data delivery, reduced processing time, improved data accuracy, fewer production incidents, and successful release execution.

Technical Skills (By Category)

Programming Languages

Essential

• Strong Python development experience.
• Strong PySpark and Spark programming experience.
• Experience using Pandas for data analysis, transformation, profiling, and validation.
• Ability to write modular, maintainable, testable, and production-ready code.
• Ability to develop reusable data-processing components, ETL logic, validation utilities, and automation scripts.

Preferred

• Experience with additional programming or scripting languages used in data engineering and distributed processing.
• Familiarity with asynchronous or event-driven data-processing patterns.
• Experience developing reusable libraries and shared data-engineering components.

Databases/Data Management

Essential

• Strong Oracle SQL experience.
• Experience with SQL and NoSQL databases.
• Understanding of data-warehouse concepts and data-mart design.
• Experience working with structured, semi-structured, and unstructured data.
• Understanding of data modelling, dimensional modelling, schemas, fact tables, dimensions, keys, joins, aggregations, and transformations.
• Experience with data quality, validation, reconciliation, completeness, accuracy, and consistency checks.
• Ability to optimize queries and investigate data-related application or pipeline issues.
• Understanding of data lineage, source-to-target mapping, metadata, data retention, and controlled data access.

Preferred

• Experience with banking or financial-services data structures and reporting requirements.
• Familiarity with data lakes, analytical data stores, and distributed data architectures.
• Experience with partitioning, indexing, compression, schema evolution, and high-volume data processing.
• Exposure to automated data-quality frameworks, data reconciliation, and metadata management.

Cloud Technologies

Essential

• Understanding of scalable data-processing and data-pipeline deployment environments.
• Ability to support development, testing, UAT, staging, and production data environments.
• Understanding of resource management, performance, availability, monitoring, and operational support for data platforms.

Preferred

• Experience with cloud-based data platforms and distributed processing services.
• Familiarity with cloud storage, managed data-processing services, data warehouses, and orchestration tools.
• Exposure to cloud-native CI/CD, infrastructure automation, and containerized data workloads.
• Understanding of processing optimization, resource utilization, and sustainable data-engineering practices.

Frameworks and Libraries

Essential

• Python.
• PySpark and Spark.
• Pandas.
• Oracle SQL.
• Hadoop.
• Hive.
• MapReduce.
• SQL and NoSQL database technologies.
• Jupyter Notebook.
• ETL and data-pipeline development frameworks.
• CI/CD tools and automated deployment workflows.
• Data validation, testing, monitoring, and troubleshooting utilities.

Preferred

• Workflow orchestration and scheduling frameworks.
• Data-quality, reconciliation, lineage, and metadata-management tools.
• Pipeline observability, logging, metrics, and alerting frameworks.
• Libraries supporting performance profiling, distributed processing, and automated testing.
• Tools supporting cloud-based data integration and analytics.

Development Tools and Methodologies

Essential

• Experience across the data-engineering lifecycle, including requirements analysis, design, development, testing, UAT support, deployment, monitoring, troubleshooting, and production support.
• Experience with ETL development and data-mart implementation.
• Familiarity with version control, code reviews, coding standards, technical documentation, and collaborative development practices.
• Experience implementing data validation, unit testing, integration testing, regression testing, and pipeline testing.
• Experience supporting CI/CD implementation for data applications and deployment workflows.
• Ability to create data mappings, transformation specifications, test plans, support procedures, and operational runbooks.
• Familiarity with Agile or iterative delivery practices.
• Experience performing root-cause analysis and resolving production data-pipeline issues.

Preferred

• Experience with automated data-pipeline deployment and release management.
• Familiarity with infrastructure as code, containers, and DevOps practices.
• Experience with pipeline monitoring, operational dashboards, alerting, and service-level reporting.
• Exposure to automated rollback, recovery, retry, and failure-handling patterns.
• Experience supporting continuous improvement and process standardization.

Security Protocols

Essential

• Understanding of secure data access, authentication, authorization, and least-privilege principles.
• Ability to protect sensitive business, customer, financial, and operational data.
• Understanding of secure database connections, credential management, data masking, encryption, and controlled data handling.
• Ability to follow security, privacy, compliance, change-management, and access-control requirements.
• Awareness of risks related to unauthorized access, data leakage, insecure scripts, exposed credentials, and uncontrolled data movement.
• Ability to maintain auditability and traceability across data transformations, deployments, and production changes.

Preferred

• Experience supporting data-security reviews, audit activities, access reviews, and compliance controls.
• Familiarity with secrets management, encryption in transit and at rest, and security monitoring.
• Experience integrating security and quality checks into CI/CD pipelines.
• Knowledge of security requirements for banking and financial-services data.

Experience Requirements

• Overall experience: Minimum 5 years of professional experience in data engineering, ETL development, data warehousing, big-data engineering, or a related technical role.
• Data-engineering experience: Strong hands-on experience building data marts, ETL pipelines, data transformations, and large-scale data-processing solutions.
• Technical experience: Strong experience with Python, PySpark, Spark, Pandas, SQL, Oracle SQL, Hadoop, Hive, MapReduce, and SQL and NoSQL databases.
• Data-warehouse experience: Experience with data-warehouse concepts, data modelling, data marts, source-to-target mapping, data transformations, and data-quality controls.
• SDLC experience: Experience across development, UAT support, bug fixing, production deployment, monitoring, and post-production support.
• Production experience: Experience troubleshooting data-pipeline failures, query issues, data-quality defects, performance problems, and production incidents.
• Domain preference: Banking and financial-services experience is highly preferred.
• Industry preference: Experience supporting enterprise-scale, high-volume, regulated, or business-critical data environments is preferred.
• Alternative experience pathways: Equivalent combinations of relevant data-engineering experience, technical training, substantial pipeline implementations, open-source contributions, or demonstrated production outcomes may be considered.
• Outcome evidence: Candidates should demonstrate contributions to reliable data delivery, improved pipeline performance, enhanced data quality, reduced processing time, successful production deployments, or reduced recurring support issues.

Day-to-Day Activities

• Design, develop, test, and optimize Python, PySpark, Spark, SQL, and ETL solutions supporting data marts and large-scale data processing.
• Analyze source data, debug Oracle SQL and PySpark issues, validate transformations, support UAT, resolve defects, and monitor production pipelines.
• Collaborate with data, application, testing, operations, and business teams through planning sessions, requirement discussions, technical reviews, release meetings, and issue-resolution activities.
• Deliver production-quality code, data mappings, test results, validation reports, deployment artifacts, technical documentation, and recommendations within the assigned scope.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, Data Science, or a related field; equivalent relevant experience may be considered.
• Minimum 5 years of hands-on experience in data engineering, ETL development, data warehousing, big-data processing, or a related technical role.
• Demonstrated experience with Python, PySpark, Spark, Pandas, Oracle SQL, ETL development, data marts, data warehousing, Hadoop, Hive, MapReduce, and SQL and NoSQL databases.
• Experience with Jupyter Notebook, CI/CD pipelines, testing, data validation, SDLC activities, production deployment, and post-production support is required.
• Commitment to continuous professional development through relevant training, technical experimentation, knowledge sharing, and ongoing learning in data engineering, distributed processing, cloud technologies, data quality, and automation.

Professional Competencies

• Applies structured analytical thinking to investigate data-quality issues, pipeline failures, query performance concerns, processing errors, and production incidents.
• Collaborates effectively with data, application, testing, operations, and business teams to deliver reliable data solutions and resolve issues.
• Communicates technical findings, data limitations, defects, delivery status, risks, and recommendations clearly to technical and non-technical stakeholders.
• Adapts to evolving data sources, processing technologies, business requirements, delivery priorities, and platform constraints through continuous learning.
• Identifies practical opportunities to improve pipeline performance, data quality, automation, maintainability, reliability, resource efficiency, and sustainability.
• Manages time and priorities effectively across development, testing, UAT support, production deployment, incident resolution, documentation, and continuous improvement.

S YNECHRON’S DIVERSITY & INCLUSION STATEMENT

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture - promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.

All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

Candidate Application Notice