Senior ETL Testing/ Data Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior ETL Testing/ Data Engineer based in India.

As a Senior ETL Testing/Data Engineer, you will design, build, test, and optimize scalable data pipelines within a cloud-native environment. You will work across data integration, data warehousing, quality assurance, and automation to ensure reliable, accurate, and high-performing data flows. The role offers broad exposure to Google Cloud Platform (GCP), BigQuery, Cloud Dataflow, Cloud Composer, Snowflake, Oracle, Teradata, and other enterprise data technologies. You will combine strong programming and SQL expertise with rigorous data validation and testing practices. Working within Agile teams, you will contribute to continuous delivery through modern CI/CD tools, version control, and automated testing frameworks. Your work will help ensure that complex data sources are transformed into trusted, high-quality datasets that support business decision-making. This is an opportunity to take ownership of critical data engineering and testing initiatives in a technically diverse environment.

Accountabilities

• Design, develop, and optimize scalable Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes using GCP services such as BigQuery, Cloud Dataflow, and Cloud Data Fusion.
• Manage and orchestrate data workflows using Cloud Composer and Apache Airflow to support reliable data movement and processing.
• Design and maintain data warehousing solutions across platforms including Snowflake, Oracle, Teradata, and SQL-based databases.
• Develop efficient, maintainable code using Java and Python for complex data transformations, processing, and business logic.
• Create advanced SQL queries using complex joins, Common Table Expressions (CTEs), window functions, and other techniques to extract and manipulate data efficiently.
• Establish and execute comprehensive data testing strategies, including source-to-target reconciliation, schema validation, regression testing, and data-driven validation.
• Validate data pipelines and transformations to ensure accuracy, completeness, consistency, and reliability across source and target systems.
• Integrate data pipelines and automated testing into Continuous Integration and Continuous Delivery (CI/CD) workflows using Jenkins, Git, and GCP Cloud Build.
• Use enterprise data integration tools such as Informatica PowerCenter and Talend to support complex integration requirements.
• Collaborate within Agile teams using Jira and Confluence to manage tasks, defects, documentation, and technical specifications.
• Investigate data quality issues, pipeline failures, and processing discrepancies, implementing effective solutions to improve reliability and performance.
• Contribute to continuous improvements in data engineering, testing, deployment, automation, and quality assurance processes.

Requirements

• 5-10 years of professional experience in data engineering, ETL testing, data quality, or a related field.
• Proven hands-on experience with Google Cloud Platform (GCP), particularly BigQuery, Cloud Storage, and Cloud Composer.
• Strong experience with relational and NoSQL databases, including PostgreSQL, MySQL, Cloud SQL, Cloud Spanner, Oracle, and Snowflake.
• Advanced programming skills in Java, including JDBC and Apache Beam Software Development Kit (SDK).
• Strong Python expertise, including experience with libraries and frameworks such as Pandas and PyTest.
• Expert-level SQL skills, including complex joins, window functions, Common Table Expressions (CTEs), and PL/SQL.
• Hands-on experience with data testing and quality assurance frameworks such as TestNG, JUnit, and Behavior-Driven Development (BDD) Cucumber.
• Strong understanding of data-driven testing methodologies, source-to-target validation, schema validation, reconciliation, and regression testing.
• Familiarity with CI/CD pipelines, Git, GitHub, Maven, and defect tracking platforms such as Jira and HP Application Lifecycle Management (ALM).
• Experience with enterprise data integration technologies such as Informatica PowerCenter and Talend is advantageous.
• Strong analytical and problem-solving skills, with the ability to investigate complex data and pipeline issues.
• Ability to work effectively in Agile environments and collaborate with technical and cross-functional stakeholders.
• Strong attention to detail and commitment to data accuracy, quality, and reliability.
• Ability to work effectively in a remote environment and manage priorities independently.

Benefits

• Contract opportunity within a data engineering and technology-focused environment.
• Remote working arrangement.
• Opportunity to work with modern GCP cloud technologies and large-scale data platforms.
• Exposure to diverse enterprise databases and data warehousing technologies.
• Opportunity to work across both data engineering and ETL testing, including automated quality assurance.
• Experience with CI/CD, cloud-native data workflows, and modern DevOps practices.
• Collaboration within Agile teams using established engineering and project management practices.
• Opportunity to contribute to scalable, high-availability data infrastructure and enterprise integration initiatives.