Data quality engineer

'

: - ( - )

Experience: 5+ yrs

Location: Bengaluru, Karnataka, India

Job Type: Full-time

We are looking for an experienced Senior Data QA Engineer to ensure the accuracy, reliability, and quality of enterprise data pipelines, ETL processes, and data platforms. The role focuses on validating data transformations, business logic, KPIs, metrics, and data quality across modern platforms such as Snowflake, Databricks, and Hive.

The ideal candidate will combine strong SQL, Python, data engineering, and test automation skills with a deep understanding of data quality and validation. You will work closely with Data Engineers, Data Analysts, Product Managers, and Engineering teams to identify potential issues early and ensure reliable, production-ready data releases.

Requirements

Key Responsibilities

• Validate ETL pipelines, data transformations, business logic, and data quality across Snowflake, Databricks, Hive, and other data platforms.
• Design and execute comprehensive test scenarios for data pipelines, data warehouses, and analytical datasets.
• Translate business and technical requirements into effective test cases covering KPIs, metrics, calculations, and business rules.
• Use advanced SQL to validate large datasets, identify anomalies, reconcile data, and investigate data-quality issues.
• Partner with Data Engineers to identify potential failure points and proactively detect defects before production releases.
• Develop automated and reusable tests for data pipelines to improve coverage and reduce regression risk.
• Contribute to and enhance existing data test automation frameworks with a focus on scalability, reliability, and maintainability.
• Validate data accuracy, completeness, consistency, and integrity across source, transformation, and target systems.
• Perform regression testing and release validation for data platform changes.
• Collaborate with Data Analysts, Product Managers, Data Engineers, and Engineering teams to resolve data-quality issues.
• Support testing across batch and distributed data-processing environments.
• Use Python to develop automation scripts, validation utilities, and data-quality testing solutions.
• Integrate testing practices into CI/CD workflows to improve release quality and development velocity.
• Investigate production data issues, perform root-cause analysis, and help implement sustainable solutions.
• Maintain test documentation, validation standards, and reusable testing assets.
• Continuously improve data testing methodologies, automation coverage, and quality processes.

What Makes You a Great Fit

• 5+ years of experience in data quality, data QA, ETL testing, data engineering testing, or a closely related role.
• Strong hands-on experience validating data pipelines, ETL processes, and data warehouses in production environments.
• Expert-level SQL skills with experience working with very large datasets, including terabyte-scale data.
• Proven ability to identify data anomalies, inconsistencies, and quality issues through efficient SQL analysis.
• Strong experience with Snowflake, Databricks, Hive, or similar modern data platforms.
• Solid proficiency in Python and experience developing automated tests for data pipelines.
• Good understanding of Apache Spark, Airflow, and modern data-processing workflows.
• Strong understanding of data warehousing, ETL/ELT concepts, data transformations, and data validation.
• Familiarity with CI/CD principles and integrating automated testing into development and deployment workflows.
• Experience building or contributing to scalable and maintainable test automation frameworks.
• Knowledge of BDD frameworks such as Behave is an advantage.
• Experience working with AWS or other cloud platforms is desirable.
• Familiarity with data-quality frameworks such as Great Expectations, Deequ, or similar custom solutions is an advantage.
• Strong analytical and problem-solving skills with excellent attention to detail.
• Excellent communication and collaboration skills with the ability to work effectively across technical and business teams.
• Bachelor's degree in Computer Science, Information Technology, Engineering, or equivalent professional experience is preferred.
• Strong ownership mindset with the ability to proactively identify quality risks and drive issues through resolution.