Senior Technical Consultant

Job Summary

seeking an experienced Data Engineer to design, develop, and optimize scalable enterprise data pipelines and transformation solutions on Google Cloud Platform (GCP).

Key Responsibilities

Data Engineering & Development

• Design and develop scalable data ingestion, transformation, and processing pipelines on GCP.
• Build reusable engineering frameworks and patterns for batch, incremental, and Change Data Capture (CDC) workloads.
• Develop and maintain data transformations using BigQuery, Dataform, SQL, and Python.
• Convert business and technical requirements into efficient data engineering solutions.
• Support high-volume enterprise data migration and modernization initiatives.

GCP Data Platform

• Design and implement solutions using GCP services such as:
• BigQuery
• Cloud Storage
• Pub/Sub
• Dataflow
• Dataproc
• Cloud Composer
• Dataform
• Optimize BigQuery datasets, queries, partitioning, clustering, and overall workload performance.

Data Integration & CDC

• Design and implement batch and near-real-time data integration patterns.
• Build CDC pipelines from enterprise source systems into GCP.
• Handle inserts, updates, deletes, schema changes, incremental loads, and error/reprocessing scenarios.
• Experience with CDC/integration technologies such as Qlik Replicate/Qlik CDC, Datastream, Informatica, or equivalent is advantageous.

Data Quality & Reconciliation

• Implement data validation and reconciliation between source and target systems.
• Develop automated controls covering row counts, duplicates, nulls, aggregates, business rules, and transformation accuracy.
• Troubleshoot data discrepancies and work with source-system, QA, and business teams to resolve issues.
• Support SIT, UAT, migration rehearsals, production cutover, and hypercare.

DevOps & Automation

• Implement CI/CD practices for data engineering workloads.
• Maintain source code and deployment artifacts using Git/GitHub.
• Integrate automated testing and deployment into pipelines using tools such as GitHub Actions, Cloud Build, Jenkins, or equivalent.
• Promote reusable coding standards, parameterized solutions, and automated deployments across environments.

Technical Leadership

• Provide technical guidance and mentoring to junior data engineers.
• Review solution designs, code, SQL, transformation logic, and deployment artifacts.
• Define reusable patterns and data engineering best practices.
• Collaborate with Data Architects, Platform Engineers, Data Modelers, DBAs, QA teams, and business stakeholders.
• Support technical troubleshooting and root-cause analysis of complex data issues.

Skill Requirements

Core skills

• Google Cloud Platform (GCP)
• BigQuery
• Dataform
• Advanced SQL
• Python
• Data integration / ETL / ELT
• CDC patterns
• Data modeling concepts
• Git / GitHub
• CI/CD
• Data reconciliation and testing

Other Requirements

• 5- 9 years of overall data engineering/ETL experience with strong hands-on cloud data engineering expertise.
• Strong experience working with large datasets and complex SQL transformations.
• Experience delivering enterprise data migration, modernization, or analytics solutions.
• Strong analytical, troubleshooting, communication, and stakeholder-management skills.

Preferred Certifications

• Google Cloud Professional Data Engineer