Senior Technical Specialist

Job Summary

Reports to

Programme Tech Lead

Capabilities covered

Standardized Data Platform Architecture; Data Toolkit Design; Data Observability, Data Validation, Data Contracts, Data Transformation, Platform Observability modules; Common CLI/SDK; Synthetic Data Generator

Location

Prudential VCC Bangalore (preferred)

Role summary

Own the common Azure Databricks and GCP architecture and the data toolkit architecture (modules, SDK, observability) and guide its deployment across LBUs.

Key Responsibilities

• Maintain the standardised cross-cloud architecture covering ingestion, processing, consumption and BI serving layers; no redesign without an approved gap.
• Maintain toolkit design (e.g. dbt, Airflow) and the common SDK architecture integrating observability, validation, transformation and contract modules.
• Guide integration of Synthetic Data Generator on UDP and deployment in PACS.
• Define versioning, backward-compatibility and release approach for the CLI/SDK and toolkit modules.
• Review LBU-specific rollout designs for conformance; document approved deviations.
• Mentor Data Engineers and platform engineers on patterns and quality gates.

Skill Requirements

Must-have skills and experience

• 7-10 years in data architecture with hands-on delivery on both Azure Databricks and GCP (BigQuery).
• Experience designing reusable engineering frameworks (SDKs, CLIs, modules, data contracts, DQ/observability).
• Strong knowledge of dbt, Airflow or equivalent orchestration and transformation tooling.
• Strong Python and SQL; solid understanding of SCD patterns and DDL generation.

Good to have

• Experience with data observability tooling and metrics APIs.
• Experience with synthetic data platforms for privacy-safe testing.

Certifications (within 90 days of assignment)

• Databricks Data Engineer Professional
• GCP Professional Data Engineer
• Azure Data Engineer / Solutions Architect
• HashiCorp Terraform Associate

Other Requirements