Data Platform Architect - Data Engineering

JOB OVERVIEW:

This role owns the architecture, engineering and operation of the enterprise data platform built on Snowflake. The incumbent is a hands-on Data Platform Architect who will design and build scalable ingestion and transformation pipelines, define the Snowflake account, security and cost model, and establish a governed, reusable data foundation that powers reporting, analytics and AI across the organisation.

EXPERIENCE:

At least 7 years in data engineering - data platform roles, with at least 4 years of hands-on Snowflake engineering and architecture, including at least 2 years in an architect or technical lead capacity.

KEY ROLES AND RESPONSIBILITIES:

- Snowflake Platform Architecture & Ownership: Own the end-to-end architecture, standards and roadmap of the enterprise Snowflake platform. Define account, database, schema and warehouse topology; environment strategy across development, test and production; the RBAC and access model; and naming and deployment standards. Act as the final technical authority on Snowflake design decisions and conduct architecture and code reviews.

- Data Engineering Delivery: Design, build and operationalise ingestion and transformation pipelines that bring data from lending, collections, treasury, finance, customer and third-party bureau sources into Snowflake. Implement ELT using dbt and Snowpark with Streams, Tasks and Dynamic Tables, and establish a layered raw-to-curated-to-consumption architecture with reusable, certified data models and a governed semantic layer.

- Performance & Cost Optimisation: Continuously tune query, warehouse and storage performance. Establish resource monitors, budgets and consumption dashboards, and drive measurable reduction in credit consumption per workload while meeting agreed SLAs. Own capacity planning for growth in data volume and user concurrency.

- Data Reliability, Quality & Operations: Define and enforce pipeline SLAs, freshness and reconciliation controls. Build automated data quality testing, lineage and observability so that business and regulatory reporting is demonstrably accurate and auditable. Own production escalation and root-cause resolution for the data platform.

- Data Governance, Security & Compliance: Establish and enforce governance covering data quality, security, privacy, lineage, retention and classification. Implement masking, row-level security and tag-based policies for PII and sensitive financial data in line with RBI/NHB and DPDP requirements. Partner with Information Security, Risk and Compliance, and support internal and regulatory audits.

- Analytics, BI & AI Enablement: Enable enterprise-wide analytics, executive dashboards and self-service consumption on the Snowflake foundation with governed KPI definitions. Build and scale AI and GenAI capabilities including intelligent assistants, knowledge platforms, RAG solutions and workflow automation, leveraging Snowflake Cortex and enterprise data assets with appropriate AI governance, monitoring and evaluation.

- Engineering Practice & Innovation: Establish CI/CD, version control, automated testing and Infrastructure-as-Code as the default way of working for the data team. Mentor data engineers, raise the engineering bar, and evaluate emerging Snowflake and cloud capabilities for adoption.

GOOD TO HAVE:

- Prior experience in BFSI, NBFC or housing finance data platforms

- Additional SnowPro Advanced tracks beyond the mandatory certification

- AWS Certified Solutions Architect or AWS Certified Data Engineer

- dbt certification; experience with Kafka, Iceberg and open table format

SKILLS AND COMPETENCIES:

Core Snowflake (Primary):

- Snowflake architecture: virtual warehouses, micro-partitions, clustering keys, caching layers, query profile analysis and performance tuning

- Snowflake data engineering features: Snowpipe and Snowpipe Streaming, Streams & Tasks, Dynamic Tables, Materialized Views, External and Iceberg Tables, Time Travel, Zero-Copy Cloning, Fail-safe

- Snowpark (Python) for transformation and data application workloads; UDFs, UDTFs and Stored Procedures

- Snowflake security and governance: RBAC design, Secure Views, Row Access Policies, Dynamic Data Masking, tag-based masking, Object Tagging, Data Classification, Access History, network policies

- Cost and capacity management: warehouse right-sizing, auto-suspend/resume, multi-cluster scaling, resource monitors, credit consumption analysis and chargeback

- Secure Data Sharing, Reader Accounts, Marketplace, cross-region replication, failover and disaster recovery

Data Engineering:

- Design and ownership of batch, micro-batch and streaming ingestion pipelines at enterprise scale

- ELT/ETL orchestration using dbt, Airflow or equivalent; CI/CD for data with Git-based workflows, automated testing and environment promotion

- Strong production-grade SQL and Python; performance engineering on high-volume datasets

- Data modelling: dimensional modelling, Data Vault 2.0, medallion/layered architectures, semantic layer design, slowly changing dimensions

- Change Data Capture from core lending systems (LOS, LMS, core banking, collections, CRM) using AWS DMS, Fivetran, Qlik Replicate, Debezium or Kafka

- Data quality engineering: reconciliation controls, automated validation, anomaly detection, SLA and freshness monitoring, lineage capture

- Semi-structured data handling (JSON, XML, Parquet, Avro) using VARIANT and schema-on-read patterns

Cloud & Platform:

- AWS (preferred): S3, Glue, Lambda, IAM, Kinesis/MSK, Secrets Manager, VPC, PrivateLink; Infrastructure-as-Code using Terraform

- Platform reliability: observability, alerting, incident management, DR and RPO/RTO design for the data estate

Governance, BI & AI:

- Data governance, cataloguing, metadata management and end-to-end lineage

- Regulatory context: RBI/NHB guidance, DPDP Act, data residency, retention and audit requirements for a housing finance company

- BI enablement on Snowflake (Power BI, Tableau or equivalent), KPI governance and self-service semantic layers

- AI/GenAI enablement on Snowflake: Cortex functions, vector data types and embeddings, RAG patterns, feature stores and responsible-AI practices

Behavioural:

- Stakeholder management and business partnering

- Strategic thinking and translation of business requirements into engineering outcomes

- Mentoring and technical leadership of data engineers

Skills

Data Engineering, Python, SQL, AWS, Data Modeling, Terraform, Data Governance, Machine Learning, Snowflake DB, Database Architecture