AI Native Data Engineer

Job Description

AI-Native Data Engineer

Role Overview

We are looking for an AI-Native Data Engineer to design, build, and maintain scalable data pipelines, integrations, and data solutions that support modern digital banking applications and enterprise platforms.

This role combines data engineering, data integration, data quality management, and AI-assisted development to transform raw data into reliable, accessible, and business-ready information. The ideal candidate can translate business and technical requirements into efficient data solutions while leveraging AI tools to accelerate development, improve data quality, and enhance engineering productivity.

The role works closely with Solutions Architects, Data Architects, Technical Leads, and development teams to implement data architectures, support application integrations, and ensure the reliability, consistency, and availability of data across enterprise systems.

This role serves as a key contributor to data-driven product development, helping organizations adopt AI-native approaches to data engineering, automation, and digital transformation.

Responsibilities

Build, maintain, and optimize data pipelines and integrations supporting enterprise applications, digital banking features, and business processes.

Prepare, extract, transform, validate, and manage data required for application features, customer journeys, operational processes, and reporting.

Implement data integration solutions connecting core banking systems, Cards Management Systems, mobile applications, databases, APIs, and downstream platforms.

Develop and maintain data models, transformation logic, and data processing workflows in alignment with established architecture and engineering standards.

Implement data quality checks, validation rules, monitoring, and error-handling mechanisms to ensure data accuracy, consistency, and reliability.

Collaborate with Solutions Architects, Data Architects, Technical Leads, and software engineers to translate architecture designs into working data solutions.

Support data-related testing, defect investigation, troubleshooting, and resolution throughout the software development lifecycle.

Ensure data pipelines and integrations are secure, scalable, maintainable, and performant.

Leverage AI-assisted development tools and automation to accelerate data engineering activities, improve code quality, and increase engineering productivity.

Maintain technical documentation covering data pipelines, transformations, integrations, data models, and operational procedures.

Qualifications

Minimum 3 years of experience in data engineering, software engineering, or a related technical role.

Strong programming and scripting skills using Python, SQL, or equivalent technologies.

Hands-on experience designing, building, and maintaining data pipelines and ETL/ELT processes.

Strong understanding of relational databases, SQL, data modeling, and database performance optimization.

Experience integrating enterprise systems through APIs, messaging platforms, databases, and other data exchange mechanisms.

Familiarity with data processing frameworks and tools such as Apache Spark, Kafka, Airflow, or equivalent technologies.

Experience working with cloud-based data services and platforms such as AWS, Microsoft Azure, or Google Cloud Platform (GCP).

Understanding of data quality management, data validation, data integrity, and data governance practices.

Familiarity with CI/CD, version control, automated testing, and DevOps practices.

Experience using AI-assisted development tools, LLMs, or AI-powered automation to improve engineering workflows.

Strong analytical, problem-solving, communication, and collaboration skills.

Detailed Expectations

1. Data Pipeline Development & Management

Design, build, maintain, and optimize data pipelines supporting enterprise applications and business requirements.

Develop ETL/ELT processes for extracting, transforming, processing, and loading data across different systems.

Implement batch and real-time data processing solutions based on application and business requirements.

Ensure pipelines are reliable, scalable, performant, and maintainable.

Monitor pipeline execution, investigate failures, and implement appropriate error-handling and recovery mechanisms.

2. Data Integration & Transformation

Build and maintain data integrations between core platforms, Cards Management Systems, mobile applications, databases, APIs, and downstream systems.

Prepare, transform, and validate data required for digital banking features, including SB App Cards journeys and reporting.

Implement data mapping, transformation logic, and interface specifications in alignment with approved data models and architecture designs.

Ensure accurate and consistent data exchange across integrated platforms.

Collaborate with application engineers to resolve integration issues and ensure seamless data flows.

3. Data Quality, Validation & Governance

Implement data quality checks, validation rules, and reconciliation mechanisms to ensure data accuracy, completeness, consistency, and integrity.

Develop monitoring mechanisms to identify data anomalies, pipeline failures, and processing issues.

Implement appropriate logging, exception handling, and alerting mechanisms.

Ensure data processing activities adhere to established security, access control, and data governance requirements.

Maintain documentation covering data definitions, transformations, lineage, validation rules, and operational procedures.

4. AI-Native Data Engineering

Leverage AI-assisted coding tools and development agents to accelerate data pipeline development, transformation, and debugging.

Utilize LLMs and AI-powered tools to automate repetitive data engineering activities, documentation, and code generation.

Explore opportunities to apply AI in data validation, anomaly detection, data quality monitoring, and pipeline optimization.

Develop reusable prompts, workflows, and automation accelerators that improve data engineering productivity.

Apply appropriate human review, testing, and validation to AI-generated code and data transformation logic.

Stay updated on emerging AI technologies, frameworks, and tools relevant to modern data engineering.

5. Testing, Troubleshooting & Release Support

Perform unit testing, integration testing, and data validation to ensure the accuracy and reliability of data pipelines and integrations.

Support end-to-end testing of application features and business journeys that depend on data integrations.

Investigate and resolve data-related defects, discrepancies, and pipeline failures.

Collaborate with Technical Leads, Solutions Architects, and QA teams to ensure data solutions meet functional and non-functional requirements.

Support deployment activities, production validation, and release readiness.

Monitor data pipeline performance and reliability following deployment and implement improvements when necessary.

6. Collaboration & Technical Execution

Work closely with Solutions Architects and Data Architects to understand and implement approved data models, integration patterns, and architecture designs.

Collaborate with Technical Leads and software engineers to define data interfaces, integration requirements, and implementation approaches.

Coordinate with business analysts, product teams, and QA teams to understand data requirements and validate expected outcomes.

Participate in technical design discussions, code reviews, and engineering refinement sessions.

Identify opportunities to improve data engineering practices, reusable components, and development standards.

7. Engineering Excellence & Continuous Improvement

Apply engineering best practices in coding, testing, version control, documentation, and deployment.

Optimize data pipelines and queries for performance, scalability, and resource efficiency.

Establish reusable components, templates, and engineering practices that improve delivery consistency.

Continuously evaluate emerging data engineering technologies, frameworks, and tools.

Identify opportunities to automate manual processes and improve data engineering efficiency.