Senior Data Engineer

Role Overview :

The Senior Data Engineer Financial Integrations will be responsible for building, operating and continuously improving reliable data and system integrations across member institution Core Banking Systems (CBS), firm's central data and technology platforms, payment and financial systems, regulatory interfaces and other third-party platforms.

The role requires strong hands-on engineering capability across APIs, data pipelines, messaging/event-driven systems, data validation, reconciliation, exception management and production operations.

The engineer will work across heterogeneous technology environments and contribute to standardised, reusable integration patterns that enable firm to onboard and operate multiple financial institutions at scale.

Educational Qualifications :

B.Tech / B.E. / MCA / Diploma in IT/CS or equivalent.

Relevant professional certifications or equivalent practical experience in data engineering, cloud, integration or financial technology will be an advantage.

Primary Job Responsibilities :

1. Financial System & CBS Integrations :

- Build and operate integrations between member institution Core Banking Systems and firm's platforms, covering data extraction, transformation, validation, transmission and loading.

- Develop and maintain integrations with banking, payment, regulatory, fraud-reporting and other ecosystem systems.

- Work with heterogeneous CBS and third-party technology environments and implement standardised integration patterns.

- Support both batch and near-real-time integration requirements based on business, operational and regulatory timelines.

2. API & Event-Driven Integrations :

- Design, develop and consume secure APIs and services using REST and applicable enterprise integration standards.

- Build integrations using message queues and event-driven architectures.

- Implement appropriate patterns for retries, timeouts, idempotency, duplicate handling, ordering, dead-letter processing and replay.

- Support API versioning, schema evolution and backward compatibility for live interfaces.

- Contribute to reusable integration components and frameworks to reduce bespoke development across institutions.

3. Data Validation & Quality :

- Implement validation and data-quality checks for inbound and outbound data flows.

- Identify and handle incomplete, invalid, duplicate, inconsistent or delayed records.

- Define transformation and mapping rules between source-system data structures and SSPs standardised data models.

- Support data profiling, quality assessment and resolution of source-system data issues.

- Ensure data integrity throughout the integration lifecycle.

4. Reconciliation & Exception Management :

- Own transaction-level and data-flow reconciliation for inbound and outbound integrations.

- Implement mechanisms for identifying missing, duplicate, delayed, rejected or mismatched records.

- Build and maintain exception-management workflows covering detection, investigation, resolution and closure.

- Maintain appropriate evidence and audit trails for reconciliation and exception handling.

- Support operational teams in resolving reconciliation breaks and recurring data-quality issues.

5. Reliability, Monitoring & Production Operations :

- Build integrations with appropriate failure-handling and recovery mechanisms.

- Implement monitoring, logging, metrics and alerting for integration health, throughput, latency, failures and data-quality exceptions.

- Diagnose and resolve production incidents involving connectivity, APIs, data pipelines, messaging and downstream systems.

- Support root-cause analysis and implementation of preventive and corrective actions.

- Ensure production interfaces operate within defined service levels and operational requirements.

6. Institution Onboarding :

- Support onboarding of new member institutions onto firm's platform.

- Perform connectivity assessment, interface configuration, data mapping and interface testing.

- Coordinate integration certification, reconciliation validation, production cutover and post-cutover support.

- Support migration from institution-specific integration approaches towards standardised firm's integration patterns.

- Contribute to repeatable onboarding frameworks and checklists that enable efficient onboarding at scale.

7. Integration Security :

- Implement secure connectivity and data exchange mechanisms for financial and sensitive data.

- Work with authentication, authorisation, encryption, TLS/mTLS, secure file transfer and related security controls.

- Follow security, access-control, secrets-management and data-protection requirements.

- Ensure appropriate logging and traceability for production integration activities.

8. Integration Standards & Documentation :

- Maintain interface specifications, API contracts, data mappings, schemas, transformation rules and integration documentation.

- Maintain operational runbooks covering monitoring, incident response, recovery and reconciliation.

- Document dependencies, failure scenarios and operational procedures for every production interface.

- Follow common integration architecture, interface standards, canonical data models, security controls and change-management processes.

9. Engineering Practices :

- Follow software engineering practices including version control, code reviews, automated testing and CI/CD.

- Develop unit, integration and interface-level test cases for production integrations.

- Participate in technical design discussions and peer reviews.

- Continuously improve integration reliability, scalability, maintainability and operational efficiency.

10. Stakeholder & Vendor Coordination :

- Coordinate with member institution technology teams, CBS vendors, technology partners and other external stakeholders.

- Investigate and resolve connectivity, data, interface and production issues across multiple external systems.

- Participate in integration planning, testing, release and incident-management activities with internal and external teams.

- Communicate technical issues, dependencies, risks and resolution plans clearly to relevant stakeholders.

Professional Skills & Experience :

Mandatory :

- 5 - 7 years of experience in data engineering, integration engineering, backend engineering or systems integration.

- Meaningful production experience with live financial systems, preferably banking, payments or financial technology platforms.

- Strong SQL and Python skills.

- Hands-on experience developing and consuming REST APIs and integrating enterprise systems.

- Production experience with message queues or event-driven systems.

- Hands-on experience integrating with Core Banking Systems, payment systems or comparable financial platforms.

- Strong understanding of transaction-level reconciliation, exception management and data validation.

- Experience building and operating near-real-time or event-driven data/integration pipelines.

- Experience with production monitoring, alerting, incident handling and failure recovery.

- Working knowledge of secure data transfer, authentication, encryption and secure integration practices.

- Ability to work with multiple technology teams, vendors and external stakeholders concurrently.

Strongly Preferred :

- Experience with Kafka, Pulsar, RabbitMQ, AMQP or equivalent messaging platforms.

- Experience with API gateways and enterprise integration platforms.

- Experience with banking/payment protocols, regulatory interfaces or financial reporting systems.

- Experience with data-quality frameworks and data lineage.

- Experience with cloud platforms and containerised environments such as Docker/Kubernetes.

- Familiarity with CI/CD and automated testing practices.

- Experience working with heterogeneous CBS environments and large-scale institution onboarding.

- Experience in production environments where high availability, auditability and transaction integrity are critical.

Key Attributes :

- Strong ownership and production mindset.

- Ability to troubleshoot complex integration and data-flow issues systematically.

- Strong attention to data accuracy, transaction integrity and operational reliability.

- Comfortable working in a heterogeneous technology environment.

- Ability to convert integration requirements into robust, maintainable engineering solutions.

- Strong collaboration skills with internal teams, banks, CBS vendors and technology partners.

- Willingness to work in a fast-scaling, mission-driven environment where systems are being built and standardised from the ground up.

Skills

Data Engineering, Data Integration, System Integration, Financial Information System, Data Validation, Data Profiling, SQL, Python, Payment System, Finacle E-Banking Application