2,362,064open jobs
147,838companies
287,832added this week
Browse all
Location
In office (Hyderabad)
Seniority
Middle · 3+ years exp
Employment
Full-Time

Confirmed on the employer's own hiring board on Nov 18, 2026. First seen by Alion on Nov 18, 2026. NPCI scores A on the Alion truth index.

Overview
Company
Impact
Profile match
The National Payments Corporation of India is the umbrella organisation for retail payment and settlement systems in India, headquartered in Mumbai. It was set up in 2008 by the Reserve Bank of India and the Indian Banks Association as a not-for-profit company and operates national platforms such as the Unified Payments Interface, the Immediate Payment Service and the RuPay card network. NPCI hires product managers, backend and Java developers, information security and identity specialists, cryptography researchers, data scientists, business development and legal staff, mainly in Mumbai and Hyderabad.
The Opportunity To design and implement real-time data streaming pipelines ensuring 24×7 availability of high-volume transactional data. Your role includes building fault-tolerant, scalable systems using modern big data technologies, integrating data from multiple sources into data lakes/lakehouses, and enabling downstream analytics. You will collaborate with internal teams (Data Science, Product, and Technology) to deliver reliable data solutions, optimize performance, and maintain compliance with NPCI standards. Job Details Job Title: Data Engineer (Streaming) Division: Data Analytics Years of Experience: 3-8 years

Education: Graduation in Computer Science/IT (preferably BE/B.Tech) or equivalent; advanced degrees are a plus

Employment Type: Full-time, Permanent Location: Hyderabad

Key

Responsibilities Design and develop real-time data pipelines using Apache Kafka and stream processing frameworks (Spark Structured Streaming / Apache Flink). Ensure 24×7 data availability with fault-tolerant, highly reliable systems. Implement ingestion, transformation, and load (ITL) patterns for data lakes/lakehouses (S3/MinIO, HDFS). Work with table formats like Iceberg, Hudi, or Paimon for ACID transactions and schema evolution. Optimize SQL queries on Trino/Hive for large-scale analytics. Develop orchestration workflows using DBT, Dagster, or Airflow for data transformations. Write efficient code in Python, Scala, and Java for data processing and automation. Collaborate with cross-functional teams to understand requirements and deliver high-quality data solutions. Monitor pipeline health, manage checkpoints, and implement observability for streaming jobs. Ensure compliance with security, governance, and audit standards.

Requirements Key Skills and

Experience Required Mandatory

Technical Skills: Apache Kafka (topics, partitions, offsets, reliability) Stream processing (Spark Structured Streaming or Apache Flink) Preferred

Technical Skills: Data Lake / Lakehouse (S3, MinIO, HDFS) Table formats: Iceberg, Hudi, Paimon SQL engines: Trino, Hive Orchestration tools: DBT, Dagster, Airflow Programming: Python, Scala, Java NoSQL databases (MongoDB, Cassandra, Redis) CI/CD (Jenkins, GitHub Actions), Linux basics BI tools: Superset, Tableau Data Quality frameworks and observability practices Other

Requirements: Strong analytical and problem-solving skills Ability to work in 24×7 environments Quick adaptability to new technologies

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,362,064 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Data Science
Similar stack
Same company
Hyderabad
Data Engineer - AI 1 hour ago
In office · Full-Time · 3+ years exp · Bachelor's Degree · Chennai
Python
SQL
Python
pySpark
Databases
Microsoft Fabric
AI/ML
Spark
Model Context Protocol
Prompt Engineering
AI Agents
DevOps
Azure
Apply
Hybrid · Full-Time · Bachelor's Degree · Bengaluru
Python
Databases
Snowflake
Databricks
Apache Kafka
AI/ML
Spark
Airflow
AI Agents
DevOps
Azure
Git
Docker
Kubernetes
Analytics
ETL/ELT
Apply
In office · 8+ years exp · Bengaluru
Python
Databases
PostgreSQL
Neo4j
pgvector
AI/ML
Model Context Protocol
AI Agents
Agentic Workflows
Analytics
Power BI
Management
UiPath
Power Automate
Apply
In office · Full-Time · 3+ years exp · Bachelor's Degree · Hyderabad
Python
SQL
Python
pySpark
Databases
Snowflake
AI/ML
Spark
DevOps
AWS
AWS Lambda
Amazon S3
AWS Step Functions
Analytics
ETL/ELT
AWS Glue
Apply
In office · Hyderabad
Python
SQL
DevOps
Rest API
GCP
Azure
CI/CD
AWS
SOAP
Analytics
ETL/ELT
Informatica
Management
Confluence
Agile
Apply
Sr. Data Engineer 1 day ago
In office · 5+ years exp · Hyderabad
Python
SQL
Python
pySpark
Databases
Databricks
Microsoft Fabric
AI/ML
Spark
dbt
DevOps
Azure DevOps
Azure
Analytics
Power BI
Azure Data Factory
Master Data Management
Management
Agile
Apply
In office · Full-Time · 4+ years exp · Hyderabad
Databases
Databricks
AI/ML
LangGraph
LangChain
DSPy
MLFlow
AI Agents
Machine Learning
DevOps
Git
GitHub
Apply
Remote (India) · Full-Time · 4+ years exp · Hyderabad
Python
SQL
Databases
Databricks
AI/ML
Spark
Model Context Protocol
AI Agents
RAG
Agentic Workflows
Multi-Agent Systems
Machine Learning
DevOps
Azure
CI/CD
Docker
AIOps
Chips/EDA
Advantest / Teradyne ATE
Apply
In office · 5+ years exp · Hyderabad
SQL
Databases
Oracle
DevOps
Rest API
SOAP
Management
Oracle ERP
Apply
In office · 5+ years exp · Hyderabad
SQL
Databases
Oracle
Analytics
ETL/ELT
Apply
See all jobs
This is one of many
2,362,064 more open roles from verified company boards, updated every day.