2,413,597open jobs
150,017companies
294,924added this week
Browse all
Location
In office (Bengaluru)
Seniority
Senior · 14+ years exp

First seen by Alion on Nov 20, 2026.

Overview
Company
Impact
Profile match
Relevance Lab is an enterprise GenAI, Cloud, Data, DevOps and Automation services company. We help organizations adopt Generative AI at scale through our 7-Dimension GenAI Maturity Framework, AI Pods model, and proprietary platforms including RLCatalyst, SPECTRA, Research Gateway, and ServiceOne.

Company Overview:

We are pleased to introduce ourselves as Relevance Lab Pvt. Ltd., based in Bangalore. Relevance Lab is a privately held IT services and consulting organization that delivers innovative solutions across cloud computing, DevOps, automation, artificial intelligence (AI), machine learning (ML), and digital transformation. We partner with our clients to build scalable, secure, and future-ready technology solutions.

Position Details:

- Position: Senior Data Platform Engineer

- Work location: Bangalore

- Number of Position: 1

- Mode: FTE

- Notice period: Immediate

- Mode of Interview: MS Teams

- Year of Experience: 14+ years

- Level: Senior Individual Contributor (IC5)

- Work Schedule: Must be able to collaborate effectively with teams across India and US time zones.

About the Role:

We are looking for an experienced Senior Data Platform Engineer to design, develop, implement, and operate scalable, reliable, secure, and cost-effective data platforms. The ideal candidate will have strong hands-on experience in data engineering, distributed systems, cloud infrastructure, data pipelines, and production software engineering. The role involves building and maintaining the underlying infrastructure responsible for data ingestion, processing, storage, transformation, and accessibility across enterprise systems. The engineer will work closely with data engineering, analytics, security, compliance, and application engineering teams to deliver high-quality data platform capabilities that support business intelligence, product analytics, operational efficiency, and regulatory reporting. The ideal candidate should possess strong programming skills, particularly in Python, along with relevant experience in technologies such as PySpark, Snowflake, AWS, Apache Kafka, Kubernetes, and infrastructure automation. Experience with the entire technology stack is not mandatory; however, candidates must demonstrate deep production experience in at least one relevant data or infrastructure domain and the ability to learn new technologies independently.

Key Responsibilities:

1. Data Platform Engineering:

- Design, develop, and maintain scalable data platform components supporting enterprise data ingestion, movement, processing, storage, and consumption.

- Build and enhance data pipelines for batch processing, streaming, and near-real-time data integration.

- Develop reusable frameworks and automation components for data ingestion, transformation, validation, reconciliation, logging, and error handling.

- Work with data warehouses, data lakes, relational databases, and distributed processing systems to support business-critical data workloads.

- Evaluate suitable technologies and architectural approaches based on scalability, reliability, maintainability, performance, and cost.

- Improve existing data platform architecture and modernize legacy data processing solutions where required.

2. Data Engineering and Distributed Processing:

- Develop and optimize data processing solutions using Python, PySpark, Spark SQL, and other suitable programming technologies.

- Design and maintain ETL/ELT pipelines for structured and semi-structured data.

- Implement batch and streaming data processing using Apache Kafka or similar messaging technologies.

- Build efficient data transformation and ingestion workflows that support large data volumes and downstream applications.

- Develop data quality checks, validation mechanisms, reconciliation processes, and exception-handling frameworks.

- Troubleshoot pipeline failures, processing bottlenecks, data inconsistencies, and performance issues.

- Work with cloud data platforms such as Snowflake and data lake technologies to support scalable data storage and processing.

3. Cloud Infrastructure and Platform Automation:

- Design and support cloud-based data infrastructure, particularly on AWS.

- Work with cloud storage, compute, networking, and data services to build reliable and cost-efficient platform solutions.

- Use infrastructure-as-code tools such as Terraform or Terragrunt to automate infrastructure provisioning and configuration.

- Work with containerization and orchestration technologies such as Docker and Kubernetes where applicable.

- Develop reusable infrastructure modules, engineering utilities, and automation frameworks to improve developer productivity.

- Implement CI/CD pipelines to automate code validation, testing, deployment, and release management.

- Improve deployment safety through automated validation, version control, rollback mechanisms, and repeatable delivery practices.

4. Software Engineering and Development:

- Write clean, modular, testable, maintainable, and production-ready code using Python or another general-purpose programming language.

- Apply sound software engineering principles, including object-oriented design where appropriate, modular architecture, exception handling, and automated testing.

- Participate in code reviews, technical design discussions, and engineering best-practice initiatives.

- Build reusable libraries, frameworks, and utilities that can be adopted across engineering teams.

- Automate repetitive operational tasks and improve the efficiency of development and support workflows.

- Diagnose complex technical issues and implement sustainable solutions rather than relying on temporary fixes.

5. Performance Optimization and Production Operations:

- Monitor data platform performance, availability, reliability, and resource utilization.

- Optimize Spark jobs, SQL queries, data models, partitioning strategies, and cloud resource configurations.

- Identify performance bottlenecks and improve processing efficiency and infrastructure cost.

- Implement observability, logging, monitoring, alerting, and operational readiness practices.

- Participate in incident investigation, root-cause analysis, remediation, and recovery planning.

- Design systems with appropriate failure handling, resilience, backup, recovery, and safe deployment practices.

- Take end-to-end ownership of assigned platform components from problem definition and design through implementation, rollout, and ongoing production support.

6. Data Security, Governance, and Compliance:

- Incorporate data security, privacy, access control, and governance requirements into platform architecture and implementation.

- Implement appropriate access management, data classification, masking, and protection mechanisms where required.

- Support data lineage, auditability, data quality, and governance automation.

- Work with security, compliance, and business stakeholders to ensure platform solutions meet applicable organizational and regulatory requirements.

- Contribute to the development of reliable data systems operating in regulated or security-sensitive environments.

7. AI-Assisted Engineering and Productivity:

- Actively use AI-assisted engineering tools to accelerate technical research, solution design, coding, debugging, testing, and documentation.

- Leverage tools such as GitHub Copilot, ChatGPT, Claude, or equivalent engineering assistants where appropriate.

- Critically validate AI-generated code, recommendations, and documentation before incorporating them into production solutions.

- Use AI responsibly while protecting confidential information, sensitive data, and proprietary code.

- Identify repeatable opportunities to incorporate AI into engineering workflows and improve development speed, quality, and operational effectiveness.

8. Technical Leadership and Collaboration:

- Independently drive complex engineering assignments with minimal day-to-day supervision.

- Break ambiguous business or technical problems into clear milestones and deliver actionable solutions.

- Communicate architectural decisions, implementation approaches, technical risks, and delivery status clearly.

- Collaborate with distributed teams across India and US time zones using effective written and asynchronous communication.

- Participate in technical design reviews, documentation, knowledge sharing, and engineering best-practice initiatives.

- Mentor engineers and help improve the technical capability of the wider team where appropriate.

Required Technical Skills:

1. Core Data Engineering:

- Strong hands-on experience in data engineering, data platform development, or data infrastructure.

- Data ingestion, ETL/ELT, data transformation, and pipeline orchestration.

- Batch processing and/or streaming data processing.

- Experience with distributed data processing or messaging technologies.

- Production experience in at least one relevant domain, such as data pipelines, data warehouses, data lakes, distributed systems, operational databases, Kubernetes, or cloud infrastructure.

2. Programming and Software Development:

- Strong proficiency in Python or another general-purpose programming language.

- Experience writing, testing, debugging, reviewing, and maintaining production-quality code.

- SQL and data processing fundamentals.

- Automation, version control, code quality, and CI/CD practices.

3. Cloud and Data Platforms:

- Experience with cloud infrastructure, preferably AWS, or equivalent cloud platforms.

- Experience with one or more relevant data technologies such as Snowflake, Apache Kafka, Apache Spark, PySpark, PostgreSQL, or data lake technologies.

- Understanding of distributed systems, scalability, fault tolerance, and system design.

4. Production Engineering:

- Experience operating production systems and resolving technical incidents.

- Knowledge of monitoring, observability, logging, deployment safety, and recovery mechanisms.

- Understanding of performance optimization, resource utilization, and infrastructure cost management.

- Ability to design reliable, secure, maintainable, and scalable solutions.

5. AI Tools:

- Active and responsible use of AI-assisted development tools in engineering workflows.

- Ability to validate AI-generated output and independently assess its correctness and suitability.

Good-to-Have Skills:

- Snowflake data warehousing, data modeling, and performance optimization.

- Apache Kafka and event-driven data architectures.

- AWS data services, cloud-native data platforms, and Lakehouse architectures.

- Databricks, Delta Lake, Spark SQL, and PySpark optimization.

- Apache Airflow or other workflow orchestration technologies.

- Terraform/Terragrunt and infrastructure as code.

- Docker, Kubernetes, Jenkins, Git, and automated CI/CD.

- PostgreSQL and other operational database technologies.

- dbt, Dagster, time-series databases, and open table formats.

- Change Data Capture (CDC), high-volume ingestion, and event streaming.

- Data governance, data classification, masking, lineage, identity integrations, and access control.

- Experience building reusable internal platforms, infrastructure modules, or developer tooling.

- Experience in fintech, banking, payments, financial services, regulatory reporting, or other regulated environments.

Experience and Qualifications:

- Senior-level experience in data engineering, data infrastructure, platform engineering, backend engineering, or distributed systems.

- Typically 10+ years of overall experience is a suggested sourcing benchmark, with substantial hands-on experience designing and operating production systems.

- Demonstrated ownership of technical projects from requirements and design through implementation, deployment, and ongoing support.

- Strong problem-solving and system-design capabilities, including the ability to evaluate architectural trade-offs involving scale, reliability, security, cost, and maintainability.

- Proven ability to learn unfamiliar technologies and become productive without extensive day-to-day supervision.

- Excellent written and verbal communication skills.

- Ability to collaborate effectively with geographically distributed engineering teams.

Skills

DevOps, Data Engineering, Distributed Systems, Cloud Infrastructure, Data Quality, Data Ingestion, Python, Snowflake DB, ETL, AWS, CI/CD Pipeline

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,413,597 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Data Science
Similar stack
Same company
Bengaluru
In office · 3+ years exp · Bachelor's Degree · Bengaluru
SAS
Life Sciences
CDISC
Apply
Hybrid · 5+ years exp · Bachelor's Degree · Bengaluru
Python
SQL
Databases
SAP HANA
Databricks
AI/ML
Embeddings
Scikit-learn
AI Agents
TensorFlow
PyTorch
LLM
RAG
Knowledge Graph
Multi-Agent Systems
Machine Learning
DevOps
GCP
Azure
AWS
Management
ServiceNow
Apply
Sr DB Engineer - SQL 3 hours ago
In office · Hyderabad
SQL
Databases
MySQL
PostgreSQL
MS SQL
Azure SQL Database
DevOps
AWS
Analytics
SSIS
Apply
In office · Full-Time · 8+ years exp · Bachelor's Degree · Mumbai
Python
SQL
Bash
Python
pySpark
Databases
Databricks
Delta Lake
Google BigQuery
AI/ML
Spark
Airflow
Great Expectations
DevOps
Terraform
GCP
GitHub Actions
Datadog
CI/CD
Jenkins
Kubernetes
Grafana
Google GKE
FinOps
IAM
Linux
Cybersecurity
SonarQube
Wiz
Dependabot
Analytics
Tableau
Power BI
ETL/ELT
Talend
Alteryx
Apply
Data Architect 2 hours ago
In office · 10+ years exp · Bachelor's Degree · Hyderabad
SQL
Databases
PostgreSQL
Oracle
Microsoft Fabric
Cybersecurity
GDPR
Analytics
Tableau
Power BI
ETL/ELT
Informatica
Talend
SSIS
Azure Data Factory
Apply
In office · Full-Time · Bengaluru
Python
SQL
Python
pySpark
Databases
Apache Kafka
AI/ML
Hadoop
Spark
Airflow
dbt
AI Agents
LLM
Machine Learning
DevOps
Terraform
AWS
Amazon S3
Analytics
ETL/ELT
AWS Glue
Apply
In office · Full-Time · 7+ years exp · Master's Degree · Bengaluru
Python
SQL
AI/ML
Machine Learning
Apply
≈ $14k – $31k per year (Estimated) · Hybrid · Full-Time · 2+ years exp · Bengaluru
AI/ML
LLM
Apply
In office · Full-Time · 5+ years exp · Bachelor's Degree · Bengaluru
Apply
In office · Full-Time · Bengaluru
AI/ML
Copilot
AWS Bedrock
Gemini
OpenAI
Copilot Studio
DevOps
Azure
AWS
Linux
Wi-Fi
Cybersecurity
CyberArk
Microsoft Entra ID
Management
ServiceNow
Agile
Apply
See all jobs
This is one of many
2,413,597 more open roles from verified company boards, updated every day.