2,391,362open jobs
149,182companies
289,228added this week
Browse all
Location
In office (Indore)
Seniority
Senior · 8+ years exp
Visa
Sponsors visas in United States; no register of this country names it

First seen by Alion on Nov 19, 2026.

Overview
Company
Impact
Profile match
InfoBeans is an Indore-based software engineering and digital transformation company that builds enterprise applications, AI agents and automation for clients in the United States, Europe and the Middle East. Founded in 2000 and listed on the NSE and BSE in India, it runs delivery centres in Indore, Pune, Bengaluru and Chennai, holds partner status with ServiceNow, Salesforce and SAP, and offers its own accelerators such as Expona AI. The company hires Python, MEAN stack and Salesforce developers, SAP ABAP and ServiceNow architects, DevOps engineers, Power BI and ETL specialists and technical project managers.

Job Description:

Senior Data Engineer Azure Databricks

Job Overview:

We are looking for an experienced Senior Data Engineer with strong hands-on expertise in Azure Databricks, PySpark, Apache Spark, Delta Lake and Azure data services. The role will focus on designing and developing scalable data engineering solutions, building batch and streaming pipelines, implementing Lakehouse architectures, and optimizing Spark workloads for performance, reliability and maintainability.

Key Responsibilities:

- Design, develop and maintain scalable data pipelines using Azure Databricks, PySpark and Spark SQL.

- Build and manage Lakehouse solutions using Bronze, Silver and Gold / Medallion architecture.

- Develop robust batch and streaming ETL/ELT pipelines for structured and semi-structured data.

- Work extensively with Delta Lake, including MERGE/upserts, schema evolution, data quality and incremental processing.

- Implement and manage data governance using Unity Catalog, including catalogs, schemas, permissions and access controls.

- Develop and orchestrate Databricks notebooks, jobs and workflows for reliable data processing.

- Integrate Databricks with Azure services such as ADLS Gen2, Azure Data Factory, Synapse Analytics, Event Hubs and Service Bus.

- Optimize Spark workloads using appropriate partitioning, caching, broadcast joins, shuffle optimization and other performance-tuning techniques.

- Apply Spark fundamentals such as lazy evaluation, transformations/actions, joins, repartitioning and data-skew handling.

- Design data models and curated datasets for analytics, reporting and downstream applications.

- Implement data quality checks, schema validation, monitoring, error handling and SLA-driven pipeline delivery.

- Develop solutions using Python and SQL and follow clean, modular and maintainable coding practices.

- Use Git, Azure DevOps and CI/CD practices for source control, deployment and release management.

- Collaborate with engineering, analytics and business teams to understand requirements and deliver reliable data products.

Required Technical Skills:

- Programming: Python, SQL

- Big Data: Apache Spark, PySpark, Spark SQL

- Databricks: Azure Databricks, notebooks, jobs/workflows, performance optimization

- Lakehouse: Delta Lake, Medallion architecture, incremental processing, MERGE, schema evolution

- Governance: Unity Catalog, access control, data governance

- Azure Data: ADLS Gen2, Azure Data Factory, Azure Synapse Analytics

- Messaging / Integration: Azure Event Hubs, Azure Service Bus

- DevOps: Git, Azure DevOps, CI/CD, Docker

- Databases: SQL Server, PostgreSQL, Oracle, Azure SQL

Qualifications & Experience:

- 8+ years of overall experience in software/data engineering.

- Strong hands-on experience in Azure Databricks and PySpark-based data engineering.

- Strong Python and SQL development skills.

- Demonstrated experience designing and supporting production-grade data pipelines.

- Strong analytical, troubleshooting and problem-solving skills.

- Good communication and ability to explain technical approaches clearly.

Skills

Data Engineering, Azure Databricks, PySpark, Python, SQL, Delta Lake, Azure Data Factory, Apache Spark, Azure, Synapse Analytics, ETL

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,391,362 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Data Science
Similar stack
Same company
Indore
Data Engineer 2 hours ago
In office · Full-Time · 3+ years exp · Bachelor's Degree · Chennai
Python
Java
SQL
Scala
Python
pySpark
Databases
Databricks
AI/ML
Spark
DevOps
GitHub Actions
Azure
CI/CD
GitHub
Linux
Unix
Management
Confluence
Jira
Agile
Scrum
Apply
Data Architect 26 min ago
In office · 15+ years exp · Bachelor's Degree · Noida
Python
SQL
Python
pySpark
Databases
Databricks
Delta Lake
AI/ML
Spark
MLFlow
Embeddings
RAG
DevOps
Rest API
Terraform
Azure
CI/CD
Analytics
ETL/ELT
Dimensional Modeling
Apply
In office · 5+ years exp · Navi Mumbai
Python
MATLAB
Design
SolidWorks
SpaceTech
ANSYS
STAR-CCM+
ECSS
Thermal Desktop
Apply
Data Scientist 5 hours ago
In office · Full-Time · 4+ years exp · Master's Degree · Bengaluru
Python
SQL
Databases
Snowflake
Neo4j
Databricks
AI/ML
Fine-tuning
Scikit-learn
SciPy
Multimodal AI
AI Agents
TensorFlow
Pandas
NumPy
Keras
PyTorch
LLM
RAG
LLM Guardrails
Machine Learning
DevOps
Azure DevOps
Azure
CI/CD
Analytics
Azure Data Factory
MedTech
ISO 13485
ISO 14971
IEC 62304
DICOM
HL7
FHIR
Verification & Validation
Apply
In office · 3+ years exp · Bengaluru
SQL
COBOL
DevOps
Unix
Apply
$80k – $100k per year · In office · 5+ years exp · Bachelor's Degree · Indore
Apply
In office · Full-Time · 5+ years exp · Indore
Apply
In office · Full-Time · 3+ years exp · Indore
Apply
In office · Full-Time · 3+ years exp · Indore
DevOps
Terraform
Red Hat
CloudFormation
CI/CD
Windows Server
AWS
Docker
Kubernetes
Amazon EC2
Amazon S3
IAM
Amazon CloudWatch
Windows
Cybersecurity
SOC 2
Life Sciences
LIMS
Apply
$3.2k – $3.5k per year · In office · Full-Time · Master's Degree · Indore
Analytics
Microsoft Excel
Apply
See all jobs
This is one of many
2,391,362 more open roles from verified company boards, updated every day.