2,298,056open jobs
145,848companies
270,309added this week
Browse all
Salary
≈ $20k – $42k per year (Estimated)
Location
In office (Hyderabad)
Seniority
Senior · 6+ years exp

First seen by Alion on Sep 23, 2026.

Overview
Company
Impact
Profile match
I n n o v a t i n g T o m o r r o w T o d a y. Prophecy Technologies - Unlock business potential with Prophecy Technologies—driving growth through AI optimization, digital solutions, and modernized infrastructure for seamless customer engagement.

Data Engineer with Scala

Job Summary :

We are seeking a highly skilled and motivated Data Engineer with 6+ years of experience in designing, developing, and maintaining scalable data platforms and pipelines.

The ideal candidate will possess strong expertise in Apache Spark, Scala, Python, SQL, and Cloud Technologies (Azure/AWS), with the ability to independently own and deliver end-to-end data engineering solutions.

The role requires working closely with business stakeholders, data scientists, and engineering teams to build reliable, high-performance, and scalable data ecosystems that support analytics, reporting, and advanced data-driven initiatives.

Key Responsibilities :

- Design, develop, and maintain large-scale batch and real-time data pipelines using Apache Spark, Scala, and Python.

- Build scalable and reliable data processing frameworks for ingesting, transforming, and integrating data from multiple sources.

- Develop and optimize complex SQL queries, stored procedures, and data models to support reporting and analytics requirements.

- Design and implement cloud-based data solutions using Azure and/or AWS services.

- Create and manage ETL/ELT workflows using cloud-native tools such as Azure Data Factory, Azure Data Lake, AWS Glue, and Amazon S3.

- Collaborate with business stakeholders, data analysts, and data scientists to understand requirements and deliver data solutions.

- Monitor, troubleshoot, and optimize data pipelines to ensure performance, reliability, and data quality.

- Implement best practices for data governance, security, scalability, and operational excellence.

- Participate in code reviews, architecture discussions, and technical design sessions.

- Support CI/CD implementation and automation of data engineering workflows.

- Work with distributed data processing systems and contribute to platform modernization initiatives.

- Mentor junior team members and contribute to knowledge-sharing activities within the team.

Experience Required :

- 6+ years of hands-on experience in Data Engineering, Data Warehousing, and Big Data technologies.

- Strong experience developing scalable data pipelines using Apache Spark, Scala, and Python in enterprise environments.

- Proven experience working with cloud platforms such as Microsoft Azure and/or AWS, including data storage, processing, and integration services.

- Advanced knowledge of SQL, including complex query development, performance tuning, data modeling, and query optimization.

- Experience designing and implementing end-to-end ETL/ELT workflows for large-scale data processing and analytics.

- Demonstrated ability to independently own and deliver data engineering solutions from requirements gathering through deployment and production support.

- Strong troubleshooting and problem-solving skills with experience resolving complex data quality, performance, and scalability challenges.

- Experience working within Agile/Scrum teams and collaborating effectively with cross-functional stakeholders to deliver business-critical data solutions.

Preferred to Have Skills :

- Experience with Azure Data Factory (ADF), Azure Synapse Analytics, or AWS Glue.

- Hands-on experience with Apache Kafka or other real-time streaming platforms.

- Knowledge of Delta Lake, Databricks, or Lakehouse architectures.

- Experience with CI/CD pipelines and DevOps practices for Data Engineering.

- Familiarity with Data Governance, Data Quality, and Metadata Management frameworks.

- Exposure to Generative AI, Machine Learning data pipelines, or Analytics platforms is an added advantage.

Skills

Data Engineering, Scala, Python, Apache Spark, SQL, AWS, ETL, Azure Data Factory, AWS Glue

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,298,056 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Data Science
Similar stack
Same company
Hyderabad
Data Scientist 3 hours ago
Hybrid · 2+ years exp · Bachelor's Degree · Karachi
Python
Java
AI/ML
Jupyter Notebook
Quantization
Scikit-learn
Knowledge Distillation
TensorFlow
Keras
PyTorch
Contrastive Learning
Model Distillation
Machine Learning
DevOps
GitHub
Apply
In office · 5+ years exp · United States
Python
SQL
Python
pySpark
Databases
Apache Kafka
AI/ML
Spark
dbt
DevOps
CI/CD
AWS
Kubernetes
SLI/SLO/SLA
Cybersecurity
HIPAA
MedTech
HL7
FHIR
Apply
Data Engineer II 1 hour ago
Remote (Brazil) · Full-Time · 5+ years exp · Brazil
Python
SQL
Databases
MS SQL
AI/ML
Spark
DevOps
Azure
Git
AWS
Analytics
Power BI
ETL/ELT
Apply
Remote (Hungary) · 2+ years exp · Budapest · Pécs · Debrecen · Szeged
Python
SQL
Python
pySpark
AI/ML
Hadoop
Spark
Pandas
LLM
DevOps
GCP
CI/CD
Git
GitLab
Analytics
Seaborn
Matplotlib
Plotly
Data Vault
Apply
In office · 5+ years exp · Navi Mumbai
Python
C++
SpaceTech
STAR-CCM+
OpenFOAM
Apply
Hybrid · Full-Time · 8+ years exp · Bachelor's Degree · Bengaluru
Python
SQL
Python
pySpark
Databases
Databricks
Delta Lake
AI/ML
Spark
MLFlow
SciPy
NumPy
Anomaly Detection
Time Series
Feature Store
Machine Learning
DevOps
CI/CD
Git
AWS
Docker
Platform Engineering
Analytics
ETL/ELT
Apply
Hybrid · Full-Time · 5+ years exp · Bachelor's Degree · Bengaluru
Python
SQL
Python
pySpark
Databases
Databricks
Delta Lake
AI/ML
Spark
MLFlow
SciPy
NumPy
Anomaly Detection
Time Series
Feature Store
Machine Learning
DevOps
CI/CD
Git
AWS
Docker
Analytics
ETL/ELT
Management
Agile
Apply
BI/Data Analyst 5 days ago
In office · Minsk
Python
SQL
Databases
MySQL
PostgreSQL
AI/ML
Spark
Pandas
Machine Learning
DevOps
Git
Analytics
Power BI
Seaborn
Matplotlib
ETL/ELT
Looker
Data Vault
Apply
In office · Full-Time · 5+ years exp · Singapore
Python
SQL
Analytics
Tableau
Power BI
Microsoft Excel
Apply
In office · Full-Time · 5+ years exp · Manila
Python
SQL
Analytics
Tableau
Power BI
Microsoft Excel
Apply
In office · Full-Time · 8+ years exp · Hyderabad
Python
SQL
Databases
Snowflake
AI/ML
Model Context Protocol
AI Agents
LLM
RAG
Agentic Workflows
Apply
In office · Full-Time · 8+ years exp · Hyderabad
Python
SQL
Databases
Snowflake
AI/ML
Model Context Protocol
AI Agents
LLM
RAG
Agentic Workflows
Apply
In office · Full-Time · 8+ years exp · Hyderabad
Python
SQL
Databases
Snowflake
AI/ML
Model Context Protocol
AI Agents
LLM
RAG
Agentic Workflows
Apply
≈ $21k – $45k per year (Estimated) · Hybrid · Full-Time · 5+ years exp · Bengaluru
Python
SQL
Databases
PostgreSQL
DevOps
GCP
Azure
AWS
Analytics
ETL/ELT
Informatica
Talend
SSIS
DataStage
Apply
≈ $20k – $43k per year (Estimated) · In office · 8+ years exp · Hyderabad
Python
SQL
Databases
Google BigQuery
AI/ML
Airflow
dbt
DevOps
Terraform
GCP
CI/CD
Docker
Analytics
ETL/ELT
Apply
GCP Data Engineer 1 day ago
Hybrid · 5+ years exp · Bachelor's Degree · Hyderabad
Python
SQL
Databases
Google BigQuery
Teradata
AI/ML
Airflow
DevOps
GCP
Platform Engineering
IAM
Analytics
ETL/ELT
Dimensional Modeling
Management
Agile
Apply
Data Engineer 1 day ago
≈ $20k – $42k per year (Estimated) · Hybrid · 5+ years exp · Hyderabad
Python
SQL
Scala
Python
pySpark
Databases
Oracle
Apache Kafka
AI/ML
Hadoop
Spark
DevOps
GCP
Azure
Analytics
ETL/ELT
Apply
In office · 2+ years exp · Bachelor's Degree · Hyderabad
Management
Oracle ERP
Apply
In office · 2+ years exp · Bachelor's Degree · Hyderabad
Management
Oracle ERP
Apply
Equity · In office · Full-Time · 10+ years exp · Bachelor's Degree · Hyderabad
DevOps
Rest API
SOAP
Management
Oracle ERP
Apply
See all jobs
This is one of many
2,298,056 more open roles from verified company boards, updated every day.