Sr. Data Engineer - Python + Pyspark + Cloud

Job Description

• Skill Set :

  SQL,cloud,pyspark

• Total Experience :

  5.00 to 10.00 Years

• No of Openings :

  1

• Job Post Date :

  02/09/2026

• Job Expiry Date :

  31/10/2026

• Domain :

  IT

• Location :

  HYDERABAD [India]

• Job Reference No :

  4105167

Job Summary

Key Responsibilities

•Design and deliver Oracle AI Data Platform solutions, including platform implementations and migrations.

•Build scalable, governed, AI ready data pipelines, software components, and services for data consolidation, integration, transformation, and data products.

•Develop Lakehouse, Medallion, streaming, and real time analytics solutions on OCI using Spark, PySpark, Delta Lake, Parquet, Flink, and distributed systems patterns.

•Build AI/ML and Generative AI data pipelines, including feature engineering, feature extraction, MLOps workflows, and vector ready datasets.

•Assess and modernize workloads by refactoring PySpark and ETL code and optimizing Spark configuration, partitioning, memory, adaptive query execution, reliability, and performance.

•Develop APIs, SDKs, web applications, and automation frameworks that simplify adoption of Oracle AI Data Platform and Generative AI services.

•Contribute to reusable migration accelerators, reference architectures, automation frameworks, and delivery methodologies.

•Contribute to engineering excellence across CSS and continuously evaluate emerging AI, cloud, and data engineering practices.

Minimum Qualifications

•5‿8 years of experience designing, building, and delivering scalable data platforms.

•Proven experience as a Data Engineer, Software Engineer, AI Engineer, Consulting Engineer, or similar technical role.

•Strong knowledge of modern cloud data platforms like Databricks.

•Strong hands on experience with Apache Spark, PySpark, Delta Lake, Python, SQL, and modern data ecosystems; experience with Scala, Java, or Kafka is a plus.

•Exposure to building AI agents and orchestration pipelines.

•Strong understanding of Spark architecture, configuration, distributed systems, performance optimization, and tuning.

•Experience modernizing analytics platforms and refactoring Spark and ETL workloads.

•Working understanding of networking concepts to support data connectivity, configuration, and troubleshooting.

•Working knowledge of DataOps, CI/CD, Infrastructure as Code, data security, governance, metadata, lineage, and observability.

•Strong communication, customer collaboration, and stakeholder management skills.

Preferred Qualifications

•OCI Architect, Data Engineering, Data Science, or AI certifications.

•Experience with LLMs, Retrieval Augmented Generation (RAG), vector databases, AI agents, and MLOps.

•Experience contributing to large scale data platform modernization programs and distributed engineering teams.

•Industry experience in Financial Services, Healthcare, Public Sector, Telecommunications, Manufacturing, or Retail.

Delivery & Collaboration Expectations

•Contribute to the technical delivery strategy for customer programs.

•Share field feedback with Oracle product and engineering teams.

•Apply and improve reusable engineering standards and delivery frameworks.

•Build expertise in Oracle cloud data engineering and AI platforms.

•Collaborate with senior architects and engineers across Oracle CSS.

•Participate in architecture reviews and customer innovation workshops.