Senior Data Engineer with Java /Python (5+ Years)

Job Description – Data Engineer

Position: Data Engineer

Experience: 5–7 Years

Relevant Experience: 5+ Years

Location: Remote

Employment Type: Full-Time

Job Summary

We are looking for an experienced Data Engineer with strong expertise in Python, SQL, Java or Scala, Apache Spark, and Microsoft Fabric. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data pipelines and data engineering solutions.

The candidate should have hands-on experience working with batch and real-time data processing, modern data frameworks, data modeling, and Microsoft Fabric Notebooks to build and operationalize efficient data workflows.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Python, Java, or Scala.
• Build and optimize data processing solutions using Apache Spark and modern data frameworks.
• Develop batch and streaming data pipelines using technologies such as Kafka, Flink, Spark, and dbt.
• Work with SQL and NoSQL databases, implementing efficient data models and storage solutions.
• Optimize SQL queries, data pipelines, and storage systems for improved performance and cost efficiency.
• Utilize Microsoft Fabric Notebooks to develop, debug, test, and operationalize data engineering workflows.
• Implement data integration, transformation, and processing solutions to support business requirements.
• Leverage Docker and Kubernetes for containerization and orchestration of data engineering workloads.
• Implement monitoring, logging, and observability practices using tools such as Datadog.
• Collaborate with cross-functional teams to ensure data quality, reliability, scalability, and performance.

Required Skills & Qualifications

• 5–7 years of overall experience in Data Engineering, with at least 5 years of relevant experience.
• Strong programming proficiency in Python, Java, or Scala.
• Hands-on experience with Apache Spark and distributed data processing.
• Practical experience with Microsoft Fabric Notebooks for data engineering workflows.
• Strong SQL skills, including query development and performance optimization.
• Experience with modern data frameworks such as Kafka, Flink, dbt, or equivalent technologies.
• Good understanding of SQL and NoSQL databases and data modeling principles.
• Experience developing both batch and/or streaming data processing solutions.
• Hands-on experience with Docker and Kubernetes.
• Knowledge of observability and monitoring tools, preferably Datadog.
• Strong analytical, problem-solving, and troubleshooting skills.

Mandatory Technical Skills

Python | SQL | Java/Scala | Apache Spark | Microsoft Fabric

Good to Have

• Experience with Kafka, Flink, and dbt.
• Exposure to cloud-based data engineering environments.
• Experience with Datadog or similar monitoring and observability platforms.
• Knowledge of data pipeline orchestration and workflow automation.

Work Details

• Work Mode: 100% Remote
• Experience: 5–7 Years
• Relevant Experience: Minimum 5 Years