Data Engineer

Responsibilities:

• Design, develop, and maintain high-performance Spark and Scala data processing applications and pipelines on Amazon EMR.
• Ensure performance, reliability, and responsiveness of applications and distributed data workloads through profiling, tuning, monitoring, and capacity planning.
• Design and develop APIs/services to support Revenue Management and Pricing platform capabilities and integrations.
• Stay abreast of developments in distributed computing, cloud-native design, and programming languages; apply improvements pragmatically.
• Collaborate with architects, engineers, and business teams to design and evolve architectures and technical solutions.
• Release high-quality code early and often using lean and Agile engineering practices.
• Promote best practices such as automated testing, code reviews, pair programming, test-driven development, and continuous integration.
• Coordinate with Operations and Infrastructure/DevOps teams for deployment, monitoring, and production support activities, including incident troubleshooting and prevention.

Requirements:

• Bachelor's degree in computer science, Information Technology, or equivalent experience.
• 4+ years of professional software development experience, specialising in designing and developing enterprise-grade, scalable, and high-performance applications.
• 4+ years of hands-on experience with Apache Spark and Scala, including building, scaling, and optimising large-scale batch data processing pipelines.
• Proven experience with Amazon EMR, including configuration, tuning, troubleshooting, and orchestration of Spark jobs in a cloud environment.
• 3+ years of hands-on experience developing applications on cloud platforms (AWS preferred; Azure acceptable), including storage/services such as S3
• Proficiency with SQL and NoSQL databases (e. g., PostgreSQL and Couchbase/Document DB), including schema design and SQL query optimisation for application performance.
• Working experience building distributed systems using Pub/Sub patterns and streaming platforms like Kafka.
• Strong experience in performance tuning across Spark jobs, application services, database queries, and cloud resource utilisation.
• Working knowledge of DevOps tools and practices such as Git, JIRA, Harness/Jenkins, and CI/CD.
• Strong communication and collaboration skills, with the ability to explain technical concepts and trade-offs to cross-functional stakeholders.

Preferred Qualifications:

• Experience with enterprise integration patterns (e. g., Apache Camel).
• Experience with microservice development and architectures (Java, Spring Boot, JPA, Hibernate) and container orchestration (OpenShift/Kubernetes).
• Experience with ReactJS or a similar modern JavaScript framework (full-stack exposure).
• Experience working with highly concurrent distributed systems (e. g., Akka-based applications).
• Domain experience in travel, lodging, pricing, or revenue management applications/services/operations.
• Exposure to enterprise/solution architecture and platform modernisation initiatives.
• Experience working in Agile/Scrum methodology.

Skills

Amazon EMR, Apache Spark, EMR, Scala, Spark