Data Engineer II

Role Overview :

As a Data Engineer II based in Mumbai, you will serve as a cornerstone of our data infrastructure, architecting and maintaining robust systems that transform raw information into actionable business intelligence.

You will work closely with cross-functional teams, including data scientists, product managers, and software engineers, to design scalable pipelines that power our core analytical products.

Your day-to-day will involve navigating complex data ecosystems to ensure high availability, performance, and data integrity, directly influencing how our leadership team makes data-driven decisions and how our customers experience our personalized service offerings.

Key Responsibilities :

- Design and implement end-to-end ETL/ELT pipelines to ingest, process, and store massive datasets, ensuring seamless data flow for downstream analytics and machine learning applications.

- Optimize data warehouse performance and data modeling strategies to reduce query latency and improve cost-efficiency across our cloud infrastructure.

- Collaborate with engineering stakeholders to integrate real-time data streaming solutions using Kafka, enabling low-latency insights for critical business operations.

- Maintain and scale Big Data clusters, ensuring high availability and fault tolerance for mission-critical data processing tasks.

- Mentor junior engineers through code reviews and technical guidance, fostering a culture of engineering excellence and best practices within the data team.

Required Skillset :

- Demonstrated proficiency in Python and Scala for building complex data processing applications and automating infrastructure tasks.

- Advanced expertise in SQL and NoSQL database management, with a proven ability to design efficient data models that support diverse analytical requirements.

- Hands-on experience with Big Data frameworks including Spark and Hadoop, coupled with a strong understanding of distributed computing principles.

- Proven track record of deploying and managing data solutions within AWS cloud environments, leveraging cloud-native tools to enhance scalability and security.

- Strong communication skills with the ability to translate complex technical concepts into clear insights for non-technical stakeholders.

- A collaborative mindset with the ability to thrive in a hybrid work environment, demonstrating adaptability and a proactive approach to problem-solving.

- A Bachelors or Masters degree in Computer Science, Engineering, or a related quantitative field, supported by 4 - 7 years of professional experience in data engineering roles.

Skills

Data Engineering, ETL, Machine Learning, Data Pipeline, Data Warehousing, Data Strategy, Data Analytics, Python, Scala, SQL, NoSQL