Data Engineer

We are seeking a highly skilled Lead Data Engineer to join our dynamic team. In this role, you will not only design and develop our cloud-based data infrastructure but also lead a team of Data Engineers to deliver robust and scalable data solutions for our BFSI customers. As a Tech Lead, you will work at the intersection of cloud technologies, data engineering, and the BFSI domain, while guiding the team to ensure project success.

Responsibilities:

• Lead, mentor, and manage a team of Data Engineers to deliver high-quality data solutions.
• Design, develop, and implement cloud-based data pipelines, ETL processes, and data integration solutions using AWS platforms.
• Collaborate with cross-functional teams to understand data requirements and design scalable data models and architectures that align with BFSI industry needs.
• Optimize data storage, processing, and retrieval for maximum performance and cost- efficiency in cloud environments.
• Implement data security and compliance measures to ensure the protection and integrity of sensitive BFSI data.
• Work closely with data scientists and analysts to enable seamless access to high-quality data for analytical purposes.
• Troubleshoot and resolve data-related issues, ensuring data availability and reliability for BFSI customers.
• Stay updated on industry best practices, emerging cloud technologies, and trends in the BFSI sector to drive continuous improvement and innovation.
• Manage project timelines, delegate tasks, and ensure the successful delivery of data engineering projects.

Requirements:

• Minimum 6 years' experience with good hands-on PySpark, Airflow, AWS.
• Advanced SQL and PySpark expertise; strong understanding of Data Engineering concepts (DW, Data Lake, OLTP vs OLAP, etc. ).
• API development experience (preferably in Python).
• Familiarity with Docker and Kubernetes.
• Experience with Airflow.
• Exposure to Hudi, Iceberg, or Delta Lake.
• Strong AWS project experience (should also be adaptable to other cloud platforms if required).
• Proven experience in leading and managing a team of Data Engineers in previous roles.
• Bachelor's degree in computer science, Engineering, or a related field. Master's degree preferred.
• Proficiency in programming languages like Python, Java, or Spark for data processing and ETL operations.
• Experience with big data technologies(e. g., Hadoop, Spark) and data processing frameworks.
• Familiarity with BFSI industry data standards, compliance requirements, and security protocols.
• Excellent problem-solving skills and the ability to work in a collaborative, cross-functional team environment.