Data Engineer
First seen by Alion on Sep 23, 2026.
Role: Hadoop Data Engineer
Experience: 8+ Years
Location: Bangalore
Work Mode: Hybrid
POSITION SUMMARY:
We are seeking a highly skilled and driven Senior Data Engineer to join our growing Data Engineering team. The ideal candidate will have strong expertise in building scalable data solutions, modern cloud-based data platforms, and distributed data processing systems.
ROLES AND RESPONSIBILITIES:
- Design, develop, and maintain data services that integrate seamlessly with LendingClub products and external vendor systems.
- Build and manage scalable data pipelines using modern technologies such as Hadoop, Spark, DBT, Trino, Airflow/Dagster, and cloud-native AWS services.
- Develop solutions on modern data platforms including Databricks and Snowflake.
- Identify opportunities to automate manual processes, optimize data delivery, improve system efficiency, and reduce cloud infrastructure costs.
- Implement and enhance frameworks for Data Quality, Observability, Governance, Lineage, and Monitoring.
- Support production environments, troubleshoot critical issues, and perform root cause analysis (RCA) to ensure platform reliability.
- Follow engineering best practices including Test-Driven Development (TDD), unit and integration testing, documentation, and knowledge-sharing through technical wikis.
REQUIRED QUALIFICATIONS:
- Strong hands-on experience with distributed data systems and big data technologies, including Hadoop, Spark, Hive, DBT, and Airflow/Dagster.
- Minimum 4+ years of production-grade Python development experience, with a focus on data engineering and pipeline development.
- Experience working with cloud platforms, preferably AWS.
- Hands-on experience with modern data platforms such as Databricks and/or Snowflake.
- Proficiency in Git, Jenkins, JIRA, Shell Scripting, and CI/CD practices.
- Solid understanding of Agile methodologies, source control management, automated testing, and Test-Driven Development.
- Experience collaborating with cross-functional teams in fast-paced, dynamic environments.
- Strong analytical thinking, problem-solving abilities, and communication skills.
- Proven track record of designing and delivering scalable, reliable, high-performance data pipelines and data platforms.
EDUCATION:
Bachelor's degree in computer science, Software Engineering, MIS or equivalent combination of education and experience.
KEY SKILLS:
Python, PySpark, Hadoop, Spark, Hive, Data Engineering, Big Data, SQL
Skills
Hadoop, Python, PySpark, Spark, Data Engineering, Hive, AWS, Databricks, SQL, Big Data, Snowflake DB