Senior Software Engineer - Open Data Platform (ODP)

ABOUT THE ROLE

We are seeking a Senior Software Engineer with at least 7 years of experience in distributed systems or data platform development. In this role, you will design, build, and scale components of the Acceldata Open Data Platform (ODP), contributing to open-source technologies and solving real-world challenges in large-scale data infrastructure.

The Acceldata Open Data Platform (ODP) brings together the best of the modern data ecosystem, including technologies such as Hadoop, Spark, Hive, Trino, Kafka, NiFi, MLFlow, Pinot, and more, to help organizations run complex analytical workloads at scale while maintaining flexibility and avoiding vendor lock-in.

Product Link: https://www.acceldata.io/open-data-platform

RESPONSIBILITIES

• Lead the design, development, and implementation of complex features for the Open Source Data Platform.
• Architect scalable and robust solutions, ensuring high performance and reliability.
• Mentor and guide junior software engineers, fostering a culture of continuous learning and growth.
• Drive technical discussions and provide expertise in technology selection and best practices.
• Contribute to the strategic direction of the Open Source Data Platform project.
• Perform thorough code reviews, ensuring code quality, maintainability, and adherence to standards.
• Work across diverse environments: Bare Metals, VM, and Kubernetes.

REQUIREMENTS

• Mandatory Linux proficiency for automation and efficient use of Linux shortcuts for troubleshooting.
• Familiarity with Kubernetes, Docker, and cloud environments.
• Ability to manage threads and debug issues related to deadlocks or memory leaks.
• Strong comprehension of garbage collection tuning and management challenges.
• Extensive hands-on experience with advanced Java, Scala, or Python, developed and prepared project build models associated with Maven, Gradle, or SBT.
• Experienced in Github & Nexus management and its use cases.
• Profound understanding of software engineering, database concepts, virtualisation and architectural patterns.
• Comprehensive knowledge and a history of significant contributions to past projects.
• Past experience of mentoring junior engineers in handling critical issues or tasks.
• Proven capability in fixing bugs and developing new features within complex software systems.
• Experience working with Big data stack or its ecosystem components, utilizing appropriate frameworks for application development.
• Demonstrated fluency in English communication.
• Ability to communicate effectively with senior leadership, presenting technical accomplishments and strategic insights.
• Strong capacity for collaboration with internal teams, external contributors, and the broader open-source community.

PREFERRED SKILLS

• Demonstrated history of contributions or a committer role in Apache projects (e.g., bug fixes, feature development, documentation).
• Proficiency in identifying performance bottlenecks within big data applications and frameworks.
• Experience with the migration of applications, Hadoop-based distributions, or data streaming platforms.
• Familiarity with distributed systems and architectural patterns optimised for scalable data processing.