Data Engineer

The Software Engineer is ultimately responsible for delivering technical solutions: starting from the project's onboard until post-launch support and including design, development, and testing. It is expected to coordinate, support, and work with multiple delocalized project teams in multiple regions. As a member of the technical staff with our Digital Meter Processing team, you will further develop the backend system that processes massive amounts of data every day, across 3 different AWS regions. Your role will involve designing, implementing, and maintaining robust, scalable solutions that leverage a Java-based system that runs in an AWS environment. You will play a key role in shaping the technical direction of our projects and mentoring other team members.

Responsibilities:

• System Deployment: Conceive, design, and build new features in the existing backend processing pipelines.
• CI/CD Implementation: Design and implement CI/CD pipelines for automated build, test, and deployment processes. Ensure continuous integration and delivery of features, improvements, and bug fixes.
• Code Quality and Best Practices: Enforce coding standards, best practices, and design principles. Conduct code reviews and provide constructive feedback to maintain high code quality.
• Performance Optimization: Identify and address performance bottlenecks in reading, processing, and writing data to the backend data stores.
• Mentorship and Collaboration: Mentor junior engineers, guiding technical aspects and best practices. Collaborate with cross-functional teams to ensure a cohesive and unified approach to software development.
• Security and Compliance: Implement security best practices for all tiers of the system. Ensure compliance with industry standards and regulations related to AWS platform security.

Requirements:

• Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field.
• Proven experience, minimum 3 years, in high-volume data processing development expertise using ETL tools such as AWS Glue or PySpark, Java, SQL, and databases such as Postgres.
• Minimum 2 years of development on an AWS platform.
• Strong understanding of CI/CD principles and tools. GitLab is a plus.
• Excellent problem-solving and debugging skills.
• Strong communication and collaboration skills with the ability to communicate complex technical concepts and align the organization on decisions.
• Sound problem-solving skills with the ability to quickly process complex information and present it clearly and simply.
• Utilizes team collaboration to create innovative solutions efficiently.

Other desirable skills:

• Knowledge of networking principles and security best practices.
• AWS certifications.
• Experience with Data Warehouses, ETL, and/or Data Lakes is very desirable.
• Experience with RedShift, Airflow, Python, Lambda, Prometheus, Grafana, and OpsGeni is a bonus.
• Exposure to the Google Cloud Platform (GCP).