Linux Scripting Engineer

The Data Engineering team at Epsilon plays a pivotal role in building the next-generation data platforms that empower Epsilon's ecosystem. We design, develop, and optimise scalable, high-performance data solutions that process terabytes of data every day, driving intelligent insights for clients across industries. This team of problem-solvers and innovators leverages modern big data, cloud, and automation technologies to deliver reliable and efficient data products. The candidate will be a member of the Data Engineering Team and be responsible for developing, Unit Testing, and implementing applications for the Data Engineering group, predominantly in the Hadoop Ecosystem and Databricks.

Responsibilities:

• Provide outstanding customer service, following detailed specifications, in the processing of incoming and outgoing client data
• Design and develop custom data solutions in Databricks.
• Develop efficient, reusable, and reliable code for data processing and transformation.
• Work with Technical Leads, Architects and data platform teams to understand and implement robust data solutions.
• Contribute to data modelling, quality, and governance initiatives to ensure trusted data delivery.
• Perform detailed analysis, troubleshooting, and RCA for production issues and optimise system reliability.
• Participate in code reviews and enforce best coding and design practices.
• Collaborate with cross-functional teams to deliver high-quality software solutions.
• Improve and optimise deployment processes and help deliver reliable solutions.
• Interact with technical leads and architects to discover solutions that help solve challenges faced by Data Engineering teams.
• Contribute to building an environment where continuous improvement of the development and delivery process is in focus, and our goal is to deliver outstanding software.

Requirements:

• You have strong experience in Data Engineering.
• You are hands-on with Shell Scripting, Python/PySpark, and enjoy working with massive datasets.
• You have experience working with AWS.
• You are self-driven and enjoy working collaboratively as part of a team.
• You can analyse, troubleshoot and resolve customer issues with strong customer focus.
• You take pride in writing efficient, maintainable code and automating processes to improve efficiency.
• You enjoy new challenges and are solution-oriented.
• BE / B. Tech / MCA - No correspondence course.
• 3-5 years of experience working in a Linux or Unix environment.
• 1-3 years of experience in Shell Scripting and SQL.
• Must have strong experience in Python/PySpark.
• Good to have knowledge and experience with Databricks.
• Good to have experience working with AWS.
• Experience with JIRA for user-story/bug tracking.
• Experience with GIT/Bitbucket.
• Experience leveraging GenAI in data processing will be a plus.