Data Engineer

We are seeking a highly skilled Data Engineer to join our dynamic Data Engineering team. The ideal candidate possesses 4-7 years of data engineering experience. An excellent candidate should have a solid understanding of Spark. PySpark and SQL, and have data pipeline experience. Hired individuals will play a crucial role in helping to build out our data engineering team to support one of our strategic pipelines and optimise for reliability, efficiency, and performance.

Additionally, Data Engineering serves as the gold standard for all other YipitData analyst teams, building and maintaining the core pipelines and tooling that power our products. This high-impact, high-visibility team is instrumental to the success of our rapidly growing business. This is a unique opportunity to be the first hire in this team, with the potential to build and lead the team as their responsibilities expand.

Responsibilities:

• Report directly to a Manager of Data Engineering, who will provide significant, hands-on training on cutting-edge data tools and techniques.
• Build and maintain end-to-end data pipelines.
• Help with setting best practices for our data modelling and pipeline builds.
• Create AI-ready analytical datasets designed with the structure, metadata, documentation, and business context needed for effective use by AI agents and insight-driven applications.
• Become an expert at solving complex data pipeline issues using PySpark and SQL.
• Collaborate with stakeholders to incorporate business logic into our central pipelines.
• Deeply learn Databricks, Spark, AI, and other ETL toolings developed internally.

Requirements:

• You hold a Bachelor's or Master's degree in Computer Science, STEM, or a related technical discipline.
• You have 4+ years of experience as a Data Engineer or in other technical functions.
• You are excited about solving data challenges and learning new skills.
• You have a great understanding of working with data or building data pipelines.
• You are comfortable working with large-scale datasets using PySpark, Delta, and Databricks.
• You understand business needs and the rationale behind data transformations to ensure alignment with organisational goals and data strategy.
• You are eager to constantly learn new technologies.
• You are a self-starter who enjoys working collaboratively with stakeholders.
• You have exceptional verbal and written communication skills.
• Experience with Airflow, dbt, Snowflake, or equivalent.