Data Engineer - GCP

Data engineer responsible for building and optimising large-scale data pipelines using PySpark, SQL, and Google Cloud (BigQuery, DataProc) while migrating and managing big data workloads across modern platforms. Works within an Agile team to develop, deploy, and enhance data-driven solutions for marketing analytics, ensuring high performance, scalability, and continuous improvement of data systems.

Responsibilities:

• Big Data, GCP, ETL, Big Data/Data Warehousing, Git (GitHub, GitLab, BitBucket, SVN), PySpark, Python, SQL, Google Cloud - Data Engineer. We are looking for energetic, high-performing and highly skilled data engineers to help shape our technology and product roadmap.
• You will be part of the fast-paced, entrepreneurial Global Campaign Tracking (GCT) team under the Enterprise Personalisation Portfolio focused on delivering the next generation of global marketing capabilities.
• The team is responsible for marketing campaign tracking, new account acquisition and bounty payments and leverages large-scale data engineering technologies, such as SQL, PySpark, GCP, BigQuery, DataProc, Adobe Analytics, Google Analytics, Hive, Kafka and Java.
• Focus: Designs, develops, solves problems, debugs, evaluates, modifies, deploys, and documents software and systems that meet the needs of customer-facing applications, business applications, and/or internal end user applications.
• Develop and maintain a large-scale data processing pipeline using PySpark, DataProc, BigQuery and SQL.
• Use BigQuery and Dataproc to migrate existing Hadoop/Spark/Hive workloads to Google Cloud.
• Proficient in BigQuery to carry out batch and interactive data analysis.
• Function as a member of an agile team by contributing to software builds through consistent development practices (tools, common components, and documentation).
• Develops and tests software, including ongoing refactoring of code, and drives continuous improvement in code structure and quality.
• Enable the deployment, support, and monitoring of software across test, integration, and production environments.

Requirements:

• A bachelor's degree in computer science, computer engineering, another technical discipline, or equivalent work experience.
• 6 - 9 years of software development experience.
• Hands-on expertise with application design, software development, and automated testing.
• Strong programming knowledge in SQL, PySpark, DataProc, and BigQuery.
• Hands-on experience in big data technologies (Spark, Hive).
• Understanding and experience with UNIX / Shell / Perl / Python scripting.
• Database query optimisation and indexing Web services design and implementation using REST / SOAP and Java is a plus.
• Experience collaborating with the business to drive requirements/Agile story analysis.
• Experience with design and coding across one or more platforms and languages as appropriate.

Bonus skills:

• Machine learning/data mining, object-orientated design and coding, and Adobe Marketing Campaign products.

Skills

BigQuery, PySpark, SQL