Data Engineer

We are looking for an experienced Data Engineer with 7+ years of experience in designing, developing, and maintaining scalable data pipelines and data infrastructure. The candidate will be responsible for building ETL/ELT processes, managing large datasets, optimising data workflows, and supporting data analytics and business intelligence initiatives.

• Design, develop, and maintain scalable data pipelines and data integration solutions.

• Build and manage ETL/ELT workflows for structured and unstructured data.

• Develop data models and optimise data warehouse solutions.

• Integrate data from multiple sources, including databases, APIs, and cloud platforms.

• Implement data processing solutions using batch and real-time streaming technologies.

• Ensure data quality, accuracy, consistency, and reliability.

• Work with data scientists, analysts, architects, and business stakeholders.

• Develop and optimise SQL queries and database performance.

• Implement data governance, security, and access control practices.

• Monitor, troubleshoot, and resolve data pipeline failures and performance issues.

• Work with cloud-based data platforms and distributed processing frameworks.

• Maintain technical documentation and follow software development best practices.

• Strong proficiency in SQL and Python.

• Hands-on experience with ETL/ELT tools and data pipeline development.

• Experience with Apache Spark, PySpark, and Hadoop.

• Knowledge of Apache Airflow or similar workflow orchestration tools.

• Experience with data warehouses such as Snowflake, Amazon Redshift, Google BigQuery, or Azure Synapse.

• Hands-on experience with AWS, Microsoft Azure, or Google Cloud Platform.

• Knowledge of relational and NoSQL databases.

• Experience with data integration, data modelling, and data transformation.

• Familiarity with Kafka or other real-time data streaming technologies.

• Experience with Git, CI/CD, and automated deployment processes.

• Understanding of data governance, data security, and data quality frameworks.

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related discipline.

• Minimum 7 years of relevant experience in Data Engineering.

• Proven experience developing and managing enterprise-scale data pipelines.

• Experience working with cloud data platforms and big data technologies.

• Strong analytical and problem-solving abilities.

• Excellent SQL and programming skills.

• Ability to handle large and complex datasets.

• Strong communication and collaboration skills.

• Ability to work independently and manage multiple priorities.

• AWS Certified Data Engineer.

• Microsoft Certified: Azure Data Engineer Associate.

• Google Cloud Professional Data Engineer.

• Databricks Data Engineer Certification.

Skills

Snowflake Cloud Data Warehouse, Data Warehousing Design, optimized SQL queries, Airflow, Apache Spark, Stream Processing, ETL Tools, AWS, Google Cloud Platform, Data Integration, Data Quality Assurance, Relational Databases, integrate Data sources, Data Science, Data Infrastructure, SQL Development