Data Engineer (overall)

About Us

Komodo Consulting is a technology and strategy firm specializing in Digital Transformation. Operating in Portugal and Poland, we provide IT Consulting & Nearshore services. We support both public and private sector organizations through two main areas:

- Consulting - with a focus on strategy, investment analysis, and digital process improvement;

- IT Team Augmentation - helping clients scale and strengthen their tech teams.

The Project

We are seeking a Data Engineer to work on a project for a Technology Company.

You will have the following responsibilities:

• Design, build, and maintain scalable data pipelines, both batch and streaming;

• Develop ETL/ELT processes to ingest, transform, and integrate data from multiple sources;

• Design and optimise data models for data warehouses, data lakes, or lakehouses;

• Ensure data quality, consistency, and reliability through validation and monitoring;

• Optimise the performance and cost of data processing and storage;

• Implement data governance, security, and access control practices;

• Automate data workflows and deployments using orchestration and CI/CD tools;

• Collaborate with data analysts, data scientists, and business teams to deliver effective data solutions;

• Troubleshoot data issues, identify root causes, and propose effective solutions;

• Produce and maintain up-to-date technical documentation.

You need to have the following skills/experience:

• 3+ years of solid experience in data engineering;

• Strong proficiency in SQL and Python (Scala or Java is a plus);

• Experience with distributed data processing frameworks such as Apache Spark or equivalents;

• Experience with at least one cloud data platform (AWS, Azure, or GCP) and related services;

• Knowledge of data warehouse and lakehouse technologies such as Snowflake, Databricks, BigQuery, Synapse, or equivalents;

• Experience with orchestration tools such as Airflow, Azure Data Factory, dbt, or equivalents;

• Solid understanding of data modelling concepts such as dimensional modelling and star schema;

• Familiarity with streaming technologies such as Kafka or equivalents;

• Experience with Git, CI/CD, and containerisation tools such as Docker is valued;

• English proficiency at B2 level or higher.

Location

Lisbon, Porto, Coimbra, or Braga (Hybrid - 2 days per week at the office).