2,324,170open jobs
146,502companies
274,492added this week
Browse all
Location
In office (London)
Seniority
Senior
Visa
Licensed UK visa sponsor
Employment
Full-Time

Confirmed on the employer's own hiring board on Nov 17, 2026. First seen by Alion on Nov 16, 2026. Optima Partners scores A on the Alion truth index.

Overview
Company
Impact
Profile match
We help large and complex organisations realise the latent value of their customers through AI and collaboration, built on clarity and trust.
Backed by BGF

Who we are

Optima Partners is a VC-backed, scaling AI and data consultancy headquartered in Edinburgh. We bring together expertise in strategy, design, data science and engineering to help leading brands solve real business challenges.

We turn data into better customer experiences, smarter decisions and lasting business value.

Our clients include Lloyds Banking Group, NatWest Group, Bank of Ireland, Nationwide, Aviva, Eon Next, OVO and GSK. We work alongside their teams to deliver practical solutions and share the skills to build on them.

Within our Data practice, you’ll work in mixed-capability pods alongside data engineers, analysts, data scientists and ML/AI engineers. You’ll bring technical depth, guide colleagues and turn data into practical solutions, clear insights and better customer experiences.

The opportunity

As a Senior Data Engineer within our Data practice, reporting to a Technical Lead, you will lead the design, build, testing and maintenance of reliable, scalable data pipelines and platforms for clients across diverse sectors. Working in multi-disciplinary teams, you’ll take ownership of technical delivery and help shape solutions from requirements through to production.

This role suits an experienced data engineer who combines strong hands-on delivery with sound technical judgement, client-facing confidence and a commitment to developing others.

You will work alongside engineers, data scientists, architects and product specialists to make pragmatic design decisions, resolve complex challenges and improve engineering standards. You’ll mentor colleagues, share knowledge and continue to develop your technical and consulting expertise.

What you will be doing

  • Lead the design, build, testing and operation of scalable data pipelines using platforms such as Databricks, taking ownership of delivery quality and production readiness.

  • Design and develop data warehouses, data lakes and lakehouse solutions, making pragmatic architecture and data modelling decisions with architects and technical leads.

  • Build and optimise robust ETL/ELT workflows, embedding data quality checks, monitoring and controls to deliver reliable, accessible data.

  • Work directly with clients and multi-disciplinary colleagues to translate business requirements into technical designs, estimate work and communicate trade-offs, risks and progress.

  • Deliver well-modelled, documented datasets that support analytics, dashboards and visualisations in tools such as Power BI, Looker and QlikView, collaborating with analysts to meet reporting needs.

  • Provide hands-on technical leadership through design and code reviews, mentoring and knowledge sharing, helping colleagues adopt consistent engineering practices.

  • Troubleshoot complex production issues and improve performance, reliability and cost efficiency through automation, testing and CI/CD. Maintain clear documentation and support effective client handover.

What skills we would like you to have

  • Strong hands-on Python and SQL skills, with experience writing maintainable, tested code for production data workloads.

  • Practical experience building and optimising distributed data processing workloads using Spark and Databricks, or comparable platforms.

  • Experience with Delta Lake or comparable lakehouse technologies, including notebooks, workflows and reliable production deployment.

  • Experience delivering and operating data solutions on at least one major cloud platform, such as AWS, Azure or GCP, including appropriate access and security controls.

  • Strong understanding of relational databases, advanced SQL and query optimisation, with the ability to diagnose and resolve performance bottlenecks.

  • Experience implementing orchestration, scheduling, dependencies and failure handling using tools such as Airflow, Apache NiFi, Talend or platform-native workflows.

  • Strong data warehousing, modelling and schema design skills, with experience creating curated datasets for analytics and an understanding of how tools such as Power BI, QlikView and Looker consume data.

  • Experience using Git, automated testing and CI/CD to support collaborative development, controlled releases and maintainable data solutions.

  • Proven experience delivering production data engineering solutions end to end, including technical design, implementation, deployment and ongoing support.

  • Experience working with clients or business stakeholders to clarify requirements, explain technical choices and manage delivery expectations, alongside mentoring engineers and reviewing their work.

  • Experience delivering iteratively within Agile or Scrum teams, breaking down work, estimating effort and identifying dependencies and technical risks.

What you will bring

  • Strong analytical and problem-solving skills, with sound judgement when balancing technical quality, delivery timelines and client priorities.

  • An inquisitive mindset and a commitment to continuous improvement, with the ability to evaluate new approaches and share practical learning across the Data practice.

  • Clear, confident communication, with the ability to explain complex technical ideas, challenge assumptions constructively and build trusted client relationships.

  • A collaborative, accountable approach, with the initiative to lead technical work, support colleagues’ development and give and receive constructive feedback.

As technology partners with leading companies such as AWS and Databricks, we’ll invest in your training and development, alongside hands-on project experience with some of the UK’s leading organisations.

What we offer

  • Competitive base salary.

  • Inclusion in our annual discretionary bonus plan with an on-target performance bonus of up to 15%.

  • Up to 37 days holiday inclusive of personal and public allocations. Seven are fixed days for Christmas and New Year, and 30 are floating days taken at your discretion, subject to client scheduling and line manager approval.

  • Private medical insurance, single cover.

  • Group life and income protection insurance.

  • Salary sacrifice pension scheme after three months’ employment.

  • Access to over 1,000 perks and discounts via an employee discount portal.

  • Dedicated development time plus tools and funding to support personal and professional development.

Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,324,170 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

Data Science
Remote, many countries
Similar stack
Same company
London
In office · Full-Time · Southampton · London
Databases
Databricks
Microsoft Fabric
AI/ML
Multimodal AI
DevOps
Azure
Platform Engineering
Apply
Equity · In office · Full-Time · 5+ years exp · London
Python
SQL
Databases
Apache Kafka
AI/ML
Spark
Dagster
dbt
Prefect
AI Agents
Agentic Workflows
DevOps
GCP
Azure
AWS
Amazon Kinesis
Apply
Data Engineer 9 hours ago
In office · 3+ years exp · Edinburgh
Python
Java
SQL
Scala
Databases
Databricks
Apache Kafka
AI/ML
Spark
dbt
Machine Learning
DevOps
GCP
Azure
AWS
Analytics
ETL/ELT
Apply
In office · 5+ years exp · United Kingdom
Python
SQL
Databases
Google BigQuery
Azure SQL Database
Microsoft Fabric
AI/ML
AutoGen
LangChain
Embeddings
Function Calling
AI Agents
Semantic Kernel
RAG
OpenAI
LLMOps
Human-in-the-Loop
DevOps
GCP
Azure
CI/CD
Analytics
Power BI
Azure Data Factory
Looker
Dimensional Modeling
Management
Power Automate
Power Apps
Apply
In office · Bachelor's Degree · London
Python
SQL
AI/ML
Machine Learning
Analytics
Tableau
Power BI
Apply
$200k – $260k per year · Remote (EMEA, APAC, United States) · Full-Time · 8+ years exp · Seattle
Databases
Databricks
ClickHouse
Apache Kafka
Google BigQuery
AI/ML
dbt
AI Agents
DevOps
GCP
Kubernetes
Analytics
ETL/ELT
Apply
Data Engineer 1 day ago
Remote (LATAM, United States) · 5+ years exp · Bachelor's Degree
Python
SQL
Python
pySpark
Databases
Databricks
AI/ML
Spark
Dagster
dbt
Prompt Engineering
Knowledge Graph
Machine Learning
DevOps
CI/CD
Analytics
ETL/ELT
Apply
Remote (LATAM, United States) · Full-Time · 5+ years exp
Python
SQL
Scala
Databases
Apache Iceberg
Delta Lake
Presto
Apache Hudi
Trino
AI/ML
Spark
Airflow
DevOps
Azure
CI/CD
Git
Amazon S3
Cybersecurity
Open Policy Agent
HIPAA
Analytics
Power BI
ETL/ELT
Apache NiFi
Looker
Superset
Apply
Remote (LATAM) · Full-Time · 4+ years exp
Python
SQL
Databases
Snowflake
Databricks
Amazon Redshift
AI/ML
Spark
Airflow
DevOps
Terraform
Git
AWS
Amazon S3
Amazon Kinesis
Analytics
AWS Glue
Dimensional Modeling
Apply
In office · Full-Time · 2+ years exp · Bachelor's Degree · London
Python
SQL
DevOps
GitHub
Apply
Head of Engineering 10 hours ago
$198k per year · Hybrid · Full-Time · London
DevOps
Azure
AWS
Apply
In office · Full-Time · London · Manchester · Belfast · Reading
Analytics
Power BI
Apply
In office · Full-Time · London · Bristol · Leeds · Manchester · Birmingham
AI/ML
AI Agents
DevOps
GCP
Azure
AWS
Apply
In office · Full-Time · London
DevOps
Incident Management
Apply
See all jobs
This is one of many
2,324,170 more open roles from verified company boards, updated every day.