{"id":4426041,"url":"https://alion.io/job/optima-partners-senior-data-engineer-2","title":"Senior Data Engineer","company":{"id":678916,"name":"Optima Partners","domain":"optimapartners.co.uk","url":"https://alion.io/company/optimapartners","size_band":"51-200","is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Teamtailor","truth_index":{"grade":"A","score":85,"open_postings":5,"ghost_share":0,"stale_share":0.4,"repost_share":0,"time_to_fill_p50_days":99,"computed_at":"2026-11-16T05:46:25Z"}},"role":"Data Science","role_family":"Data Science","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["London, United Kingdom"],"countries":["GB"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agile","optional":false},{"name":"Airflow","optional":false},{"name":"Apache NiFi","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Delta Lake","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"Git","optional":false},{"name":"Looker","optional":false},{"name":"Power BI","optional":false},{"name":"Python","optional":false},{"name":"Scrum","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Talend","optional":false}],"status":"live","first_seen_at":"2026-11-16T16:37:09Z","employer_posted_date":"2026-11-16","last_verified_at":"2026-11-17T01:59:09Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"Who we are\nOptima Partners is a VC-backed, scaling AI and data consultancy headquartered in Edinburgh. We bring together expertise in strategy, design, data science and engineering to help leading brands solve real business challenges.\nWe turn data into better customer experiences, smarter decisions and lasting business value.\nOur clients include Lloyds Banking Group, NatWest Group, Bank of Ireland, Nationwide, Aviva, Eon Next, OVO and GSK. We work alongside their teams to deliver practical solutions and share the skills to build on them.\nWithin our Data practice, you’ll work in mixed-capability pods alongside data engineers, analysts, data scientists and ML/AI engineers. You’ll bring technical depth, guide colleagues and turn data into practical solutions, clear insights and better customer experiences.\nThe opportunity\nAs a Senior Data Engineer within our Data practice, reporting to a Technical Lead, you will lead the design, build, testing and maintenance of reliable, scalable data pipelines and platforms for clients across diverse sectors. Working in multi-disciplinary teams, you’ll take ownership of technical delivery and help shape solutions from requirements through to production.\nThis role suits an experienced data engineer who combines strong hands-on delivery with sound technical judgement, client-facing confidence and a commitment to developing others.\nYou will work alongside engineers, data scientists, architects and product specialists to make pragmatic design decisions, resolve complex challenges and improve engineering standards. You’ll mentor colleagues, share knowledge and continue to develop your technical and consulting expertise.\nWhat you will be doing\nLead the design, build, testing and operation of scalable data pipelines using platforms such as Databricks, taking ownership of delivery quality and production readiness.\n\nDesign and develop data warehouses, data lakes and lakehouse solutions, making pragmatic architecture and data modelling decisions with architects and technical leads.\n\nBuild and optimise robust ETL/ELT workflows, embedding data quality checks, monitoring and controls to deliver reliable, accessible data.\n\nWork directly with clients and multi-disciplinary colleagues to translate business requirements into technical designs, estimate work and communicate trade-offs, risks and progress.\n\nDeliver well-modelled, documented datasets that support analytics, dashboards and visualisations in tools such as Power BI, Looker and QlikView, collaborating with analysts to meet reporting needs.\n\nProvide hands-on technical leadership through design and code reviews, mentoring and knowledge sharing, helping colleagues adopt consistent engineering practices.\n\nTroubleshoot complex production issues and improve performance, reliability and cost efficiency through automation, testing and CI/CD. Maintain clear documentation and support effective client handover.\n\nWhat skills we would like you to have\nStrong hands-on Python and SQL skills, with experience writing maintainable, tested code for production data workloads.\n\nPractical experience building and optimising distributed data processing workloads using Spark and Databricks, or comparable platforms.\n\nExperience with Delta Lake or comparable lakehouse technologies, including notebooks, workflows and reliable production deployment.\n\nExperience delivering and operating data solutions on at least one major cloud platform, such as AWS, Azure or GCP, including appropriate access and security controls.\n\nStrong understanding of relational databases, advanced SQL and query optimisation, with the ability to diagnose and resolve performance bottlenecks.\n\nExperience implementing orchestration, scheduling, dependencies and failure handling using tools such as Airflow, Apache NiFi, Talend or platform-native workflows.\n\nStrong data warehousing, modelling and schema design skills, with experience creating curated datasets for analytics and an understanding of how tools such as Power BI, QlikView and Looker consume data.\n\nExperience using Git, automated testing and CI/CD to support collaborative development, controlled releases and maintainable data solutions.\n\nProven experience delivering production data engineering solutions end to end, including technical design, implementation, deployment and ongoing support.\n\nExperience working with clients or business stakeholders to clarify requirements, explain technical choices and manage delivery expectations, alongside mentoring engineers and reviewing their work.\n\nExperience delivering iteratively within Agile or Scrum teams, breaking down work, estimating effort and identifying dependencies and technical risks.\n\nWhat you will bring\nStrong analytical and problem-solving skills, with sound judgement when balancing technical quality, delivery timelines and client priorities.\n\nAn inquisitive mindset and a commitment to continuous improvement, with the ability to evaluate new approaches and share practical learning across the Data practice.\n\nClear, confident communication, with the ability to explain complex technical ideas, challenge assumptions constructively and build trusted client relationships.\n\nA collaborative, accountable approach, with the initiative to lead technical work, support colleagues’ development and give and receive constructive feedback.\n\nAs technology partners with leading companies such as AWS and Databricks, we’ll invest in your training and development, alongside hands-on project experience with some of the UK’s leading organisations.\nWhat we offer\nCompetitive base salary.\n\nInclusion in our annual discretionary bonus plan with an on-target performance bonus of up to 15%.\n\nUp to 37 days holiday inclusive of personal and public allocations. Seven are fixed days for Christmas and New Year, and 30 are floating days taken at your discretion, subject to client scheduling and line manager approval.\n\nPrivate medical insurance, single cover.\n\nGroup life and income protection insurance.\n\nSalary sacrifice pension scheme after three months’ employment.\n\nAccess to over 1,000 perks and discounts via an employee discount portal.\n\nDedicated development time plus tools and funding to support personal and professional development.","description_format":"text","description_chars":6246,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Health insurance","Professional development"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["AI Consulting & Integration"],"lifecycle":[{"event":"open","at":"2026-11-17T01:59:09Z"}],"visa":[{"country":"GB","licensed_sponsor":true,"adverts_offering_sponsorship_24m":0,"sanctioned_now":false,"evidence":"Licensed UK visa sponsor (Skilled Worker)","filings_12m":null,"filings_prev_12m":null,"green_card_filings_12m":null,"median_offered_wage_usd":null,"route":"Skilled Worker","cap_exempt":false,"checked_at":"2026-11-16T08:17:55+00:00","sources":["UK Home Office: register of licensed sponsors (workers)"],"filings_for_role_12m":0}],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":99,"reasons":["conf:1","win:early"],"computed_at":"2026-11-17T03:52:43Z"},"pay":null,"html_url":"https://alion.io/job/optima-partners-senior-data-engineer-2","json_url":"https://alion.io/job/optima-partners-senior-data-engineer-2.json","meta":{"generated_at":"2026-11-17T03:52:43Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":7274,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":678916},"rest":"https://alion.io/mcp/rest/get_company?id=678916"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Foptima-partners-senior-data-engineer-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Foptima-partners-senior-data-engineer-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Foptima-partners-senior-data-engineer-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/optima-partners-senior-data-engineer-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Foptima-partners-senior-data-engineer-2"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4426041}},{"tool":"search_jobs","arguments":{"company":"optimapartners.co.uk","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4426041},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}