{"id":714008,"url":"https://alion.io/job/optimapartners-core-data-engineer","title":"Core Data Engineer","company":{"id":678916,"name":"Optima Partners","domain":"optimapartners.co.uk","url":"https://alion.io/company/optimapartners","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Teamtailor","truth_index":{"grade":"A","score":90,"open_postings":5,"ghost_share":0,"stale_share":0.2,"repost_share":0,"time_to_fill_p50_days":84,"computed_at":"2026-11-01T06:10:25Z"}},"role":"Data Science","role_family":"Data Science","seniority":"junior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Edinburgh, United Kingdom"],"countries":["GB"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":44000,"max_usd":85000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":17},"experience_years_min":2,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Airflow","optional":false},{"name":"Amazon Redshift","optional":false},{"name":"Apache NiFi","optional":false},{"name":"AWS","optional":false},{"name":"Azure","optional":false},{"name":"CI/CD","optional":false},{"name":"Databricks","optional":false},{"name":"Docker","optional":false},{"name":"ETL/ELT","optional":false},{"name":"GCP","optional":false},{"name":"Git","optional":false},{"name":"Google BigQuery","optional":false},{"name":"Hadoop","optional":false},{"name":"Java","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Python","optional":false},{"name":"Scala","optional":false},{"name":"Snowflake","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Talend","optional":false},{"name":"Agile","optional":true},{"name":"Scrum","optional":true}],"status":"live","first_seen_at":"2025-10-10T16:35:06Z","employer_posted_date":"2025-10-10","last_verified_at":"2026-11-02T00:35:28Z","board_verified":true,"closed_at":null,"days_open":387,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":386},"description":"The Role\nAs a Core Data Engineer, you will play a pivotal role in designing, building, and optimising data pipelines, architectures, and workflows for our diverse client base. You’ll work within dynamic, multi-disciplinary project teams to implement scalable data solutions while adhering to best practices and emerging trends in data engineering.\nThis is an exciting and valuable opportunity for an enthusiastic, earlier-stage data engineer to advance their career across a variety of commercial sectors. At Optima, you’ll work within cross-functional teams, collaborating with experts in different fields and gaining exposure to varied business challenges.\nOptima provides an inclusive space for professional development, with a focus on continuous learning and knowledge sharing. Engineering at Optima is a fast-growing team with strong partnerships in the wider data science community.\nWith the chance to work alongside seasoned engineers, scientists, data architects, and product specialists, you’ll be contributing to our collective effort to stay at the forefront of a fast-paced, data-driven world. Moreover, there are chances to learn and grow as an engineer in different disciplines, as well as a clear path to career development and promotion within the role. \n\nKey Responsibilities\n\nData Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources.\n\nData Architecture: Develop and optimise data architectures, including data warehouses, data lakes, and other storage solutions.\n\nETL/ELT Processes: Create and maintain reliable ETL/ELT workflows that ensure data quality and accessibility.\n\nCollaboration: Work closely with data scientists, analysts, and client stakeholders to understand requirements and deliver impactful solutions.\n\nPerformance Optimisation: Monitor and enhance the performance of data systems to ensure minimal downtime and fast query responses.\n\nAutomation and Tools: Identify opportunities for automation and recommend tools to improve data engineering workflows.\n\nDocumentation: Maintain detailed technical documentation for all solutions and processes.\n\nTechnical Skills\n\nProgramming: Proficiency in Python, Java, Scala, or similar languages.\n\nBig Data Technologies: Hands-on experience with big data tools e.g. (Databricks, Apache Spark, Hadoop).\n\nCloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks.\n\nExpertise in relational databases (e.g. postgres, sql server)\n\nData Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend.\n\nData Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design.\n\nVersion Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines for deployment.\n\nExperience\n2+ years of experience in data engineering or a similar role.\n\nProven track record of delivering data solutions in a consulting or client-facing environment is a plus.\n\nExperience with Agile or Scrum methodologies is beneficial.\n\nPersonal Qualities\n\nStrong problem-solving and analytical thinking abilities.\n\nExcellent communication skills to explain complex technical concepts to non-technical stakeholders.\n\nA team-oriented mindset with a focus on collaboration and adaptability\n\nThe Company\nOptima Partners is an advanced data and business consultancy headquartered in Edinburgh, UK. We are a practitioner-led organisation that collaborates with top consumer brands to drive transformation and foster customer-centricity through our expertise in customer strategy, innovative design, and advanced data science and engineering.\nWe help our clients get closer to their customers, to truly understand them, and deliver exactly the right products, engagement, and experiences across all channels and interactions. We specialise in unlocking latent value within organizations through a three-pronged approach that focuses on identifying and enhancing value for customers, within the customer base, and within the business itself. In doing so, we foster sustainable value, paving the way for consistent business growth.\nWe work with leading consumer brands to tackle and overcome complex business and customer problems to drive transformation and champion customer-centric agendas. We are proud to include some of the leading UK and global brands among our current clients such as Lloyds Banking Group, NatWest Group, Bank of Ireland, Nationwide, Aviva, Biogen, Eon Next, OVO, Virgin Media O2, BT, HMD Global, Centrica, and GSK. We are obsessive about delivering value for our clients and work in a collaborative, engaged, and creative way with our colleagues and clients. We strive to support the transition of knowledge and capability into strategic teams.","description_format":"text","description_chars":4817,"description_truncated":false,"requirements":{"experience_years_min":2,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"phd","optional":false},"security_clearance":false,"languages":[]},"benefits":["Continuous learning","Professional development"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":[],"lifecycle":[{"event":"open","at":"2026-09-11T05:29:22Z"}],"visa":[{"country":"GB","licensed_sponsor":true,"evidence":"Licensed UK visa sponsor (Skilled Worker)","filings_12m":null,"filings_prev_12m":null,"green_card_filings_12m":null,"median_offered_wage_usd":null,"route":"Skilled Worker","cap_exempt":false,"checked_at":"2026-10-31T06:23:40+00:00","sources":["UK Home Office: register of licensed sponsors (workers)"],"filings_for_role_12m":0}],"liveness":{"score":8,"band":"cold","label":"Long shot","p_open":1,"p_active":0.276,"p_room":0.28,"age_days":386,"expected_fill_days":84,"reasons":["conf:0","win:tail","crowd:junior"],"computed_at":"2026-11-01T06:10:25Z"},"pay":null,"html_url":"https://alion.io/job/optimapartners-core-data-engineer","json_url":"https://alion.io/job/optimapartners-core-data-engineer.json","meta":{"generated_at":"2026-11-02T01:34:40Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2471,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-03T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":678916},"rest":"https://alion.io/mcp/rest/get_company?id=678916"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Foptimapartners-core-data-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Foptimapartners-core-data-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Foptimapartners-core-data-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/optimapartners-core-data-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Foptimapartners-core-data-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":714008}},{"tool":"search_jobs","arguments":{"company":"optimapartners.co.uk","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":714008},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}