Staff Data Solutions Engineer

Job description:

The Staff Data Solutions Engineer plays a critical role in shaping, designing and delivering data and AI solutions that translate complex business needs into scalable, secure and reusable capabilities. Operating at staff level, this role provides senior technical leadership across data products, data platforms and AI-enabled solutions, with clear accountability for end-to-end system design from high-level architecture through to low-level implementation design.

This is a hands-on engineering leadership role. The postholder independently identifies problems to solve, leads the technical design of solutions and acts as technical lead across one large-scale or multiple medium-scale delivery workstreams. A strong command of modern engineering patterns and practices is essential, as this role sets the standard for others to follow.

The role carries clear technical leadership accountability without line management responsibility. It guides delivery teams, influences internal and third-party engineering outcomes, and ensures solutions are well designed, maintainable, secure and fit for enterprise use. The postholder is also expected to mentor others and actively contribute to engineering excellence across the wider team.

You will work with:

This role sits within the Central Engineering team within Finance Technology and works closely with data engineers, AI and software engineers, product managers, architects, cyber, platform and business stakeholder teams. It also partners with central technology and architecture teams to ensure solutions are designed for reuse, interoperability and future scalability, rather than being developed in isolation.

You will deliver:

• End-to-end system design: Translate high-level architectural direction into detailed, buildable solution designs. This includes decomposing architectural patterns, such as event-driven ingestion, medallion lakehouse or RAG pipelines, into concrete engineering specifications including table schemas, transformation logic, interface contracts, retry and error-handling strategies, SLA thresholds and observability requirements. Produce HLD artefacts that communicate solution intent and platform choices, as well as LLD artefacts that delivery engineers can implement without ambiguity. Design outputs should be reviewable through architecture governance and usable as the baseline for technical assurance.
• Strong engineering practices and patterns: Set and reinforce standards for software design, code quality, testing, CI/CD, observability and documentation. Advocate for and apply established engineering patterns, including event-driven architecture, microservices, pipeline orchestration, RAG and layered data architecture, with sound judgement on when each pattern is appropriate. Apply these patterns effectively across data-intensive pipelines, platforms and applications.
• Scalable data infrastructure and product delivery: Design, build and maintain reliable data pipelines, data products and integration layers that move, process and serve data across the enterprise at scale.
• AI-enabled solution engineering: Integrate AI and machine learning capabilities into production-grade applications, including LLM-based solutions, retrieval-augmented generation, AI agents and unstructured data processing. Evaluate AI tooling pragmatically, prioritising business value and sustainable delivery over experimentation.
• Reusable, scalable and pragmatic solution design: Apply clear judgement on when to build custom capability, configure existing platforms, adopt SaaS solutions or use low-code/no-code approaches to deliver faster and more sustainable outcomes. Ensure technology decisions consider broader enterprise value beyond the immediate use case by identifying opportunities for common patterns, shared services, platform-based delivery, code reuse and the active reduction of technical debt.
• Technical governance and design assurance: Work with architecture, cyber, data governance and platform teams to ensure solutions meet enterprise standards for security, resilience, compliance, performance and operational readiness.
• Technical leadership across internal and partner teams: Provide technical direction and assurance across internal engineering teams, third-party partners and delivery squads to ensure engineering quality, architectural alignment, sustainable ownership and long-term maintainability of delivered solutions.
• Operational excellence and lifecycle ownership: Design for supportability, observability, reliability and cost effectiveness. Define service reliability expectations, SLA requirements and appropriate site-reliability engineering practices for systems owned or led.
• Hands-on delivery and prototyping: Translate ambiguous problems into practical technology options, solution designs and working prototypes. Use AI tools to accelerate design and build activities and validate ideas rapidly with users.
• Technical mentorship: Mentor engineers across the team, contribute to data engineering learning and development, and participate in the evolution of engineering communities, standards and practices.

What you will need to be successful Essential:

• Bachelor's degree in Computer Science or a related field.
• 8+ years of hands-on experience designing, building, productionising and maintaining reliable, scalable data infrastructure and data products in complex environments.
• Demonstrable experience producing HLD and LLD artefacts for data or technology solutions, including solution architecture diagrams, data flow designs, component specifications, API contracts, data models and operational runbooks.
• Ability to take architectural decisions and decompose them into engineering specifications that developers can build from without further clarification.
• Strong command of software engineering best practices, including technical design review, unit testing, CI/CD pipeline development, monitoring and alerting, code review and documentation.
• Development experience in one or more object-oriented programming languages, with Python preferred. Scala, Java or C# are also applicable.
• Expert SQL knowledge and experience working with large-scale distributed data systems.
• Experience integrating AI and machine learning capabilities into production systems, including LLMs via APIs, RAG patterns, vector search, embeddings or AI agents.
• Deep knowledge and hands-on experience across the full data lifecycle.
• Strong collaborator management skills and the ability to lead through technical influence.
• A continuous learning and improvement perspective.

Desired:

• Experience with AI/ML frameworks such as LangChain, LlamaIndex, Hugging Face, Weaviate or similar ecosystems.
• Familiarity with agentic frameworks such as CrewAI, AutoGen or similar LLM orchestration toolkits.
• Experience with big data technologies such as Spark, Hadoop or Hive.
• Familiarity with cloud data platforms such as AWS, Azure or GCP, and infrastructure-as-code practices.
• Experience with data governance tools such as Alation or Collibra, and enterprise data management standards.
• No prior experience in the energy industry is required.

Travel Requirement:

Up to 10% travel should be expected with this role

Relocation Assistance:

This role is eligible for relocation within country

Remote Type:

This position is a hybrid of office/remote working

Skills

AI Agents, C#, Embeddings, Java, LLM, Python, RAG, SQL, Scala, Vector Search