Senior Applied AI Engineer – Prompting & Evaluation

"At Klipboard we've introduced a flexible hybrid work policy, where employees spend three days in the office and two days working from home. This approach promotes a balanced work environment that combines office collaboration with the comfort and convenience of remote work."

Klipboard provides specialist software, services and support to deliver fully integrated trading and business management solutions to companies in the distributive trade - wherever they are in the world. With a unique depth of knowledge and experience in ERP/SaaS solutions, Klipboard has a wide range of clients includes wholesalers, distributors, merchants and retailers from small traders to multinational enterprises. Klipboard has offices in the UK, Ireland, The Netherlands, South Africa, Kenya and North America. Our mission is simple: to design and deliver high performance, integrated ERP solutions that enable our distributive trade customers to source effectively, stock efficiently, sell profitably and service competitively.

Klipboard is a global, growing business that embraces AI and emerging technologies to enhance customer outcomes, collaboration, and continuous improvement. We’re looking for people who are curious about or fluid with AI, open to change, and excited to learn how technology can improve the way we work and help our customers which is always supported by strong human insight and communication.

This is a high-impact senior engineering role for someone who wants to shape how a large engineering organisation uses AI in real products and real development work. You will set patterns, create evaluation standards, coach engineers and help Klipboard turn AI from individual experimentation into repeatable, scalable engineering practice.

The role is two jobs in one: the craft of getting reliable, measurable results from large language models through prompt engineering, context design and evaluation; and making everyone around you better at it. With hundreds of engineers across the UK and South Africa, the difference between an organisation where a few people prompt well and one where everyone does is enormous. Your success is measured less by what you personally produce and more by how quickly the engineers around you improve.

Key Responsibilities:

Role Accountabilities

• Practise the craft at a high level - designing, building and refining prompts, context strategies and agentic workflows for AI features in our products.
• Grow the engineers around you - coaching, pairing, workshops and building shared assets that spread good practice across hundreds of engineers.
• Set standards and keep the organisation honest - defining what good looks like for prompting, evaluation and context engineering, and maintaining those standards as tools change.

Key Activities and Contributions

• Design, build and refine prompts, context strategies and agentic workflows for AI features in Klipboard products, where a confidently wrong price, part match or stock answer is worse than no answer.
• Build evaluation into everything - test prompts against real business cases at scale, measure quality honestly, and treat “it looked fine when I tried it“ as the start of the work, not the end.
• Work across models and providers, choosing pragmatically on quality, cost and latency rather than loyalty, and keeping up as models, tools and providers change quickly.
• Get deep into the domain detail with product teams and subject matter experts, because a prompt is only as good as the understanding of the business problem behind it.
• Coach engineers across teams to get dramatically more from AI coding tools and LLMs, through pairing, reviews, workshops and patient one-to-one help.
• Build and maintain shared prompt patterns, reusable templates, evaluation harnesses and internal guides, so good practice spreads as assets rather than folklore.
• Run an internal community of practice where engineers share what is working, what failed and why, and make it somewhere people actually want to show up.
• Help define what good looks like for prompting, context engineering and evaluation across R&D, and keep those standards current as tools and models change.
• Share measurable results with engineering leadership, including where AI tooling is genuinely paying off and where it is not.

Systems, Tools and Technology

• Large language model APIs across multiple providers (GPT, Claude, Llama and equivalents)
• AI coding tools: GitHub Copilot, Cursor or equivalents
• Prompt engineering, context design and agentic workflow frameworks
• Evaluation harnesses, evaluation datasets and automated test suites for AI outputs
• Retrieval-augmented generation (RAG), vector search, embeddings and retrieval pipelines (desirable)
• LLM orchestration frameworks (LangChain, Semantic Kernel or equivalents - desirable)

Technical and Professional Expertise

• Strong software engineering background with several years building and shipping production software.
• Substantial hands-on experience with large language models: prompt design, context engineering and structured outputs, in real work rather than personal experiments alone.
• Experience building evaluations or test harnesses for LLM outputs, and using them to improve quality measurably.
• Clear evidence of growing other engineers - through mentoring, teaching, internal training, communities of practice or similar.
• Daily working fluency with AI coding tools such as GitHub Copilot, Cursor or equivalents, with a practical view of where they help and where they mislead.

Core Responsibilities and Contributions

• Deliver AI features in Klipboard products with proper evaluation and measurable quality standards.
• Embed prompt engineering and evaluation best practice across the R&D organisation.
• Be the sceptical voice when something demos well but will not hold up in production; be the encouraging one when a genuinely good idea needs backing.
• Maintain high standards for safety, accuracy and data handling in everything that ships - customers run their businesses on the answers our software gives them.
• Build and sustain a living library of prompt patterns, templates and guides.
• Meet engineers where they are, including those working in long-established codebases, where getting real value from AI tools is some of the highest-leverage work in the company.

Customer Experience

• Ensure AI features in Klipboard products give accurate, safe and trustworthy answers - customers run their businesses on our output.
• Work with product teams and subject matter experts to understand the business problem behind each feature properly before building.
• Prevent confidently wrong answers - incorrect prices, part matches or stock levels - from reaching customers.

Key Outcomes and Activities

• Engineers across several teams can point to specific things they now do better because of coaching and knowledge-sharing.
• At least one AI feature or workflow shipped per quarter with proper evaluation behind it.
• A living, used library of prompt patterns, templates and guides in place and growing.
• Community of practice running, well attended, and producing outputs teams actually reuse.
• Engineering leadership has a clearer, evidence-based picture of where AI is paying off.

People, Collaboration & Culture

• Generous with knowledge and genuinely energised by other people getting better. If the instinct is to guard expertise rather than spread it, this is the wrong role.
• Rigorous and honest about results - measures, shows working, and says so when data disagrees with the enthusiasm.
• Patient with engineers at every level of AI maturity, from those who live in these tools to those who are quietly sceptical of the whole thing.
• Curious about the trades Klipboard customers work in, because the best prompts are full of domain understanding.
• Calm about the pace of change - treats model and tooling shifts as the interesting part of the job, not a threat.
• Works closely with engineering leadership, architecture, product and cross-site engineering teams across the UK and South Africa.

Additional Responsibilities (as aligned to company job architecture)

• Some travel between Klipboard's UK and South Africa offices as part of the coaching and community-building remit.
• Keep standards current as models, tools and providers change rapidly.
• Represent Klipboard's engineering practice externally where appropriate - talks, writing or open-source contributions.

Key Relationships (Internal and external)

Internal

• Engineering leadership and architecture teams
• R&D engineering teams across Newcastle, Tankersley, Hungerford, Johannesburg and Cape Town / Stellenbosch
• Product managers and product owners
• Subject matter experts in Klipboard's product verticals (distributive trades, rental, automotive)

External

• AI and LLM platform and tooling providers (as needed)

Required Qualifications and Experience

• Strong software engineering background with several years building and shipping production software - this role needs an engineer first.
• Substantial hands-on experience with large language models: prompt design, context engineering and structured outputs, in real work rather than personal experiments alone.
• Experience building evaluations or test harnesses for LLM outputs, and using them to improve quality measurably.
• Clear evidence of growing other engineers - through mentoring, teaching, internal training, communities of practice or similar.
• Daily working fluency with AI coding tools such as GitHub Copilot, Cursor or equivalents, with a practical view of where they help and where they mislead.

Preferred Qualifications and Experience

• Retrieval-augmented generation, agentic workflows, tool use or fine-tuning in production settings.
• Experience with LLM APIs, orchestration frameworks, vector search, embeddings, retrieval pipelines, evaluation datasets or automated test suites for AI outputs.
• Experience across more than one model provider, with a feel for their different strengths.
• Domain exposure to any of Klipboard's sectors: distributive trades, rental, retail, automotive aftermarket parts or garage management.
• Experience helping teams adopt AI tooling in established or legacy codebases, not just greenfield projects.
• Public sharing of your work: internal talks given externally, writing or open-source contributions.
• Klipboard is embracing AI at pace across our products and ways of working. We’re looking for people who are curious about how AI can enhance productivity, decision-making and customer outcomes, and who are open to learning and adapting as this space evolves.

What Success in This Role Looks Like

• Engineers across several teams can point to specific things they now do better because of coaching and knowledge-sharing.
• At least one AI feature or workflow shipped with proper evaluation behind it, and the team involved could now repeat the process without you.
• A living, used library of prompt patterns, templates and guides that did not exist before.
• The community of practice is running, well attended, and producing things teams actually reuse.
• Engineering leadership has a clearer, evidence-based picture of where AI is paying off across the organisation.

Company Info

You may also have seen from our recent posts that we are excited to begin sharing our new company name - Klipboard. Kerridge Commercial Systems (KCS) is becoming Klipboard and our new brand is designed to bring together our expertise across distribution, automotive, retail, rental, transport management, manufacturing, and field service management. We have offices based across the world and we are looking for talented individuals to join our growing teams. Due to our growth over the last few years it is an exciting time to join us as we enter our next chapter! At Klipboard we've introduced a flexible hybrid work policy, where employees spend three days in the office and two days working from home. This approach promotes a balanced work environment that combines office collaboration with the comfort and convenience of remote work."

Equal Opportunities

As a global company, we value and respect the diversity of our workforce, aiming to empower everyone to embrace each other's differences. We are committed to creating an inclusive workplace where diversity, equity, and inclusion are integral to our company and culture. We recognize the benefits of a diverse workforce, where creativity and valuing differences enable us all to thrive and sparks innovation.

If you require any help, adjustments and/or support during the interview and offer process then please advise our TA or HR team.

Research shows that women and other underrepresented groups are less likely to apply for a role unless they meet every listed requirement. However, we recognise that skills and experience come in many forms, and we encourage you to apply even if you don’t meet every criterion. If you are passionate about this role and believe you have the right mindset and transferrable skills, we would love to hear from you!

To all recruitment agencies: Klipboard does not accept agency speculative resumes. At present we only accept CV’s from Agencies on our PSL who have been assigned specific position/s. Please do not forward resumes to our careers site or direct to Klipboard employee as this does not constitute an introduction and Klipboard retrospectively will not be liable for any candidate ownership or fees related to unsolicited resumes.