{"id":2205882,"url":"https://alion.io/job/klipboard-senior-applied-ai-engineer-prompting-evaluation","title":"Senior Applied AI Engineer – Prompting & Evaluation","company":{"id":2718560,"name":"Klipboard","domain":"klipboard.com","url":"https://alion.io/company/klipboard-com","size_band":"51-200","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"ADP","truth_index":{"grade":"B","score":77,"open_postings":11,"ghost_share":0,"stale_share":0.909,"repost_share":0,"time_to_fill_p50_days":45,"computed_at":"2026-10-28T05:59:10Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Johannesburg, South Africa"],"countries":["ZA"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":34000,"max_usd":88000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":2365},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"Claude","optional":false},{"name":"Context Engineering","optional":false},{"name":"Copilot","optional":false},{"name":"Cursor","optional":false},{"name":"Embeddings","optional":false},{"name":"Llama","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"Prompt Engineering","optional":false},{"name":"RAG","optional":false},{"name":"Fine-tuning","optional":true},{"name":"Function Calling","optional":true},{"name":"LangChain","optional":true},{"name":"LLM","optional":true},{"name":"Semantic Kernel","optional":true},{"name":"Structured Outputs","optional":true}],"status":"live","first_seen_at":"2026-09-07T13:19:00Z","employer_posted_date":"2026-09-07","last_verified_at":"2026-10-29T02:17:27Z","board_verified":true,"closed_at":null,"days_open":51,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":51},"description":"\"At Klipboard we've introduced a flexible hybrid work policy, where employees spend three days in the office and two days working from home. This approach promotes a balanced work environment that combines office collaboration with the comfort and convenience of remote work.\"\nKlipboard provides specialist software, services and support to deliver fully integrated trading and business management solutions to companies in the distributive trade - wherever they are in the world. With a unique depth of knowledge and experience in ERP/SaaS solutions, Klipboard has a wide range of clients includes wholesalers, distributors, merchants and retailers from small traders to multinational enterprises. Klipboard has offices in the UK, Ireland, The Netherlands, South Africa, Kenya and North America. Our mission is simple: to design and deliver high performance, integrated ERP solutions that enable our distributive trade customers to source effectively, stock efficiently, sell profitably and service competitively.\nKlipboard is a global, growing business that embraces AI and emerging technologies to enhance customer outcomes, collaboration, and continuous improvement. We’re looking for people who are curious about or fluid with AI, open to change, and excited to learn how technology can improve the way we work and help our customers which is always supported by strong human insight and communication.\nThis is a high-impact senior engineering role for someone who wants to shape how a large engineering organisation uses AI in real products and real development work. You will set patterns, create evaluation standards, coach engineers and help Klipboard turn AI from individual experimentation into repeatable, scalable engineering practice.\nThe role is two jobs in one: the craft of getting reliable, measurable results from large language models through prompt engineering, context design and evaluation; and making everyone around you better at it. With hundreds of engineers across the UK and South Africa, the difference between an organisation where a few people prompt well and one where everyone does is enormous. Your success is measured less by what you personally produce and more by how quickly the engineers around you improve.\nKey Responsibilities: \nRole Accountabilities\nPractise the craft at a high level - designing, building and refining prompts, context strategies and agentic workflows for AI features in our products.\nGrow the engineers around you - coaching, pairing, workshops and building shared assets that spread good practice across hundreds of engineers.\nSet standards and keep the organisation honest - defining what good looks like for prompting, evaluation and context engineering, and maintaining those standards as tools change.\nKey Activities and Contributions\nDesign, build and refine prompts, context strategies and agentic workflows for AI features in Klipboard products, where a confidently wrong price, part match or stock answer is worse than no answer.\nBuild evaluation into everything - test prompts against real business cases at scale, measure quality honestly, and treat “it looked fine when I tried it“ as the start of the work, not the end.\nWork across models and providers, choosing pragmatically on quality, cost and latency rather than loyalty, and keeping up as models, tools and providers change quickly.\nGet deep into the domain detail with product teams and subject matter experts, because a prompt is only as good as the understanding of the business problem behind it.\nCoach engineers across teams to get dramatically more from AI coding tools and LLMs, through pairing, reviews, workshops and patient one-to-one help.\nBuild and maintain shared prompt patterns, reusable templates, evaluation harnesses and internal guides, so good practice spreads as assets rather than folklore.\nRun an internal community of practice where engineers share what is working, what failed and why, and make it somewhere people actually want to show up.\nHelp define what good looks like for prompting, context engineering and evaluation across R&D, and keep those standards current as tools and models change.\nShare measurable results with engineering leadership, including where AI tooling is genuinely paying off and where it is not.\nSystems, Tools and Technology\nLarge language model APIs across multiple providers (GPT, Claude, Llama and equivalents)\nAI coding tools: GitHub Copilot, Cursor or equivalents\nPrompt engineering, context design and agentic workflow frameworks\nEvaluation harnesses, evaluation datasets and automated test suites for AI outputs\nRetrieval-augmented generation (RAG), vector search, embeddings and retrieval pipelines (desirable)\nLLM orchestration frameworks (LangChain, Semantic Kernel or equivalents - desirable)\nTechnical and Professional Expertise\nStrong software engineering background with several years building and shipping production software.\nSubstantial hands-on experience with large language models: prompt design, context engineering and structured outputs, in real work rather than personal experiments alone.\nExperience building evaluations or test harnesses for LLM outputs, and using them to improve quality measurably.\nClear evidence of growing other engineers - through mentoring, teaching, internal training, communities of practice or similar.\nDaily working fluency with AI coding tools such as GitHub Copilot, Cursor or equivalents, with a practical view of where they help and where they mislead.\nCore Responsibilities and Contributions\nDeliver AI features in Klipboard products with proper evaluation and measurable quality standards.\nEmbed prompt engineering and evaluation best practice across the R&D organisation.\nBe the sceptical voice when something demos well but will not hold up in production; be the encouraging one when a genuinely good idea needs backing.\nMaintain high standards for safety, accuracy and data handling in everything that ships - customers run their businesses on the answers our software gives them.\nBuild and sustain a living library of prompt patterns, templates and guides.\nMeet engineers where they are, including those working in long-established codebases, where getting real value from AI tools is some of the highest-leverage work in the company.\nCustomer Experience\nEnsure AI features in Klipboard products give accurate, safe and trustworthy answers - customers run their businesses on our output.\nWork with product teams and subject matter experts to understand the business problem behind each feature properly before building.\nPrevent confidently wrong answers - incorrect prices, part matches or stock levels - from reaching customers.\nKey Outcomes and Activities\nEngineers across several teams can point to specific things they now do better because of coaching and knowledge-sharing.\nAt least one AI feature or workflow shipped per quarter with proper evaluation behind it.\nA living, used library of prompt patterns, templates and guides in place and growing.\nCommunity of practice running, well attended, and producing outputs teams actually reuse.\nEngineering leadership has a clearer, evidence-based picture of where AI is paying off.\nPeople, Collaboration & Culture\nGenerous with knowledge and genuinely energised by other people getting better. If the instinct is to guard expertise rather than spread it, this is the wrong role.\nRigorous and honest about results - measures, shows working, and says so when data disagrees with the enthusiasm.\nPatient with engineers at every level of AI maturity, from those who live in these tools to those who are quietly sceptical of the whole thing.\nCurious about the trades Klipboard customers work in, because the best prompts are full of domain understanding.\nCalm about the pace of change - treats model and tooling shifts as the interesting part of the job, not a threat.\nWorks closely with engineering leadership, architecture, product and cross-site engineering teams across the UK and South Africa.\nAdditional Responsibilities (as aligned to company job architecture)\nSome travel between Klipboard's UK and South Africa offices as part of the coaching and community-building remit.\nKeep standards current as models, tools and providers change rapidly.\nRepresent Klipboard's engineering practice externally where appropriate - talks, writing or open-source contributions.\nKey Relationships (Internal and external)\nInternal\nEngineering leadership and architecture teams\nR&D engineering teams across Newcastle, Tankersley, Hungerford, Johannesburg and Cape Town / Stellenbosch\nProduct managers and product owners\nSubject matter experts in Klipboard's product verticals (distributive trades, rental, automotive)\nExternal\nAI and LLM platform and tooling providers (as needed)\nRequired Qualifications and Experience\nStrong software engineering background with several years building and shipping production software - this role needs an engineer first.\nSubstantial hands-on experience with large language models: prompt design, context engineering and structured outputs, in real work rather than personal experiments alone.\nExperience building evaluations or test harnesses for LLM outputs, and using them to improve quality measurably.\nClear evidence of growing other engineers - through mentoring, teaching, internal training, communities of practice or similar.\nDaily working fluency with AI coding tools such as GitHub Copilot, Cursor or equivalents, with a practical view of where they help and where they mislead.\nPreferred Qualifications and Experience\nRetrieval-augmented generation, agentic workflows, tool use or fine-tuning in production settings.\nExperience with LLM APIs, orchestration frameworks, vector search, embeddings, retrieval pipelines, evaluation datasets or automated test suites for AI outputs.\nExperience across more than one model provider, with a feel for their different strengths.\nDomain exposure to any of Klipboard's sectors: distributive trades, rental, retail, automotive aftermarket parts or garage management.\nExperience helping teams adopt AI tooling in established or legacy codebases, not just greenfield projects.\nPublic sharing of your work: internal talks given externally, writing or open-source contributions.\nKlipboard is embracing AI at pace across our products and ways of working. We’re looking for people who are curious about how AI can enhance productivity, decision-making and customer outcomes, and who are open to learning and adapting as this space evolves.\nWhat Success in This Role Looks Like\nEngineers across several teams can point to specific things they now do better because of coaching and knowledge-sharing.\nAt least one AI feature or workflow shipped with proper evaluation behind it, and the team involved could now repeat the process without you.\nA living, used library of prompt patterns, templates and guides that did not exist before.\nThe community of practice is running, well attended, and producing things teams actually reuse.\nEngineering leadership has a clearer, evidence-based picture of where AI is paying off across the organisation.\nCompany Info\nYou may also have seen from our recent posts that we are excited to begin sharing our new company name - Klipboard. Kerridge Commercial Systems (KCS) is becoming Klipboard and our new brand is designed to bring together our expertise across distribution, automotive, retail, rental, transport management, manufacturing, and field service management. We have offices based across the world and we are looking for talented individuals to join our growing teams. Due to our growth over the last few years it is an exciting time to join us as we enter our next chapter! At Klipboard we've introduced a flexible hybrid work policy, where employees spend three days in the office and two days working from home. This approach promotes a balanced work environment that combines office collaboration with the comfort and convenience of remote work.\"\nEqual Opportunities\nAs a global company, we...","description_format":"text","description_chars":13352,"description_truncated":true,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity","Hybrid work"],"hiring_locations":[{"name":"South Africa","iso":"ZA","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["ERP & Business Management Software"],"lifecycle":[{"event":"open","at":"2026-10-09T23:22:30Z"}],"visa":[],"liveness":{"score":31,"band":"fade","label":"Fading","p_open":1,"p_active":0.568,"p_room":0.55,"age_days":50,"expected_fill_days":45,"reasons":["conf:1","stale_co","velocity","win:tail"],"computed_at":"2026-10-28T05:59:10Z"},"pay":null,"html_url":"https://alion.io/job/klipboard-senior-applied-ai-engineer-prompting-evaluation","json_url":"https://alion.io/job/klipboard-senior-applied-ai-engineer-prompting-evaluation.json","meta":{"generated_at":"2026-10-29T04:50:22Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":7540,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-30T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":2718560},"rest":"https://alion.io/mcp/rest/get_company?id=2718560"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fklipboard-senior-applied-ai-engineer-prompting-evaluation"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fklipboard-senior-applied-ai-engineer-prompting-evaluation"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fklipboard-senior-applied-ai-engineer-prompting-evaluation"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/klipboard-senior-applied-ai-engineer-prompting-evaluation\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fklipboard-senior-applied-ai-engineer-prompting-evaluation"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":2205882}},{"tool":"search_jobs","arguments":{"company":"klipboard.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":2205882},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}