Senior/Lead Quality Engineer - Chatbot
Jobgether
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior/Lead Quality Engineer - Chatbot based in Canada.
This role offers the opportunity to define quality engineering practices for next-generation AI-powered products.
You will join a multidisciplinary engineering team building conversational AI capabilities and ensuring reliable, trustworthy user experiences.
As a founding quality expert for the chatbot product, you will shape testing strategies, release processes, and quality standards from the ground up.
The role combines software quality engineering, automation, AI testing, and experimentation in a fast-moving technical environment.
You will work closely with software engineers, machine learning specialists, and product teams to improve reliability and accelerate innovation.
This is an opportunity for a technically curious engineer who wants to influence how quality is built for dynamic, LLM-powered systems.
Accountabilities:
The Senior/Lead Quality Engineer will be responsible for establishing and evolving quality practices for AI-driven conversational products. This role will combine hands-on testing, technical collaboration, and strategic quality improvements to ensure reliable releases and strong user confidence.
- Develop and continuously improve quality strategies, testing frameworks, and release processes for chatbot products.
- Establish evaluation methodologies to measure chatbot quality, reliability, performance, and regression risks.
- Design testing approaches for dynamic and non-deterministic AI systems that go beyond traditional testing methods.
- Identify, validate, and improve AI safety measures, including protection against prompt injection, unexpected outputs, and model-related risks.
- Collaborate closely with software engineers through code reviews, technical discussions, and debugging sessions to improve quality throughout the development lifecycle.
- Analyze existing testing practices, identify gaps, research new approaches, and recommend practical improvements.
- Design proof-of-concepts for innovative testing tools and methodologies, helping productionize solutions that deliver value.
- Support release readiness by defining quality signals, validation processes, and acceptance criteria.
- Contribute performance testing strategies for AI and LLM-powered workflows.
- Share knowledge, promote quality engineering best practices, and help build a strong quality culture across engineering teams.
- Mentor engineers and influence testing practices within cross-functional product teams.
Requirements:
The ideal candidate has strong software quality engineering experience combined with technical curiosity and an interest in testing AI-driven systems. They should be comfortable operating in uncertain environments and creating practical solutions for complex engineering challenges.
- Strong hands-on experience in software quality engineering, SDET, test automation, or a similar engineering role.
- Experience testing chatbot, conversational AI, RAG, LLM-powered applications, or other systems with dynamic and non-deterministic behavior.
- Proficiency in at least one programming language with the ability to understand production code and collaborate effectively with software engineers.
- Experience designing and implementing automated testing strategies and quality frameworks.
- Strong analytical and investigative mindset with the ability to explore complex problems and turn findings into actionable improvements.
- Ability to evaluate technical trade-offs and prioritize quality initiatives based on risk, user impact, and business objectives.
- Excellent written and verbal communication skills, with the ability to explain technical concepts and testing decisions clearly.
- Strong collaboration skills and experience working with Product, Engineering, and Machine Learning teams.
- Experience working in agile, fast-paced, and evolving environments.
Nice to have:
- Experience with AI security, safety testing, or responsible AI practices.
- Experience testing conversational AI platforms, chatbot systems, or agent-based workflows.
- Previous experience as the first Quality Engineer within a team.
- Experience mentoring engineers or influencing quality practices across organizations.
- Passion for staying current with emerging technologies and sharing new methodologies with teams.
Benefits:
- Fully remote work flexibility from Canada.
- Competitive compensation package aligned with experience, skills, and market standards.
- Stock options and opportunity to participate in the growth of an innovative technology company.
- Company-provided laptop, remote setup support, and coworking budget.
- Flexible working hours and location independence.
- Generous paid time off in addition to local public holidays.
- Enhanced parental leave benefits.
- Health and retirement benefits.
- Annual learning and development budget to support professional growth.
- Opportunities to participate in impactful AI projects shaping the future of work.
- Virtual and in-person team events, including annual work opportunities and company gatherings.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.