Quality Assurance Engineer

We are seeking a highly motivated and detail-oriented Quality Assurance (QA) Engineer to join our team. You will play a key role in ensuring the reliability, accuracy, and user experience of our conversational AI systems. This role involves testing dialogue models, integrations with NLP/LLM-based backends, and end-to-end conversational workflows across multiple platforms (chat, voice, and integrations with enterprise systems).

The core responsibilities for the job include the following:

Test Planning and Strategy:

• Develop and execute comprehensive test plans for conversational AI features, including NLP pipelines, dialogue flows, integrations, and APIs.
• Design functional, regression, performance, and exploratory test cases specific to conversational interfaces.

Conversational Testing:

• Validate intent recognition, entity extraction, context handling, and multi-turn dialogue management.
• Test conversational flows for accuracy, completeness, and user experience.
• Perform scenario-based and edge-case testing to capture unexpected user inputs.

Automation and Tooling:

• Build automated test scripts for API endpoints, dialogue flows, and ML model outputs.
• Work with frameworks like Postman, Cypress, PyTest, or custom NLP testing tools.
• Contribute to CI/CD pipelines with automated regression tests.
• Collaborate with data scientists, ML engineers, and product managers to define acceptance criteria for AI-driven features.
• Provide feedback on conversational design and model performance.
• Collaborate with DevOps on load testing and monitoring production systems.

Quality Advocacy:

• Ensure conversational AI products meet high standards for accuracy, robustness, fairness, and usability.
• Help define and track KPIs (e. g., intent accuracy, response latency, conversation success rate).

Requirements:

• 3+ years of QA/testing experience, preferably with AI, ML, or NLP products.
• Strong understanding of API testing, automated frameworks, and scripting languages (Python, JavaScript, or similar).
• Experience testing chat agents, voice agents, or conversational interfaces is a plus.
• Familiarity with NLP concepts: intents, entities, context, LLMs, and dialogue management frameworks (e. g., Rasa, Dialogflow, and LLM APIs).
• Experience with CI/CD pipelines and version control (Git).
• Excellent problem-solving, analytical, and communication skills.
• Exposure to A/B testing and model evaluation.
• Experience with cloud platforms (AWS, GCP, Azure).
• Familiarity with accessibility and localization testing for multilingual agents.

Skills

API, Cypress, JavaScript, Postman, PyTest, Python, QA, quality, testing