Tester for testing Context Studio

Job Description: Tester - Context Studio (AI Agents & Dashboards)

Experience: 5-8 years

Location: WFH

About the role

We're looking for an analytical, detail-oriented tester to validate AI agents and dashboards built on Context Studio. You'll support the development of critical dashboards, help make real-time data insights reliable for business decisions, and ensure compliance with data governance and security standards.

Key responsibilities

• Test AI agents configured in Context Studio for accuracy, consistency and handling of edge cases
• Turn business questions and KPIs into test scenarios with clearly defined expected results
• Check agent responses and insights against trusted source data
• Find and report context gaps, wrong metric definitions, misleading or made-up responses, and inconsistent behaviour
• Run regression tests whenever business context, data models or agent settings change
• Test critical dashboards for data accuracy, real-time freshness, filters, drill-downs and usability
• Check compliance with data governance and security standards, including access controls, protection of sensitive data and use of approved data sources
• Work with business stakeholders, data engineers and developers to clarify requirements and resolve defects
• Log defects clearly, track them to closure, and share quality status and release-readiness reports

Required skills

• 5-8 years of experience in software or data testing
• A strong tester mindset: curious and sceptical, with exploratory testing skills and attention to detail
• Business orientation: understands KPIs, metrics and business processes, and can judge whether an insight makes business sense
• Understanding of AI agents: how they use context to answer questions, and common failures such as hallucinations, ambiguity and inconsistent answers
• Experience testing dashboards, reports or data-driven applications
• Basic SQL knowledge to check data independently
• Good written and verbal communication

Good to have

• Working knowledge of Snowflake
• Exposure to testing AI/LLM-based applications or evaluation frameworks
• Familiarity with data governance and data privacy practices
• Basic scripting (e.g., Python) for test automation or data checks
• QA certification (e.g., ISTQB)