Quality Assurance Automation Engineer
First seen by Alion on Sep 16, 2026.
Role Overview :
We are seeking a seasoned QA Automation Engineer with a specialized focus on AI/ML testing to join our high-growth SaaS engineering team. In this role, you will be responsible for architecting and executing robust automation frameworks that ensure the reliability and performance of complex AI-driven features. You will collaborate closely with data scientists, product managers, and backend developers to bridge the gap between traditional software quality and the non-deterministic nature of machine learning models. By implementing advanced testing strategies, you will directly influence the quality of our product releases, ensuring that our AI solutions deliver consistent, high-value outcomes for our global enterprise clients.
Key Responsibilities :
- Design and maintain scalable automation frameworks using Playwright and Python to validate complex SaaS workflows and UI components.
- Develop specialized test suites for AI/ML models to evaluate model accuracy, bias, and performance, ensuring our intelligent features meet rigorous production standards.
- Collaborate with cross-functional engineering teams to integrate automated testing into CI/CD pipelines, reducing manual intervention and accelerating deployment cycles.
- Analyze complex system behaviors and edge cases to proactively identify potential failures, safeguarding the user experience against regressions.
- Mentor junior team members on best practices in automation and AI testing methodologies to foster a culture of technical excellence and continuous improvement.
Required Skillset :
- Demonstrated expertise in building end-to-end automation frameworks using Playwright and Python within a fast-paced SaaS environment.
- Proven ability to design and execute test strategies for AI/ML-based applications, including data validation and model output verification.
- Strong analytical mindset with the ability to communicate complex technical quality metrics to both technical stakeholders and product leadership.
- Exceptional problem-solving skills and the ability to thrive in a remote-first work environment, maintaining high levels of autonomy and collaboration across distributed teams.
- A Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field, supported by 5 - 10 years of professional experience in software quality assurance.
Skills
Automation Testing, AI/ML Testing, SaaS, Playwright Testing, Python, Software Quality Assurance, GUI Testing, Artificial Intelligence, Machine Learning