QA Automation Engineer
First seen by Alion on Oct 15, 2026.
QA Automation Engineer
Python + Playwright / Java + Selenium / TOSCA & Enterprise AI Systems
About the Role:
We are seeking a highly technical QA Automation Engineer to anchor the quality, reliability, and precision of our Enterprise AI and core software systems. In this role, you will be responsible for designing, building, and maintaining robust test automation frameworks using Python with Playwright, Java with Selenium, and TOSCA. Alongside core automation, you will establish verification layers for enterprise AI systems and Retrieval-Augmented Generation (RAG) applications, ensuring web applications, API microservices, data pipelines, and AI components are resilient and production-ready.
Key Responsibilities:
- Automation Framework Development & UI Testing: Build, extend, and maintain scalable test automation frameworks using Python with Playwright and Java with Selenium. Implement Hybrid Page Object Model (POM) and utilize TOSCA for enterprise test automation.
- API & Backend Automation: Architect and execute automated test suites for RESTful microservices and backend systems using RestAssured and Postman.
- Data & ETL Testing: Perform rigorous data feed validation, verify reporting processes, and conduct market data verification. Leverage advanced SQL to ensure data pipeline integrity.
- CI/CD, Version Control & BDD: Integrate test suites into CI/CD pipelines (Jenkins, Azure DevOps). Manage version control via Git (GitHub, GitLab, Bitbucket). Author BDD test coverage using Cucumber and TestNG.
- Enterprise AI, Agentic & RAG System Evaluation: Design automated evaluation suites to validate response quality in AI applications (faithfulness, answer relevance). Validate Agentic Search Routers and data grounding across semantic layers.
Required Experience and Qualifications:
- Overall Experience: 7 - 10 years of professional experience in software QA automation and testing complex distributed systems.
- Mandatory Core Skills: Extensive hands-on experience with Python + Playwright and Java + Selenium. Proficiency with TOSCA is highly desirable.
- API & Backend Testing: Strong expertise in testing APIs and microservices using RestAssured and Postman.
- Frameworks & Methodologies: Deep knowledge of Hybrid Page Object Model (POM), TestNG, and Cucumber (BDD).
- Data & ETL Testing: Proficient in SQL, with a background in data feed validation, reporting processes, and market data verification.
- AI & LLM Testing Foundations: Hands-on experience testing AI/ML pipelines, LLM orchestrations (e.g., LangChain, LlamaIndex), and validating RAG architectures.
- CI/CD & Version Control: Experience integrating tests using Git and CI/CD tools (Jenkins or Azure DevOps).
- Certifications: ISTQB Certified Tester and ICAgile Certified Professional credentials.
Preferred Qualifications:
- Performance Testing: Experience with performance and load testing frameworks (e.g., Locust, JMeter) for measuring API response times.
- Cloud & Infrastructure: Familiarity with cloud platforms (AWS or Azure), Docker, and cloud API gateways.
Skills
Automation Testing, Python, Java, Selenium, TOSCA, RestAssured, Playwright Testing, Software Quality Assurance, ETL/Datawarehouse Testing, ISTQB, Application Testing