{"id":4375066,"url":"https://alion.io/job/billennium-test-automation-engineer-3","title":"Test Automation Engineer","company":{"id":1572,"name":"Billennium","domain":"billennium.com","url":"https://alion.io/company/billennium","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"services","is_intermediary":false,"listed_via":null,"ats_vendor":"Traffit","truth_index":{"grade":"A","score":92,"open_postings":6,"ghost_share":0,"stale_share":0.333,"repost_share":0,"time_to_fill_p50_days":59,"computed_at":"2026-11-16T05:46:25Z"}},"role":"QA","role_family":"QA","seniority":null,"employment_type":"contractor","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Warsaw, Poland","Kraków, Poland","Gdańsk, Poland","Wrocław, Poland"],"countries":["PL"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AWS","optional":false},{"name":"CI/CD","optional":false},{"name":"HTTPX","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"Playwright","optional":false},{"name":"Pytest","optional":false},{"name":"Python","optional":false},{"name":"Rest API","optional":false},{"name":"TypeScript","optional":false},{"name":"WebSockets","optional":false},{"name":"A2A","optional":true},{"name":"Amazon CloudWatch","optional":true},{"name":"Amazon S3","optional":true},{"name":"API Gateway","optional":true},{"name":"AWS Bedrock","optional":true},{"name":"AWS Bedrock AgentCore","optional":true},{"name":"AWS Lambda","optional":true},{"name":"AWS Strands Agents","optional":true},{"name":"Burp Suite","optional":true},{"name":"Claude Code","optional":true},{"name":"Copilot","optional":true},{"name":"CrewAI","optional":true},{"name":"DeepEval","optional":true},{"name":"Docker","optional":true},{"name":"DynamoDB","optional":true},{"name":"GitLab CI","optional":true},{"name":"LangChain","optional":true},{"name":"LangGraph","optional":true},{"name":"LLM","optional":true},{"name":"OWASP Top 10","optional":true},{"name":"OWASP ZAP","optional":true},{"name":"Promptfoo","optional":true},{"name":"Ragas","optional":true},{"name":"Snowflake","optional":true}],"status":"live","first_seen_at":"2026-11-16T13:28:46Z","employer_posted_date":null,"last_verified_at":"2026-11-16T13:28:46Z","board_verified":false,"closed_at":null,"days_open":0,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":0},"description":"Key Responsibilities:\nTest Strategy: Define the test strategy and test pyramid with developers and platform engineers, and keep it up to date as the platform grows.\n\nTest Repository Ownership: Set up and own the automation repository: structure, conventions, shared fixtures, code review and reporting. Keep it simple enough that developers can add tests themselves.\n\nAPI Test Automation: Build functional, schema, negative and regression suites for the REST API. Cover long-running asynchronous flows: agent deployments, image builds, provisioning of dev environments and WebSocket log streaming.\n\nUI Test Automation: Build a stable Playwright E2E suite for the critical user journeys: workspaces, agent deployment, marketplace, evaluation, monitoring and admin.\n\nAgent Integration & Smoke Testing: Deploy the supported agent templates through the platform. Check that endpoints respond, auth is enforced, logs and traces arrive, MCP tool calls succeed and memory persists.\n\nCI/CD & Test Environments: Run the suites in CI as merge-request gates, nightly regression runs and post-deploy smoke tests. Automate the setup and cleanup of short-lived test workspaces and test data in AWS.\n\nCollaboration and Troubleshooting: Work closely with developers, AI engineers, and platform teams to identify automation opportunities and improve overall software quality. Conduct root cause analysis for identified issues and collaborate with development teams for resolution.\n\nQualifications:\n5+ years in software quality engineering. You have defined a test strategy for a web platform or SaaS product. You have also set up and owned a test automation repository (not only contributed tests to one), including its structure, conventions, CI integration, code reviews and long-term maintenance, in Python or TypeScript.\n\n Strong hands-on experience automating REST API tests (e.g., pytest + httpx, Playwright API testing), including: schema validation against OpenAPI (contract testing is a plus), asynchronous flows (polling, state machines, queues, WebSockets), auth and access control (OAuth2/OIDC, JWT, role- and tenant-based access, negative cases).\n\nProven experience with Playwright on React/SPA applications. You have kept a large suite stable and kept flaky tests measured and under control.\n\nExperience building quality gates in GitLab CI (or a similar tool) with Docker. Experience testing cloud-hosted applications and debugging through logs, metrics and traces.\n\nYou understand how LLM-based agents behave (tools, memory, non-determinism). You use AI assistants (e.g., Claude Code, GitHub Copilot) every day in your own work: writing tests, analysing failures and documenting. You review AI-generated code critically.\n\nStrong collaboration and reporting skills; ability to work in cross-functional and international teams. Strong analytical and problem-solving skills with attention to detail.\n\nNice to have: \nAWS experience (Lambda, API Gateway, DynamoDB, S3, CloudWatch, Bedrock AgentCore).\n\nExperience with Strands or other agent frameworks (e.g., LangGraph, CrewAI).\n\nFamiliarity with LLM evaluation tools (e.g., DeepEval, Promptfoo, Ragas).\n\nExperience with MCP / A2A servers and protocols.\n\nKnowledge of vector-based retrieval systems.\n\nExperience in monitoring & observability.\n\nExperience with full-stack application testing.\n\nSecurity testing: working knowledge of the OWASP Top 10 (web and API) and hands-on penetration testing experience (e.g., Burp Suite, OWASP ZAP)\n\nKnowledge of GxP or regulated environments is a plus.\n\nStrong analytical mindset and attention to detail.\n\nWhat we offer:\nComprehensive benefits - enjoy Udemy for Business, private medical care, Multisport card, veterinary package, language lessons, and shopping vouchers.\n\nCareer growth - access opportunities for professional development and learning, including perks related to our official partnerships with global IT giants: Microsoft, AWS, Snowflake, Salesforce & more.\n\nGlobal collaboration - work with a diverse, international team.\n\nInnovative environment - be part of a forward-thinking and growth-oriented workplace.\n\nEngaging community – Work with passionate professionals and participate in team-building events, hackathons, and CSR initiatives to make an impact beyond work.\n\nTeam-building events including our company tradition (annual company event in Mazury).\n\nA pleasant surprise to start your journey with us in the form of a welcome pack.\n\nTech stack\nPython\nTypeScript\nPlaywright\nREST API\nGitlab CI\nDocker\nOpenAPI","description_format":"text","description_chars":4514,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Professional development"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["IT Outsourcing & Dedicated Teams","Enterprise Apps Implementation Partners","Cloud Consulting & Migration","IT Staffing & Staff Augmentation"],"lifecycle":[{"event":"open","at":"2026-11-16T14:25:35Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":59,"reasons":["seen:0","win:early"],"computed_at":"2026-11-16T20:50:17Z"},"pay":null,"html_url":"https://alion.io/job/billennium-test-automation-engineer-3","json_url":"https://alion.io/job/billennium-test-automation-engineer-3.json","meta":{"generated_at":"2026-11-16T20:50:17Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":6720,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1572},"rest":"https://alion.io/mcp/rest/get_company?id=1572"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fbillennium-test-automation-engineer-3"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fbillennium-test-automation-engineer-3"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fbillennium-test-automation-engineer-3"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/billennium-test-automation-engineer-3\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fbillennium-test-automation-engineer-3"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":4375066}},{"tool":"search_jobs","arguments":{"company":"billennium.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":4375066},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}