2,306,856open jobs
146,345companies
271,979added this week
Browse all
Location
Hybrid (Warsaw, Kraków, Gdańsk, Wrocław, Poland)
Employment
Contractor

First seen by Alion on Nov 16, 2026. Billennium scores A on the Alion truth index.

Overview
Company
Impact
Profile match
Billennium is an international IT services and software development company specializing in cloud solutions, enterprise application development, and digital transformation. Headquartered in Poland with global delivery centers, the company provides end-to-end technology services, managed IT, and staff augmentation to enterprise clients worldwide. By partnering with leading cloud platforms like AWS, Microsoft, and Salesforce, Billennium helps organizations optimize their digital infrastructure and streamline business operations.

Key Responsibilities:

  • Test Strategy: Define the test strategy and test pyramid with developers and platform engineers, and keep it up to date as the platform grows.

  • Test Repository Ownership: Set up and own the automation repository: structure, conventions, shared fixtures, code review and reporting. Keep it simple enough that developers can add tests themselves.

  • API Test Automation: Build functional, schema, negative and regression suites for the REST API. Cover long-running asynchronous flows: agent deployments, image builds, provisioning of dev environments and WebSocket log streaming.

  • UI Test Automation: Build a stable Playwright E2E suite for the critical user journeys: workspaces, agent deployment, marketplace, evaluation, monitoring and admin.

  • Agent Integration & Smoke Testing: Deploy the supported agent templates through the platform. Check that endpoints respond, auth is enforced, logs and traces arrive, MCP tool calls succeed and memory persists.

  • CI/CD & Test Environments: Run the suites in CI as merge-request gates, nightly regression runs and post-deploy smoke tests. Automate the setup and cleanup of short-lived test workspaces and test data in AWS.

  • Collaboration and Troubleshooting: Work closely with developers, AI engineers, and platform teams to identify automation opportunities and improve overall software quality. Conduct root cause analysis for identified issues and collaborate with development teams for resolution.

Qualifications:

  • 5+ years in software quality engineering. You have defined a test strategy for a web platform or SaaS product. You have also set up and owned a test automation repository (not only contributed tests to one), including its structure, conventions, CI integration, code reviews and long-term maintenance, in Python or TypeScript.

  • Strong hands-on experience automating REST API tests (e.g., pytest + httpx, Playwright API testing), including: schema validation against OpenAPI (contract testing is a plus), asynchronous flows (polling, state machines, queues, WebSockets), auth and access control (OAuth2/OIDC, JWT, role- and tenant-based access, negative cases).

  • Proven experience with Playwright on React/SPA applications. You have kept a large suite stable and kept flaky tests measured and under control.

  • Experience building quality gates in GitLab CI (or a similar tool) with Docker. Experience testing cloud-hosted applications and debugging through logs, metrics and traces.

  • You understand how LLM-based agents behave (tools, memory, non-determinism). You use AI assistants (e.g., Claude Code, GitHub Copilot) every day in your own work: writing tests, analysing failures and documenting. You review AI-generated code critically.

  • Strong collaboration and reporting skills; ability to work in cross-functional and international teams. Strong analytical and problem-solving skills with attention to detail.

Nice to have:

  • AWS experience (Lambda, API Gateway, DynamoDB, S3, CloudWatch, Bedrock AgentCore).

  • Experience with Strands or other agent frameworks (e.g., LangGraph, CrewAI).

  • Familiarity with LLM evaluation tools (e.g., DeepEval, Promptfoo, Ragas).

  • Experience with MCP / A2A servers and protocols.

  • Knowledge of vector-based retrieval systems.

  • Experience in monitoring & observability.

  • Experience with full-stack application testing.

  • Security testing: working knowledge of the OWASP Top 10 (web and API) and hands-on penetration testing experience (e.g., Burp Suite, OWASP ZAP)

  • Knowledge of GxP or regulated environments is a plus.

  • Strong analytical mindset and attention to detail.

What we offer:

  • Comprehensive benefits - enjoy Udemy for Business, private medical care, Multisport card, veterinary package, language lessons, and shopping vouchers.

  • Career growth - access opportunities for professional development and learning, including perks related to our official partnerships with global IT giants: Microsoft, AWS, Snowflake, Salesforce & more.

  • Global collaboration - work with a diverse, international team.

  • Innovative environment - be part of a forward-thinking and growth-oriented workplace.

  • Engaging community – Work with passionate professionals and participate in team-building events, hackathons, and CSR initiatives to make an impact beyond work.

  • Team-building events including our company tradition (annual company event in Mazury).

  • A pleasant surprise to start your journey with us in the form of a welcome pack.

Tech stack

  • Python
  • TypeScript
  • Playwright
  • REST API
  • Gitlab CI
  • Docker
  • OpenAPI
Free account
Stop reading job ads. Get the ones that fit.
One free account turns this page into a shortlist built around your stack, your level and your pay.
Match on every job. Stack, seniority, pay and location, scored against your profile.
2,306,856 open roles. Read straight off company career pages, refreshed every day.
Unlimited applications. Every one you send is tracked in one place, on-site or on a company board.
3 tailored CVs a month. Rewritten for the exact job you are applying to. Included free.
Create a free account Continue with Google
Free forever. No card. Under a minute.

Your match

How well do you fit this role?
Two answers are enough for a real match. No account needed.
Check my fit
Answers stay in this browser until you create an account.

Recommended for you based on this role

QA
Similar stack
Same company
Warsaw
Lead QA Engineer 4 hours ago
Remote (Poland) · Full-Time · Poland
Python
JavaScript
TypeScript
DevOps
CI/CD
Docker
Cybersecurity
Shift-Left Security
Management
Jira
QA
Playwright
Apply
Senior QA Engineer 4 hours ago
Remote (Poland) · Full-Time · Poland
Python
JavaScript
TypeScript
SQL
AI/ML
Copilot
Cursor
Frontend
GraphQL
DevOps
Rest API
GitHub Actions
GitLab CI
CI/CD
Jenkins
Docker
Bitbucket
Management
Jira
Agile
QA
Playwright
Apply
Senior QA Engineer 6 hours ago
Hybrid · Contractor · Warsaw · Gdańsk · Gdynia
Java
DevOps
CI/CD
Management
Confluence
Jira
Agile
QA
JMeter
Playwright
Swagger
Postman
Apply
Manual Tester 5 hours ago
Remote (Poland) · Full-Time · Gliwice
SQL
DevOps
Rest API
CI/CD
Management
Confluence
Jira
Agile
Scrum
QA
Swagger
Postman
Insomnia
Apply
Hybrid · Contractor · Warsaw
Management
Confluence
Apply
Manual QA Tester 6 hours ago
Hybrid · Warsaw
SQL
DevOps
Azure DevOps
Kibana
Azure
Management
Confluence
Jira
Apply
Remote (Poland) · Contractor · Warsaw
Python
JavaScript
DevOps
GitLab CI
CI/CD
Jenkins
Cybersecurity
Shift-Left Security
Management
Agile
QA
Selenium
Cypress
Playwright
Rest-Assured
Apply
≈ $64k – $133k per year (Estimated) · In office · Full-Time · 7+ years exp · Darmstadt · Warsaw
Life Sciences
ELISA
Apply
In office · Contractor · Warsaw
Analytics
Microsoft Excel
Apply
In office · Warsaw
Analytics
Power BI
Apply
See all jobs
This is one of many
2,306,856 more open roles from verified company boards, updated every day.