{"id":2002498,"url":"https://alion.io/job/svitla-systems-ai-qa-engineer-on-device-testing","title":"AI QA Engineer – On-Device Testing","company":{"id":2216,"name":"Svitla Systems","domain":"svitla.com","url":"https://alion.io/company/svitla-systems","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Career site","truth_index":{"grade":"A","score":86,"open_postings":7,"ghost_share":0,"stale_share":0.571,"repost_share":0,"time_to_fill_p50_days":49,"computed_at":"2026-10-12T06:55:09Z"}},"role":"QA","role_family":"QA","seniority":null,"employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":[],"countries":[],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Appium","optional":false},{"name":"Claude Code","optional":false},{"name":"Computer Vision","optional":false},{"name":"Cypress","optional":false},{"name":"Edge AI","optional":false},{"name":"JavaScript","optional":false},{"name":"Jira","optional":false},{"name":"JMeter","optional":false},{"name":"Machine Learning","optional":false},{"name":"Playwright","optional":false},{"name":"Postman","optional":false},{"name":"Python","optional":false},{"name":"Red Teaming","optional":false},{"name":"Robot Framework","optional":false},{"name":"Ruby","optional":false},{"name":"Selenium","optional":false},{"name":"WebDriverIO","optional":false}],"status":"live","first_seen_at":"2026-10-07T10:01:38Z","employer_posted_date":"2026-10-07","last_verified_at":"2026-10-12T05:29:43Z","board_verified":true,"closed_at":null,"days_open":5,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":5},"description":"Svitla Systems Inc. is looking for an AI QA Engineer - On-Device & Mobile Testing for a full-time position (40 hours per week) in London. Our client is an American mobile computing company. It's a central innovation team with engineers and researchers in London, Europe, the US, India, and Shanghai. They provide AI innovation and acceleration for the Business Units and are working on CV and GenAI solutions with an emphasis on device inference (Mostly Qualcomm chipsets). Researchers create/fine-tune models and hand them over to the engineering team to optimize, quantize, etc. The team then builds an application solution with customers to show value.\nThe position is based in London and requires visiting the office approximately three times per week.\nThe AI QA Engineer is a critical role responsible for ensuring the performance, safety, and reliability of our cutting-edge AI/ML models. You will be at the forefront of our development lifecycle, designing and executing comprehensive evaluation strategies to identify model weaknesses, potential biases, and critical edge cases.\nThis role requires a blend of analytical rigor, technical aptitude, and a deep curiosity about how AI models behave in real-world scenarios. You will not just find bugs but provide the actionable insights that drive model improvement and guide our research and development efforts.\nRequirements\nProven experience in quality assurance, preferably within the AI/ML domain.\nA deep understanding of the machine learning lifecycle and the common failure modes of AI models.\nA proactive approach to collecting and curating test data to cover edge cases, with experience in data validation and managing large datasets.\nHands-on experience with any modern UI automation framework (e.g., Playwright, Cypress, Robot Framework, Selenium, WebdriverIO).\nExperience with mobile test automation using Appium or a similar mobile automation tool.\nAPI testing experience (e.g., Postman).\nExperience with scripting and test automation (Python preferred, but other languages such as Java, JavaScript, or Ruby are also suitable). Ability to design, execute, and maintain automated test scenarios and validation workflows.\nHands-on experience testing on physical Android devices: installing APKs, updating software, and performing both manual and automated testing on real devices. Experience with emulators or cloud devices alone is not sufficient.\nStrong analytical and problem-solving skills, with meticulous attention to detail and the ability to identify patterns in complex data.\nExperience with bug tracking systems (e.g., Jira) and test case management tools.\nFamiliarity with using AI tools to create agents and skills and to generate code that works for testing on-device apps. Preferred tool: Claude Code.\nWill be a plus\nFamiliarity with computer vision or other specific AI domains relevant to our work.\nNative Android tooling (Android Studio, ADB).\nJMeter or other performance testing experience.\nResponsibilities\nEvaluation Strategy & Benchmark Development\nDesign, develop, and maintain a comprehensive suite of test cases and evaluation benchmarks. Proactively identify potential model failure points, including edge cases, adversarial inputs, and sources of bias.\nError Analysis & Failure Triage\nConduct systematic error analysis to categorize model failures and identify underlying patterns. Triage defects, prioritize them based on severity and impact, and work with the development team to ensure resolution.\nData Sourcing & Curation\nSource, curate, and manage high-quality datasets for model evaluation and testing. This includes performing data annotation and validation to ensure the integrity of our ground-truth data.\nExploratory & Adversarial Testing (Red Teaming)\nPerform unscripted, exploratory testing to discover unexpected model behaviors. Participate in red teaming exercises to intentionally challenge our models and identify potential safety and security vulnerabilities.\nFunctional Testing & Automation\nValidate AI-powered applications, APIs, and end-to-end user workflows to ensure functionality, usability, and reliability. Design, implement, and maintain automated tests for application-level validation using modern automation frameworks and AI-assisted tooling where appropriate.\nTest Environment Management\nSet up, maintain, and troubleshoot testing and demonstration environments to ensure a stable and reliable evaluation pipeline.\nReporting & Insights\nAnalyze and synthesize test results into clear, actionable reports for both technical and non-technical stakeholders. Translate complex findings into concrete recommendations for model improvement.\nProcess Improvement\nActively participate in post-hoc evaluation reviews and contribute to the continuous improvement of our testing methodologies, tools, and overall quality assurance processes.","description_format":"text","description_chars":4852,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["DevOps & Platform Engineering","IT Outsourcing & Dedicated Teams","Custom Software Development"],"lifecycle":[{"event":"open","at":"2026-10-07T12:04:13Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"Filed H-1B visas in the last two years","filings_12m":0,"filings_prev_12m":1,"green_card_filings_12m":0,"median_offered_wage_usd":null,"route":null,"cap_exempt":false,"checked_at":"2026-10-03T21:08:04+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)"],"filings_for_role_12m":0}],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":4,"expected_fill_days":49,"reasons":["conf:1","velocity","win:early","comp:brand"],"computed_at":"2026-10-12T06:55:09Z"},"pay":null,"html_url":"https://alion.io/job/svitla-systems-ai-qa-engineer-on-device-testing","json_url":"https://alion.io/job/svitla-systems-ai-qa-engineer-on-device-testing.json","meta":{"generated_at":"2026-10-12T12:01:55Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":18824,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-13T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":2216},"rest":"https://alion.io/mcp/rest/get_company?id=2216"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fsvitla-systems-ai-qa-engineer-on-device-testing"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fsvitla-systems-ai-qa-engineer-on-device-testing"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fsvitla-systems-ai-qa-engineer-on-device-testing"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/svitla-systems-ai-qa-engineer-on-device-testing\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fsvitla-systems-ai-qa-engineer-on-device-testing"}]}