{"id":2499125,"url":"https://alion.io/job/vi-senior-qa-engineer","title":"Senior QA Engineer","company":{"id":1143,"name":"Vi","domain":"vi.co","url":"https://alion.io/company/vi-co","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Comeet","truth_index":null},"role":"QA","role_family":"QA","seniority":"senior","employment_type":null,"work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"board_field","remote_working_hours":null,"hiring_geo_confidence":"structured","locations":[],"countries":[],"hiring_countries":["US"],"hiring_countries_total":1,"salary":null,"salary_estimate":{"min_usd":96000,"max_usd":187000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":87},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"LLM","optional":false},{"name":"LLM Guardrails","optional":false},{"name":"Python","optional":false},{"name":"Speech Recognition","optional":false},{"name":"Structured Outputs","optional":false},{"name":"Text-to-Speech","optional":false},{"name":"Voice Agents","optional":false},{"name":"HIPAA","optional":true},{"name":"Twilio","optional":true}],"status":"live","first_seen_at":"2026-10-07T13:38:00Z","employer_posted_date":"2026-10-07","last_verified_at":"2026-10-14T10:20:47Z","board_verified":true,"closed_at":null,"days_open":6,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":6},"description":"Description\nVi Operate puts AI agents into the daily operations of large health systems, providers, and pharma organizations. Augment is the voice capability underneath it: the agents that call patients and providers, hold a real conversation, and take action in the systems where the work actually lives. This is a dedicated quality role for that work.\nTesting a voice agent is not like testing software that returns the same answer twice. A similar call can go three different ways. A model update can quietly change behavior that nothing in a unit test would catch. When a call goes badly, the cause could be the speech recognition, the model, the voice synthesis, the telephony layer, or the integration underneath, and the transcript alone won't tell you which. Making that testable, repeatable, and catchable before a client hears it is the job.\nYou will own quality for agents that call real patients, providers, and caregivers. That means building the test and evaluation systems our engineers develop against, deciding what has to pass before something ships, and being the person who can take a bad call recording and say what actually went wrong. We need someone who has done classical QA properly and has since tested voice and LLM systems in production, because this role needs all three.\nWhat You'll Own\nVoice testing and debugging. Call recordings, transcripts, latency traces, and logs. Isolating which layer failed: speech to text, the model, text to speech, telephony, or the integration behind it. Turning a vague \"the call went badly\" into a reproducible ticket an engineer can act on.\nEvaluation and regression testing. Golden test sets, simulated calls, judged transcripts, accuracy on classification and extraction, and structured output validation. Regression coverage that still holds when the underlying model changes.\nTesting things that aren't deterministic. Deciding what \"passing\" means when the output varies run to run. Setting tolerances, handling flake honestly rather than by rerunning until green, and keeping the suite trustworthy enough that engineers act on a failure.\nRelease criteria. What has to be true before an agent goes live for a client, and what has to stay true afterward. You set the bar and you hold it, including when a deployment date is pushing on it.\nClassical QA for the rest of the product. Test plans, regression suites, bug triage, reproductions, and CI integration for everything around the agent.\nQuality in production. Sampling live calls after release, spotting drift and degradation, and getting it in front of the right engineer before the client raises it.\nBreaking it on purpose. Red-teaming the agent before someone outside does. Prompt injection through the documents it reads, callers who talk it off script or push it toward giving medical advice, people claiming to be someone they aren't to get patient information out of it. You test that the guardrails and access controls actually hold under pressure, not just that they exist.\nWhat We're Looking For\nQA on production voice agents. You have tested a system that talks to people on the phone, and you can walk through a real failure you diagnosed: what you heard, where you looked, and which layer turned out to be at fault. This is required and it's what we screen hardest on.\nQA on LLM systems. You have tested something built on a language model in production, not as a side project, and you can describe how you decided whether it was working.\nReal classical QA depth. Test strategy, test planning, regression suites, bug triage, and release process. You have worked somewhere QA had teeth.\nTesting non-deterministic systems. You have dealt with output that varies and made it testable anyway, and you can explain the tradeoffs you made to get there.\nPython and test automation. You write your own tooling and wire it into CI rather than filing a ticket for someone else to.\nJudgment about severity. You can tell the difference between a cosmetic defect and something that will mislead a patient on a phone call, and you can defend that call to an engineer who disagrees.\nA system mindset. You look at the whole system rather than one piece, and your first instinct is to find where it comes apart. You think about the seams between components, the inputs nobody planned for, and the person on the other end trying to get something out of it that they shouldn't.\nNice To Have\nBackground in IVR, contact center, or CCaaS quality (Genesys, NICE, Five9, Amazon Connect, Twilio, or similar)\nAudio and network condition testing: codecs, jitter, packet loss, barge-in and interruption handling\nFamiliarity with evaluation tooling such as LLM-as-judge harnesses, prompt regression frameworks, or simulated conversation testing\nHealthcare or pharma experience, and comfort working under HIPAA\nExperience standing up a QA function rather than joining one\nLoad and performance testing on real-time systems\nSpanish proficiency, enough to listen to a call and judge whether the agent handled it","description_format":"text","description_chars":5012,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[{"language":"Spanish","level":"Advanced (C1)","optional":true}]},"benefits":[],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Medical AI","Health Data & Interoperability"],"lifecycle":[{"event":"open","at":"2026-10-14T05:28:58Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":6,"expected_fill_days":29,"reasons":["conf:1","win:early"],"computed_at":"2026-10-14T06:34:16Z"},"pay":null,"html_url":"https://alion.io/job/vi-senior-qa-engineer","json_url":"https://alion.io/job/vi-senior-qa-engineer.json","meta":{"generated_at":"2026-10-14T12:50:35Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":15706,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-15T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":1143},"rest":"https://alion.io/mcp/rest/get_company?id=1143"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fvi-senior-qa-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fvi-senior-qa-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fvi-senior-qa-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/vi-senior-qa-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fvi-senior-qa-engineer"}]}