{"id":82627,"url":"https://alion.io/job/tekion-senior-software-development-test-engineer-ai-2","title":"Senior Software Development Test Engineer - AI","company":{"id":804,"name":"Tekion","domain":"tekion.com","url":"https://alion.io/company/tekion","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"B","score":81,"open_postings":12,"ghost_share":0,"stale_share":0.75,"repost_share":0,"time_to_fill_p50_days":48,"computed_at":"2026-10-17T05:46:00Z"}},"role":"QA","role_family":"QA","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":14000,"max_usd":34000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":28},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"AI Agents","optional":false},{"name":"CI/CD","optional":false},{"name":"Embeddings","optional":false},{"name":"Hallucination","optional":false},{"name":"Java","optional":false},{"name":"LLM","optional":false},{"name":"LLM Guardrails","optional":false},{"name":"Machine Learning","optional":false},{"name":"Python","optional":false},{"name":"RAG","optional":false},{"name":"Synthetic Data","optional":false},{"name":"AWS","optional":true},{"name":"Azure","optional":true},{"name":"DeepEval","optional":true},{"name":"Docker","optional":true},{"name":"Function Calling","optional":true},{"name":"GCP","optional":true},{"name":"Kubernetes","optional":true},{"name":"LangSmith","optional":true},{"name":"MLFlow","optional":true},{"name":"Ragas","optional":true},{"name":"Tool Use","optional":true},{"name":"Weights & Biases","optional":true}],"status":"live","first_seen_at":"2026-07-09T12:37:31Z","employer_posted_date":"2026-07-09","last_verified_at":"2026-10-17T07:46:50Z","board_verified":true,"closed_at":null,"days_open":99,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":99},"description":"About Tekion:\nPositively disrupting an industry that has not seen any innovation in over 50 years, Tekion has challenged the paradigm with the first and fastest cloud-native automotive platform that includes the revolutionary Automotive Retail Cloud (ARC) for retailers, Automotive Enterprise Cloud (AEC) for manufacturers and other large automotive enterprises and Automotive Partner Cloud (APC) for technology and industry partners. Tekion connects the entire spectrum of the automotive retail ecosystem through one seamless platform. The transformative platform uses cutting-edge technology, big data, machine learning, and AI to seamlessly bring together OEMs, retailers/dealers and consumers. With its highly configurable integration and greater customer engagement capabilities, Tekion is enabling the best automotive retail experiences ever. Tekion employs close to 3,000 people across North America, Asia and Europe.\nAbout the Role\nWe are looking for highly motivated Senior AI Software Development Engineers in Test (AI\u0002SDET) to join Tekion’s ML Engineering team. Our team builds and ships the machine learning \nmodels and AI agents that power the Automotive Retail Cloud (ARC) - from Service AI Agents and F&I automation to the AI-powered Analytics Agent - trained on real-time data spanning 1,700+ dealership rooftops.\nIn this role, you will be responsible for building confidence in the quality, correctness, and reliability of the models and AI agents we deliver. You will work closely with ML Engineers, Data\nScientists, the AI Platform team, Product Management, and Applied AI teams to define robust validation strategies, automate testing across the model lifecycle, and ensure our AI features meet the highest quality bar before they reach dealers and their customers.Because you are testing AI systems, you will go beyond deterministic assertions: you will design\nevaluation frameworks for non-deterministic model and agent outputs, measure accuracy and consistency, catch hallucinations and regressions, and continuously expand coverage. You will also use AI and LLMs yourself - to generate test cases, synthetic data, and automation scaffolding - raising both the speed and the depth of quality engineering.\nWhat You’ll Do\nDevelop a deep understanding of Tekion’s AI agents, ML models, business domains, and the data that feeds them.\n\nOwn end-to-end quality for the ML models, AI agents, and AI-powered features shipped by the ML Engineering team.\n\nDesign and implement automated testing frameworks for model inference, prompt pipelines, agentic workflows, RAG/retrieval systems, and the APIs that serve them.\n\nValidate model correctness, accuracy, latency, and behavioral consistency across model versions, prompt changes, and upgrades.\n\nDefine and execute evaluation strategies for AI/LLM-powered capabilities by:\n\nCreating and curating evaluation datasets (evals) and ground-truth sets\n\nMeasuring response accuracy, relevance, and consistency\n\nIdentifying hallucinations, unsafe outputs, and edge cases\n\nBuilding LLM-as-judge and automated scoring pipelines\n\nContinuously improving evaluation coverage as models and agents evolve\n\nUse AI/LLMs to accelerate quality engineering - generating test cases, synthetic test data,and automation scaffolding.\n\nBuild regression suites that detect quality drift when models, prompts, embeddings, or training data change.\n\nValidate integration of ML models and agents into ARC products, including end-to-end flows and fallback behavior.\n\nDrive root cause analysis for production model and behavioral issues, and partner with ML Engineers on preventive improvements.\n\nDefine quality metrics, guardrails, and automated monitoring to detect model degradation before it impacts customers.\n\nChampion quality and evaluation engineering best practices across the ML Engineering organization.\n\nWhat We’re Looking For\n5-8 years in SDET, quality engineering, or software engineering, with a strong track record of building test automation frameworks.\n\nStrong programming skills in Python (preferred for ML tooling) and/or Java, with the ability to write production-quality automation code.\n\nHands-on experience testing data-intensive or ML/AI systems - or a strong software\u0002testing background with demonstrated ML/LLM fluency.\n\nSolid understanding of ML concepts: model training and inference, evaluation metrics (precision/recall, F1, etc.), and the non-deterministic nature of model outputs.\n\nExperience designing evaluation frameworks, or working with eval datasets, benchmarks, or LLM-as-judge approaches.\n\nFamiliarity with LLM/agent concepts - prompting, RAG, embeddings, vector search - and generative failure modes (hallucination, drift, prompt sensitivity).\n\nExperience with CI/CD, test orchestration, and API/integration testing.\n\nStrong analytical and debugging skills; comfortable performing root cause analysis across model, data, and code layers.\n\nExcellent collaboration and communication skills to partner across ML, data, and product teams.\n\nNice to Have\n\nExperience with ML/eval tooling such as MLflow, Weights & Biases, LangSmith, Ragas, DeepEval, or provider eval suites.\n\nExperience with cloud platforms (AWS/GCP/Azure) and containerized environments (Docker, Kubernetes).\n\nExposure to model observability, drift detection, and production ML monitoring.\n\nExperience testing agentic systems, tool use, or multi-step LLM workflows.\n\nPrior experience in a high-scale SaaS or data platform environment.\n\nCurrent Tekion Employees: Please apply via the Internal Job Board in Ashby\nNote: Tekion recently transitioned to a new recruiting tool and we appreciate your patience and feedback as we adjust to our new system!\nTekion is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, victim of violence or having a family member who is a victim of violence, the intersectionality of two or more protected categories, or other applicable legally protected characteristics.\nFor more information on our privacy practices, please refer to our Applicant Privacy Notice here.","description_format":"text","description_chars":6314,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Artificial Intelligence","Automotive Software"],"lifecycle":[{"event":"open","at":"2026-07-09T12:37:31Z"}],"visa":[],"liveness":{"score":12,"band":"cold","label":"Long shot","p_open":1,"p_active":0.426,"p_room":0.28,"age_days":99,"expected_fill_days":48,"reasons":["conf:1","stale_co","velocity","win:tail","crowd:"],"computed_at":"2026-10-17T05:46:00Z"},"pay":null,"html_url":"https://alion.io/job/tekion-senior-software-development-test-engineer-ai-2","json_url":"https://alion.io/job/tekion-senior-software-development-test-engineer-ai-2.json","meta":{"generated_at":"2026-10-17T10:09:39Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":13292,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-18T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":804},"rest":"https://alion.io/mcp/rest/get_company?id=804"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ftekion-senior-software-development-test-engineer-ai-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ftekion-senior-software-development-test-engineer-ai-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ftekion-senior-software-development-test-engineer-ai-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/tekion-senior-software-development-test-engineer-ai-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ftekion-senior-software-development-test-engineer-ai-2"}]}