{"id":82615,"url":"https://alion.io/job/tekion-staff-software-development-test-engineer-3","title":"Staff Software Development Test Engineer","company":{"id":804,"name":"Tekion","domain":"tekion.com","url":"https://alion.io/company/tekion","size_band":"1001-5000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"B","score":81,"open_postings":12,"ghost_share":0,"stale_share":0.75,"repost_share":0,"time_to_fill_p50_days":48,"computed_at":"2026-10-16T05:45:00Z"}},"role":"QA","role_family":"QA","seniority":"staff","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Bengaluru, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":24000,"max_usd":55000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":191},"experience_years_min":8,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Anthropic","optional":false},{"name":"CI/CD","optional":false},{"name":"DeepEval","optional":false},{"name":"Docker","optional":false},{"name":"Feature Store","optional":false},{"name":"GitHub Actions","optional":false},{"name":"Grafana","optional":false},{"name":"Hallucination","optional":false},{"name":"Hugging Face","optional":false},{"name":"Jenkins","optional":false},{"name":"Kibana","optional":false},{"name":"LangChain","optional":false},{"name":"LangSmith","optional":false},{"name":"LlamaIndex","optional":false},{"name":"LLM","optional":false},{"name":"Machine Learning","optional":false},{"name":"MLFlow","optional":false},{"name":"Multi-Agent Systems","optional":false},{"name":"NumPy","optional":false},{"name":"OpenAI","optional":false},{"name":"OpenTelemetry","optional":false},{"name":"Pandas","optional":false},{"name":"Postman","optional":false},{"name":"Pytest","optional":false},{"name":"Python","optional":false},{"name":"RAG","optional":false},{"name":"Rest-Assured","optional":false},{"name":"SQL","optional":false},{"name":"Tool Use","optional":false},{"name":"TruLens","optional":false},{"name":"AWS","optional":true},{"name":"AWS Bedrock","optional":true},{"name":"Azure","optional":true},{"name":"Cypress","optional":true},{"name":"Feast","optional":true},{"name":"GCP","optional":true},{"name":"Hypothesis","optional":true},{"name":"JMeter","optional":true},{"name":"Kubeflow","optional":true},{"name":"Kubernetes","optional":true},{"name":"Locust","optional":true},{"name":"Playwright","optional":true},{"name":"PyTorch","optional":true},{"name":"Scikit-learn","optional":true},{"name":"Synthetic Data","optional":true},{"name":"TensorFlow","optional":true},{"name":"Terraform","optional":true},{"name":"Vertex AI","optional":true},{"name":"Weights & Biases","optional":true}],"status":"live","first_seen_at":"2026-07-17T11:37:45Z","employer_posted_date":"2026-07-17","last_verified_at":"2026-10-16T20:14:47Z","board_verified":true,"closed_at":null,"days_open":91,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":91},"description":"About Tekion:\nPositively disrupting an industry that has not seen any innovation in over 50 years, Tekion has challenged the paradigm with the first and fastest cloud-native automotive platform that includes the revolutionary Automotive Retail Cloud (ARC) for retailers, Automotive Enterprise Cloud (AEC) for manufacturers and other large automotive enterprises and Automotive Partner Cloud (APC) for technology and industry partners. Tekion connects the entire spectrum of the automotive retail ecosystem through one seamless platform. The transformative platform uses cutting-edge technology, big data, machine learning, and AI to seamlessly bring together OEMs, retailers/dealers and consumers. With its highly configurable integration and greater customer engagement capabilities, Tekion is enabling the best automotive retail experiences ever. Tekion employs close to 3,000 people across North America, Asia and Europe.\nRoles & Responsibilities\nGenerative AI & LLM Evaluation\nBuild automated testing suites to detect hallucinations, bias, toxicity, and prompt injection vulnerabilities across LLM-powered products\n\nImplement automated evaluations for RAG systems measuring context relevance, groundedness, and answer faithfulness using frameworks like RAGAS or DeepEval\n\nDesign test beds to validate multi-agent workflows - tool-calling accuracy, multi-step reasoning, memory, and autonomous decision loops\n\nBuild and run automated conversation simulations - scripted and synthetic user journeys - to stress-test agent behaviour across intents, edge cases, and multi-turn dialog flows\n\nCreate prompt regression frameworks to assess how changes in system prompts, temperature, and sampling parameters impact output consistency\n\nData Quality Assurance\nStatistically validate AI data outputs - distributions, precision/recall, error pattern analysis - to catch silent data quality failures before production\n\nProgrammatically audit data ingestion, transformation, and feature store pipelines for schema drift and data corruption\n\nValidate vector DB indexing, embedding semantic similarity accuracy, and retrieval latency\n\nVerify quality, diversity, and privacy compliance of synthetic datasets used for model training and evaluation\n\nClassical ML & Deep Learning Validation\nMaintain automated suites tracking ML metrics - Precision, Recall, F1, ROC-AUC - and deep learning loss curves across model versions\n\nImplement continuous monitoring scripts to detect data and concept drift on live inference endpoints\n\nAutomation Engineering & CI/CD\nBuild and maintain scalable test automation frameworks for APIs, backend services, and model endpoints\n\nEmbed AI evaluation and data QA suites into MLOps and CI/CD pipelines so quality failures block releases automatically\n\nDefine and track AI quality KPIs and communicate release readiness to engineering and product teams\n\nExperience of 8+ years SDET role\nTechnical Skills & Frameworks\nCore Programming\nPython - expert level; test automation, eval pipelines, data analysis (Pandas, NumPy, Pytest)\n\nSQL - data output validation, ground truth querying, pipeline data quality checks\n\nGenAI & Evaluation\nRAGAS / TruLens / DeepEval / Promptflow etc - LLM evaluation frameworks for measuring faithfulness, hallucination rate, and task success\n\nLangChain / LangSmith / LlamaIndex - agent workflow testing, prompt tracing, and LLM response debugging\n\nOpenAI / Anthropic / Hugging Face APIs - direct LLM endpoint testing and output consistency validation\n\nVector DBs - retrieval quality testing, embedding validation, and latency benchmarking\n\nPandas / NumPy etc. - statistical analysis for output validation and error pattern investigation, data profiling, schema validation, and pipeline integrity checks\n\nAPI & Automation\nPytest - modular, reusable test framework for AI eval and automation suites\n\nPostman / REST Assured / Requests - API contract validation and service-level integration testing\n\nMLOps & CI/CD\nMLflow - tracking model versions and eval runs to detect regressions across updates\n\nDocker / GitHub Actions / Jenkins - containerised test environments and deployment pipeline automation\n\nObservability\nGrafana / Kibana / OpenTelemetry - monitoring AI system health, output drift, and distributed tracing across agent pipelines\n\nGood to Have\nCloud AI Services - AWS Bedrock, Azure OpenAI, or GCP Vertex AI for testing managed model endpoints and cloud-deployed agents\n\nMLOps Platforms - MLflow, Kubeflow, Weights & Biases, or Feast feature stores for experiment tracking and model governance\n\nML Frameworks - Scikit-learn, TensorFlow, or PyTorch familiarity for understanding model internals and validating training pipelines\n\nInfrastructure as Code - Docker, Kubernetes, Terraform for managing containerised test environments at scale\n\nUI Automation - Playwright or Cypress for end-to-end conversational AI application testing\n\nPerformance Engineering - Locust or JMeter for load testing heavy AI inference endpoints under peak traffic\n\nStatistical Hypothesis Testing - t-tests, confidence intervals, significance testing to distinguish real quality signal from noise\n\nSynthetic Data Generation - using LLMs to generate diverse test cases and evaluation datasets at scale\n\nCurrent Tekion Employees: Please apply via the Internal Job Board in Ashby\nNote: Tekion recently transitioned to a new recruiting tool and we appreciate your patience and feedback as we adjust to our new system!\nTekion is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, victim of violence or having a family member who is a victim of violence, the intersectionality of two or more protected categories, or other applicable legally protected characteristics.\nFor more information on our privacy practices, please refer to our Applicant Privacy Notice here.","description_format":"text","description_chars":6068,"description_truncated":false,"requirements":{"experience_years_min":8,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Automotive Software"],"lifecycle":[{"event":"open","at":"2026-07-17T11:37:45Z"}],"visa":[],"liveness":{"score":17,"band":"cold","label":"Long shot","p_open":1,"p_active":0.479,"p_room":0.36,"age_days":90,"expected_fill_days":48,"reasons":["conf:1","stale_co","velocity","win:tail","crowd:"],"computed_at":"2026-10-16T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/tekion-staff-software-development-test-engineer-3","json_url":"https://alion.io/job/tekion-staff-software-development-test-engineer-3.json","meta":{"generated_at":"2026-10-16T21:14:21Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":2029,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":804},"rest":"https://alion.io/mcp/rest/get_company?id=804"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Ftekion-staff-software-development-test-engineer-3"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Ftekion-staff-software-development-test-engineer-3"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Ftekion-staff-software-development-test-engineer-3"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/tekion-staff-software-development-test-engineer-3\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Ftekion-staff-software-development-test-engineer-3"}]}