{"id":2985612,"url":"https://alion.io/job/aurora-senior-software-engineer-ml-evaluation-introspection-infrastructure","title":"Senior Software Engineer, ML Evaluation & Introspection Infrastructure","company":{"id":40481,"name":"Aurora Innovation","domain":"aurora.tech","url":"https://alion.io/company/aurora-tech","size_band":"501-1000","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Mountain View, United States","Seattle, United States","Pittsburgh, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":{"min":162000,"max":260000,"currency":"USD","period":"year","gross":null,"usd_annual":260000},"salary_estimate":null,"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":true,"technologies":[{"name":"AI Agents","optional":false},{"name":"Amazon Aurora","optional":false},{"name":"AWS","optional":false},{"name":"Embeddings","optional":false},{"name":"Kubernetes","optional":false},{"name":"Python","optional":false},{"name":"SQL","optional":false},{"name":"Agentic Workflows","optional":true},{"name":"Interpretability","optional":true},{"name":"LLM Evaluation","optional":true}],"status":"live","first_seen_at":"2026-10-19T23:58:54Z","employer_posted_date":"2026-10-19","last_verified_at":"2026-10-25T06:20:37Z","board_verified":true,"closed_at":null,"days_open":5,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":5},"description":"Who we are\nAurora’s mission is to deliver the benefits of self-driving technology safely, quickly, and broadly.\nThe Aurora Driver will create a new era in mobility and logistics, one that will bring a safer, more efficient, and more accessible future to everyone.\nAt Aurora, you will tackle massively complex problems alongside other passionate, intelligent individuals, growing as an expert while expanding your knowledge. For the latest news from Aurora, visit aurora.tech or follow us on LinkedIn.\nWhat we are looking for\nWe’re searching for a Senior Software Engineer, ML Evaluation & Introspection Infrastructure to join our Behavior Analysis team, reporting to the team’s engineering manager. Aurora’s trucks already haul freight on Texas highways with no one on board, and every new version of the learned model that drives them has to prove, on real driving data, that it is better and safer than the one before. Behavior Analysis builds the systems that make that call: the evals that compare model versions, the embedding search that makes every driving scenario findable, and the introspection tools that show why a model did what it did. Engineers build more than 100 training and eval datasets a week through our tools. We’re a small team, so each engineer owns whole systems, and we work agent-first: our tools are built for AI agents to drive as easily as engineers do. You’ll own core pieces of that loop and help lead what comes next: putting LLMs, VLMs and agents to work on analysis, so questions that take days to answer today take minutes.\nIn this role you will\nBuild and scale evaluation pipelines that compare model versions on large driving datasets: offline evals, slice-based metrics, regression detection, and model-vs-model reports.\n\nOwn embedding and vector-search pipelines over driving data, versioned per model, so engineers can search, cluster and compare how any model represents the world. This is where you’d start: owning embedding generation and search end to end.\n\nBuild model introspection infrastructure (attention, attribution, ablations) that turns “why did the model do that?” from a research project into a query.\n\nHelp define how frontier LLMs, VLMs and agents change ML analysis here: triaging, labeling, clustering and summarizing driving scenarios and eval results. We already have a prototype using LLMs for data curation, and you’d help decide where it goes next.\n\nWork side by side with the ML engineers training these models to decide what to measure, then make those measurements trustworthy, observable and cheap enough to run on every candidate model.\n\nBuild the web app and CLI that engineers and AI agents use to find datasets, training runs, evals and their costs, designed with agents as first-class users.\n\nRequired qualifications\nTypically 5+ years of professional software engineering experience building backend, data or ML infrastructure.\n\nStrong Python, and proficiency with SQL.\n\nDesigned and operated large-scale batch or data pipelines on cloud infrastructure, including backfills, retries and cost management.\n\nBuilt systems that measure or evaluate something: metrics, experimentation, evaluation or analytics platforms.\n\nExperience with distributed compute and container orchestration (e.g. Kubernetes, AWS Batch or similar).\n\nDesirable qualifications\nML evaluation infrastructure: eval harnesses, golden datasets, regression detection, A/B model comparison.\n\nEmbeddings and vector search, or model introspection and interpretability tooling.\n\nUsing LLMs, VLMs or agentic workflows to automate analysis, labeling or data curation.\n\nAutonomous vehicles, robotics or other large-scale sensor data.\n\nObservability for data systems (metrics, alerting, SLOs), and everyday use of AI coding agents.\n\nComing from backend or data engineering and want to learn ML evaluation? Apply anyway: we’ll teach you the ML side.\n\nFor roles based in Mountain View, CA: The salary range for this position is $162,000 - $260,000 per year. \nAurora’s pay ranges are determined by role, level, and location. Within the range, the successful candidate’s starting base pay will be determined based on factors including job-related skills, experience, qualifications, relevant education or training, and market conditions. These ranges may be modified in the future. The successful candidate will also be eligible for an annual bonus, equity compensation, and benefits.\nWorking at Aurora\nAt Aurora, we bring together extraordinarily talented and experienced people united by the strength of our values. We operate with integrity, set outrageous goals, and build a culture where we win together - all without any jerks.\nWe believe in-person work increases collaboration, empathy, and our ability to lead effectively. As a result, Aurorans are expected to work from the office at least 3 days per week, with some roles requiring more frequent in-office presence, up to 5 days per week, based on operational needs.\nOurCareers page provides insight into what it is like to work at Aurora, and you can find all the latest updates in our Newsroom.\nOur commitment to safety\nAt the core of everything we do is our commitment to safety. Building best-in-class self-driving technology will take time, and we believe that each employee at Aurora has a role in contributing to safety, every step of the way. Aurora expects commitment to our safety policies from every employee, and seeks candidates who take an active responsibility, can contribute to building an atmosphere of trust, and invest in the organization’s long-term success by prioritizing working safely, no matter what.\nOur commitment to inclusion\nAurora considers candidates without regard to their race, color, religion, national origin, age, sex, gender, gender identity, gender expression, sexual orientation, marital status, pregnancy status, parent or caregiver status, ancestry, political affiliation, veteran and/or military status, physical or mental disability, or any other status protected by federal or state law. Aurora considers qualified applicants with criminal histories, consistent with applicable federal, state, and local law. We are also committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, you may contact us at .\nFor California applicants, information collected and processed as part of your application and any job applications you choose to submit is subject to Aurora’s California Employment Privacy Policy.","description_format":"text","description_chars":6627,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity"],"hiring_locations":[{"name":"United States","iso":"US","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Autonomous Driving","Logistics"],"lifecycle":[{"event":"open","at":"2026-10-21T08:21:37Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":5,"expected_fill_days":77,"reasons":["conf:3","velocity","win:early"],"computed_at":"2026-10-25T05:49:26Z"},"pay":{"stated_usd_annual":260000,"is_top_pay":true},"html_url":"https://alion.io/job/aurora-senior-software-engineer-ml-evaluation-introspection-infrastructure","json_url":"https://alion.io/job/aurora-senior-software-engineer-ml-evaluation-introspection-infrastructure.json","meta":{"generated_at":"2026-10-25T09:44:42Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":13943,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-26T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":40481},"rest":"https://alion.io/mcp/rest/get_company?id=40481"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Faurora-senior-software-engineer-ml-evaluation-introspection-infrastructure"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Faurora-senior-software-engineer-ml-evaluation-introspection-infrastructure"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Faurora-senior-software-engineer-ml-evaluation-introspection-infrastructure"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/aurora-senior-software-engineer-ml-evaluation-introspection-infrastructure\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Faurora-senior-software-engineer-ml-evaluation-introspection-infrastructure"}]}