{"id":2472773,"url":"https://alion.io/job/level-senior-applied-ai-engineer","title":"Senior Applied AI Engineer","company":{"id":4417780,"name":"Level","domain":"level.me","url":"https://alion.io/company/level-14","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":{"grade":"A","score":88,"open_postings":28,"ghost_share":0,"stale_share":0.5,"repost_share":0,"time_to_fill_p50_days":null,"computed_at":"2026-10-16T05:45:00Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Austin, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":129000,"max_usd":245000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":1213},"experience_years_min":7,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"AI Agents","optional":false},{"name":"Anthropic SDK","optional":false},{"name":"AWS","optional":false},{"name":"Chain-of-Thought","optional":false},{"name":"Claude","optional":false},{"name":"CrewAI","optional":false},{"name":"Embeddings","optional":false},{"name":"Function Calling","optional":false},{"name":"GCP","optional":false},{"name":"Human-in-the-Loop","optional":false},{"name":"Hybrid Search","optional":false},{"name":"In-Context Learning","optional":false},{"name":"Instructor","optional":false},{"name":"LangGraph","optional":false},{"name":"LlamaIndex","optional":false},{"name":"LLM","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"LLM Guardrails","optional":false},{"name":"Mastra","optional":false},{"name":"OpenAI Agents SDK","optional":false},{"name":"Outlines","optional":false},{"name":"Prompt Engineering","optional":false},{"name":"Pydantic AI","optional":false},{"name":"Python","optional":false},{"name":"Structured Outputs","optional":false},{"name":"Tool Use","optional":false},{"name":"TypeScript","optional":false},{"name":"DSPy","optional":true},{"name":"GraphRAG","optional":true},{"name":"LangChain","optional":true},{"name":"Model Context Protocol","optional":true}],"status":"live","first_seen_at":"2026-05-26T15:33:16Z","employer_posted_date":"2026-05-26","last_verified_at":"2026-10-16T10:36:28Z","board_verified":true,"closed_at":null,"days_open":142,"trust":{"level":"stale","repost_count":0,"flags":["stale"],"days_open":142},"description":"Applied AI Engineer\nLocation: Onsite 5 days a week in Austin, TX, relocation is offered.\nLevel is a learning technology company dedicated to helping students build real academic and life skills with confidence and joy. We combine proven curriculum principles with world class interactive design to make meaningful practice something students want to come back to, not something they struggle through.\nWe support what teachers, schools, and parents are already doing by increasing student engagement with high quality, standards aligned practice that reinforces classroom learning. That’s why we’re building:\nFor students: Rewarding and motivating learning experiences that meet students where they are to build real academic proficiency.\nFor educators: Tools that fit naturally into instruction and help students stay engaged while reducing teacher workload.\nFor parents:Activities that help your kid catch up or get ahead, build confidence, and minimize homework battles, whether learning at school or at home.\nWe are committed to helping every person maximize their potential and live a life of meaning. It’s a difficult problem that requires brilliant people and tremendous effort over time.\nWhat You'll Do:\nShip production agentic systems- Design and build agents and agentic workflows that solve a defined problem end to end. You own prompts, tools, retrieval, guardrails, observability, cost and latency budgets, and rollout.\n\nAutomate SME workflows- Identify high-leverage operational toil (review pipelines, content QA, labeling and ops loops, support triage, internal copilots) and partner with the SMEs running those workflows to define success criteria, validate outputs, and replace meaningful chunks of that work with AI systems they trust. SMEs are co-owners of quality from the start of the project.\n\nOwn the evaluation loop and the golden datasets that anchor it- Build offline evals, LLM-as-judge with calibration, regression suites, and online metrics. Maintain versioned, decontaminated golden datasets covering intents, difficulty, edge cases, and adversarial inputs, and continuously enrich them with real production failures and SME-validated labels. The measurement plan is what decides whether a feature ships.\n\nMake AI features safe- Treat what you deliver as a regulated product. Design for relevant compliance frameworks (e.g., COPPA, FERPA) from day one, run safety and bias evals before launch and continuously after, and build the human-in-the-loop and content-filtering controls AI features need before they reach end users.\n\nHand off what you ship- Production features leave your hands with documentation, runbooks, an eval harness, and dashboards. An embed is not complete until the receiving team has shipped a change to the system without you in the room. Typical embed length is 4 to 12 weeks.\n\nMake AI features easier for the rest of engineering to build- Internal libraries, patterns, and playbooks so other teams can ship AI features without your direct involvement.\n\nWhat You Need:\n7+ years professional experience as an Engineer with at least 1+ years of hands-on experience building agentic systems on at least one modern stack (LangGraph, the Anthropic SDK / Claude Agent SDK, OpenAI Agents SDK, Pydantic-AI, Mastra, LlamaIndex, CrewAI, or a homegrown stack). We care that you have built and operated agentic systems in production, not which framework.\n\nStrong Python familiarity plus one typed language for production services (TypeScript, Go, or similar). Cloud experience (AWS or GCP) and containerized deployment.\n\nSenior or staff-level software engineering foundation (formal or autodidact), with several years of production environment experience and a track record of leading systems to launch.\n\nMultiple shipped LLM-powered features in a production environment, with concrete stories about what broke, how you fixed it, and what you would do differently.\n\nPractical knowledge of common agentic patterns: ReAct, tool use with structured schemas, prompt chaining, routing, orchestrator-workers, evaluator-optimizer / reflection, and human-in-the-loop. You can decide when a deterministic workflow is the right answer instead of an autonomous agent.\n\nHands-on experience in a production environment with retrieval: chunking, embeddings, hybrid search, re-ranking, metadata filtering, and the failure modes of each. Working knowledge of grounding techniques that anchor generated answers in retrieved evidence (citation and quote extraction, faithfulness and refusal evals, post-hoc consistency checks).\n\nStrong prompt-engineering practice: zero-shot, few-shot, and many-shot patterns; example selection and ordering; in-context learning and chain-of-thought.\n\nComfort with structured output and validation in production (provider-native structured outputs, Instructor, Pydantic-AI, Outlines, or a comparable approach).\n\nDisciplined evaluation practice. You do not rely on subjective review to decide whether a system is ready.\n\nStrong written and verbal communication. You can explain an architectural trade-off to an executive and to a junior engineer in the same week.\n\nYou are comfortable using AI coding tools heavily in your implementation workflow while you own problem framing, design choices, and verification. We measure your output by working systems delivered, not lines of code written.\n\nNice to Have:\nAdvanced retrieval experience: GraphRAG, agentic retrieval, evaluation-driven retrieval tuning, and hybrid retrieval at scale.\n\nDirect or transferable experience with safety, privacy, and policy constraints in user-facing AI. K-12 or other regulated-domain experience is a strong plus.\n\nExperience with prompt-optimization frameworks (DSPy, TEXTGRAD, AdalFlow) where they paid off in production.\n\nA public repo, package, gist, or technical write-up of meaningful AI work, or a representative project you can describe in detail under your confidentiality constraints.\n\nOpen-source contributions to AI tooling (frameworks, agents, evals, MCP servers).\n\n*A note on our interview process\nOne of our interview rounds is an AI-assisted coding session. Bring your own setup (IDE, AI coding assistant, agentic tools, whatever you actually use day to day) and solve a realistic problem live with AI in the loop. We are looking at how you collaborate with AI tools: how you prompt, validate output, catch bad suggestions, decide when to override, and produce code that meets production standards. It is not a cleanroom algorithm round.","description_format":"text","description_chars":6487,"description_truncated":false,"requirements":{"experience_years_min":7,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":true,"industries":[],"lifecycle":[{"event":"open","at":"2026-10-13T20:21:59Z"}],"visa":[],"liveness":{"score":5,"band":"cold","label":"Long shot","p_open":1,"p_active":0.195,"p_room":0.28,"age_days":142,"expected_fill_days":24,"reasons":["conf:0","stale_co","velocity","win:tail","crowd:"],"computed_at":"2026-10-16T05:45:00Z"},"pay":null,"html_url":"https://alion.io/job/level-senior-applied-ai-engineer","json_url":"https://alion.io/job/level-senior-applied-ai-engineer.json","meta":{"generated_at":"2026-10-16T12:17:08Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":11401,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-17T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4417780},"rest":"https://alion.io/mcp/rest/get_company?id=4417780"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Flevel-senior-applied-ai-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Flevel-senior-applied-ai-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Flevel-senior-applied-ai-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/level-senior-applied-ai-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Flevel-senior-applied-ai-engineer"}]}