{"id":2383086,"url":"https://alion.io/job/apple-ai-evaluations-engineer-us-decision-intelligence-2","title":"AI Evaluations Engineer, US Decision Intelligence","company":{"id":12,"name":"Apple","domain":"apple.com","url":"https://alion.io/company/apple","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Apple Jobs","truth_index":{"grade":"B","score":77,"open_postings":1008,"ghost_share":0,"stale_share":0.938,"repost_share":0,"time_to_fill_p50_days":40,"computed_at":"2026-10-14T06:34:16Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Austin, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":202000,"max_usd":385000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":1124},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"Anthropic","optional":false},{"name":"CI/CD","optional":false},{"name":"FAISS","optional":false},{"name":"Gemini","optional":false},{"name":"Hadoop","optional":false},{"name":"Langfuse","optional":false},{"name":"LLM","optional":false},{"name":"Milvus","optional":false},{"name":"OpenAI","optional":false},{"name":"Pinecone","optional":false},{"name":"PostgreSQL","optional":false},{"name":"Python","optional":false},{"name":"RAG","optional":false},{"name":"Snowflake","optional":false},{"name":"Spark","optional":false},{"name":"SQL","optional":false},{"name":"Context Engineering","optional":true},{"name":"Embeddings","optional":true},{"name":"RabbitMQ","optional":true},{"name":"Recommender Systems","optional":true},{"name":"Redis","optional":true}],"status":"live","first_seen_at":"2026-09-18T15:48:06Z","employer_posted_date":"2026-10-12","last_verified_at":"2026-10-14T08:41:18Z","board_verified":true,"closed_at":null,"days_open":25,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":25},"description":"Imagine what you could do here. At Apple, new ideas have a way of becoming outstanding products, services, and customer experiences very quickly. Bring passion and dedication to your job, and there's no telling what you could accomplish.\nApple’s Sales organization generates the revenue needed to fuel our ongoing development of products and services. This, in turn, enriches the lives of hundreds of millions of people around the world. We are, in many ways, the face of Apple to our largest customers.\nApple's US Decision Intelligence (DI) team is looking for a talented individual who is passionate about crafting, implementing, and operating AI solutions that have a direct and measurable impact on Apple Sales and its customers.\nDescription\nWe’re seeking a visionary AI Evaluations Engineer to own the end-to-end evaluation pipeline for our AI products and agentic workflows. This role will focus on implementing and maintaining evaluation frameworks, instrumentation, and workflows that help us understand how well our AI systems perform, where they fail, and how they improve over time. You own the evaluation gate and the standards.\nThis role will operate in both capacities, to augment existing AI roadmap, as well as innovate and trailblaze new frontier-technology projects, crafting AI experiences that reduce time to insight and catalyze decision making.\nMinimum Qualifications\n5+ years of experience in data and AI-related fields such as AI engineering, software development, ML engineering, data science, or QA roles.\nEagerness and ability to learn new skills and solve dynamic problems in an encouraging and expansive environment.\nStrong Python skills.\nHands-on experience with AI evaluation techniques, such as Golden datasets, LLM-as-a-Judge, or rubric-based scoring.\nExperience with different LLM ecosystems (OpenAI, Anthropic, Gemini, etc.), RAG pipelines, vector databases (e.g., Pinecone, FAISS, Milvus, PostgreSQL).\nProficiency in SQL and experience with at least one major data analytics platform, such as Hadoop, Spark, or Snowflake.\nExperience with CI/CD or release validation workflows.\nExperience working with data science teams on insights generation leveraging LLMs.\nStrong time management skills with the ability to collaborate across multiple teams.\nAble to balance competing priorities, long-term projects, and ad hoc requirements.\nAbility to work in a fast-paced, dynamic, constantly evolving business environment.\nHands-on experience with Langfuse or similar tools for LLM observability.\nComfortable working with product/domain experts to translate fuzzy correctness criteria into measurable rubrics or metrics.\nB.S. degree in Computer Science/Engineering, or equivalent work experience\nPreferred Qualifications\nSound communication skills - expert at messaging domain and technical content, at a level appropriate for the audience. Strong ability to gain trust with stakeholders and senior leadership.\nFamiliarity with embeddings, retrieval algorithms, agents, and data modeling for vector and graph databases.\nOther complementary technologies for distributed systems architecture and asynchronous messaging, agent communication, and caching like RabbitMQ, Redis, and Valkey are preferred.\nExperience working across global teams to ensure alignment of product development.\nApplied knowledge of GenAI and RAG strategies, microservices, recommendation systems, and context engineering.\nWorking knowledge of agent evaluation concepts like trajectory vs. end-to-end vs. component-level evaluation, tool-call correctness.\nAdvanced degree (MS or Ph.D.) in Economics, Electrical Engineering, Statistics, Data Science, or a similar quantitative field is preferred.","description_format":"text","description_chars":3690,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"bachelor","optional":true},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Operating Systems","Laptops","Smartwatches & Fitness Trackers","Tablets"],"lifecycle":[{"event":"open","at":"2026-10-12T19:41:51Z"}],"visa":[{"country":"US","licensed_sponsor":true,"evidence":"H-1B filings in 12 months: 6,350 · green card filings: 37","filings_12m":6350,"filings_prev_12m":6527,"green_card_filings_12m":37,"median_offered_wage_usd":177355,"route":null,"cap_exempt":false,"checked_at":"2026-10-03T21:08:04+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","US Department of Labor: PERM disclosure data (green cards)"],"filings_for_role_12m":2959}],"liveness":{"score":47,"band":"ok","label":"Likely open","p_open":1,"p_active":0.527,"p_room":0.9,"age_days":25,"expected_fill_days":40,"reasons":["conf:3","stale_co","win:mid","comp:brand"],"computed_at":"2026-10-14T06:34:16Z"},"pay":null,"html_url":"https://alion.io/job/apple-ai-evaluations-engineer-us-decision-intelligence-2","json_url":"https://alion.io/job/apple-ai-evaluations-engineer-us-decision-intelligence-2.json","meta":{"generated_at":"2026-10-14T10:23:44Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":12083,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-15T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":12},"rest":"https://alion.io/mcp/rest/get_company?id=12"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fapple-ai-evaluations-engineer-us-decision-intelligence-2"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fapple-ai-evaluations-engineer-us-decision-intelligence-2"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fapple-ai-evaluations-engineer-us-decision-intelligence-2"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/apple-ai-evaluations-engineer-us-decision-intelligence-2\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fapple-ai-evaluations-engineer-us-decision-intelligence-2"}]}