{"id":2992015,"url":"https://alion.io/job/apple-applied-researcher-on-device-multimodal-reasoning","title":"Applied Researcher: On-Device Multimodal Reasoning","company":{"id":12,"name":"Apple","domain":"apple.com","url":"https://alion.io/company/apple","size_band":"5000+","is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Apple Jobs","truth_index":{"grade":"A","score":94,"open_postings":180,"ghost_share":0,"stale_share":0.011,"repost_share":0.044,"time_to_fill_p50_days":153,"computed_at":"2026-11-18T05:48:20Z"}},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":null,"work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Sunnyvale, United States"],"countries":["US"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":227000,"max_usd":500000,"period":"year","method":"role_country_seniority_unknown","sample_n":4194},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Chain-of-Thought","optional":false},{"name":"Computer Vision","optional":false},{"name":"Core ML","optional":false},{"name":"Knowledge Distillation","optional":false},{"name":"KV Cache","optional":false},{"name":"llama.cpp","optional":false},{"name":"LLM","optional":false},{"name":"Machine Learning","optional":false},{"name":"MLX ML","optional":false},{"name":"Model Distillation","optional":false},{"name":"Multimodal AI","optional":false},{"name":"Post-training","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"Quantization","optional":false},{"name":"SFT","optional":false},{"name":"SGLang","optional":false},{"name":"vLLM","optional":false},{"name":"VLM","optional":false},{"name":"CLIP","optional":true},{"name":"DINOv2","optional":true},{"name":"GRPO","optional":true},{"name":"Mamba","optional":true},{"name":"Self-Supervised Learning","optional":true},{"name":"Speculative Decoding","optional":true},{"name":"TensorRT","optional":true},{"name":"TensorRT-LLM","optional":true},{"name":"World Models","optional":true}],"status":"live","first_seen_at":"2026-08-05T22:59:27Z","employer_posted_date":"2026-11-18","last_verified_at":"2026-11-18T16:50:39Z","board_verified":true,"closed_at":null,"days_open":105,"trust":{"level":"ok","repost_count":2,"flags":[],"days_open":102},"description":"The Video Computer Vision (VCV) organization is an applied research and engineering team developing real-time, on-device Computer Vision and Machine Perception technologies across Apple products. Within VCV, our team builds next-generation multimodal AI systems that combine on-device multimodal encoders, large language models, and foundation models to create intelligent systems capable of understanding, reasoning, and acting across language, vision, audio, and tools. Our work is deeply integrated into the Apple ecosystem, partnering across hardware, software, and ML teams to deliver real-time, scalable, and privacy-preserving experiences reaching millions of users.\nDescription\nWe are seeking an Applied Researcher with deep expertise in multimodal reasoning at small model scale - making vision-language models in the smaller regime (under ~10B parameters, down to sub-1B) think, plan, and act reliably under strict compute, memory, and latency constraints. In this role you will own the reasoning side of the on-device multimodal stack: designing compact VLMs that reason over images, video, and 3D scene content; compressing and distilling the reasoning process itself; and engineering the decoding and inference path that makes multi-step reasoning affordable on an Apple device. This role offers the unique opportunity to define what on-device intelligence looks like for hundreds of millions of users. You'll push the boundaries of what small models can achieve - enabling real-time multimodal understanding and multi-step reasoning without reliance on cloud connectivity. You'll collaborate with hardware teams, compiler engineers, and ML researchers to unlock capabilities that few organizations can deliver at Apple's scale and quality bar.\nThis role spans multiple dimensions of efficient on-device reasoning - including VLM architecture and connector design, reasoning post-training (SFT/RL), chain-of-thought compression, speculative and structured decoding, visual token reduction, quantization and distillation, and hardware-aware inference optimization.\nA core focus of this role is efficient reasoning: compressed and latent chain-of-thought, reasoning distillation from frontier teachers, adaptive test-time compute (knowing when - and how long - to think), speculative and structured decoding, KV-cache compression, and visual-token efficiency. A second focus is reasoning over real-time visual perception experts. Rather than consuming pixels alone, the VLM should be able to invoke and reason over the outputs of specialist on-device vision models - feed-forward 3D scene and geometry estimators (VGGT-style reconstruction, depth, camera pose), human body and hand mesh/pose recovery, object detectors, localizers, and trackers - and fuse those structured, metric outputs into its reasoning about the scene. This raises real research questions: how to represent geometry, body parameters, and detections compactly in a token-budgeted context; how to schedule which experts run at which frame rate within a real-time budget; and how to train a small model to invoke, trust, and cross-check them. Efficiency is treated as a first-class metric here: reasoning quality is measured at a fixed latency, memory, and power budget.\nMinimum Qualifications\nMS in Computer Science, Machine Learning, AI, Computer Vision, or a related field (or equivalent practical experience)\nStrong foundation in deep learning, with specific experience in LLM or VLM training, post-training, or inference optimization\nDemonstrated experience working with multimodal models (vision-language models, multimodal LLMs) in resource-constrained environments, including hands-on work with reasoning quality, decoding, or model compression\nProficiency in Python and modern deep learning frameworks (PyTorch preferred), with familiarity with inference and optimization toolchains (quantization, distillation, pruning - e.g., vLLM/SGLang, llama.cpp, MLX, CoreML)\nPreferred Qualifications\nPhD with research in efficient multimodal reasoning, LLM reasoning, model compression, efficient inference/decoding, or lightweight VLM architectures\nExperience training or post-training vision-language models end-to-end - connector/projector design, visual instruction tuning, resolution and token-budget trade-offs, small-model recipes\nHands-on experience with reasoning techniques: chain-of-thought distillation and compression, latent/implicit reasoning, reward-guided decoding, RL for reasoning (GRPO, RLVR, STaR), or test-time compute allocation\nExpertise in decoding and serving optimizations: speculative decoding, structured/grammar-constrained generation, KV-cache quantization and eviction, continuous batching, long-context inference\nExperience combining LLMs with real-time perception models - 3D reconstruction and geometry (VGGT, DUSt3R/MASt3R-style, SLAM, monocular depth), human pose and body/hand mesh recovery (SMPL-family), detection, segmentation, or tracking - and with spatial or 3D-grounded reasoning and embodied/spatial VQA\nExperience deploying LLM or multimodal models on mobile or edge hardware (CoreML, MLX, TensorRT-LLM, or equivalent), with attention to ANE/GPU kernel and memory constraints\nExperience with quantization-aware training, mixed-precision inference, and knowledge distillation for vision-language models\nFamiliarity with efficient vision encoders and self-supervised/joint-embedding pretraining (V-JEPA, I-JEPA, MAE, DINO/DINOv2, SigLIP, CLIP), Mamba/SSM vision backbones, or streaming architectures for real-time video with fixed memory budgets\nInterest in Video-LLMs, long-video reasoning, and world models for prediction and planning\nPublication record in top-tier venues is a plus (NeurIPS, ICML, ICLR, CVPR, ECCV, ACL, MLSys, ICRA, etc.)","description_format":"text","description_chars":5766,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":{"level":"master","optional":true},"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Operating Systems","Laptops","Smartwatches & Fitness Trackers","Tablets"],"lifecycle":[{"event":"open","at":"2026-10-21T10:08:55Z"},{"event":"close","at":"2026-10-21T16:58:38Z"},{"event":"reopen","at":"2026-11-11T11:18:52Z"},{"event":"close","at":"2026-11-11T22:29:36Z"},{"event":"reopen","at":"2026-11-12T04:23:18Z"},{"event":"close","at":"2026-11-12T16:09:35Z"},{"event":"reopen","at":"2026-11-13T04:11:20Z"},{"event":"close","at":"2026-11-13T10:47:28Z"},{"event":"reopen","at":"2026-11-13T23:46:56Z"},{"event":"close","at":"2026-11-17T04:54:28Z"},{"event":"reopen","at":"2026-11-18T16:44:08Z"}],"visa":[{"country":"US","licensed_sponsor":true,"adverts_offering_sponsorship_24m":0,"sanctioned_now":false,"evidence":"H-1B filings in 12 months: 6,518 · green card filings: 214","filings_12m":6518,"filings_prev_12m":7008,"green_card_filings_12m":214,"median_offered_wage_usd":176830,"route":null,"cap_exempt":false,"checked_at":"2026-11-18T07:32:00+00:00","sources":["US Department of Labor: LCA disclosure data (H-1B, H-1B1, E-3)","US Department of Labor: PERM disclosure data (green cards)"],"filings_for_role_12m":2981}],"liveness":{"score":42,"band":"fade","label":"Fading","p_open":1,"p_active":0.589,"p_room":0.72,"age_days":105,"expected_fill_days":153,"reasons":["conf:6","repost:2","win:mid","crowd:brand"],"computed_at":"2026-11-18T23:43:46Z"},"pay":null,"html_url":"https://alion.io/job/apple-applied-researcher-on-device-multimodal-reasoning","json_url":"https://alion.io/job/apple-applied-researcher-on-device-multimodal-reasoning.json","meta":{"generated_at":"2026-11-18T23:43:46Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":5342,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-19T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.1,"usd":0.01,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":12},"rest":"https://alion.io/mcp/rest/get_company?id=12"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fapple-applied-researcher-on-device-multimodal-reasoning"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"free","price":{"credits":0,"usd":0},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fapple-applied-researcher-on-device-multimodal-reasoning"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.1,"usd":0.01},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fapple-applied-researcher-on-device-multimodal-reasoning"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/apple-applied-researcher-on-device-multimodal-reasoning\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fapple-applied-researcher-on-device-multimodal-reasoning"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":2992015}},{"tool":"search_jobs","arguments":{"company":"apple.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":2992015},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}