{"id":2497629,"url":"https://alion.io/job/upwind-security-ai-engineering-lead","title":"AI Engineering Lead","company":{"id":9362,"name":"Upwind Security","domain":"upwind.io","url":"https://alion.io/company/upwind-security","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Comeet","truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":"lead","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Ramat Gan, Israel"],"countries":["IL"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":127000,"max_usd":304000,"period":"year","method":"global_role_cell_scaled_by_country","sample_n":1041},"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Agentic Workflows","optional":false},{"name":"AI Agents","optional":false},{"name":"eBPF","optional":false},{"name":"Function Calling","optional":false},{"name":"LangChain","optional":false},{"name":"LangGraph","optional":false},{"name":"LLM","optional":false},{"name":"Python","optional":false},{"name":"Structured Outputs","optional":false},{"name":"Tool Use","optional":false}],"status":"live","first_seen_at":"2026-10-14T05:08:40Z","employer_posted_date":"2026-10-14","last_verified_at":"2026-10-14T05:14:45Z","board_verified":true,"closed_at":null,"days_open":0,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":0},"description":"Description\nUpwind is a next-generation Cloud Security Platform that leverages runtime context to identify and prioritize critical risks, providing precise insights and efficient cloud security management. Unlike traditional tools, Upwind uses runtime data proactively for risk prioritization and posture insights, ensuring teams focus on what truly matters. With industry-leading efficiency and eBPF-powered sensors, Upwind delivers comprehensive capabilities, including agentless cloud posture discovery, real-time threat protection, and integrated API security. From misconfigurations to malware defense, Upwind ensures end-to-end, cost-effective cloud infrastructure protection.\nAt Upwind, you'll have the opportunity to work on some of the most challenging problems in applied AI and security, turning cutting-edge research and LLM capabilities into products that operate in real customer environments.\nWe're looking for an AI Engineering Lead to help drive the quality, intelligence, and continuous improvement of Upwind's AI and agentic systems.\nThis is a hands-on AI engineering lead role focused on data, evaluation, and agentic workflows. You'll build the systems and methodologies that allow us to understand how our agents perform, identify where they fail, and continuously improve their reasoning, tool usage, and outcomes.\nYou'll work closely with AI Researchers, Engineers, and Product teams to turn experiments into reliable, high-performing AI systems - from defining evaluation datasets and metrics to improving agentic workflows and deploying them at scale.\nWhat You’ll Build:\nAgentic Workflows: Design and continuously improve multi-step agent workflows, including reasoning, tool usage, planning, state management, and recovery strategies.\nAI Evaluation: Build evaluation frameworks, datasets, metrics, and automated pipelines to measure the quality and reliability of LLMs and agents.\nAI Data Infrastructure: Build the data pipelines and feedback loops required to collect, curate, analyze, and learn from real-world agent interactions\nAI Observability: Build visibility into agent behavior and performance, including trajectories, tool calls, latency, failures, and outcome quality.\nResponsibilities:\nOwn and evolve the evaluation strategy for Upwind's AI agents and LLM-powered features.\nDefine meaningful metrics and automated evaluation pipelines for measuring agent quality, accuracy, reliability, and task completion.\nDesign and improve agentic workflows to make agents more capable, reliable, efficient, and predictable.\nBuild feedback loops from production usage back into evaluation datasets and development workflows.\nDevelop tooling for running experiments and comparing models, prompts, workflows, and agent configurations.\nWork closely with AI Researchers to turn research ideas and experiments into production-quality AI capabilities.\nCollaborate with backend engineers to integrate AI workflows and evaluation systems into our production platform.\nEstablish best practices for testing and validating non-deterministic AI systems.\nMonitor AI systems in production and continuously identify opportunities for improvement.\nTake end-to-end ownership of projects- from defining the problem and building the first experiment to productionizing and measuring its impact.\nRequirements\nStrong engineering experience, with a track record of owning and delivering complex technical systems.\nHands-on experience building LLM-powered applications, agents, or AI systems.\nStrong understanding of LLM evaluation, experimentation, and data-driven development.\nExperience designing and implementing LLM-as-a-Judge evaluations, including evaluation criteria, scoring rubrics, structured outputs, and validation of judge reliability.\nStrong problem-solving skills and the ability to analyze complex AI behavior and turn observations into concrete improvements.\nExperience working with agentic workflows, tool calling, multi-step reasoning, or AI orchestration.\nAbility to work closely with researchers and engineers and bridge the gap between experimentation and production.\nComfortable working in a fast-paced environment where requirements and technologies evolve quickly.\nStrong Advantages:\nExperience with LLM evaluation frameworks and automated evaluation pipelines.\nProven experience with LLM-as-a-Judge methodologies, including calibrating judges against human evaluations, measuring evaluator consistency, and reducing evaluation bias\nExperience with LangGraph, LangChain, or comparable agent orchestration frameworks.\nExperience working with security, infrastructure, or other complex technical domains.\nExperience building production AI systems where correctness and reliability are critical.\nStrong Python skills and experience working with data processing and AI/ML tooling.\nWhy Upwind?\nYou'll have the opportunity to work on AI systems that are solving real, complex security problems- not demos or prototypes.\nYou'll be part of a team building the next generation of AI-powered security agents, with the opportunity to influence how we evaluate, improve, and scale these systems from the ground up.","description_format":"text","description_chars":5131,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"Israel","iso":"IL","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Cloud Security"],"lifecycle":[{"event":"open","at":"2026-10-14T05:14:45Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":0,"expected_fill_days":23,"reasons":["conf:1","win:early"],"computed_at":"2026-10-14T06:34:16Z"},"pay":null,"html_url":"https://alion.io/job/upwind-security-ai-engineering-lead","json_url":"https://alion.io/job/upwind-security-ai-engineering-lead.json","meta":{"generated_at":"2026-10-14T08:47:52Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":9529,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-10-15T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":9362},"rest":"https://alion.io/mcp/rest/get_company?id=9362"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fupwind-security-ai-engineering-lead"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fupwind-security-ai-engineering-lead"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fupwind-security-ai-engineering-lead"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/upwind-security-ai-engineering-lead\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fupwind-security-ai-engineering-lead"}]}