{"id":3571555,"url":"https://alion.io/job/velixo-applied-ai-engineer","title":"Applied AI Engineer","company":{"id":3113674,"name":"Velixo","domain":"velixo.com","url":"https://alion.io/company/velixo","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":null,"employment_type":"full_time","work_mode":"remote","remote_scope":"stated_countries","remote_scope_basis":"posting_text","remote_working_hours":null,"hiring_geo_confidence":"inferred","locations":[],"countries":[],"hiring_countries":["CA"],"hiring_countries_total":1,"salary":null,"salary_estimate":null,"experience_years_min":null,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"ChatGPT","optional":false},{"name":"Claude","optional":false},{"name":"Copilot","optional":false},{"name":"Langfuse","optional":false},{"name":"LLM","optional":false},{"name":"LLM Evaluation","optional":false},{"name":"Model Context Protocol","optional":false},{"name":"Prompt Caching","optional":false},{"name":"Promptfoo","optional":false},{"name":"Python","optional":false},{"name":"TypeScript","optional":false}],"status":"live","first_seen_at":"2026-10-31T16:55:18Z","employer_posted_date":null,"last_verified_at":"2026-10-31T16:55:18Z","board_verified":false,"closed_at":null,"days_open":3,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":3},"description":"About Velixo\nVelixo builds Excel-native reporting, planning, and automation for finance teams running Acumatica, Sage Intacct, Business Central, and MYOB. Founded in 2017 and headquartered in Montreal, we serve finance and accounting teams who live in spreadsheets and need their ERP data to be live, trustworthy, and writable. \n\nVelixo is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.\n\nThis is a fully remote position. While we welcome applications from across Canada, we are giving priority to candidates located in Quebec at this time.\n\nThe role \nYou will own how well Velixo Intelligence works and what it costs us to run. That means building the evaluation infrastructure that tells us whether a model change made our product better or worse, and driving down the cost per action without degrading quality. \nThis is a measurement-first engineering role. Today we ship AI features and form opinions about quality from anecdotes. You will replace that with evidence. \nAbout Velixo Intelligence Velixo Intelligence is our MCP-based gateway that lets finance teams query their ERP in natural language and perform governed writebacks from Claude, ChatGPT, and Copilot. Reads run through GIQL, our query language. Writes go through the ERP's own screen APIs so that every permission, validation, and audit rule still applies. \nIt launched in public preview this year. Usage is growing, the surface area is expanding, and our understanding of the unit economics has not kept pace. That gap is the reason this role exists. \nWhat you will own Evaluation Build and maintain regression suites for GIQL query generation, writeback correctness, and tool selection \n\nDefine what \"good\" means for each action type and make it measurable \n\nRun structured evaluations before every model upgrade or prompt change, and produce a clear ship or no-ship recommendation \n\nTrack quality over time so we notice degradation before customers do \n\nCost engineering Instrument every Gateway action for tokens, cache behavior, model, latency, and tenant \n\nEstablish and maintain the cost-per-action baseline that the rest of the company plans against \n\nReduce cost through prompt compression, prompt caching, context pruning, and routing work to smaller models where evaluation shows quality holds \n\nIdentify the expensive tail: which customers, which query shapes, which failure and retry loops \n\nModel strategy Track new model releases across providers and evaluate them against our workloads rather than published benchmarks \n\nMaintain a current view of the price and performance frontier for what we do \n\nRecommend when to migrate, when to wait, and when to run models in parallel \n\nPartnership with Finance Supply the unit cost data behind our credit pricing and plan allowances \n\nModel the margin impact of proposed pricing changes alongside our VP Finance \n\nFlag when product decisions will move the cost curve before they ship \n\nYou will not own pricing decisions. You will own the numbers those decisions depend on. \nFirst 90 days Days 1 to 30: Full instrumentation of the Gateway. A dashboard showing cost per action type, per tenant, and per model that the exec team checks weekly. \n\nDays 31 to 60: A working evaluation suite covering our highest-volume action types, with a documented baseline. \n\nDays 61 to 90: A prioritized cost reduction roadmap with sized estimates, and at least one shipped optimization with measured before-and-after quality. \n\nWhat we are looking for Substantial experience building on LLM APIs in production, not in demos or notebooks \n\nYou have built an eval harness before and can explain why the naive version of it misleads you \n\nHands-on with the current tooling landscape: tracing and observability platforms such as Langfuse, eval frameworks such as promptfoo. You have opinions about which of these earn their keep and which add ceremony without adding signal. \n\nComfortable with token accounting, context window management, and caching strategies \n\nStrong analytical instincts and fluency with data. You reach for a query or a notebook before you reach for an opinion. \n\nAble to write clearly about tradeoffs for a non-technical audience, because your conclusions will land in board decks and pricing meetings \n\nPython or TypeScript proficiency \nNice to have Experience with MCP or other tool-calling frameworks \n\nBackground in ERP, accounting, or financial systems \n\nPrior exposure to usage-based or credit-based pricing models \n\nExperience at a small company where you set your own priorities \n\nWhy this role is interesting Most companies bolt a chat box onto their product and hope. We are building governed, auditable write access to financial systems of record, where a wrong answer has real consequences and a hallucinated journal entry is unacceptable. The evaluation problem here is genuinely hard and genuinely matters. \nYou will also have unusual visibility. The numbers you produce will directly shape what we charge and what we build. \nOriginally posted on Himalayas","description_format":"text","description_chars":5081,"description_truncated":false,"requirements":{"experience_years_min":null,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"Canada","iso":"CA","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Data & Analytics","Data Integration & ETL Tools","Hospitals & Health Systems"],"lifecycle":[{"event":"open","at":"2026-10-31T17:28:15Z"}],"visa":[],"liveness":{"score":86,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.86,"p_room":1,"age_days":3,"expected_fill_days":27,"reasons":["seen:3","win:early"],"computed_at":"2026-11-04T06:34:25Z"},"pay":null,"html_url":"https://alion.io/job/velixo-applied-ai-engineer","json_url":"https://alion.io/job/velixo-applied-ai-engineer.json","meta":{"generated_at":"2026-11-04T07:14:21Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":11356,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-05T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":3113674},"rest":"https://alion.io/mcp/rest/get_company?id=3113674"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fvelixo-applied-ai-engineer"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fvelixo-applied-ai-engineer"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fvelixo-applied-ai-engineer"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/velixo-applied-ai-engineer\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fvelixo-applied-ai-engineer"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3571555}},{"tool":"search_jobs","arguments":{"company":"velixo.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3571555},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}