{"id":947578,"url":"https://alion.io/job/datasnipper-senior-software-engineer-llm-ops-evals","title":"Senior Software Engineer - LLM Ops & Evals","company":{"id":4724,"name":"DataSnipper","domain":"datasnipper.com","url":"https://alion.io/company/datasnipper","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":"Ashby","truth_index":null},"role":"AI/ML","role_family":"AI/ML","seniority":"senior","employment_type":"full_time","work_mode":"hybrid","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Amsterdam, Netherlands"],"countries":["NL"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":92000,"max_usd":179000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":23},"experience_years_min":5,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"Azure","optional":false},{"name":"GCP","optional":false},{"name":"Grafana","optional":false},{"name":"LLM","optional":false},{"name":"OpenTelemetry","optional":false},{"name":"Platform Engineering","optional":false},{"name":"Python","optional":false},{"name":"Terraform","optional":false},{"name":"OCR","optional":true},{"name":"Synthetic Data","optional":true}],"status":"live","first_seen_at":"2026-09-15T20:04:33Z","employer_posted_date":"2026-09-15","last_verified_at":"2026-11-04T08:18:58Z","board_verified":true,"closed_at":null,"days_open":49,"trust":{"level":"ok","repost_count":null,"flags":[],"days_open":49},"description":"Every AI call in DataSnipper goes through us. Product teams do not talk to model providers directly, they talk to our gateway. We are responsible for how inference is routed, how it fails over, what it costs, and how anyone can tell whether the output is any good.\nThe second half of the job is evaluation. We are building the platform teams use to measure AI quality: versioned datasets, experiment tracking, and evaluation runs they can act on. It is a hard problem and largely an open one, so you will have real influence over how we solve it.\nThis is a small team with a large blast radius. You will own real production systems, set the standards other teams build against, and see your work in front of hundreds of thousands of users in audit and finance.\nAbout DataSnipper\nAudit and finance are still massively manual and we are changing that. DataSnipper is a $1B, bootstrapped unicorn with 600,000+ users across 180+ countries, already embedded in the daily workflows of top audit and accounting firms.\nNow, we are taking things further with our Excel Agent, bringing AI directly into where the work actually happens. Unlike generic AI tools, we do not sit on the sidelines. Our AI operates inside Excel, with access to real documents and audit evidence, meaning it does not just generate answers, it does the work, with full traceability.\nWe are not just applying AI, we are redefining how audit gets done. If you want to build something category-defining at scale, this is the place.\nWhat you will do\nTechnical Delivery\nOwn the LLM gateway: routing, provider failover, rate limits, retries, and cost attribution across multiple model providers\n\nDeploy, version and deprecate models across clouds, regions and environments, including quota and capacity planning, managed as infrastructure as code\n\nBuild the shared evaluation platform: versioned datasets, experiment tracking, run and result schemas, reporting, and trace linkage back to the run\n\nOwn the infrastructure for async and long-running AI workloads\n\nReliability, Security & On-Call\nOwn observability for AI traffic: latency, retries and fallbacks, token usage, cost and errors, per team and per use case\n\nTake part in the on-call rotation, run incidents, and close the follow-ups\n\nImplement the security and compliance controls the platform is held to: retention, access control, RBAC and SSO\n\nCollaboration & Impact\nDefine and maintain clean integration contracts between the platform and the teams that consume it\n\nPartner with product and ML engineers to turn their requirements into platform capabilities that are self-service rather than a request queue\n\nWhat you will bring\nMust-Have\n5+ years in backend or platform engineering, with strong production Python\n\nExperience building or running LLM inference infrastructure: a gateway or routing layer with multiple providers, failover, rate limiting and cost attribution\n\nExperience with cloud at the infrastructure level and infrastructure as code (we run across Azure and GCP with Terraform)\n\nExperience running a shared service in production: on-call, incidents, postmortems, SLOs\n\nHands-on experience with observability tools (OpenTelemetry, Grafana), including instrumenting services and designing dashboards and alerts\n\nComfort with privacy and compliance work: PII handling, anonymisation, retention, access control\n\nExperience building platform or shared-service capabilities consumed by multiple internal teams\n\nNice-to-Have\nExperience with LLM or agent evaluation\n\nTemporal or another durable workflow engine\n\nSelf-hosted inference, capacity planning, load testing\n\nSynthetic data or document anonymisation pipelines\n\nDocument AI: VLMs, OCR, structured extraction and the metrics that go with it\n\nDomain experience in audit, accounting or fintech\n\nWhat we expect\nOwnership: You own work end-to-end, anticipate issues, and ensure high-quality delivery without close supervision\n\nGrowth Mindset: You encourage open feedback exchange and provide clear, balanced feedback that helps others grow\n\nCollaboration: You build strong cross-functional relationships and influence peers through expertise, data, and empathy\n\nAdaptability: You navigate ambiguity calmly, model positive behavior, and help peers adjust through clear communication\n\nJudgment: You exercise sound judgment in ambiguous situations, balance speed and accuracy, and adjust priorities proactively\n\nRecruitment steps\nRecruiter screen\n\nHiring Manager interview\n\nPeer programming session\n\nSystem design interview\n\nFinal interviews with Engineering leadership","description_format":"text","description_chars":4550,"description_truncated":false,"requirements":{"experience_years_min":5,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":[],"hiring_locations":[{"name":"Netherlands","iso":"NL","kind":"country"}],"hiring_excludes":[],"relocation_offered":false,"industries":["Document AI","Accounting & Tax Software"],"lifecycle":[{"event":"open","at":"2026-09-15T21:51:24Z"}],"visa":[],"liveness":{"score":64,"band":"ok","label":"Likely open","p_open":1,"p_active":0.854,"p_room":0.75,"age_days":49,"expected_fill_days":61,"reasons":["conf:2","velocity","win:late"],"computed_at":"2026-11-04T06:34:25Z"},"pay":null,"html_url":"https://alion.io/job/datasnipper-senior-software-engineer-llm-ops-evals","json_url":"https://alion.io/job/datasnipper-senior-software-engineer-llm-ops-evals.json","meta":{"generated_at":"2026-11-04T12:20:27Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":18554,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-05T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":4724},"rest":"https://alion.io/mcp/rest/get_company?id=4724"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fdatasnipper-senior-software-engineer-llm-ops-evals"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fdatasnipper-senior-software-engineer-llm-ops-evals"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fdatasnipper-senior-software-engineer-llm-ops-evals"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/datasnipper-senior-software-engineer-llm-ops-evals\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fdatasnipper-senior-software-engineer-llm-ops-evals"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":947578}},{"tool":"search_jobs","arguments":{"company":"datasnipper.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":947578},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}