Senior LLM Engineer
Confirmed on the employer's own hiring board on Oct 30, 2026. First seen by Alion on Oct 28, 2026. Dedagroup scores A on the Alion truth index.
We are one of the leading technology groups in Italy, with operations in the United States. We operate as a Business & Technology Accelerator for companies, financial institutions, and public sector organizations, supporting the evolution of their IT and digital strategies. Through an ecosystem of companies that share values, business strategies, and a common vision, we enable our clients to unlock the full potential of technology, amplifying its positive impact on both the economy and society. This role is dedicated to our US market.
What to expect
As an LLM Engineer, you build the LLM features in our products: turning a credit-union staff member’s question into a correct query against the semantic layer and explaining the answer, extracting the facts that matter from incoming content against rules users define, and writing the narrative around easurement results. You write the prompts and agent logic, build the retrieval, and add the guardrails that stop a model from leaking one credit union’s data to another or inventing a number. Above all, you measure: every feature has an evaluation set, and no prompt or model change ships without a score. The parts the products share-one evaluation harness, one prompt repository, one guardrail library-are built once and used everywhere, and you work with the business analyst who defines the metrics with the credit unions, the backend engineers who serve the features, and the engineers who run the models. The data is regulated member financial data, so a model sees only what the person asking may see, and every figure in an answer traces back to the query that produced it. We work in Python, with part of the existing LLM code in Java/Spring Boot, and with models behind an interface so that one can replace another; we hire for the understanding underneath the tool and expect you to carry it across. Your ultimate goal is LLM features a credit union can trust with its own numbers-and a score that shows why it can.
What you will do
Questions over Data: Turn a staff member’s question into a query against the semantic layer-metric, dimensions, filters, time range-run it, and explain the answer in plain language, including saying so when the data cannot answer the question.
Extraction & Alerts: Extract the facts that matter from incoming content against the rules users define, each with a citation back to its source, so that an alert can be checked rather than believed.
Narratives over Results: Generate narratives around measurement results that say only what the numbers support, with every figure in the text matching the result it came from.
Prompts, Agents & Retrieval: Write the prompts and agent logic and build the retrieval each feature needs, keeping prompts versioned in the shared repository rather than buried in code, and flows as deterministic as the task allows.
Guardrails: Build the shared guardrail library-tenant scoping enforced outside the model, grounding checks that reject a number not found in the data, input and output validation, and refusal when a request is out of scope.
Evaluation: Build and run the shared evaluation harness and an evaluation set for every feature-reference questions, labelled extractions, adversarial cases-wired into CI so that no prompt or model change ships without a score, and fed with the failures production finds.
Running What Ships: Trace every call, watch quality, latency and cost per feature, and be able to say which model and which prompt version produced any answer a user questions.
What we expect from you
Scope You Have Owned: Experience as an LLM, AI or ML engineer-or as a backend or data engineer who has built LLM features-in which you shipped at least one LLM feature to real users and measured it: built its evaluation set and let the scores decide what shipped. We weigh what you have owned, not the years it took.
Evaluation Design: You can build an evaluation set for a task-what goes in, how it is labelled, which metric decides-and you know where exact match, programmatic checks and LLM-as-judge each work and how to calibrate a judge against human labels; promptfoo, Ragas, DeepEval, Langfuse, LangSmith or a harness you built yourself all count.
Prompting & Agents: Production experience with structured outputs, tool calling and multi-step agent flows, and the judgment to replace an agent step with plain code where that will do; LangGraph, LlamaIndex, Semantic Kernel or plain Python count, as long as you know what the framework does for you.
Retrieval: You have built retrieval that works-chunking, embeddings, hybrid lexical and vector search, reranking, metadata filters-and measured it separately from the generation that uses it.
SQL & Semantic Layers: Solid SQL-PostgreSQL preferred; any serious relational database counts-enough to tell whether a generated query answers the question asked, and an understanding of how a semantic layer defines metrics and dimensions.
Guardrails & Isolation: You treat prompt injection, leakage between tenants and invented figures as real threats, and design against them with authorization outside the model, grounding checks and validated outputs, not with instructions in a prompt.
Python & Engineering Practice: Solid Python, Git, code review, tests, CI and containers, with models treated as swappable components behind an interface, hosted or open-weight. LLM features are software: versioned, reviewed and released like the backend.
Ownership & Communication: You can explain to a product manager why a change raised one score and lowered another, and you say early when a feature cannot reach the quality it needs.
Nice to Haves: Text-to-SQL or semantic-layer querying in production (dbt Semantic Layer, Cube, LookML or similar); Information extraction from documents and web content (HTML and PDF parsing, schema-guided extraction, citations); Open-weight models served in-house (vLLM, TGI, Ollama), and fine-tuning or distillation for a narrow task; LLM observability and tracing (Langfuse, LangSmith, OpenTelemetry); Search engines and vector stores (OpenSearch, Elasticsearch, pgvector); Financial services or other regulated data (GLBA, GDPR), particularly where a regulation shaped a design; Dagster or another orchestrator for batch LLM jobs; Java/Spring Boot, which part of the existing LLM code and the backend serving these features run on.