AI Machine Learning Engineer

We are the leading Tech Company in Europe, supporting digital innovation for over 45 years.

We are committed to bringing digital technology into every business. How? By putting customers first and becoming the go-to partner for professionals, small, medium and large companies, offering customised solutions and services to optimise business processes, from HR and customer relations to e-invoicing and digital payments.

The Research and Development team is the beating heart of the Group's innovation. We are constantly committed to researching and developing new technologies and solutions to anticipate market needs and exceed our customers' expectations. We invest significantly in artificial intelligence (AI), exploring its potential to develop innovative solutions and improve our products and services.

Join an agile team where you can enhance your technical skills by participating in challenging and scalable projects in the field of AI Engineering.

In a fast-paced development environment focused on growth and innovation, you will be responsible for designing the tooling and pipelines that allow us to test AI agents and applications with the same rigor applied to software testing: systematically, continuously, and with clear pass/fail criteria.

In particular, you will:

• Design and build testing harnesses that evaluate AI agent behavior, tool use, and outputs, as well as the applications that integrate them, against defined quality criteria.

• Develop and maintain a central quality context layer - shared datasets, golden traces, eval sets, and metadata that ground testing across agents, applications, and teams.

• Build and operate quality gates integrated into CI/CD for both agent and application code, blocking regressions before they reach production.

• Design and track KPIs and quality metrics (accuracy, latency, hallucination rate, task success rate, drift, application-level reliability and performance) with dashboards and automated monitoring/alerting.

• Develop EL(T) pipelines to collect, transform, and structure test data, traces, application logs, and evaluation results for analysis.

• Collaborate with AI/ML engineers, application developers, data engineers, and product stakeholders to define what "quality" means for each agentic system and application, and translate that into automated checks.

• Continuously improve test coverage, harness reliability, and evaluation methodology as agent and application capabilities evolve.

• Document testing frameworks, quality standards, and harness usage for the broader engineering team.

What skills are we looking for?

• At least 2 years of experience in AI/ML engineering.

• Experience building or working with testing frameworks, harnesses, or evaluation pipelines - for traditional applications, LLM/agent-based systems, or both.

• Familiarity with CI/CD and DevOps practices and integrating automated checks/gates into pipelines.

• Experience with at least one cloud platform (AWS, Azure, or GCP).

• Understanding of how LLMs and AI agents work (tool use, context, orchestration), even if primarily from a testing/QA angle.
• Hands-on experience with AI agent frameworks (MCP, LangChain, PydanticAI, LangGraph) or LLM eval tools (LangFuse, Ragas, DeepEval, Braintrust).

• Comfortable defining and tracking metrics/KPIs and building monitoring dashboards.

• Strong problem-solving skills and attention to detail - you like breaking things systematically to find what's wrong.
• Excellent knowledge of the English language (C1 level). Knowledge of the Italian language will be considered a plus.

Do you recognize yourself in the skills we’re looking for? That’s a great start. We’re looking for people who reflect our values through their growth mindset and their ability to turn challenges into opportunities.

• ENTREPRENEURSHIP - is the engine that drives us to excel, innovate and challenge ourselves for our own success and that of our company. We are proactive, responsible and customer- centric, always ready to question the status quo and achieve extraordinary results.
• INNOVATION - is the guiding light that illuminates our path as we move through an environment full of different experiences, knowledge and perspectives. We are not afraid of making mistakes; we work in an environment that is fertile for the development of innovative solutions.
• INCLUSION - is the sentiment that guides our relationships amongst individuals, each of whom has a different history and experience. We advocate for an inclusive work environment that values each person's unique characteristics.

Do you identify with this? Apply now in just a few clicks and track your application on our portal! Here, we give space to those who want to get involved, grow and leave their mark.

What we offer:

We believe that by creating value for our people, we promote a culture of excellence, innovation and ethical and responsible business practices.

We are a tech company, and our approach to work is far from being conventional! We believe in a dynamic environment where innovation, growth and wellbeing intertwine every day. Here, work is not just a set of tasks, but an opportunity to express talent, experiment and make a difference.

• WELLBEING: we take care of your well-being! We offer private health insurance with comprehensive coverage and a 13th salary, to tangibly recognize your contribution and support you both inside and outside of work.
• FLEXIBILITY: we believe in mutual trust! You won't clock in and out, and you'll work in a flexible, hybrid way. And to make it easier for you, we'll give you everything you need to be productive and connected remotely.
• DEVELOPMENT AND GROWTH: we invest in people! You'll take part in a personalised development programme to build your skills and contribute to the achievement of the company's goals.
• DIVERSITY AND INCLUSIVITY: we value differences! You will work in an inclusive environment based on collaboration and respect. Our leadership is focused on trust and the growth of each talent.
• IMPACT: we work with pride! We are committed to creating shared value: for our customers, our partners and the community. In every project, we focus on sustainability and innovation.