AI data analyst (LLM development and evaluation)

//Careers

AI data analyst (LLM development and evaluation)

Tallinn, Estonia - hybrid · Starting ASAP

Feelingstream provides conversation analytics for large contact centres. Our platform turns customer conversations - phone calls, chats, emails, and feedback - into clear insight using multilingual speech-to-text, analytics, and automation.

Our customers include banks, telecom companies, utilities, transportation providers, and insurance companies across Scandinavia and the Baltics.

Data security sits at the core of our operations. Feelingstream is ISO 27001 certified, and we are preparing for the EU AI Act - our product is being developed to meet the high-risk AI requirements expected in 2026.

We are developing and monitoring the quality of our AI features, including summaries, quality scoring, sales monitoring, and detection of unwanted calls (failure demand).

We need a dedicated person who can define what “good” looks like, build reliable evaluation datasets, and help our product and engineering teams improve results over time.

What you will do

• Define evaluation frameworks for AI outputs (scoring guides, acceptance criteria, and edge cases).
• Build and maintain “gold datasets” (sampling, labelling guidelines, consistency checks).
• Track performance over time (quality metrics, false positives/negatives, drift, stability by language and topic).
• Run error analysis on transcripts and metadata to identify root causes and improvement opportunities.
• Work with product and engineering to translate findings into clear improvement tasks.
• Create lightweight reporting so teams can see quality trends and progress.

What we are looking for

• Strong data analysis skills and experience building analysis datasets.
• Python for data work (pandas, notebooks, automation).
• Solid grasp of statistics and practical experiment design (sampling, confidence, A/B testing basics).
• Comfort working with text-heavy data (transcripts) and imperfect operational data.
• Clear communication skills, including writing short, structured analysis summaries for non-analysts.

Nice to have

• Experience evaluating LLM or NLP outputs (human review scoring guides, scoring consistency, inter-annotator agreement).
• Knowledge of contact centre operations, quality assurance, or compliance.
• Power BI (or similar).
• Experience working with anonymised or privacy-sensitive data.

What Feelingstream can offer

• Structured onboarding and training.
• Tallinn-based office with hybrid work.
• Focus on outcomes and delivery, not time spent online.
• Work that is directly linked to customer outcomes in large organisations.
• Opportunity to grow into product analytics and regulated AI delivery.

How to apply

Send your CV and salary expectations. If you can, include 2-3 short examples of analytical work you have done (what you measured, how you validated it, what changed as a result).

For more information, contact Head of Technology, Lauri Ilison: [email protected]. We will fill the role as soon as we find the right person.

By applying for this job you agree with our Recruitment Privacy Statement.

Apply now All openings