AI Engineer LLM Systems
AI Engineer - LLM Systems // Evals @ Ventora
Location: Remote
Employment Type: Full-time
Level: Mid-level / Senior
About Ventora
Ventora is an AI platform that helps people turn ideas into real products - from business planning and MVP creation to payments, marketing, and market validation.
The core platform is already built and in use. We’re now entering a stage of active growth and are looking for an AI-first engineer to help us make our AI systems smarter, more reliable, and easier to improve.
This is a hands-on product engineering role with real ownership. You’ll build LLM-powered features, improve AI pipelines, and create practical evaluation systems that show us what works and what needs to be fixed.
Your Mission
Help Ventora ship better AI systems - and make their quality measurable.
You’ll work across the product’s AI pipeline: business plan generation, app building, code generation, review, and marketing.
You’ll:
- Build and improve LLM-powered product features
- Develop multi-stage AI agents and workflows
- Integrate and test new models
- Improve prompts, context, routing, and fallbacks
- Create automated evals and regression tests
- Measure quality, cost, speed, and reliability
- Turn evaluation findings into product improvements
- Contribute to regular product engineering when needed
This is not a research-only or eval-only position. You’ll ship production code and see your work directly affect the product and its users.
What You’ll Work On
AI Systems & Product Features
Build and improve AI-powered features across Ventora’s product journey.
This may include multi-stage workflows such as plan → build → review, model integrations, structured outputs, tool calling, streaming, caching, prompt improvements, and new AI agents.
You’ll also contribute to production features, bug fixes, and pipeline development in the main codebase.
Evals & Quality
Help us understand whether each product change actually makes the AI better.
You’ll create practical evaluation systems for generated applications, business plans, marketing content, AI reliability, and release regressions. Depending on the task, you may use automated checks, browser tests, LLM judges, human-reviewed datasets, and side-by-side comparisons. When something fails, you’ll help find the cause, improve the prompt or pipeline, ship the fix, and add a regression test so the same issue does not return.
The goal is simple: catch problems before users do and make product decisions based on evidence.
What Success Looks Like
- New AI features and improvements reach production regularly
- AI quality becomes visible and measurable
- Regressions are caught before release
- Generated products become more functional and reliable
- Evaluation findings consistently turn into shipped fixes
- Model quality, cost, and latency stay under control
Over time, you’ll become a key technical partner for the founders, product team, and engineers shaping Ventora’s AI platform.
What You Bring
Required:
- 3+ years of experience in AI/ML, applied data science, or backend engineering
- Hands-on experience building production applications with LLMs
- Experience with model APIs such as OpenAI, Anthropic, or OpenRouter
- Understanding of prompts, agents, tool calling, structured outputs, and multi-stage AI workflows
- Ability to turn unclear questions like “Is this version better?” into practical, measurable tests
- Strong ownership, product thinking, and clear communication
- Regular use of AI coding tools such as Codex, Claude Code, Cursor, or similar
- Comfortable working in Linux environments and with Docker
You don’t need experience with every framework or evaluation platform. We care more about strong engineering fundamentals, curiosity, and the ability to l"}
Вакансии
Специалисты
Кандидатам
Работодателям
Вакансии
Специалисты
Кандидатам
Работодателям
menu
AI Engineer LLM Systems
Прямой работодатель Atlantix( www.atlantix.cc )
Миддл
- Сеньор
Информационные технологии
- Разработка
- SaaS/PaaS
11 августа
Удаленная работа
Опыт работы любой
Работодатель Atlantix
Короткая ссылка: geekjob.ru/hidX
Откликнуться
Описание вакансии
AI Engineer - LLM Systems // Evals @ Ventora
Location: Remote
Employment Type: Full-time
Level: Mid-level / Senior
About Ventora
Ventora is an AI platform that helps people turn ideas into real products - from business planning and MVP creation to payments, marketing, and market validation.
The core platform is already built and in use. We're now entering a stage of active growth and are looking for an AI-first engineer to help us make our AI systems smarter, more reliable, and easier to improve.
This is a hands-on product engineering role with real ownership. You'll build LLM-powered features, improve AI pipelines, and create practical evaluation systems that show us what works and what needs to be fixed.
Your Mission
Help Ventora ship better AI systems - and make their quality measurable.
You'll work across the product's AI pipeline: business plan generation, app building, code generation, review, and marketing.
You'll:
- Build and improve LLM-powered product features
- Develop multi-stage AI agents and workflows
- Integrate and test new models
- Improve prompts, context, routing, and fallbacks
- Create automated evals and regression tests
- Measure quality, cost, speed, and reliability
- Turn evaluation findings into product improvements
- Contribute to regular product engineering when needed
This is not a research-only or eval-only position. You'll ship production code and see your work directly affect the product and its users.
What You'll Work On
AI Systems & Product Features
Build and improve AI-powered features across Ventora's product journey.
This may include multi-stage workflows such as plan → build → review, model integrations, structured outputs, tool calling, streaming, caching, prompt improvements, and new AI agents.
You'll also contribute to production features, bug fixes, and pipeline development in the main codebase.
Evals & Quality
Help us understand whether each product change actually makes the AI better.
You'll create practical evaluation systems for generated applications, business plans, marketing content, AI reliability, and release regressions. Depending on the task, you may use automated checks, browser tests, LLM judges, human-reviewed datasets, and side-by-side comparisons. When something fails, you'll help find the cause, improve the prompt or pipeline, ship the fix, and add a regression test so the same issue does not return.
The goal is simple: catch problems before users do and make product decisions based on evidence.
What Success Looks Like
- New AI features and improvements reach production regularly
- AI quality becomes visible and measurable
- Regressions are caught before release
- Generated products become more functional and reliable
- Evaluation findings consistently turn into shipped fixes
- Model quality, cost, and latency stay under control
Over time, you'll become a key technical partner for the founders, product team, and engineers shaping Ventora's AI platform.
What You Bring
Required:
- 3+ years of experience in AI/ML, applied data science, or backend engineering
- Hands-on experience building production applications with LLMs
- Experience with model APIs such as OpenAI, Anthropic, or OpenRouter
- Understanding of prompts, agents, tool calling, structured outputs, and multi-stage AI workflows
- Ability to turn unclear questions like "Is this version better?" into practical, measurable tests
- Strong ownership, product thinking, and clear communication
- Regular use of AI coding tools such as Codex, Claude Code, Cursor, or similar
- Comfortable working in Linux environments and with Docker
You don't need experience with every framework or evaluation platform. We care more about strong engineering fundamentals, curiosity, and the ability to learn and ship.
Nice to Have
- Experience with LLM evals, datasets, scorers, or CI quality gates
- TypeScript or Node.js experience
- Browser automation, SQL, or production monitoring experience
- Familiarity with tools such as promptfoo, DeepEval, Ragas, Braintrust, LangSmith, or Langfuse
- Experience with agent frameworks, RAG evaluation, vision models, or AI red-teaming
- A background in mathematics, statistics, physics, machine learning, or another quantitative field
- Early-stage startup or side-project experience
Why Join Ventora
- Build real AI systems used in production
- Influence both the product and its technical foundation
- Own projects from idea to launch and measurement
- Work directly with founders, product, and engineering
- Experiment with new models and AI development workflows
- Help define quality standards for an AI-native platform
- Grow into a leading role as the product and team expand
If you enjoy building with LLMs, care about whether AI systems actually work, and want your work to have a direct product impact, we'd be glad to talk.
Join us and help build the next stage of Ventora!
Специализация
Информационные технологии
Разработка
Отрасль и сфера применения
SaaS/PaaS
Уровень должности
Миддл
Сеньор
Откликнуться на вакансию
Быстрый отклик и регистрация/авторизацияИли быстрая регистрация/авторизация через OAuth
* Продолжая регистрацию/авторизацию вы подтверждаете свое согласие на хранение и обработку своих персональных данныхв соответствии с Федеральным законом № 152-ФЗ "О персональных данных", а так же подтверждаете, что прочитали и принимаете Пользовательское соглашение и согласие на обработку персональных данных, Условия использования сайта, Политику Конфиденциальности иРазъяснение использования личных данных,и согласны получать информацию и рассылкиот Geekjob через телекомуникационные средства личной связи.
Вакансии от"Atlantix"
Еще интересные вакансии
Москва
RU
AI Product Manager в RUFP
Rufp.ru
office
10 августа
AR
Lead Product Analyst / Analytics Engineer - AI & Experimentation
Artworkout.app
remote
7 августа
LE
E-mail/SMM Marketing Manager
Lev Haolam
remote
6 августа
LE
CRM Marketing Manager (Email Social)
Lev Haolam
remote
6 августа
London
500K - 1000K ₽
CTO / Head of Engineering
Kanzum
relocateremote
5 августа
Junior/Middle Full-Stack Developer (AI-First)
Insinet
remote
3 августа
Junior+/Middle QA Engineer
Starplay Games
remote
3 августа
500 $
Junior+ QA Engineer
ITDEAL GROUP
remote
3 августа
Новосибирск
TH
Senior CV Engineer / Team Lead
The One (агентство)
office
31 июля
Москва
150K ₽
PL
DevOps Engineer
PlaysDev
remote
31 июля
Еще...
Эбаут
Ви ар зе дрим тим оф крэзи спешиалистс энд рекрутерс.Ви лов программерс энд ви олвэйс хелп ту файнд зе дрим ворк фор гуд пипл.
Доказательный рекрутинг
NEWHR Medium Blog
Исследования рынка труда
Дайджесты вакансий
Статьи о карьере в IT
NewHR Podcast
Блог нашего FullStack CTO
Контакты
Обратная связь
Условия использования
Конфиденциальность
Услуги и цены
Партнеры
Сейчас на сайте 28482 кандидатов и более 10000 компаний.В среднем 16 откликов на вакансию
Ⓒ Geekjob
Мы используем куки, потому что без кук наш сайт не работал бы, другие сайты не работали бы, да и вообще весьинтернет не работал бы
window.Vacancy = {"id":"6a7af7879319f6a4ef049fba","ci":"661cd76bb72af3b82a0a80db","ic":"661cd5b901ba39cb8504f155","lang":"en","position":"AI Engineer LLM Systems