Generative AI/LLM Application Engineer
First seen by Alion on Aug 17, 2026.
GenAI / LLM Application Engineer
Role Summary :
We are looking for a GenAI / LLM Application Engineer to build production-grade AI assistants, retrieval systems, and workflow automation solutions. The role requires hands-on experience with multi-agent orchestration, RAG pipelines, search and retrieval infrastructure, backend APIs, and user-facing chatbot experiences.
The ideal candidate can take a business workflow, break it into AI-enabled subtasks, connect the right tools and services, and deliver reliable, measurable outcomes.
Key Responsibilities :
- Design and implement multi-agent orchestration systems that can decompose complex user requests into smaller tasks.
- Build retrieval-augmented generation pipelines for enterprise documents and knowledge bases.
- Develop and optimize search flows using vector search, semantic search, keyword search, and hybrid retrieval.
- Integrate LLMs with external tools, APIs, and enterprise systems such as ticketing, HR, or support platforms.
- Create backend services using Python and FastAPI to power AI features and APIs.
- Work with LangChain and LangGraph to implement agent workflows, tool calling, and retrieval logic.
- Build and maintain chatbot interfaces using React, Chainlit, or similar frontend frameworks.
- Manage token limits, prompt quality, context handling, and conversation history strategies.
- Evaluate response quality using human review, recall vs precision analysis, and automated evaluation methods.
- Collaborate with product, backend, and business teams to translate workflows into AI solutions.
Required Skills :
- Strong experience in Python backend development.
- Hands-on experience with FastAPI or similar API frameworks.
- Practical knowledge of LangChain and/or LangGraph.
- Experience building RAG systems with chunking, embeddings, retrieval, and prompt assembly.
- Familiarity with Elasticsearch or other vector/search databases.
- Understanding of embeddings, transformers, attention, tokenization, and generation settings such as temperature, top-p, and top-k.
- Ability to design prompt and context management strategies for longer conversations.
- Experience integrating third-party APIs and enterprise tools.
- Strong problem-solving ability and production-oriented engineering mindset.
Preferred Skills :
- Experience with MCP and tool-based agent architectures.
- Knowledge of reranking, hybrid search, BM25, and HNSW.
- Experience with Azure AI Search, Azure OpenAI, blob storage, or other Azure services.
- Experience with React or Chainlit for AI chatbot interfaces.
- Exposure to automated evaluation frameworks such as Ragas.
- Familiarity with distributed systems, event-driven workflows, or microservices.
Qualifications :
- 5+ years of software engineering experience, with at least part of that focused on AI or backend development.
- Proven ability to ship features end to end, from backend logic to user-facing experience.
- Experience working in production environments with measurable performance or quality outcomes.
Skills
AI Integration, LLM, Generative AI, Python, FastAPI, React.js, LangChain, ElasticSearch, LangGraph, Azure, Full Stack