Senior Technical Architect

Job Summary

LLMOps Engineer

Role Summary

The LLMOps Engineer is responsible for operationalizing, scaling, monitoring, and governing Large Language Model (LLM)-based systems in production. This role focuses on ensuring LLM applications are reliable, performant, cost-efficient, observable, and secure across their lifecycle-from deployment to continuous optimization

Key Responsibilities

Key Responsibilities

• Deploy and manage LLM-based applications across cloud and hybrid environments.
• Build and maintain production-grade inference pipelines for LLM workloads.
• Design and implement LLMOps pipelines for model, prompt, and configuration lifecycle management.
• Implement monitoring for latency, throughput, cost, errors, drift, and hallucinations.
• Build dashboards and alerts to ensure SLA and reliability targets.
• Optimize inference cost and performance using caching, batching, routing, and autoscaling strategies.
• Enforce security, access control, audit logging, and governance controls for LLM usage.
• Collaborate with Responsible AI teams to operationalize safety and compliance guardrails.
• Enable AI Engineers and Fullstack teams through reusable LLMOps frameworks and platforms.

Skill Requirements

Required Skills & Experience

• Strong understanding of LLMs, Generative AI architectures, and inference workflows.
• Hands-on experience running LLM-based systems in production environments.
• Strong experience with cloud platforms (Azure, AWS, or GCP).
• Hands-on experience with Docker, Kubernetes, and container orchestration.
• Experience with CI/CD for ML or AI applications.
• Programming and automation skills in Python or similar languages.
• Familiarity with MLOps / LLMOps tooling, monitoring, and logging systems.

Experience Level

• 6-10 years of overall experience in DevOps, Platform, or ML/AI Engineering roles.
• 2-4+ years of experience operating ML or GenAI systems in production.

Success Measures

• Stable, scalable, and cost-efficient LLM systems running in production.
• Reduced production incidents and faster recovery for AI services.
• Faster and safer enterprise adoption of GenAI solutions enabled through robust LLMOps practices.

Other Requirements

Good Communication Skills