AI DevOps Engineer

About The Role

We are seeking a highly motivated AI DevOps Engineer to join Deltek's growing AI Center of Excellence.The ideal candidate combines deep expertise in LLMOps, MLOps, and DevOps with hands-on experience in Generative AI, Large Language Models (LLMs), SLMs, and cloud computing.

The AI DevOps Engineer will partner with AI data scientists, architects, AI solutions engineers, and platform engineering teams to deliver production-grade AI services while ensuring security, scalability, governance, observability, and operational excellence. The role also contributes to AI solution development, so that operational practices are shaped by real delivery experience. This role reports to the Senior AI Solutions Architect.

Key Responsibilities

LLMOps & Generative AI DevOps

• Design and operate LLMOps pipelines for prompt management, evaluation, versioning, release, and rollback of LLM-based applications.
• Run and scale LLM gateways and model-serving layers across providers such as GPT, Claude, Gemini, Llama, and Mistral, including routing, rate limiting, caching, and failover.
• Operate Retrieval-Augmented Generation (RAG) platforms, including vector database management, embedding refresh, and index lifecycle.
• Build automated evaluation frameworks to measure quality, groundedness, hallucination rates, safety, latency, and cost per request.
• Support deployment and monitoring of AI agents and agentic workflows, including MCP and A2A integrations.
• Track token usage and spend, and optimize model selection, context size, and inference configuration for cost and performance.
  
  

MLOps & Model Lifecycle Management

• Design and implement end-to-end MLOps pipelines for model training, validation, registration, deployment, monitoring, and retirement.
• Automate model training, testing, and promotion across development, staging, and production environments.
• Manage experiment tracking, model registries, feature pipelines, and reproducible environments.
• Monitor model performance, data drift, concept drift, and operational metrics, and trigger retraining or remediation.
  
  

DevOps, Cloud & Platform Engineering

• Build and maintain CI/CD pipelines for AI applications, models, and infrastructure using Git-based workflows.
• Provision and manage cloud infrastructure on AWS (preferred), Azure, or Google Cloud using Infrastructure as Code such as Terraform, CloudFormation, or Bicep.
• Containerize and orchestrate AI workloads with Docker and Kubernetes, including GPU scheduling, autoscaling, and serverless options.
• Implement observability for AI platforms: metrics, logs, traces, dashboards, alerting, and on-call runbooks.
• Drive reliability practices, including incident response, root cause analysis, capacity planning, and disaster recovery.
• Manage AI platform cost through tagging, budgets, right-sizing, and FinOps reporting.
  
  

AI Solutions Engineering

• Collaborate with data scientists and solution teams to productionize machine learning, Generative AI, and RAG solutions.
• Develop Python-based APIs, microservices, and integrations that connect AI services with enterprise systems and data platforms.
• Contribute to prompt engineering, fine-tuning, and model customization efforts where operational input improves outcomes.
• Provide reusable templates, SDKs, and reference architectures that help teams adopt AI platform standards quickly.
  
  

AI Governance, Security & Compliance

• Ensure compliance with Responsible AI, security, privacy, and regulatory requirements across the AI lifecycle.
• Implement secure design and operation: secrets management, identity and access controls, network isolation, and data protection.
• Put guardrails in place, such as content filtering, prompt-injection defenses, PII handling, and audit logging.
  
  

As the recognized global standard for project-based businesses, Deltek delivers software and information solutions to help organizations achieve their purpose. Our market leadership stems from the work of our diverse employees who are united by a passion for learning, growing and making a difference. At Deltek, we take immense pride in creating a balanced, values-driven environment, where every employee feels included and empowered to do their best work. Our employees put our core values into action daily, creating a one-of-a-kind culture that has been recognized globally. Thanks to our incredible team, Deltek has been named one of America's Best Midsize Employers by Forbes, a Best Place to Work by Glassdoor, a Top Workplace by The Washington Post and a Best Place to Work in Asia by World HRD Congress. www.deltek.com

Required Qualifications

Education

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Engineering, or a related technical field.
  
  

Experience

• 5+ years of experience in DevOps, platform engineering, software engineering, or machine learning engineering.
• 2+ years of hands-on experience building and operating MLOps or LLMOps pipelines and platforms in production environments.
• 2+ years of experience supporting Generative AI, Agentic AI, or AI/ML solutions in production.
  
  

Technical Skills

MLOps & LLMOps

• MLflow, Kubeflow, Azure ML, SageMaker, or similar platforms for tracking, registry, and pipelines.
• LLM evaluation, prompt versioning, and observability tooling, such as LangSmith, Langfuse, or similar.
• Model serving and inference optimization, such as vLLM, KServe, Triton, or managed endpoints.
• Monitoring of model quality, data drift, hallucinations, latency, and token cost.
  
  

DevOps & Platform Engineering

• Docker and Kubernetes (EKS, AKS, or GKE), including Helm and GitOps tools such as Argo CD or Flux.
• CI/CD automation with GitHub Actions, Azure DevOps, or Jenkins.
• Infrastructure as Code: Terraform, CloudFormation, or Bicep.
• Observability stack: Prometheus, Grafana, OpenTelemetry, CloudWatch, Datadog, or similar.
• Scripting and automation in Python and Bash.
  
  

Programming & Engineering

• Strong expertise in Python; working knowledge of Java, JavaScript, or similar languages.
• Experience building APIs and microservices with FastAPI or Flask.
• Solid understanding of algorithms, data structures, APIs, and software design principles.
  
  

Artificial Intelligence & Generative AI

• Machine Learning and Deep Learning concepts and frameworks, including PyTorch, TensorFlow, and Scikit-learn.
• Large Language Models (LLMs) and SLMs, prompt engineering, and fine-tuning concepts.
• Retrieval-Augmented Generation (RAG), vector embeddings, and semantic search.
• AI agents and agentic workflows using LangChain, LlamaIndex, Semantic Kernel, MCP, and A2A.
  
  

Data & Cloud Platforms

• SQL and NoSQL databases; data pipelines, ETL, and data modeling.
• AWS (preferred), including Bedrock and SageMaker; Azure OpenAI Service; Google Vertex AI.
• Cloud security fundamentals: IAM, networking, encryption, and secrets management.
  
  

20%

Deltek is committed to the protection and promotion of your privacy. In connection with your application for employment with us at Deltek, it is necessary for us to collect, store and use information about you (Personal Data) to administer and evaluate your application. We are the controller of the Personal Data you provide us and will process any such Personal Data in accordance with applicable law and the statements contained in this

Candidate Privacy Notice

Additionally, we have not sold and do not sell Personal Data you provide to us through the job application process.

Important: Protect Yourself from Recruitment Scams

Bad actors or scammers may try to impersonate Deltek and send fake job offers to people. Messages from Deltek about employment opportunities will be from an @deltek.com account or [HIDDEN TEXT], never from free services like Gmail or Yahoo. Please look carefully at the email address that provides any job offer, as some fake accounts are created to look like a legitimate domain name or email address. We will also never ask you to pay money at any point in the hiring process, whether for training, equipment, background checks, or anything else. If you receive a suspicious offer claiming to be from Deltek, do not share personal or financial information. Report any suspicious communication to [HIDDEN TEXT] and consider reporting it to law enforcement.