Lead DevOps/MLOps Engineer

We are looking for a Lead DevOps/MLOps Engineer to join our Data Science Department and work with the ML Engineering team to build out and harden our on-premises, GPU-backed ML platform. In this role, you will help enable multiple ML and LLM use cases to run simultaneously, reliably, and with low latency, while scaling the platform to support a wider user base across the organization.

Unlock the potential of remote work in Kazakhstan, giving you the flexibility to work from home or access our offices in Astana, Almaty or Karaganda.

Responsibilities

• Build and harden the on-premises GPU-backed ML platform to support concurrent ML and LLM workloads

• Troubleshoot and resolve daily resource bottlenecks and latency issues across competing production and development use cases

• Deploy and maintain the NVIDIA GPU Operator and manage GPU resources across Kubernetes and OpenShift clusters

• Implement GPU sharing and isolation strategies to optimize memory and fault isolation across workloads

• Scale the GPU environment to accommodate growing compute demands as new H200 servers are added

• Maintain and improve platform services such as MLflow and the Agent Gateway

• Manage monitoring pipelines using NVIDIA DCGM exporter, Prometheus and Grafana to track GPU utilization and performance

• Collaborate with the Deep Learning and Traditional ML teams to support shared infrastructure, including Ray-based training workloads

• Ensure platform reliability and security given the confidential and sensitive nature of data handled on-premises

• Partner with the Data Analytics team to support the in-house agentic platform as it scales to a broader employee base

Requirements

• 5+ years of experience running GPU workloads on Kubernetes or OpenShift in production, ideally on-premises

• Practical experience deploying and troubleshooting the NVIDIA GPU Operator, with clear understanding of each component's function

• Understanding of how GPUs differ from CPUs as a scheduled resource, including memory limits, utilization semantics and scheduling limitations

• Knowledge of GPU sharing and isolation methods such as MIG, time-slicing and MPS, along with their trade-offs

• Solid Linux fundamentals, including cgroups, namespaces, systemd resource controls and process-level resource management

• Experience with Prometheus, Grafana and NVIDIA DCGM, including GPU metrics at process or pod level

• Working knowledge of the CUDA stack, including drivers, CUDA toolkit and container runtime version compatibility

• Experience deploying with Helm and operating MLOps tools such as MLflow and Airflow on Kubernetes

Nice to have

• Familiarity with LLM serving engines such as vLLM, SGLang or NVIDIA Triton

• Experience with Ray, KubeRay and GPU-aware schedulers such as Kueue or Volcano

• Hands-on work with H100/H200-class hardware and GitLab CI

We offer/Benefits

We connect like-minded people:

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth:

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers
• Discounts in local language schools, including online courses for the Kazakh language

We cover it all:

• Participation in the Employee Stock Purchase Plan
• Monetary bonuses for engaging in the referral program
• Medical & family care package
• Six trust days per year (sick leave without a medical certificate)
• Coverage of psychology sessions of your choice
• Benefits package (sports activities, a variety of stores and services)
• Housing support program: preferential mortgage access via Otbasy Bank partnership

EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.