Production Engineer - AI Storage & HPC Infrastructure

Job Description

• Skill Set :

  Python,Kubernetes,HPC

• Total Experience :

  5.00 to 15.00 Years

• No of Openings :

  1

• Job Post Date :

  03/11/2026

• Job Expiry Date :

  30/11/2026

• Domain :

  IT

• Location :

  SAN JOSE [United States]

• Job Reference No :

  4116320

Job Summary

Role: Production Engineer - AI Storage & HPC Infrastructure

Location: United States, Pacific time zone preferred

Duration: Fulltime

We are seeking a highly skilled Production Engineer - AI Storage & HPC Infrastructure to support and optimize next-generation AI and High-Performance Computing (HPC) environments.

In this role, you will help build, operate, and scale high-performance storage platforms that support large-scale machine learning, AI training, advanced analytics, and distributed computing workloads. The ideal candidate will combine expertise in storage systems, Linux, automation, performance engineering, and infrastructure operations to ensure reliable and efficient access to critical datasets and model artifacts across enterprise-scale environments.

This is a hands-on engineering role involving storage architecture, performance optimization, automation, observability, and production support.

Key Responsibilities

• Design, deploy, and support distributed file systems and object storage platforms used by large-scale AI, analytics, and HPC workloads.

• Manage storage environments supporting shared research, analytics, model training datasets, and checkpoint storage requirements.

• Troubleshoot and optimize storage performance across metadata services, throughput, latency, caching, networking, and data-access patterns.

• Build and maintain automation for provisioning, capacity expansion, upgrades, recovery processes, and lifecycle management.

• Manage data movement, replication, migration, and synchronization across multiple clusters, regions, and cloud environments while maintaining security and data integrity.

• Develop tooling and automation to improve storage operations, scalability, monitoring, and reliability.

• Partner with platform engineering, infrastructure, security, and development teams to ensure highly available and resilient storage services.

• Participate in on-call support, incident response, root cause analysis, and continuous service improvement initiatives.

• Implement observability, alerting, health monitoring, testing strategies, and controlled production rollouts.

Required Skills & Experience

• Strong experience designing, operating, or supporting large-scale distributed storage platforms, high-performance file systems, or enterprise data infrastructure.

• Deep hands-on Linux administration and troubleshooting experience.

• Strong understanding of file systems, storage protocols, networking, concurrency, and performance optimization.

• Experience diagnosing production issues related to storage performance, data access, throughput bottlenecks, and system reliability.

• Hands-on programming or automation experience using one or more of the following:

  • Python

  • Go

  • C++

  • Rust

• Experience driving end-to-end reliability improvements, performance optimization initiatives, or large-scale operational enhancements.

• Strong debugging, analytical, and problem-solving skills.

• Experience working in highly available production environments with operational ownership responsibilities.

Preferred Qualifications

• Experience with Kubernetes, container orchestration platforms, and cloud-native infrastructure.

• Strong understanding of distributed filesystem concepts including metadata management, caching, consistency, failover, recovery, and high availability.

• Hands-on experience with one or more enterprise storage technologies such as:

  • Lustre

  • ZFS / OpenZFS

  • VAST Data

  • Pure Storage

  • Hammerspace

  • Similar large-scale distributed storage platforms

• Experience supporting AI, Machine Learning, Data Engineering, or HPC infrastructures.

• Understanding of high-throughput data ingestion, checkpoint management, data staging, and accelerator-cluster storage requirements.

• Experience with Infrastructure as Code (IaC) and automation tools including:

  • Terraform

  • Helm

  • Kubernetes Operators

• Experience implementing observability and monitoring solutions using metrics, logging, tracing, benchmarking, and capacity-planning tools.

• Familiarity with object storage technologies, S3-compatible storage, lifecycle management, tiering, replication, and cross-region data movement.

Preferred Profile

• 5+ years of experience in Storage Engineering, Platform Engineering, Site Reliability Engineering (SRE), Infrastructure Engineering, HPC Operations, or Cloud Infrastructure.

• Experience in enterprise-scale production environments supporting mission-critical workloads.

• Strong communication skills and ability to collaborate with globally distributed engineering teams.

• Self-driven engineer with a passion for automation, reliability, scalability, and operational excellence.

The pay range for this role is USD $140,000- $160,000 per annum including any bonuses or variable pay. Tech Mahindra also offers benefits like medical, vision, dental, life, disability insurance and paid time off (including holidays, parental leave, and sick leave, as required by law). Ask our recruiters for more details on our Benefits package. The exact offer terms will depend on the skill level, educational qualifications, experience and location of the candidate.

AI tools may assist in the recruitment process; however, all hiring decisions are made by the recruitment team based on a comprehensive evaluation of candidates.

“Tech Mahindra is an Equal Employment Opportunity employer. We promote and support a diverse workforce at all levels of the company. All qualified applicants will receive consideration for employment without regard to race, religion, color, sex, age, national origin, or disability. All applicants will be evaluated solely on the basis of their ability, competence, and performance of the essential functions of their positions with or without reasonable accommodations. Reasonable accommodations also are available in the hiring process for applicants with disabilities. Candidates can request a reasonable accommodation by contacting the company ADA Coordinator at [email protected].”