AI Ops Engineer

The core responsibilities for the job include the following:

Multi-Cloud Infrastructure Architecture:

• Design production-grade infrastructure across AWS, GCP, and Azure.
• Architect private, low-latency interconnects between clouds: AWS Direct Connect, GCP Cloud Interconnect, Azure ExpressRoute, Dedicated cross-cloud networking solutions.
• Deploy multi-region infrastructure for HA and DR.
• Implement IaC (Terraform, Pulumi, CloudFormation) across all clouds.

AI/ML Services & API Integration:

• Deploy and optimize Google Gemini APIs, Vertex AI APIs, and Bedrock APIs.
• Implement ASR/STT services from Deepgram, Google Cloud Speech-to-Text, Azure Speech Services, Whisper, Baseten, More.
• Configure TTS services: Google Cloud TTS, Azure Speech, and ElevenLabs.
• Implement model serving infrastructure for fine-tuned models.

Security & Network Engineering:

• Design Zero Trust network architectures across multi-cloud.
• Configure VPCs, VNets, security groups, NACLs, and firewall rules.
• Implement private endpoints and PrivateLink configurations.
• Set up VPN tunnels, peering connections, and transit gateways.
• Implement secrets management, encryption, and key rotation.
• Maintain compliance: SOC 2 ISO 27001 and ISO/IEC 42001 if not practical, theoretical understanding of AI-regulated compliances like ISO/IEC 42001:2023 and ISO/IEC 27001 is a must.

Compute & Container Orchestration:

• Create and manage VMs, instance groups, and auto-scaling.
• Deploy Kubernetes clusters (EKS, GKE, AKS).
• Implement GPU compute infrastructure, NVIDIA A100 H100 and TPUs.
• Optimize compute costs while meeting performance SLAs.

Performance & Reliability:

• Design for sub-100ms latency in voice AI pipelines.
• Implement monitoring and observability: Datadog, Grafana, CloudWatch, and Cloud Monitoring.
• Build automated incident response and self-healing infrastructure.
• Conduct performance testing, load testing, and capacity planning.

Requirements:

• 6+ years of hands-on cloud infrastructure experience
• 3+ years working across multiple cloud providers simultaneously
• Proven track record with production AI/ML workloads
• Deep expertise in at least 2 of AWS, GCP, and Azure
• Experience with real-time voice/audio systems

:

Technical Skills Must-Have:

• VM Management: AWS EC2 GCP Compute Engine, and Azure VMs, creation, configuration, hardening, and lifecycle management.
• Advanced Networking: VPCs, subnets, route tables, NAT gateways, Load Balancers (ALB, NLB, Cloud Load Balancing, Azure LB), DNS (Route 53 Cloud DNS, Azure DNS), Private Connectivity, VPN tunnels, Direct Connect / Cloud Interconnect / ExpressRoute, PrivateLink / Private Service Connect.
• Cross-Cloud Networking: Transit gateways, hub-spoke architectures, multi-cloud mesh.
• Container Orchestration: Kubernetes (EKS, GKE, AKS), Docker, Helm, Service Mesh (Istio, Linkerd).
• Infrastructure as Code: Terraform (required), CloudFormation, Pulumi, and ARM templates.
• Security: IAM, RBAC, security groups, NACLs, WAF, DDoS protection, secrets management (Vault, Secrets Manager).
• CI/CD: GitHub Actions, GitLab CI, Cloud Build, CodePipeline, Security scanning integration.

AI/ML Infrastructure Must Have:

• Google Gemini APIs / Vertex AI or equivalent LLM platforms.
• Production STT deployment (Deepgram, Google Speech, Azure Speech, Whisper).
• Production TTS deployment (Google TTS, Azure TTS, ElevenLabs).
• Model serving patterns, GPU allocation, and inference optimization.
• Real-time streaming protocols (WebRTC, WebSocket, gRPC).

Nice to Have:

• LiveKit, Twilio, or similar real-time communication platforms.
• Telephony/VoIP background (SIP trunking, PSTN integration).
• MLOps: model versioning, A/B testing, and canary deployments.
• FinOps: cost optimization, reserved/spot instances.

Certifications:

• AWS Solutions Architect Professional.
• GCP Professional Cloud Architect.
• Azure Solutions Architect Expert.
• AI governance frameworks (ISO/IEC 42001:2023).