{"id":3418916,"url":"https://alion.io/job/teradata-ai-infrastructure-engineer-generative-ai-model-serving-go-rust-gpu","title":"AI Infrastructure Engineer Generative AI Model Serving (Go / Rust / GPU)","company":{"id":58848,"name":"Teradata","domain":"teradata.com","url":"https://alion.io/company/teradata","size_band":null,"is_staffing_agency":false,"employer_type":"direct","is_intermediary":false,"listed_via":null,"ats_vendor":null,"truth_index":null},"role":"DevOps","role_family":"DevOps","seniority":"middle","employment_type":"full_time","work_mode":"on_site","remote_scope":null,"remote_scope_basis":null,"remote_working_hours":null,"hiring_geo_confidence":"structured","locations":["Hyderabad, India"],"countries":["IN"],"hiring_countries":[],"hiring_countries_total":0,"salary":null,"salary_estimate":{"min_usd":11000,"max_usd":31000,"period":"year","method":"role_seniority_country_remote_cell","sample_n":30},"experience_years_min":3,"visa_sponsorship":false,"relocation_package":false,"has_equity":false,"technologies":[{"name":"AI Agents","optional":false},{"name":"ArgoCD","optional":false},{"name":"AWQ","optional":false},{"name":"CI/CD","optional":false},{"name":"CUDA","optional":false},{"name":"Databricks","optional":false},{"name":"Docker","optional":false},{"name":"Embeddings","optional":false},{"name":"GitHub Actions","optional":false},{"name":"Go","optional":false},{"name":"GPTQ","optional":false},{"name":"Grafana","optional":false},{"name":"gRPC","optional":false},{"name":"Hallucination","optional":false},{"name":"Helm","optional":false},{"name":"Hugging Face","optional":false},{"name":"Jenkins","optional":false},{"name":"KServe","optional":false},{"name":"Kubernetes","optional":false},{"name":"KV Cache","optional":false},{"name":"llama.cpp","optional":false},{"name":"LLM","optional":false},{"name":"ONNX","optional":false},{"name":"ONNX Runtime","optional":false},{"name":"OpenAI","optional":false},{"name":"OpenTelemetry","optional":false},{"name":"Prometheus","optional":false},{"name":"Python","optional":false},{"name":"PyTorch","optional":false},{"name":"Quantization","optional":false},{"name":"RAG","optional":false},{"name":"Ray","optional":false},{"name":"Ray Serve","optional":false},{"name":"Reranking","optional":false},{"name":"Rest API","optional":false},{"name":"Rust","optional":false},{"name":"SGLang","optional":false},{"name":"SLI/SLO/SLA","optional":false},{"name":"Snowflake","optional":false},{"name":"Speculative Decoding","optional":false},{"name":"TensorRT-LLM","optional":false},{"name":"Teradata","optional":false},{"name":"Terraform","optional":false},{"name":"Triton","optional":false},{"name":"Triton Inference Server","optional":false},{"name":"vLLM","optional":false},{"name":"CUTLASS","optional":true},{"name":"Edge AI","optional":true},{"name":"InfiniBand","optional":true},{"name":"LangChain","optional":true},{"name":"LiteRT","optional":true},{"name":"LlamaIndex","optional":true},{"name":"LLM Evaluation","optional":true},{"name":"NCCL","optional":true},{"name":"SLURM","optional":true},{"name":"TensorFlow","optional":true},{"name":"TensorRT","optional":true},{"name":"TFLite","optional":true}],"status":"live","first_seen_at":"2026-10-28T17:33:36Z","employer_posted_date":null,"last_verified_at":"2026-10-28T17:33:36Z","board_verified":false,"closed_at":null,"days_open":2,"trust":{"level":"not_scored","repost_count":null,"flags":[],"days_open":2},"description":"Our Company:\n\nAt Teradata, we believe that people thrive when empowered with better information. Teradata Autonomous Knowledge Platform activates enterprise intelligence by unifying data, knowledge and business context to achieve tangible outcomes. With Teradata, organizations can provide agents with full context for impact when it matters. Our solution lets businesses connect and scale on premises, in the cloud, or through a hybrid approach. Teradata delivers real business value with AI. \n\nPower the Platform Behind Enterprise Generative AI\n\nAt Teradata, we're not just managing data; we're unleashing its full potential. Our ClearScape Analytics™ platform and Enterprise Vector Store help the world's largest enterprises derive value from their most complex data. Generative AI and agentic systems are the next frontier, and they run on infrastructure that has to be fast, efficient, and dependable at scale. \n\nAs an AI Infrastructure Engineer, you will build the serving layer that runs large language models and other generative models inside the Teradata Vantage ecosystem. You will work on GPU-accelerated inference, high-performance Go and Rust services, and the cloud-native platform that lets teams ship models to production with confidence. You will own systems end to end, from a CUDA kernel profile to a Kubernetes rollout. \n\nWhat You'll Do\n\nDesign, build, and operate scalable, low-latency model-serving platforms for LLMs and generative models, covering batching, streaming, routing, autoscaling, and multi-tenant isolation. \nDevelop high-performance control-plane and data-plane services in Go and performance-critical components (tokenizers, schedulers, runtimes, gRPC/HTTP gateways, native extensions) in Rust. \nOptimize GPU inference for throughput, latency, and cost using continuous batching, KV-cache management, quantization (INT8/FP8/AWQ/GPTQ), speculative decoding, tensor/pipeline parallelism, and efficient memory use. \nIntegrate and extend inference engines such as vLLM, TensorRT-LLM, Triton Inference Server, Ray Serve, KServe, SGLang, llama. cpp, and ONNX Runtime; contribute fixes upstream where it makes sense. \nBuild and manage GPU cluster infrastructure on Kubernetes (NVIDIA GPU Operator, device plugins, MIG/time-slicing, node pools, scheduling and bin-packing) across cloud and on-prem environments. \nOwn the full delivery path: CI/CD pipelines, model packaging and registries, canary and blue/green rollouts, infrastructure as code, and automated performance regression testing. \nBuild AI observability into everything: metrics, logs, and traces for token throughput, time-to-first-token, GPU utilization, queue depth, error and hallucination signals, using OpenTelemetry, Prometheus, and Grafana. \nProvide clean APIs, SDKs, and developer tooling (OpenAI-compatible endpoints, internal services, and lightweight UIs or dashboards) so AI engineers and agent builders can deploy and monitor models self-serve. \nSupport agentic and RAG workloads by serving embedding, reranking, and generation models alongside the Enterprise Vector Store, with predictable latency and graceful degradation. \nDrive reliability, security, and cost efficiency: SLOs, load and chaos testing, capacity planning, GPU cost attribution, and secure model and tenant isolation. \n\nWho You'll Work With\n\nYou will collaborate daily with AI researchers, ML engineers, data scientists, and platform and SRE teams who champion diversity, equity, and inclusion as fundamental to our success. You'll be part of a cohesive team delivering high-quality, highly visible AI/ML functionality within the Teradata Vantage platform. You will report to the Sr. Manager, Software Engineering, who will champion your growth and empower your contributions. \n\nWhat Makes You a Qualified Candidate\n\n3+ years of professional software engineering experience in backend systems, distributed systems, or AI/ML infrastructure. \nStrong, production-level proficiency in Go (concurrency, gRPC, profiling, service design) and working proficiency, or a demonstrated passion for learning, Rust (ownership, async runtimes such as Tokio, FFI, performance tuning). \nHands-on experience with GPU computing for ML: CUDA fundamentals, GPU profiling (Nsight, nvidia-smi/DCGM), memory and bandwidth tradeoffs, and multi-GPU or multi-node inference. \nExperience serving LLMs or generative models in production, with practical knowledge of inference optimization techniques and the serving frameworks listed above. \nSolid understanding of transformer architectures and the LLM inference lifecycle (prefill vs. decode, attention, KV cache, sampling), plus familiarity with PyTorch and Hugging Face tooling. \nStrong system design skills and a deep understanding of distributed systems, networking, concurrency, and failure modes. \nExperience with containers and orchestration (Docker, Kubernetes, Helm) and CI/CD (GitHub Actions, Jenkins, Argo CD, or similar), plus infrastructure as code (Terraform). \nFull-stack engineering experience: comfortable building REST/gRPC APIs and a pragmatic front end (React or similar) for dashboards, playgrounds, or admin tools; Python for glue, tooling, and ML integration. \nExperience with observability practices and tools (OpenTelemetry, Prometheus, Grafana, tracing) applied to ML or LLM workloads. \nYou are obsessive about reliability, debuggability, and ensuring AI systems behave predictably and deterministically when needed. \nYou code daily, own systems end to end, and are knowledgeable about open-source tools and know how to leverage and extend them. \n\nWhat You Bring\n\nA Bachelor's or Master's degree in Computer Science, Engineering, or a related field, or equivalent practical experience. \nGenuine excitement for AI, LLMs, and the systems that make them efficient and affordable to run at scale. \nStrong communication skills and a bias toward measuring, benchmarking, and iterating. \nFamiliarity with modern data platforms such as Teradata, Snowflake, or Databricks is a plus. \n\nBonus Points\n\nExperience writing or tuning CUDA / Triton (OpenAI) / CUTLASS kernels, or contributing to inference engines. \nExperience with on-device or edge ML inference (ONNX, TFLite, llama. cpp, quantized small language models), embedded or mobile deployment. \nKnowledge of RAG pipelines, vector databases, embeddings, and agent frameworks (LangChain, LlamaIndex, ReAct-style agents). \nExperience with model monitoring, drift detection, evaluation harnesses, and responsible AI practices. \nOpen-source contributions, technical writing, or public projects in ML systems, Go, or Rust. \nExperience with service meshes, RDMA/InfiniBand/NCCL, Slurm, or Ray clusters.\n\nWhy We Think You'll Love Teradata\n\nWe prioritize a people-first culture because we know our people are at the very heart of our success. We embrace a flexible work model because we trust our people to make decisions about how, when, and where they work. We focus on well-being because we care about our people and their ability to thrive both personally and professionally. We are an anti-racist company because our dedication to Diversity, Equity, and Inclusion is more than a statement. It is a deep commitment to doing the work to foster an equitable environment that celebrates people for all of who they are.\n\nTeradata invites all identities and backgrounds in the workplace. We work with deliberation and intent to ensure we are cultivating collaboration and inclusivity across our global organization. We are proud to be an equal opportunity and affirmative action employer. We do not discriminate based upon race, color, ancestry, religion, creed, sex (including pregnancy, childbirth, breastfeeding, or related conditions), national origin, sexual orientation, age, citizenship, marital status, disability, medical condition, genetic information, gender identity or expression, military and veteran status, or any other legally protected status.","description_format":"text","description_chars":7918,"description_truncated":false,"requirements":{"experience_years_min":3,"management_years_min":null,"team_size_min":null,"manages_managers":false,"education":null,"security_clearance":false,"languages":[]},"benefits":["Equity","Flexible schedule"],"hiring_locations":[],"hiring_excludes":[],"relocation_offered":false,"industries":["Data Warehouses & Lakehouses"],"lifecycle":[{"event":"open","at":"2026-10-28T20:09:00Z"}],"visa":[],"liveness":{"score":90,"band":"hot","label":"Hiring now","p_open":1,"p_active":0.903,"p_room":1,"age_days":2,"expected_fill_days":24,"reasons":["seen:2","velocity","win:early"],"computed_at":"2026-10-31T05:58:55Z"},"pay":null,"html_url":"https://alion.io/job/teradata-ai-infrastructure-engineer-generative-ai-model-serving-go-rust-gpu","json_url":"https://alion.io/job/teradata-ai-infrastructure-engineer-generative-ai-model-serving-go-rust-gpu.json","meta":{"generated_at":"2026-10-31T14:00:23Z","cache_seconds":300,"methodology":"https://alion.io/methodology","terms":"https://alion.io/terms","contact":"https://alion.io/contact","api":"https://alion.io/developers","about":"Alion is a live layer of people, companies and AI agents: who they are, whether they are real and active right now, what they do and how to work with them, readable by people and by agents and paid per call.","catalog":"https://alion.io/catalog.json","usage":{"tier":"crawler_verified","counted_by":"address","units_charged":1,"used_today":21746,"day_limit":null,"remaining_today":null,"minute_limit":300,"resets_at":"2026-11-01T00:00:00Z"}},"offers":[{"id":"company.slices","title":"One company in depth, by slice","status":"live","price":{"credits":0.02,"usd":0.002,"plus_per_slice":{"credits":0.05,"usd":0.005}},"unit":"per company, plus each slice with data","note":"the employer in depth","call":{"mcp_tool":"get_company","arguments":{"id":58848},"rest":"https://alion.io/mcp/rest/get_company?id=58848"},"human":"https://alion.io/catalog?offer=company.slices&for=job%2Fteradata-ai-infrastructure-engineer-generative-ai-model-serving-go-rust-gpu"},{"id":"market.stats","title":"A market slice: pay, demand and time to fill","status":"live","price":{"credits":1,"usd":0.1},"unit":"per slice","note":"pay, demand and time to fill for this role and place","call":{"mcp_tool":"market_stats"},"human":"https://alion.io/catalog?offer=market.stats&for=job%2Fteradata-ai-infrastructure-engineer-generative-ai-model-serving-go-rust-gpu"},{"id":"job.search","title":"Open jobs by role, technology, place, pay and visa","status":"live","price":{"credits":0.02,"usd":0.002},"unit":"per posting in a list","note":"similar open postings","call":{"mcp_tool":"search_jobs"},"human":"https://alion.io/catalog?offer=job.search&for=job%2Fteradata-ai-infrastructure-engineer-generative-ai-model-serving-go-rust-gpu"},{"id":"company.verify","title":"Is this company real and active right now","status":"pilot","price":null,"unit":"per company","request":{"url":"https://alion.io/catalog/request","method":"POST","body":"{\"offer\": \"company.verify\", \"for\": \"job/teradata-ai-infrastructure-engineer-generative-ai-model-serving-go-rust-gpu\", \"note\": \"what you need it for\"}"},"human":"https://alion.io/catalog?offer=company.verify&for=job%2Fteradata-ai-infrastructure-engineer-generative-ai-model-serving-go-rust-gpu"}],"agent":{"mcp":"https://alion.io/mcp/jobs","mcp_open":"https://alion.io/mcp/open","next":[{"tool":"get_job","arguments":{"id":3418916}},{"tool":"search_jobs","arguments":{"company":"teradata.com","detail":"compact"}},{"tool":"match_people","arguments":{"job_id":3418916},"note":"people open to offers who fit this vacancy"}],"note":"The same data through MCP tools: https://alion.io/mcp/open needs no sign-in (compact searches, postings, people); https://alion.io/mcp with an Alion account has everything. Docs: https://alion.io/developers"}}