AI Engineer

You will build the company's AI Platform: the shared layer that lets every product team ship LLM features and agents without rebuilding serving, agent runtime, retrieval or evaluation. This is a platform engineering role with internal teams as your users.

The platform has four parts: model serving and an LLM gateway, an agent runtime built on harnesses such as DeepSeek Harness, Codex, shared NLP and RAG services, and an evaluation and observability stack.

Trách nhiệm công việc

Model serving and LLM gateway

• Serve open-source LLMs, embedding and reranking models on GPUs with vLLM and SGLang, behind OpenAI-compatible endpoints.
• Build the LLM gateway: one API across self-hosted and external models, with routing, fallback, rate limits, quotas, API keys and per-team cost tracking.
• Tune throughput and latency: batching, quantization, KV and prompt caching, autoscaling, GPU capacity planning.

Agent runtime and harness

• Offer the agent harness as a platform service: sandboxed execution, session storage, permissions, headless runs in CI/CD and scheduled jobs.
• Extend harnesses with plugins, model adapters and tool registries so teams can run any platform model in them.
• Maintain a shared catalog of tools, MCP servers and skills that teams can reuse, with access control per team.
• Provide templates and an SDK for common agent patterns: single-agent, multi-agent, subagents, human-in-the-loop.

Shared NLP and RAG services

• Build retrieval as a service: ingestion, chunking, embedding, hybrid search, reranking, per-tenant indexes.
• Provide reusable NLP components: classification, information extraction, summarization, Vietnamese document processing.

Evaluation, observability and governance

• Build the eval platform: benchmark task sets, automated scoring, LLM-as-judge, regression gates before model or prompt releases.
• Provide tracing, logging and session replay for every model call and agent run.
• Enforce guardrails: prompt injection defenses, PII handling, tool permissions, audit logs.

Yêu cầu

• Bachelor's degree in Computer Science, Artificial Intelligence, Applied Mathematics or equivalent.
• 2+ years of NLP/LLM experience, including at least one LLM or agent system running in production.
• Experience building shared services or internal platforms used by other teams: API design, multi-tenancy, versioning, SLAs.
• Experience serving LLMs with vLLM or SGLang: deploying OpenAI-compatible endpoints on GPUs, tuning batching, quantization and KV cache, and benchmarking throughput and latency.
• Strong Python and backend skills (FastAPI or similar, async, queues); reading and writing TypeScript/Node.js is a big advantage, since many harnesses are written in TypeScript.
• Solid understanding of how agents work: agent loop, tool calling, context window management, prompt caching, structured output.
• Hands-on with at least one agent harness or coding agent (DeepSeek Harness, Codex or similar) at the customization level, not only as an end user.
• Experience building RAG and working with vector databases.
• Strong NLP fundamentals: Transformers, tokenization, embeddings, evaluation metrics.
• Solid infrastructure skills: Docker, Kubernetes, Linux, CI/CD, monitoring.

Nice to have

• You have built or operated an internal ML/AI platform or LLM gateway serving several product teams.
• You have written a plugin, MCP server or skill for an agent harness and have a public repo to show.
• GPU cluster operations: scheduling, autoscaling, multi-GPU and multi-node inference.
• Knowledge of agent safety: prompt injection, sandboxing, tool permissions.