Security Engineer

About METR

We are a nonprofit research organization that develops scientific methods to assess AI capabilities, risks, and mitigations, with a specific focus on threats related to AI R&D automation and misalignment.

METR has consistently set precedents for catastrophic AI risk evaluations, including the first independent safety evaluations (working informally with Anthropic and OpenAI in 2022), the first loss-of-control evaluations and first agentic dangerous capability evaluations, the first evaluations using finetuning (mentioned briefly here),the first independent evaluations using internal information about training, the first review partnership for company risk analysis, the first embedded redteaming, and the first evaluations of internal deployments.

We’ve been consulted and/or favorably referenced by groups on opposite ends of various spectra, including a16z, Khosla, Gary Marcus, Obama, and Dean Ball, and are known for producing one of the most positive results on AI capabilities (the time horizon trend) and the most negative (our downlift study). We’re generally referenced as the canonical third party assessor, e.g. as the obvious candidate to verify conditional pause agreements.

We believe it is robustly good for policymakers and civil society to have a clear understanding of risks from AI systems, and we are extremely excited to build a team of ambitious, excellent people to tackle one of the most important challenges of our time.

About the role

Security at METR is becoming its own dedicated team, and you would be one of its first hires. It is extremely important that we continue to be an organization that frontier AI labs, governments, and the public trust with sensitive model access and confidential information. As misalignment incidents become more extreme and confidential information about models and frontier AI labs becomes more valuable, we expect to be under increasingly heavy pressure.

For us, security encompasses managing endpoints and securing development environments, cloud platform security, safely sandboxing agents and evaluations, VPN and VPC networking, application code reviews, account provisioning and access control, and helping ensure we use the best practices across all of our workflows.

What this role looks like

• Offensive security: You would be the first person on the team with an offensive security background. You'll run targeted red-team exercises against our own systems and build automated AI red teaming.

• High-context detection and response: You will build AI systems that can quickly triage and respond to threats, both from internal agents and external attackers.

• Blue-team engineering: Detection engineering, telemetry pipelines, incident response, and hardening across our cloud infrastructure, endpoints, and identity systems.

• Securing a unique attack surface: METR's evaluation infrastructure runs frontier AI agents, including early checkpoints of unreleased models, executing untrusted, model-generated code at scale on multi-day tasks.

• Enabling bleeding-edge research: You'll work closely with our researchers to make dangerous-capability experiments safe to run. We often face extreme reward hacking and evaluation awareness during our pre-deployment evaluations, and expect internal threats from agents to become more extreme.

Why this role matters

• METR handles some of the most sensitive artifacts in AI - pre-release frontier model access, confidential lab information, and transcripts with raw chain-of-thought. Labs and policymakers trust us with this because of our security posture, and keeping that trust is necessary for everything else we do.

• As AI agents are used more aggressively by malicious actors for cyber offense operations, and METR's salience rises in the public eye, we expect to face increasingly sophisticated attacks. Strengthening security at METR can be one of the highest-leverage roles to ensure third parties continue to have access to confidential information necessary to inform the world about current risks.

• METR is one of the first organizations to see and closely study misalignment incidents that involve models breaking out of sandboxes, attacking our infrastructure, manipulating graders, and more. We also may pursue incident investigations embedded in frontier labs, in which case internal experience with similar failures will be critical.

Required skills

• Deep security expertise: You have strong fundamentals across systems, networks, cloud, and identity.
• Offensive security: You have experience acting like an attacker, whether through red teaming, penetration testing, or adversarial research.
• AI/LLM engineering: You build with AI: agent pipelines, LLM-powered tooling, automated workflows, and understand current limitations of those tools.
• AWS: You should know AWS very well, including a deep understanding of IAM policies.

We don't screen on certifications, degrees, or years of experience.

Nice to haves

• Detection engineering at scale: Experience with SIEM/detection pipelines, writing and tuning detections, and threat hunting.

• Cloud and container security: AWS (especially non-trivial IAM), Kubernetes, and infrastructure-as-code environments.

• Incident response: You've led or worked severe incidents, ideally those involving AI agents.

• AI security research: Familiarity with prompt injection, agent containment, model supply-chain risks, or red teaming AI systems themselves.

Ideally you have experience with a good portion of these technologies:

• AWS: cloud-native software platforms
• EKS
• Lambda
• ECS
• IAM (in-depth)
• SQS
• CloudWatch
• SecurityHub & GuardDuty
• PostgreSQL: RLS, serverless Aurora
• Pulumi: IaC
• DataDog: SIEM
• Okta: IdP
• Google Workspace: IdP
• Tailscale: networking
• CrowdStrike Falcon: endpoint security