Question: Design an AI coding agent sandbox
Companies: Anthropic, Cursor
Difficulty: Hard
Problem statement
Agents run untrusted code for many users. Design isolated, fast-starting sandboxes with file and network controls.
Requirements
Functional
- Sandbox lifecycle and execution: create a sandbox from a base image (language runtime + repo checkout), run commands with streamed stdout/stderr, exit codes and per-command timeouts, snapshot/resume between agent turns, and destroy on task completion or idle timeout (e.g. 15 min).
- File controls: each sandbox gets a private writable workspace. The agent can read, write, list and diff files there. Host and other tenants' files are never visible. The result is exported as a git patch or kept for the next session.
- Network controls: egress is denied by default. A per-session allowlist (package registries, the user's git remote) goes through an egress proxy. Cloud metadata endpoints and the internal network are always blocked, and every connection is logged for audit.
Non-functional
- Isolation first: treat all code as hostile. Use a hardware-virtualized boundary (a Firecracker-style microVM or gVisor) rather than containers that share the host kernel. Set per-sandbox CPU, memory, disk and PID quotas so no tenant can starve its neighbours, and keep no long-lived credentials inside the sandbox.
- Fast start: target p50 time-to-ready under 500 ms and p99 under 2 s, using a warm pool of pre-booted VMs plus snapshot restore. Per-command overhead should stay under 50 ms, because one agent task can make dozens of tool calls.
- Scale and cost (assumed load): 1M tasks/day × 10 min avg ≈ 1M × 600 s / 86,400 s ≈ 7k concurrent sessions, so ~20k at a 3× peak. At 2 vCPU / 4 GB each, memory is the binding constraint: 80 TB ÷ 768 GB hosts ≈ 105 hosts. CPU can be oversubscribed ~4× because sandboxes mostly sit idle while the LLM thinks (40k vCPU ÷ 4 ÷ 96 ≈ 105 hosts). The control plane should be 99.9% available. A single sandbox crash is acceptable as long as the agent can retry on a fresh one.