Design LeetCode

Source: https://www.hack2hire.com/question-bank/questions/system-design/69d98d1ed94fcdd01529ff91
Company: Snowflake, Meta, Uber
Difficulty: Medium

Problem statement

Users solve coding problems in the browser; run their submissions safely against hidden tests, return verdicts quickly, and support live contests with leaderboards.

Requirements

We are building an online judge like LeetCode: users browse coding problems, write code in the browser, submit it to run against hidden test cases in a sandbox, get a verdict within seconds, and compete in timed contests with a live leaderboard.

Clarify early: which languages (say 10), whether "Run" against sample tests is in scope alongside "Submit", contest size (100k concurrent participants), and whether the leaderboard must be exact in real time or may lag a few seconds (a few seconds is fine).

Functional requirements

• List and filter problems (difficulty, tag, solved status); view a problem's statement, examples, and starter code per language.
• Submit code in a supported language; it runs against hidden tests and returns a verdict: Accepted, Wrong Answer, Time Limit Exceeded, Memory Limit Exceeded, Runtime Error, or Compile Error, plus runtime and memory.
• Run code against sample or custom input without recording a submission.
• Contests: fixed start and end time, a problem set hidden until start, scoring by problems solved, then total time plus a penalty per wrong attempt.
• Leaderboard: top N and "my rank" during and after a contest.

Non-functional requirements

• Safety first: untrusted code must not reach the network, the host, other users' code, or the hidden tests' expected outputs.
• Verdict p50 under 3 s, p99 under 10 s outside contests; under 15 s p99 at contest peak.
• Never lose an accepted submission; judging is at-least-once and idempotent.
• Fair: the same code gets the same verdict regardless of load (CPU-time limits, not wall-clock on a busy host).
• Problem browsing is read-heavy and cacheable; 99.9% available.
• Leaderboard reads at contest peak served in under 100 ms; staleness up to ~5 s is acceptable.

Back-of-the-envelope estimation

• 10M monthly users, 1M daily; ~5M submissions/day normally → ~60/s average, ~200/s peak.
• Contest: 100k participants, 4 problems, 90 minutes, ~10 submissions each → 1M submissions; the first 10 minutes and the last 15 see ~3× average → ~500 submissions/s at peak.
• Judging cost: ~50 test cases, ~2 s CPU per submission on average (compile + run) → 500/s × 2 s = 1,000 busy cores at contest peak; plan ~1,500 cores with headroom, ~200 normally.
• Storage: 5M × 2 KB code ≈ 10 GB/day of code, ~3.6 TB/year; cheap in a database plus object storage for old rows.
• Test data: 3,000 problems × ~5 MB of inputs/outputs ≈ 15 GB; cached on every judge host.
• Leaderboard reads: 100k users polling every 5 s → 20k reads/s.

API design

GET  /api/problems?difficulty=medium&tag=graph&cursor=..&limit=50
  → {items: [{slug, title, difficulty, tags, acceptance_rate, solved_by_me}], next_cursor}

GET  /api/problems/{slug}
  → {slug, title, statement_md, examples: [{input, output}],
     limits: {time_ms, memory_mb}, starter_code: {python: "...", cpp: "..."}}

POST /api/submissions
  {problem_slug, language, code, contest_id?, idempotency_key}
  → 202 {submission_id, status: "queued"}

GET  /api/submissions/{id}
  → {status: "queued|running|done", verdict, passed: 37, total: 50,
     runtime_ms, memory_kb, failed_case?: {index, input?, expected?, actual?}}
  (or subscribe: GET /api/submissions/{id}/events  — server-sent events)

POST /api/run   {problem_slug, language, code, stdin} → 202 {run_id}

GET  /api/contests/{id}/leaderboard?offset=0&limit=50
  → {as_of, rows: [{rank, user, score, penalty_s, problems: [{solved, attempts, time_s}]}],
     me: {rank, score, penalty_s}}

Hidden test inputs are never returned; failed_case shows only for public examples or after a contest ends.

Data model

Authoritative state lives in Postgres (or MySQL): problems, submissions, contest results. Redis holds derived, rebuildable views (the leaderboard and the solved set). Test files live in object storage.

The boundary: contest_results is the source of truth for scoring. The leaderboard can always be recomputed from it:

SELECT user_id,
       COUNT(*) FILTER (WHERE solved)                                   AS score,
       SUM(EXTRACT(EPOCH FROM first_ac_at - c.start_at) + wrong_before_ac * c.penalty_s)
         FILTER (WHERE solved)                                          AS penalty
FROM contest_results r JOIN contests c ON c.id = r.contest_id
WHERE contest_id = $1
GROUP BY user_id
ORDER BY score DESC, penalty ASC;

That query over 100k users × 4 rows is fine once at the end, not 20k times a second. Redis caches it as a sorted set lb:{contest_id} with member user_id and score solved × 10^9 − penalty_s, so a higher score always ranks first and lower penalty breaks ties.

High-level design

flowchart LR
  U[Browser] --> CDN[CDN]
  CDN --> GW[API gateway]
  GW --> PS[Problem service]
  GW --> SS[Submission service]
  GW --> LB[Leaderboard service]
  PS --> DB[(Postgres)]
  PS --> RC[(Redis cache)]
  SS --> DB
  SS --> Q[[Submission queue]]
  Q --> J[Judge workers]
  J --> S3[(Test data S3)]
  J --> RQ[[Result stream]]
  RQ --> RH[Result handler]
  RH --> DB
  RH --> ZS[(Redis leaderboard)]
  LB --> ZS
  RH -. push verdict .-> GW

1. Browse problems. The list is a cached, paginated query; public fields are cached at the CDN for 60 s. solved_by_me comes from a per-user Redis set solved:{user} merged in by the problem service.
2. View a problem. The statement, examples and starter code are immutable per version, so they cache at the CDN and in Redis with a long TTL; publishing a new version changes the cache key.
3. Submit. The submission service checks auth, the rate limit (e.g. 1 submission per 5 s per user), and contest window; writes the row as queued; publishes {submission_id} to the queue (Kafka or SQS, partitioned by language pool); returns 202. The client listens on SSE (or polls every 1 s).
4. Judge. A worker takes the job, fetches the code, compiles in a sandbox, runs each test with CPU and memory limits, compares output, and publishes the result. The result handler updates submissions, and for contest submissions upserts contest_results and updates the Redis sorted set, then pushes the verdict to the waiting client.
5. Leaderboard. The leaderboard service reads ZREVRANGE for the page and ZREVRANK for "me", fills in names from a user cache, and responds; the response caches for 2 s per page.

Deep dives

Isolating untrusted code. Treat every submission as hostile: fork bombs, infinite loops, reading /etc, opening sockets, filling disk.

Layer it: a microVM or gVisor sandbox per run, inside it no network namespace, a read-only root filesystem with the language toolchain, a small tmpfs (64 MB) for the binary and output, a seccomp allowlist, cgroup limits (cpu.max, memory.max at the problem limit + runtime overhead, pids.max = 64), and output capped at 1 MB. Run as an unprivileged user with no capabilities. Judge hosts sit in their own VPC with no outbound route. Expected outputs never enter the sandbox: the harness feeds input on stdin, captures stdout, and compares outside.

Execution pipeline and test harness. Compile once (C++ ~1–2 s, cached by hash of code + compiler flags so resubmits skip it), then run tests in order, stopping at the first failure (that is all the verdict needs, and it saves CPU). Measure CPU time via cgroup cpu.stat, not wall-clock, so a busy host doesn't cause false TLE; also kill at 3× the limit in wall time to catch sleeping code. Languages differ in speed, so time limits are per language (e.g. Python gets 3× C++). Problems with many valid outputs use a per-problem checker program (float tolerance, any valid path) run outside the sandbox. For LeetCode-style function signatures, a per-language driver wraps the user's function: it parses the input, calls Solution().twoSum(...), and prints a serialized result. Workers are idempotent: if a job is redelivered, they skip it when the row is already done.

Container pool management. Cold-starting a sandbox for every submission wastes 150–300 ms and CPU. Keep a warm pool per language per host (e.g. 20 Python, 20 Java with JVM pre-started), each used once and destroyed, with a replacement started in the background. Never reuse a sandbox across users, since state could leak. Size pools by the recent mix of languages; Java and C++ pools need the most memory. Test data is cached on local SSD per host keyed by (problem_id, testset_version), so S3 is hit only on a miss.

Contest burst scaling. Contests are scheduled, so pre-scale: 15 minutes before start, raise the judge fleet to the forecast (≈ participants × 0.005 cores, ~1,500 cores for 100k) and pre-warm test data for the contest problems on every host. Keep separate queues for contest and practice traffic; contest jobs get priority and their own capacity, so a contest never starves behind a practice backlog and vice versa. If the queue still grows, autoscale on queue depth and age (target oldest job under 10 s). Since scoring uses submission time, not judging time, a slow queue delays verdicts but does not change ranks. Rate limit per user (one in-flight contest submission) to stop spam resubmission.

Leaderboard at contest scale. Recomputing the SQL ranking per read would be 20k aggregate queries/s. Instead the result handler updates state incrementally: on an Accepted for a problem not yet solved, compute that user's new total and ZADD lb:{contest} score user; wrong answers only bump wrong_before_ac in Postgres and change nothing visible. ZREVRANGE for a page and ZREVRANK for "me" are O(log N + page). 100k members is ~10 MB, one Redis primary with replicas for reads handles 20k reads/s easily. The top pages (most of the traffic) are also cached for 2 s, so Redis sees a few hundred page reads per second. If Redis is lost, rebuild it from contest_results with the SQL above in seconds. After the contest closes, freeze the final ranking into a contest_standings table and serve from there. For ranked ties with the same score and penalty, add earlier last-AC time as a tiebreaker inside the fractional part of the score.

Trade-offs

• Async queue vs synchronous judging: the queue absorbs contest bursts and survives worker crashes, at the cost of an extra hop and a push channel to return the verdict.
• gVisor/microVM vs plain containers: much stronger isolation for 10–20% CPU overhead and more complex operations; a plain container is a single kernel bug away from compromise.
• Stop at first failed test: saves CPU and returns faster, but users don't learn how many later tests would have failed.
• Redis sorted set as a derived view: fast reads and cheap updates, but it can lag Postgres by a moment and must be rebuildable; Postgres stays the authority.
• Leaderboard cache of 2 s: cuts load by 100× but rankings can look a couple of seconds stale.
• Pre-scaling for contests: idle cores before the start cost money, but scaling reactively would add minutes of queueing exactly when fairness matters most.
• CPU-time limits: fair under load, but sleeping or I/O-bound code needs a separate wall-clock cap.

Interviewer expectations

Mid-level (L4)

• Covers the core flows: list, view, submit, poll for the verdict, contest leaderboard.
• Puts submissions on a queue processed by workers instead of judging inside the web request.
• Knows user code must run in a container with time and memory limits and no network.

Senior (L5)

• Layers isolation (gVisor or microVM, seccomp, cgroups, read-only filesystem, no egress) and keeps expected outputs out of the sandbox.
• Designs the leaderboard as a Redis sorted set with a composite score, derived from an authoritative results table and rebuildable from it.
• Sizes the judge fleet from submissions/s × CPU per submission, and plans warm pools and pre-scaling for contests.

Staff (L6+)

• Addresses fairness: CPU-time measurement, per-language limits, verdicts unaffected by queue delay, priority isolation between contest and practice traffic.
• Explains idempotent at-least-once judging, test-set versioning, and rejudging after a test fix.
• Discusses cost (pool sizing, compile caching, stop-at-first-failure) and how the system degrades when judging falls behind.