Design YouTube

Upload videos, transcode them into many formats, and stream them worldwide with adaptive bitrate, along with views, likes and comments.

We are building a video platform: creators upload large video files, the system transcodes them into an adaptive-bitrate ladder, and viewers worldwide stream them from a CDN, with view counts, likes and comments alongside. Scope out live streaming, ads, recommendations and search beyond a title lookup.

Functional requirements

• Upload a video (up to 256 GB, typically 50 MB–2 GB) through a resumable session that survives dropped connections.
• Transcode each upload into several resolutions (144p–4K) and codecs (H.264, VP9/AV1), packaged as HLS and DASH segments with manifests.
• Play a video with adaptive bitrate: the player switches rendition per segment as bandwidth changes; seeking is instant.
• Show title, channel, view count, like count and paged comments on the watch page.
• Like/unlike once per user; post and list comments; remember each viewer's resume position.
• Creator sees processing status (uploading, processing, ready, blocked) and can set visibility (private, unlisted, public).

Non-functional requirements

• Playback start (time to first frame) under 2 s p95 globally; rebuffer ratio under 0.5%.
• Read-heavy and extremely skewed: a tiny share of videos takes most traffic, and a video can go from 0 to 10M views in an hour.
• Uploads are never lost once acknowledged (11 nines durability on the raw object).
• Time from upload complete to playable in SD: under 5 minutes for a 10-minute video.
• View and like counts are eventually consistent (seconds to a minute); playback metadata is highly available.

Back-of-the-envelope estimation

• 2B monthly users, 5B video plays/day ≈ 58K plays/s average, ~200K/s peak.
• Uploads: 500 hours of video per minute ≈ 30K hours/hour ≈ 8.3 hours of video ingested every second.
• Raw storage: ~1 GB per hour of 1080p source → 720K hours/day × 1 GB ≈ 720 TB/day raw.
• Encoded ladder is ~1.5× the source across all renditions → ~1 PB/day of packaged output; ~3 copies via erasure coding across regions → ~1.5 EB/year before tiering.
• Egress: ~50M concurrent streams at peak × ~3 Mbps average ≈ 150 Tbps, which only a CDN with ISP-embedded caches can serve.
• Segments of 4 s: a 10-minute video at 6 renditions × 2 codecs ≈ 1,800 segment objects.
• Transcode compute: ~1 CPU-hour per hour of video per rendition with fast presets → 720K × 12 ≈ 8.6M CPU-hours/day ≈ 360K cores busy, so encoding must be parallel and use hardware encoders where possible.

POST /v1/uploads
  {title, description, size_bytes, content_type, sha256?}
  Idempotency-Key: <client uuid>
  → 201 {upload_id, video_id, part_size: 16777216, part_count, expires_at}

PUT  /v1/uploads/{upload_id}/parts/{n}   (or presigned object-store URL per part)
  body: bytes  → 200 {etag}

GET  /v1/uploads/{upload_id}             → {received_parts: [1..37, 39], status}
POST /v1/uploads/{upload_id}/complete    {parts: [{n, etag}]}
  → 202 {video_id, status: "processing"}   (repeat calls return the same result)

GET  /v1/videos/{video_id}/playback
  → 200 {manifest_urls: {hls, dash}, token, expires_at, resume_at_s,
         title, channel, view_count, like_count, duration_s}
  → 409 {code: "processing"} | 451 {code: "blocked"} | 404

POST   /v1/videos/{video_id}/like        → 204   (idempotent; DELETE to unlike)
GET    /v1/videos/{video_id}/comments?cursor=…&limit=20 → {items, next_cursor}
POST   /v1/videos/{video_id}/comments    {text, parent_id?}, Idempotency-Key
POST   /v1/events/heartbeat              {video_id, position_s, rendition, rebuffer_ms}

• Clients upload bytes straight to object storage with presigned part URLs; API servers never proxy video.
• Errors: 413 over size limit, 410 for an expired session, 429 with Retry-After. Lists use opaque cursors ((created_at, comment_id) encoded), never offsets.
• The player never fetches the raw file. It fetches a manifest from the CDN, then segments; the manifest is the contract between packaging and playback.

Invariants: a video is public and playable only when status is ready and moderation passed; a manifest only references segments that already exist; a (video_id, user_id) like row exists at most once; packaged paths are never overwritten, only superseded by a new manifest_version.

flowchart LR
  U[Uploader] --> API[Upload API]
  U -->|"presigned parts"| RAW[(Raw blob store)]
  API --> MDB[(Video metadata DB)]
  RAW -->|"object committed"| Q[Job queue]
  Q --> SPL[Splitter]
  SPL --> ENC[Encoder fleet]
  ENC --> PKG[Packager]
  PKG --> PB[(Packaged blob store)]
  PKG --> MDB
  Q --> MOD[Moderation]
  MOD --> MDB
  V[Viewer] --> PBS[Playback API]
  PBS --> MDB
  V --> CDN[CDN edge]
  CDN --> MID[Regional mid tier]
  MID --> PB
  V --> EV[Event ingest]
  EV --> CNT[Counter service]

1. Upload. The client creates a session (idempotency key → same upload_id on retry), uploads 16 MB parts in parallel to presigned URLs, then calls complete. The API verifies part ETags, asks the object store to stitch the multipart object, writes status=processing, and emits a video.uploaded event.
2. Process and publish. The splitter cuts the source at keyframes into ~10 s chunks; the encoder fleet transcodes chunk × rendition in parallel; the packager writes fMP4/CMAF segments and per-rendition playlists, then the master manifest last. Moderation (hash match + classifier) runs in parallel. When the minimum ladder (360p, 720p H.264) is ready and moderation passes, the video flips to ready; higher renditions and AV1 arrive later and bump manifest_version.
3. Watch. The player calls the playback API, which checks status, visibility and region rules, returns signed manifest URLs plus resume position. The player pulls the manifest and segments from the nearest CDN edge; misses go to a regional mid tier, then origin. Heartbeats every 10–30 s feed views, QoE metrics and watch position.

Failure modes the design assumes: encoder workers die mid-chunk (tasks are retried, outputs are deterministic paths, so retries overwrite identically); an origin region is down (packaged objects are replicated, CDN fails over to another origin); the counter service lags (watch page shows a slightly stale count).

Resumable upload and idempotent completion. The session record holds part_count and the object-store multipart ID. On reconnect the client calls GET /uploads/{id} to learn which parts landed and re-sends only the missing ones; parts are idempotent by number. complete runs a conditional update state: uploading → committing so two concurrent completes cannot both enqueue processing; the second sees committing/committed and returns the same 202. The video.uploaded event is written via an outbox row in the same transaction, so a crash between commit and publish does not strand the video. Content hashes (per-part MD5, optional whole-file SHA-256) catch corruption; a sweeper aborts multipart uploads past expires_at (24–72 h) to free storage.

Transcoding into renditions, segments and manifests. Encoding a 2-hour 4K file serially takes hours; splitting at GOP boundaries into 10 s chunks gives ~720 independent tasks per rendition, done in minutes on a large fleet. A DAG per video: probe → split → encode(chunk, rendition) → concat/package → manifest → publish. Ladder example:

Use CMAF fMP4 segments so one set of files serves both HLS and DASH. Keyframes are aligned across renditions (fixed GOP of 2 s, segments of 4 s) so the player can switch at any segment boundary. Prioritize: SD first for time-to-publish, then HD, and encode AV1 only once a video crosses a view threshold, since AV1 costs ~5–10× the CPU but saves ~30% egress, which only pays off on popular videos.

Playback bootstrap and the manifest contract. The playback API is the only dynamic call per view, and it is cheap: one metadata read (cached in-region for 5–30 s, invalidated on block) and a signed token (HMAC over video_id, expiry, optional IP/ASN) that the CDN validates at the edge without calling home. The master manifest lists renditions and codecs; the player picks a start rendition from its bandwidth estimate (low by default to start fast), then uses buffer-based ABR (BOLA-style) to step up or down. Segments are immutable and served with Cache-Control: max-age=1y; the master manifest has a short TTL (60 s) so new renditions appear. Seeking maps time → segment index, so it costs one segment fetch.

Hot videos and cache fill under read skew. Three tiers: ISP-embedded edge caches → regional mid-tier → origin. Request collapsing at each tier ensures one miss per object per tier, not one per viewer, so a viral video's first segment hits origin a handful of times. For a suddenly trending video, the view-velocity signal triggers prewarm: push the first 30 s of the popular renditions to edges in regions where it is trending. Long-tail videos (most of the catalog, tiny share of plays) are served from mid-tier or origin; edges cache the head of the distribution by LFU. Origin shields are sized for the cold-miss rate, not total traffic: at a 95% edge hit ratio and 150 Tbps, origin+mid serves ~7.5 Tbps.

Views, likes and comments at scale. A view counts after ~30 s of playback (heartbeat), deduplicated by (viewer, video, 1 h bucket) and filtered for bots in a stream job. Counts go to a sharded counter (N shards per hot video, N grows with rate) and are summed every few seconds into the displayed count. Likes write the (video_id, user_id) row conditionally (insert-if-absent), then increment the counter only when the insert succeeded, so retries never double count. Comments are partitioned by video; a viral video with 1M comments is a large partition, so bucket by (video_id, month) for very hot videos and serve "top comments" from a precomputed list.

• Publishing on a minimum ladder gets videos live in minutes but first viewers on fast networks get 720p until 1080p finishes.
• Chunked parallel encoding cuts latency ~100× but costs extra keyframes at chunk boundaries and a concat/verify step.
• AV1 only for popular videos saves compute on the long tail but leaves egress savings on the table for the 2% of plays from never-promoted videos.
• Immutable, versioned segment paths make caching trivial but leave old versions in storage until a cleanup job reclaims them.
• Eventually consistent counters give cheap writes at 200K events/s but a view count can trail reality by a minute; an exact count would need a hot-row transaction per view.
• Running moderation in parallel with encoding shortens time to publish but means a fast-ready video may need a hold step; strict gates (no public until moderation passes) protect users at the cost of minutes of delay for every creator.
• Signed tokens validated at the edge avoid a call home per segment but cannot be revoked before expiry, so keep TTLs short (a few hours).

Mid-level (L4)

• Separates upload, processing and playback, with the blob store holding video and a DB holding metadata.
• Knows video must be transcoded into several resolutions and served through a CDN.
• Gives a working API for upload, watch and comments.

Senior (L5)

• Designs resumable multipart upload with idempotent completion and direct-to-blob transfer.
• Explains ABR: renditions, aligned keyframes, segments, manifests, and why segments are immutable and cacheable.
• Parallelizes encoding by chunk, publishes on a minimum ladder, and runs the counters as sharded, eventually consistent aggregates.
• Sizes storage, egress and encoding compute with numbers.

Staff (L6+)

• Treats egress as the dominant cost: tiered CDN, request collapsing, prewarm for viral videos, codec choice driven by popularity.
• Defines invariants and failure handling: outbox for upload events, deterministic retry-safe encoding outputs, versioned manifests, cleanup of orphaned parts and stale packages.
• Weighs moderation gating against time to publish, and protects playback availability when metadata or counters degrade.