Design Spotify

Source: https://www.hack2hire.com/question-bank/questions/system-design/69d8023ac4c1bc791ee22f16
Company: Meta, Microsoft, Google, Amazon
Difficulty: Medium

Problem statement

Stream music to hundreds of millions of listeners: catalog, search, playlists, smooth playback over bad networks, and offline downloads.

Requirements

We are building a music streaming service: a licensed catalog of ~100M tracks that hundreds of millions of people search, organize into playlists, and play on phones, desktops and speakers, including on flaky mobile networks and fully offline.

Functional requirements

• Play any track with playback starting in well under a second; seek, skip, gapless next track.
• Search the catalog (tracks, artists, albums, playlists, podcasts) with type-ahead suggestions.
• Create, edit, reorder, follow and collaborate on playlists; "Liked Songs" is just a special playlist.
• A personalized home feed (recently played, daily mixes, new releases from followed artists).
• Download playlists and albums for offline play, honoring the licence (re-check online at least every 30 days).
• Free tier: shuffle-only on mobile plus ads; Premium: on-demand, high quality, offline. One active playback stream per account at a time, with handoff between devices ("Connect").
• Out of scope: artist uploads/ingestion UI, royalties accounting detail, social messaging.

Non-functional requirements

• Time to first audio < 300 ms p50, < 1 s p95 for warm tracks; rebuffering below 0.5% of listening time.
• Playback availability 99.99%; search and home can degrade (cached or stale results) without stopping music.
• Catalog changes (new release at midnight local time, takedowns) visible in search within minutes; takedowns enforced on playback immediately.
• Playlist edits durable and visible on the user's other devices within a few seconds; no lost edits under concurrent edits.
• Audio content protected (encrypted files, per-device keys) because labels require it.

Back-of-the-envelope estimation

• 600M MAU, 250M DAU, ~25M peak concurrent streams.
• Bitrates: 24 kbps (data saver) to 320 kbps; average ~160 kbps → 25M × 160 kbps ≈ 4 Tbps of peak egress. This is a CDN problem, not an origin problem.
• Catalog storage: 100M tracks × ~4 min × ~5 encodings (Ogg/AAC at 24/96/160/320 plus lossless) ≈ 100M × 25 MB ≈ 2.5 PB at origin, triple-replicated in object storage. Popularity is extremely skewed: the top ~1% of tracks serve most plays, so edge hit rate is > 95%.
• Play starts: 250M DAU × 40 tracks/day ≈ 10B/day ≈ 115k/s average, ~350k/s peak manifest lookups.
• Search: ~3B queries/day including keystrokes ≈ 35k/s avg, ~100k/s peak.
• Playlists: ~5B playlists, median 50 tracks; ~60 bytes per entry → ~15 TB of entries.
• Play events for history, royalties and recommendations: 10B/day × ~300 B ≈ 3 TB/day into the event log.

API design

All calls are authenticated with an OAuth access token; the client also sends a device id.

GET /v1/tracks/{track_id}/stream?quality=auto
→ 200 { track_id, duration_ms, gain_db,
        files: [{format:"ogg", bitrate:96, file_id, size}, {..., bitrate:320}],
        cdn_urls: ["https://audio-fa.cdn/…/{file_id}?sig=…&exp=…", "<fallback cdn>"],
        key_ref: "k_9f…",  ttl_s: 3600 }
→ 403 { code: "not_available_in_region" | "subscription_required" }
→ 409 { code: "playing_elsewhere", active_device: "Kitchen speaker" }

POST /v1/keys   { key_ref, device_id } → { wrapped_key }          # decryption key, per device
GET  /v1/search?q=tay&types=track,artist&limit=10&market=US&cursor=…
→ 200 { top_hit, tracks:[{id,name,artists,album,art_url,explicit}], artists:[…], next_cursor }
GET  /v1/search/suggest?q=tay  → { suggestions:[{text, type, id}] }   # autocomplete, < 50 ms

POST /v1/playlists                     { name, public } → { playlist_id, snapshot_id }
POST /v1/playlists/{id}/ops            { base_snapshot_id, ops:[{op:"add", uri, position},
                                         {op:"move", from, to, count}, {op:"remove", item_id}] }
→ 200 { snapshot_id }   # server rebases ops on newer snapshots
GET  /v1/playlists/{id}?since_snapshot=… → full or delta
GET  /v1/me/home       → { sections:[{title, reason, items:[…]}], ttl_s }
POST /v1/me/player/transfer  { device_id, play: true }   # Connect handoff
POST /v1/offline/licenses    { device_id, track_ids:[…] } → { licenses:[{track_id, wrapped_key, expires_at}] }
POST /v1/events (batched)    [{type:"play", track_id, ms_played, reason_end, ts, offline:true}]

Data model

Core invariants: a track plays only if the user's market has active rights at play time; a playlist has a total order agreed by all devices after sync; an account has at most one active audio stream.

Playlist positions use fractional keys (e.g. "a0V" between "a0" and "a1") so an insert or move writes one row instead of renumbering; occasionally rebalance long playlists in the background.

High-level design

flowchart LR
  C[Client app] --> E[Edge API gateway]
  C --> CDN[Audio CDNs]
  CDN --> O[(Object storage origin)]
  E --> PB[Playback service]
  E --> S[Search service]
  E --> PL[Playlist service]
  E --> H[Home service]
  E --> SES[Session service]
  PB --> CAT[(Catalog cache)]
  PB --> K[Key service]
  S --> IDX[(Search index)]
  PL --> W[(Wide column store)]
  H --> F[(Precomputed feeds)]
  C -- events --> EV[Event pipeline Kafka]
  EV --> REC[Recommendation batch jobs]
  REC --> F
  ING[Catalog ingest] --> CAT
  ING --> IDX
  ING --> O

1. Play a song. The client asks the playback service for a manifest. It checks rights for the user's market and subscription, asks the session service whether this device holds the playback lease, and returns file ids for each bitrate, signed CDN URLs (two CDNs) and a key reference. The client fetches the first ~10 seconds at a low bitrate via HTTP range requests, gets the wrapped key, decrypts and starts playing, then steps up quality. Near the end of a track it prefetches the next one's manifest and first chunk, which is what makes skips feel instant.
2. Search. Each keystroke (debounced ~100 ms) calls suggest, served from a prefix index with popularity scores; Enter or a pause runs the full query against the inverted index with per-market filtering, then a light ranker blends text match, popularity and the user's own affinity.
3. Load the home feed. The home service reads home:{user_id} (built nightly and refreshed by near-real-time signals), filters out unavailable or recently played items, re-ranks a few hundred candidates by context (time of day, device), and returns ~10 sections. Artwork and track metadata are hydrated from the catalog cache in one batch call.
4. Playlist edits. The client sends ops with the snapshot it last saw. The playlist service applies them in a single-partition transaction (lightweight transaction or conditional write on the header's snapshot_id), rebasing on newer snapshots if needed, and publishes "playlist changed" to the user's other devices over their push channel; they pull the delta.
5. Offline. The client downloads the files like a normal stream at the chosen quality, stores them encrypted, and requests offline licences that expire in 30 days; offline plays are logged locally and uploaded on reconnect.

Why this holds together: the hot path (bytes) never touches our services after the manifest; everything stateful (playlists, sessions) is partitioned by a single key so it scales horizontally; and the expensive personalization runs offline, leaving cheap reads at request time.

Deep dives

Adaptive bitrate and playback orchestration. Music differs from video: tracks are short (~4 min, 4–10 MB at 320 kbps), so the whole file usually downloads in a few seconds on a good network. Strategy:

• Start at a bitrate chosen from recent measured throughput (default 96 kbps on cellular, 160 on Wi-Fi) so first audio needs only ~50–100 KB.
• Download in range requests of ~256 KB; keep a buffer target of 30 s on cellular, the whole track on Wi-Fi. Measure throughput per chunk with an EWMA.
• Switch bitrate only at chunk boundaries the codec can splice cleanly (files are encoded with aligned frames and the same gain), so switches are inaudible. Step down after 2 slow chunks; step up only after sustained headroom of 1.5× the next bitrate.
• On a stall, retry the same range on the second CDN after ~800 ms instead of waiting for a TCP timeout. Most "bad network" problems are lossy cellular links, so fast failover beats long timeouts.
• Cache recently and frequently played tracks on disk (an LRU of ~1–10 GB). About a third of plays are repeats, so they hit the local cache and need no network at all.
• Prefetch the next track's head when the current one has ~10 s left; for shuffle, the client picks the next item early so it can prefetch.

Personalized home feed. Two stages. Offline: daily batch jobs over the play log train embeddings for users and tracks (collaborative filtering on co-listening plus audio and text features), then build candidate sets per user: daily mixes from taste clusters, "jump back in" from history, new releases from followed artists, editorial playlists by affinity. Write ~500 candidates per user into the feed store (250M × ~20 KB ≈ 5 TB). Online: at request time, merge in fresh signals from the last hour (a streaming job keeps a short "recent sessions" vector), drop anything unavailable, apply diversity rules (no two sections dominated by one artist) and rank sections with a light model in < 50 ms. For cold-start users, use onboarding picks plus regional charts. Precomputing keeps p99 low; the cost is staleness, which the online layer covers.

Search: autocomplete, ranking and freshness. Suggest uses an in-memory prefix structure (an FST or edge n-gram index) holding the top ~10M entities by popularity, sharded by first character pairs and replicated in every region, so it answers in < 20 ms. Full search queries the inverted index with fuzzy matching (edit distance 1–2), transliteration and aliases ("The Weeknd" vs "weekend"). Ranking blends BM25 text score, global and regional popularity (plays in the last 7 days, log-scaled), user affinity (the user's own library and history get a strong boost), and click-through learned from search logs. Freshness: catalog ingest writes to the source of truth and emits a change event; an indexer consumes it and updates the index within ~1 minute. Scheduled releases are indexed ahead of time with available_from, and a query-time filter hides them until then, so a midnight drop appears exactly on time. Takedowns are enforced at playback (rights check on every manifest), so index lag can at worst show a result that refuses to play, never play something unlicensed.

Concurrent session enforcement and device handoff. Each device holds a long-lived connection (WebSocket) to the session service, keyed by account. Starting playback acquires a lease session:{account_id} with a 30 s TTL renewed by heartbeat; the write is a compare-and-set on lease_owner. If another device already holds it, the newer device wins (users expect "play here" to work) and the service pushes "stop" to the old device. Manifest responses carry the lease epoch; the CDN URLs are short-lived, so a device ignoring "stop" loses access within minutes. Handoff (Connect) transfers state: the controller sends transfer, the service copies {track_id, position_ms, queue, shuffle} to the target, which starts at the position and acquires the lease. Offline devices keep playing downloaded music without a lease; this is a deliberate licence carve-out, bounded by the 30-day re-check.

Offline downloads and DRM. Each file is encrypted once with a per-file content key (AES-128-CTR, which allows random access for seeking). Online, the key service wraps the content key with a device key generated at app install. Offline licences bind track_id + device_id + expires_at, limited to 10,000 tracks on 5 devices per account. On reconnect the client renews licences in bulk and deletes tracks whose rights lapsed in its market. Because files are shared, encryption costs nothing per user and CDNs cache them normally.

Trade-offs

• CDN-first delivery gives cheap global scale and low latency but we give up per-request control; signed URLs with short expiry and per-device keys restore enough control.
• Client-side ABR on whole files is simple and works for offline, but needs smart client logic on every platform and wastes some bytes when users skip early (mitigated by the low-bitrate first chunk).
• Precomputed home feeds keep serving cheap and fast but are up to a day stale; the online re-ranker covers recent behaviour at the cost of a second system.
• Search index is eventually consistent with the catalog (about a minute), accepted because rights are enforced at playback, not in search.
• Newest device wins for sessions is simple and matches user intent, but a shared account can bounce playback between two people; that is a product choice that favours paid family plans.
• Fractional position keys make edits single-row writes but positions grow and need occasional rebalancing; snapshot-based rebasing beats full CRDTs for a list that is rarely edited concurrently.
• Wide-column store for playlists scales writes and partitions well but gives up cross-playlist transactions; nothing needs them.

Interviewer expectations

Mid-level (L4)

• Separates metadata services from audio bytes and puts audio on a CDN with object storage behind it.
• Defines a working stream, search and playlist API and a sensible schema for tracks and playlists.
• Mentions multiple bitrates and client-side buffering.

Senior (L5)

• Does the bandwidth and storage math and concludes delivery is a CDN and edge-hit-rate problem.
• Designs ABR for short tracks: low-bitrate start, range requests, CDN failover, next-track prefetch, local cache.
• Explains the home feed as offline candidate generation plus online re-ranking, and search with autocomplete, ranking signals and index freshness.
• Handles playlist ordering and concurrent edits without renumbering.

Staff (L6+)

• Treats licensing as a system constraint: per-market rights checked at playback, scheduled releases, takedowns, offline licence expiry and DRM key design.
• Designs single-stream enforcement and device handoff with leases, and explains failure behaviour (session service down → allow playback, log, reconcile).
• Prioritizes degradation: music keeps playing when search, home or recommendations are down; quantifies cache hit rates and their cost impact.
• Connects the play-event pipeline to royalties, recommendations and abuse detection, and calls out where exactly-once matters (royalties) versus where it does not.