Design a news feed

Design the home feed for a social network with 1B users: posts from people you follow, ranked, fresh within seconds.

Functional requirements

• Users see a feed of posts from people and pages they follow, ranked.
• New posts appear within seconds for active viewers.

Non-functional requirements

• Feed load under 200 ms p99; 1B users, 300M daily active.
• Celebrity accounts with 100M followers must not melt the write path.

Back-of-the-envelope estimation

• 300M DAU × 10 feed loads/day ≈ 35k reads/s average, 100k/s peak.
• 300M × 1 post/day ≈ 3.5k posts/s. Average 300 followers → ~1M feed inserts/s if we fan out on write.

GET /feed?cursor= → {items: [post], next_cursor}; POST /posts.

• posts(id, author_id, body, media, created_at) in a sharded store by post_id.
• follows(follower_id, followee_id) sharded by follower, with a reverse index.
• feeds:{user_id} in Redis: a capped list of the last ~800 post ids with scores.

flowchart LR
  W[Write post] --> PS[(Post store)]
  W --> FQ[Fan-out queue]
  FQ --> FW[Fan-out workers] --> FC[(Feed cache)]
  R[Read feed] --> FC
  R --> CEL[Celebrity posts merge]
  R --> RANK[Ranker]

Fan-out on write vs read. Push post ids into followers' feed caches for normal users (fast reads). For accounts above ~10k followers, don't fan out; merge their recent posts at read time. This hybrid bounds write amplification.

Ranking. Candidate generation (followed posts + some recommended) → a lightweight model scores ~500 candidates → a heavier model reranks the top 50, with diversity rules (not five posts in a row from one author).

Inactive users. Skip fan-out for users inactive for 30 days; rebuild their feed on next login.

Pagination. Cursor on (score, post_id) so new posts don't shift pages.

• Fan-out on write costs storage and write throughput for fast reads; the celebrity threshold is a tunable knob.
• Ranking freshness vs cost: cache the ranked page for a minute per user.