Design a Slack-like chat system

Workspaces, channels, threads and direct messages with real-time delivery, unread state, search and history, for large companies.

We are building a workplace chat service for large companies: workspaces that hold public and private channels, threads and DMs, with real-time delivery, per-user unread state, full history and search.

Functional requirements

• Users belong to one or more workspaces; a workspace has channels (public or private), DMs (1:1) and group DMs (up to 9 people).
• Send, edit and delete messages; reply in a thread; react with emoji; attach files and images.
• Messages reach every online member within about a second, on every device they have open.
• Scroll back through full channel history; jump to a message from a link or search hit.
• Unread counts and mention badges per channel, kept in sync across desktop, web and mobile.
• Search messages and files the user is allowed to see.
• Offline users get mobile push for DMs and mentions.

Out of scope: voice/video huddles, the app/bot platform, enterprise key management, compliance exports (mentioned only where they shape the design).

Non-functional requirements

• Send-to-delivery p99 under 500 ms within a region for online recipients.
• No acknowledged message is ever lost; per-channel order is the same for every reader.
• 99.99% availability for send and read; search may lag by a few seconds.
• Channels from 2 members up to 500k members (company-wide announcement channels).
• Tenant isolation: one workspace's data never shows in another's queries or search; data residency per enterprise.

Back-of-the-envelope estimation

• 50M daily active users, 20M concurrently connected at peak, each with ~1.5 devices → ~30M open WebSockets. At ~100k sockets per gateway host, ~300 hosts plus headroom.
• 50M users × 40 messages/day = 2B messages/day ≈ 23k/s average, ~100k/s peak.
• Average message 500 bytes with metadata → 1 TB/day, ~365 TB/year before replication; ×3 replicas ≈ 1.1 PB/year.
• Fanout: median channel has ~10 online members, but a 100k-member channel with 30k online turns one post into 30k socket pushes. Deliveries ≈ 100k/s × ~20 ≈ 2M pushes/s peak.
• Reads dominate writes ~10:1 (opening channels, scrolling, reconnect catch-up).
• Files: 5% of messages carry an attachment of ~1 MB → 100M files/day ≈ 100 TB/day in object storage.

Send a message (HTTPS, idempotent):

POST /v1/channels/{channel_id}/messages
Headers: Idempotency-Key: <client_msg_id uuid>
{ "text": "ship it", "thread_ts": null, "file_ids": ["F1"] }
→ 201 { "message_id": "M9", "channel_id": "C1", "seq": 48213, "ts": 1760262000123, "client_msg_id": "..." }

The client shows the message optimistically, keyed by client_msg_id, and swaps in seq when the 201 arrives. Retries with the same key return the original message.

WebSocket delivery contract. One socket per device: wss://gw.example.com/ws?token=.... Server frames:

{ "type": "message.new", "channel_id": "C1", "seq": 48213, "message": {...} }
{ "type": "message.updated" | "message.deleted", "channel_id": "C1", "seq": 48250, "target_seq": 48213 }
{ "type": "read.updated", "channel_id": "C1", "last_read_seq": 48213 }   // from another device
{ "type": "channel.hint", "channel_id": "C7", "latest_seq": 902 }        // large channels, see deep dive

Every event in a channel carries a channel seq. The client tracks last_seen_seq per channel; a gap means "fetch the missing range". Client sends ping every 30 s and ack frames are not needed: correctness comes from seq gaps plus history fetch, not from per-message acks.

Fetch history (cursor by seq):

GET /v1/channels/{id}/messages?before_seq=48213&limit=50
GET /v1/channels/{id}/messages?after_seq=48100&limit=200
GET /v1/channels/{id}/threads/{root_seq}/replies?after_seq=0
→ { "messages": [...], "has_more": true }

Recovery contract (reconnect):

POST /v1/sync { "cursors": { "C1": 48100, "C7": 880 }, "since_event_id": "E123" }
→ { "channels": { "C1": { "latest_seq": 48213, "unread": 12, "mentions": 1 } }, "events": [...], "resync_required": ["C7"] }

The server returns small gaps inline; for large gaps it returns only counts and the client pages history lazily when the user opens the channel.

Supporting: POST /v1/channels/{id}/read {last_read_seq}, POST /v1/files (returns presigned upload URL), GET /v1/search?q=...&in=C1&from=U2, POST /v1/channels, POST /v1/channels/{id}/members, PUT /v1/messages/{id} (edit), DELETE /v1/messages/{id}.

Storage trade-off: a wide-column store gives cheap appends and range scans by (channel, seq) at petabyte scale, but no cross-row transactions, so seq is assigned elsewhere (next section). Slack itself runs messages on sharded MySQL (Vitess) keyed by channel, which also works: the key choice (partition by channel, order by seq) matters more than the engine.

Query stories: open channel → latest 50 by (channel, bucket) desc; sidebar → user_channels by user + channels.last_seq for unread dots; thread view → replies by (channel, root_seq); deep link → point read by (channel, seq).

flowchart LR
  C[Clients] --> LB[Load balancer]
  LB --> GW[WebSocket gateways]
  LB --> API[Message API]
  API --> CS[Channel sequencer]
  API --> MDB[(Message store)]
  API --> META[(Channel and member DB)]
  API --> K[Event log Kafka]
  K --> FO[Fanout service]
  FO --> PR[(Presence and routing Redis)]
  FO --> GW
  K --> PUSH[Push notifier]
  K --> IDX[Search indexer]
  IDX --> ES[(Search cluster)]
  C --> CDN[CDN]
  CDN --> S3[(Object storage)]

1. Connect. The client opens a WebSocket to a gateway. The gateway authenticates, then registers user_id → (gateway_id, conn_id) in the presence/routing Redis with a 60 s TTL refreshed by heartbeats. It calls /v1/sync to catch up.
2. Flow 1: DM. POST hits the Message API. It checks membership, gets the next seq for the DM channel from the sequencer, writes the message (quorum write), appends to Kafka partitioned by channel_id, and returns 201. The fanout service looks up the two members' connections in Redis and pushes message.new to the right gateways. If the recipient has no live connection, the push notifier sends APNs/FCM.
3. Flow 2: small group channel (≤ ~1k members). Same write path. Fanout reads the member list (cached per channel, invalidated on join/leave), intersects with online users, groups connections by gateway and sends one batched frame per gateway. Gateway pushes to each local socket.
4. Flow 3: large channel (> ~1k members). Same write. Fanout does not push the full message to every member. It publishes to a channel topic that gateways subscribe to only if they hold a socket currently viewing that channel; everyone else gets a coalesced channel.hint (latest seq) at most once every few seconds and pulls the message when they open the channel.
5. Read. Opening a channel calls history by seq from the message store, served through a per-channel cache of the latest ~200 messages.

Caching boundaries: latest-N messages per hot channel in Redis (write-through on send, invalidated on edit/delete); channel member lists in fanout memory with versioned invalidation; user sidebar (channel list + last_read) in Redis; file bytes at the CDN with signed URLs. Never cache permission decisions longer than the membership version they were computed from.

Realtime delivery and connection management. Gateways are stateful and dumb: hold sockets, forward frames, no business logic. Each host handles ~100k sockets with epoll and ~20 KB per connection (≈2 GB RAM). Routing state lives in Redis: conn:{user_id} → set of gateway_id:conn_id. Fanout groups recipients by gateway and sends one RPC per gateway per message, so a 300-recipient message costs ~30 RPCs, not 300. On deploy or host loss, sockets drop; clients reconnect with jittered backoff (1 s, 2 s, 4 s, cap 30 s, ±50%) so 100k clients do not stampede. Gateways drain over 5 minutes during deploys by sending a reconnect frame in batches. A frame that fails to deliver is not retried by the server: the client's seq gap detection and /v1/sync repair it.

Large channel fanout: push vs. pull threshold.

A 300k-member #announcements post with 100k online would be 100k frames in one burst; with viewer-only push it is maybe 5k frames plus one hint per online member, batched with other channel hints into a single sidebar update every 2–5 s. Unread badges for large channels come from comparing channels.last_seq with last_read_seq at read time instead of incrementing a counter per member. Mentions are the exception: @here/@channel in a big channel are rate-limited and admin-gated, and a direct @user mention is always pushed.

Message ordering and delivery guarantees. Each channel gets a dense, monotonically increasing seq from a sequencer: INCR seq:{channel_id} on a Redis shard owned by the channel (with the value fenced and persisted via channels.last_seq in MySQL on failover, skipping ahead by 1,000 to avoid reuse). Per-channel order is all a chat needs; there is no global order. The API assigns seq, writes the message, then publishes; Kafka is partitioned by channel_id, so consumers see each channel's events in order. Guarantees: at-least-once delivery to devices, deduplicated on the client by message_id, made complete by gap detection (client has 48210 and receives 48213 → fetch 48211–48212). Sends are idempotent via client_msg_id stored with a 24 h unique index. If the API crashes after taking a seq but before writing, the seq stays a hole; the client sees a gap, fetches, finds nothing, and moves on (history returns missing: [48211] as an explicit tombstone after a 10 s grace).

Multi-device sync and unread tracking. Read state is a single integer per (user, channel): last_read_seq. Unread count = messages with seq > last_read_seq (excluding own), computed from the message store and capped at "99+". When a device marks read, the server only moves the cursor forward (max(old, new)), writes it to Redis, and broadcasts read.updated to the user's other devices so their badges clear. Mention counts are tracked separately in user_mentions(user_id, channel_id, seq), since mentions must survive even when the badge is muted. Mobile push is suppressed if any device marked the channel read within the last 30 s or the desktop is active.

Deletes, media and search. Deletes are tombstones: the row keeps deleted=true, text is cleared, and a message.deleted event with a new seq propagates through the same path, so offline clients apply it on sync. Caches drop the entry; the search indexer deletes the document; thread roots show "This message was deleted" when replies exist. Retention policies run as a background job deleting by partition bucket. Files: client requests POST /v1/files, uploads straight to S3 with a presigned URL (multipart above 100 MB), then sends the message with file_id. An async pipeline virus-scans, generates thumbnails and extracts text for search; downloads use short-lived signed CDN URLs checked against channel membership.

Search: the indexer consumes Kafka and writes documents {workspace_id, channel_id, message_id, author, text, ts}. At query time the service loads the user's channel IDs (cached, up to tens of thousands) and filters with a terms query on channel_id, plus public channels in the workspace. Large enterprises get dedicated indexes (also needed for residency); small workspaces share indexes routed by workspace_id. Indexing lag target: under 5 s. Ranking mixes BM25 with recency and the user's affinity to the channel and author.

• Per-channel seq gives simple gap detection and consistent order, but the sequencer is a hot spot per channel; one extremely busy channel is capped by a single Redis key (~100k ops/s, far above real chat rates).
• Viewer-only push for large channels cuts fanout by 10–50×, but members not looking see new messages a few seconds late and pay a fetch when they open the channel.
• Read-time unread counts avoid N writes per message, but cost a count query per channel at sidebar load; cap and cache them.
• Wide-column message storage scales appends cheaply but gives up transactions and secondary indexes; edits, reactions and reply counts need careful idempotent updates.
• No server-side delivery acks keeps gateways stateless, but recovery depends on clients doing gap fetches correctly.
• Shared search indexes are cheaper for small tenants; dedicated ones isolate noisy neighbours and satisfy residency, at higher cost.

Mid-level (L4)

• Clear requirements and estimates; WebSockets for real-time, HTTP for send and history.
• A message table partitioned by channel and ordered by time or seq, with pagination.
• A working send → store → deliver flow and mobile push for offline users.

Senior (L5)

• Per-channel sequence numbers, idempotent sends, client gap detection and a reconnect sync contract.
• Gateway routing via a presence store, batched fanout per gateway, and backoff on reconnect.
• Push vs. pull for large channels with a concrete threshold; read cursors instead of per-message unread rows.
• Tombstones for deletes, presigned uploads, ACL-filtered search.

Staff (L6+)

• Quantifies fanout amplification and shows how hints, coalescing and @channel controls bound it.
• Explains failure modes: sequencer failover and seq holes, gateway drains, Kafka lag, and what the client sees in each.
• Covers tenant isolation, data residency, retention and legal hold, and how they shape sharding and the search index.
• Argues storage engine choice from access patterns rather than brand, and names what would change at 10× scale.