Design WhatsApp (a Chat System) — System Design Interview Walkthrough

The thing that makes a chat system hard is real-time delivery: how do you get a message to a recipient instantly when they're online, reliably when they're offline, to all their devices, and to everyone when it's a group? Persistent connections (WebSocket) plus a connection registry, a durable per-recipient inbox, and pub/sub routing answer all of it. Everything else — receipts, ordering, history — hangs off that. This applies the standard system design framework to a real-time problem.

Unlike a URL shortener (request/response, read-heavy), chat is stateful and push-based — the server has to reach out to clients. That's the whole challenge, and it is why a CRUD-only candidate struggles here.

1. Requirements

Functional: 1:1 messaging, group messaging, online/last-seen presence, delivery + read receipts, message history, push notifications when the recipient is offline, and multi-device support. Out of scope (say so): voice/video calls; media is stored in an object store + CDN and the message carries a URL.

Non-functional, quantified: messages feel instant (sub-500 ms delivery for online users); highly available; durable (never lose a message); ordered within a conversation; end-to-end encrypted; and able to handle hundreds of millions of concurrent connections.

2. Capacity estimation

Numbers make the constraint obvious. Assume 1 billion users, ~200 million concurrently connected, and ~50 billion messages/day.

• Message write rate: 50B / 86,400 ≈ ~580k messages/sec average, multiplied by inbox fan-out (a copy per recipient) and peak factor — call it well over a million writes/sec.
• Storage: at ~100 bytes per message, 50B/day ≈ ~5 TB/day, or ~9 PB over five years of retained history (the per-recipient inbox copies are transient — TTL'd once delivered). Large, but it's ordinary disk in a wide-column store; not the hard part.
• Bandwidth: 5 TB/day ÷ 86,400 ≈ ~58 MB/s ingress, and about the same egress. That figure is the whole point — a few hundred MB/s is nothing for a fleet this size, so the bottleneck is emphatically not bandwidth.
• Connections: a tuned gateway holds ~1–2 million long-lived WebSocket connections, so 200M concurrent ÷ ~1M ≈ ~200 gateway servers just to hold the sockets. Size the fleet to ~70% utilization (~285) so a dead gateway's million sockets have somewhere to reconnect.
• Registry memory: the session registry is just userId/deviceId → gatewayId — ~200M entries × ~100 bytes ≈ ~20 GB of Redis, trivially sharded. That's the one in-memory structure that must scale with connections; everything else is on the sockets.

The headline: connections are the capacity constraint — not bandwidth, not CPU. The whole design is organized around holding hundreds of millions of cheap, mostly-idle sockets and routing between them. The bandwidth math above is what proves that, instead of just asserting it.

3. The core challenge: connections

HTTP request/response can't push. So clients hold a persistent WebSocket to a fleet of connection (gateway) servers. The problem this creates: when User A sends to User B, which server holds B's connection?

• Maintain a session registry (Redis): userId/deviceId → gatewayServerId, updated on connect/disconnect.
• To deliver, find B's gateway and route the message there, which pushes it down B's socket.

This connection layer is the heart of the design — call it out first. Here is the system you're building:

4. The API (a message protocol, not REST)

Because the transport is a WebSocket, the "API" is a small catalog of message types in both directions, not REST endpoints:

client → server:  sendMessage {chatId, clientMsgId, ciphertext}
                  ack {messageId}              // delivered / read receipts
                  createChat {participants}
                  getLastSeen {userId}
                  ping                         // heartbeat

server → client:  newMessage {chatId, messageId, seq, senderId, ciphertext}
                  ackStatus {messageId, status: sent|delivered|read}
                  presence {userId, status}
                  pong

A clientMsgId on send makes the operation idempotent — a retried sendMessage after a flaky connection is deduplicated server-side instead of producing two messages.

5. Sending a message (the send path)

The receipt states above (sent → delivered → read) fall straight out of the path:

1. A's client sends sendMessage over its WebSocket to its gateway.
2. The gateway persists the message to the message store and writes an inbox entry per recipient (durability before delivery), and assigns a per-conversation sequence number. A is told sent.
3. The gateway publishes to B's channel (Section 7).
   • B online: B's gateway pushes newMessage down B's socket; B acks; the inbox entry is marked delivered (and deleted), A sees delivered.
   • B offline: the message stays in B's inbox; a push notification (APNs/FCM) is queued. B drains its inbox on reconnect.
4. When B reads the chat, B sends a read ack; A sees read.

6. Data model & storage

Chat is write-heavy (every message is a write, fanned out per recipient) with time-ordered reads per conversation — a textbook fit for a wide-column store (Cassandra), plus a few supporting tables:

The inbox is the table that makes "we never lose a message" true: the message is durably written there before any delivery attempt, and only removed once the device acks. Without it, an offline (or crashing) recipient loses messages.

Why a wide-column store and not MySQL. The workload is a firehose of tiny writes plus ordered range-reads of one conversation's history — exactly what an LSM-tree engine (Cassandra, HBase, Bigtable) is built for. It buffers writes in memory and flushes them to disk in sorted runs, so a million small appends a second don't each pay a random-write seek, and "the last N messages of this conversation" is a cheap sequential scan of one partition. A row-store would thrash on per-message writes. Durability for the live message store is replication (factor ~3 across AZs) — small, hot rows reconstruct cheaply that way — while the bulk media blobs and cold message archives lean on erasure coding (Reed–Solomon: k data + m parity shards survive any m losses at a fraction of 3× replication's storage cost). Either way, a message is never single-copy.

7. Routing at scale (Bad → Good → Great)

"Look up B's gateway and connect to it" works on a whiteboard but not at 200 gateways. Frame the progression:

• Bad — naive load balancer: B's messages hit a random gateway that doesn't hold B's socket. Broken.
• Good — consistent hashing via a coordinator: keep the user → gateway map in ZooKeeper/etcd; gateways open direct connections to each other. Works, but every gateway must discover and connect to every other.
• Great — Redis Pub/Sub channels: each connected user subscribes its gateway to a channel user:{userId}. To deliver, the sending gateway just publishes to user:{B}; whichever gateway holds B's socket is subscribed and pushes it down. Gateways never need each other's addresses. A single Redis node handles ~100k+ publishes/sec; shard channels across a cluster.

8. Ordering, receipts & gap recovery

Why assign a sequence number at all — why not just sort by arrival time? Because server-receive timestamps don't order messages reliably. Picture two messages into the same chat: M1 reaches the gateway at T1, M2 at T2 just after. If they take different paths through the fleet, recipient B can receive M2 before M1 — so B's transcript disagrees with A's, or with B's other device. Wall clocks drift between servers and can't rescue it. The fix is to stop trusting time and assign an explicit order:

• Ordering is guaranteed per conversation by a monotonic sequence number assigned server-side (via a per-conversation Redis INCR); clients render in sequence order, not arrival order. Global ordering across all conversations is unnecessary and far more expensive — per-conversation is what users actually perceive, and it's what keeps every one of a user's devices consistent. (Some systems take the other side of the trade: accept NTP-synced server timestamps and the rare out-of-order render, betting that showing a message instantly beats perfect order. Name it; per-conversation sequencing is the stronger default once multiple devices must converge.)
• Delivery is at-least-once + idempotent client dedup by clientMsgId — simpler and safer than exactly-once.
• Gap recovery: each newMessage carries its seq; if a client sees 5 then 7, it knows 6 is missing and requests a re-sync. Heartbeats (ping/pong every ~20–30s) also carry the latest seq, so a client that missed messages while its socket was half-dead detects and recovers.

9. Presence at scale

True real-time presence for everyone is expensive — writing "online" on every heartbeat would be millions of writes/sec. The trick is to write lastSeen only on disconnect (a conditional write), and answer "online" live by checking whether a socket exists in the session registry. "Last seen 5 minutes ago" needs no constant writes; "online" is a registry lookup. That asymmetry is what makes presence affordable at all.

Two optimizations stack on top, and naming them shows you've thought past the happy path:

• Debounce the "online" broadcast. When a user connects, wait a few seconds before telling their contacts they're online — flaky mobile networks flap connect/disconnect constantly, and you don't want to fan out a presence storm for a socket that drops a second later.
• Pull presence for the viewport, don't push it to everyone. A client doesn't subscribe to all 1,000 contacts' statuses; it pulls status for the handful of chats currently on screen (plus on app start and when opening a new chat) and tolerates a slightly stale "last seen." Presence is a pull-on-demand read, not a broadcast.

10. Group messaging (the fan-out)

A group message goes to every member. For typical groups (WhatsApp caps membership, historically around 256), fan out on send: write once to the message store, then write an inbox entry per member and publish to each member's channel. Past a threshold (say ~25+ active participants), publishing N times per message gets expensive, so switch to a group channel chat:{chatId} that members subscribe to — one publish instead of N. Naming that threshold, and the switch, is what separates a strong answer.

11. Encryption & multi-device

• End-to-end encryption. WhatsApp uses the Signal protocol (the double ratchet): messages are encrypted on the sender's device and only the recipient's devices can decrypt them, so the server stores and routes ciphertext it cannot read. This keeps the server a dumb relay for bodies — but it means features like server-side search aren't possible, and key exchange must be managed per device.
• Multi-device. Each device has its own keys, so the sender encrypts a copy of each message per recipient device, and the inbox and receipts become per-device (the (userId, deviceId) key above). A device that's been offline syncs its inbox on reconnect.
• Media is encrypted, uploaded to an object store via a presigned URL (bypassing the gateway's bandwidth), and the message carries the URL + decryption key; recipients fetch from the CDN.

12. Failure modes interviewers probe

13. Trade-offs, follow-ups & what to say at each level

Trade-offs to name unprompted: durability vs latency (persist-before-deliver costs a write on the hot path — the right trade for messaging); at-least-once + dedup over exactly-once; presence accuracy vs write cost; fan-out-on-send vs a group channel past the threshold.

Follow-ups a senior interviewer asks next: offline-push topology (a dedicated Notification service forwards to APNs/FCM, which deliver to the device; users opt in per device), message search (hard under E2EE — done client-side), media pipeline (thumbnails, transcoding), typing indicators (ephemeral, fire-and-forget, never stored), and multi-region (route users to a home region; cross-region delivery via the message bus).

What to say at each level. Mid: WebSocket + session registry + persist-then-deliver + 429-style backpressure. Senior: the inbox model, the LSM-store choice, pub/sub routing, per-conversation ordering and gap recovery, presence-on-disconnect, and the group threshold. Staff+: persist-before-deliver as an explicit durability-vs-latency trade, erasure-coded media/archive durability, multi-device + E2EE key management, multi-region, the reconnect-storm mitigation, and hot-partition sharding — unprompted.

Why interviewers use this one

It moves you off the comfortable request/response model into stateful, push-based, real-time territory — connection management, durable delivery, ordering, and encryption — which a CRUD-only candidate hasn't thought about. It's the same 7-step framework, applied to a problem where the server must reach the client. For a different flavor of fan-out, compare the read-side fan-out in designing Instagram.

Amit Singh is a Senior SDE at Amazon, a Claude Certified Architect, and the instructor at AlgoEngineer. Real-time systems are a favorite in senior loops; we run them as live mock interviews in our System Design course.