High‑level design of WhatsApp

Backend Fundamentals · System Design · 8 min read

1. What problem does this solve?

WhatsApp is a global instant‑messaging platform that must deliver messages, media, and presence information to millions of users with low latency, high reliability, and end‑to‑end security.
The core problem is how to build a distributed, real‑time, fault‑tolerant system that scales to billions of messages per day while keeping data private and ensuring a smooth user experience.

• Why it matters

  • Users expect instant delivery; delays erode trust.
  • Data breaches would expose sensitive conversations.
  • The service must survive node failures, network partitions, and traffic spikes.

• Consequences of ignoring the problem

  • Message loss or duplication → user frustration.
  • Poor scalability → outages during peak times.
  • Weak security → legal penalties and brand damage.

• Real‑world appearance

  • Messaging apps (Telegram, Signal, Slack).
  • IoT telemetry pipelines.
  • Real‑time collaboration tools (Google Docs, Figma).

2. Mental model

Analogy: The Postal Service

Just as a postal system must route, store, and deliver mail reliably, WhatsApp routes, stores, and delivers messages while preserving privacy.

Engineering model

1. Client → API Gateway (HTTPS/WebSocket)
2. API Gateway → Message Broker (Kafka)
3. Broker → Microservices (Message, Presence, Media)
4. Microservices → Datastores (PostgreSQL, MongoDB, Redis)
5. Microservices → Clients (WebSocket, Push)

3. Deep explanation

Internals

Important terminology

• End‑to‑End Encryption (E2EE) – only sender/receiver can read the payload.
• At‑least‑once delivery – message may be delivered multiple times but never lost.
• Offline message queue – store messages until the recipient is online.
• Presence – online/offline status, typing indicators.
• Sharding – partitioning data across nodes.
• Consistent hashing – balanced distribution of shards.

Trade‑offs

Failure modes

• Broker outage → message loss if not replicated.
• Database partition → stale reads, write failures.
• Encryption key loss → data unrecoverable.
• Push notification failure → missed offline messages.

Limitations

• E2EE prevents server‑side content filtering.
• Media size limits require CDN and transcoding.
• Global latency: users in remote regions may experience higher RTT.

Where it fits

The design sits at the core of a microservice architecture: each service is independently deployable, scales horizontally, and communicates via a broker. It interacts with authentication, billing, analytics, and compliance services.

4. How it works

Step‑by‑step flow

1. User A composes a message.
2. Client encrypts payload with Signal Protocol → ciphertext.
3. Client sends POST /messages to API Gateway (HTTPS).
4. Gateway authenticates JWT, rate‑limits, forwards to Message Service.
5. Message Service writes metadata to PostgreSQL (sender, receiver, timestamp).
6. Service publishes event message.created to Kafka.
7. Presence Service consumes event, checks if User B is online via Redis.
8. If online → Kafka → WebSocket Service → pushes to User B’s socket.
9. If offline → Kafka → Offline Queue Service → stores in MongoDB.
10. When User B reconnects → WebSocket Service pulls from MongoDB, sends via WebSocket.
11. Media attachments are uploaded to Media Service → stored in S3 → URL returned to client.
12. All services log to ELK stack for observability.

Mermaid diagram

sequenceDiagram
    participant UA as User A
    participant UA_Client as Client
    participant GW as API Gateway
    participant MsgSvc as Message Service
    participant DB as PostgreSQL
    participant Kafka as Kafka
    participant PresSvc as Presence Service
    participant WS as WebSocket Service
    participant OfflineSvc as Offline Queue
    participant DB2 as MongoDB
    participant UB as User B
    participant UB_Client as Client

    UA->>UA_Client: Compose message
    UA_Client->>UA_Client: Encrypt (Signal)
    UA_Client->>GW: POST /messages
    GW->>MsgSvc: Forward request
    MsgSvc->>DB: INSERT message meta
    MsgSvc->>Kafka: Publish message.created
    Kafka->>PresSvc: Consume message.created
    PresSvc->>Redis: Check online status
    alt User B online
        PresSvc->>Kafka: Publish presence.online
        Kafka->>WS: Consume presence.online
        WS->>UB_Client: Push via WebSocket
    else
        PresSvc->>Kafka: Publish presence.offline
        Kafka->>OfflineSvc: Consume presence.offline
        OfflineSvc->>DB2: Store message
    end
    UB_Client->>UB: Reconnect
    UB_Client->>WS: Subscribe
    WS->>OfflineSvc: Pull offline messages
    OfflineSvc->>DB2: Retrieve
    OfflineSvc->>WS: Push to UB_Client

5. Practical implementation

Below is a minimal Node.js/TypeScript example that shows:

• An Express API to receive a message.
• Publishing to Kafka.
• A WebSocket server that consumes the Kafka topic and pushes to connected clients.
• Redis for presence.

Prerequisites: Docker Compose with Kafka, Redis, and PostgreSQL.

5.1 docker-compose.yml

version: '3.8'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.5.0
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
  kafka:
    image: confluentinc/cp-kafka:7.5.0
    depends_on: [zookeeper]
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
  redis:
    image: redis:7
  postgres:
    image: postgres:15
    environment:
      POSTGRES_USER: whatsapp
      POSTGRES_PASSWORD: whatsapp
      POSTGRES_DB: whatsapp

5.2 src/server.ts

import express from 'express';
import { Kafka } from 'kafkajs';
import { Server } from 'ws';
import { createClient } from 'redis';
import { Pool } from 'pg';

const app = express();
app.use(express.json());

// PostgreSQL pool
const pool = new Pool({
  user: 'whatsapp',
  host: 'postgres',
  database: 'whatsapp',
  password: 'whatsapp',
  port: 5432,
});

// Redis client for presence
const redis = createClient();
redis.on('error', err => console.error('Redis error', err));
await redis.connect();

// Kafka producer
const kafka = new Kafka({ brokers: ['kafka:9092'] });
const producer = kafka.producer();
await producer.connect();

// WebSocket server
const wss = new Server({ port: 8080 });
const clients = new Map<string, WebSocket>(); // userId -> ws

wss.on('connection', ws => {
  // Expect first message to be userId
  ws.once('message', async msg => {
    const { userId } = JSON.parse(msg.toString());
    clients.set(userId, ws);
    await redis.set(`presence:${userId}`, 'online', { EX: 60 });
    console.log(`User ${userId} connected`);
  });

  ws.on('close', async () => {
    // Find userId by ws
    for (const [uid, socket] of clients.entries()) {
      if (socket === ws) {
        clients.delete(uid);
        await redis.del(`presence:${uid}`);
        console.log(`User ${uid} disconnected`);
        break;
      }
    }
  });
});

// Kafka consumer for message delivery
const consumer = kafka.consumer({ groupId: 'ws-group' });
await consumer.connect();
await consumer.subscribe({ topic: 'messages', fromBeginning: false });

consumer.run({
  eachMessage: async ({ topic, partition, message }) => {
    const payload = JSON.parse(message.value!.toString());
    const { to, from, text } = payload;
    const ws = clients.get(to);
    if (ws) {
      ws.send(JSON.stringify({ from, text }));
      console.log(`Delivered to ${to}`);
    } else {
      console.log(`User ${to} offline, message queued`);
      // In production, push to offline queue
    }
  },
});

// API endpoint
app.post('/messages', async (req, res) => {
  const { from, to, text } = req.body;
  // Persist metadata
  await pool.query(
    'INSERT INTO messages (sender, receiver, body, ts) VALUES ($1,$2,$3,now())',
    [from, to, text]
  );
  // Publish to Kafka
  await producer.send({
    topic: 'messages',
    messages: [{ value: JSON.stringify({ from, to, text }) }],
  });
  res.status(202).send({ status: 'queued' });
});

app.listen(3000, () => console.log('API listening on 3000'));

Key lines

• producer.send → decouples API from delivery.
• clients map → naive in‑memory presence; replace with Redis in prod.
• consumer.run → pushes to WebSocket if online, otherwise would enqueue.

Run: docker compose up then ts-node src/server.ts.

6. Production considerations

7. Common mistakes

8. Senior engineer perspective

Architecture decisions

• Microservices vs. Monolith – microservices give independent scaling but add operational overhead.
• Kafka vs. Redis Streams – Kafka offers durability and replay; Redis Streams is lighter but less durable.
• PostgreSQL vs. NoSQL for messages – PostgreSQL ensures ACID; NoSQL offers schema flexibility.
• Edge CDN for media – reduces latency, offloads origin servers.
• Push vs. WebSocket – Push for mobile, WebSocket for desktop/web.

Trade‑offs

• Latency vs. Reliability – WebSocket gives instant delivery but requires keep‑alive; push notifications have higher latency but work offline.
• Cost vs. Performance – CDN + object storage reduces load on origin but adds cost.
• Complexity vs. Flexibility – Kafka cluster is complex but supports replay; simpler queues trade durability.

Bottlenecks

• Kafka broker throughput – need enough partitions.
• Redis memory – presence data can grow; use eviction policies.
• Media transcoding – CPU‑heavy; offload to worker pool.
• Database write amplification – batch inserts to reduce lock contention.

Failure scenarios

• Kafka partition loss – enable replication factor ≥ 3.
• Redis cluster split‑brain – use Sentinel or Cluster mode.
• Media CDN outage – fallback to origin S3.
• Key compromise – implement forward secrecy, key rotation.

Monitoring

• Latency histograms for API, Kafka, WebSocket.
• Throughput counters per topic.
• Presence churn metrics.
• Error rates for message delivery.
• Resource utilization (CPU, memory, network).

When NOT to use this approach

• Small‑scale internal chat – a monolith with SQLite may suffice.
• Low‑traffic IoT telemetry – a simple MQTT broker could be cheaper.
• Highly regulated data – may need on‑premises storage, not cloud CDN.

9. Interview questions

10. Hands‑on task

Implement a simple presence service:

1. Extend the WebSocket server to broadcast presence changes (online, offline, typing).
2. Store presence in Redis with TTL.
3. Create an HTTP endpoint /presence/:userId that returns the current status.
4. Write a small test script that simulates two clients connecting, sending a typing event, and disconnecting.
5. Verify that the presence endpoint reflects the correct state.

11. Quick revision

• WhatsApp is a distributed, real‑time system built on microservices.
• End‑to‑end encryption (Signal Protocol) protects payloads; server never sees plaintext.
• Kafka decouples API from delivery, provides replay and ordering.
• Redis is used for fast presence lookups; TTL ensures stale data expires.
• WebSocket delivers instant messages; push handles offline users.
• Offline queue (MongoDB) stores messages until the recipient reconnects.
• Media lives in S3 + CDN; transcoded on upload.
• Scalability achieved via partitioning, sharding, and autoscaling.
• Reliability via replication, idempotency, dead‑letter queues.
• Observability: tracing, metrics, logs.
• Common pitfalls: ignoring key rotation, single DB, no offline handling, duplicate messages.
• Senior decisions: Kafka vs. Redis Streams, CDN vs. direct S3, microservice boundaries, cost vs. performance trade‑offs.

12. References

• WhatsApp Business API
• Signal Protocol
• Kafka Documentation
• Redis Documentation
• Node.js Express
• TypeScript Handbook
• WebSocket RFC 6455
• OpenTelemetry
• Prometheus
• ELK Stack