How Message Queues Handle Traffic Spikes

System Design · Days 41–44 · 3 min read

Video Reference: How Message Queues Handle Traffic Spikes

1. Asli Problem Kya Hai? (The Core Why)

Picture Swiggy during the 5‑am breakfast rush. Orders keep pouring in, but the backend services (order‑processor, payment, inventory) can’t keep up instantly.
If we process everything synchronously, each request waits for the whole chain to finish → latency spikes.
A sudden 200% surge (say a flash sale on Zerodha) can crash the API layer or cause timeouts, leading to bad user experience and revenue loss.
So, we need a mechanism that buffers bursts, decouples producers from consumers, and keeps the system responsive while still guaranteeing throughput.

2. Core Architecture & Mechanisms

2.1 Producer & Consumer Decoupling

Producers (frontend, mobile apps) publish messages to a queue (e.g., Kafka, RabbitMQ) without waiting for the consumer to finish.
This gives the system a buffer that absorbs spikes.

2.2 Back‑pressure & Rate Limiting

The queue can throttle incoming messages or move them to a dead‑letter queue if consumers are lagging.
This prevents the whole pipeline from getting clogged.

2.3 Parallelism & Scaling

Multiple consumer instances (micro‑services) read from the same topic/queue.
Horizontal scaling is easy: just add more workers.
Each worker processes a partition (Kafka) or a queue slot (RabbitMQ), giving linear throughput.

2.4 Exactly‑Once vs At‑Least‑Once

Kafka’s offset mechanism ensures at‑least‑once delivery; idempotent consumers or transactional writes give exactly‑once semantics.
In a payment system, you’d opt for exactly‑once to avoid double‑charging.

2.5 Persistence & Durability

Messages are stored on disk (Kafka log) or replicated across nodes (RabbitMQ mirrored queues).
So a node failure doesn’t lose data—important for critical flows like IRCTC seat booking.

2.6 Monitoring & Metrics

Key metrics: queue depth, consumer lag, latency per message.
Alerting on a sudden spike in queue depth helps pre‑empt outages.

3. Architecture Flow

graph TD
    A[User Request (Swiggy Order)] -->|Publish| B[Message Queue (Kafka)]
    B -->|Read| C1[Order Processor]
    B -->|Read| C2[Payment Service]
    B -->|Read| C3[Inventory Service]
    C1 -->|Ack| B
    C2 -->|Ack| B
    C3 -->|Ack| B
    B -->|Forward| D[Response to User]

4. Production Case Study

Netflix uses Kafka extensively for its micro‑service ecosystem.
During a new season launch, viewership can jump 4× overnight.
Kafka’s partitioning allows Netflix to spin up hundreds of consumer instances to process watch events in real‑time.
They also employ Kafka Streams to aggregate metrics (e.g., concurrent viewers) and push alerts to the monitoring dashboard.
Result: zero downtime, consistent user experience, and real‑time analytics.

5. Trade-offs (Pros vs Cons)

Pros	Cons
Decoupling – producers and consumers operate independently, easing maintenance.	Complexity – added infrastructure and operational overhead.
Scalability – horizontal scaling of consumers increases throughput linearly.	Latency – extra hop (queue) can add milliseconds to each request.
Fault Tolerance – persistence and replication protect against data loss.	Data Ordering – not guaranteed across partitions; may need additional logic.
Back‑pressure – queue depth signals consumer load, preventing system overload.	Resource Utilization – requires dedicated hardware/cluster, increasing costs.
Event Replay – ability to reprocess messages for debugging or auditing.	Operational Burden – need for monitoring, alerting, and capacity planning.

6. System Design Interview Cheat Sheet

Decouple producers and consumers using a queue; handle spikes with buffering.
Use partitioning (Kafka) or mirrored queues (RabbitMQ) for horizontal scaling.
Implement back‑pressure: throttle producers or move to dead‑letter queues when lag > threshold.
Choose exactly‑once semantics for idempotent operations (payments); otherwise, at‑least‑once is fine.
Monitor queue depth and consumer lag; set alerts for rapid growth.