Timeouts — Analogy: Imagine a waiter in a busy restaurant who keeps a customer’s order in the kitchen. If the kitchen takes forever, the waiter will eventually give up and ask the customer…
Retries — If you never retry, the letter may never reach your friend. If you retry too aggressively, you overload the post office.
Exponential backoff — Exponential backoff mitigates these issues by spacing retries increasingly farther apart, giving the target service time to recover while still giving the caller a chance to…
Jitter — Motivation When a backend system talks to another service, the round‑trip time (RTT) is rarely constant. Network congestion, load spikes, GC pauses, or even the way a database…
Circuit breakers — Motivation In a distributed system, a service may call another service, a database, or an external API. If the callee is slow, overloaded, or down, the caller can become blocked…
Bulkheads — Motivation In a distributed backend, a single failure can ripple through the entire system. If one component consumes too many resources (CPU, memory, DB connections, external API…
Graceful Degradation — In a distributed backend, components rarely work perfectly all the time. When a downstream service (e.g., a recommendation engine, a payment gateway, or a cache) fails or becomes…
Failover — Motivation In a distributed backend, a single point of failure (SPOF) can bring the whole service down. Failover is the mechanism that detects a failure and automatically switches…
Health checks — Think of a health check like a doctor’s check‑up for a server.
Liveness probes — Motivation In a distributed backend, a process can appear to be running (the OS shows a PID, the container is up) but its internal state may be corrupted, a deadlock may have…
Readiness probes — Readiness probes let the orchestrator (e.g., Kubernetes) know when a pod is ready to receive traffic, separating liveness (is the process alive?) from readiness (is it ready to…
Error budgets — Balancing reliability and feature velocity In a production system, every change can introduce bugs or performance regressions. Teams need a systematic way to decide when to ship…
SLO — Motivation In a real‑world backend, multiple teams ship features, fix bugs, and deploy changes continuously. Clients expect a certain level of reliability, but developers often…
SLA — In a production backend, you have many moving parts: databases, caches, message queues, external APIs, and the application code itself. Clients (web browsers, mobile apps, other…
SLI — Motivation When you run a backend service, you need to know how well it is performing. Clients expect low latency, high availability, and correct results. Without a quantitative…