Reverse proxies

Backend Fundamentals · System Design · 9 min read

1. What problem does this solve?

Motivation
When a client talks to a backend service, the client usually talks to a single endpoint (e.g., api.example.com). In reality, that endpoint is often a gateway that hides a cluster of servers, handles TLS, performs load balancing, and can add security or caching layers. A reverse proxy is the software that sits in that gateway position.

What problem exists

Where it appears in real backend systems

• API gateways (e.g., Kong, Tyk, AWS API Gateway)
• Edge servers / CDNs (e.g., Cloudflare, Fastly)
• In‑cluster service meshes (Envoy, Istio)
• Classic load balancers (NGINX, HAProxy, Traefik)
• Web servers that also act as reverse proxies (Apache, NGINX)

2. Mental model

Analogy – The Receptionist
Think of a corporate office building. Clients arrive at the front desk (the reverse proxy). The receptionist checks the visitor list, forwards the visitor to the correct department, may give a badge (add headers), and can even answer simple questions (rate limiting). The receptionist also ensures that only authorized visitors get inside (TLS termination, authentication).

Engineering model
A reverse proxy is a server that:

1. Listens on a public IP/port (e.g., 443).
2. Receives an HTTP(S) request from a client.
3. Decides which backend server should handle the request.
4. Forwards the request (possibly rewriting the URL, headers, or body).
5. Receives the backend’s response and streams it back to the client.

It can also terminate TLS, cache responses, enforce security policies, and provide observability hooks.

3. Deep explanation

Internals

Important terminology

Trade‑offs

Failure modes

Limitations

• Cannot proxy protocols that are not HTTP/HTTPS (unless custom logic).
• Adds latency (one extra hop).
• Requires careful configuration to avoid leaking sensitive headers.
• Scaling the proxy itself can become a bottleneck if not horizontally scalable.

Where it fits in a backend architecture

┌───────────────────────┐
│  Client (Browser/API) │
└────────────┬──────────┘
             │
             ▼
┌───────────────────────┐
│  Reverse Proxy (NGINX│
│  / Envoy / Traefik)   │
└───────┬───────┬───────┘
        │       │
        ▼       ▼
┌───────────────┐  ┌───────────────┐
│  Service A    │  │  Service B    │
│  (FastAPI)    │  │  (Node.js)    │
└───────────────┘  └───────────────┘

The reverse proxy sits between the public internet and the internal service mesh, providing a single entry point.

Interaction with related concepts

• Load balancer – Often the reverse proxy doubles as a load balancer.
• Service mesh – Envoy can act as both a sidecar and a gateway.
• API gateway – Adds authentication, rate limiting, and request transformation.
• CDN – Edge reverse proxies cache static assets globally.
• Observability – Prometheus metrics, OpenTelemetry traces are emitted by the proxy.

4. How it works

Step‑by‑step flow

1. Client sends an HTTPS request to api.example.com.
2. TLS handshake occurs at the reverse proxy; the client’s certificate is verified (if mutual TLS).
3. Proxy parses the request, matches the path (/users/*) to a backend pool.
4. Load balancer selects an instance (e.g., users-01.internal:8000).
5. Proxy forwards the request, rewriting the Host header to users.internal.
6. Backend processes the request and sends a response.
7. Proxy receives the response, optionally caches it, and streams it back to the client.
8. Client receives the response over the same TLS session.

sequenceDiagram
    participant C as Client
    participant P as Reverse Proxy
    participant U as Users Service

    C->>P: HTTPS GET /users/123
    P->>P: TLS Handshake
    P->>U: HTTP GET /123 (Host: users.internal)
    U->>P: HTTP 200 {body}
    P->>C: HTTPS 200 {body}

5. Practical implementation

Below is a minimal Node.js reverse proxy using http-proxy-middleware that:

• Terminates TLS
• Routes /api/* to a FastAPI backend
• Adds an X-Forwarded-For header
• Handles WebSocket upgrades

// proxy.ts
import express from 'express';
import { createProxyMiddleware } from 'http-proxy-middleware';
import https from 'https';
import fs from 'fs';
import path from 'path';

const app = express();

// 1️⃣ TLS termination
const sslOptions = {
  key: fs.readFileSync(path.join(__dirname, 'certs', 'privkey.pem')),
  cert: fs.readFileSync(path.join(__dirname, 'certs', 'fullchain.pem')),
};

// 2️⃣ Proxy middleware
const apiProxy = createProxyMiddleware('/api', {
  target: 'http://localhost:8000', // FastAPI backend
  changeOrigin: true,
  pathRewrite: { '^/api': '' }, // strip /api prefix
  onProxyReq: (proxyReq, req) => {
    // Add X-Forwarded-For header
    const clientIp = req.headers['x-forwarded-for'] || req.socket.remoteAddress;
    proxyReq.setHeader('X-Forwarded-For', clientIp as string);
  },
  ws: true, // support WebSocket
  logLevel: 'debug',
});

// 3️⃣ Mount the proxy
app.use(apiProxy);

// 4️⃣ Start HTTPS server
https.createServer(sslOptions, app).listen(443, () => {
  console.log('Reverse proxy listening on https://localhost');
});

Dockerfile

# Dockerfile
FROM node:20-alpine

WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .

# Expose HTTPS port
EXPOSE 443

CMD ["node", "proxy.ts"]

docker‑compose.yml

version: "3.8"
services:
  proxy:
    build: .
    ports:
      - "443:443"
    volumes:
      - ./certs:/app/certs
    depends_on:
      - api
  api:
    image: tiangolo/uvicorn-gunicorn-fastapi:python3.10
    volumes:
      - ./app:/app
    environment:
      - MODULE_NAME=main
      - VARIABLE_NAME=app
    expose:
      - "8000"

Run docker compose up --build and hit https://localhost/api/hello. The proxy will forward to the FastAPI backend.

6. Production considerations

7. Common mistakes

8. Senior engineer perspective

What changes at scale

• Architecture – Move from a single proxy to a gateway cluster behind a global load balancer (e.g., AWS ALB). Use a service mesh (Envoy) for intra‑cluster traffic.
• Design decisions – Choose between edge (public) vs internal proxies. Decide on TLS termination location (edge vs backend). Pick a caching strategy (CDN vs local).
• Trade‑offs – Balancing latency vs security. For example, terminating TLS at the edge reduces backend load but introduces a new trust boundary.
• Bottlenecks – CPU for TLS, memory for connection pools, network egress costs. Use hardware acceleration (AES-NI) or offload to dedicated proxies.
• Failure scenarios – Proxy crash, misconfiguration, network partition. Use redundancy, health checks, and circuit breakers.
• Monitoring – Collect metrics per backend, latency histograms, error rates. Use distributed tracing to pinpoint where latency is introduced.
• Cost/performance – Running multiple proxy instances increases compute cost. Offloading TLS to a CDN can reduce egress costs.
• When NOT to use – For a single, low‑traffic service, a simple NGINX instance may suffice. Avoid over‑engineering.

9. Interview questions

10. Hands‑on task

Task:
Create a Docker‑Compose stack that includes:

1. A Node.js reverse proxy (using http-proxy-middleware).
2. A FastAPI backend that serves /hello and /ws (WebSocket echo).
3. A simple React front‑end that calls /api/hello and opens a WebSocket to /api/ws.

Requirements

• The proxy should terminate TLS (use self‑signed certs).
• The proxy must rewrite /api/* to the FastAPI backend.
• The React app should display the response from /api/hello and echo messages over the WebSocket.
• Verify that the proxy logs the request path and client IP.

Deliverables

• docker-compose.yml
• proxy/ folder with proxy.ts and Dockerfile
• backend/ folder with FastAPI main.py
• frontend/ folder with a minimal React app
• README with instructions to run and test

11. Quick revision

• Reverse proxy sits between clients and backends, forwarding requests.
• Terminates TLS to offload CPU and centralize certs.
• Routes based on host/path, can load‑balance, rewrite URLs, and add headers.
• Health checks keep the backend pool healthy.
• Common features: caching, WAF, rate limiting, WebSocket support.
• Failure modes: misrouting, missing health checks, TLS expiry.
• Production concerns: scalability, reliability, security, observability, performance.
• Senior‑level design: multi‑AZ, service mesh, CDN edge, distributed tracing.
• Typical tech stack: NGINX, Envoy, Traefik, http‑proxy‑middleware, HAProxy.
• Key metrics: request latency, error rate, backend health, throughput.

12. References

• NGINX Reverse Proxy Documentation
• Envoy Proxy Documentation
• Traefik Documentation
• http-proxy-middleware GitHub
• FastAPI Documentation
• OpenTelemetry for Node.js
• Prometheus Node Exporter
• Let’s Encrypt Certbot