Stateless vs Stateful Services

Backend Fundamentals · Core Backend Fundamentals · 8 min read

1. What problem does this solve?

In production, most backends aim to be stateless so that any instance can serve any request. When state is required, it is externalized (database, cache, message queue) so that the service itself remains stateless.

2. Mental model

Analogy: The Post Office

• Stateless: The clerk never keeps track of the letter after processing; any clerk can handle any letter.
• Stateful: The clerk must remember the letter’s location, so only that clerk can retrieve it.

Engineering model

• Stateless: Each request is independent; the service has no persistent memory between calls.
• Stateful: The service retains data (session, cache, in‑memory state) that influences subsequent requests.

3. Deep explanation

Internals

Important terminology

• Session: A logical grouping of requests belonging to the same user.
• Sticky session: Load balancer routing that keeps a user’s requests to the same instance.
• External state store: DB, Redis, Kafka, etc. used to persist state.
• Idempotency: Ability to safely retry a request without side effects.

Trade‑offs

Failure modes

• Stateless: Failure of external store (DB, cache) can cause service errors.
• Stateful: Instance crash loses in‑memory state unless persisted; sticky sessions can cause uneven load.

Limitations

• Stateless services cannot keep per‑user data in RAM; must use external stores.
• Stateful services cannot scale beyond the memory of a single instance unless state is sharded or replicated.

Where it fits in architecture

• Stateless: API gateways, microservice endpoints, background workers.
• Stateful: Session stores, in‑memory caches, real‑time analytics engines.

Interaction with related concepts

• Caching: Stateless services often use external caches (Redis).
• Idempotency keys: Needed for stateless services to safely retry.
• Event sourcing: Moves state to an event log, enabling stateless event handlers.
• Container orchestration: Stateless services are easier to deploy with Kubernetes; stateful services require StatefulSets or external volumes.

4. How it works

Stateless request flow

sequenceDiagram
    participant Client
    participant LB as Load Balancer
    participant Service as Stateless Service
    participant DB as PostgreSQL
    participant Cache as Redis

    Client->>LB: GET /profile
    LB->>Service: GET /profile
    Service->>Cache: GET user:123
    Cache->>Service: null
    Service->>DB: SELECT * FROM users WHERE id=123
    DB->>Service: {id:123, name:"Alice"}
    Service->>Cache: SET user:123 {id:123, name:"Alice"}
    Service->>Client: 200 {id:123, name:"Alice"}

Stateful request flow

sequenceDiagram
    participant Client
    participant LB
    participant Service
    participant SessionStore

    Client->>LB: POST /login
    LB->>Service: POST /login
    Service->>SessionStore: CREATE session {userId:123}
    SessionStore->>Service: sessionId=abc123
    Service->>Client: 200 Set-Cookie: session=abc123
    Client->>LB: GET /dashboard
    LB->>Service: GET /dashboard
    Service->>SessionStore: GET session abc123
    SessionStore->>Service: {userId:123}
    Service->>Client: 200 {dashboard data}

5. Practical implementation

Stateless service with external Redis cache (Node.js)

// src/server.ts
import express from 'express';
import { createClient } from 'redis';

const app = express();
const redis = createClient(); // defaults to localhost:6379

redis.on('error', console.error);
await redis.connect();

app.get('/profile/:id', async (req, res) => {
  const id = req.params.id;
  // 1️⃣ Try cache
  const cached = await redis.get(`user:${id}`);
  if (cached) {
    return res.json(JSON.parse(cached));
  }

  // 2️⃣ Fallback to DB (pseudo)
  const user = await fakeDbQuery(id); // replace with real DB call

  // 3️⃣ Store in cache
  await redis.set(`user:${id}`, JSON.stringify(user), {
    EX: 60 * 5, // 5 minutes
  });

  res.json(user);
});

app.listen(3000, () => console.log('Server listening on 3000'));

Key lines

• redis.get and redis.set keep the service stateless; all state lives in Redis.
• No per‑request memory beyond local variables.
• If the service restarts, it still works because the cache is external.

Stateful service example: in‑memory session store (Node.js)

// src/sessionService.ts
import express from 'express';
import { v4 as uuidv4 } from 'uuid';

const app = express();
app.use(express.json());

const sessions: Record<string, { userId: string }> = {};

app.post('/login', (req, res) => {
  const { userId } = req.body;
  const sessionId = uuidv4();
  sessions[sessionId] = { userId };
  res.cookie('session', sessionId, { httpOnly: true });
  res.send('Logged in');
});

app.get('/profile', (req, res) => {
  const sessionId = req.cookies.session;
  const session = sessions[sessionId];
  if (!session) return res.status(401).send('Unauthenticated');
  res.json({ userId: session.userId });
});

app.listen(3001, () => console.log('Stateful server listening on 3001'));

Why this is stateful

• sessions object lives in the process memory.
• If the process restarts, all sessions are lost.
• Load balancer must use sticky sessions to keep a user on the same instance.

6. Production considerations

7. Common mistakes

8. Senior engineer perspective

Architecture decisions

• Choose stateless for API gateways, microservices, background workers.
• Use stateful only when latency is critical and data can be kept in RAM (e.g., in‑memory cache, real‑time analytics).
• Externalize state: Redis for sessions, PostgreSQL for durable data, Kafka for event streams.

Trade‑offs & bottlenecks

• Cache hit ratio: Low hit ratio turns stateless service into a DB bottleneck.
• Network latency: External store latency can dominate request time.
• Consistency: Distributed caches need eventual consistency; consider read‑your‑write patterns.
• Cost: External stores (Redis, DB) add operational cost; weigh against in‑memory RAM cost.

Failure scenarios

• Cache outage: Fallback to DB; add circuit breaker.
• DB outage: Use read replicas, graceful degradation.
• Network partition: Ensure service can still serve cached data or return 503.

Monitoring

• Cache hit/miss ratio
• External store latency
• Instance memory usage (for stateful services)
• Circuit breaker metrics

Cost/performance

• Stateless: Lower per‑instance cost, but higher external store cost.
• Stateful: Higher RAM cost, but lower network latency.

When NOT to use stateless

• When you need real‑time in‑memory state that cannot tolerate network latency (e.g., high‑frequency trading).
• When the state is small and short‑lived and can be kept in a lightweight in‑process cache without external store overhead.

9. Interview questions

10. Hands‑on task

Task: Build a simple stateless “product catalog” API that caches product data in Redis.

• Requirements:
  1. GET /products/:id returns product details.
  2. Cache product data for 10 minutes.
  3. If Redis is down, fallback to a mock in‑memory DB.
  4. Add a health check endpoint /health that reports Redis status.
• Deliverables:
  • src/server.ts with Express, Redis client, and fallback logic.
  • Dockerfile to run the service.
  • README with instructions to run locally and test the cache hit/miss.

11. Quick revision

• Stateless services: no per‑request memory; all state externalized.
• Stateful services: keep data in RAM; require sticky sessions or replication.
• External stores (DB, Redis, Kafka) are the backbone of stateless design.
• Idempotency keys protect against duplicate requests in stateless APIs.
• Cache hit ratio directly impacts latency and DB load.
• Circuit breakers prevent cascading failures when external stores fail.
• Sticky sessions are a sign of stateful design; avoid them for true horizontal scaling.
• Monitoring cache metrics (hits/misses, latency) is essential for performance tuning.
• In production, stateless services are easier to deploy, scale, and recover.
• Use stateful services only when latency or data size justifies in‑memory storage.

12. References

• Node.js Redis client documentation
• Express.js API documentation
• Redis best practices
• Kubernetes StatefulSets
• Circuit Breaker pattern
• PostgreSQL documentation
• FastAPI documentation
• Next.js API routes