Memory metrics

Backend Fundamentals · Observability · 8 min read

1. What problem does this solve?

Memory metrics let you measure, understand, and control how much RAM your backend processes consume.
Without them you risk:

• Out‑of‑Memory (OOM) crashes that bring down services and cause downtime.
• Hidden memory leaks that slowly degrade performance and increase infrastructure costs.
• Mis‑scaled deployments – allocating too little memory wastes resources, too much wastes money.
• Unpredictable latency – garbage collection pauses can spike response times if you don’t know how much memory is in play.

In real systems, memory metrics surface in:

• Node.js servers (heap, RSS, external memory).
• Python/FastAPI apps (process memory, garbage‑collector stats).
• Databases (PostgreSQL shared buffers, Redis memory usage).
• Container orchestrators (Docker/Kubernetes memory limits/requests).
• Monitoring stacks (Prometheus, Grafana, Datadog).

They are the foundation for capacity planning, alerting, and performance tuning.

2. Mental model

Analogy

Think of a backend process as a house.

• House size → total RAM allocated.
• Furniture → objects allocated in the heap.
• Garbage → unused objects that should be removed.
• Household chores → garbage collection.

If you keep adding furniture without cleaning up, the house eventually becomes overcrowded and you can’t move in new guests (OOM). Memory metrics are the utility bills that tell you how much space you’re using and when you need to tidy up.

Engineering model

• RSS (Resident Set Size): physical memory the process occupies.
• Heap: dynamic memory for JS objects.
• External: memory used by native bindings (e.g., buffers).
• Stack: call‑stack frames (small, usually ignored).
• GC cycles: how often and how long the engine pauses to reclaim heap.

Monitoring these gives you a clear picture of the “house” and its “utility usage”.

3. Deep explanation

Internals

• V8 splits heap into new space (young generation) and old space (tenured objects).
• Garbage collection is generational: young objects are collected more frequently.
• Node.js exposes process.memoryUsage() which returns an object with rss, heapTotal, heapUsed, external, arrayBuffers.
• Prometheus metrics like nodejs_heap_size_bytes are derived from these values.

Trade‑offs

• Higher heap limits → fewer GC pauses but more memory consumption.
• Lower heap limits → more GC but lower cost.
• Enabling heap snapshots → useful for debugging but expensive in production.

Failure modes

• Memory leaks: objects referenced unintentionally, never freed.
• Fragmentation: GC cannot reclaim memory due to layout.
• OOM: process killed by OS or container runtime.
• GC thrashing: too frequent GC causing high CPU usage.

Limitations

• Metrics are snapshots; they don’t show historical trends unless stored.
• process.memoryUsage() reports process‑level memory; not per‑request.
• GC pause metrics require instrumentation (e.g., v8-profiler or node --trace_gc).

Where it fits

• Observability stack: Prometheus scrape, Grafana dashboards, alerting.
• CI/CD: automated tests that fail if memory usage exceeds thresholds.
• Auto‑scaling: Kubernetes HPA can use memory metrics to trigger pod scaling.

Interaction with related concepts

• CPU: GC pauses consume CPU; high CPU + high memory → performance bottleneck.
• Networking: Large payloads increase buffer memory.
• Databases: Connection pools consume memory; monitor pool size.
• Container limits: Set --memory flag; OOM kills if exceeded.

4. How it works

1. Collect: process.memoryUsage() is called periodically (e.g., every 10 s).
2. Transform: Convert raw numbers into Prometheus metrics (nodejs_heap_size_bytes, nodejs_heap_used_bytes, etc.).
3. Expose: Serve /metrics endpoint.
4. Scrape: Prometheus pulls the endpoint.
5. Store & Alert: Grafana visualizes; alerts fire when thresholds are breached.
6. Investigate: On alert, take heap snapshot or enable GC tracing to diagnose leaks.

sequenceDiagram
    participant App as Node.js App
    participant Prom as Prometheus
    participant Graf as Grafana
    App->>App: process.memoryUsage()
    App->>Prom: /metrics
    Prom->>Prom: scrape /metrics
    Prom->>Graf: query metrics
    Graf->>Graf: display dashboard
    Graf->>App: alert (if threshold)

5. Practical implementation

5.1 Express + prom-client

// src/metrics.ts
import { collectDefaultMetrics, register, Gauge } from 'prom-client';
import { Request, Response, NextFunction } from 'express';

// Collect Node.js default metrics (CPU, memory, etc.)
collectDefaultMetrics();

// Custom memory gauges
const heapUsed = new Gauge({
  name: 'nodejs_heap_used_bytes',
  help: 'V8 heap used in bytes',
});
const heapTotal = new Gauge({
  name: 'nodejs_heap_total_bytes',
  help: 'V8 heap total in bytes',
});
const rss = new Gauge({
  name: 'nodejs_rss_bytes',
  help: 'Resident Set Size in bytes',
});
const external = new Gauge({
  name: 'nodejs_external_bytes',
  help: 'External memory in bytes',
});

export function memoryMetricsMiddleware(
  req: Request,
  res: Response,
  next: NextFunction
) {
  const mem = process.memoryUsage();
  heapUsed.set(mem.heapUsed);
  heapTotal.set(mem.heapTotal);
  rss.set(mem.rss);
  external.set(mem.external);
  next();
}

// src/server.ts
import express from 'express';
import { memoryMetricsMiddleware } from './metrics';
import { register } from 'prom-client';

const app = express();

app.use(memoryMetricsMiddleware);

app.get('/health', (_, res) => res.send('OK'));

app.get('/metrics', async (_, res) => {
  res.set('Content-Type', register.contentType);
  res.end(await register.metrics());
});

app.listen(3000, () => console.log('Server listening on :3000'));

What it does

• collectDefaultMetrics() automatically exposes CPU, memory, and GC stats.
• Custom gauges keep the latest values; they’re updated on every request.
• /metrics endpoint is Prometheus‑friendly.

5.2 Docker memory limits

# Dockerfile
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
EXPOSE 3000
CMD ["node", "dist/server.js"]

Run with a memory cap:

docker run -d -p 3000:3000 --memory=512m --memory-swap=512m myapp

If the process exceeds 512 MiB, Docker will OOM‑kill it.

5.3 Heap snapshot on high memory

import { writeFileSync } from 'fs';
import { createHeapSnapshot } from 'v8';

function maybeSnapshot() {
  const mem = process.memoryUsage();
  if (mem.heapUsed > 400 * 1024 * 1024) { // >400 MiB
    const file = `heap-${Date.now()}.heapsnapshot`;
    writeFileSync(file, createHeapSnapshot());
    console.log(`Heap snapshot written to ${file}`);
  }
}

Call maybeSnapshot() in a background interval or after a request.

6. Production considerations

7. Common mistakes

8. Senior engineer perspective

When used at scale

• Clustered Node: Each worker has its own memory profile; aggregate across workers.
• Micro‑services: Each service exposes its own /metrics; central Prometheus scrapes all.
• Serverless: Memory limits are part of the function configuration; monitor per‑invocation usage.

Design decisions

Bottlenecks & failure scenarios

• GC thrashing: High allocation rate → frequent GC → CPU saturation.
• Memory fragmentation: Long‑running services may hit OOM even with free space.
• Container OOM kills: Sudden spikes (e.g., bulk uploads) can kill workers.
• Prometheus scrape failures: If /metrics is slow, Prometheus may miss data.

Monitoring & cost

• Dashboards: Show heap usage, RSS, GC pause, and request latency side‑by‑side.
• Alerting: Thresholds at 80 % of container limit; alert on GC pause > 50 ms.
• Cost: More memory → higher instance cost; GC pauses → higher CPU cost.

When NOT to use

• Stateless, short‑lived functions (e.g., Lambda) – memory metrics are less useful; rely on provider metrics.
• Very small services with <10 MiB usage – overhead of instrumentation outweighs benefit.

9. Interview questions

10. Hands‑on task

Goal: Add a memory‑leak detector to the existing Express app.

1. Create a background job that runs every minute.
2. Record heapUsed to a local file (mem.log).
3. If heapUsed grows by >10 % over 5 minutes, write a warning to the console and trigger a heap snapshot.
4. Expose a /memlog endpoint that returns the last 10 log entries.
5. Run the app in Docker with a 256 MiB memory limit and observe the detector in action when you simulate a leak (e.g., by adding a global array that grows).

11. Quick revision

• process.memoryUsage() → RSS, heapTotal, heapUsed, external.
• Heap: V8’s managed memory; RSS: total physical memory.
• GC pause → latency spikes; monitor via nodejs_gc_duration_seconds.
• Prometheus metrics: nodejs_heap_size_bytes, nodejs_heap_used_bytes, nodejs_rss_bytes.
• Docker memory limits: --memory, --memory-swap.
• Heap snapshots: v8.createHeapSnapshot().
• Common leak signs: steady heapUsed growth, high GC frequency.
• Alert thresholds: 80 % container memory, GC pause > 50 ms.
• Security: protect /metrics with auth or network policies.
• Scaling: aggregate per‑worker metrics; use HPA on memory.
• Debugging: enable --trace_gc, use heapdump, correlate with request latency.

12. References

• Node.js process.memoryUsage()
• Prometheus client for Node.js
• V8 Heap Snapshot API
• Docker memory limits
• Kubernetes memory requests/limits
• Prometheus best practices
• Node.js GC flags