Cloud Native Architecture & 12-Factor Microservices Patterns

Skill metadata

Name cloud-native-microservices-patterns · Level senior · Tags cloud-native microservices 12-factor kubernetes architecture platform-engineering

"Cloud-native service design: 12-factor compliance, liveness, readiness and startup probes, graceful shutdown with SIGTERM handling and connection draining, structured logging with correlation IDs, and idempotency. Use when pods are killed during a deploy and drop in-flight requests, when Kubernetes restarts a container that is merely slow to start, or when refactoring a service to run correctly on Kubernetes."

Source: skills/platform-engineering/cloud-native-microservices-patterns/SKILL.md

When to Use This Skill

Triggers — load this skill when:

• A service is being designed or refactored for Kubernetes
• Probes, shutdown handling, or connection draining behave incorrectly
• Logging, config, or state handling violates 12-factor and hurts operability

Route elsewhere when:

• Manifest and chart mechanics -> helm-kubernetes-deployment
• Inter-service traffic policy -> api-gateway-service-mesh
• Event-driven decomposition -> serverless-event-driven-architecture

1. 12-Factor Production Checklist for Microservices

• Codebase: One codebase tracked in revision control, many deploys.
• Dependencies: Explicitly declare and isolate dependencies (lockfiles, container layers).
• Config: Store configuration in the environment (process.env, os.environ), never in code.
• Backing Services: Treat databases, caches, and queues as attached resources.
• Build, Release, Run: Strict separation between build stage and execution stage.
• Processes: Execute the app as one or more stateless, shareable processes.
• Port Binding: Export services via port binding (e.g. :8080).
• Concurrency: Scale out via the process model (horizontal pod scaling).
• Disposability: Maximize robustness with fast startup and graceful shutdown (SIGTERM handling).
• Dev/Prod Parity: Keep development, staging, and production as similar as possible.
• Logs: Treat logs as unbuffered event streams (stdout/stderr in JSON format).
• Admin Processes: Run admin/management tasks as one-off processes (K8s Jobs).

---

2. Graceful Shutdown Implementation (Node.js / Express Example)

const server = app.listen(process.env.PORT || 8080);

const shutdown = (signal) => {
  console.log(`Received ${signal}. Gracefully closing HTTP server...`);
  server.close(() => {
    console.log('HTTP server closed. Disconnecting database pools...');
    db.pool.end(() => {
      console.log('Database pool closed. Exiting process.');
      process.exit(0);
    });
  });

  // Force shutdown after 15s timeout
  setTimeout(() => {
    console.error('Graceful shutdown timeout exceeded. Forcing exit.');
    process.exit(1);
  }, 15000);
};

process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));

---

3. Best Practices & Anti-Patterns

• Do: Always implement both livenessProbe (is the process deadlocked?) and readinessProbe (is the app ready to serve traffic?).
• Don't: Never exit immediately upon receiving SIGTERM; allow in-flight HTTP requests 5-15 seconds to drain.

---

4. Graceful Shutdown Timing & Request Correlation

Dropped requests during a deploy are almost always a race: Kubernetes sends SIGTERM and removes the pod from Endpoints at the same time, and in-flight proxies still hold the old address for a moment.

The sequence that does not drop traffic:

1. preStop sleeps (5–10s) so Endpoint removal propagates to every proxy before shutdown starts;
2. SIGTERM flips readiness to false and stops accepting new connections;
3. in-flight requests drain, with a deadline shorter than the grace period;
4. the process exits before terminationGracePeriodSeconds expires — otherwise SIGKILL truncates whatever is still running.

spec:
  terminationGracePeriodSeconds: 45      # must exceed preStop + drain deadline
  containers:
    - name: api
      lifecycle:
        preStop:
          exec: { command: ["/bin/sh", "-c", "sleep 10"] }
      startupProbe:                       # slow boot must not be read as "wedged"
        httpGet: { path: /healthz, port: 8080 }
        failureThreshold: 30
        periodSeconds: 5

process.on('SIGTERM', async () => {
  isReady = false;                                    // readiness flips first
  server.closeIdleConnections?.();
  await new Promise((r) => server.close(r));          // stop accepting, drain in-flight
  await Promise.race([pool.end(), sleep(10_000)]);    // bounded dependency close
  process.exit(0);
});

Correlation IDs make the resulting logs usable: accept traceparent (or X-Request-Id), generate one when absent, propagate it on every outbound call, and emit it as a field on every structured log line — trace_id in logs is what turns three services' logs into one story, and it is the join key back to the trace.