Build production Docker images with multi-stage builds

Introduction

When you ship a FastAPI GenAI service in a single-stage image built from python:3.12, the artifact you push to your registry weighs 1.2 GB or more, carries the C toolchain plus development headers your runtime never executes, and runs every request as root inside the container. A push that should take twenty seconds takes three minutes, cold-starts in Kubernetes stall behind a 1 GB pull, and a single remote code execution bug in your dependency tree gives the attacker root on the host's user namespace. By the end of this lesson you will be able to write a two-stage Dockerfile that separates build tooling from runtime, drop privileges to a dedicated non-root user, expose a working HEALTHCHECK, and configure BuildKit cache mounts so CI rebuilds finish in under a minute when only application code changed.

Key Terminology

• Multi-stage build — a Dockerfile pattern that uses multiple FROM instructions so artifacts can be copied between stages with COPY --from=…; matters here because the final runtime image excludes the compiler toolchain and intermediate files that the builder needed.
• Non-root user — a container process running under a UID other than 0 (created with useradd and selected via the USER directive); matters because it prevents an attacker who achieves code execution from escalating to host-level privileges in the pod's user namespace.
• Layer cache — the per-instruction filesystem snapshot Docker reuses when an instruction's inputs are unchanged; matters because correct COPY ordering and BuildKit cache mounts collapse a 3-minute dependency install into a fraction of a second on repeat CI builds.
• HEALTHCHECK — a Dockerfile instruction that defines a command Docker (and orchestrators reading the OCI manifest) run periodically to mark a container healthy, unhealthy, or starting; matters because Kubernetes uses this signal alongside its own probes to decide when to route traffic to a pod.

Concepts

Two-stage architecture: builder vs runtime

A multi-stage Dockerfile uses one FROM to build (with compilers, headers, pip cache) and a second FROM to ship (only the interpreter plus the resolved dependency tree). Stage one installs into a virtualenv at /app/.venv so the entire dependency closure is a single copyable directory. Stage two starts from python:3.12-slim, copies /app/.venv and your source, and discards everything else from stage one. The runtime image lands near 140 MB instead of 1.2 GB, and an attacker who finds a remote code execution path no longer has gcc, apt, or development headers available to pivot with (see Code Walkthrough).

Non-root execution

Containers default to UID 0 inside the container. Even in rootless Kubernetes setups, running as root inside the container removes a defense layer that costs almost nothing to add. Create a dedicated user and group with fixed UID/GID (1001 is a common choice that avoids collisions with the base image's system users), use --shell /bin/false so the account cannot be used for interactive login, and switch with the USER directive after all COPY operations so file ownership is correct. The CMD then executes as that user, and any privilege-escalation path inside Python or its native extensions ends at a UID with no system access (see Code Walkthrough).

Layer caching and BuildKit cache mounts

Docker invalidates a layer when the instruction's inputs change, and every subsequent layer with it. Two implications: copy requirements.txt before the rest of the source, so application edits do not invalidate the pip install layer; and enable BuildKit's --mount=type=cache,target=/root/.cache/pip so the wheel download cache persists across builds even when requirements.txt itself changes. Together these turn a clean-context CI rebuild from 3 minutes of dependency resolution into seconds for the cached path.

Health checks that mean something

HEALTHCHECK only verifies whatever command you put in it. A curl localhost:8000 proves the socket binds; it does not prove the service can serve a request. The endpoint must touch the dependencies that traffic touches — at minimum the database — so an unhealthy pod gets rotated out of service before user requests fail. Keep the check cheap (SELECT 1, a process-memory read) so it does not become its own outage trigger.

Code Walkthrough

Having just studied the two-stage architecture and non-root execution in the Concepts section, the Dockerfile below ties both ideas together with cache-friendly COPY ordering and a HEALTHCHECK that calls a real endpoint.

Code snippetdockerfile
1# syntax=docker/dockerfile:1
2
3# ── Stage 1: Builder ─────────────────────────────────────────────────────────
4FROM python:3.12 AS builder
5WORKDIR /app
6
7# Isolate the dependency closure into a virtualenv — it becomes a single copy target
8RUN python -m venv /app/.venv
9ENV PATH="/app/.venv/bin:$PATH"
10
11# Copy requirements first; editing app code won't bust this cache layer
12COPY requirements.txt .
13RUN --mount=type=cache,target=/root/.cache/pip \
14    pip install --upgrade pip && pip install -r requirements.txt
15
16# Application source goes in last so code-only rebuilds skip the pip install
17COPY app ./app
18
19# ── Stage 2: Runtime ─────────────────────────────────────────────────────────
20FROM python:3.12-slim AS runtime
21
22# Create a dedicated non-root user before copying any files
23RUN groupadd --gid 1001 appgroup \
24    && useradd --uid 1001 --gid appgroup --shell /bin/false --create-home appuser
25
26WORKDIR /app
27
28# Copy only the resolved dependency tree and application source out of stage one
29COPY --from=builder /app/.venv /app/.venv
30COPY --from=builder /app/app   ./app
31
32ENV PATH="/app/.venv/bin:$PATH" \
33    PYTHONDONTWRITEBYTECODE=1 \
34    PYTHONUNBUFFERED=1
35
36EXPOSE 8000
37
38# Probe runs every 30 s; container is marked unhealthy after 3 consecutive failures
39HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
40    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"
41
42# Drop to appuser only after all COPY operations so files keep their original ownership
43USER appuser
44
45CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

The # syntax=docker/dockerfile:1 comment must be the first line to activate BuildKit features, including the --mount=type=cache directive used during the pip install. The builder stage uses the full python:3.12 image so native extensions like uvloop and pydantic-core can compile; the runtime stage starts from python:3.12-slim, which ships without a C compiler or development headers — that gap is what collapses the image from 1.2 GB to roughly 140 MB. Placing COPY requirements.txt before COPY app ./app is deliberate: Docker replays cache top-to-bottom, so a code-only change reuses the pip layer entirely. The --mount=type=cache directive further preserves the wheel cache between runs even when requirements.txt does change, keeping repeat builds fast.

The health endpoint the HEALTHCHECK polls should do more than return a bare 200 OK — it should exercise the actual dependency chain. The implementation below checks the database connection and memory pressure so a pod that can't reach its database is removed from the load-balancer rotation before real traffic hits it:

Code snippetpython
1# app/health.py
2from fastapi import APIRouter, Depends
3from sqlalchemy.ext.asyncio import AsyncSession
4from sqlalchemy import text
5import psutil
6import time
7from app.database import get_db
8
9router = APIRouter(tags=["health"])
10
11@router.get("/health")
12async def health_check(db: AsyncSession = Depends(get_db)):
13    checks: dict[str, object] = {}
14    start = time.monotonic()
15    try:
16        await db.execute(text("SELECT 1"))
17        checks["database"] = "healthy"
18    except Exception as e:
19        checks["database"] = f"unhealthy: {e}"
20    memory = psutil.virtual_memory()
21    checks["memory"] = "warning" if memory.percent > 90 else "healthy"
22    elapsed_ms = (time.monotonic() - start) * 1000
23    all_healthy = all(
24        v == "healthy" for v in [checks["database"], checks["memory"]]
25    )
26    return {
27        "status": "healthy" if all_healthy else "degraded",
28        "checks": checks,
29        "response_time_ms": round(elapsed_ms, 2),
30    }

The SELECT 1 query exercises the full connection-pool path without touching real data; capturing the exception text in the response body lets an operator diagnose the failure without grep-ing logs. Memory above 90% downgrades the status to degraded rather than unhealthy — the distinction matters because Kubernetes stops routing traffic to unhealthy pods, so a high-memory condition should warn rather than immediately shed load. PYTHONUNBUFFERED=1 in the Dockerfile ensures that every log line the health check emits (and that uvicorn emits) reaches the Kubernetes log collector immediately rather than buffering in the Python runtime.

You'll know it works when docker build . completes and reports a runtime image under 200 MB, docker inspect --format '{{.Config.User}}' <container_id> prints appuser, curl localhost:8000/health returns {"status":"healthy",...}, and a second docker build . with no file changes reuses every layer and finishes in under five seconds.

Do's and Don'ts

Now that you've seen the Dockerfile, health endpoint, and the discipline-specific mapping, the rules below codify the habits that keep production images small, secure, and cache-friendly.

Do's

1. ✓Do copy requirements.txt before your source — placing the dependency manifest in its own COPY keeps the expensive pip install layer cached when only application code changed, which is the common case in CI.
2. ✓Do drop to a non-root USER after all COPY instructions finish — switching earlier leaves you fighting file ownership; switching at the end gives you correct permissions plus a privilege drop for the runtime process.
3. ✓Do point HEALTHCHECK at an endpoint that touches real dependencies — a check that only proves the socket is open lets a pod with a broken database stay in the load-balanced set.

Don'ts

1. ✗Don't ship the builder stage as your runtime image — leaving gcc, make, and development headers in the production image inflates pulls and hands an attacker a toolkit if they get code execution.
2. ✗Don't hardcode 0.0.0.0-binding ports below 1024 — non-root processes cannot bind privileged ports without extra capabilities; sticking to 8000+ keeps the USER appuser switch friction-free.
3. ✗Don't use --no-cache-dir together with a BuildKit cache mount — the flag disables pip's wheel cache, which is exactly what the mount is preserving; pick one strategy and let it work.