Use Kustomize overlays for environment management

Introduction

When you copy a Deployment YAML three times — once for dev, staging, and prod — the duplicates drift within a week. A staging fix never lands in prod, a prod tweak goes missing in dev, and the next incident retro reads "we forgot to update the other two files." Kustomize removes that failure mode by letting you keep one base manifest and express each environment as a small, reviewable patch on top of it. By the end of this lesson you'll be able to lay out a base/-plus-overlays/ tree, write a strategic merge patch that adjusts only what differs per environment, and run kustomize build to render the final manifests for kubectl apply.

Key Terminology

• Base — the canonical set of Kubernetes manifests (Deployment, Service, ConfigMap) plus a kustomization.yaml that all overlays start from; in this lesson the base holds the shape every environment must share.
• Overlay — a directory with its own kustomization.yaml that references the base and applies environment-specific patches; one overlay per environment is the unit you swap when deploying to dev, staging, or prod.
• Strategic merge patch — a partial Kubernetes manifest that names the target by apiVersion/kind/metadata.name and merges only the fields it sets; it is the primary way overlays change replica counts, resource limits, or container env vars without rewriting the base.
• kustomize build — the command that resolves an overlay against its base and emits the final, rendered YAML to stdout; the output is what kubectl apply -f - or ArgoCD actually deploys.
• commonLabels — a top-level kustomization field that adds labels to every resource the overlay emits, including patched and overlay-only resources, so you don't have to label each manifest by hand.

Concepts

Kustomize takes a different approach from Helm. Instead of parameterized templates with variable substitution, it works with plain Kubernetes YAML and applies patches to transform a base into environment-specific variants. Bases stay valid, deployable resources — not Go template fragments — so reviewers can read them as Kubernetes manifests directly and kubectl apply -k base/ against a sandbox always works.

The base-plus-overlays layout separates what is common from what varies. The base holds the canonical Deployment, Service, and ConfigMap. Each overlay has its own kustomization.yaml that references the base and patches the small set of fields that should differ per environment: replica counts, resource limits, LLM endpoint, and any environment-only resources such as a PodDisruptionBudget for prod (see Code Walkthrough).

Resolution order matters when you debug a rendered manifest. kustomize build first loads the base, then applies strategic merge patches, then JSON patches, then commonLabels, then namespace transformers. Because commonLabels run after patches, labels are attached to overlay-only resources (like the prod-only PDB) automatically — you do not need to add them by hand.

Code Walkthrough

The snippet below renders two artifacts: the base kustomization.yaml and a strategic merge patch that any per-environment overlay applies to change replicas and resource limits. The same OverlayPatch model drives dev (one replica, tight limits) and prod (more replicas, higher limits) — only the field values differ.

Code snippetpython
1from typing import Optional
2import yaml
3from pydantic import BaseModel, Field
4
5class KustomizeBase(BaseModel):
6    resources: list[str] = Field(
7        default=["deployment.yaml", "service.yaml", "configmap.yaml"]
8    )
9    common_labels: dict[str, str] = Field(
10        default={"app": "ai-service", "managed-by": "kustomize"}
11    )
12
13class OverlayPatch(BaseModel):
14    target_name: str = "ai-service"
15    replicas: Optional[int] = None
16    cpu_limit: Optional[str] = None
17    memory_limit: Optional[str] = None
18
19def render_base_kustomization(base: KustomizeBase) -> str:
20    return yaml.dump(
21        {
22            "apiVersion": "kustomize.config.k8s.io/v1beta1",
23            "kind": "Kustomization",
24            "resources": base.resources,
25            "commonLabels": base.common_labels,
26        },
27        default_flow_style=False,
28    )
29
30def render_strategic_merge_patch(patch: OverlayPatch) -> str:
31    manifest: dict = {
32        "apiVersion": "apps/v1",
33        "kind": "Deployment",
34        "metadata": {"name": patch.target_name},
35        "spec": {},
36    }
37    if patch.replicas is not None:
38        manifest["spec"]["replicas"] = patch.replicas
39
40    limits: dict[str, str] = {}
41    if patch.cpu_limit:
42        limits["cpu"] = patch.cpu_limit
43    if patch.memory_limit:
44        limits["memory"] = patch.memory_limit
45    if limits:
46        manifest["spec"]["template"] = {
47            "spec": {
48                "containers": [
49                    {
50                        "name": patch.target_name,
51                        "resources": {"limits": limits},
52                    }
53                ]
54            }
55        }
56    return yaml.dump(manifest, default_flow_style=False)
57
58prod_patch = OverlayPatch(replicas=5, cpu_limit="1000m", memory_limit="1Gi")
59print(render_strategic_merge_patch(prod_patch))

KustomizeBase describes which files the base ships and the labels every rendered resource will carry. OverlayPatch uses Optional fields so an overlay only contains what it actually changes; absent fields stay at the base value. render_strategic_merge_patch identifies the target by apiVersion/kind/metadata.name and emits only the fields the overlay touches, which is exactly what Kustomize needs to merge cleanly. To wire it into an overlay, write the rendered YAML to overlays/prod/replica-patch.yaml and list it under patchesStrategicMerge in overlays/prod/kustomization.yaml.

You'll know it works when kustomize build overlays/prod prints a Deployment with replicas: 5 and the higher limits, while kustomize build overlays/dev against the same base still prints the base values unchanged.

Do's and Don'ts

Do's

1. ✓Do keep the base deployable on its own — kubectl apply -k base/ against a sandbox should produce a working service, which forces the base to stay honest and makes overlay diffs the only place environment drift can hide.
2. ✓Do put only environment differences in the overlay — replica counts, resource limits, LLM endpoints, PDBs; everything shared belongs in the base so a fix lands everywhere at once.
3. ✓Do review kustomize build overlays/<env> output in PRs — the rendered YAML is the contract with the cluster, and reviewing patches alone misses merge surprises like an accidentally cleared base field.

Don'ts

1. ✗Don't fork the base per environment — if you find yourself copying base files into an overlay, you've recreated the duplication Kustomize exists to remove.
2. ✗Don't patch fields that aren't in the base — strategic merge needs the field path to exist in the target; add it to the base first, then override in the overlay.
3. ✗Don't skip commonLabels on overlays that add resources — overlay-only manifests like a prod PDB inherit labels from the overlay's kustomization, and dropping that field leaves them unlabeled and hard to query.