Free lesson · GenAI Platform Engineering

Test Helm charts before deployment

You will build a testing pipeline for Helm charts. Install helm-unittest and write unit tests: test that deployment replicas match values, test that LLM provider ConfigMap is created when llm.enabled=true, test that resource limits are set correctly. Run helm lint to catch template syntax errors. Use helm template to render manifests without installing and validate them with kubeval for K8s schema compliance. Build a CI job that runs helm-unittest, helm lint, and kubeval on every PR modifying chart files. Implement snapshot testing: save rendered manifests as golden files and fail if they change unexpectedly.

Course: DevOps Foundations for GenAI Engineers · Chapter 5 · Infrastructure as Code

Free to read — no subscription required.

Introduction

When you push a Helm chart that renders fine with dev values but breaks under production values, the failure surfaces during helm upgrade against a live cluster — after the rollout has already started and pods are being replaced. A missing conditional, a bad indent in a helper, or a string-where-int type mismatch is enough to leave the namespace in a half-applied state that on-call has to unwind by hand. Testing charts in CI moves that failure detection to the pull request, where it is cheap, safe, and traceable.

By the end of this lesson you'll be able to assemble a three-stage validation pipeline — helm lint, helm-unittest, and kubeval/kubeconform — that renders every values file you ship, asserts the template logic, and checks each rendered manifest against the Kubernetes API schema for your target cluster version before the chart is allowed to reach a cluster.

Key Terminology

  • helm lint — Helm's built-in static check for chart syntax, missing required values, and template parse errors; it matters because it's the cheapest, fastest gate and catches the most common authoring mistakes before any rendering happens.
  • helm-unittest — A Helm plugin that runs YAML-defined test cases against rendered templates, asserting on specific paths in the output; it matters because it locks template conditionals and helpers against regression as values files multiply across environments.
  • kubeval / kubeconform — CLI validators that check rendered manifests against the Kubernetes OpenAPI schema for a pinned cluster version; they matter because they catch deprecated API versions, unknown fields, and type mismatches that lint and unit tests cannot see.
  • values file — The YAML inputs (values.yaml, values-prod.yaml, …) that parameterize a chart per environment; it matters because every distinct values file is a distinct render and must be validated independently — the dev render passing tells you nothing about prod.
  • strict validation — A kubeval/kubeconform mode that rejects manifests containing fields not in the schema; it matters because it turns typos like replcia into hard CI failures instead of silently-ignored YAML keys.

Concepts

The testing pipeline runs in sequence, with each stage catching a different class of error. Early stages are fast and broad; later stages are slower but catch subtle schema violations that earlier stages cannot detect. Charts move forward only when every stage passes for every values file in scope (see Code Walkthrough).

Loading diagram...

Layered validation, fastest gate first

helm lint runs in under a second and rejects charts that fail to parse or are missing required values — keep it as the first job in CI so authoring typos never consume more expensive compute. helm-unittest then exercises the templates against representative values overrides and asserts on rendered paths; this is where you lock in behaviour like "the HPA template only renders when autoscaling.enabled=true". Finally, helm template renders every values file and kubeval/kubeconform validates each rendered manifest against the OpenAPI schema for your target Kubernetes version.

One render per values file

A chart is not validated until every values file it ships is rendered and checked. Production-only paths — extra sidecars, stricter resource limits, the HPA, network policies — frequently live behind if blocks that the default values.yaml never exercises. The pipeline must iterate over values-dev.yaml, values-staging.yaml, values-prod.yaml, etc., and treat any single failure as a chart-level failure.

Pin the cluster version

kubeval --kubernetes-version 1.28.0 (or the equivalent kubeconform -kubernetes-version) is what makes the schema check meaningful. Without a pinned version the validator falls back to a default that may not match your fleet, and deprecated API versions slip through. Pin to the lowest cluster version you currently run.

Code Walkthrough

The two snippets below demonstrate the middle and last stages of the pipeline: a Pydantic model that emits the YAML helm-unittest consumes (locking template logic), and a wrapper that renders the chart and validates every manifest against the Kubernetes schema (locking the API contract).

Code snippetpython
1from pydantic import BaseModel, Field 2from typing import Any 3import yaml 4 5class HelmTestAssertion(BaseModel): 6 equal: dict[str, Any] | None = None 7 match_regex: dict[str, str] | None = None 8 is_not_null: dict[str, bool] | None = None 9 is_null: dict[str, bool] | None = None 10 11class HelmTestCase(BaseModel): 12 name: str 13 set_values: dict[str, Any] = Field(default_factory=dict) 14 assertions: list[HelmTestAssertion] = Field(default_factory=list) 15 16class HelmTestSuite(BaseModel): 17 suite_name: str 18 template: str 19 tests: list[HelmTestCase] = Field(default_factory=list) 20 21def generate_test_suite(suite: HelmTestSuite) -> str: 22 suite_dict: dict[str, Any] = { 23 "suite": suite.suite_name, 24 "templates": [suite.template], 25 "tests": [], 26 } 27 for test in suite.tests: 28 entry: dict[str, Any] = {"it": test.name} 29 if test.set_values: 30 entry["set"] = test.set_values 31 asserts: list[dict[str, Any]] = [] 32 for a in test.assertions: 33 if a.equal: 34 asserts.append({"equal": a.equal}) 35 if a.match_regex: 36 asserts.append({"matchRegex": a.match_regex}) 37 if a.is_not_null: 38 asserts.append({"isNotNull": a.is_not_null}) 39 if a.is_null: 40 asserts.append({"isNull": a.is_null}) 41 entry["asserts"] = asserts 42 suite_dict["tests"].append(entry) 43 return yaml.dump(suite_dict, default_flow_style=False)

HelmTestAssertion models the four assertion kinds helm-unittest understands; HelmTestCase binds a name to a set block (per-test values overrides) and a list of assertions; HelmTestSuite groups cases that target the same template file. generate_test_suite emits the exact YAML shape (it / set / asserts) the plugin parses, so the same Pydantic objects can drive both code-side test generation and the on-disk test fixtures.

Code snippetpython
1import subprocess 2import tempfile 3from pathlib import Path 4 5class ValidationResult(BaseModel): 6 resource: str 7 valid: bool 8 errors: list[str] = Field(default_factory=list) 9 10class ManifestValidator(BaseModel): 11 kubernetes_version: str = "1.28.0" 12 strict: bool = True 13 14 def validate_chart(self, chart_path: str, values_file: str) -> list[ValidationResult]: 15 render = subprocess.run( 16 ["helm", "template", "test-release", chart_path, "--values", values_file], 17 capture_output=True, text=True, 18 ) 19 if render.returncode != 0: 20 return [ValidationResult(resource="chart", valid=False, errors=[render.stderr])] 21 22 with tempfile.NamedTemporaryFile(mode="w", suffix=".yaml", delete=False) as tmp: 23 tmp.write(render.stdout) 24 tmp_path = tmp.name 25 26 cmd = ["kubeval", tmp_path, "--kubernetes-version", self.kubernetes_version] 27 if self.strict: 28 cmd.append("--strict") 29 check = subprocess.run(cmd, capture_output=True, text=True) 30 31 results: list[ValidationResult] = [] 32 for line in check.stdout.splitlines(): 33 if "PASS" in line or "WARN" in line or "ERR" in line: 34 results.append(ValidationResult( 35 resource=line.split(" - ")[0].strip(), 36 valid="PASS" in line, 37 errors=[] if "PASS" in line else [line], 38 )) 39 Path(tmp_path).unlink(missing_ok=True) 40 return results

ManifestValidator first runs helm template against one values file; if rendering fails there is no manifest to validate and the method returns the helm error directly. The rendered YAML is written to a temp file and passed to kubeval with the pinned Kubernetes version and (optionally) --strict so unknown fields fail the check. The parsed output yields one ValidationResult per resource, which the CI job can assert on. Call validate_chart once per values file in your matrix.

You'll know it works when CI rejects a PR that introduces replcia: 3 into values-prod.yaml with a kubeval ERR line naming the resource — and accepts the same PR once the typo is fixed.

Do's and Don'ts

Do's

  1. Do run all three stages on every PR that touches a chart — lint, helm-unittest, and kubeval/kubeconform — because each catches a class of error the others miss.
  2. Do render and validate every values file you ship — production-only conditionals are invisible to a dev render.
  3. Do pin --kubernetes-version to the lowest cluster version you run — deprecated API versions slip through when the validator falls back to a default.

Don'ts

  1. Don't treat helm lint passing as "the chart works" — lint only checks syntax and required values, not rendered output.
  2. Don't skip --strict on kubeval/kubeconform — non-strict mode silently accepts typos like replcia as valid YAML.
  3. Don't validate only the default values.yaml — the values file that exercises your HPA, sidecars, and network policies is the one that finds bugs.

This lesson is free to read. Its 3 hands-on labs — real code, in a cloud IDE — are part of the GenAI Platform Engineering subscription.

From · cancel anytime

More free lessons in DevOps Foundations for GenAI Engineers

All free lessons in GenAI Platform Engineering