Free lesson · GenAI Data Engineering
Deploy infrastructure with Crossplane + Helm + Kustomize
Use Crossplane for cloud resource provisioning (AlloyDB, GCS), Helm for application packaging, Kustomize for environment overlays, and Argo CD for GitOps delivery.
Course: GenAI Data Pipelines · Chapter 18 · Production Capstone on GKE
Free to read — no subscription required.
Introduction
When you provision cloud resources through console clicks or imperative scripts, you accumulate configuration drift, undocumented changes, and environments that cannot be reliably reproduced — so the next disaster-recovery rebuild surprises you with a half-day outage. GitOps fixes this by treating Git as the single source of truth: Crossplane represents cloud resources as Kubernetes objects, Helm packages the application, Kustomize patches per-environment differences, and ArgoCD continuously reconciles the cluster to match. By the end of this lesson you'll be able to author Crossplane claims for AlloyDB and GCS, layer Kustomize overlays over a Helm chart, and let ArgoCD reconcile both into a running environment.
Key Terminology
- CompositeResourceClaim (XRC) — a Crossplane custom resource that lets application teams request cloud infrastructure (AlloyDB, GCS buckets) through a simplified Kubernetes manifest. It matters because the XRC is the developer-facing API in this lesson — every claim ArgoCD syncs is an XRC.
- Composition — the platform-team-authored template that maps an XRC to the provider-specific resources it actually requires (VPC peering, IAM bindings, the AlloyDB instance itself). It matters because Compositions are how you hide GCP-specific knobs from application teams.
- Kustomize overlay — a directory of patches layered on top of a base Helm-rendered manifest set to adjust replica counts, resource requests, and ConfigMaps per environment without forking the chart. It matters because the lesson uses one chart for dev, staging, and prod.
- ArgoCD reconciliation loop — the controller process that diffs the Git-declared manifests against live cluster state and applies the delta on a fixed interval or via webhook-triggered sync after merge. It matters because sync timing decides whether your deploy is near-instant or lags 3 minutes.
Concepts
This lesson assembles a GitOps deployment stack: Crossplane CompositeResourceClaims that let developers request cloud resources through short Kubernetes manifests, Helm charts layered with Kustomize overlays for per-environment packaging, and an ArgoCD reconciliation loop that syncs the Git-declared state to the cluster.
Helm Charts with Kustomize Overlays
The deployment strategy layers Helm charts for application packaging with Kustomize overlays for per-environment customization. Base Helm values define the application structure, while Kustomize patches adjust replica counts, resource requests, and environment-specific ConfigMaps without duplicating the entire chart.
Code Walkthrough
Crossplane CompositeResourceClaims
Crossplane lets platform teams define simplified interfaces for cloud resources. A CompositeResourceClaim (XRC) allows application developers to request an AlloyDB instance or a GCS bucket through a short Kubernetes manifest without knowing the underlying provider-specific configuration. The platform team defines the Composition that maps these claims to actual GCP resources.
CrossplaneClaimGenerator produces Crossplane claim manifests from a high-level configuration dictionary. The class formats claims that ArgoCD can apply, abstracting away provider-specific details behind team-friendly interfaces.
Code snippet python
1import yaml 2from pathlib import Path 3 4class CrossplaneClaimGenerator: 5 def __init__(self, output_dir: str): 6 self.output_dir = Path(output_dir) 7 self.output_dir.mkdir( 8 parents=True, exist_ok=True, 9 ) 10 11 def generate_alloydb_claim( 12 self, 13 name: str, 14 cpu_count: int = 2, 15 memory_gb: int = 16, 16 database: str = "pipeline_db", 17 ) -> Path: 18 claim = { 19 "apiVersion": 20 "database.platform.io/v1alpha1", 21 "kind": "AlloyDBClaim", 22 "metadata": {"name": name}, 23 "spec": { 24 "parameters": { 25 "cpuCount": cpu_count, 26 "memoryGb": memory_gb, 27 "databaseName": database, 28 "enablePgvector": True, 29 "backupSchedule": "0 2 * * *", 30 }, 31 "compositionSelector": { 32 "matchLabels": { 33 "provider": "gcp", 34 "service": "alloydb", 35 }, 36 }, 37 }, 38 } 39 path = self.output_dir / f"{name}.yaml" 40 path.write_text( 41 yaml.dump(claim, default_flow_style=False), 42 ) 43 return path 44 45 def generate_gcs_claim( 46 self, 47 name: str, 48 location: str = "US", 49 lifecycle_days: int = 90, 50 ) -> Path: 51 claim = { 52 "apiVersion": 53 "storage.platform.io/v1alpha1", 54 "kind": "BucketClaim", 55 "metadata": {"name": name}, 56 "spec": { 57 "parameters": { 58 "location": location, 59 "lifecycleDays": lifecycle_days, 60 "versioning": True, 61 }, 62 }, 63 } 64 path = self.output_dir / f"{name}.yaml" 65 path.write_text( 66 yaml.dump(claim, default_flow_style=False), 67 ) 68 return path
- Lines 1-9: The constructor creates an output directory for generated YAML manifests, ensuring the GitOps repository structure exists before writing claim files.
- Lines 11-41: The generate_alloydb_claim method produces a Crossplane claim for an AlloyDB instance with pgvector enabled and automated backup scheduling, using the platform team's Composition to handle provider-specific details like VPC peering and IAM bindings.
- Lines 43-65: The generate_gcs_claim method creates a bucket claim with lifecycle policies and versioning enabled, abstracting the storage configuration behind a simple interface that application developers can request without GCP console access.
ArgoCD Reconciliation Loop
ArgoCD watches the Git repository and detects when manifests diverge from the running cluster state. When a pull request merges new Crossplane claims or updated Helm values, ArgoCD triggers a sync operation that applies the changes. The reconciliation loop runs every three minutes by default, but webhook-triggered syncs provide near-instant deployment after merge.
Do's and Don'ts
Now that you have worked through the implementation, the practices below separate a durable approach from a fragile one.
Do's
- ✓Author Crossplane Compositions on the platform team and expose only narrow XRC schemas (CPU, memory, database name) so application developers cannot leak provider-specific IAM, VPC, or networking knobs into their claims.
- ✓Keep Helm chart values minimal and push environment differences (replica counts, resource requests, ConfigMap data) into Kustomize overlays so the same chart version ships to dev, staging, and prod with auditable per-env patches.
- ✓Pin ArgoCD
Applicationresources to a specific Git revision or semver tag rather thanHEAD, so a bad merge tomaindoes not auto-sync into every cluster before review.
Don'ts
- ✗Don't
kubectl applyCrossplane claims or Helm releases directly to the cluster — every change must land through a Git commit so ArgoCD reconciliation stays the single source of truth and out-of-band edits get reverted on the next sync. - ✗Don't embed secrets (database passwords, GCP service account keys) in Helm
values.yamlor Kustomize patches; route them through External Secrets Operator or Crossplane provider configs that pull from Secret Manager at apply time. - ✗Don't shorten the ArgoCD reconciliation interval below the default three minutes to "fix" slow deploys — wire a Git webhook for instant sync after merge and leave the polling interval alone to avoid hammering the API server.
This lesson is free to read. Its 3 hands-on labs — real code, in a cloud IDE — are part of the GenAI Data Engineering subscription.
From · cancel anytime · Already a subscriber? Sign in →
More free lessons in GenAI Data Pipelines
- Ch 16Build event-driven triggers with Kafka and KEDA autoscaling
- Ch 16Version datasets with DVC backed by GCS
- Ch 16Connect pipeline agents via MCP for autonomous orchestration
- Ch 16Implement pipeline observability with OTel, Prometheus, Grafana
- Ch 17Implement model cascading for cost reduction
- Ch 18Design end-to-end architecture on GKE Autopilot
- Ch 18Deploy infrastructure with Crossplane + Helm + KustomizeYou are here