Deploy onboarding system with ArgoCD integration

Introduction

Engineers often manage team onboarding manually — creating namespaces, applying secrets, and rolling out default configs by hand — which leads to drift between environments and slow, error-prone bootstrapping. GitOps eliminates that drift by declaring the desired state in a repository and letting a controller reconcile it continuously. By the end of this lesson, you will define an ArgoCD Application that points at your onboarding Helm chart, configure sync waves to sequence database migrations before the API service, and verify deployment health through the onboarding endpoint.

Key Terminology

• GitOps — A deployment model in which the desired cluster state is committed to a Git repository and a controller reconciles the live cluster to match it continuously, eliminating the configuration drift that results from manual kubectl operations during team onboarding.
• ArgoCD Application — A Kubernetes custom resource (apiVersion: argoproj.io/v1alpha1, kind: Application) that declares the Helm chart source (repoURL, path, targetRevision), the destination namespace, and the sync policy ArgoCD uses to manage the onboarding stack.
• Sync wave — An ordering mechanism controlled by the argocd.argoproj.io/sync-wave annotation that determines which resources ArgoCD applies first; assigning "0" to the migrations job ensures the schema migration completes before the API pods at wave 2 are scheduled.
• Automated sync policy — The syncPolicy.automated block with prune: True and selfHeal: True that causes ArgoCD to remove orphaned resources and correct any out-of-band cluster changes so live state always converges to what is committed in Git.
• CreateNamespace sync option — A syncOptions entry that instructs ArgoCD to create the target namespace (e.g., platform-production) on first sync, removing the need for a manual namespace bootstrapping step when a new environment is provisioned.
• Helm value overrides — Environment-specific YAML files (values-{environment}.yaml) referenced in the Application's source.helm.valueFiles block that customize replica counts, resource limits, and integration URLs per environment without modifying the chart itself.

Concepts

Git as the Operational Source of Truth

When engineers provision namespaces, apply secrets, and roll out default configs by hand, each cluster becomes a unique snowflake — slightly different from staging, slightly different from what the runbook describes. GitOps inverts this relationship: the repository is authoritative, and the cluster is a derived artifact. ArgoCD watches the chart repository and continuously compares live cluster state against what Git describes. Any gap — whether from a manual kubectl apply or a config edit made directly on a running pod — triggers reconciliation back to the committed state.

This is what syncPolicy.automated with selfHeal: True enforces in generate_argocd_application. It is not just a convenience setting; it is the mechanism that makes the onboarding stack's desired state durable across incidents and human interventions. prune: True complements self-heal by removing resources that were deleted from the chart but remain in the cluster, so dead objects do not accumulate across deploy cycles.

Ordering Resources with Sync Waves

A Helm chart renders all its resources simultaneously, but Kubernetes cannot guarantee that a database migration job finishes before an API deployment starts unless ordering is explicitly enforced. Sync waves are ArgoCD's mechanism for this: resources tagged with a lower wave number are applied first and must reach a healthy state before higher-wave resources are even scheduled.

In generate_helm_values, the migrations job carries argocd.argoproj.io/sync-wave: "0" alongside argocd.argoproj.io/hook: "PreSync". Wave 0 runs first; only after the migration job completes successfully does ArgoCD schedule the API pods (wave 2). This sequencing prevents the API from starting against a database whose schema is not yet in the expected state — a startup failure that is easy to miss in production because the pod may partially initialize before a schema check triggers a crash (see Code Walkthrough).

Separating Configuration from Credentials

A single Helm chart should serve multiple environments without encoding any environment-specific assumptions. The Application manifest's source.helm.valueFiles field references values-{environment}.yaml, so replica counts, resource limits, and integration URLs live in the values file rather than in the chart. The generate_helm_values function illustrates this: replicaCount is 3 in production and 1 otherwise, and GATEWAY_URL, QUOTA_URL, and REGISTRY_URL point to cluster-internal service names appropriate for each environment.

Credentials stay out of both the chart and the values files entirely. The postgresql.auth.existingSecret: "onboarding-db-creds" reference tells the chart to read database credentials from a Kubernetes Secret that was provisioned separately — outside the GitOps flow — so sensitive values are never committed to the repository. This pattern keeps the chart and its values files safe to store in a shared or version-controlled repository while ensuring secrets remain under a separate access-control boundary.

Code Walkthrough

Now that you understand how ArgoCD Application resources, Helm charts, and sync waves coordinate a GitOps deployment, the implementation binds those concepts together in two artifacts: an Application manifest that tells ArgoCD where to find the chart and how to sync it, and a Helm values structure that configures the onboarding stack and enforces wave ordering.

ArgoCD Application manifest

The generate_argocd_application function produces the manifest ArgoCD reads to manage the onboarding stack. The source block points to the Helm chart repository and selects environment-specific value overrides; syncPolicy.automated enables prune-and-self-heal so the cluster state always converges to what is committed in Git, and CreateNamespace=true lets ArgoCD bootstrap the target namespace on first sync.

Code snippetpython
1def generate_argocd_application(environment: str) -> dict:
2    return {
3        "apiVersion": "argoproj.io/v1alpha1",
4        "kind": "Application",
5        "metadata": {
6            "name": f"onboarding-{environment}",
7            "namespace": "argocd",
8        },
9        "spec": {
10            "project": "platform",
11            "source": {
12                "repoURL": "https://github.com/org/platform-charts",
13                "path": "charts/onboarding",
14                "targetRevision": "main" if environment == "production" else "develop",
15                "helm": {"valueFiles": [f"values-{environment}.yaml"]},
16            },
17            "destination": {
18                "server": "https://kubernetes.default.svc",
19                "namespace": f"platform-{environment}",
20            },
21            "syncPolicy": {
22                "automated": {"prune": True, "selfHeal": True},
23                "syncOptions": ["CreateNamespace=true"],
24            },
25        },
26    }

Helm values and sync-wave ordering

The Helm values control replica counts, resource limits, and the database migration job. Sync waves enforce the deployment sequence described in the Concepts section: the migration job carries argocd.argoproj.io/sync-wave: "0" so it runs and completes before the API pods (wave 2) are scheduled. The existingSecret reference keeps database credentials out of the chart itself, and the integration URLs wire the onboarding service to the gateway, quota, and registry dependencies.

Code snippetpython
1def generate_helm_values(environment: str) -> dict:
2    is_prod = environment == "production"
3    return {
4        "api": {
5            "replicaCount": 3 if is_prod else 1,
6            "image": {"repository": "gcr.io/project/onboarding-api", "tag": "latest"},
7            "resources": {
8                "requests": {"cpu": "250m", "memory": "512Mi"},
9                "limits": {"cpu": "1000m", "memory": "1Gi"},
10            },
11            "env": {
12                "GATEWAY_URL": "http://litellm-gateway:4000",
13                "QUOTA_URL": "http://quota-service:8000",
14                "REGISTRY_URL": "http://model-registry:8000",
15            },
16        },
17        "migrations": {
18            "enabled": True,
19            "annotations": {
20                "argocd.argoproj.io/hook": "PreSync",
21                "argocd.argoproj.io/sync-wave": "0",
22            },
23        },
24        "postgresql": {
25            "enabled": True,
26            "auth": {"existingSecret": "onboarding-db-creds"},
27            "primary": {"persistence": {"size": "10Gi"}},
28        },
29    }

Confirm that argocd app sync onboarding-<environment> completes without errors and that the Application status shows Healthy and Synced in the ArgoCD UI — those two indicators mean the migration job completed in wave 0, the API pods passed their health checks, and the cluster state matches the chart committed in Git.

Do's and Don'ts

Building on the Application manifest, sync-wave ordering, and credential-handling patterns above, the following guidance distils the highest-leverage practices to apply when deploying the onboarding system with ArgoCD — and the pitfalls that most often break a GitOps rollout.

Do's

1. ✓Do set syncPolicy.automated with both prune: True and selfHeal: True — without both flags, manual changes applied directly to the cluster won't be reverted and deleted Git resources will linger, undermining the GitOps guarantee that the cluster state always converges to what is committed.
2. ✓Do assign argocd.argoproj.io/sync-wave: "0" to the database migration job and a higher wave (e.g., "2") to the API pods — wave ordering is what guarantees the migration completes before the API service is scheduled; reversing or collapsing the waves causes the API to start against an uninitialized schema.
3. ✓Do reference database credentials via existingSecret: "onboarding-db-creds" rather than inlining them in Helm values — the existingSecret pattern keeps secrets out of the chart repository entirely, so rotating credentials never requires a chart commit or a redeployment of the Application manifest.

Don'ts

1. ✗Don't omit CreateNamespace=true from syncOptions when deploying to a new environment — without it, ArgoCD will fail the sync with a namespace-not-found error on first apply, requiring a manual kubectl create namespace step that reintroduces the out-of-band cluster changes GitOps is meant to eliminate.
2. ✗Don't point both production and non-production Applications at the same targetRevision: "main" — production should track main while staging tracks develop; pointing all environments at main means untested changes are promoted to production the moment they merge, bypassing the environment progression the generate_argocd_application function encodes.
3. ✗Don't confirm a successful deployment by checking only the ArgoCD UI sync status without also verifying the onboarding endpoint — an Application can show Synced even if the migration job exited non-zero in wave 0 and the API is serving against a broken schema; the Healthy indicator on the API pods and a live endpoint check together confirm that wave ordering and schema initialization completed correctly.