Build multi-container agent pods

Introduction

Engineers often need to run an AI agent alongside supporting services—a model proxy, a logging sidecar, or a shared-storage helper—but packaging multiple processes into a single deployable unit without the right primitives leads to fragile, hard-to-debug containers. Kubernetes solves this with the pod, a group of co-scheduled containers that share a network namespace and loopback interface. By the end of this lesson, you will know how to author a multi-container pod manifest and use Podman to validate it locally before deploying to a cluster.

Key Terminology

• Pod — A Kubernetes scheduling unit that groups one or more containers under a shared network namespace and loopback interface, allowing co-located processes like agent and model-proxy to communicate over localhost without external routing.
• Shared Network Namespace — The networking context that all containers within a pod inherit, meaning every container sees the same loopback (lo) interface and can reach sibling containers via localhost:<port> as if they were local processes on the same host.
• Sidecar Container — A secondary container co-deployed in the same pod as the main application container to provide a supporting concern—in this lesson, the model-proxy sidecar handles model traffic so the agent container can delegate all proxy logic to http://localhost:8080.
• Pod Manifest — A Kubernetes YAML file (e.g., agent-pod.yaml) that declares the pod's metadata, spec, and the ordered list of containers, including each container's image, environment variables, and exposed ports.
• podman play kube — A Podman command that interprets a Kubernetes pod manifest and instantiates the pod locally, enabling developers to catch image-pull errors, environment-variable typos, and port conflicts before applying the manifest to a live cluster.
• podman pod logs — A Podman command (used with -f to stream) that aggregates and tails log output from all containers in a named pod, allowing you to confirm that agent reached the proxy and that model-proxy is listening without requiring cluster access.

Concepts

Why Pods Exist: Co-location Without Coupling

Kubernetes does not schedule individual containers — it schedules pods. A pod is the smallest deployable unit, and its defining property is that all containers inside it share a single network namespace. This means they share one IP address and one loopback interface. From any container's perspective, a sibling container's port is reachable on localhost, exactly as if it were a process running directly on the same machine.

This design solves a recurring problem in agent deployments: your agent process needs a tightly coupled helper — a model proxy, a logging collector, a secrets refresher — but that helper is not part of the agent's own image. Splitting them into separate pods forces you to route traffic through Kubernetes Services, manage separate IP addresses, and accept network hops that add latency and operational surface area. Putting them in the same pod collapses all of that: the helper is a sidecar container, and the call is just http://localhost:<port>.

The Sidecar Pattern in Practice

When the agent container needs to call a language model, it points to http://localhost:8080 — an environment variable (MODEL_PROXY_URL) that bakes in the assumption that a proxy is always reachable on that port. The model-proxy container, declared alongside it in spec.containers, binds to port 8080 and handles all outbound model traffic. Neither container needs to know the other's IP; the shared loopback makes the address constant regardless of which node the pod lands on.

This is the sidecar pattern: the main container owns the business logic, and the sidecar owns a cross-cutting concern. Because both containers live and die together — Kubernetes schedules and terminates them as a unit — there is no window where the agent is running but the proxy is absent (see Code Walkthrough for the manifest that wires this up).

Local Validation with podman play kube

Before a pod manifest reaches a cluster, it should be validated locally. Podman accepts the same YAML that kubectl apply does, and podman play kube instantiates the full pod on your workstation using locally built images. This catches the most common pre-deploy failures — a missing image tag, a misspelled environment variable, a port already bound on the host — without consuming cluster resources or requiring credentials.

The local workflow mirrors the cluster workflow intentionally: build both images with podman build, start the pod with podman play kube, and verify steady state with podman pod ps. Once agent-pod shows Running with both containers healthy and podman pod logs -f agent-pod shows the expected startup messages from each container, the manifest is ready to commit. The cluster will see exactly what Podman ran locally.

Code Walkthrough

Now that you understand how pods group containers under a shared network namespace, the next step is to express that structure in a manifest and validate it locally with Podman before promoting it to a cluster.

The manifest below declares a two-container pod: the main agent container runs your agent process and communicates with a model-proxy sidecar over localhost:8080. Both containers share the pod's loopback interface, so no Kubernetes Service or external routing is required for that intra-pod call.

Code snippetyaml
1# agent-pod.yaml
2apiVersion: v1
3kind: Pod
4metadata:
5  name: agent-pod
6  labels:
7    app: agent
8spec:
9  containers:
10    - name: agent
11      image: localhost/agent:1.0.0
12      env:
13        - name: MODEL_PROXY_URL
14          value: "http://localhost:8080"
15      ports:
16        - containerPort: 9000
17    - name: model-proxy
18      image: localhost/model-proxy:1.0.0
19      ports:
20        - containerPort: 8080

Before applying this manifest to a live cluster, use podman play kube to spin up the pod locally. Podman interprets the same Kubernetes YAML that kubectl apply accepts, letting you catch image-pull errors, environment-variable typos, and port conflicts on your workstation without touching the cluster.

Code snippetbash
1# Build both images with Podman
2podman build -t localhost/agent:1.0.0 ./agent
3podman build -t localhost/model-proxy:1.0.0 ./model-proxy
4
5# Run the pod from the manifest
6podman play kube agent-pod.yaml
7
8# Stream logs from both containers
9podman pod logs -f agent-pod

Once the pod is running, the agent container should log that it reached the proxy on http://localhost:8080, and the model-proxy container should log that it is listening on port 8080. If either container exits with a non-zero code, podman pod logs will surface the failure immediately—no cluster access needed.

Check that podman pod ps shows agent-pod in Running state with both containers healthy before committing the manifest to your repository.

Do's and Don'ts

Having walked through building multi-container agent pods above, the following Do's and Don'ts distill it into practice.

Do's

1. ✓Do route intra-pod agent-to-sidecar traffic over localhost — because all containers in a pod share the same network namespace and loopback interface, setting MODEL_PROXY_URL=http://localhost:8080 eliminates the need for a Kubernetes Service or external routing, keeping latency near zero and the topology simple.
2. ✓Do run podman play kube agent-pod.yaml locally before applying to a cluster — Podman interprets the same YAML kubectl apply accepts, so image-pull errors, environment-variable typos, and port conflicts surface on your workstation where iteration is fast and cluster access is not required.
3. ✓Do confirm podman pod ps shows the pod in Running state with both containers healthy before committing the manifest — checking that the agent container logs a successful reach to http://localhost:8080 and model-proxy logs that it is listening on port 8080 proves the shared-namespace wiring is correct, not just that the containers started.

Don'ts

1. ✗Don't add a Kubernetes Service or external routing to connect the agent container to the model-proxy sidecar — because they share the pod's loopback interface, treating intra-pod communication like inter-pod communication adds unnecessary network hops and breaks the co-scheduling guarantee that makes the pattern work.
2. ✗Don't skip podman build for both images before running podman play kube — if either localhost/agent:1.0.0 or localhost/model-proxy:1.0.0 is absent or stale, podman play kube fails with an image-pull error that looks identical to a manifest mistake, wasting debugging time on the wrong layer.
3. ✗Don't ignore a non-zero container exit code surfaced by podman pod logs — a container that crashes and restarts silently can make the pod appear Running in podman pod ps while the agent is repeatedly failing to reach the proxy; podman pod logs -f agent-pod is the only local signal that distinguishes a healthy startup from a crash loop before the manifest reaches the cluster.