Expose the LLM chat API via an Ingress resource

Introduction

When you've deployed an LLM chat API behind a ClusterIP Service, no one outside the cluster can reach it — your frontend can't call it, a demo audience can't see it, and external monitors can't probe it. Provisioning a separate cloud load balancer per Service quickly becomes expensive and unmanageable as your GenAI platform grows. The Ingress resource solves this by giving you a single HTTP/HTTPS entry point that routes external traffic to the correct backend Service based on the request's host and path.

By the end of this lesson you'll be able to install an Ingress controller, write an Ingress resource that exposes your LLM chat API on a public hostname, split traffic across paths and hosts, and verify the routing from outside the cluster.

Key Terminology

• Ingress resource — a Kubernetes object that declares HTTP/HTTPS routing rules (host + path → Service); it's a spec, not a running component, so it does nothing on its own.
• Ingress controller — a pod running a reverse proxy (NGINX, Traefik, Envoy) that watches Ingress resources and configures itself to actually route the traffic they describe.
• Host-based routing — switching backends on the HTTP Host header so chat.example.com and embeddings.example.com can share one entry point.
• Path-based routing — switching backends on the URL path so /api/chat and /api/embeddings reach different Services under the same hostname.
• ingressClassName — the field on an Ingress that picks which controller handles it; required whenever the cluster runs more than one controller.

Concepts

Ingress resource vs. Ingress controller

An Ingress resource is purely declarative. Writing one says "when a request matches host X and path Y, send it to Service Z" — but nothing actually listens until an Ingress controller is running. The controller is typically an NGINX pod that watches the Kubernetes API for Ingress resources and rewrites its proxy config in response. Installing the controller also creates a LoadBalancer Service that provisions one cloud load balancer with a stable external IP — that IP is the single front door for every Ingress in the cluster.

The external user hits the cloud load balancer, which forwards raw TCP to the Ingress controller pod. The controller terminates TLS — meaning it completes the HTTPS handshake, decrypts the incoming request, and forwards plain HTTP onward to the backend — then inspects the Host header and URL path, and forwards the request to the ClusterIP Service that fronts your chat pods. Terminating TLS at the controller centralizes certificate management and removes cryptographic work from the application pods.

Host-based and path-based routing

Routing rules live in the Ingress spec.rules array. Each rule binds a hostname to a set of paths, and each path binds to a backend Service. You can use these dimensions independently or together (see Code Walkthrough):

• Host-based: chat.example.com → llm-chat-svc, embeddings.example.com → embeddings-svc. Both DNS records resolve to the same controller IP; the controller switches on Host.
• Path-based: under one hostname, /api/chat → llm-chat-svc, /api/embeddings → embeddings-svc. The controller longest-prefix-matches the path.

This lets a single Ingress controller — and a single cloud load balancer — serve every public endpoint of your GenAI platform.

pathType and defaultBackend

Two details determine the edges of routing behavior:

• pathType is Prefix for URL-prefix matches and Exact for full-path equality. Prefix: / is the catch-all under a hostname.
• defaultBackend on the Ingress spec catches requests that hit the controller but match no rule (e.g. an unknown hostname). Without it, unmatched requests get a generic 404 from NGINX.

Code Walkthrough

Now that you've seen the routing dimensions and the role of pathType, the manifest below ties those concepts together in one Ingress: host-based routing for chat.example.com, path-based routing that splits /api (chat backend) from /health (probe backend), and a defaultBackend to soak up everything else.

Code snippetyaml
1# llm-chat-ingress.yaml
2apiVersion: networking.k8s.io/v1
3kind: Ingress
4metadata:
5  name: llm-chat-ingress
6spec:
7  ingressClassName: nginx
8  defaultBackend:
9    service:
10      name: default-backend-svc
11      port:
12        number: 80
13  rules:
14  - host: chat.example.com
15    http:
16      paths:
17      - path: /api
18        pathType: Prefix
19        backend:
20          service:
21            name: llm-chat-svc
22            port:
23              number: 80
24      - path: /health
25        pathType: Prefix
26        backend:
27          service:
28            name: health-svc
29            port:
30              number: 80

• ingressClassName: nginx selects the NGINX controller you installed (e.g. helm install ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx --create-namespace).
• The host rule restricts these routes to requests whose Host header is chat.example.com; other hostnames fall through to defaultBackend.
• Path rules match longest-prefix-first: chat.example.com/api/v1/messages hits llm-chat-svc, chat.example.com/health hits health-svc, anything else under that host (e.g. /) hits the default backend.

Apply it and exercise the route from outside the cluster without DNS by forcing curl to resolve the hostname to the controller's external IP:

Code snippetbash
1kubectl apply -f llm-chat-ingress.yaml
2
3INGRESS_IP=$(kubectl get svc ingress-nginx-controller \
4  -n ingress-nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
5
6curl --resolve chat.example.com:80:$INGRESS_IP \
7  -H "Content-Type: application/json" \
8  -d '{"message":"hello"}' \
9  http://chat.example.com/api/v1/chat

You'll know it works when kubectl get ingress llm-chat-ingress shows an ADDRESS populated with the controller's external IP, kubectl describe ingress llm-chat-ingress lists chat pod endpoints under each rule, and the curl above returns the chat API's JSON response (not a 404 from the default backend).

Do's and Don'ts

Having just walked through a working Ingress manifest, keep these practices in mind to keep your routing reliable and cost-effective.

Do's

1. ✓Set ingressClassName explicitly — clusters frequently run multiple controllers (one for public traffic, another for internal), and omitting it leaves the routing ambiguous.
2. ✓Front every public Service through the same Ingress controller — you pay for one cloud load balancer instead of one per Service, and TLS certificates live in one place.
3. ✓Add a defaultBackend — unmatched requests then hit a controlled response page instead of NGINX's generic 404, which is easier to debug and friendlier to clients.

Don'ts

1. ✗Don't create an Ingress without a controller installed — the resource will be accepted but no traffic flows, and an empty ADDRESS column in kubectl get ingress is the only clue.
2. ✗Don't let path rules overlap carelessly — mix Exact and Prefix with intent, or you'll get surprising routing where a more specific path is shadowed by a broader one.
3. ✗Don't expose admin or debug endpoints under the same hostname as the chat API — give them their own host rule (admin.example.com) so you can later restrict access with auth, IP allowlists, or a separate controller.