Host: Welcome back. You're in AI Developer Platform Engineering — a course about building the internal tools and infrastructure that let AI teams ship models and applications without getting stuck in plumbing. This chapter is called Internal Developer Platform Vision, and it's about designing the self-service systems that a platform team offers to the rest of the engineering organization. Today's skill is observability — making your platform visible so you can actually run it in production.
Expert: Let me paint the picture. Imagine you're on a platform engineering team at a mid-sized company — maybe three hundred engineers, a handful of AI products in production. Your team has built what's called an internal developer platform. That's basically a self-service system where any engineer in the company can click a button and get a new environment, a new model endpoint, or a new data pipeline — without filing a ticket and waiting three days. It's working. People love it. And then one Tuesday morning, Slack lights up. "The platform is slow." "My provisioning request failed." "Is something broken?" And here's the problem — you have no idea. You built the platform, but you didn't build the instruments to watch it. You're flying blind. This is the moment every platform team hits, and the fix is instrumentation — teaching your system to report on itself, continuously, so you know what's happening before your users tell you.
Host: So this is exercise one of three in this sequence, and it's your starting point — no prior exercise to build on. Let's get concrete. What exactly will we build today?
Expert: You'll add instrumentation to a platform's internal web service — the part that handles requests like "create me a new environment" or "give me a model endpoint." Right now, that service just does its job silently. Your job is to make it talk. Specifically, you'll have it report three things. First, how many requests it's handling — just raw counts, ticking up every time something comes in. Second, how many of those requests are failing versus succeeding. And third, how long each request is taking, broken down into buckets — so you can see not just the average, but the shape of the distribution. Fast requests, slow requests, really slow requests. The tool you'll use for this is called Prometheus. Prometheus is an open-source monitoring system — it's the industry standard for this kind of work. Your service exposes its numbers, Prometheus scrapes them on a schedule, and now you have a living picture of your platform's health. Two specific kinds of measurements matter here — a counter, which is just a number that goes up, and a histogram, which bundles measurements into buckets so you can see distributions, not just averages.
Host: Okay, before someone starts typing — what's the thing that trips people up here?
Expert: The tempting mistake is to instrument everything, everywhere, with maximum detail. Tag every request with the user ID, the environment name, the request body size, the time of day. It feels thorough. It's actually a trap. Every unique combination of tags creates a new time series that Prometheus has to store, and the cost explodes fast. This is called high cardinality, and it's the number one way teams blow up their monitoring bill and slow down their dashboards. The rule of thumb: tag by things with a small, bounded set of values — the endpoint name, the HTTP status code, maybe the service name. Never tag by user ID, request ID, or anything unbounded. Start small, add tags deliberately, and your future self will thank you.
Host: Great. So what will someone walk away able to do after this exercise?
Expert: After this, you'll be able to take any internal service and add production-grade observability to it — so your team can actually see what's happening when something goes wrong at three in the morning. This is a foundational building block. Every dashboard, every alert, every on-call rotation downstream depends on the numbers you're exposing here. It's the kind of skill that turns a platform from a demo into something your organization can actually depend on — and it's exactly the kind of contribution that makes a platform team credible inside a larger engineering org.
Host: And what's coming next?
Expert: Next, you'll go deeper on one of these three measurements — the latency histograms. You'll build a more sophisticated version that tracks how long provisioning operations take, with configurable bucket boundaries, percentile computation, and alert thresholds wired in. Today's exercise gets the instrumentation in place. The next one turns those raw numbers into the signal your team will actually page on when things go wrong. After that, you'll build the dashboard that visualizes all of it. Three exercises, one complete observability story for your platform. Thanks for listening, and good luck with the build.
Want to go deeper? Explore disciplines with hands-on labs, quizzes, and chapter podcasts.