Generators & Iterators — chapter audio overview

Podcast Script: Generators & Iterators Host: Welcome to LLM Foundations — this is Chapter 1 of 20, covering Generators and Iterators in Python. Now, if you are coming into this wondering why a course on large language models opens with iteration patterns, that is fair — but here is the framing. Your organization has invested in building genuine production capability, not simply wiring up API calls. And the difference between a team that ships a working demo and a team that ships a system handling millions of tokens per day comes down to foundations like the one we are covering today. Streaming responses, processing large datasets, building agent data flows — every one of those applications rests on a pattern Python calls the iteration protocol. In this chapter you will build the mental model, and then apply it across five hands-on lab objectives. We will explore how Python iterates under the hood, how the yield keyword creates lazy values, how generator expressions let you compose processing inline, how chaining generators forms pipelines, and how the itertools module provides battle-tested building blocks. To start — what is happening when Python executes a for loop? Expert: Great place to start, because most developers use for loops every day without understanding the protocol underneath. Here is the mental model. When Python iterates over a collection — a list, a file, a range — it is following a formal contract called the iteration protocol. Any object that wants to be iterable has to provide two methods. The first is an iter method, which returns an iterator. The second is a next method, which returns the following value in the sequence. When there are no more values, the next method raises a special signal called StopIteration, and that signal tells Python's for loop to stop. Now, notice I said two different words there — iterable and iterator. These sound similar but they are distinct. An iterable is any object that can produce an iterator when requested. A list is iterable. A file is iterable. A range is iterable. But the list itself does not do the iterating — it hands you a separate iterator object, and that iterator tracks state and returns the next value each time. This matters because iterators are single-use. Once you consume all the values, the iterator is exhausted. The list, on the other hand, you can iterate over multiple times because each pass Python requests a fresh iterator. Python provides two built-in functions that let you drive this protocol manually. The iter function takes an iterable and returns its iterator. The next function takes an iterator and returns the following value. You can use these to step through a sequence one value at a time, which is useful for understanding what is happening and for fine-grained control. You can also pass a default value to the next function, and if the iterator is exhausted, you receive the default instead of an exception. Where this matters in production: when you are processing a stream of tokens from a language model, or reading lines from a file larger than memory, or iterating over records in a dataset, you are using this protocol. Understanding it means you can debug iteration bugs, compose iterators cleanly, and recognize when a section of code exhausts a stream before it should. That last one is a classic production failure — code that iterates over the same generator twice and silently produces empty results on the second pass. Host: So the iteration protocol is the foundation — two methods, iter and next, plus the StopIteration signal. And the key subtlety is that iterators are single-use while iterables like lists can provide fresh iterators on every pass. Now, implementing those two methods in a full class every time you need custom iteration would be painful. What is the Pythonic way? Expert: This is where generators come in, and honestly, they are one of the most elegant features in Python. A generator is a special form of iterator that you create with a simple function — no class required. The magic is the yield keyword. When you write a function and place yield inside it, Python transforms that function into a generator function. Calling it does not execute the body — instead, it returns a generator object, which is an iterator. Here is the mental model for yield: picture it as pause-and-return. When the for loop requests the next value, Python executes the function body until it hits a yield statement. At that point, the yielded value is returned to the caller, and the complete state of the function is frozen — local variables, the position in the code, every detail. The next time a value is requested, execution resumes right after the yield, with all state intact. When the function finally returns, Python raises StopIteration automatically. This matters for two reasons. First, code simplicity. Writing a counter generator takes a handful of lines, versus a full class with two methods. Second, and more important, memory efficiency. A generator uses constant memory regardless of how many values it produces. If you compute a million squares as a list, Python allocates roughly eight megabytes to hold them all. If you compute the same values as a generator, the object itself is a couple of hundred bytes, and only one value exists in memory at any point in time. For AI applications processing datasets that do not fit in RAM, that difference is not a nice-to-have — it is the difference between successful execution and process termination. Generators can also have multiple yield statements. You might yield a header, then loop and yield items, then yield a footer — and each yield is a separate pause point. This is the natural pattern for producing structured output like formatted reports or streaming API responses. Each yield hands control back to the consumer, which processes the value and then requests the following one. One more concept here: the yield-from form. This lets a generator delegate to a sub-generator. Instead of manually iterating over a sub-generator and re-yielding each value, you write yield from and Python handles the delegation. This is essential when you are flattening nested structures or composing pipelines out of smaller generator functions. It keeps your code compact and avoids the bug-prone pattern of hand-rolled delegation loops. Host: So yield pauses the function, preserves state, and resumes on the following call — giving you an iterator without implementing the protocol yourself. And the memory story is dramatic: constant bytes regardless of the data size. Now, Python also has a compact syntax for simple generators, closely related to list comprehensions. When do I reach for that form? Expert: Generator expressions are the compact form, and the syntax is almost identical to a list comprehension — same pattern, same filtering, but with parentheses instead of square brackets. That one-character change produces a fundamentally different object. Square brackets build a complete list in memory immediately. Parentheses produce a generator that yields values lazily, one at a time, as you iterate. Why does this matter in practice? Consider summing the squares of a million numbers. With a list comprehension, Python allocates the complete list first, then sums it — double the memory work. With a generator expression passed to the sum function, values are produced and consumed one at a time. Same result, a tiny fraction of the memory footprint. The same pattern applies to the any and all functions, which short-circuit as soon as they find a match or non-match. Pair them with a generator expression and you receive efficient, early-terminating logic. There is a nice syntactic detail here. When a generator expression is the only argument to a function, you do not need extra parentheses — the function's own parentheses serve double duty. So you end up with very clean, readable code. You will see this pattern everywhere in production Python: sum of a generator, any of a generator, join of a generator. It is idiomatic. But there is a limit. Generator expressions are for simple transformations — one filter, one map operation, a transform you can express on a single line. The moment your logic needs multiple yields, exception handling, complex branching, or a descriptive name, switch to a full generator function. Do not try to cram complex multi-line logic into a generator expression. It becomes unreadable fast, and you lose the ability to add docstrings or comments explaining what is happening. One more gotcha: generator expressions, like all generators, are single-use. If you assign one to a variable, iterate over it once, and then try to iterate again, the second pass produces nothing. This catches people. If you need to iterate multiple times, either materialize the values into a list, or recreate the generator expression. Host: So the compact form is perfect for simple transformations fed directly into consuming functions like sum, any, and join — but the moment the logic grows, promote it to a full generator function. Now, individual generators are useful, but the real power comes from connecting them. How does the pipeline pattern work? Expert: This is where generators stop being a neat syntactic feature and become an architectural pattern. Data pipelines. The pattern is simple but powerful: each stage is a generator that takes an iterable as input, transforms each value, and yields the result to the following stage. You chain them together, and data flows through the complete pipeline one item at a time — never loaded entirely into memory. Here is a concrete example. Imagine processing a multi-gigabyte file of JSON records. Stage one is a generator that reads the file line by line and yields each raw line. Stage two takes those lines and filters out empty ones. Stage three parses each line as JSON, skipping any that fail to parse. Stage four extracts a specific field from each parsed record. You wire these up by passing the output of each stage as the input to the following stage, and then you iterate over the final result. Data flows through: read one line, filter it, parse it, extract the field, produce the value, repeat. At no point does the complete dataset sit in memory. This composes beautifully. Each stage is a small, testable generator function. You can swap stages, add new ones, reorder them. And because the complete pipeline is lazy, nothing executes until you actually consume values at the end. That lazy evaluation is the superpower — you can define a pipeline that conceptually operates on a terabyte of data, but it only processes what you consume. The yield-from statement I mentioned earlier plays a role here too. If you have a generator that needs to produce values from several sub-generators in sequence, yield from handles the delegation cleanly. It is also the right pattern for recursive generators — flattening nested structures, traversing tree-shaped data, operations like that. A production tip: keep each stage focused on one transform. Do not build deeply nested generator chains that are hard to debug. Three to five stages is a comfortable range. Beyond that, you start losing the ability to reason about where a bug is happening. And always test with a small dataset first. Run the pipeline on a thousand records and verify the output before pointing it at the full production file. Host: Pipelines give you memory-efficient composition — each stage a small generator, data flowing one item at a time. Python also ships with a standard library module full of pre-built iterator tools. What is in itertools that I should understand? Expert: The itertools module is a standard library collection of highly optimized iterator functions, implemented in C for maximum performance. Learning it is part of writing professional Python. I will cover the three most important for AI applications. First, islice. This is the iterator equivalent of list slicing. Regular Python slicing with square brackets creates a new list, which defeats the purpose when you are working with generators or infinite streams. The islice function lets you take a slice without materializing the data. You can grab the first five values from an infinite counter, skip a hundred and take the following fifty, or apply a start, stop, and step pattern for sampling. It is essential for pagination, previewing large files, and batching operations over streams. Second, chain. This combines multiple iterables into a single sequential stream. You pass in several lists, several generators, whatever — and chain yields from each in turn. No copying, no extra memory. A common production use is adding a header and a footer around a data stream by chaining a header value, the data generator, and a footer value. There is also a chain-from-iterable variant that flattens a collection of collections — handy when you have nested iterables. Third, groupby. This groups consecutive elements that share a key. The critical word is consecutive — groupby does not sort for you. You must sort the data by the grouping key first, then groupby iterates through and batches neighbors together. This is the idiomatic way to aggregate log entries by type, or count runs of repeated values. Other functions worth understanding: count for an infinite counter with optional start and step, cycle to repeat an iterable forever, repeat for a constant value, takewhile and dropwhile for conditional slicing, and zip_longest for pairing sequences of different lengths with a fill value. Every one of these is a building block. When you find yourself writing a custom loop to perform one of these operations, stop and check itertools first. The C implementation will be faster and more reliable than hand-rolled code. One warning: itertools also provides a tee function that splits one iterator into multiple independent copies. It sounds useful, but under the hood it stores values in memory to feed the slower consumer. Use it sparingly and only when you understand the memory implications. Host: So itertools gives you islice for lazy slicing, chain for combining streams, and groupby for aggregation on pre-sorted data — plus a complete set of other building blocks. Let's return to language models. How do these patterns show up in production AI systems, and what production wisdom should I carry forward? Expert: Several patterns dominate. First, streaming LLM responses. When you call a language model API with streaming enabled, the response arrives as a sequence of token chunks over time. The natural form for handling this is a generator that yields each token as it arrives. The calling code iterates and can print tokens to the screen for real-time display, accumulate them into a complete response, count them, or feed them into subsequent processing — all while the model is still generating. This is how every production chat interface feels responsive. Second, processing training data. Fine-tuning a model typically means feeding it a large dataset of example conversations in a specific format, often JSON lines. The dataset might be tens of gigabytes. A generator reads the file line by line, parses each JSON record, formats it into the required structure — usually a messages list with roles and content — and yields one training example at a time. The complete dataset never loads into memory. Pair this with a batch iterator, built on islice, and you can upload examples in groups of a hundred or a thousand, maintaining high throughput without memory pressure. Third, agent data flows. Agents process incoming messages through multiple transformation stages — parsing, content extraction, chunking long text into pieces that fit the model's context window, formatting for the following stage. Each stage is a generator. Compose them into a pipeline, iterate at the end, and you have a clean, memory-efficient agent data flow. Now the production wisdom. If you remember no other detail from this chapter: use generators for any dataset that does not comfortably fit in memory, and chain them into pipelines for clean composition. Respect the single-use nature of iterators — a generator, once consumed, produces no more values, and silent empty results on a second pass is one of the most common production bugs in this space. Do not reinvent itertools; the C-optimized functions are faster and more tested than hand-rolled code. And the top never-do: do not convert a generator into a list just to iterate it. That defeats the complete memory-efficiency purpose. Two more considerations. Generators that hold file handles or network connections need proper resource cleanup — use context managers inside the generator so resources release even when iteration is abandoned early. And generators are not thread-safe by default. If you share one across threads, you need explicit synchronization. Host: In the lab, you will apply each of these hands-on. Objective one has you implement the iteration protocol from scratch in a custom class, so the iter and next methods become muscle memory. Objective two walks you through building generators with yield, including memory comparisons against list-based code. Objective three gives you practice with generator expressions and the consuming functions like sum, any, and all. Objective four has you compose a complete data pipeline with multiple chained generator stages. And objective five puts itertools into your hands — islice, chain, groupby, and the other building blocks. Each lab has its own audio overview that goes deeper into the exercise. Here is what you now understand. You understand Python's iteration protocol — the iter and next methods, the StopIteration signal, and the distinction between iterables and iterators. You understand how the yield keyword transforms a function into a generator that pauses and resumes while preserving state, producing values lazily with constant memory. And you understand how to chain generators into data pipelines and apply the itertools module to build efficient, production-grade data flows. You now have the depth to evaluate data processing approaches for your team's AI infrastructure — and to explain the memory and streaming trade-offs to stakeholders choosing between list-based and generator-based code. The chapter quiz will focus on Python's iteration protocol, the behavior of yield, the single-use nature of generators, and when to reach for itertools building blocks. Pay particular attention to the difference between iterables and iterators, and to the memory characteristics of generator expressions versus list comprehensions. In the following chapter, we move from generators to async programming basics — the same lazy-evaluation mindset, applied to concurrent operations like parallel API calls to language models. Generators prepare the ground for async, because both rest on the idea of suspending and resuming execution. See you in Chapter 2.