Generators & yield — How Iteration Works Without Eating Memory
The mechanism behind yield "pausing" function execution
Lists load everything into memory. 100 million items = 100 million items worth of memory.
# Eats all memory
numbers = [x * 2 for x in range(100_000_000)]
# Almost no memory
numbers = (x * 2 for x in range(100_000_000))
The second is a generator expression. Values aren't pre-created — computed one at a time on each next() call.
What yield Does
Normal functions destroy the stack frame on return. Local variables, execution position — all gone.
yield is different. It returns a value while preserving the stack frame. Next next() call resumes from the line after yield. Local variables intact.
CPython Internals
In CPython, generators are PyGenObject. gi_frame holds the preserved frame, gi_code stores bytecode. next() calls _PyEval_EvalFrameDefault resuming from the preserved frame.
Unlike normal function calls, no new frame is created — the existing frame is reused. This is why generators are lightweight.
Key Points
yield returns a value while preserving the function stack frame
next() resumes from the line after yield — local variables preserved
In CPython, frame preserved in PyGenObject.gi_frame
send() injects values at yield points — basis for coroutines