Community Article

Iterators, Generators, and Async Generators

One protocol, three layers. The iterator protocol with its single next method, generators as sugar over it, and async generators for streaming data with back-pressure. The lazy pipeline pattern I reach for every week.

Iterators, Generators, and Async Generators

iterators

generators

async-await

iteration-patterns

stream-processing

By @owentoure

November 24, 2025

Updated May 18, 2026

174 views

4.4 (11)

When I was first introduced to generators, I read three articles, watched a video, and walked away with the impression that they were a clever solution to an obscure problem. Five years later, I use them constantly: streaming a paginated API, backpressure-aware queue draining, lazy parser pipelines, infinite sequences for property tests. The thing that flipped my perception was learning that iterators, generators, and async generators are not three separate features. They are one protocol with three layers. Once that lands, the syntax stops being decorative and starts being useful.

The argument I want to make here is that you should learn the iterator protocol first, see how generators are sugar over the protocol second, and treat async generators as the same idea over async data third. The order matters, because the protocol is the one piece you need; the rest is shorthand.

The iterator protocol, one method, two return shapes

An iterator is any object that has a next method. Calling next returns an object of the shape { value, done }. When done is false, the consumer reads value and calls next again. When done is true, the iterator has nothing left to yield.

function rangeIterator(start, end) {
    let current = start;
    return {
        next() {
            if (current < end) {
                return { value: current++, done: false };
            }
            return { value: undefined, done: true };
        },
    };
}

const it = rangeIterator(1, 4);
it.next();   // { value: 1, done: false }
it.next();   // { value: 2, done: false }
it.next();   // { value: 3, done: false }
it.next();   // { value: undefined, done: true }

That is the entire protocol. Two pieces (a next method and the { value, done } shape), and any object that satisfies them is an iterator.

The second half of the protocol, the iterable half, says that an object is iterable if it has a method at the well-known symbol Symbol.iterator that returns an iterator. for ... of, spread, destructuring, Promise.all, and dozens of other constructs all consume iterables; they call obj[Symbol.iterator]() to get an iterator and then drive it.

const myRange = {
    [Symbol.iterator]() {
        return rangeIterator(1, 4);
    },
};
for (const n of myRange) console.log(n);   // 1, 2, 3
[...myRange];                                // [1, 2, 3]

Built-in iterables (arrays, strings, Map, Set, NodeList in modern browsers) all implement this same pair: an object with a Symbol.iterator method that returns an iterator with a next method. There is nothing else to learn at this layer.

Generators are sugar over the iterator protocol

Writing iterators by hand is tedious. You have to maintain state on a closure or object, write a next method, return the right shape, and remember to set done correctly. Generators automate the entire thing.

function* range(start, end) {
    for (let i = start; i < end; i++) {
        yield i;
    }
}

const it = range(1, 4);
it.next();   // { value: 1, done: false }
it.next();   // { value: 2, done: false }
it.next();   // { value: 3, done: false }
it.next();   // { value: undefined, done: true }

function* produces a generator function. Calling it returns an iterator (specifically, an iterator that is also iterable, because generators set up Symbol.iterator to return themselves). Each yield pauses the function and emits the value as { value: X, done: false }. When the function returns (or runs off the end), the iterator emits { value: undefined, done: true }.

The win is not just less code; it is the ability to keep arbitrary execution state on the stack between yields. Recursive generators, generators that emit interleaved values from multiple sources, generators that thread state through many calls without allocating an explicit state object: all of it falls out for free.

function* take(iter, n) {
    let i = 0;
    for (const v of iter) {
        if (i >= n) return;
        yield v;
        i++;
    }
}
function* naturals() {
    let n = 1;
    while (true) yield n++;
}

[...take(naturals(), 5)];   // [1, 2, 3, 4, 5]

naturals is an infinite generator; take consumes a finite prefix. Neither generator holds the unproduced values in memory; each yield produces one value, the consumer pulls the next, and that is it. The lazy evaluation is the entire reason generators are valuable for streams.

`yield*` delegates to another iterable

The detail that turned generators from "interesting" to "useful in real code" for me was yield*. Inside a generator, yield* followed by an iterable forwards every value from that iterable, then continues with the rest of the body.

function* concat(...iters) {
    for (const it of iters) {
        yield* it;
    }
}

[...concat([1, 2], [3, 4], naturals())];   // infinite, but the first 4 values are 1,2,3,4

yield* makes generators composable. You can write small generators (take, map, filter, concat, interleave) and combine them like Unix pipes. The standard Array methods are the eager equivalent; the lazy generator equivalents are sometimes faster, often more memory-efficient, and almost always more readable for streaming data.

Generators receive values too: the two-way protocol

A generator's yield expression has a return value. When the consumer calls it.next(x), the x becomes the value of the most recent yield expression inside the generator. That makes generators bidirectional: the generator can emit values out, and the caller can push values back in.

function* echo() {
    while (true) {
        const incoming = yield;
        console.log('received', incoming);
    }
}

const e = echo();
e.next();              // run up to first yield
e.next('hello');       // received hello
e.next('world');       // received world

I rarely use this in product code; I use it constantly in test utilities, mocks, and parser combinators. The bidirectional protocol is what makes co-routine-style generators (the foundation of pre-async-await libraries like co) possible.

Async generators: same idea, async-aware

An async generator is a generator that yields promises and is consumed with for await ... of. The protocol is the same, with next returning a promise of { value, done } instead of the bare object.

async function* paginated(url) {
    let next = url;
    while (next) {
        const res = await fetch(next);
        const page = await res.json();
        yield* page.items;
        next = page.next_url;
    }
}

for await (const item of paginated('/api/things?page=1')) {
    console.log(item);
}

This is the pattern I reach for whenever I am consuming a paginated API and want the consumer to see one item at a time without loading the whole result into memory. The generator handles pagination internally; the consumer treats the result as a stream. The same shape works for cursor-based DB queries, server-sent events, file streams, and any source where the data arrives in chunks but downstream code wants one item at a time.

The for await ... of loop is doing two things on every iteration: calling await on the promise returned by next, then unpacking the { value, done } from the resolved object. The await is what makes back-pressure natural: the producer cannot get ahead of the consumer, because the consumer's loop body must finish before the next await runs.

A pattern I use weekly: lazy pipeline composition

The combination of generators and async generators turns into a small library of useful primitives. The four I keep in a personal utils file:

async function* mapAsync(source, fn) {
    for await (const v of source) yield await fn(v);
}
async function* filterAsync(source, predicate) {
    for await (const v of source) if (await predicate(v)) yield v;
}
async function* takeAsync(source, n) {
    let i = 0;
    for await (const v of source) {
        if (i++ >= n) return;
        yield v;
    }
}
async function collect(source) {
    const out = [];
    for await (const v of source) out.push(v);
    return out;
}

Compose: await collect(takeAsync(filterAsync(mapAsync(paginated(url), enrich), keep), 100)). The pipeline never materialises the full stream; values flow through one at a time, with back-pressure baked in. For the kinds of workloads I write (data ingestion, ETL, log processing), this composition pattern is more useful than any third-party stream library I have tried, and it is built entirely from language features.

Where I would not use them

Generators are not free. The state machine the engine builds for function* has more overhead than a plain function, the heap allocation of the iterator object happens on every call, and the for ... of driver has more bookkeeping than a for loop over an array index. For tight numeric loops with a known finite size, a plain array and a plain for loop is faster.

Async generators have an extra cost: each await is a microtask. Iterating through 100,000 items via for await ... of enqueues 100,000 microtasks, even if every value is already resolved. For most network-bound workloads this is in the noise; for in-memory transformations, plain generators or array methods are usually a better fit.

The decision rule I use: if the data source is naturally streaming or paginated, use a generator. If it is in-memory and finite, use array methods. If you are mixing the two in a hot path, profile.

Iterators, Generators, and Async Generators

The iterator protocol, one method, two return shapes

Generators are sugar over the iterator protocol

yield* delegates to another iterable

Generators receive values too: the two-way protocol

Async generators: same idea, async-aware

A pattern I use weekly: lazy pipeline composition

Where I would not use them

Further reading

`yield*` delegates to another iterable