Lazy Pipelines

Array.map eagerly creates a result array. A generator transformation can instead yield one transformed value for each requested input. Compose small synchronous generators to filter and map a source without materializing intermediate arrays.

Laziness changes when side effects happen and when errors become visible. A pipeline can be constructed successfully but fail only when consumed. A consumer that asks for one result should not cause the entire source to be processed. This is useful for large finite data sources and bounded workers.

Do not confuse laziness with low total memory in every API. Spreading the entire pipeline into an array collects everything. An infinite source still cannot be fully collected. Likewise, the final task pool bounds active mapper calls but retains all output for ordered return; it is not a streaming-output API.

Use explicit loops inside generator helpers while learning. The yield sites show exactly which work happens for each request, and generator cleanup composes with early loop exit.

Predict, run, and explain

Write the output and trace the important state changes before checking it. Then run the program. Each editor is independent. Async examples end with main() so this editor waits for their work.

function* doubled(values) {
  for (const value of values) {
    console.log(`read ${value}`);
    yield value * 2;
  }
}

const pipeline = doubled([1, 2, 3]);

console.log("created");
console.log(pipeline.next().value);
pipeline.return();

created
read 1
2

Observe work that a consumer never requested

A lazy transform performs work only as downstream demand advances it. This can avoid processing unused items, but it also changes when effects and errors occur. A function returning an iterator has not necessarily completed its calculations when it returns.

Composition order matters. Filtering before transforming can reduce mapper calls; transforming before filtering may be required if the predicate depends on the transformed value. Preserve the intended values and order before claiming an optimization.

let calls = 0;

function* doubled(values) {
  for (const value of values) {
    calls++;
    yield value * 2;
  }
}

const cursor = doubled([1, 2, 3]);

console.log(calls);
console.log(cursor.next().value);
console.log(calls);
cursor.return();

0
2
1

Only one transformation ran. The unused inputs never reached the multiplication. This is useful evidence of laziness, beyond merely returning an object with next.

const values = [1, 2, 3, 4];
let first = 0;
let second = 0;
const a = values
  .filter(n => n > 2)
  .map(n => {
    first++;
    return n * 2;
  });
const b = values
  .map(n => {
    second++;
    return n * 2;
  })
  .filter(n => n > 4);

console.log(a.join(","), b.join(","));
console.log(first, second);

6,8 6,8
2 4

These predicates are deliberately equivalent for this transformation. Equal outputs do not mean equal work. For a different predicate or effectful mapper, reordering may change the contract.

Try the rule on new inputs

Count mapper calls for zero, one and all requested outputs. Add a failure on a later unused input and explain when it should become observable. Do not confuse lazy input consumption with constant result memory.

Your turn

// Yield matching values; collect only at the final display boundary.

function* positive(values) {
  for (const value of values)
    if (value > 0) {
      yield value;
    }
}

console.log([...positive([-1, 0, 2, 3])].join(","));

2,3

Change it and test it

Add a transformation that squares each value, then compose it after positive. Log input reads and stop after one yielded result. Which values were actually examined? Explain how this differs from mapping and filtering an entire array first.