The Problem: Memory Blow‑ups

Every PHP developer has hit the memory limit when loading a massive CSV or a JSON payload into an array. file('huge.csv') or json_decode(file_get_contents('big.json'), true) pulls the entire file into RAM, and the script crashes once it exceeds memory_limit. I’ve seen production jobs fail silently because a nightly import grew from 50 MB to 300 MB over a few months.

Enter Generators

Since PHP 5.5, generators let you iterate over data without materialising the whole collection. A generator function uses yield instead of return, pausing execution and handing one value at a time to the caller. The runtime keeps only the current value in memory, so you can process gigabytes of data with a few megabytes of RAM.

Key point: Generators implement the Iterator interface automatically, so they work with foreach, iterator_to_array() (if you really need an array), and any library that expects a traversable.

A Real‑World Example: Streaming a CSV

Imagine a nightly job that reads a 2 GB CSV of order records, transforms each row, and writes the result to a database. With a generator we never load more than one line.


/**
 * Yield one associative array per CSV line.
 *
 * @param string $path Path to the CSV file
 * @return Generator>
 */
function csvRows(string $path): Generator
{
    $handle = fopen($path, 'r');
    if ($handle === false) {
        throw new RuntimeException("Cannot open $path");
    }

    // Read header once
    $header = fgetcsv($handle);
    if ($header === false) {
        fclose($handle);
        return; // empty file
    }

    while (($row = fgetcsv($handle)) !== false) {
        // Combine header with values
        yield array_combine($header, $row);
    }

    fclose($handle);
}

// Usage in the import script
foreach (csvRows('/var/data/orders.csv') as $record) {
    // Transform – e.g. normalize dates, cast numbers
    $record['created_at'] = DateTime::createFromFormat('Y-m-d H:i:s', $record['created_at'])->format('U');
    $record['amount'] = (float) $record['amount'];

    // Persist – batch inserts are still possible
    $pdo->prepare('INSERT INTO orders (id, created_at, amount) VALUES (?, ?, ?)')
        ->execute([$record['id'], $record['created_at'], $record['amount']]);
}

The generator opens the file once, reads the header, then yields each row. The foreach loop pulls rows one‑by‑one, so memory stays flat.

Why Generators Win

  • Constant memory footprint – only the current row lives in RAM.
  • Lazy evaluation – processing starts immediately; you don’t wait for the whole file to load.
  • Composable – you can chain generators (filter, map, batch) without intermediate arrays.
  • Native syntax – no extra libraries, just a function with yield.

Gotchas and Best Practices

  1. Don’t rewind. Generators are forward‑only. If you need multiple passes, either re‑run the generator or cache results selectively.
  2. Handle resources carefully. In the example the file handle is closed after the loop ends. If you break early, use a finally block or register a shutdown function to avoid leaks.
  3. Type‑hint the return. Since PHP 7.1 you can declare : Generator (or : Iterator) for static analysis tools.
  4. Batch when writing. Even though reading is streaming, database inserts benefit from transactions or bulk statements. Accumulate a few hundred rows, then execute a single multi‑row INSERT.

Wrapping Up

Generators turned a nightly import that used to OOM at 1 GB into a stable job that runs comfortably under 30 MB. The pattern applies anywhere you stream data: parsing large XML with XMLReader, consuming paginated API responses, or even generating on‑the‑fly reports. Next time you reach for file() or json_decode() on a big payload, ask yourself whether a generator can do the job one piece at a time. Your memory limit will thank you.