Why Generators Matter

When a script pulls thousands of rows from a database or reads a massive CSV, loading everything into an array can blow the memory limit and slow the whole request. Generators let you iterate over a data source one item at a time, keeping only the current record in memory. That difference is the line between a smooth background job and a fatal Allowed memory size exhausted error.

The Real‑World Problem

Imagine a nightly import that processes a 200 MB CSV of product updates. The naïve approach reads the file with fgetcsv into a giant array, then loops over it to update the catalog. On a modest VPS the script crashes halfway through. The business needs the import to finish reliably without provisioning a larger instance.

Generator‑Based Solution

Wrap the parsing logic in a generator function. Each yield hands a single row back to the caller, so the script never holds more than one line in memory.

function csvRows(string $path): Generator
{
    $handle = fopen($path, 'r');
    if ($handle === false) {
        throw new RuntimeException("Cannot open $path");
    }

    // Skip header if present
    fgetcsv($handle);

    while (($row = fgetcsv($handle)) !== false) {
        // Normalise columns – example: id, sku, price, stock
        yield [
            'id'    => (int)$row[0],
            'sku'   => $row[1],
            'price' => (float)$row[2],
            'stock' => (int)$row[3],
        ];
    }

    fclose($handle);
}

// Usage in the import script
foreach (csvRows('/var/import/products.csv') as $product) {
    // Process one product – e.g. upsert into DB
    $repository->upsert($product);
    // Optional: flush entity manager every 500 rows to keep DB memory low
    if ($repository->getCount() % 500 === 0) {
        $entityManager->flush();
        $entityManager->clear();
    }
}
$entityManager->flush(); // final flush

How It Works

The csvRows function returns a Generator object. When the foreach starts, PHP executes the function until the first yield, then pauses. Each subsequent iteration resumes the function right after that yield, producing the next row. The file handle stays open the whole time, but only one line lives in PHP’s memory at any moment.

Key point: Generators implement Iterator, so they work with any foreach, iterator_to_array (if you really need an array), or libraries that accept traversables.

Best Practices & Gotchas

  • Close resources explicitly. The example calls fclose after the loop ends. If the loop breaks early (e.g., an exception), wrap the iteration in a try/finally or use a Generator that implements __destruct to guarantee cleanup.
  • Avoid iterator_to_array on huge streams. That defeats the purpose by materialising the whole dataset.
  • Combine with batch flushing. As shown, periodically flushing the ORM keeps the database side memory‑friendly too.
  • Type‑hint the return as Generator. It gives static analysis tools a clear contract and enables IDE autocompletion.
  • Yield by reference only when necessary. yield &$value can be useful for modifying the original data, but it adds complexity and can surprise callers.

Wrapping Up

Generators are a small language feature with a big impact on resource‑heavy tasks. By turning a “load‑everything‑then‑process” pipeline into a lazy stream, you keep memory usage flat, improve start‑up latency, and make the code easier to reason about. Next time you face a bulky import, export, or API pagination, reach for a generator first — your future self (and the ops team) will thank you.