Why Generators Matter in PHP

When I first started working with PHP, I tended to load entire files or query results into arrays before processing them. It felt natural—grab everything, loop, and you’re done. The problem appears when the data set grows beyond a few megabytes. Memory spikes, execution slows, and sometimes the script hits the memory limit and dies. That’s where generators become a quiet hero. A generator lets you produce values one at a time, keeping only the current item in memory. The rest of the data stays on disk or in the database until you actually need it.

Real‑world scenario: importing a massive CSV

Imagine you receive a nightly feed from a partner that contains product updates. The CSV file is 500 MB, with over two million rows. Each row needs validation, a lookup against the product table, and possibly an insert or update. Loading the whole file into an array would easily exceed the default 128 MB PHP memory limit, causing the import to fail. Instead of splitting the file manually or raising the limit (which only postpones the problem), we can stream the file line by line using a generator.

The generator‑based solution

Below is a production‑ready snippet that opens a CSV, yields each row as an associative array, and lets the caller decide what to do with that row. Notice the use of SplFileObject for robust file handling and the yield keyword that turns the function into a generator.


setFlags(SplFileObject::READ_CSV);
    $file->setCsvControl($delimiter, $enclosure);

    // Grab the header row first. If the file is empty we simply return.
    if ($file->eof()) {
        return; // generator ends immediately
    }
    $header = $file->fgetcsv();
    if ($header === false) {
        throw new RuntimeException('Unable to read header from CSV file.');
    }

    // Now iterate over the remaining rows.
    while (!$file->eof()) {
        $row = $file->fgetcsv();
        if ($row === false) {
            // In case of a malformed line we skip it but continue.
            continue;
        }
        // Combine header with data to produce an associative array.
        yield array_combine($header, $row);
    }
}

// Example usage: process each row without ever loading the whole file.
foreach (csvRows('/var/imports/products.csv') as $product) {
    // Validate required fields.
    if (empty($product['sku']) || empty($product['price'])) {
        error_log('Skipping invalid product: ' . json_encode($product));
        continue;
    }

    // Pretend $db is a PDO instance.
    global $db;
    $stmt = $db->prepare(
        'INSERT INTO products (sku, name, price, updated_at) '
        . 'VALUES (:sku, :name, :price, NOW()) '
        . 'ON DUPLICATE KEY UPDATE name = VALUES(name), price = VALUES(price), updated_at = NOW()'
    );
    $stmt->execute([
        ':sku'  => $product['sku'],
        ':name' => $product['name'] ?? '',
        ':price'=> (float)$product['price'],
    ]);
}
?>




    

        
        
        
    

    
What’s happening under the hood


When PHP encounters yield, it does not execute the rest of the function immediately. Instead, it returns an object that implements the Iterator interface. Each time the caller advances the iterator (via foreach or manual next() calls), PHP resumes the generator exactly after the last yielded value, runs until it hits another yield or the end, and then pauses again. This means only the current row lives in memory; the file handle stays open, but we never buffer more than a single line.



Because we use SplFileObject with the READ_CSV flag, PHP handles quoted fields, escaped delimiters, and varying line endings for us. The generator abstraction lets the consumer focus on business logic—validation, database work, or whatever else—while worrying less about I/O details.



When to reach for a generator


    
    
  • Reading large files (logs, CSV, XML) line by line.
    • 
    
  • Processing results from a database cursor where you don’t want to fetch all rows at once.
    • 
    
  • Creating infinite sequences, like generating IDs or timestamps on demand.
    • 
    
  • Chaining operations: you can pipe one generator into another, forming a lazy processing pipeline.
    • 




A few gotchas


    
    
  • Generators are forward‑only; you cannot rewind or seek to a previous value without recreating the generator.
    • 
    
  • If you throw an exception inside the generator, it propagates to the caller at the point of iteration—handle it accordingly.
    • 
    
  • Because the file stays open for the duration of the iteration, avoid long‑running tasks inside the loop that could block other processes; consider off‑loading heavy work to a queue if needed.
    • 




In my day‑to‑day work, this pattern has saved countless late‑night debugging sessions caused by memory exhaustion. It’s simple to drop into existing code, requires no external libraries, and scales gracefully as the data size grows. Give it a try the next time you face a bulky import or export—you’ll likely wonder why you ever loaded everything into an array at once.