The problem with eager enumeration

When you call each, map, or select on a large collection, Ruby builds the entire result in memory before the next step runs. That works fine for a few thousand records, but once you start streaming log files, API paginations, or database cursors with millions of rows, the memory footprint explodes and the GC starts thrashing.

Lazy enumerators let you chain transformations without materializing intermediate arrays.

Enter Enumerator::Lazy

Calling lazy on any Enumerable returns a lazy wrapper. Methods like map, select, flat_map, and take become lazy versions that yield one element at a time, pulling from the source only when the pipeline demands it.

# A simple eager pipeline — builds three full arrays
users = User.all
active = users.select(&:active?)
names  = active.map(&:name)
first_five = names.first(5)

Switching to lazy is a one‑line change:

# Lazy pipeline — only five User objects are ever instantiated
first_five = User.all.lazy
  .select(&:active?)
  .map(&:name)
  .first(5)

Real‑world scenario: processing a massive log file

Imagine a nightly job that scans a 2 GB nginx access log, extracts JSON payloads from a specific endpoint, and pushes them to a message queue. Loading the whole file into an array is impossible on a modest CI container.

def stream_payloads(path, endpoint)
  File.foreach(path).lazy               # lazy line enumerator
    .select { |line| line.include?(endpoint) }
    .map    { |line| JSON.parse(line) rescue nil }
    .compact
    .map    { |hash| hash["payload\