Dart Extension Method for Chunking Iterables: Batch Processing Made Simple
The problem
When you pull a large list of items from a REST endpoint — think thousands of product IDs — you often need to send them downstream in smaller batches. The API you call might only accept 100 IDs per request, or you want to throttle database writes. Writing a manual loop each time clutters the business logic and invites off‑by‑one bugs.
The solution: an extension method
Dart’s extension methods let you add functionality to existing types without subclassing. By extending Iterable<T> we get a reusable chunked method that works on any collection — lists, sets, or even lazy streams converted to iterables.
Implementation
extension IterableChunked<T> on Iterable<T> {
/// Splits the iterable into chunks of [size].
/// The last chunk may contain fewer elements.
/// Throws [ArgumentError] if [size] <= 0.
Iterable<List<T>> chunked(int size) sync* {
if (size <= 0) {
throw ArgumentError('Chunk size must be positive');
}
final buffer = <T>[];
for (final element in this) {
buffer.add(element);
if (buffer.length == size) {
yield List.unmodifiable(buffer);
buffer.clear();
}
}
if (buffer.isNotEmpty) {
yield List.unmodifiable(buffer);
}
}
}
The method is a synchronous generator (sync*) so it streams chunks lazily — no intermediate list of all chunks is allocated. List.unmodifiable protects callers from accidentally mutating the internal buffer.
Usage example
Future<void> uploadProductImages(List<String> imageUrls) async {
const batchSize = 50;
for (final batch in imageUrls.chunked(batchSize)) {
// Each batch is a List<String> of at most 50 URLs.
await api.uploadImages(batch);
// Optional: small delay to respect rate limits.
await Future.delayed(const Duration(milliseconds: 200));
}
}
Notice how the calling code reads like plain English: "for each batch in imageUrls chunked by 50". No index arithmetic, no manual slicing.
Why it works
- Zero‑cost abstraction: the extension compiles to a static method; the generator yields one chunk at a time.
- Type safety: the generic
<T>preserves the element type, so the returnedList<T>matches the source. - Composability: you can chain
.where(...).chunked(100)or.map(...).chunked(10)without extra boilerplate.
If you already use
package:collection, itsIterableExtension.chunkeddoes the same thing. The snippet above shows how to roll your own when you want to avoid an extra dependency or need a tiny tweak (e.g., custom error handling).
Caveats
Because the method returns an Iterable<List<T>>, it is lazy. If you need random access or multiple passes, materialise it first: final batches = imageUrls.chunked(50).toList();. Also, the extension lives in the file where it’s declared — import that file wherever you need chunking.
Final thoughts
Small, focused extensions like chunked keep domain code clean and push repetitive plumbing into a tested, reusable spot. I’ve dropped this into three projects this quarter and it has eliminated a whole class of “how many items left?” bugs. Give it a try next time you face a batching requirement — you’ll wonder how you lived without it.