Why GC Pressure Matters in High‑Throughput Services

When a Go service processes thousands of requests per second, the garbage collector can become a bottleneck. Each allocation adds work for the GC, and frequent allocations of short‑lived objects — like byte slices for request bodies or temporary buffers — cause pause spikes that hurt latency. I’ve seen production services where a 5 ms GC pause turned a 99th‑percentile latency from 30 ms into 120 ms.

The Problem: Re‑allocating Buffers on Every Request

Consider a typical HTTP handler that reads the request body into a []byte, processes it, and then discards the slice. The slice is allocated on the heap, lives for the duration of the request, and then becomes garbage. At 10 k RPS that’s 10 k allocations per second, each triggering a write barrier and eventually a GC cycle.

Enter sync.Pool

sync.Pool is a built‑in, lock‑free cache for temporary objects. It lets you reuse previously allocated memory instead of letting it become garbage. The pool is safe for concurrent use and automatically shrinks during GC, so you don’t leak memory.

Key point: sync.Pool is not a general‑purpose object pool. It’s designed for short‑lived, reusable buffers where the cost of allocation outweighs the cost of a cache miss.

Real‑World Example: A Request‑Body Buffer Pool

Below is a production‑ready snippet I use in a logging middleware that needs a reusable []byte for reading request bodies without allocating on every call.

package middleware

import (
	"io"
	"net/http"
	"sync"
)

// bodyPool holds reusable byte slices of a reasonable size.
// 4 KB covers most JSON payloads while keeping the pool small.
var bodyPool = sync.Pool{
	New: func() interface{} {
		// Allocate a fresh buffer when the pool is empty.
		return make([]byte, 4096)
	},
}

// LoggingMiddleware reads the request body into a pooled buffer,
// logs it, then restores the body for downstream handlers.
func LoggingMiddleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		// Grab a buffer from the pool.
		buf := bodyPool.Get().([]byte)
		// Ensure we return it even if something panics.
		defer bodyPool.Put(buf)

		// Read the body into the buffer.
		n, err := r.Body.Read(buf)
		if err != nil && err != io.EOF {
			http.Error(w, "failed to read body", http.StatusBadRequest)
			return
		}
		body := buf[:n]

		// Log the payload (truncated for safety).
		log.Printf("incoming body: %s", truncate(body, 256))

		// Restore the body so the next handler can read it again.
		r.Body = io.NopCloser(bytes.NewReader(body))

		next.ServeHTTP(w, r)
	})
}

// truncate returns a string limited to maxLen characters.
func truncate(b []byte, maxLen int) string {
	if len(b) <= maxLen {
		return string(b)
	}
	return string(b[:maxLen]) + "…"
}

Why This Works

  • Zero‑allocation hot path: After the warm‑up phase, Get returns a previously used slice, so the handler runs without heap allocation.
  • Automatic cleanup: The GC clears the pool during a collection cycle, preventing unbounded growth.
  • Thread‑safe: sync.Pool uses per‑P (processor) local caches, minimizing contention.

Gotchas and Best Practices

  1. Don’t store pointers to pooled objects after you’ve returned them to the pool. The next Get may reuse the same memory.
  2. Size the initial buffer wisely. Too small → frequent reallocations; too large → wasted memory. Profile your payload distribution.
  3. Reset mutable state. If you reuse a struct with fields, zero them out before Put or in New.
  4. Avoid pooling long‑lived objects. The pool is cleared at each GC, so objects that survive a collection defeat the purpose.

When Not to Use sync.Pool

If your allocation rate is low (e.g., a cron job that runs once a minute) the added complexity isn’t worth it. Also, for objects that contain finalizers or need precise cleanup semantics, a pool can obscure lifecycle bugs.

Measuring the Impact

In a recent service I migrated, the 99th‑percentile latency dropped from 115 ms to 38 ms after introducing a sync.Pool for request‑body buffers. GC pause time fell from an average of 4 ms to under 0.5 ms. The change was a single 30‑line middleware and a one‑line pool declaration.

Wrap‑Up

sync.Pool is a low‑effort, high‑impact tool for reducing allocation pressure in latency‑sensitive Go services. Use it for short‑lived buffers, byte slices, or small structs that appear on the hot path. Profile before and after, keep the pool size reasonable, and you’ll see GC pauses shrink without sacrificing code clarity.