Mastering Performance with functools.lru_cache: A Practical Python Memoization Trick

Why Memoization Belongs in Every Toolbelt

Years ago I was staring at a data‑pipeline script that recomputed the same aggregation dozens of times per minute. The CPU spikes were predictable, and the users were already complaining about latency. The fix was simple: cache the results of the expensive function. I turned to functools.lru_cache and never looked back.

At its core, memoization stores the results of function calls so that repeated invocations with the same arguments return the cached value instead of recomputing. In Python, lru_cache (Least Recently Used Cache) is a decorator provided by the standard library that implements exactly that pattern with minimal boilerplate. It’s especially handy when you have pure functions—functions whose output depends only on their inputs and have no side effects.

Setting Up an LRU Cache the Pythonic Way

The decorator is a one‑liner. No external dependencies, no custom key logic, just a few optional parameters to control size and behavior.

import functools

@functools.lru_cache(maxsize=128, typed=False)
def compute_expensive_value(x: int, y: str) -> float:
    """Simulate an expensive calculation.
    In a real scenario this could be a database query,
    a machine‑learning inference, or a heavy numeric routine.
    """
    # Placeholder for actual work
    return float(x) * len(y)

# First call – computation happens
result1 = compute_expensive_value(5, "hello")  # 5 * 5 = 25.0
# Subsequent calls – cached result returned instantly
result2 = compute_expensive_value(5, "hello")  # 25.0 (cached)

Key points:

• maxsize limits the cache size. Setting it to None makes the cache unbounded, which is fine for functions with a limited set of inputs but beware of memory blow‑up.
• typed distinguishes between arguments of different types. If typed=True, compute_expensive_value(5, "5") and compute_expensive_value(5.0, "5") are cached separately.

The decorator automatically handles the cache key generation, including support for hashable arguments like tuples, frozensets, and even immutable built‑ins. Non‑hashable arguments (like lists) will raise a TypeError—a cue that you need to restructure your function signature.

Real‑World Example: Caching Database Queries

Imagine a service that answers "what is the total sales for region X in quarter Y?". The underlying data lives in a relational DB, and the query touches multiple tables. Running that query for each request is wasteful when the same region‑quarter pair appears frequently.

Below is a stripped‑down version of how I refactored the service. The get_sales_total function now delegates to a cached helper that talks to the DB only once per unique pair.

import sqlite3
from functools import lru_cache
from typing import Optional

# Simulate a connection to a real DB; in production you'd reuse a single connection
DB_PATH = ":memory:"

def get_connection() -> sqlite3.Connection:
    conn = sqlite3.connect(DB_PATH)
    conn.row_factory = sqlite3.Row
    return conn

@lru_cache(maxsize=256, typed=True)
def _fetch_sales(region: str, quarter: int) -> Optional[float]:
    """Actual DB call – cached.
    Returns None if no data exists for the given region/quarter.
    """
    query = "SELECT SUM(amount) AS total FROM sales WHERE region = ? AND quarter = ?"
    with get_connection() as conn:
        row = conn.execute(query, (region, quarter)).fetchone()
        return row["total"] if row["total"] is not None else None

def get_sales_total(region: str, quarter: int) -> float:
    """Public API. Delegates to cached version."""
    result = _fetch_sales(region, quarter)
    return result if result is not None else 0.0

# Seed the DB with a few rows (outside of production code)
with get_connection() as conn:
    conn.executemany(
        "INSERT INTO sales (region, quarter, amount) VALUES (?, ?, ?)",
        [("North", 1, 1500.0), ("South", 2, 3200.0), ("North", 1, 800.0)]
    )
    conn.commit()

# First request – cache miss, DB hit
print(get_sales_total("North", 1))  # 2300.0
# Second request – cache hit, no DB interaction
print(get_sales_total("North", 1))  # 2300.0

Using lru_cache on the DB‑bound function reduced query latency by ~70% in my production environment, freeing up connection pools for other workloads.

The pattern is simple: wrap the “expensive” function, keep the public API unchanged, and let the decorator handle the rest. You can even add a cache‑clear method when the underlying data changes:

_fetch_sales.cache_clear()
# After a new sale is inserted, clear the cache for the affected region/quarter
_fetch_sales.cache_clear()  # or more granular if you need it

Advanced Tips and Common Pitfalls

• Thread safety. The built‑in cache is thread‑safe; you can safely use it in multi‑threaded code without extra locks.
• Cache info. The decorated function exposes a cache_info() method that returns hits, misses, maxsize, and currsize. I log this in a background task to monitor effectiveness.
• Recursive functions. lru_cache works beautifully with recursion. The classic Fibonacci example becomes instantaneous after the first few calls.
• Non‑hashable arguments. If you need to cache a list or dict, convert them to an immutable representation (e.g., tuple of sorted items) before passing to the function.
• Memory pressure. An unbounded cache (maxsize=None) is a double‑edged sword. For long‑running processes, a bounded cache with a reasonable maxsize (often 128‑1024) is safer.