The Mystery: Deleting Dead Code Makes Execution 2x Slower

In low-level C++ programming, conventional wisdom suggests that eliminating unused class members should either keep performance identical or slightly improve it by reducing memory footprint. However, developers occasionally encounter a bizarre phenomenon: removing a completely unused variable causes the program to slow down significantly (sometimes by 1.5x to 2x).

Consider this minimal example with a custom circular buffer:

#include <vector>
#include <array>

class CyclicArray {
    static constexpr int L = 1 << 10; // 1024 ints = 4096 bytes

    std::array<int, L> data_;
    int head_, size_; // head_ is never actually used

public:
    CyclicArray() : data_{}, head_{0}, size_{0} {}

    int operator[](int index) const { return this->data_[index % L]; }

    void push_back(const int value) {
        this->data_[this->size_ % L] = value;
        this->size_++;
    }
};

When running a recurrence loop across a std::vector<CyclicArray>, removing the unused member int head_ causes runtime to jump from 1.2 seconds to 2.4 seconds. Why does this happen?

The Core Reason: Object Size and Cache Line Strides

The performance drop is not caused by the variable itself, but by the change in the total byte size (stride) of CyclicArray. Modifying the struct layout changes how adjacent elements in a contiguous std::vector map to CPU cache sets and memory subsystems.

1. Calculating the Object Layout

  • std::array<int, 1024> occupies 1024 * 4 = 4096 bytes (exactly 4 KB, a power of 2).
  • With head_ and size_: The class size is 4096 + 4 + 4 = 4104 bytes (or 4096 + 8).
  • Without head_: The class size drops to 4096 + 4 = 4100 bytes (or 4096 + 4).

Because all instances of CyclicArray are stored consecutively in a std::vector, the address distance between A[i] and A[i+1] changes from 4104 bytes to 4100 bytes.

Hardware Bottlenecks in Action

1. L1/L2 Cache Set Associativity and Cache Aliasing

Modern CPU caches (such as L1 Data Cache, typically 32 KB or 48 KB with 8-way associativity) do not store data at arbitrary locations. Memory addresses are mapped into specific cache sets using the middle bits of the physical address.

When a loop accesses multiple elements at matching offsets within strides close to powers of two (like 4096 bytes):

for (int i = 1; i <= K; i++) {
    A[i].push_back(A[i-1][j] ^ A[i+1][j]);
}

The loop simultaneously reads from A[i-1][j], reads from A[i+1][j], and writes to A[i][size_ % L]. Because j % L maps to identical relative offsets inside data_, the memory addresses accessed across adjacent array elements compete for the exact same cache sets.

With a stride of 4100 bytes, address offsets align in a way that repeatedly causes cache set thrashing (evicting cache lines prematurely). With 4104 bytes, the 8-byte offset distributes cache accesses more evenly across the available cache ways, avoiding pathological conflict misses.

2. 4K Aliasing (False Store-Forwarding Conflicts)

Modern Intel and AMD processors utilize store-forwarding buffers to pass data directly from a recent store to a subsequent load without going through L1 cache. However, the CPU speculates on address matches using bits 0 to 11 (a 4KB page boundary offset).

When loads (A[i-1][j], A[i+1][j]) and stores (A[i][...]) have identical lower 12-bit address offsets, the CPU may incorrectly detect a memory dependency. This causes a 4K aliasing store-forwarding stall, forcing the CPU pipeline to flush and wait around 10–20 cycles per iteration.

How to Diagnose and Fix Cache Stride Issues

1. Diagnose with Performance Profilers

You can verify hardware cache conflicts using Linux perf:

perf stat -e L1-dcache-load-misses,L1-dcache-loads,resource_stalls.sb ./benchmark

When the slowdown occurs, you will see a massive spike in L1 cache misses or store-forwarding stalls.

2. Fix #1: Add Explicit Padding

To avoid power-of-two stride bottlenecks, intentionally pad data structures to offset cache line alignments:

class CyclicArray {
    static constexpr int L = 1 << 10;
    std::array<int, L> data_;
    int size_;
    // Explicit cache padding to prevent conflict misses / 4K aliasing
    char padding_[60]; 
};

3. Fix #2: Transpose Data / Structure of Arrays (SoA)

If you have multiple cyclic buffers accessed synchronously in a grid, separate the array storage from the control variables:

// Instead of Array of Structures (AoS):
// std::vector<CyclicArray> A(K + 2);

// Use Structure of Arrays (SoA) or a flat buffer:
std::vector<int> raw_data((K + 2) * L);
std::vector<int> sizes(K + 2, 0);

Summary

Removing an unused struct member changes the stride between consecutive elements in contiguous memory. When the stride lands near hardware-sensitive boundaries (like multiples of 64-byte cache lines or 4KB page boundaries), it triggers cache associativity conflicts and store-forwarding stalls. Keeping strides misaligned from power-of-two multiples ensures optimal cache utilization.