When microbenchmarking on modern out-of-order Intel x86 architectures like Skylake (including Skylake-X), theoretical throughput limits often suggest that simple arithmetic loops should achieve ideal 1-cycle-per-iteration performance. However, real-world benchmarks frequently demonstrate results around 1.45 to 1.5 cycles per iteration.

The Problem Breakdown

Consider the following x86-64 loop executed for 1,000,000 iterations:

align 32
.loop:
    add rdi, rdi
    sub rsi, rdi
    and rdx, rdx
    dec rax
    jnz .loop
    ret

In theory, this loop looks prime for 1 cycle/iteration:

  • dec rax and jnz .loop undergo macro-fusion, transforming into a single fused micro-op (µop) executed on execution Port 0 or Port 6.
  • There are 4 unfused ALU µops per iteration (add, sub, and, fused dec/jnz).
  • Intel Skylake features 4 general-purpose ALU execution ports (Ports 0, 1, 5, 6).
  • While sub rsi, rdi depends on add rdi, rdi, out-of-order execution should overlap instructions across iterations seamlessly.

Yet hardware performance counters measuring CPU_CLK_UNHALTED.THREAD_P return roughly 1,450,000 cycles for 1,000,000 iterations (~1.45 c/iter). Why does the CPU fail to hit 1.0 c/iter?

Root Causes for the 1.5 Cycle Bottleneck

1. Macro-Fusion and Frontend Allocation Width

While the execution pipeline on Skylake can execute 4 ALU µops per cycle across ports 0, 1, 5, and 6, the issue often stems from front-end allocation and rename constraints. The Skylake pipeline can allocate/rename up to 4 µops per cycle from the Decoded Stream Buffer (DSB / µop cache) or the Instruction Decode Queue (IDQ).

However, branch handling and macro-fused pairs have subtle quirks in the µop cache and loop stream detector (LSD):

  • A macro-fused branch occupies an allocation slot. In loops containing 4 fused-domain µops, subtle boundary alignments in the µop cache lines (which store sets of up to 6 µops per 32-byte cache window) can cause uneven delivery rates (e.g., alternating between 3 and 4 µops per cycle).
  • If the Loop Stream Detector (LSD) is disabled (which Intel did via microcode updates for many Skylake processors due to errata), delivery relies directly on the DSB, creating potential bubble cycles.

2. Limited Branch Port Distribution (Port 0 and Port 6)

While general ALU operations can execute on ports 0, 1, 5, and 6, conditional branches can only execute on Port 0 or Port 6 (prior to Skylake, only Port 6; on Skylake, both 0 and 6). Similarly, add and sub compete for these same ports alongside ports 1 and 5.

When combined with intra-iteration dependencies (add -> sub), the scheduler faces a tight scheduling window:

  • add rdi, rdi has an unbroken dependency chain across iterations if rdi isn't reset, meaning latency on rdi accumulates if not independently scheduled. Here, rdi accumulates powers of 2.
  • The reservation station (RS) attempts to pick ready instructions, but with dec/jnz vying for ports 0/6 every cycle and sub waiting on add, structural port pressure develops on Ports 0 and 6.

3. The Execution Port Imbalance (The Real Bottleneck)

Let's look closely at port availability:

  • and rdx, rdx: Ports 0, 1, 5, 6
  • dec/jnz: Ports 0, 6
  • add rdi, rdi: Ports 0, 1, 5, 6
  • sub rsi, rdi: Ports 0, 1, 5, 6

Because the macro-fused dec/jnz must go to Port 0 or 6, half of the available execution bandwidth for that specific µop is constrained. If the Out-of-Order scheduler routes add or sub to Port 0 or Port 6 in cycle N, the branch may be forced to wait for cycle N+1, immediately pushing the loop throughput toward an average of ~1.33 to 1.5 cycles per iteration.

How to Verify and Diagnose

You can verify execution port utilization using perf or Intel VTune with micro-architectural event counters:

perf stat -e \\
  cycles,\\
  instructions,\\
  uops_executed.port_0,\\
  uops_executed.port_1,\\
  uops_executed.port_5,\\
  uops_executed.port_6 \\
  ./your_benchmark

If ports 0 and 6 are saturated while ports 1 and 5 exhibit idle bubbles, port contention between the branch and the arithmetic instructions is the primary culprit.

How to Achieve 1.0 Cycle/Iteration

To reach 1.0 cycle per iteration or better, apply loop unrolling:

align 32
.unrolled_loop:
    ; Iteration 1
    add rdi, rdi
    sub rsi, rdi
    and rdx, rdx
    ; Iteration 2
    add rdi, rdi
    sub rsi, rdi
    and rdx, rdx

    sub rax, 2
    jnz .unrolled_loop

Unrolling amortizes the cost of the branch µop across multiple arithmetic operations, avoiding port contention on Ports 0 and 6 and enabling the dynamic scheduler to distribute independent µops evenly across ports 0, 1, 5, and 6.