Path: ...!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!eternal-september.org!.POSTED!not-for-mail
From: anton@mips.complang.tuwien.ac.at (Anton Ertl)
Newsgroups: comp.arch
Subject: An execution time puzzle
Date: Mon, 10 Mar 2025 07:33:18 GMT
Organization: Institut fuer Computersprachen, Technische Universitaet Wien
Lines: 76
Message-ID: <2025Mar10.083318@mips.complang.tuwien.ac.at>
Injection-Date: Mon, 10 Mar 2025 09:10:58 +0100 (CET)
Injection-Info: dont-email.me; posting-host="9f8e121acd1049a94952a1c3822533e3";
	logging-data="1326562"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX19PR4bxrfcdM7NKkSE28k3s"
Cancel-Lock: sha1:0y4o8+GyIT7Ojf+6mzyn58M0TA4=
X-newsreader: xrn 10.11
Bytes: 3397

I have the sequence

     1	add    $0x8,%rbx
     2	sub    $0x8,%r13
     3	mov    %rbx,0x0(%r13)
     4	mov    %rdx,%rbx
     5	mov    (%rbx),%rax
     6	jmp    *%rax
     7	mov    %r8,%r15
     8	add    $0x10,%rbx
     9	mov    0x0(%r13),%rbx
    10	mov    -0x10(%r15),%rax
    11	mov    %r15,%rdx
    12	add    $0x8,%r13
    13	sub    $0x8,%rbx
    14	jmp    *%rax

The contents of the registers and memory are such that the first jmp
continues at the next instruction in the sequence and the second jmp
continues at the top of the sequence.  I measure this sequence with
perf stat on a Zen4, terminating it with Ctrl-C, and get output like:

       21969657501      cycles
       27996663866      instructions  #    1.27  insn per cycle

I.e., about 11 cycles for the whole sequence of 14 instructions.  In
trying to unserstand where these 11 cycles come from, I asked
llvm-mca with

cat xxx.s|llvm-mca-16 -mcpu=znver4 --iterations=1000

and it tells me that it thinks that 1000 iterations take 2342 cycles:

Iterations:        1000
Instructions:      14000
Total Cycles:      2342
Total uOps:        14000

Dispatch Width:    6
uOps Per Cycle:    5.98
IPC:               5.98
Block RThroughput: 2.3

So llvm-mca does not predict the actual performance correctly in this
case and I still have no explanation for the 11 cycles.

Does anybody have an explanation?

The indirect jumps predict very well (0.03% mispredictions), so that's
not the reason.  So the jumps and all instructions that only produce
(intermediate) results consumed by the jumps should not contribute to
the latency: instructions 5,6,10,14.

Instruction 8 produces a dead result (overwritten by instruction 9)
and therefore does not contribute to the latency.  Instruction 4 and
(in the previous iteration) 11 produce results that are only used in
latency-irrelevant instructions.  This leaves us with:

     1	add    $0x8,%rbx
     2	sub    $0x8,%r13
     3	mov    %rbx,0x0(%r13)
     7	mov    %r8,%r15
     9	mov    0x0(%r13),%rbx
    12	add    $0x8,%r13
    13	sub    $0x8,%rbx

One idea is that in this case the hardware alias analysis and 0-cycle
store-to-load forwarding fails for storing and reloading a value
to/from 0(%r13) (instructions 3 and 9), but I would expect a latency
of 6 cycles (1 cycle from instruction 1, 0 from 3, 4 from 9, 1 from
13) from that, not 11.

- anton
-- 
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
  Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>