Instruction scheduling model for the KNL

[hfinkel]

Bugzilla Link	26418
Version	trunk
OS	All
Blocks	#31672
CC	@legrosbuffle,@topperc,@davezarzycki,@Delena,@gchatelet,@igor-breger,@RKSimon,@MattPD,@naromero77,@phoebewang,@vchuravy

Extended Description

We need to have a real instruction scheduling model for the KNL. Aside from anything else, it seems reasonable to switch the default to be more like Silvermont than like Haswell, plus there is a significant amount of information here (under the "MICROARCHITECTURE" heading):

https://software.intel.com/en-us/articles/what-disclosures-has-intel-made-about-knights-landing

plus from Avinash's HotChips presentation:

http://www.hotchips.org/wp-content/uploads/hc_archives/hc27/HC27.25-Tuesday-Epub/HC27.25.70-Processors-Epub/HC27.25.710-Knights-Landing-Sodani-Intel.pdf

[hfinkel]

There's now a significant amount of published information:

Knights Landing: Second-Generation Intel Xeon Phi Product
Avinash Sodani, et al. IEEE Micro. Vol. 36 (2). 2016

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7453080

[hfinkel]

There's now a significant amount of published information:

Knights Landing: Second-Generation Intel Xeon Phi Product
Avinash Sodani, et al. IEEE Micro. Vol. 36 (2). 2016

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7453080

Here are quotes from this article (and some of my commentary) relevant to instruction scheduling and some related issues...

"Front-end unit. The core’s FEU comprises a 32-Kbyte instruction cache (IL1) ... In case of a hit, the instruction cache can deliver up to 16 bytes per cycle. These bytes are then sent to a two-wide decoder."

It is important to note in this context that AVX-512 instructions can be up to 12 bytes long (if you have a memory operand with a non-compressed displacement). We'll need to take care not to generate displacements that require 4 bytes, and, I presume, otherwise try not to schedule long instructions next to each other.

"Most instructions are decoded into a single micro-op, ... Decoded micro-ops are placed into a 32-entry instruction queue."

"The allocation unit reads two micro-ops per cycle from the instruction
queue. It assigns the necessary pipeline resources required by the micro-ops, such as reorder buffer (ROB) entries (72), rename buffer entries (72), store data buffers (16), gather-scatter table entries (4), and reservation station entries."

"After the allocation unit, the micro-ops are sent to one of three execution
units—IEU, MEU, or VPU—depending on their opcode types. Some micro-ops
could get sent to more than one execution unit. For example, an Add instruction that has a memory source will be sent to the MEU to read the memory source, and then to the IEU to execute the Add operation."

"Integer execution unit. The IEU executes integer micro-ops ... There are two IEUs in the core. Each IEU contains one 12-entry reservation station that issues one micro-op per cycle... Most operations have one-cycle latency and are supported by both IEUs. But a few have three- or five-cycle latency (for example, "multiplies") and are only supported by one of the IEUs."

"Memory execution unit. Up to two memory operations, either load or store, can be executed in the MEU in a given cycle. Memory operations are issued ... from the 12-entry memory reservation station. While stores are kept in the store buffer after address translation, they can forward data to dependent loads. Stores are committed to memory in the program order, one per cycle."

"L1 data cache (DL1) supports two simultaneous 512-bit reads and one 512-
bit write, with a load-to-use latency of four cycles for integer loads and five cycles for floating-point loads."

"The MEU supports unaligned memory accesses without any penalties and supports accesses that split into two cache lines with a two-cycle penalty"

"Vector processing unit... provides support for x87, MMX, Streaming
SIMD Extensions (SSE), AVX, and AVX-512 instructions, as well as integer divides. Two VPUs are connected to the core... the allocation unit dispatching instructions directly into the VPUs... The VPUs are mostly symmetrical, and each can provide a steadystate throughput of one AVX-512 instruction per cycle,... One of the VPUs is extended to provide support for the legacy floating-point instructions, such as x87, MMX, and a subset of byte and word SSE instructions."

"Each VPU contains a 20-entry floating-point reservation station that issues one
micro-op per cycle out of order. The floating-point reservation station differs from the IEU and MEU reservation stations in that it does not hold source data ...; the floating-point micro-ops read their source data from the floating-point rename buffer and the floating-point register file after they issue from the floating-point reservation station, spending an extra cycle between the reservation station and execution compared to integer and memory micro-ops. Most floating-point arithmetic operations have a latency of six cycles, whereas the rest of the operations have a latency of two or three cycles, depending on the operation type."

"On a KNL tile, the two cores share a... L2 cache. The BIU also contains an L2 hardware prefetcher that is trained on requests coming from the cores. It supports up to 48 independent prefetch streams... A KNL core supports up to four hardware contexts"

This fact is relevant for cost modeling for higher-level loop transformations (e.g. loop fusion and fission). We want to keep relevant streaming operations below the hardware limit of (48 / 2 (cores per tile) / 4 (threads per core) == 6 per thread).

[hfinkel]

The KNL is now covered in Intel's Architecture Optimization Manual (http://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-optimization-manual.html), specifically chapter 16.

[RKSimon]

Does anyone have access to KNL hardware that they could run llvm-exegesis uops/latency tests on?

This will require some hacking as KNL still uses the Haswell model at the moment - maybe moving to the SLM model for the tests, along with the necessary perf counter mappings (see https://sourceforge.net/p/perfmon2/libpfm4/ci/master/tree/lib/events/intel_knl_events.h).

[RKSimon]

Does anyone have access to KNL hardware that they could run llvm-exegesis
uops/latency tests on?

This will require some hacking as KNL still uses the Haswell model at the
moment - maybe moving to the SLM model for the tests, along with the
necessary perf counter mappings (see
https://sourceforge.net/p/perfmon2/libpfm4/ci/master/tree/lib/events/
intel_knl_events.h).

The perf counter mappings are now in place, so we now need someone with KNL hardware to run llvm-exegesis (with libpfm4) to get the raw schedule data.

[llvmbot]

Does anyone have access to KNL hardware that they could run llvm-exegesis
uops/latency tests on?

This will require some hacking as KNL still uses the Haswell model at the
moment - maybe moving to the SLM model for the tests, along with the
necessary perf counter mappings (see
https://sourceforge.net/p/perfmon2/libpfm4/ci/master/tree/lib/events/
intel_knl_events.h).

Unfortunately, the compiler explorer link I posted in the other bug was set to use KNL because I was also testing gcc5.5 in the same window at some point it it was the only "arch" options that 5.5 supported that had AVX-512 (and I wanted to see vectorization with AVX512 instructions because I have gcc5.5 on a skylake-avx512 platform). Sorry

[RKSimon]

@​davezarzycki You mentioned on D89952 that you have access to a KNL box, would you be in a position to get llvm-exegesis running on it to see whether we'll be able to create a true scheduler model instead of just using the haswell model?

I'm not sure if libPFM has access to suitable pipe counters but uops/latency reports might already work.

[davezarzycki]

In theory, sure. How much work does this involve? Is there a command or script I can run from a recent LLVM build to generate all of the required data?

[legrosbuffle]

Instructions are here: https://llvm.org/docs/CommandGuide/llvm-exegesis.html

the tl;dr is that you can generate the data with:

llvm-exegesis -mode=latency -opcode-index=-1
llvm-exegesis -mode=uops -opcode-index=-1

Make sure to sync past 7e2ffe7, as there was an issue with recent instructions.

Then, creating a full model from scratch is a significant piece of work. You can make this easier by starting with an existing model and using llvm-exegesis analysis mode to fix this detected issues.

[davezarzycki]

Will the fact that LLVM considers KNL/KNM to be Haswell derived create any problems when using llvm-exegesis? I ask because KNL/KNM is actually Silvermont derived.

[legrosbuffle]

No, you should be good.
Note that you might want to fork the new model from SLM though, to minimize diffs.

[RKSimon]

You will need to build/install a recent copy of perfmon:

https://sourceforge.net/p/perfmon2/libpfm4/ci/master/tree/

And then rebuild llvm-exegesis from scratch to ensure cmake sees thats its installed.

[davezarzycki]

Is libpfm-4.10.1 not new enough? The uops pass of llvm-exegesis is failing completely (with kernel 5.8.16 if it matters):

invalid event attribute - cannot create event uops_retired:any
llvm-exegesis error: Unable to create counter with name 'uops_retired:any'

[davezarzycki]

And the latency pass is weirdly labeling KNL as "tremont". For example:

---

mode: latency
key:
instructions:
- 'VCMPSSZrrb_Int K3 XMM0 XMM13 i_0x1'
- 'VPMOVUSQWZrrkz XMM0 K3 ZMM5'
config: ''
register_initial_values:
- 'XMM0=0x0'
- 'XMM13=0x0'
- 'MXCSR=0x0'
- 'ZMM5=0x0'
cpu_name: tremont
llvm_triple: x86_64-unknown-linux-gnu
num_repetitions: 10000
measurements:

• { key: latency, value: 5.0421, per_snippet_value: 10.0842 }
  error: ''
  info: Repeating two instructions
  assembled_snippet: 4883EC10C7042400000000C744240400000000C744240800000000C744240C00000000F30F6F04244883C4104883EC10C7042400000000C744240400000000C744240800000000C744240C00000000F3440F6F2C244883C4104883EC04C70424801F00000FAE14244883C40462D17E18C2DD0162F27ECB14E862D17E18C2DD0162F27ECB14E862D17E18C2DD0162F27ECB14E862D17E18C2DD0162F27ECB14E862D17E18C2DD0162F27ECB14E862D17E18C2DD0162F27ECB14E862D17E18C2DD0162F27ECB14E862D17E18C2DD0162F27ECB14E8C3
  ...
  Check generated assembly with: /usr/bin/objdump -d /tmp/snippet-bd8c0e.o

---

[legrosbuffle]

Looks like a bad pfm event mapping. libpfm, I see "uops_retired:all", but llvm/lib/Target/X86/X86PfmCounters.td has " "uops_retired:any" for slm, so you might want to try to fix that (I don't have access to a KNL to test it)