Profiling, SIMD and the shipped instruction set¶
The rule for speed is measure, then write SIMD where a measurement says so. System.Numerics is already intrinsic (see Math), so most code needs nothing. This chapter says how to
look, what the shipped programs are compiled for, and where hand-written vector code exists.
Where to look¶
- The dev overlay (Dev tools): the CPU frame time history, the GPU time of each pass (
FrameLoop.Timings, from timestamp queries, per frame in flight), the render graph and the draw, culled and light counts. Start here: it says whether a frame is CPU- or GPU-bound. - RenderDoc shows the same pass names (
frame.Scope(name)andrecorder.Label(name)are debug labels). - CoreCLR (the editor and
dotnet run):dotnet-trace collect -- <program>anddotnet-counters monitor(install them withdotnet tool install -g dotnet-trace dotnet-counters). - NativeAOT (a shipped build): a native program, so the platform's profiler reads it:
perf record -gon Linux, Instruments on macOS, Windows Performance Analyzer or Visual Studio on Windows../build.sh aotkeeps the symbols (Apogee.Runtime.dbgnext to the program on Linux,.pdbfiles on Windows). - Micro-benchmarks:
./build.sh bench [filter]runs BenchmarkDotNet overSamples/Benchmarks(a sample: benchmarks are run by hand) in Release (the raw results go toBin/benchmarks, which is not committed) (for example./build.sh bench '*Culling*' --job short). BenchmarkDotNet is used there only (MIT, a test dependency, never shipped), and CI builds the project but does not run it: benchmarks are noisy on shared machines. Remember that BenchmarkDotNet's NativeAOT mode compiles for the CPU it runs on, which is not what a game ships with; to measure the shipped baseline publish a small program with-p:PublishAot=trueinstead.
The instruction set of a shipped game¶
The JIT asks the CPU which instructions exist when the program starts. NativeAOT cannot: it compiles for a fixed set, and a CPU without it does not start the program. Measured with a small program on an AVX2 machine:
IlcInstructionSet |
Vector128.IsHardwareAccelerated |
Vector256.IsHardwareAccelerated, Avx2.IsSupported, Fma.IsSupported |
|---|---|---|
| (not set) | true | false |
x86-64-v2 |
true | false |
x86-64-v3 |
true | true |
So the default of x64 is x86-64-v2 (SSE4.2 and POPCNT, practically every PC since 2009), and AVX2 code is simply compiled out of a default NativeAOT build; it is not a run-time check. Our choice, in
Directory.Build.props for PublishAot builds:
- x64:
x86-64-v2, written down so that it is a decision.-p:ApogeeAotInstructionSet=x86-64-v3builds for AVX2, BMI and FMA (Intel since Haswell in 2013, AMD since Excavator in 2015 and Zen) at the price of the older CPUs, which then do not start the game; the next section says what that buys. - osx-arm64:
apple-m1(atomics, CRC, rounding multiply, dot products): every Apple-silicon Mac has at least that. - Other arm64 and anything else keep the compiler's default (ARMv8.0 with NEON).
The CoreCLR editor and development runs use the JIT and so do get AVX2 on a CPU that has it. That is why the code that can use 256-bit vectors checks Vector256.IsHardwareAccelerated and falls back.
Where there is SIMD¶
Only where it was measured to pay, and with a scalar twin that the tests compare it with.
Frustum culling (Apogee.Core.Numerics.FrustumCulling, used by SceneRenderer): the bounds of every draw are kept as six arrays (centers and half extents, structure of arrays, padded to a multiple of eight),
and each frame the five planes of the camera's view volume are tested against all of them, four or eight boxes per instruction, producing a bit per draw. The three kernels (CullScalar, CullVector128,
CullVector256) do the same arithmetic in the same order, so they set the same bits, which FrustumCullingTests checks on random boxes and frusta; Cull picks the widest that is hardware accelerated.
Because the scene's boxes are relative to its origin, the frustum is moved to them with Frustum.Offset (the plane constants absorb the offset) and the arrays never change.
Measured for 100 000 boxes (Core i7-14700K, .NET 10):
| scalar | Vector128 (NativeAOT x86-64-v2, the shipped default) |
Vector256 (NativeAOT x86-64-v3; CoreCLR) |
|
|---|---|---|---|
| time | 1.66 ms | 0.18 ms (9x) | 0.064 ms (26x) |
So a shipped game culls 100 000 draws in under a fifth of a millisecond without AVX2, and AVX2 would save about 0.1 ms more per 100 000: not worth turning players away. For 1 000 boxes the whole cull is about
1.5 µs (128-bit). ./build.sh bench '*Culling*' reproduces the table; *RelativePosition* measures Double3.RelativeTo (100 000 positions: about 0.11 ms, scalar double code, which is fast enough that it
has no SIMD twin).
Not done, on purpose (nothing measured yet, so nothing written): density-map sweeps and the instancer (terrain), skinning, particles. They get a benchmark first.