@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/bank-conflict.md

@@ -1,217 +0,0 @@
----
-title: Bank Conflict Avoidance
-status: draft
-evidence_level: measured
-applies_to_pattern_class:
-- cuda-core
-applies_to_ops:
-- reduction
-- scan
-- transpose
-- stencil
-- normalization
-- elementwise
-requires_sm: '>=5.0'
-requires_features: []
-single_kernel_useful: true
-cuda_version_tested: '12.9'
-driver_version_tested: 570.124.06
-toolchain: nvcc 12.9 + ptxas 12.9
-measured_on: H200-SXM
-source:
-- path: spec
-  anchor: Reference
-artifacts:
-  code: artifacts/experience/hw-probes/smem-bank-conflict/smem_bank_conflict_probe.cu
-  build: nvcc -arch=sm_90a -O3 -std=c++17 -lineinfo -o smem_bank_conflict_probe smem_bank_conflict_probe.cu
-  introspection: ''
-  profile: ''
-related_apis: []
-related_skills:
-- coalescing
-- vectorized-access
-- warp-primitives
-id: skill-bank-conflict
-type: skill
-vendor: nvidia
-tags:
-- cuda-cpp
-- tma
-- vectorized-loads
-- shared-memory-optimization
-- swizzling
-- tma-multicast
-- fused-kernel
-- gemm
-applies_to:
-- general
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L1450-L1482
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L1452-L1454
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L1604-L1655
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L720-L726
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L892-L900
-architectures:
-- sm90
-- sm90a
-languages:
-- cuda-cpp
-hardware_features:
-- tma
-techniques:
-- vectorized-loads
-- shared-memory-optimization
-- swizzling
-- tma-multicast
-kernel_types:
-- fused-kernel
-- gemm
-confidence: experimental
-artifact_dir: artifacts/experience/hw-probes/smem-bank-conflict
----
-## What
-
-Bank conflict avoidance is a shared-memory layout technique that ensures threads within a warp access different memory banks when loading from or storing to `__shared__` memory.
-
-Shared memory on all CUDA devices with compute capability >= 5.0 (including H200, sm_90a) is organized into **32 banks**, each 32 bits (4 bytes) wide. Successive 32-bit words are assigned to successive banks in round-robin fashion:
-
-```
-Address (bytes):   0    4    8   12   ...  124  128  132  ...
-Bank:              0    1    2    3   ...   31    0    1   ...
-```
-
-When a warp issues a shared-memory load or store:
-
-- **No conflict**: if all 32 threads access 32 distinct banks, the entire request is serviced in a single transaction.
-- **N-way conflict**: if N threads access **different addresses** that map to the **same bank**, the hardware serializes the access into multiple rounds. This reduces effective bandwidth.
-- **Broadcast**: if multiple threads read the **same address** in the same bank, the word is multicast to all requesting threads with no penalty.
-
-The bank index for a 32-bit word at byte address `A` is:
-
-```
-bank(A) = (A / 4) % 32
-```
-
-## Why
-
-Bank conflicts directly multiply the latency of shared-memory accesses.  On H200, a conflict-free dependent `ld.shared` takes approximately **30 cycles** (measured).  A 32-way bank conflict -- where all 32 threads hit the same bank with different addresses -- raises this to **91 cycles** (3.03x baseline), representing significant serialization overhead.
-
-In bandwidth-sensitive kernels that rely on shared memory as a scratchpad (matrix transpose, tiled GEMM, reduction, stencil), bank conflicts can eliminate most of the bandwidth advantage of shared memory over global memory. The CUDA Best Practices Guide reports that removing 32-way bank conflicts from a matrix transpose kernel improved effective bandwidth from 140.2 GB/s to 199.4 GB/s on Tesla V100 (~42% improvement).
-
-## When to use
-
-Apply bank conflict avoidance whenever a kernel uses `__shared__` memory and the access pattern has a stride that is a multiple of 32 elements (128 bytes) per warp.  Common scenarios:
-
-- **Matrix transpose**: storing to `smem[threadIdx.x][threadIdx.y]` when `threadIdx.x` is the fast-moving dimension creates stride-32 access patterns that cause 32-way bank conflicts.
-
-- **Tiled matrix multiplication (C=AAT)**: copying a column of A into a column of shared memory while using row-major layout creates stride-32 writes.
-
-- **Reduction to shared memory**: writing partial sums to `smem[warpId]` when 32 warps are active causes all warps' lane-0 threads to hit bank 0.
-
-- **Column-wise access of 2D shared arrays**: any `smem[row][col]` access where consecutive threads vary `row` (with a fixed `col`) produces a stride equal to the row width, which is a 32-way conflict when the width is a multiple of 32.
-
-## When NOT to use
-
-- **No shared memory**: if the kernel does not use `__shared__` memory, bank conflicts do not apply.
-
-- **Stride-1 access**: if consecutive threads read consecutive 32-bit words (`smem[threadIdx.x]` style), the access is already conflict-free.
-
-- **Broadcast pattern**: if all threads read the same shared-memory address, the hardware broadcasts the value with no penalty.
-
-- **Global-memory-bound kernels**: if the kernel is bottlenecked by global memory bandwidth and shared memory is used lightly, optimizing bank conflicts may not yield visible speedup.  Profile first with `l1tex__data_bank_conflicts_pipe_lsu_mem_shared_op_ld.sum` via NCU to confirm conflicts are significant before applying padding.
-
-## Avoidance techniques
-
-### 1. Padding (+1 trick)
-
-The simplest and most common fix.  Add one element of padding to the inner dimension of a 2D shared array:
-
-```cuda
-// BEFORE: 32-way bank conflict when accessing column-wise
-__shared__ float tile[32][32];
-
-// AFTER: conflict-free -- stride becomes 33, and 33 % 32 = 1
-__shared__ float tile[32][33];
-```
-
-This works because padding shifts each row by one bank, so column-wise access (stride = row width) no longer maps all threads to the same bank. The extra element per row wastes 4 bytes x 32 rows = 128 bytes of shared memory, which is negligible.
-
-### 2. Index swizzling
-
-For more complex layouts (e.g., matrix tiles loaded via TMA or cp.async), swizzling the shared-memory index remaps addresses so that column-wise and row-wise accesses both avoid conflicts without wasting memory.  The programming guide documents `CU_TENSOR_MAP_SWIZZLE_128B` for this purpose (L12822-L12838).
-
-### 3. Access pattern redesign
-
-Sometimes the access pattern itself can be changed.  For example, instead of writing to `smem[threadIdx.x][threadIdx.y]` (stride-32 in the first index), transpose the indices: `smem[threadIdx.y][threadIdx.x]` (stride-1 in the second index).  This is shown in the programming guide matrix transpose example (L1624-L1639).
-
-## Classical example: padded matrix transpose tile
-
-```cuda
-#define TILE 32
-
-__global__ void transpose(const float* __restrict__ in,
-                          float*       __restrict__ out,
-                          int rows, int cols) {
-    // +1 padding eliminates bank conflicts on column-wise read
-    __shared__ float tile[TILE][TILE + 1];
-
-    int x = blockIdx.x * TILE + threadIdx.x;
-    int y = blockIdx.y * TILE + threadIdx.y;
-
-    // Coalesced load from global → conflict-free write to smem row
-    if (x < cols && y < rows)
-        tile[threadIdx.y][threadIdx.x] = in[y * cols + x];
-    __syncthreads();
-
-    // Transposed indices for output
-    x = blockIdx.y * TILE + threadIdx.x;
-    y = blockIdx.x * TILE + threadIdx.y;
-
-    // Column-wise read from smem (conflict-free due to padding)
-    // → coalesced write to global
-    if (x < rows && y < cols)
-        out[y * rows + x] = tile[threadIdx.x][threadIdx.y];
-}
-```
-
-Without the `+1` padding, the `tile[threadIdx.x][threadIdx.y]` read in the second phase would cause a 32-way bank conflict, because consecutive threads (varying `threadIdx.x`) would access addresses with stride 32.
-
-## Measured Characteristics
-
-- smem-bank-conflict stride sweep probe: On H200 (sm_90a, CUDA 12.9), dependent pointer-chasing in shared memory measured **30.01 cycles per load** conflict-free (stride-1), **91.00 cycles** under 32-way bank conflict (stride-32, 3.03x baseline), and **30.01 cycles** after applying the +1 padding trick (stride-33, fully restored).  Broadcast (all threads reading the same address) also measured 30.01 cycles, confirming zero-penalty hardware multicast.  2-way conflicts (stride-2) showed negligible overhead (31.01 cycles, 1.03x).
-
-## Health thresholds
-
-The conflict-free case is defined, not measured: a shared-memory request that
-hits 32 distinct banks is serviced in **one wavefront**. Every additional
-wavefront for the same request is one extra serialised pass.
-
-```
-conflict factor = l1tex__data_pipe_lsu_wavefronts_mem_shared_op_ld.sum
-                / smsp__inst_executed_op_shared_ld.sum
-```
-
-| Factor | Reads as |
-|---|---|
-| 1.0 | conflict-free |
-| 2-4 | mild — often a stride that hits every 2nd or 4th bank |
-| up to 32 | worst case: all 32 lanes land in one bank, fully serialised |
-
-⚠️ **The wavefront count alone is not evidence of a conflict.** A kernel with
-heavy shared-memory traffic has a large numerator simply because it issues many
-requests. Without dividing by the request count you are reading traffic, not
-conflict.
-
-A second ratio, `wavefronts / l1tex__data_pipe_lsu_cycles_active_mem_shared_op_ld.sum`,
-gives wavefronts completed per active cycle; a low value points at the same
-serialisation from the throughput side.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/cache-load-hints/apis.md

@@ -1,201 +0,0 @@
----
-title: Cache Load Hints APIs
-status: draft
-source:
-- path: spec
-  anchor: Reference
-apis:
-- func_name: __ldg
-  namespace: cuda-runtime
-  kind: load-intrinsic-readonly
-  signature: T __ldg(const T* ptr);
-  notes: 'Non-coherent read-only cache load (PTX ld.global.nc). On sm_70+ the compiler auto-emits this for `const __restrict__` pointers; explicit __ldg is redundant on H200. Measured on sm_9.0a: 0.1-0.4% difference vs default (within noise).'
-  ten_api_raw: wiki/nvidia/api-definitions/runtime/__ldg.md
-- func_name: __ldca
-  namespace: cuda-runtime
-  kind: load-intrinsic-cache-all
-  signature: T __ldca(const T* ptr);
-  notes: Cache-all load (PTX ld.global.ca, SASS LDG.E.STRONG.SM on H200). Distinct from the compiler default on `const __restrict__` pointers (which lower to ld.global.nc / LDG.E.CONSTANT); __ldca forces the L1 + L2 cache-all path. Use for any load where data is reused within the SM but the pointer is not const-qualified (otherwise default does the same thing via the .nc path).
-  ten_api_raw: wiki/nvidia/api-definitions/runtime/__ldca.md
-- func_name: __ldcg
-  namespace: cuda-runtime
-  kind: load-intrinsic-cache-global
-  signature: T __ldcg(const T* ptr);
-  notes: 'Cache-global load (PTX ld.global.cg). Caches in L2 only, bypasses L1. Measured on H200: 2.26x slower than default at L2-resident 16-pass regime because L1 staging is bypassed. Use ONLY when data is not reused within the SM.'
-  ten_api_raw: wiki/nvidia/api-definitions/runtime/__ldcg.md
-- func_name: __ldcs
-  namespace: cuda-runtime
-  kind: load-intrinsic-cache-streaming
-  signature: T __ldcs(const T* ptr);
-  notes: 'Cache-streaming load (PTX ld.global.cs). L1 + L2 but tagged evict-first. Measured on H200: identical to default in un-contended L2 (8 MiB buffer in 60 MiB L2); the evict-first tag only matters when another workload is fighting for L2. Legacy P4.'
-  ten_api_raw: wiki/nvidia/api-definitions/runtime/__ldcs.md
-- func_name: __ldlu
-  namespace: cuda-runtime
-  kind: load-intrinsic-last-use
-  signature: T __ldlu(const T* ptr);
-  notes: Last-use load (PTX ld.global.lu). Marks the cache line for eviction after this load retires. Effect is on subsequent kernels' L2 state; not observable in single-kernel microbenches. Retained as inferred pending multi-kernel probe.
-  ten_api_raw: wiki/nvidia/api-definitions/runtime/__ldlu.md
-- func_name: __ldcv
-  namespace: cuda-runtime
-  kind: load-intrinsic-volatile
-  signature: T __ldcv(const T* ptr);
-  notes: 'Cache-volatile load (PTX ld.global.cv). Always re-fetches (bypasses cache tag check). Use ONLY for flags written by other threads/kernels (correctness). Measured on H200: 2.26x slower than default at L2 regime — same penalty as __ldcg — with no compensating benefit for non-volatile data.'
-  ten_api_raw: wiki/nvidia/api-definitions/runtime/__ldcv.md
-- func_name: __stwb
-  namespace: cuda-runtime
-  kind: store-intrinsic-write-back
-  signature: void __stwb(T* ptr, T value);
-  notes: Write-back store (PTX st.global.wb, the default). Line stays in L2 and may be re-read by same SM. Not re-measured on H200; retained from legacy Skill 4.
-- func_name: __stcg
-  namespace: cuda-runtime
-  kind: store-intrinsic-cache-global
-  signature: void __stcg(T* ptr, T value);
-  notes: Cache-global store (PTX st.global.cg). L2 only, bypass L1. Symmetric with __ldcg. Not re-measured on H200.
-- func_name: __stcs
-  namespace: cuda-runtime
-  kind: store-intrinsic-cache-streaming
-  signature: void __stcs(T* ptr, T value);
-  notes: Cache-streaming store (PTX st.global.cs). L1 + L2 evict-first. Symmetric with __ldcs. Not re-measured on H200.
-- func_name: __stwt
-  namespace: cuda-runtime
-  kind: store-intrinsic-write-through
-  signature: void __stwt(T* ptr, T value);
-  notes: Write-through store (PTX st.global.wt). Writes visible to system memory immediately; useful for cross-device / host-observable writes. Not re-measured on H200.
-- func_name: ld.global.nc
-  namespace: ptx
-  kind: ptx-load-readonly
-  signature: ld.global.nc.type d, [a];
-  notes: Non-coherent read-only load. Emitted by __ldg and by compiler from const __restrict__ parameters. On H200 sm_9.0a, the L1/TEX are unified so this is the same cache as .ca.
-- func_name: ld.global.ca
-  namespace: ptx
-  kind: ptx-load-cache-all
-  signature: ld.global.ca.type d, [a];
-  notes: Cache-all (default). Caches in L1 + L2. Emitted by __ldca and by default C loads.
-- func_name: ld.global.cg
-  namespace: ptx
-  kind: ptx-load-cache-global
-  signature: ld.global.cg.type d, [a];
-  notes: Cache-global. Caches in L2 only; bypasses L1. Emitted by __ldcg.
-- func_name: ld.global.cs
-  namespace: ptx
-  kind: ptx-load-cache-streaming
-  signature: ld.global.cs.type d, [a];
-  notes: Cache-streaming. Caches in L1 + L2 with evict-first priority. Emitted by __ldcs.
-- func_name: ld.global.lu
-  namespace: ptx
-  kind: ptx-load-last-use
-  signature: ld.global.lu.type d, [a];
-  notes: Last-use. Caches normally but tags the line for eviction after this load. Emitted by __ldlu.
-- func_name: ld.global.cv
-  namespace: ptx
-  kind: ptx-load-cache-volatile
-  signature: ld.global.cv.type d, [a];
-  notes: Cache-volatile. Always re-fetches, bypassing tag check. Emitted by __ldcv.
-- func_name: st.global.wb
-  namespace: ptx
-  kind: ptx-store-write-back
-  notes: Write-back (default). L1 + L2. Emitted by default C stores and __stwb.
-- func_name: st.global.cg
-  namespace: ptx
-  kind: ptx-store-cache-global
-  notes: L2 only, bypass L1. Emitted by __stcg.
-- func_name: st.global.cs
-  namespace: ptx
-  kind: ptx-store-cache-streaming
-  notes: Cache-streaming evict-first. Emitted by __stcs.
-- func_name: st.global.wt
-  namespace: ptx
-  kind: ptx-store-write-through
-  notes: Write-through to system memory. Emitted by __stwt.
-- func_name: cudaFuncSetCacheConfig
-  namespace: cuda-runtime
-  kind: cache-config-setter
-  signature: cudaError_t cudaFuncSetCacheConfig(const void* func, cudaFuncCache cacheConfig);
-  notes: 'Sets the preferred L1 / shared memory balance for a kernel. Legacy pitfall P10: on Ampere+ the split is hardware-managed; this call has no observable effect. Not re-measured on H200 in this probe.'
-- func_name: cudaFuncAttributePreferredSharedMemoryCarveout
-  namespace: cuda-runtime
-  kind: func-attribute
-  notes: Newer alternative to cudaFuncSetCacheConfig; expresses desired shared-memory carveout as a percentage. Also may be a no-op on sm_9.0a.
-id: api-cache-load-hints-ref
-type: api-definition
-vendor: nvidia
-func_name: Cache Load Hints APIs
-namespace: runtime
-header: cuda_runtime.h
-signature: See documentation
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L25083-L25230
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/parallel-thread-execution/cuda_parallel-thread-execution_index.html.md
-  anchor: L10400-L10490
-architectures:
-- sm90
-- sm90a
-languages:
-- ptx
-- cuda-cpp
-techniques:
-- vectorized-loads
-- cache-policy
-- data-reuse
-- shared-memory-optimization
-confidence: inferred
-evidence_level: inferred
-tags:
-- vectorized-loads
-- cache-policy
-- data-reuse
-- shared-memory-optimization
-- ptx
-- cuda-cpp
----
-## Core load intrinsics (measured on H200 sm_9.0a)
-
-| Intrinsic   | PTX operator       | H200 measured behavior |
-|-------------|--------------------|------------------------|
-| *default* (plain pointer) | `ld.global.ca` | Baseline; L1 + L2. Rare in modern code — usually auto-promoted to `.nc` by `const __restrict__`. |
-| *default* (`const __restrict__`) | `ld.global.nc` | SASS `LDG.E.CONSTANT`. Auto-emitted by nvcc for read-only pointers; identical opcode to `__ldg`. Audited 2026-04-23. |
-| `__ldg`     | `ld.global.nc`     | SASS `LDG.E.CONSTANT`. **0.1-0.4% diff vs default** — noise. Same opcode as default when the pointer is `const __restrict__`. |
-| `__ldca`    | `ld.global.ca`     | SASS `LDG.E.STRONG.SM`. **Distinct from default under `const __restrict__`** — forces the L1+L2 cache-all path instead of the read-only `.nc` path. |
-| `__ldcg`    | `ld.global.cg`     | **2.26x slower** at L2-resident reuse regime (L1 bypass). |
-| `__ldcs`    | `ld.global.cs`     | Identical to default in un-contended L2; evict-first tag inactive. |
-| `__ldlu`    | `ld.global.lu`     | Not re-measured (cross-kernel effect). |
-| `__ldcv`    | `ld.global.cv`     | **2.26x slower** at L2-resident reuse. Use only for correctness (flag polling). |
-
-## Store intrinsics (not re-measured on H200)
-
-| Intrinsic   | PTX operator       | Purpose |
-|-------------|--------------------|---------|
-| *default*   | `st.global.wb`     | Write-back; line stays in L2. |
-| `__stwb`    | `st.global.wb`     | Explicit write-back. |
-| `__stcg`    | `st.global.cg`     | L2 only, bypass L1. Symmetric with `__ldcg`. |
-| `__stcs`    | `st.global.cs`     | Evict-first. Symmetric with `__ldcs`. |
-| `__stwt`    | `st.global.wt`     | Write-through to system memory. |
-
-## Cross-references
-
-- **10-api-raw stubs** (already exist): `__ldg.md`, `__ldca.md`,
-  `__ldcg.md`, `__ldcs.md`, `__ldcv.md`, `__ldlu.md`. This skill's
-  measured characteristics feed directly into the `probed_by:` field
-  of those stubs after `probe_loop crosslink`.
-- **`l2-access-policy` skill** (l2-access-policy): the stream-level window picks
-  which L2 lines survive under pressure; this skill picks which
-  cache levels the *load instruction* visits. Complementary axes —
-  can and should be used together for hot buffers.
-- **`coalescing` skill**: coalescing fixes the access *pattern*;
-  cache hints steer *where cached bytes live*. Fix coalescing first;
-  hints are a downstream concern.
-- **`vectorized-access` skill**: wider loads amortize the per-load
-  hint decision over more data. The probe's DRAM regime result
-  (all variants tie at 29% SOL) suggests single-float loads are
-  not the bottleneck — `float4` would shift DRAM throughput up
-  without changing the hint-choice verdict.
-
-## Related Probes
-
-- sources/experience/hw-probes/cache-hint.md —
-  6 variants × 2 regimes sweep. Load-bearing results: `__ldg` ≡
-  default on H200; `__ldcg` and `__ldcv` 2.26× slower on L2-resident
-  reuse; `__ldcs` is a null in un-contended L2.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/cache-load-hints/pitfalls.md

@@ -1,266 +0,0 @@
----
-title: Cache Load Hints — Pitfalls
-status: verified
-related_skill: wiki/nvidia/foundations/memory/cache-load-hints/skill.md
-source:
-- path: spec
-  anchor: Reference
-experience_refs:
-- sources/experience/hw-probes/cache-hint.md
-id: pitfall-cache-load-hints
-type: pitfall
-vendor: nvidia
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L25083-L25230
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/parallel-thread-execution/cuda_parallel-thread-execution_index.html.md
-  anchor: L10400-L10490
-architectures:
-- sm90
-- sm90a
-languages:
-- ptx
-- cuda-cpp
-techniques:
-- cache-policy
-- data-reuse
-- shared-memory-optimization
-confidence: inferred
-evidence_level: inferred
-tags:
-- cache-policy
-- data-reuse
-- shared-memory-optimization
-- ptx
-- cuda-cpp
----
-Legacy P1–P5 came from L3-sandbox verification on pre-H200 hardware.
-P6–P11 are L3-sandbox findings copied from the legacy KB. The H200
-probe on 2026-04-23 re-measured P6, P7, and the `__ldcg` /
-`__ldcv` regressions; annotations below distinguish
-"**Measured on H200 sm_9.0a**" results from legacy claims.
-
-## P1. Overusing `__ldg` on data that is modified (staleness)
-
-**Symptom**: Stale data read because `__ldg` uses the non-coherent
-read-only cache path (PTX `ld.global.nc`). Writes by the same or
-another kernel are not reflected.
-**Detection**: Correctness failure when kernel reads data that was
-recently written. Texture/RO cache is not invalidated by global
-stores during the same launch.
-**Fix**: Only use `__ldg` for data truly read-only for the kernel
-execution. For data possibly modified by the same kernel, use
-default loads (or drop the `__restrict__` qualifier so the compiler
-cannot auto-emit `ld.global.nc`).
-**Not re-measured on H200** — the probe kernel never writes to the
-buffer. Retained as legacy-anecdotal pending a read-after-write
-correctness probe.
-**Source**: PG §5.4.8.3 (L25083-L25130).
-
-## P2. Cache hints ignored by compiler or hardware
-
-**Symptom**: No performance change despite using a cache-hint
-intrinsic.
-**Detection**: Profile shows identical cache behavior with and
-without the hint. Compiler may reorder or merge loads; hardware may
-ignore advisory operators.
-**Fix**: Verify with `nvcc -ptx` that the expected
-`ld.global.{ca,cg,cs,lu,cv,nc}` instruction is present. On H200 the
-templated probe confirms the compiler emits distinct instructions
-per template (the wall-clock delta at L2 regime is the evidence —
-if all had merged, __ldcg would not be 2.26× slower than default).
-**Source**: PG §2.2.3.6; probe 2026-04-23.
-
-## P3. L1 bypass (`__ldcg`) causing L2 pressure — and SM-local reuse loss
-
-**Symptom (legacy, A100)**: "Using `__ldcg` everywhere causes L2
-miss rate to increase because more traffic goes through L2."
-**Symptom (measured on H200 sm_9.0a, 2026-04-23)**: The wall-clock
-cost is concrete and large: **2.26× slowdown** at L2-resident
-reuse regime (8 MiB buffer, 16 inner passes). The L2 miss-rate
-framing is a red herring on H200 — the actual problem is that
-bypassing L1 forces every read to route L2→registers, and L2 BW
-per SM is lower than L1 BW per SM.
-**Detection**: NCU `l1tex__t_requests_pipe_lsu_mem_global_op_ld.sum
-= 0` for kernels using `__ldcg`; compare wall-clock to a default
-variant on the same data.
-**Fix**: Only use `__ldcg` when the buffer is truly not reused
-within the SM's residency window. In single-kernel reductions /
-normalizations / scans that pass over input multiple times, this
-condition is almost never met — use default / `__ldca` / `__ldg`
-instead. The legacy framing "save L1 capacity for other data"
-requires the "other data" to actually benefit from L1; verify
-before reaching for `__ldcg`.
-**Source**: PG §5.4.8.3; probe 2026-04-23.
-
-## P4. Streaming hint (`__ldcs`) on reused data
-
-**Symptom (legacy)**: "Using `__ldcs` on data that is actually
-accessed multiple times causes repeated cache misses."
-**Symptom (measured on H200 sm_9.0a, 2026-04-23)**: **Not observed
-in un-contended L2.** With an 8 MiB buffer in a 60 MiB L2 and no
-competing traffic, `__ldcs` tags lines evict-first but the tags
-never trigger because there is no memory pressure to resolve.
-Measured wall-clock is **identical to default** (6043.67 GB/s
-both at 16 inner passes).
-**Updated fix**: The hazard still exists — but only in **contended
-L2** regimes. If you are applying `__ldcs` expecting a "streaming
-benefit" in a single-kernel workload, measure first; it is very
-likely a null. Single-stream MVPs should default-load.
-**Follow-up probe**: `sources/experience/hw-probes/cache-hint-contended/`
-(open).
-**Source**: PG §5.4.8.3; probe 2026-04-23.
-
-## P5. `cudaFuncSetCacheConfig` overridden by runtime
-
-**Symptom**: Requesting `cudaFuncCachePreferL1` but kernel still
-gets the default L1/shared split.
-**Detection**: Profile shows shared-memory carveout unchanged.
-**Fix (legacy)**: Use `cudaFuncSetAttribute` with
-`cudaFuncAttributePreferredSharedMemoryCarveout` for more direct
-control on newer architectures.
-**Note on H200**: Legacy P10 claims the config is entirely
-hardware-managed on Ampere+; not re-measured by this probe.
-Retained as inferred.
-**Source**: PG §3.2.6.
-
-## P6. `const __restrict__` already emits the read-only path — `__ldg` is redundant
-
-**Symptom (legacy)**: "The skill description itself acknowledges
-this: 'The compiler may already generate `ld.global.nc` for
-`const __restrict__` pointers' — in practice this means `__ldg`
-is a no-op on any reasonably modern toolchain."
-**Symptom (measured on H200 sm_9.0a, 2026-04-23)**: Confirmed.
-`__ldg` vs `default` wall-clock spread is **0.1%** at L2 regime
-and **0.4%** at DRAM regime — both within noise. Legacy P6
-**upgraded from inferred to measured**.
-**Fix**: Write `const __restrict__` on your parameters and let the
-compiler emit the right instruction. Explicit `__ldg` is not
-wrong but is not optimization either — it's documentation. Use it
-when the compiler cannot prove immutability (non-const pointer,
-aliased args) or when matching an existing codebase style.
-**Source**: PG §5.4.8.3; BP §12.2.5 (L1289-L1330); probe 2026-04-23.
-
-## P7. Simple streaming kernel: default ≈ streaming hint
-
-**Symptom (legacy)**: "On simple streaming kernels the default
-load path already behaves like `__ldcg`; the hint only matters
-when there is measurable L1 contention from competing data that
-would benefit from L1 residency."
-**Symptom (measured on H200 sm_9.0a, 2026-04-23)**: Refined.
-**At DRAM regime** (256 MiB single-pass), all six variants are
-identical at ~1.38 TB/s — consistent with legacy P7.
-**At L2 regime** (8 MiB reused), `default` is ~2.26× **faster**
-than `__ldcg` because the default uses L1 staging and `__ldcg`
-bypasses it. So legacy P7's "default behaves like __ldcg" is
-only true in the DRAM-bound regime; in the L2-resident regime the
-two are dramatically different.
-**Upgraded P7 fix**: Default-load unless you have measured
-evidence of L1 contention from competing data. "Saving L1" is
-not a free optimization on H200.
-**Source**: Level-3 sandbox verification (2026-04-04); probe 2026-04-23.
-
-## P8. Metric shifts from cache hints are misleading without wall-clock gain
-
-**Symptom (legacy)**: "DRAM bytes written dropped ~14.8% with
-`__stcs`, suggesting the cache-streaming hint altered write-back
-behavior, but this did not translate to any execution time
-improvement — metric shifts from cache hints can be misleading
-without end-to-end speedup."
-**Context on H200 sm_9.0a, 2026-04-23**: Confirmed on the load
-side as well. The DRAM regime NCU `dram__bytes_read.sum` is
-identical across all six load variants (268 MB per launch,
-matches the 256 MiB × 1 pass buffer), yet wall-clock is also
-identical. Metric-level "hints are working" evidence is not
-evidence of speedup.
-**Fix**: Always measure wall-clock or effective BW as the
-primary signal. Per-section NCU metric deltas are corroborating
-at best; a metric shift with no wall-clock delta means the hint
-is either irrelevant or the bottleneck has moved elsewhere.
-**Source**: Level-3 sandbox verification (2026-04-04); probe 2026-04-23.
-
-## P9. Store cache hints only matter when stores are the bottleneck
-
-**Symptom (legacy)**: "Store cache hints only matter when stores
-are the bottleneck; in load-latency-bound kernels they are
-irrelevant and may even slightly degrade performance by bypassing
-L2 write-back coalescing."
-**Not re-measured on H200** — this probe is read-only.
-`sources/experience/hw-probes/store-hint/` (open) is the follow-up.
-**Fix (legacy)**: Profile first. If the kernel is load-bound,
-skip store hints; if store-bound, compare `__stwb` (default) vs
-`__stcs` (evict-first) vs `__stcg` (L2-only) vs `__stwt`
-(write-through) and pick the winner.
-**Source**: Level-3 sandbox verification (2026-04-04).
-
-## P10. `cudaFuncSetCacheConfig` has no observable effect on Ampere+
-
-**Symptom (legacy)**: "On Ampere and later architectures, the
-L1/shared memory split is managed automatically by hardware;
-`cudaFuncSetCacheConfig` has no observable effect and may give
-a false sense of tuning."
-**Not re-measured on H200 sm_9.0a**. Retained as inferred. The
-probe does not exercise this API, and legacy P5 already covers
-the typical symptom (config request ignored).
-**Fix**: Don't rely on `cudaFuncSetCacheConfig` for L1 sizing on
-sm_90a; if you truly need more shared memory (≥ 48 KB per block)
-use `cudaFuncSetAttribute` with
-`cudaFuncAttributeMaxDynamicSharedMemorySize`.
-**Source**: Level-3 sandbox verification (2026-04-04).
-
-## P11. `__ldlu` benefit invisible in single-kernel microbench
-
-**Symptom (legacy)**: "The `__ldlu` hint did reduce DRAM write
-volume (likely fewer dirty L2 evictions), but this benefit is
-invisible in runtime for memory-bandwidth-bound kernels that
-are already underutilized — the cache pressure relief only
-matters when other concurrent kernels or data streams are
-competing for cache capacity."
-**Consistent with H200 2026-04-23 probe**: The same regime-gap
-observed for `__ldcs` (P4) applies to `__ldlu`. Single-stream
-microbench cannot show a cross-kernel L2-pressure-relief
-benefit; belongs in the contended follow-up probe.
-**Fix**: Retain `__ldlu` on known last-use reads (final iteration
-of a loop over a temp buffer the kernel is done with) as a low-
-cost documentation hint, but do not expect a measurable speedup
-in a single-kernel harness.
-**Follow-up probe**: `sources/experience/hw-probes/cache-hint-contended/`
-(open).
-**Source**: Level-3 sandbox verification (2026-04-04).
-
-## P12. `__ldcv` is 2.26× slower than default with no compensating benefit (NEW — measured)
-
-**Symptom (measured on H200 sm_9.0a, 2026-04-23)**:
-`__ldcv` at L2-resident 16-pass regime runs at 2673 GB/s vs
-default's 6044 GB/s — **the same ~44% of default** as `__ldcg`.
-**Cause**: `__ldcv` always re-fetches (PTX `ld.global.cv` bypasses
-the cache tag check). On H200's L1/L2 hierarchy, the measured
-wall-clock behaves as if L1 is bypassed; whether L2 is also
-bypassed is not distinguishable by wall-clock (both collapse to
-the same BW). Either way, **there is no performance use case for
-`__ldcv`**.
-**Fix**: Use `__ldcv` only for **correctness** — when polling a
-flag that another thread or kernel writes to and you need to see
-updated values. Never as a performance knob, never "to force a
-fresh read".
-**Source**: Discovered by the H200 probe 2026-04-23 (not in
-legacy KB).
-
-## P13. Template-load-variant pitfall when mixing const-qualification (NEW — probe-surfaced)
-
-**Symptom**: When switching a load site between default `*p` and
-`__ldg(p)` inside the same kernel template, the compiler may emit
-*different* cache operators than expected because `const` /
-`__restrict__` inference differs between the two paths.
-**Detection**: `nvcc -ptx` on each template specialization; confirm
-the expected `ld.global.{nc, ca, cg, cs, lu, cv}` appears.
-**Fix**: When writing a benchmark that compares variants, keep the
-pointer type identical across all templates (the probe's
-`stream_read<V>` does this) and rely on the intrinsic dispatch —
-not on qualifier changes — to select the cache operator. This
-isolates the cache-operator variable from the const-inference
-variable.
-**Source**: Probe 2026-04-23 methodology note (no symptom
-observed, but preventive discipline).