@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/cache-load-hints.md

@@ -1,373 +0,0 @@
----
-title: Cache Load Hints (__ldg / __ldca / __ldcg / __ldcs / __ldcv / __ldlu)
-status: verified
-evidence_level: measured
-cuda_version_tested: '12.9'
-driver_version_tested: 570.124.06
-toolchain: nvcc 12.9.86 + ptxas 12.9
-measured_on: H200-SXM
-applies_to_pattern_class:
-- cuda-core
-applies_to_ops:
-- elementwise
-- indexing
-- normalization
-- reduction
-requires_sm: '>=3.5'
-requires_features:
-- ptx-ld-cache-operators
-single_kernel_useful: true
-source:
-- path: spec
-  anchor: Reference
-artifacts:
-  code: artifacts/experience/hw-probes/cache-hint/cache_hint_probe.cu
-  build: artifacts/experience/hw-probes/cache-hint/build.sh
-  introspection: artifacts/experience/hw-probes/cache-hint/device.json
-  profile: ''
-related_apis:
-- __ldg
-- __ldca
-- __ldcg
-- __ldcs
-- __ldcv
-- __ldlu
-- ld.global.nc
-- ld.global.ca
-- ld.global.cg
-- ld.global.cs
-- ld.global.cv
-- ld.global.lu
-related_skills:
-- l2-access-policy
-- coalescing
-- vectorized-access
-- register-pressure
-experience_refs:
-- sources/experience/hw-probes/cache-hint.md
-id: skill-cache-load-hints
-type: skill
-vendor: nvidia
-tags:
-- cuda-cpp
-- pipeline-stages
-- vectorized-loads
-- cache-policy
-- data-reuse
-- fused-kernel
-- ptx
-applies_to:
-- general
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L25083-L25130
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/parallel-thread-execution/cuda_parallel-thread-execution_index.html.md
-  anchor: L10400-L10490
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1289-L1330
-architectures:
-- sm90
-- sm90a
-languages:
-- ptx
-- cuda-cpp
-techniques:
-- pipeline-stages
-- vectorized-loads
-- cache-policy
-- data-reuse
-kernel_types:
-- fused-kernel
-confidence: experimental
-artifact_dir: artifacts/experience/hw-probes/cache-hint
----
-## What
-
-CUDA exposes **six** per-load cache operators that steer a global-memory
-read into a specific cache level. At the PTX level they are
-`ld.global.{ca, cg, cs, lu, cv}` plus the non-coherent `ld.global.nc`;
-in CUDA C they appear as `__ldca / __ldcg / __ldcs / __ldlu / __ldcv`
-plus the specially-named `__ldg` (which emits `ld.global.nc`).
-
-Measured instruction mapping on H200 sm_9.0a (verified via `nvcc -ptx` + `cuobjdump --dump-sass` on the probe's six template specialisations, 2026-04-23):
-
-| Intrinsic | PTX emitted | SASS emitted | Caches used | Intended use |
-|-----------|-------------|--------------|-------------|--------------|
-| *default* (`*p` with `const __restrict__`) | `ld.global.nc` | `LDG.E.CONSTANT` | Read-only / L1-TEX | Compiler auto-promotes read-only pointers to the non-coherent path; identical to `__ldg` at HW level |
-| `__ldg`     | `ld.global.nc` | `LDG.E.CONSTANT`    | Read-only / L1-TEX    | Read-only data known immutable for kernel lifetime |
-| `__ldca`    | `ld.global.ca` | `LDG.E.STRONG.SM`   | L1 + L2               | Explicit cache-all; distinct from default on H200 (default is `.nc` not `.ca` when pointer is `const __restrict__`) |
-| `__ldcg`    | `ld.global.cg` | `LDG.E.STRONG.GPU`  | L2 only (bypass L1)   | Data not reused within SM |
-| `__ldcs`    | `ld.global.cs` | `LDG.E.EF`          | L1 + L2, **evict-first** | Streaming one-shot data |
-| `__ldlu`    | `ld.global.lu` | (not in probe)      | L1 + L2, **last-use** | Final read before discard |
-| `__ldcv`    | `ld.global.cv` | `LDG.E.STRONG.SYS`  | **always re-fetch** from memory | Volatile / cross-SM synchronization |
-
-Two things are specific to sm_9.0a (H200) and critical to using the
-hints correctly:
-
-1. The L1 and Tex caches are **unified** (one physical L1/TEX unit). `__ldg`'s old benefit — tapping a parallel texture pipeline — evaporates. Measured: `__ldg` and default differ by < 0.5% on H200, AND both lower to the same SASS opcode `LDG.E.CONSTANT` (audit 2026-04-23).
-2. With `const __restrict__` parameters, the compiler already emits
-   the read-only form (`ld.global.nc` or an equivalent optimized form)
-   automatically. Explicit `__ldg` is redundant on modern toolchains
-   and only helps when the compiler cannot prove immutability (no
-   `const`, aliased pointers, etc.).
-
-The skill is therefore not "always sprinkle cache hints"; it is a
-narrow toolbox with one measurably useful operator (`__ldcs` under
-contended L2) and one measurably dangerous one (`__ldcg` when the
-data is L2-resident and reused within an SM).
-
-## Why
-
-On H200's unified L1/TEX, the **only measurable per-load decision**
-is which levels of the hierarchy to use: L1+L2 vs L2-only vs
-L1+L2+evict-first vs bypass-all. Measured wall-clock differences:
-
-- **DRAM-bound single-pass reads** (working set > L2, each element
-  read once): all six operators within **0.5%** — hint choice is free
-  and irrelevant. Spend your optimization time on ILP or
-  vectorized-access instead.
-- **L2-resident reused reads** (working set < L2, multiple passes
-  per launch): choosing `__ldcg` or `__ldcv` costs **2.26×** wall-
-  clock because L1 is bypassed. The compiler's default (`ld.global.nc` with `const __restrict__`, else `ld.global.ca`)
-  or its semantic alias (`__ldca`, or `__ldg` on read-only data) is
-  the right choice.
-- **Evict-first (`__ldcs`)** shows no measurable effect in an
-  under-contended L2 — it tags lines for eviction preference but
-  only matters when another stream is fighting for the same L2
-  capacity. Same regime-gap pattern as `l2-access-policy` skill.
-
-The legacy skill's framing ("use `__ldcg` when data is not reused by
-the same SM") is **a trap** for typical cuda-core kernels, because
-single-kernel reuse patterns (multiple passes in reduction /
-normalization / scan) are exactly the SM-local reuse that L1 staging
-accelerates. `__ldcg` is for multi-kernel pipelines where the reuse
-is cross-SM (one block writes, a different block in a later launch
-reads) — a pattern that doesn't show up in cuda-core MVP.
-
-## When to use
-
-### S1. Default / `__ldca` — the correct choice for almost everything
-
-```cuda
-float v = in[i];                 // default with const __restrict__: ld.global.nc / LDG.E.CONSTANT (audited)
-// or explicitly:
-float v = __ldca(&in[i]);        // ld.global.ca / LDG.E.STRONG.SM (distinct from default!)
-```
-
-With `const __restrict__`, default lowers to `ld.global.nc` (identical to `__ldg`); without the qualifier, or with explicit `__ldca`, it lowers to `ld.global.ca`. Both forms are safe for any read pattern
-where the data is reused — within a warp, within a block, within
-an SM, across loop iterations. **This is the right choice in ≥ 90%
-of kernels.**
-
-### S2. `__ldg` — equivalent to default on H200, harmless legacy pattern
-
-```cuda
-__global__ void k(const float* __restrict__ in, ...) {
-    float v = __ldg(&in[i]);     // ld.global.nc
-}
-```
-
-Maps to `ld.global.nc` via the non-coherent read-only path. On H200
-the unified L1/TEX makes this the same physical cache as the default
-load, and measured 0.1-0.4% spread vs `default` is within noise.
-Writing `__ldg` is *not wrong* — it documents read-only intent and
-matches older codebases — but it is not *optimization*. Prefer
-`const __restrict__` on the parameter and let the compiler emit the
-same instruction automatically.
-
-### S3. `__ldcg` — only for cross-SM non-reuse patterns
-
-```cuda
-// ONLY when this buffer is NOT touched again by any thread in this
-// kernel's SM residency window. Typical use: producer kernel writes,
-// different consumer kernel (possibly on a different SM) reads once.
-float v = __ldcg(&in[i]);        // ld.global.cg (L2 only, bypass L1)
-```
-
-Maps to `ld.global.cg` (cache-global, bypass L1). **Measured on H200:
-2.26× slower** than default when the buffer is L2-resident and
-re-read 16 times within a single kernel launch. The "save L1 capacity
-for other data" argument (legacy Skill 2) only holds when:
-
-- The "other data" actually exists (another resident buffer benefits
-  from staying in L1), AND
-- The `__ldcg`-loaded buffer is NOT re-read by the same SM during
-  the same launch.
-
-Single-kernel reductions, normalizations, and scans typically make
-multiple passes over the input — so they are firmly **not** the
-right place for `__ldcg`.
-
-### S4. `__ldcs` — measurably a null on under-contended L2
-
-```cuda
-float v = __ldcs(&in[i]);        // ld.global.cs (evict-first)
-```
-
-Maps to `ld.global.cs`. Marks the cache line for preferential
-eviction when L2 is under pressure. **Measured on H200 at L2 regime
-(8 MiB buffer, 16 inner passes): identical to default** — because a
-60 MiB L2 with 8 MiB of hot data has no pressure to resolve.
-
-The legitimate use case is a **contended L2** — when a concurrent
-stream is touching > 20 MiB of its own data and would otherwise
-evict our streaming buffer. In single-kernel MVPs, drop `__ldcs` and
-use the default.
-
-### S5. `__ldcv` — for correctness, not performance
-
-```cuda
-int ready = __ldcv(&flag);       // ld.global.cv (always re-fetch)
-```
-
-Maps to `ld.global.cv` (cache-volatile). Bypasses all caches and
-always re-fetches from L2/DRAM. **Use only when polling a flag
-written by another thread/kernel** (correctness). Measured: 2.26×
-slower than default at L2 regime on H200, with no compensating
-benefit for non-volatile data.
-
-### S6. `__ldlu` — last-use eviction hint (not re-measured on H200)
-
-```cuda
-float v = __ldlu(&temp[i]);      // ld.global.lu (last-use)
-```
-
-Maps to `ld.global.lu`. Tags the line for eviction after this load.
-Its effect is on **subsequent** L2 pressure from later kernels — a
-cross-kernel optimization that a single-kernel harness cannot
-surface. Retained as inferred pending a multi-kernel probe
-(`sources/experience/hw-probes/cache-hint-contended/`, open).
-
-## When NOT to use
-
-- **DRAM-bound streaming kernels**. Measured: all six variants land
-  at ~1.38 TB/s within 0.5% on H200. The hint is free and irrelevant.
-- **Any time L1 staging is useful**. That includes every typical
-  reduction / normalization / scan / elementwise-with-shared-input
-  pattern. `__ldcg` / `__ldcv` cost 2.26× in those regimes.
-- **As a substitute for `cudaStreamAttributeAccessPolicyWindow`**.
-  The runtime-level window (skill `l2-access-policy`) and
-  per-instruction hints are *complementary*, not alternatives. For
-  a hot buffer shared across kernels, use the window **and** default
-  loads inside the kernel.
-- **To replace `const __restrict__` hygiene**. The compiler's
-  auto-emit path covers all the `__ldg` use cases when pointers are
-  correctly qualified. Fix your parameter declarations first.
-
-## Measured Characteristics
-
-Measured on H200-SXM (sm_9.0a, CUDA 12.9, driver 570.124.06) using
-sources/experience/hw-probes/cache-hint/ —
-6 load variants × 2 working-set regimes. Full record:
-sources/experience/hw-probes/cache-hint.md.
-
-### DRAM regime (256 MiB, single pass) — variants collapse to DRAM BW
-
-| variant   | GB/s_eff | vs default |
-| --------- | -------: | ---------: |
-| default   |  1377.44 |      1.000 |
-| `__ldg`   |  1383.35 |      1.004 |
-| `__ldca`  |  1381.52 |      1.003 |
-| `__ldcg`  |  1382.66 |      1.004 |
-| `__ldcs`  |  1384.26 |      1.005 |
-| `__ldcv`  |  1384.26 |      1.005 |
-
-Spread: **0.5%**. At DRAM-bound scale the per-load cache hint has
-zero effect; every variant runs at ~29% HBM3e SOL.
-
-### L2 regime (8 MiB, 16 inner passes) — L1 bypass is the only measurable loss
-
-| variant       | GB/s_eff | vs default | note |
-| ------------- | -------: | ---------: | ---- |
-| default       |  6043.67 |      1.000 | L1+L2 staged |
-| `__ldg`       |  6052.39 |      1.001 | same as default |
-| `__ldca`      |  6034.97 |      0.998 | same as default |
-| **`__ldcg`**  | **2668.13** | **0.441** | **L1 bypass** |
-| `__ldcs`      |  6043.67 |      1.000 | evict-first = null under un-contended L2 |
-| **`__ldcv`**  | **2673.23** | **0.442** | bypass-all behaves like L1 bypass |
-
-Key measured findings:
-
-- **`__ldg` vs default = 0.1% on L2, 0.4% on DRAM** — within noise.
-  Legacy pitfall P6 ("const __restrict__ already emits the right
-  instruction") upgraded from inferred to measured.
-- **`__ldcg` / `__ldcv` are 2.26× slower at L2 regime** because L1
-  staging is bypassed. For any kernel pattern with SM-local reuse
-  (including the typical grid-stride multi-pass pattern) these are
-  the wrong choice.
-- **`__ldcs` is a measured null at L2 regime**. Legacy pitfall P4
-  ("streaming hint on reused data") confirmed on H200 in the
-  un-contended case; the hint matters only when another stream is
-  fighting for L2.
-- **DRAM regime is variant-independent** at 0.5% spread. Spend
-  optimization effort on ILP / vector-load width instead.
-
-Follow-up probes open:
-
-- `sources/experience/hw-probes/cache-hint-contended/` — add a second
-  stream touching 50+ MiB of distinct data so the evict-first tag
-  (`__ldcs`) and last-use tag (`__ldlu`) can be measured.
-- `sources/experience/hw-probes/store-hint/` — counterpart for
-  `__stcs` / `__stwb` / `__stwt`; legacy Skill 4 not re-measured.
-
-## Principles
-
-1. **Default / `__ldca` / `__ldg` are equivalent on H200 for
-   `const __restrict__` pointers.** Pick whichever the codebase
-   reads clearest; don't chase micro-optimizations between them.
-2. **Never bypass L1 on data with SM-local reuse.** `__ldcg` and
-   `__ldcv` are 2.26× slower when the buffer is L2-resident and
-   re-read, which is the typical reduction / normalization pattern.
-3. **`__ldcv` is for correctness, not performance.** Use it to poll
-   flags written by other kernels. Never to "bypass stale caches"
-   in a single-kernel context — the caches are coherent for
-   non-`__ldg` loads within a kernel.
-4. **Cache hints compose with the runtime-level policy window.**
-   The `cudaStreamAttributeAccessPolicyWindow` picks which lines
-   L2 keeps; the cache hint picks which level the *load instruction*
-   routes through. They operate on different axes.
-
-## Open questions
-
-- Q1. Does `__ldcs` measurably help when a concurrent stream is
-  evicting the hot buffer? Follow-up probe
-  `sources/experience/hw-probes/cache-hint-contended/` (open).
-- Q2. **RESOLVED** (audit 2026-04-23): `nvcc -arch=sm_90a -O3 -ptx` emits six distinct `ld.global.{nc,ca,cg,cs,lu,cv}` instructions — no compiler folding. `cuobjdump --dump-sass` confirms six distinct SASS opcodes (`LDG.E.CONSTANT` / `LDG.E.STRONG.SM` / `LDG.E.STRONG.GPU` / `LDG.E.EF` / `LDG.E.STRONG.SYS`). The DRAM-regime 0.5% collapse is 'all hints honoured identically at DRAM-bound scale', not compiler folding. Default and `__ldg` share the same SASS (`LDG.E.CONSTANT`) with `const __restrict__` pointers, explaining the 0.1-0.4% wall-clock identity. See probe record §"PTX / SASS audit".
-- Q3. How do `__stcs` / `__stwb` / `__stwt` store hints behave on
-  H200? Legacy Skill 4 claimed use cases but was never re-measured.
-  Follow-up probe `sources/experience/hw-probes/store-hint/` (open).
-- Q4. Is `cudaFuncSetCacheConfig` observable on sm_9.0a? Legacy
-  pitfall P10 claims "no observable effect on Ampere+". Not
-  re-measured in this pass — belongs in a dedicated `L1/shared
-  carveout` probe (also open).
-
-## Legacy references
-
-- `legacy_sandbox_path`:
-  `corpus/nvidia/legacy-optimization/memory/cache-load-hints/skill.md`.
-  Legacy kept six sub-skills (S1 `__ldg`, S2 `__ldcg`, S3 `__ldcs`,
-  S4 store hints, S5 `cudaFuncSetCacheConfig`, S6 `__ldlu`); this
-  port reorganises by the H200 measurement:
-  - S1 folded into S2 (same wall-clock on H200) and demoted.
-  - S2 (cache-bypass L1) promoted to "measurably harmful in
-    SM-local-reuse patterns" (S3 in this skill).
-  - S3 (streaming) kept but marked as null outside contended regimes.
-  - S4 (store hints) deferred to a separate probe; retained in
-    legacy section as inferred.
-  - S5 (cache-config) routed to the `register-pressure` / shared-
-    carveout space (pending probe).
-  - S6 (`__ldlu`) retained as inferred pending contended-L2 probe.
-- Legacy pitfalls P1 (staleness), P2 (hints ignored), P5 (config
-  override), P9 (store hints irrelevant when load-bound), P10
-  (sm_80+ auto-managed L1/shared), P11 (`__ldlu` single-kernel
-  null) are retained anecdotally. P6 (redundant `__ldg`), P7
-  (default ≈ `__ldcg` on streaming), P8 (DRAM-bytes shift without
-  wall-clock impact) are **upgraded to measured** by the H200 probe.
-- **Related but distinct**:
-  - `wiki/nvidia/foundations/memory/l2-access-policy/` operates at the stream-level
-    — which lines L2 keeps under pressure. Composes with this skill.
-  - `wiki/nvidia/foundations/memory/coalescing/` fixes the address pattern; cache
-    hints are a downstream concern.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/coalescing/apis.md

@@ -1,129 +0,0 @@
----
-func_name: __ldg
-namespace: runtime
-header: cuda_runtime.h
-signature: T __ldg(const T* address)
-status: documented
-has_end_to_end_example: false
-source:
-- path: spec
-  anchor: Reference
-id: api-coalescing-ref
-type: api-definition
-vendor: nvidia
-title: Apis
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L24550-L24554
-architectures:
-- sm90
-- sm90a
-languages:
-- ptx
-- cuda-cpp
-techniques:
-- vectorized-loads
-- cache-policy
-confidence: inferred
-evidence_level: inferred
-tags:
-- vectorized-loads
-- cache-policy
-- ptx
-- cuda-cpp
----
-# Global Memory Coalescing API Reference
-
-This file lists the APIs touched by the coalescing skill. Each entry records the function name, namespace, and signature as found in upstream documentation.
-
-## Read-Only Load
-
-### `__ldg`
-
-- **Namespace**: runtime (CUDA C++ intrinsic)
-- **Header**: `cuda_runtime.h` (via `sm_32_intrinsics.h`)
-- **Signature**: `T __ldg(const T* address)`
-- **PTX**: `ld.global.nc` (non-coherent load through read-only / texture cache)
-- **Semantics**: Loads a value from global memory through the read-only data cache (L1/Tex cache path). The data must not be modified during the kernel's lifetime. On sm_35+, this provides a separate cache path from the normal L1, reducing pressure on the L1 and potentially improving bandwidth for read-only data. On sm_70+ (including H200), the compiler automatically generates `ld.global.nc` for `const __restrict__` pointers, making explicit `__ldg()` calls unnecessary in most cases.
-- **Supported types**: all C++ fundamental types, CUDA vector types (float2, float4, int2, int4, etc., except x3 variants), `__half`, `__half2`, `__nv_bfloat16`, `__nv_bfloat162`.
-- **Relation to coalescing**: `__ldg()` does not change the coalescing behavior -- the same 32-byte transaction rules apply. However, the separate cache path can improve effective bandwidth when the working set exceeds L1 capacity.
-- **Source**: Programming guide L24550-L24554
-
-### `__restrict__` pointer qualifier
-
-- **Namespace**: runtime (C++ extension)
-- **Header**: none (language keyword recognized by `nvcc`)
-- **Signature**: `void kernel(const float* __restrict__ in, float* __restrict__ out)`
-- **Semantics**: Promises that the pointer does not alias other pointers, enabling the compiler to: (a) cache loads in registers without re-reading, (b) reorder loads and stores, and (c) for `const __restrict__` pointers, automatically use `ld.global.nc` (equivalent to `__ldg()`).
-- **Relation to coalescing**: `__restrict__` does not directly affect coalescing, but the resulting `ld.global.nc` loads use the read-only cache path, improving bandwidth utilization for read-heavy kernels.
-- **Source**: Programming guide L22292-L22350
-
-## Vector Types for Wider Loads
-
-### `float4`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h` (included via `cuda_runtime.h`)
-- **Size**: 16 bytes, alignment 16 bytes
-- **Members**: `.x`, `.y`, `.z`, `.w`
-- **Constructor**: `make_float4(x, y, z, w)`
-- **Semantics**: A 128-bit vector type. When used for global memory access, a `float4` load compiles to a single 128-bit (`LDG.128`) instruction. A warp of 32 threads each loading one `float4` issues 32 x 16 = 512 bytes, serviced by 16 coalesced 32-byte transactions. This is perfectly coalesced and transfers 4x the data per transaction compared to scalar `float` loads.
-- **Usage pattern**:
-  ```cuda
-  __global__ void vec_load(const float4* __restrict__ in, float* out, int n) {
-      int tid = blockIdx.x * blockDim.x + threadIdx.x;
-      if (tid < n / 4) {
-          float4 v = in[tid];
-          out[tid * 4 + 0] = v.x;
-          out[tid * 4 + 1] = v.y;
-          out[tid * 4 + 2] = v.z;
-          out[tid * 4 + 3] = v.w;
-      }
-  }
-  ```
-- **Source**: Programming guide L22699 (float4 size=16, alignment=16)
-
-### `float2`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h`
-- **Size**: 8 bytes, alignment 8 bytes
-- **Members**: `.x`, `.y`
-- **Constructor**: `make_float2(x, y)`
-- **Semantics**: A 64-bit vector type. A `float2` load compiles to a single 64-bit (`LDG.64`) instruction. A warp of 32 threads each loading one `float2` issues 32 x 8 = 256 bytes, serviced by 8 coalesced 32-byte transactions. Useful when the data count is not a multiple of 4 but is a multiple of 2.
-- **Source**: Programming guide L22691 (float2 size=8, alignment=8)
-
-### `int4` / `uint4`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h`
-- **Size**: 16 bytes, alignment 16 bytes
-- **Members**: `.x`, `.y`, `.z`, `.w`
-- **Constructor**: `make_int4(x, y, z, w)` / `make_uint4(x, y, z, w)`
-- **Semantics**: 128-bit integer vector types. Same coalescing properties as `float4`. Often used as a generic 16-byte load type via `reinterpret_cast<const int4*>(ptr)` for type-punned vectorized loads.
-- **Source**: Programming guide L22635 (int4 size=16, alignment=16)
-
-## Other Cache-Hint Load Intrinsics
-
-### `__ldcg`, `__ldca`, `__ldcs`, `__ldlu`, `__ldcv`
-
-- **Namespace**: runtime
-- **Header**: `cuda_runtime.h`
-- **Signatures**:
-  - `T __ldcg(const T* address)` -- cache at global level, evict from L1
-  - `T __ldca(const T* address)` -- cache at all levels
-  - `T __ldcs(const T* address)` -- streaming load, evict-first
-  - `T __ldlu(const T* address)` -- last-use hint
-  - `T __ldcv(const T* address)` -- volatile (bypass cache)
-- **Relation to coalescing**: These do not change coalescing behavior; they control cache residency policy. `__ldcs` is useful for streaming kernels (elementwise, reduction) where data is read once and should not pollute L2. `__ldcg` bypasses L1 but keeps data in L2.
-- **Source**: Programming guide L24559-L24563
-
-## Related Probes
-
-- `__ldg` — end-to-end example, build command, and H200 measurement. See probe record.
-- `__ldcv` — end-to-end example, build command, and H200 measurement. See probe record.
-- `__ldca` — end-to-end example, build command, and H200 measurement. See probe record.
-- `__ldcs` — end-to-end example, build command, and H200 measurement. See probe record.
-- `__ldcg` — end-to-end example, build command, and H200 measurement. See probe record.
-- `__ldlu` — end-to-end example, build command, and H200 measurement. See probe record.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/coalescing/pitfalls.md

@@ -1,157 +0,0 @@
----
-title: Global Memory Coalescing - Pitfalls
-status: draft
-evidence_level: spec
-applies_to_pattern_class:
-- cuda-core
-applies_to_ops:
-- elementwise
-- reduction
-- normalization
-- scan
-- gather
-- scatter
-requires_sm: '>=6.0'
-single_kernel_useful: true
-source:
-- path: spec
-  anchor: Reference
-id: pitfall-coalescing
-type: pitfall
-vendor: nvidia
-architectures:
-- '*'
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L606-L634
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L1389-L1393
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L1446-L1448
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L569-L603
-languages:
-- cuda-cpp
-techniques:
-- cache-policy
-- data-reuse
-- shared-memory-optimization
-kernel_types:
-- fused-kernel
-confidence: source-reported
-tags:
-- cache-policy
-- data-reuse
-- shared-memory-optimization
-- fused-kernel
-- cuda-cpp
----
-## P1: Array-of-Structures (AoS) layout
-
-**Symptom**: Kernel achieves far less than expected memory bandwidth despite sequential thread indexing.
-
-**Cause**: When data is stored as an array of structures (e.g., `struct { float x, y, z; } points[N]`), consecutive threads reading `points[tid]` each load a full struct. The hardware sees addresses spaced `sizeof(struct)` apart per thread. For a 12-byte struct, the warp touches more 32-byte segments than necessary because the x, y, and z fields of different elements are interleaved.
-
-**Example**:
-```cuda
-// BAD: AoS -- threads read interleaved fields
-struct Point { float x, y, z; };
-__global__ void process_aos(Point* points, float* out, int n) {
-    int tid = blockIdx.x * blockDim.x + threadIdx.x;
-    if (tid < n)
-        out[tid] = points[tid].x + points[tid].y;  // stride = 12 bytes
-}
-
-// GOOD: SoA -- each field is contiguous
-__global__ void process_soa(float* x, float* y, float* out, int n) {
-    int tid = blockIdx.x * blockDim.x + threadIdx.x;
-    if (tid < n)
-        out[tid] = x[tid] + y[tid];  // stride = 4 bytes (coalesced)
-}
-```
-
-**Fix**: Convert to Structure-of-Arrays (SoA) layout: separate arrays for each field. If AoS is required by an external interface, load the data into separate shared memory arrays (SoA in shared memory) as a staging step.
-
-**Detection**: Profile with `ncu` and check the `l1tex__t_sectors_pipe_lsu_mem_global_op_ld.sum` metric. If it is significantly higher than `l1tex__t_requests_pipe_lsu_mem_global_op_ld.sum * 4` (for float), the access is not coalesced.
-
-## P2: Row-major 2D access with wrong thread dimension mapping
-
-**Symptom**: Matrix kernels run at a fraction of expected bandwidth.
-
-**Cause**: In a 2D thread block, `threadIdx.x` is the fastest-varying dimension. If `threadIdx.x` maps to the row index and `threadIdx.y` to the column index of a row-major matrix, consecutive threads (consecutive `threadIdx.x` values) access elements that are `width` apart in memory -- a large stride.
-
-**Example**:
-```cuda
-// BAD: threadIdx.x maps to row (large stride in row-major)
-int row = blockIdx.x * blockDim.x + threadIdx.x;
-int col = blockIdx.y * blockDim.y + threadIdx.y;
-float val = matrix[row * width + col];
-
-// GOOD: threadIdx.x maps to column (contiguous in row-major)
-int row = blockIdx.y * blockDim.y + threadIdx.y;
-int col = blockIdx.x * blockDim.x + threadIdx.x;
-float val = matrix[row * width + col];
-```
-
-**Fix**: Ensure `threadIdx.x` maps to the contiguous storage dimension (column for row-major, row for column-major).
-
-## P3: Misaligned base addresses
-
-**Symptom**: 5 transactions instead of 4 for a warp reading 32 consecutive floats.
-
-**Cause**: When the starting address of the warp's access is not aligned to a 32-byte boundary, the accessed range spans one extra 32-byte segment. For example, if the base address has a 4-byte offset, the 128 bytes accessed by the warp span 5 segments instead of 4 (best-practices guide L569-L577).
-
-**Fix**: Use `cudaMalloc` (which returns 256-byte-aligned pointers) and ensure that per-warp offsets maintain alignment. Common violations: manually computed offsets (e.g., `ptr + 1` for a float array shifts all subsequent warps by 4 bytes), or packed struct arrays with non-power-of-two sizes.
-
-**Mitigation**: On sm_60+ with L1 caching, the impact is modest (~10% bandwidth loss per the best-practices guide L603-604) because adjacent warps reuse the over-fetched cache lines. Still, aligning base addresses is a free optimization.
-
-## P4: Indirect (gather) access with random indices
-
-**Symptom**: Memory bandwidth is 10-30x below peak despite the kernel being memory-bound.
-
-**Cause**: When threads access `data[index[tid]]` and the `index` array contains scattered values, the warp touches up to 32 distinct cache lines. No code-level change to the kernel can fix this -- the access pattern is inherently non-coalesced.
-
-**Fix**: Algorithmic approaches:
-- Sort the work items by their index values so that nearby threads access nearby memory (e.g., sort particles by cell index before a stencil kernel).
-- Use binning / tiling to group elements that access the same region.
-- Pre-fetch into shared memory if the same data is accessed multiple times.
-
-**Detection**: `ncu` metric `smsp__sass_average_data_bytes_per_sector_mem_global_op_ld.pct` significantly below 100% indicates poor coalescing.
-
-## P5: Writing the transpose without shared memory staging
-
-**Symptom**: Transpose kernel achieves ~12.5% of peak bandwidth.
-
-**Cause**: A naive transpose `C[col * N + row] = A[row * N + col]` has coalesced reads (consecutive `threadIdx.x` maps to consecutive columns in row-major A) but non-coalesced writes (consecutive `threadIdx.x` maps to rows in the output, which are `N` elements apart). The write side issues up to 32 transactions per warp.
-
-**Fix**: Load a tile into shared memory with coalesced reads, synchronize, then write from shared memory with coalesced writes (swapping row/column indices). This is the canonical shared-memory-tiled transpose (programming guide L1486-L1601).
-
-## P6: Block size not a multiple of warp size
-
-**Symptom**: Under-utilized warps at block boundaries, poor coalescing at the edges.
-
-**Cause**: The best-practices guide (L1118) states: "The number of threads per block should be a multiple of 32 threads, because this provides optimal computing efficiency and facilitates coalescing." With non-multiple block sizes, the last warp in each block is partially filled, wasting memory transaction capacity.
-
-**Fix**: Always use block sizes that are multiples of 32 (common choices: 128, 256, 512).
-
-## P7: Confusing L1 cache rescue with actual coalescing
-
-**Symptom**: Slightly misaligned or stride-2 kernels perform better than expected, leading developers to ignore coalescing.
-
-**Cause**: On sm_60+, L1 caching is enabled by default. Adjacent warps often reuse cache lines fetched by neighbors, partially masking the cost of non-coalesced access (best-practices guide L604: "the impact is not as large as we might have expected"). This can make stride-2 access appear to lose only ~10-20% bandwidth rather than the theoretical 50%.
-
-**Risk**: Relying on L1 cache rescue is fragile. With larger strides, random access, or working sets that exceed L1 capacity, the cache cannot help and the full coalescing penalty applies.
-
-**Fix**: Always write coalesced access patterns. Treat L1 cache rescue as a safety net, not a strategy.
-
-## P8: Forgetting to coalesce the store side
-
-**Symptom**: Kernel reads are coalesced but overall bandwidth is poor.
-
-**Cause**: Developers focus on coalescing reads but forget that stores follow the same rules. Non-coalesced stores are equally costly (and on GDDR memory with ECC, even worse due to read-modify-write overhead for partial sector writes -- best-practices guide L551).
-
-**Fix**: Apply the same stride-1 mapping to output arrays. If the algorithm inherently requires non-coalesced output (e.g., scatter), use shared memory staging or atomics as appropriate.