@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/layout-transform.md

@@ -1,231 +0,0 @@
----
-title: Memory Layout Transformation (AoS ↔ SoA, Pitched, Byte Permute)
-status: verified
-evidence_level: measured
-cuda_version_tested: '12.9'
-driver_version_tested: 570.124.06
-toolchain: nvcc 12.9.86 + ptxas 12.9
-measured_on: H200-SXM
-applies_to_pattern_class:
-- cuda-core
-applies_to_ops:
-- elementwise
-- indexing
-- transpose
-- reduction
-requires_sm: '>=7.0'
-requires_features:
-- coalesced-memory-model
-single_kernel_useful: true
-source:
-- path: spec
-  anchor: Reference
-artifacts:
-  code: artifacts/experience/hw-probes/aos-vs-soa/aos_vs_soa_probe.cu
-  build: artifacts/experience/hw-probes/aos-vs-soa/build.sh
-  introspection: artifacts/experience/hw-probes/aos-vs-soa/device.json
-  profile: ''
-related_apis:
-- cudaMallocPitch
-- cudaMemcpy2D
-- cudaMalloc3D
-- prmt.b32
-- cp.async.bulk.tensor
-related_skills:
-- coalescing
-- shared-memory-cache
-- vectorized-access
-- bank-conflict
-id: skill-layout-transform
-type: skill
-vendor: nvidia
-tags:
-- cuda-cpp
-- tma
-- fp8
-- pipeline-stages
-- vectorized-loads
-- cache-policy
-- shared-memory-optimization
-- swizzling
-- fused-kernel
-- quantization
-- ptx
-applies_to:
-- general
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L606-L634
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L1379-L1411
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L1413-L1483
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/parallel-thread-execution/cuda_parallel-thread-execution_index.html.md
-  anchor: L13528-L13530
-architectures:
-- sm90
-- sm90a
-languages:
-- ptx
-- cuda-cpp
-hardware_features:
-- tma
-- fp8
-techniques:
-- pipeline-stages
-- vectorized-loads
-- cache-policy
-- shared-memory-optimization
-- swizzling
-kernel_types:
-- fused-kernel
-- quantization
-confidence: experimental
-artifact_dir: artifacts/experience/hw-probes/aos-vs-soa
----
-## What
-
-**Layout transformation** is the decision of *how data is organized in memory before a kernel touches it*, separate from *how the kernel accesses that memory*. The two are coupled by the hardware's 32-byte coalesced-transaction rule (PG §2.2.4.1, L1379-L1411): adjacent threads must land in the same 32-byte segment, or the warp issues multiple transactions and wastes bandwidth proportional to the stride.
-
-Four classical choices sit under this umbrella, each trading **when** the cost is paid for **where** the win appears:
-
-1. **AoS → SoA** — convert `struct{x,y,z}[N]` into `{x[N], y[N], z[N]}` so each hot field becomes a stride-1 array.
-2. **Pitched 2D allocation** — let `cudaMallocPitch` pad row width so every row starts at a coalescing-aligned address.
-3. **In-kernel transpose** — use shared memory to pivot a tile so both the read *and* the write sides are coalesced; mechanism owned by the sibling `shared-memory-cache` skill.
-4. **Byte permutation (`prmt.b32`)** — sub-word reordering inside 32-bit registers for tight formats (RGBA↔BGRA, packed int8/fp8).
-
-BP §10.2.1.4 (L606-L634) quantifies the stakes: stride 2 → 50 % bandwidth efficiency; stride 32 → 12.5 %. Any kernel whose natural access pattern has stride > 1 on the fastest-moving dim is paying this tax until a layout transform is applied.
-
-## Why
-
-Unlike coalescing (which is a kernel-side fix) and bank-conflict avoidance (which is a smem-side fix), layout-transform is a **pre-kernel** fix: it changes the data itself so that coalescing becomes automatic. That makes it expensive up front (a conversion kernel or a different allocation API) but free-forever for every subsequent kernel.
-
-The strategic question is **amortization**: if one downstream kernel benefits, the transform is usually not worth it; if many kernels over the lifetime of the buffer benefit, it almost always is. BP §10.2.1.4's closing line — "non-unit-stride global memory accesses should be avoided whenever possible" — is a cost bound, not a hard rule. The skill below is the set of techniques for actually avoiding them, plus the guardrails for when the cost of the transform does *not* amortize (see pitfall P3).
-
-## When to use
-
-### S1. AoS → SoA conversion (primary sub-skill)
-
-When the input is a struct-of-N-fields array and each kernel touches a **subset** of fields (the common case: particle systems where position-only kernels don't touch velocity, or feature-vector pipelines where one pass normalizes only a subset of columns), split the AoS buffer into one flat array per field.
-
-```cuda
-struct Particle { float x, y, z, vx, vy, vz; };     // 24 B per element
-
-// Kernel reads only `.x`. In AoS it strides 24 B per thread (wastes
-// 20/24 of every L2 transaction); in SoA it strides 4 B (coalesced).
-__global__ void move_x_soa(const float* __restrict__ x,
-                           float*       __restrict__ x_out,
-                           int n) {
-    int i = blockIdx.x * blockDim.x + threadIdx.x;
-    if (i < n) x_out[i] = x[i] * 2.0f;
-}
-```
-
-The one-time conversion kernel:
-
-```cuda
-__global__ void aos_to_soa(const Particle* __restrict__ in,
-                           float* __restrict__ x, float* __restrict__ y,
-                           float* __restrict__ z,
-                           /* ...vx,vy,vz */,
-                           int n) {
-    int i = blockIdx.x * blockDim.x + threadIdx.x;
-    if (i < n) {
-        Particle p = in[i];                 // coalesced 24-B read per thread
-        x[i] = p.x;  y[i] = p.y;  z[i] = p.z;   // each store is stride-1
-        // ...
-    }
-}
-```
-
-BP §10.2.1.4 (L606-L634) quantifies the upside: the AoS-read-one-field pattern has stride equal to `sizeof(struct) / sizeof(field)`.
-
-### S2. Pitched 2D allocation
-
-When a 2D array's row width in bytes is not a multiple of the coalescing granularity (128 B on current architectures), adjacent rows start at misaligned offsets and every row's first warp pays a coalescing penalty. `cudaMallocPitch` rounds each row up to a well-aligned pitch:
-
-```cuda
-float* d_mat;
-size_t pitch;                                // returned, in BYTES
-cudaMallocPitch(&d_mat, &pitch, cols * sizeof(float), rows);
-
-// Indexing:
-float* row_k = (float*)((char*)d_mat + k * pitch);
-
-// Host ↔ device copy MUST use cudaMemcpy2D and the pitch:
-cudaMemcpy2D(d_mat, pitch,
-             h_data, cols * sizeof(float),  // host pitch (tight)
-             cols * sizeof(float), rows,
-             cudaMemcpyHostToDevice);
-```
-
-Use this when `cols * sizeof(elem) % 128 != 0` and the kernel's access pattern is row-oriented. For row widths that are already 128 B multiples, plain `cudaMalloc` is cheaper — pitched allocation increases total footprint (see pitfall P4).
-
-### S3. In-kernel transpose (pointer; mechanism in shared-memory-cache)
-
-The `[TILE][TILE+1]`-padded shared-memory transpose is the standard out-of-place layout transform when no SoA split is possible (dense matrix transpose, batched axis swap). The **kernel** lives in `wiki/nvidia/foundations/memory/shared-memory-cache/skill.md` §S2; this skill records the *layout decision* — when transposing ahead-of-time beats running every downstream kernel with non-coalesced access. Measured on H200 (4096² fp32): padded tiled transpose hits 1685 GB/s; the per-step cost is ~0.08 ms. If > 3 downstream kernels benefit, the transpose amortizes.
-
-### S4. Byte permutation with `prmt.b32`
-
-When the data is sub-word — e.g. `uchar4` pixels, packed int8 for mixed-precision inference, packed fp8 scales — and the required layout change is a within-register byte reorder, inline-PTX `prmt.b32` avoids round-tripping through memory:
-
-```cuda
-// Reverse the 4 bytes inside a 32-bit word (RGBA → ABGR).
-__device__ __forceinline__ unsigned int byte_reverse(unsigned int a) {
-    unsigned int r;
-    asm("prmt.b32 %0, %1, 0, 0x00010203;" : "=r"(r) : "r"(a));
-    return r;
-}
-```
-
-The selector `c` in `prmt.b32 d, a, b, c` is a 4-nibble control picking bytes 0..7 from the concatenation `(a || b)` (PTX ISA L13528). This is a register-only op, faster than any memory-based swizzle when the reorder fits inside 32 bits. For wider reorders, chain multiple `prmt.b32` calls or fall back to S3.
-
-## When NOT to use
-
-- **One-shot data.** If the buffer is read once and discarded, the conversion kernel's own memory traffic matches the traffic you would have paid with the bad layout. Amortization requires ≥ ~3 downstream kernels (derived from the probe: AoS→SoA kernel runs at ~HBM-peak, and so does each SoA kernel; the break-even is roughly (1 + 1) transform + SoA ops vs (stride-N) AoS ops).
-- **Kernel hits all fields.** If every downstream kernel touches every field of the struct (the "hot data set = full struct" case), AoS and SoA are equivalent at the HBM level. SoA can then be worse because three separate buffers have three separate cache-line streams with weaker L1 locality.
-- **Row width already aligned.** Don't add `cudaMallocPitch` for row widths that are already multiples of 128 B. The pitch padding adds footprint without adding coalescing (pitfall P4).
-- **Sub-word permutation that fits a compile-time constant shuffle.** If the compiler already produces a single `prmt.b32` from a C++ expression involving `__byte_perm`, handwritten inline asm adds maintenance cost without perf gain — check SASS before committing.
-
-## Measured Characteristics
-
-Measured on H200-SXM (sm_90a, CUDA 12.9, driver 570.124.06) using sources/experience/hw-probes/aos-vs-soa/ — AoS-vs-SoA probe on `Particle{x,y,z,vx,vy,vz}` (24 B struct), N = 16,777,216 elements, kernel reads one field. Unlocked clock logged at 1980 MHz. Full record: sources/experience/hw-probes/aos-vs-soa.md.
-
-| Kernel                        | Median ms | Useful BW GB/s | DRAM SoL | Warp cyc/issue | Speedup |
-| ----------------------------- | --------: | -------------: | -------: | -------------: | ------: |
-| `aos_read_one_field`          |    0.1111 |           1208 |   89.16% |         110.77 |   1.00× |
-| `soa_read_one_field`          |    0.0558 |           2405 |   43.03% |      **51.34** |   1.99× |
-| `soa_vectorized` (float4)     | **0.0372**|       **3613** |   71.31% |         110.66 |**2.99×**|
-| `aos_to_soa_convert`          |    0.2017 |           3993 |   81.12% |         101.40 |     n/a |
-
-**Amortization break-even: 3.65 downstream calls** (measured). The conversion kernel costs 0.2017 ms; each downstream SoA kernel saves 0.0553 ms vs keeping the data in AoS.
-
-Key measured findings:
-
-- **Stride penalty is ~2× on H200, not the naive 6×** (struct bytes over field bytes). L2 partially absorbs wasted sectors and the lower instruction count on SoA gives extra headroom. This is a measured correction to the folklore "stride-N = N× slower".
-- **DRAM SOL alone is misleading**: the AoS kernel hits 89 % DRAM SOL because HBM is moving ~4.4 TB/s of *bytes*, but only 1.2 TB/s of *useful* bytes (24 % efficiency). Always compare to a stride-1 baseline when investigating a memory-bound kernel (pitfall P5).
-- **Vectorization on SoA is free 1.5×** on top of the AoS→SoA transform (float4 loads pack 128 B of useful data per warp transaction).
-- **Conversion kernel itself hits 81 % DRAM SOL and 71 % L1/TEX hit rate** — the AoS→SoA split is not a bandwidth waster, it's just a fixed one-pass cost that amortizes after ~4 downstream kernels.
-- S2 (`cudaMallocPitch`), S3 (in-kernel transpose; mechanism in sibling `shared-memory-cache` probe), S4 (`prmt.b32`) are **not measured by this probe**. S3 is covered by the sibling skill's 2026-04-21 probe; S2 and S4 remain `inferred` until follow-up probes land.
-
-## Principles
-
-1. **Layout is a *pre-kernel* fix; coalescing is a *kernel* fix.** Both change the same metric (effective bandwidth), but at different points in the pipeline. Use layout transform when the access pattern cannot be changed from inside the kernel (e.g. you must read `.x` only).
-2. **Amortize or don't transform.** A layout-transform pays off only when ≥ several downstream kernels use the new layout. One-shot buffers should keep their ingress layout.
-3. **SoA splits should be by access-locality, not by struct fields.** If `.x` and `.y` are always accessed together, leave them as `struct{x,y}[N]`. Only split along access boundaries.
-4. **`prmt.b32` is register-local.** Sub-word layout changes that can be expressed as byte-permute fit in one instruction; extending the idea to cross-register swaps requires memory ping-pong and becomes a variant of S3.
-
-## Open questions
-
-- Q1. What is the break-even number of downstream SoA kernels vs (1 × AoS→SoA convert + N × AoS kernels) on H200 for the 24 B struct shape of the probe? Planned follow-up probe: `aos-vs-soa/amortization-curve/`.
-- Q2. Does `cp.async.bulk.tensor` (TMA with a tensor descriptor, sm_90+) perform an implicit layout transform during copy? If so, pairs of S1 + TMA may collapse into a single-instruction path. Blocked on the `wiki/nvidia/hardware/tma/` entry (pending bucket F bootstrap).
-- Q3. On H200, when does pitched allocation's footprint overhead cost more in L2 miss rate than it saves in coalescing? Legacy pitfall P9 claims L2 hit rate dropped 62.7 %→50.1 % on an earlier device; needs re-measurement.
-
-## Legacy references
-
-- `legacy_sandbox_path`: `KernelPilot/optimization/memory/layout-transform/skill.md`. Original 5 sub-skills are routed as: Skill 1 (shared-mem transpose) → lives in sibling `shared-memory-cache` S2 (mechanism); layout decision pointer retained here as S3. Skill 2 (AoS→SoA) → kept as primary S1. Skill 3 (in-place row-to-column reorder) → retained conceptually inside S3 as the in-place variant of the transpose mechanism. Skill 4 (`cudaMallocPitch`) → kept as S2. Skill 5 (`prmt.b32`) → kept as S4.
-- Legacy Level-3 sandbox findings P5-P9 are retained in `pitfalls.md`; they were run on pre-H200 hardware and require re-measurement.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/register-pressure/apis.md

@@ -1,84 +0,0 @@
----
-func_name: cudaFuncGetAttributes
-namespace: runtime
-header: cuda_runtime_api.h
-signature: cudaError_t cudaFuncGetAttributes(struct cudaFuncAttributes *attr, const void *func)
-status: documented
-has_end_to_end_example: false
-source:
-- path: spec
-  anchor: Reference
-id: api-register-pressure-ref
-type: api-definition
-vendor: nvidia
-title: Apis
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L28314
-architectures:
-- sm90
-- sm90a
-languages:
-- cuda-cpp
-techniques:
-- register-budgeting
-- shared-memory-optimization
-confidence: inferred
-evidence_level: inferred
-tags:
-- register-budgeting
-- shared-memory-optimization
-- cuda-cpp
----
-# Register Pressure API Reference
-
-This file lists the APIs touched by the register-pressure skill. Each entry records the function name, namespace, and signature as found in upstream documentation.
-
-## Detection APIs
-
-### cudaFuncGetAttributes
-
-- **Namespace**: runtime
-- **Header**: `cuda_runtime_api.h`
-- **Signature**: `cudaError_t cudaFuncGetAttributes(struct cudaFuncAttributes *attr, const void *func)`
-- Returns `cudaFuncAttributes` containing `numRegs` (registers per thread) and `localSizeBytes` (local memory per thread; non-zero implies spilling).
-
-### cudaOccupancyMaxActiveBlocksPerMultiprocessor
-
-- **Namespace**: runtime
-- **Header**: `cuda_runtime_api.h`
-- **Signature**: `cudaError_t cudaOccupancyMaxActiveBlocksPerMultiprocessor(int *numBlocks, const void *func, int blockSize, size_t dynamicSMemSize)`
-- Computes the maximum number of active blocks per SM for a given kernel and block size. When registers are the binding constraint, the result will be lower than the thread/smem limits would allow.
-
-## Control APIs
-
-### __launch_bounds__
-
-- **Namespace**: runtime (compiler directive)
-- **Signature**: `__launch_bounds__(maxThreadsPerBlock, minBlocksPerMultiprocessor)`
-- Annotates a `__global__` function with occupancy targets. The compiler derives a register ceiling L = regsPerSM / (maxTPB * minBlocks).
-- Cross-reference: [compiler-hints skill](../../compute/compiler-hints.md)
-
-### __maxnreg__
-
-- **Namespace**: runtime (compiler directive)
-- **Signature**: `__maxnreg__(maxNumberRegistersPerThread)`
-- Directly caps per-thread register count. Mutually exclusive with `__launch_bounds__` on the same kernel.
-- Cross-reference: [compiler-hints skill](../../compute/compiler-hints.md)
-
-## Diagnostic Flags
-
-### --maxrregcount=N
-
-- **Type**: nvcc compiler flag
-- Caps register count for all `__global__` functions in the compilation unit. Ignored for kernels with `__maxnreg__`.
-
-### -Xptxas=-v
-
-- **Type**: nvcc compiler flag
-- Reports per-kernel register count, shared memory, and spill stores/loads at compile time. Primary tool for detecting register pressure.
-
-## Related Probes
-
-- `cudaOccupancyMaxActiveBlocksPerMultiprocessor` — end-to-end example, build command, and H200 measurement. See probe record.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/register-pressure/pitfalls.md

@@ -1,97 +0,0 @@
----
-title: Register Pressure -- Pitfalls
-parent_skill: register-pressure
-id: pitfall-register-pressure
-type: pitfall
-vendor: nvidia
-architectures:
-- sm90
-- sm90a
-languages:
-- cuda-cpp
-techniques:
-- pipeline-stages
-- register-budgeting
-- loop-unrolling
-- shared-memory-optimization
-kernel_types:
-- quantization
-confidence: inferred
-evidence_level: inferred
-tags:
-- pipeline-stages
-- register-budgeting
-- loop-unrolling
-- shared-memory-optimization
-- quantization
-- cuda-cpp
----
-## Pitfall 1: Blindly maximizing occupancy via --maxrregcount causes catastrophic spilling
-
-**Symptom**: Kernel runs much slower after adding `--maxrregcount=N` despite higher reported occupancy from `cudaOccupancyMaxActiveBlocksPerMultiprocessor`.
-
-**Root cause**: Forcing a low register cap (e.g., 32) on a kernel that naturally needs more (e.g., 64) causes the compiler to spill excess values to local memory. Local memory resides in the global memory space (L1/L2/HBM), with latencies 100-1000x worse than register access.
-
-**Detection**: Compile with `-Xptxas=-v` and check for non-zero "spill stores" and "spill loads". Also check `cudaFuncGetAttributes` for `localSizeBytes > 0`.
-
-**Measured example**: On H200, a 48-accumulator kernel at `--maxrregcount=32` showed 660 bytes spill stores + 784 bytes spill loads per thread, causing a 4.84x slowdown despite doubling occupancy from 50% to 100% (see probe record).
-
-**Fix**: Remove or increase the `--maxrregcount` value. Use `-Xptxas=-v` to find the natural register count, then set the cap at or above that value. Alternatively, use `__launch_bounds__` which lets the compiler make occupancy-aware decisions rather than imposing a hard cap.
-
-## Pitfall 2: __launch_bounds__ with excessive minBlocksPerMultiprocessor
-
-**Symptom**: Adding `__launch_bounds__(256, 8)` causes spilling on a kernel that ran without spills when no launch bounds were specified.
-
-**Root cause**: With `minBlocksPerMultiprocessor=8` and `maxTPB=256`, the register ceiling is L = 65536 / (256 * 8) = 32 registers per thread. If the kernel naturally needs more than 32 registers, the compiler must spill.
-
-**Detection**: Compare `-Xptxas=-v` output before and after adding launch bounds. Look for non-zero spill counts in the annotated variant.
-
-**Fix**: Reduce `minBlocksPerMultiprocessor` to a value where L exceeds the kernel's natural register demand. For a kernel needing 64 registers with block size 256: 65536 / (256 * L_target) >= 64, so max blocks = 4. Use `__launch_bounds__(256, 4)` or `__launch_bounds__(256, 3)`.
-
-## Pitfall 3: Silent register pressure increase from loop unrolling
-
-**Symptom**: Adding `#pragma unroll` or increasing the unroll factor causes a performance regression, even though the loop body is simple.
-
-**Root cause**: Full unrolling of a loop with N iterations creates N copies of all loop-body variables, multiplying register pressure by up to N. If this pushes the kernel past an occupancy cliff (e.g., from 4 blocks/SM to 3 blocks/SM), the occupancy drop hurts more than the unrolling helped.
-
-**Detection**: Compare register counts with `-Xptxas=-v` before and after the unroll pragma. Watch for occupancy cliffs by computing `floor(65536 / (regs_per_thread * block_size / 32))` warps.
-
-**Fix**: Use `#pragma unroll K` with a bounded unroll factor instead of unbounded `#pragma unroll`. For FP32 on H200, K=4 matches the FMA pipeline depth and is usually sufficient.
-
-## Pitfall 4: Large local arrays silently placed in local memory
-
-**Symptom**: A kernel declares a small array (e.g., `float buf[64]`) inside the kernel function and shows unexpectedly high local memory usage.
-
-**Root cause**: The programming guide states that "Arrays for which [the compiler] cannot determine that they are indexed with constant quantities" are placed in local memory. If the array index is runtime-variable, the entire array goes to local memory regardless of register availability. Even with constant indexing, large arrays exceed the register budget and are placed in local memory.
-
-**Detection**: Check `-Xptxas=-v` for local memory usage or `localSizeBytes` from `cudaFuncGetAttributes`.
-
-**Fix**: Replace runtime-indexed arrays with explicit scalar variables when the access pattern is known at compile time. For genuinely variable-indexed buffers, consider using shared memory instead.
-
-## Pitfall 5: Confusing register count with register pressure
-
-**Symptom**: A kernel uses 64 registers/thread and 0 spills, but the programmer adds `--maxrregcount=32` because "64 registers is too many."
-
-**Root cause**: Register count alone does not indicate a problem. A kernel using 64 registers with 0 spills at 50% occupancy may be performing optimally. Register pressure is only a problem when either (a) the kernel is spilling, or (b) the kernel is latency-bound and would benefit from higher occupancy.
-
-**Detection**: Before reducing registers, measure whether the kernel is compute-bound or memory-bound. If compute-bound, more registers (and more ILP) may help. If memory-bound, more occupancy (fewer registers) may help -- but only if it does not cause spilling.
-
-**Fix**: Profile first, tune second. Use `cudaOccupancyMaxActiveBlocksPerMultiprocessor` to check current occupancy, and `-Xptxas=-v` to check spill status. Only reduce registers if profiling shows the kernel is latency-bound with low occupancy and no spilling would result from the reduction.
-
-## Pitfall 6: --maxrregcount is per-file, not per-kernel
-
-**Symptom**: A file contains two kernels -- one lightweight (needs 16 regs) and one heavy (needs 80 regs). Setting `--maxrregcount=48` helps the heavy kernel's occupancy but the lightweight kernel was already fine.
-
-**Root cause**: `--maxrregcount` applies to all `__global__` functions in a compilation unit. It cannot be targeted at individual kernels.
-
-**Detection**: Check `-Xptxas=-v` output for all kernels in the file.
-
-**Fix**: Use `__maxnreg__(N)` or `__launch_bounds__` per-kernel instead of the file-wide `--maxrregcount`. Alternatively, split kernels into separate compilation units with different flags.
-
-## Pitfall 7: __maxnreg__ and __launch_bounds__ are mutually exclusive
-
-**Symptom**: Compilation error when both `__maxnreg__(N)` and `__launch_bounds__(M, K)` are applied to the same kernel.
-
-**Root cause**: The programming guide explicitly states "The `__launch_bounds__()` and `__maxnreg__()` qualifiers cannot be applied to the same kernel together."
-
-**Fix**: Choose one mechanism. `__launch_bounds__` is generally preferred because it lets the compiler balance register usage against the stated occupancy target. `__maxnreg__` is a harder cap for when you need exact register control.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/register-pressure.md

@@ -1,251 +0,0 @@
----
-title: Register Pressure
-status: draft
-evidence_level: measured
-applies_to_pattern_class:
-- cuda-core
-applies_to_ops:
-- elementwise
-- reduction
-- normalization
-- scan
-requires_sm: '>=9.0'
-requires_features: []
-single_kernel_useful: true
-cuda_version_tested: '12.9'
-driver_version_tested: 570.124.06
-toolchain: nvcc 12.9 + ptxas 12.9
-measured_on: H200-SXM
-source:
-- path: spec
-  anchor: Reference
-artifacts:
-  code: artifacts/experience/hw-probes/register-pressure/reg_pressure_probe.cu
-  build: nvcc -arch=sm_90a -O3 -std=c++17 -Xptxas=-v -o probe_default reg_pressure_probe.cu
-  introspection: ''
-  profile: ''
-related_apis:
-- cudaFuncGetAttributes
-- cudaOccupancyMaxActiveBlocksPerMultiprocessor
-- __launch_bounds__
-- __maxnreg__
-related_skills:
-- compiler-hints
-- ilp
-id: skill-register-pressure
-type: skill
-vendor: nvidia
-tags:
-- cuda-cpp
-- persistent-kernel
-- pipeline-stages
-- cache-policy
-- register-budgeting
-- loop-unrolling
-- shared-memory-optimization
-- fused-kernel
-applies_to:
-- general
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1064-L1065
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1093-L1094
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L1215
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L1219-L1223
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L22818-L22837
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1089
-architectures:
-- sm90
-- sm90a
-languages:
-- cuda-cpp
-techniques:
-- persistent-kernel
-- pipeline-stages
-- cache-policy
-- register-budgeting
-- loop-unrolling
-- shared-memory-optimization
-kernel_types:
-- fused-kernel
-confidence: experimental
-artifact_dir: artifacts/experience/hw-probes/register-pressure
----
-## What
-
-Register pressure is the condition where a kernel's demand for per-thread registers exceeds what is available without sacrificing either occupancy or performance. On H200 (sm_90a), each SM has 65,536 32-bit registers shared among all resident threads. The maximum per-thread register count is 255. Register allocation determines the fundamental tradeoff between occupancy (how many warps can co-reside on an SM) and per-thread capability (how many values each thread can hold without spilling to slow local memory).
-
-The three key mechanisms at play:
-
-1. **Occupancy is bounded by register usage.** With block size 256 and 64 registers/thread, only 4 blocks fit: 65536 / (256 * 64) = 4 blocks = 1024 threads = 32 warps = 50% occupancy. With 32 registers/thread, 8 blocks fit = 2048 threads = 64 warps = 100% occupancy.
-
-2. **Register spilling goes to local memory.** When the compiler cannot fit all live variables into the available register budget, it spills values to local memory. Despite the name "local", this storage resides in the global memory space (device DRAM), cached through L1 and L2. The programming guide states: "Local memory is thread local storage similar to registers and managed by NVCC, but the physical location of local memory is in the global memory space."
-
-3. **Spill cost dominates occupancy gain.** A register access costs zero extra cycles. A spill load/store traverses the L1/L2/HBM hierarchy with latencies ranging from tens to hundreds of cycles. Our probe shows that doubling occupancy via forced spilling caused a 4.84x slowdown.
-
-The compiler manages register allocation automatically using heuristics that balance register usage, spill cost, and instruction count. The programmer can influence this via `__launch_bounds__`, `__maxnreg__`, and `--maxrregcount` (see the compiler-hints skill).
-
-Register allocation also interacts with ILP: more independent accumulator chains require more registers. The ILP skill discusses this tradeoff -- specifically, 4 accumulators for FP32 on H200 is the sweet spot for pipeline saturation, using approximately 4 extra registers per chain.
-
-## Why
-
-Register pressure matters because registers are the fastest storage on the GPU (zero extra latency) and the most constrained shared resource determining occupancy.
-
-On H200 with 65,536 registers per SM and 2,048 maximum threads per SM:
-- Perfect occupancy (100%) allows only 65536 / 2048 = **32 registers per thread**.
-- Most non-trivial kernels need 40-80 registers per thread, meaning occupancy is often register-limited to 50-75%.
-- This is by design: the best practices guide notes "Higher occupancy does not always equate to higher performance -- there is a point above which additional occupancy does not improve performance."
-
-The register-vs-occupancy tradeoff has an optimal range that depends on the kernel's workload:
-
-- **Compute-bound kernels** benefit from more registers (less recomputation, more ILP capacity) even at reduced occupancy, because ILP within each warp hides arithmetic latency.
-- **Memory-bound kernels** benefit from higher occupancy to hide memory latency through warp-level parallelism (TLP), so fewer registers per thread is preferable -- provided no spilling occurs.
-- **The boundary condition is spilling.** Any occupancy gain that comes at the cost of register spilling is almost always a net loss.
-
-## When to use
-
-Register pressure management is relevant in these situations:
-
-- **After observing non-zero spill stores/loads in `-Xptxas=-v` output.** Any spill count > 0 is a signal. Small spills (< 16 bytes) may be tolerable; large spills (> 100 bytes per thread) are a performance emergency.
-
-- **When `cudaOccupancyMaxActiveBlocksPerMultiprocessor` reports fewer blocks than expected.** If the register count per thread is the binding constraint (rather than shared memory or block size), register pressure management can increase occupancy.
-
-- **When adding ILP accumulators or loop unrolling causes a performance regression.** The additional registers consumed by ILP may push the kernel past an occupancy cliff (e.g., from 4 blocks/SM to 3 blocks/SM).
-
-- **When a kernel has many live variables.** Stencil computations, matrix operations with tile buffering, or kernels processing multiple elements per thread can easily exceed 64 registers.
-
-## When NOT to use
-
-- **Do not blindly minimize registers.** The goal is not "fewest registers" but "best overall throughput." A kernel at 50% occupancy with zero spills will often outperform the same kernel at 100% occupancy with spills.
-
-- **Do not add `--maxrregcount` without checking spill output.** This flag is a blunt instrument that affects all kernels in a compilation unit. If it causes spilling, it will hurt performance. Always verify with `-Xptxas=-v`.
-
-- **Do not over-constrain `__launch_bounds__`.** Setting `minBlocksPerMultiprocessor` too high forces the compiler to reduce register usage, potentially causing spilling. See the compiler-hints skill for proper usage.
-
-- **Do not sacrifice ILP to save registers unless profiling shows register pressure is the bottleneck.** Reducing from 4 accumulators to 1 saves ~3 registers but costs a 4x throughput reduction in compute-bound phases (see the ILP skill).
-
-## Techniques to reduce register pressure
-
-1. **`__launch_bounds__(maxTPB, minBlocks)`** -- Tells the compiler the expected launch configuration so it can compute a register ceiling L = regsPerSM / (maxTPB * minBlocks). The compiler will reduce register usage to stay at or below L, spilling only if necessary. (Link: compiler-hints skill)
-
-2. **`__maxnreg__(N)` or `--maxrregcount=N`** -- Directly caps the per-thread register count. Cannot be combined with `__launch_bounds__` on the same kernel. Only use when you know the target register count will not cause excessive spilling.
-
-3. **Reduce ILP factor** -- If a kernel uses N independent accumulator chains and is register-limited, reducing N frees N registers at the cost of reduced per-thread throughput. The sweet spot on H200 is typically 4 chains for FP32 (matching the 4-cycle FMA pipeline depth).
-
-4. **Break large kernels into smaller sequential stages** -- A monolithic kernel with many live variables across phases forces all variables to be live simultaneously. Splitting into multiple kernel launches (persistent-kernel or multi-pass) allows each stage to use fewer registers. The overhead is kernel launch latency (typically 3-10 us).
-
-5. **Manual register tiling via shared memory** -- Explicitly store intermediate values in shared memory instead of registers. This trades register pressure for shared memory pressure and adds shared memory access latency (~20-30 cycles on H200), but shared memory is far faster than local memory spills (~100+ cycles for L2/HBM).
-
-6. **Use smaller data types** -- `__half` or `__nv_bfloat16` values can be packed two per 32-bit register, halving register consumption for accumulator arrays. This also doubles throughput for supported operations.
-
-## How to detect register pressure
-
-### Compile-time: `-Xptxas=-v`
-
-```bash
-nvcc -arch=sm_90a -O3 -std=c++17 -Xptxas=-v -o kernel kernel.cu
-```
-
-Output format:
-```
-ptxas info : Function properties for _Z<mangled>
-    <N> bytes stack frame, <S> bytes spill stores, <L> bytes spill loads
-ptxas info : Used <R> registers, ...
-```
-
-- `Used R registers`: per-thread register count
-- `S bytes spill stores` / `L bytes spill loads`: non-zero = spilling
-- Additional warning flags: `-Xptxas=-warn-spills` (warn on any spill), `-Xptxas=-warn-lmem-usage` (warn on any local memory use)
-
-### Runtime: `cudaFuncGetAttributes`
-
-```cpp
-cudaFuncAttributes attr;
-cudaFuncGetAttributes(&attr, myKernel);
-printf("regs=%d, localSize=%zu\n", attr.numRegs, attr.localSizeBytes);
-```
-
-- `attr.numRegs`: register count per thread
-- `attr.localSizeBytes`: local memory per thread (> 0 implies spilling)
-
-### Runtime: occupancy query
-
-```cpp
-int numBlocks;
-cudaOccupancyMaxActiveBlocksPerMultiprocessor(
-    &numBlocks, myKernel, blockSize, dynamicSmem);
-float occupancy = (float)(numBlocks * blockSize) /
-                  deviceProp.maxThreadsPerMultiProcessor;
-```
-
-## Reading the occupancy math on H200
-
-H200 (sm_90a) register file parameters:
-- 65,536 registers per SM
-- 255 max registers per thread
-- 2,048 max threads per SM (64 warps)
-- Register allocation granularity: 256 registers per warp
-
-Occupancy formula (register-limited case):
-```
-warps_by_regs = floor(65536 / (regs_per_thread * 32))
-  -- but regs_per_thread is rounded up to nearest multiple of 8
-     (256 regs per warp / 32 threads per warp = 8 reg granularity)
-occupancy = min(warps_by_regs, 64) / 64
-```
-
-Example register-occupancy table (block size 256):
-
-| regs/thread | warps possible | blocks/SM | occupancy |
-|---|---|---|---|
-| 32 | 64 | 8 | 100% |
-| 40 | 51 | 6 | 75% |
-| 48 | 42 | 5 | 62.5% |
-| 56 | 36 | 4 | 50% |
-| 64 | 32 | 4 | 50% |
-| 80 | 25 | 3 | 37.5% |
-| 96 | 21 | 2 | 25% |
-| 128 | 16 | 2 | 25% |
-| 160 | 12 | 1 | 12.5% |
-| 255 | 8 | 1 | 12.5% |
-
-Note: the actual occupancy depends on the interplay of register count, shared memory per block, and block size. Registers are often the binding constraint.
-
-## Measured Characteristics
-
-- Register pressure spill tradeoff probe: On H200 (sm_90a, CUDA 12.9), a 48-accumulator FMA kernel compiled with default register allocation used **64 registers/thread** with **zero spills** (0.1738 ms median, 4 blocks/SM, 50% occupancy). The same kernel compiled with `--maxrregcount=32` used **32 registers/thread** but generated **660 bytes of spill stores** and **784 bytes of spill loads** per thread (0.8414 ms median, 8 blocks/SM, 100% occupancy). Result: **4.84x slowdown** despite doubled occupancy, confirming that spill cost vastly outweighs occupancy gain when the spill volume is large.
-
-## Health thresholds
-
-Spill detection is exact, not a threshold: local memory exists in a kernel only
-because the compiler put it there.
-
-```
-smsp__inst_executed_op_local_ld.sum == 0
-smsp__inst_executed_op_local_st.sum == 0      =>  no spill at all
-```
-
-Any non-zero value means some value did not fit in the register file. NCU also
-reports the SASS-level variants (`smsp__sass_inst_executed_op_local_*`), which
-count the same events.
-
-⚠️ **Spill is not binary in its cost.** A small spill whose working set stays
-resident in L1 can be nearly free, while a spill in the innermost loop of a
-memory-bound kernel is not. The counter tells you spill happened; whether it
-matters is a separate question, answered by whether the local traffic shows up
-in the L1/L2 sector counts.
-
-`launch__registers_per_thread` is the knob on the other side: raising it
-reduces spill and lowers occupancy, and the right value is whichever of those
-two costs is smaller for this kernel.