@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/vectorized-access/apis.md

@@ -1,157 +0,0 @@
----
-func_name: float4
-namespace: runtime
-header: vector_types.h
-signature: struct float4 { float x, y, z, w; }
-status: documented
-has_end_to_end_example: false
-source:
-- path: spec
-  anchor: Reference
-id: api-vectorized-access-ref
-type: api-definition
-vendor: nvidia
-title: Apis
-architectures:
-- '*'
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L22574-L22702
-languages:
-- ptx
-- cuda-cpp
-techniques:
-- vectorized-loads
-- cache-policy
-- register-budgeting
-confidence: inferred
-evidence_level: inferred
-tags:
-- vectorized-loads
-- cache-policy
-- register-budgeting
-- ptx
-- cuda-cpp
----
-# Vectorized Memory Access API Reference
-
-This file lists the APIs touched by the vectorized-access skill. Each entry records the type name, namespace, and relevant properties as found in upstream documentation. All size and alignment values are from the programming guide Table 42 (L22578).
-
-## 128-bit Vector Types (16 bytes)
-
-### `float4`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h` (included via `cuda_runtime.h`)
-- **Size**: 16 bytes
-- **Alignment**: 16 bytes
-- **Members**: `.x`, `.y`, `.z`, `.w` (each `float`)
-- **Constructor**: `make_float4(float x, float y, float z, float w)`
-- **PTX load instruction**: `LDG.128` (128-bit global load)
-- **PTX store instruction**: `STG.128` (128-bit global store)
-- **Semantics**: The primary 128-bit vector type for float data. When used for global memory access, a `float4` load compiles to a single 128-bit instruction. A warp of 32 threads each loading one `float4` requests 32 x 16 = 512 bytes, serviced by 16 coalesced 32-byte transactions.
-- **Source**: Programming guide L22699 (float4 size=16, alignment=16), L22735-L22753 (vector type members and constructors)
-
-### `int4`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h`
-- **Size**: 16 bytes
-- **Alignment**: 16 bytes
-- **Members**: `.x`, `.y`, `.z`, `.w` (each `signed int`)
-- **Constructor**: `make_int4(int x, int y, int z, int w)`
-- **Semantics**: 128-bit integer vector type. Often used as a generic 16-byte load type via `reinterpret_cast<const int4*>(ptr)` for type-punned vectorized loads when the actual data type differs from float.
-- **Source**: Programming guide L22635 (int4 size=16, alignment=16)
-
-### `uint4`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h`
-- **Size**: 16 bytes
-- **Alignment**: 16 bytes
-- **Members**: `.x`, `.y`, `.z`, `.w` (each `unsigned int`)
-- **Constructor**: `make_uint4(unsigned int x, unsigned int y, unsigned int z, unsigned int w)`
-- **Semantics**: Unsigned 128-bit integer vector type. Same memory behavior as `int4`. Commonly used for raw byte loading of 16-byte chunks when the data needs manual unpacking (e.g., loading 8 half-precision values as one `uint4` and then reinterpreting the components).
-- **Source**: Programming guide L22635 (uint4 size=16, alignment=16)
-
-### `longlong2`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h`
-- **Size**: 16 bytes
-- **Alignment**: 16 bytes
-- **Members**: `.x`, `.y` (each `signed long long`)
-- **Constructor**: `make_longlong2(long long x, long long y)`
-- **Semantics**: 128-bit vector of two 64-bit integers. Generates `LDG.128`. Useful for type-punned 128-bit loads when the data elements are 8 bytes each (e.g., `double`, `int64_t`).
-- **Source**: Programming guide L22667-L22670 (longlong2 size=16, alignment=16)
-
-### `double2`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h`
-- **Size**: 16 bytes
-- **Alignment**: 16 bytes
-- **Members**: `.x`, `.y` (each `double`)
-- **Constructor**: `make_double2(double x, double y)`
-- **Semantics**: 128-bit vector of two double-precision floats. Generates `LDG.128`. This is the widest available vector load for double data (no `double4` with 16-byte alignment exists in the standard types).
-- **Source**: Programming guide L22707-L22710 (double2 size=16, alignment=16)
-
-## 64-bit Vector Types (8 bytes)
-
-### `float2`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h`
-- **Size**: 8 bytes
-- **Alignment**: 8 bytes
-- **Members**: `.x`, `.y` (each `float`)
-- **Constructor**: `make_float2(float x, float y)`
-- **PTX load instruction**: `LDG.64` (64-bit global load)
-- **Semantics**: A 64-bit vector type. A `float2` load compiles to a single 64-bit instruction. A warp of 32 threads each loading one `float2` issues 32 x 8 = 256 bytes, serviced by 8 coalesced 32-byte transactions. Useful when the data count is not a multiple of 4 but is a multiple of 2, or when 128-bit loads would cause excessive register pressure.
-- **Source**: Programming guide L22691 (float2 size=8, alignment=8)
-
-### `int2` / `uint2`
-
-- **Namespace**: runtime (built-in type)
-- **Header**: `vector_types.h`
-- **Size**: 8 bytes
-- **Alignment**: 8 bytes
-- **Members**: `.x`, `.y` (each `int` / `unsigned int`)
-- **Constructor**: `make_int2(int x, int y)` / `make_uint2(unsigned int x, unsigned int y)`
-- **Semantics**: 64-bit integer vector types. Generate `LDG.64`. Often used for 8-byte type-punned loads.
-- **Source**: Programming guide L22627-L22630 (int2/uint2 size=8, alignment=8)
-
-## Constructor Functions
-
-### `make_float4`
-
-- **Namespace**: runtime
-- **Header**: `vector_types.h`
-- **Signature**: `float4 make_float4(float x, float y, float z, float w)`
-- **Semantics**: Factory function that constructs a `float4` value from four float components.
-- **Source**: Programming guide L22744-L22753
-
-### `make_float2`
-
-- **Namespace**: runtime
-- **Header**: `vector_types.h`
-- **Signature**: `float2 make_float2(float x, float y)`
-- **Semantics**: Factory function that constructs a `float2` value from two float components.
-- **Source**: Programming guide L22744-L22753
-
-### `make_int4`
-
-- **Namespace**: runtime
-- **Header**: `vector_types.h`
-- **Signature**: `int4 make_int4(int x, int y, int z, int w)`
-- **Semantics**: Factory function that constructs an `int4` value.
-- **Source**: Programming guide L22744-L22753
-
-### `make_uint4`
-
-- **Namespace**: runtime
-- **Header**: `vector_types.h`
-- **Signature**: `uint4 make_uint4(unsigned int x, unsigned int y, unsigned int z, unsigned int w)`
-- **Semantics**: Factory function that constructs a `uint4` value.
-- **Source**: Programming guide L22744-L22753

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/vectorized-access/pitfalls.md

@@ -1,185 +0,0 @@
----
-title: Vectorized Memory Access - Pitfalls
-status: draft
-evidence_level: spec
-applies_to_pattern_class:
-- cuda-core
-applies_to_ops:
-- elementwise
-- reduction
-- normalization
-- scan
-requires_sm: '>=6.0'
-single_kernel_useful: true
-source:
-- path: spec
-  anchor: Reference
-id: pitfall-vectorized-access
-type: pitfall
-vendor: nvidia
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L22574-L22702
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1062
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L22735-L22753
-architectures:
-- sm90
-- sm90a
-languages:
-- cuda-cpp
-techniques:
-- pipeline-stages
-- vectorized-loads
-- register-budgeting
-- kernel-fusion
-kernel_types:
-- fused-kernel
-confidence: source-reported
-tags:
-- pipeline-stages
-- vectorized-loads
-- register-budgeting
-- kernel-fusion
-- fused-kernel
-- cuda-cpp
----
-## P1: Misaligned pointer for vector load
-
-**Symptom**: Runtime crash (misaligned address fault) or silent corruption when loading float4 from a pointer that is not 16-byte aligned.
-
-**Cause**: `float4` has a 16-byte alignment requirement (programming guide Table 42, L22578). If `reinterpret_cast<const float4*>(ptr)` is used on a pointer `ptr` that is not 16-byte aligned, the resulting load triggers an alignment fault. This can happen when:
-- The base pointer has an offset applied (e.g., `ptr + 1` shifts by 4 bytes, breaking 16-byte alignment).
-- The data starts at a struct member with insufficient alignment.
-- A sub-array slice begins at a non-aligned offset.
-
-**Example**:
-```cuda
-// BAD: ptr+1 is 4-byte aligned, not 16-byte aligned
-float* ptr = d_array;  // 256-byte aligned from cudaMalloc
-float4 v = reinterpret_cast<const float4*>(ptr + 1)[tid];  // CRASH
-
-// GOOD: ensure offset is a multiple of 4 floats (16 bytes)
-float4 v = reinterpret_cast<const float4*>(ptr)[tid];  // OK
-// or with aligned offset:
-float4 v = reinterpret_cast<const float4*>(ptr + 4 * k)[tid];  // OK if k is integer
-```
-
-**Fix**: Ensure the pointer passed to `reinterpret_cast<float4*>()` is 16-byte aligned. Base pointers from `cudaMalloc` are 256-byte aligned, so they are safe. Any offset from the base must be a multiple of 4 elements (for float) or 16 bytes in general. If alignment cannot be guaranteed, fall back to scalar loads for the misaligned portion.
-
-**Detection**: The kernel will abort with `misaligned address` if CUDA error checking is enabled. In debug mode (`-G` flag), the CUDA runtime detects the fault immediately.
-
-## P2: Out-of-bounds read when N is not a multiple of 4
-
-**Symptom**: Reading beyond the allocated buffer, causing incorrect results or memory access violations.
-
-**Cause**: When using `float4` loads with `n` total floats and `n` is not divisible by 4, the last `float4` load at index `n/4` reads `n/4 * 4` through `n/4 * 4 + 3`, which may extend past the end of the allocation.
-
-**Example**:
-```cuda
-// BAD: n=1025, n/4=256, thread 256 reads bytes 4096-4111 but only 4100 allocated
-int tid = blockIdx.x * blockDim.x + threadIdx.x;
-if (tid < (n + 3) / 4) {  // This allows index 256 which overflows
-    float4 v = reinterpret_cast<const float4*>(input)[tid];
-}
-```
-
-**Fix**: Two approaches:
-1. **Pad the allocation**: allocate `((n + 3) / 4) * 4` floats and zero-pad the tail. This is the simplest and most efficient approach.
-2. **Separate tail loop**: process the bulk with float4 loads and handle the remaining `n % 4` elements with scalar loads:
-```cuda
-int n4 = n / 4;
-if (tid < n4) {
-    float4 v = reinterpret_cast<const float4*>(input)[tid];
-    // ... process v
-}
-// Tail: only first (n % 4) threads handle remainder
-int tail_start = n4 * 4;
-if (tid < (n - tail_start)) {
-    float s = input[tail_start + tid];
-    // ... process s
-}
-```
-
-## P3: Incorrect reinterpret_cast with non-float types
-
-**Symptom**: Garbage values or type-aliasing bugs when vector-loading data of a different base type.
-
-**Cause**: `reinterpret_cast<const float4*>(half_ptr)` will load 16 bytes (4 floats = 128 bits), but if the data is actually `__half` (2 bytes each), those 16 bytes correspond to 8 half-precision values, not 4 floats. The values will be misinterpreted.
-
-**Example**:
-```cuda
-// BAD: half data reinterpreted as float4
-__half* h_ptr = ...;
-float4 v = reinterpret_cast<const float4*>(h_ptr)[tid];  // Wrong types!
-
-// GOOD: use the correct vector width for the data type
-// For __half, use __half2 (loads 2 halfs = 4 bytes) or load via uint4
-// and manually unpack:
-uint4 raw = reinterpret_cast<const uint4*>(h_ptr)[tid];  // 16 bytes = 8 halfs
-__half2* h2 = reinterpret_cast<__half2*>(&raw);  // 4 half2 values
-```
-
-**Fix**: Match the vector type to the underlying data type. For float data, use `float4`. For half-precision, use `__half2` for 2-element loads or load raw bytes via `uint4` and manually unpack. For `int` data, use `int4`. The general rule: the vector type's base must match the data element type.
-
-## P4: Increased register pressure from vector loads
-
-**Symptom**: Kernel occupancy drops after switching to vectorized loads, negating the bandwidth benefit.
-
-**Cause**: Each `float4` variable occupies 4 registers (one per component: `.x`, `.y`, `.z`, `.w`). If a kernel loads multiple `float4` values simultaneously (e.g., two inputs for a fused operation), that is 8 registers just for the loaded data. The best-practices guide (L1062) notes that "there is no register-related reason to pack data into vector data types such as `float4`" -- the register cost is the same as 4 separate float registers. However, if the kernel was already near the register limit, the additional temporary registers needed during the unpack/compute/repack sequence may push it over the threshold.
-
-**Example**: A kernel using `__launch_bounds__(256, 4)` (256 threads/block, 4 blocks/SM, requiring <= 64 registers/thread on H200) may exceed 64 regs when switching from scalar to float4 loads if the compute phase is complex.
-
-**Fix**: Check register usage with `--ptxas-options=-v` after adding vectorized loads. If register count increases significantly:
-- Reduce the vector width from float4 to float2.
-- Use `__launch_bounds__` to constrain registers.
-- Manually schedule the unpack/compute sequence to reduce live registers.
-
-**Detection**: Compile with `nvcc --ptxas-options=-v` and compare register count before and after vectorization.
-
-## P5: Assuming vectorized loads help compute-bound kernels
-
-**Symptom**: No performance improvement (or even slight regression) after adding vectorized loads.
-
-**Cause**: Vectorized loads reduce the number of memory instructions, which only helps when the kernel is memory-instruction-bound or memory-bandwidth- bound. If the kernel is compute-bound (the arithmetic pipeline is the bottleneck, not the memory subsystem), reducing load instructions has no effect on overall throughput.
-
-**Fix**: Before vectorizing, determine the kernel's bottleneck:
-- If `ncu` shows memory throughput close to peak and compute throughput well below peak, the kernel is memory-bound -- vectorize.
-- If compute throughput is close to peak and memory throughput is low, the kernel is compute-bound -- vectorization will not help.
-- Without `ncu`, use a rough estimate: if the kernel does less than ~10 FLOPs per byte loaded, it is likely memory-bound.
-
-## P6: Using float3 / int3 for "vectorized" loads
-
-**Symptom**: No performance improvement or even slower execution when using 3-component vector types.
-
-**Cause**: `float3` has size 12 bytes and alignment 4 bytes (programming guide Table 42, L22695-L22698). Unlike `float4` (16 bytes, alignment 16) and `float2` (8 bytes, alignment 8), `float3` does not map to a single wide load instruction. The compiler typically generates three separate 32-bit loads for a `float3`. The odd size also means that consecutive `float3` values are not naturally aligned for wide loads.
-
-**Fix**: Use `float4` with an ignored `.w` component, or use `float2` plus a scalar load. Never use `float3` / `int3` for performance-critical memory access.
-
-## P7: Forgetting to vectorize the store side
-
-**Symptom**: Kernel shows only partial improvement after vectorizing loads.
-
-**Cause**: Vectorized loads halve or quarter the load instruction count, but if the stores remain scalar, the store side still issues 4x the instructions. In memory-bound kernels where the write bandwidth is significant (e.g., elementwise ops with 1:1 read:write ratio), un-vectorized stores leave performance on the table.
-
-**Fix**: Apply the same `float4` pattern to outputs:
-```cuda
-float4 vc;
-vc.x = va.x + vb.x;
-vc.y = va.y + vb.y;
-vc.z = va.z + vb.z;
-vc.w = va.w + vb.w;
-reinterpret_cast<float4*>(c)[tid] = vc;  // STG.128 -- vectorized store
-```
-
-## P8: Breaking coalescing with strided vector access
-
-**Symptom**: Vectorized loads show no improvement or performance regression compared to scalar loads.
-
-**Cause**: If consecutive threads do not access consecutive `float4` elements -- for example, `((float4*)ptr)[tid * 2]` with stride 2 between threads -- the loads are not coalesced. Each `float4` access is 16 bytes wide, so a stride-2 pattern means the warp touches addresses 32 bytes apart per thread, wasting half the fetched data. Vectorized access must be used together with coalesced access patterns to achieve full benefit.
-
-**Fix**: Ensure the access index is `tid` (linear, stride-1 across threads within a warp). The vectorization-plus-coalescing pattern is: `reinterpret_cast<const float4*>(ptr)[blockIdx.x * blockDim.x + threadIdx.x]`.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/memory/vectorized-access.md

@@ -1,241 +0,0 @@
----
-title: Vectorized Memory Access
-status: draft
-evidence_level: measured
-applies_to_pattern_class:
-- cuda-core
-applies_to_ops:
-- elementwise
-- reduction
-- normalization
-- scan
-requires_sm: '>=6.0'
-requires_features: []
-single_kernel_useful: true
-cuda_version_tested: '12.9'
-driver_version_tested: 570.124.06
-toolchain: nvcc 12.9 + ptxas 12.9
-measured_on: H200-SXM
-source:
-- path: spec
-  anchor: Reference
-artifacts:
-  code: artifacts/experience/hw-probes/vectorized-access/vectorized_load_probe.cu
-  build: nvcc -arch=sm_90a -O3 -std=c++17 -lineinfo -o vectorized_load_probe vectorized_load_probe.cu
-  introspection: ''
-  profile: ''
-related_apis:
-- float4
-- float2
-- int4
-- uint4
-- make_float4
-- make_float2
-- make_int4
-- make_uint4
-related_skills:
-- coalescing
-- bank-conflict
-id: skill-vectorized-access
-type: skill
-vendor: nvidia
-tags:
-- cuda-cpp
-- vectorized-loads
-- cache-policy
-- register-budgeting
-- kernel-fusion
-- tile-scheduling
-- fused-kernel
-applies_to:
-- general
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L22574-L22702
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L22735-L22753
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L991-L1001
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1062
-architectures:
-- sm90
-- sm90a
-languages:
-- cuda-cpp
-techniques:
-- vectorized-loads
-- cache-policy
-- register-budgeting
-- kernel-fusion
-- tile-scheduling
-kernel_types:
-- fused-kernel
-confidence: experimental
-artifact_dir: artifacts/experience/hw-probes/vectorized-access
----
-## What
-
-Vectorized memory access is a technique where each CUDA thread loads or stores multiple elements in a single memory instruction by using CUDA's built-in vector types (`float4`, `float2`, `int4`, `uint4`, `longlong2`, etc.). Instead of each thread issuing a 32-bit (4-byte) scalar load, a thread issuing a `float4` load reads 128 bits (16 bytes) in one instruction. The compiler generates a single wide memory instruction (`LDG.128` for 128-bit loads, `LDG.64` for 64-bit loads) rather than multiple narrow instructions.
-
-The key vector types and their memory widths:
-
-| Type | Size (bytes) | Alignment (bytes) | Load instruction |
-|---|---|---|---|
-| `float` (scalar) | 4 | 4 | `LDG.32` |
-| `float2` | 8 | 8 | `LDG.64` |
-| `float4` | 16 | 16 | `LDG.128` |
-| `int4` / `uint4` | 16 | 16 | `LDG.128` |
-| `longlong2` | 16 | 16 | `LDG.128` |
-| `double2` | 16 | 16 | `LDG.128` |
-
-The alignment requirements are defined in the programming guide (Table 42, L22578): `float4` requires 16-byte alignment, `float2` requires 8-byte alignment. Memory allocated by `cudaMalloc` is guaranteed to be at least 256-byte aligned, so the base pointer always satisfies these requirements.
-
-The canonical usage pattern is to `reinterpret_cast` a `float*` pointer to a `float4*` pointer and load one `float4` per thread:
-
-```cuda
-__global__ void vec_kernel(const float* __restrict__ in,
-                           float*       __restrict__ out,
-                           int n) {
-    int tid = blockIdx.x * blockDim.x + threadIdx.x;
-    if (tid < n / 4) {
-        float4 v = reinterpret_cast<const float4*>(in)[tid];
-        // Process v.x, v.y, v.z, v.w
-        float result = v.x + v.y + v.z + v.w;
-        out[tid] = result;
-    }
-}
-```
-
-## Why
-
-Vectorized access improves memory-bound kernel performance by reducing the number of load/store instructions the SM must issue to move the same amount of data. On H200, our microbenchmark measured a **2.59x effective bandwidth improvement** with float4 loads compared to scalar float loads (see Measured Characteristics below).
-
-The improvement comes from two effects:
-
-1. **Fewer instructions per byte**: a `float4` load moves 16 bytes with one instruction, vs. 4 bytes with one scalar load. This reduces pressure on the SM's instruction scheduler and load/store units. The scalar kernel must issue 4x more load instructions, which can saturate the instruction issue rate before saturating the memory bandwidth.
-
-2. **Higher per-warp transaction efficiency**: a warp of 32 threads each loading a `float4` requests 32 x 16 = 512 bytes, serviced by 16 coalesced 32-byte transactions. While this is more transactions than the scalar case (32 x 4 = 128 bytes, 4 transactions), each transaction is fully utilized. The net effect is that the vectorized kernel saturates the memory subsystem with fewer warps and fewer instructions.
-
-The best-practices guide (L1062) notes that there is "no register-related reason to pack data into vector data types such as `float4` or `int4` types." The benefit is purely on the memory-instruction side: fewer load instructions to move the same data.
-
-The best-practices guide (L991-L1001) also observes that "best performance is achieved when using asynchronous copies with an element of size 8 or 16 bytes," reinforcing that 128-bit-wide memory operations are the highest-throughput path on modern NVIDIA GPUs.
-
-## When to use
-
-- **Elementwise kernels** (activation, scaling, bias-add, fused pointwise): these kernels are typically memory-bound and each thread processes a contiguous span of elements. Replacing scalar loads with `float4` loads is straightforward and yields immediate benefit.
-
-- **Reduction input phase**: the initial load of reduction input from global memory benefits from vectorized access. Each thread loads 4 values at once and accumulates them locally before the warp/block reduction.
-
-- **Normalization kernels** (layer-norm, batch-norm): the per-element read/write phases are memory-bound and benefit from vectorized access.
-
-- **Any kernel where N is divisible by the vector width** (4 for float4, 2 for float2): the element count must divide evenly by the vector width to avoid out-of-bounds reads. If it does not divide evenly, a scalar tail loop is needed (see pitfalls.md).
-
-- **Already-coalesced kernels**: vectorized access is a "next-level" optimization on top of coalescing. A coalesced scalar kernel (stride-1) can often be further improved by switching to vectorized loads.
-
-## When NOT to use
-
-- **N is not a multiple of the vector width and tail handling is complex**: if the data layout makes it difficult to handle the remainder elements cleanly, the code complexity may not justify the benefit. In practice, padding the input to a multiple of 4 is the simplest solution.
-
-- **Gather/scatter with non-contiguous access**: vectorized loads only help when consecutive threads access consecutive memory. If the access pattern is indexed or strided, casting to `float4*` will load the wrong data. The memory must be contiguous per-thread-group.
-
-- **Register pressure is already critical**: each `float4` occupies 4 registers (one per component). If the kernel is already register-bound and close to the occupancy limit, adding vector loads may push it over the register threshold and reduce occupancy, negating the benefit.
-
-- **Data type is not 4-byte aligned or has odd sizes**: `float4` requires 16-byte alignment. If the underlying data type is not a multiple of 4 bytes (e.g., 3-byte packed RGB), vector loads cannot be used directly. A type-punned load via `uint4` may work but requires care.
-
-- **The kernel is compute-bound**: if the kernel's bottleneck is arithmetic throughput rather than memory bandwidth, vectorized loads will not help.
-
-## Classical example: scalar vs vectorized elementwise add
-
-```cuda
-// Scalar version: each thread loads and stores one float
-__global__ void vecadd_scalar(const float* __restrict__ a,
-                              const float* __restrict__ b,
-                              float*       __restrict__ c,
-                              int n) {
-    int tid = blockIdx.x * blockDim.x + threadIdx.x;
-    if (tid < n)
-        c[tid] = a[tid] + b[tid];
-}
-
-// Vectorized version: each thread loads and stores one float4 (4 floats)
-__global__ void vecadd_float4(const float* __restrict__ a,
-                              const float* __restrict__ b,
-                              float*       __restrict__ c,
-                              int n) {
-    int tid = blockIdx.x * blockDim.x + threadIdx.x;
-    if (tid < n / 4) {
-        float4 va = reinterpret_cast<const float4*>(a)[tid];
-        float4 vb = reinterpret_cast<const float4*>(b)[tid];
-        float4 vc;
-        vc.x = va.x + vb.x;
-        vc.y = va.y + vb.y;
-        vc.z = va.z + vb.z;
-        vc.w = va.w + vb.w;
-        reinterpret_cast<float4*>(c)[tid] = vc;
-    }
-}
-```
-
-The vectorized version uses `n/4` threads (launched with `n/4` total threads) instead of `n` threads. Each thread issues one `LDG.128` per input array and one `STG.128` for the output, versus four `LDG.32` and four `STG.32` in the scalar version.
-
-## Tail handling pattern
-
-When `n` is not guaranteed to be a multiple of 4:
-
-```cuda
-__global__ void vecadd_with_tail(const float* __restrict__ a,
-                                 const float* __restrict__ b,
-                                 float*       __restrict__ c,
-                                 int n) {
-    int tid = blockIdx.x * blockDim.x + threadIdx.x;
-    int n4 = n / 4;
-    // Vectorized body
-    if (tid < n4) {
-        float4 va = reinterpret_cast<const float4*>(a)[tid];
-        float4 vb = reinterpret_cast<const float4*>(b)[tid];
-        float4 vc;
-        vc.x = va.x + vb.x;
-        vc.y = va.y + vb.y;
-        vc.z = va.z + vb.z;
-        vc.w = va.w + vb.w;
-        reinterpret_cast<float4*>(c)[tid] = vc;
-    }
-    // Scalar tail for remaining elements
-    int tail_start = n4 * 4;
-    int tail_idx = tail_start + tid;
-    if (tid < (n - tail_start) && tail_idx < n) {
-        c[tail_idx] = a[tail_idx] + b[tail_idx];
-    }
-}
-```
-
-## Measured Characteristics
-
-- vectorized-access bandwidth probe: On H200 (sm_90a, CUDA 12.9), scalar float loads achieved an effective bandwidth of 1462.96 GB/s (median latency 0.1835 ms for 256 MB), while float4 vectorized loads achieved 3783.77 GB/s (median latency 0.0709 ms for the same 256 MB). The vectorized access was **2.59x faster** than scalar access. The working set (256 MB) exceeded the L2 cache (51.2 MB), ensuring HBM-bound measurement. The vectorized kernel reached 77% of theoretical HBM peak bandwidth (4916.7 GB/s), while the scalar kernel reached only 30%.
-
-## Health thresholds
-
-The instruction-count saving from vectorising is a division, not a benchmark
-result: one `LDG.E.128` replaces the four `LDG.E.32` that would fetch the same
-16 bytes.
-
-| Access width | Loads per 16 bytes per lane | Reduction vs scalar |
-|---|---|---|
-| 32-bit (`float`) | 4 | 1x (baseline) |
-| 64-bit (`float2`) | 2 | **2x** |
-| 128-bit (`float4`) | 1 | **4x** |
-
-So "vectorising cuts load instructions by 2-4x" is an upper bound fixed by the
-access width, and it is reached only when the data is naturally aligned to that
-width.
-
-⚠️ **Fewer instructions is not automatically less time.** The saving is real
-when the LSU issue rate is the limit; if the kernel is already bound by DRAM
-bandwidth, the same bytes move either way and the instruction count is not what
-is costing you. Confirm the LSU pipe is near peak before spending alignment
-work on this.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/meta/code-extraction/pitfalls.md

@@ -1,58 +0,0 @@
----
-id: pitfall-code-extraction
-type: pitfall
-vendor: nvidia
-title: Pitfalls
-architectures:
-- '*'
-languages:
-- cuda-cpp
-hardware_features:
-- cluster
-techniques:
-- tma-multicast
-kernel_types:
-- gemm
-- quantization
-confidence: inferred
-evidence_level: inferred
-tags:
-- cluster
-- tma-multicast
-- gemm
-- quantization
-- cuda-cpp
----
-# Code-extraction meta-skill — pitfalls
-
-## 1. Don't conflate "non-aligned to canonical kernel" with "non-aligned to executing kernel"
-
-When extracting a non-aligned-tail-handling skill from cutlass, verify the chosen test shape is non-aligned to the **kernel template that actually executes it**, not just the canonical cooperative kernel. A 64³ datapoint, for example, is a perfect tile match for the small-tile fallback (`<64, 64, 32>`), so the predicate-tail path never fires there even though 64³ is non-aligned to the cooperative `<128, 128, 32>` tile. The lesson generalizes: for any extraction that exercises a fallback / smaller-tile / different-cluster code path, the alignment check must be re-run against the executing kernel's CtaTile, not the canonical one.
-
-(BitLesson reference: `BL-20260428-shape-non-aligned-on-executing-kernel`.)
-
-## 2. Don't conflate scale effects with strategy cost in A/B comparisons
-
-When the "without strategy" anchor is at a different problem size than the "with strategy" target, the throughput delta confounds the strategy cost with absolute-scale effects (wave quantization, occupancy ramp, cluster-multicast efficiency). For compile-time strategies (no runtime toggle), the closest practical A/B is the same kernel binary at adjacent shapes; the extraction must explicitly document this as a proxy with remaining confounders, not a clean isolation.
-
-(BitLesson reference: `BL-20260428-isolate-boundary-from-scale`.)
-
-## 3. Direct same-input numeric comparison vs analytic surrogate
-
-When verifying the extracted skill's correctness, prefer a direct same-A-and-B comparison against a reference implementation (e.g. `reference_gemm` for GEMM) over an analytic surrogate (e.g. "the all-ones input gives `C[0] = K`"). The analytic surrogate misses any per-element bug that happens not to affect `C[0]`. The aligned-GEMM landing in this KB uses `gemm_compare.cu` for exactly this reason.
-
-## 4. Source-corpus backlinks are a hard requirement, not a nice-to-have
-
-Every extracted skill must cite at least one third-party walkthrough in `corpus/nvidia/...` (typically a Colfax tutorial or NVIDIA blog). Citing only the upstream cutlass `*.hpp` sources is insufficient because cutlass headers are dense template-metaprogramming and don't explain the algorithmic intent. The third-party walkthrough is the explanation; the cutlass header is the implementation.
-
-## 5. The canonical artifact-bundle layout is fixed
-
-Every probe lands at `artifacts/experience/<api-probes|hw-probes>/<topic>/{<probe>.cu, helper.h, build.sh, run.sh, device.json, profiles/*.csv}`. Don't deviate. The reference is `artifacts/experience/hw-probes/warp-divergence-cost/`. Reviewers will reject probes with a non-standard layout, even if the actual measurements are correct.
-
-## 6. `evidence_level: measured` requires environment fields
-
-When you set `evidence_level: measured`, the linter will require `cuda_version_tested`, `driver_version_tested`, `toolchain`, and `measured_on` to be non-empty. Don't set `measured` without filling these in; use `evidence_level: spec` for documentation-only landings.
-
-## 7. Meta-skill does not grant verification
-
-This skill describes the procedure for extracting and landing skills. Following the procedure does NOT promote a downstream skill to `status: verified`; that promotion still requires on-hardware verification (canonical-bundle reproduction + numeric-correctness gate + reviewer approval). The meta-skill is the recipe, not the certification.