@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/techniques/persistent-kernels.md

@@ -1,232 +0,0 @@
----
-id: technique-persistent-kernels
-title: Persistent Kernels with CLC
-type: technique
-architectures:
-- sm100
-languages:
-- '*'
-tags:
-- persistent-kernel
-- clc
-- tile-scheduling
-confidence: source-reported
-evidence_level: spec
-reproducibility: snippet
-prerequisites:
-- hw-clc
-related:
-- hw-clc
-- technique-tile-scheduling
-- pattern-tail-effect
-sources:
-- doc-nvidia-tuning-guide
-- blog-tcgen05-tutorial
-- doc-cutlass-blackwell
-artifact_dir: artifacts/kernels/persistent-kernels
----
-
-## Overview
-
-Persistent kernels launch exactly as many CTAs as SMs, and each CTA processes multiple output tiles in a loop rather than exiting after one tile. On Blackwell, the CLC (Cluster Launch Control) hardware unit replaces software-based tile scheduling with a hardware-assisted mechanism. Each CTA queries the CLC for its next tile assignment and can cancel itself when no work remains, using the `try_cancel` pattern.
-
-## CLC Loop Pattern
-
-The core persistent kernel loop on Blackwell uses CLC to dynamically assign tiles:
-
-```cuda
-// Persistent kernel with CLC tile scheduling (Blackwell SM100)
-__global__ void __launch_bounds__(512)
-persistent_gemm_clc(const __grid_constant__ GemmParams params)
-{
-    // CLC-managed persistent loop: each CTA processes multiple tiles
-    while (true) {
-        // Query CLC for next tile assignment
-        // Returns tile coordinates (tile_m, tile_n) or signals termination
-        TileCoord tile;
-        bool has_work = clc_try_get_tile(&tile);
-
-        if (!has_work) {
-            // No more tiles to process -- CTA exits
-            // clc_try_cancel atomically checks if all tiles are done
-            if (clc_try_cancel()) {
-                return;  // CTA terminates
-            }
-            continue;  // Race condition: another CTA may have generated work
-        }
-
-        // Standard GEMM tile computation
-        int tile_m = tile.m;
-        int tile_n = tile.n;
-
-        // TMA producer loads A[tile_m, :] and B[:, tile_n] tiles
-        // MMA consumer accumulates K-dimension
-        // Epilogue writes C[tile_m, tile_n]
-        compute_gemm_tile(params, tile_m, tile_n);
-    }
-}
-```
-
-At the PTX level, the CLC interaction uses dedicated instructions:
-
-```ptx
-// CLC tile acquisition in PTX
-// clusterctl.try_cancel cancels the CTA if no work remains
-.reg .pred  %has_work;
-.reg .b32   %tile_m, %tile_n;
-
-TILE_LOOP:
-    // Attempt to get next tile from CLC hardware scheduler
-    clusterctl.query.async.shared  [%smem_tile_desc];
-    clusterctl.wait;
-
-    // Check if valid tile was assigned
-    ld.shared.b32  %has_work, [%smem_tile_desc + 0];
-    @!%has_work bra TRY_CANCEL;
-
-    // Extract tile coordinates
-    ld.shared.b32  %tile_m, [%smem_tile_desc + 4];
-    ld.shared.b32  %tile_n, [%smem_tile_desc + 8];
-
-    // ... compute tile ...
-
-    bra TILE_LOOP;
-
-TRY_CANCEL:
-    // Atomically try to cancel this CTA
-    clusterctl.try_cancel  %cancelled;
-    @!%cancelled bra TILE_LOOP;   // Another CTA may have pushed work
-    ret;
-```
-
-## Comparison: CLC vs Static Stride (Hopper)
-
-On Hopper (SM90), persistent kernels use a static stride pattern where each CTA computes tiles at fixed intervals:
-
-```cuda
-// Hopper-style static stride persistent kernel
-__global__ void hopper_persistent_gemm(GemmParams params)
-{
-    int cta_id = blockIdx.x;
-    int total_ctas = gridDim.x;
-    int total_tiles = params.num_tiles_m * params.num_tiles_n;
-
-    // Static stride: CTA i handles tiles i, i+total_ctas, i+2*total_ctas, ...
-    for (int tile_idx = cta_id; tile_idx < total_tiles; tile_idx += total_ctas) {
-        int tile_m = tile_idx / params.num_tiles_n;
-        int tile_n = tile_idx % params.num_tiles_n;
-        compute_gemm_tile(params, tile_m, tile_n);
-    }
-}
-```
-
-| Aspect | Hopper Static Stride | Blackwell CLC |
-|--------|---------------------|---------------|
-| Scheduling | Software loop with fixed stride | Hardware CLC unit assigns tiles |
-| Load balancing | Fixed; uneven if tile costs vary | Dynamic; CLC rebalances automatically |
-| Tail effect | Last wave may have partial occupancy | CLC minimizes by giving fast CTAs more tiles |
-| Launch overhead | Grid launch for each new problem | CLC can chain multiple problems |
-| Termination | Implicit when loop ends | Explicit `try_cancel` |
-| L2 locality | Depends on stride pattern | CLC can apply swizzled raster |
-
-## CUTLASS PersistentTileSchedulerSm100
-
-CUTLASS 4.5.0 provides `PersistentTileSchedulerSm100` that wraps the CLC hardware:
-
-```cuda
-// CUTLASS SM100 persistent tile scheduler (simplified)
-template <class TileShape>
-struct PersistentTileSchedulerSm100 {
-
-    // Initialize the CLC with the problem geometry
-    CUTLASS_DEVICE static void init(
-        dim3 problem_tiles,
-        void* clc_smem_buffer)
-    {
-        if (threadIdx.x == 0) {
-            // Program CLC with total tile count and scheduling policy
-            clc_init(clc_smem_buffer,
-                     problem_tiles.x,  // tiles along M
-                     problem_tiles.y,  // tiles along N
-                     ClcPolicy::SwizzledRaster);
-        }
-        __syncthreads();
-    }
-
-    // Shared storage for CTA-wide CLC result broadcast
-    // __shfl_sync is warp-local and cannot reach warps 1-15.
-    struct SharedClcState {
-        int tile_m, tile_n;
-        int valid;       // 1 = got tile, 0 = no more work
-        int cancelled;
-    };
-
-    // Get next tile assignment from CLC
-    CUTLASS_DEVICE static bool get_next_tile(
-        void* clc_smem_buffer,
-        SharedClcState& shared_clc,
-        int& tile_m,
-        int& tile_n)
-    {
-        if (threadIdx.x == 0) {
-            int m, n;
-            bool v = clc_query_tile(clc_smem_buffer, m, n);
-            shared_clc.tile_m = m;
-            shared_clc.tile_n = n;
-            shared_clc.valid  = v ? 1 : 0;
-        }
-        __syncthreads();  // All warps see the result
-        tile_m = shared_clc.tile_m;
-        tile_n = shared_clc.tile_n;
-        return shared_clc.valid != 0;
-    }
-
-    // Try to cancel the CTA when no more work
-    CUTLASS_DEVICE static bool try_cancel(
-        void* clc_smem_buffer,
-        SharedClcState& shared_clc)
-    {
-        if (threadIdx.x == 0) {
-            shared_clc.cancelled = clc_try_cancel(clc_smem_buffer) ? 1 : 0;
-        }
-        __syncthreads();
-        return shared_clc.cancelled != 0;
-    }
-};
-```
-
-## Performance Impact
-
-The tcgen05-tutorial progression demonstrates the impact of persistent kernels:
-
-```
-Without persistence (static grid):  940 TFLOPS  (62% of peak)
-With CLC persistent scheduling:    1476 TFLOPS  (98% of cuBLAS)
-```
-
-The 57% improvement comes from:
-1. **Eliminated tail effect**: CLC dynamically assigns tiles, so fast-completing CTAs absorb extra work rather than sitting idle while the last wave finishes.
-2. **Reduced launch overhead**: A single kernel launch covers all tiles; no need to re-launch grids.
-3. **Better L2 cache utilization**: CLC can apply a swizzled raster pattern that improves spatial locality across neighboring tiles.
-
-## When to Use
-
-- **Large GEMM problems**: Persistent kernels are most beneficial when the number of output tiles exceeds the SM count by at least 2-3x.
-- **Grouped GEMMs / MoE**: CLC can chain multiple problem instances, eliminating inter-kernel launch gaps.
-- **Workloads with uneven tile cost**: CLC's dynamic scheduling naturally handles variable-cost tiles (e.g., triangular attention masks).
-
-## Caveats
-
-- CLC is SM100-only; Hopper kernels must use software-based scheduling.
-- The `try_cancel` pattern introduces a potential race that must be handled with a retry loop.
-- For very small problems (fewer tiles than SMs), CLC overhead may not justify the complexity. A simple single-wave grid launch suffices.
-
-## Full Reference Implementation
-
-Verbatim upstream code lives in [`artifacts/kernels/persistent-kernels/full/`](../../../artifacts/kernels/persistent-kernels/full/); labeled derived variants (each with the required `// provenance: derived from ...; not upstream code` header) live in [`artifacts/kernels/persistent-kernels/variants/`](../../../artifacts/kernels/persistent-kernels/variants/). Every file's SHA-256 and upstream-pinning metadata is in `PROVENANCE.yaml` inside each bundle.
-
-Query via:
-
-```bash
-python3 scripts/get_page.py technique-persistent-kernels --include-code
-```

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/techniques/ping-pong-scheduling.md

@@ -1,87 +0,0 @@
----
-id: technique-ping-pong-scheduling
-title: Ping-Pong Scheduling
-type: technique
-architectures:
-- sm100
-languages:
-- '*'
-tags:
-- ping-pong-scheduling
-- warp-specialization
-- tmem
-- pipeline-stages
-confidence: source-reported
-evidence_level: spec
-reproducibility: snippet
-prerequisites:
-- hw-tmem
-- technique-warp-specialization
-related:
-- kernel-flash-attention-4
-- technique-double-buffering
-sources:
-- blog-flash-attention-4
-- doc-flash-attention-4
-- blog-tcgen05-tutorial
-artifact_dir: artifacts/kernels/ping-pong-scheduling
----
-
-# Ping-Pong Scheduling
-
-## Overview
-
-Ping-pong scheduling alternates two query tiles within a single CTA so the softmax warpgroup never stalls waiting for MMA. Introduced in FlashAttention-4 to exploit Blackwell's asymmetric hardware (2× tensor cores, same SFU count as Hopper).
-
-## Pattern
-
-```cuda
-// Two 128-token query tiles per CTA, alternating through the mainloop
-// Warpgroup 0: softmax for tile A while MMA runs on tile B
-// Warpgroup 1: softmax for tile B while MMA runs on tile A
-
-__global__ void fa4_ping_pong_attn(...) {
-    int wg = warp_group_id();
-
-    // TMEM holds accumulators for BOTH tiles
-    uint32_t tmem_A = tmem_alloc(256);
-    uint32_t tmem_B = tmem_alloc(256);
-
-    for (int k = 0; k < num_kv_tiles; k++) {
-        if (wg == 0) {
-            // Compute softmax for tile A (previous MMA output)
-            softmax_normalize(tmem_A);
-            // Issue MMA for tile B next
-            tcgen05_mma(Q_B_smem, K_smem[k], tmem_B);
-        } else {
-            softmax_normalize(tmem_B);
-            tcgen05_mma(Q_A_smem, K_smem[k], tmem_A);
-        }
-        mbarrier_arrive(&ping_pong_sync);
-        mbarrier_wait(&ping_pong_sync);
-    }
-}
-```
-
-## Why It Helps on Blackwell
-
-- Tensor core throughput doubled (B200 vs H100) but SFU count unchanged
-- Single-tile schedule would leave SFU idle while MMA runs, and vice versa
-- Ping-pong keeps both units 100% busy
-- FA4 achieves 1605 TFLOPS BF16 (71% utilization) with this pattern
-
-## When To Use
-
-- Compute-bound attention kernels on Blackwell
-- Kernels where softmax/epilogue is SFU-heavy
-- Not useful on Hopper (balance is different)
-
-## Full Reference Implementation
-
-Verbatim upstream code lives in [`artifacts/kernels/ping-pong-scheduling/full/`](../../../artifacts/kernels/ping-pong-scheduling/full/); labeled derived variants (each with the required `// provenance: derived from ...; not upstream code` header) live in [`artifacts/kernels/ping-pong-scheduling/variants/`](../../../artifacts/kernels/ping-pong-scheduling/variants/). Every file's SHA-256 and upstream-pinning metadata is in `PROVENANCE.yaml` inside each bundle.
-
-Query via:
-
-```bash
-python3 scripts/get_page.py technique-ping-pong-scheduling --include-code
-```

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/techniques/pipeline-stages.md

@@ -1,223 +0,0 @@
----
-id: technique-pipeline-stages
-title: Software Pipelining and Multi-Stage Buffering
-type: technique
-architectures:
-- sm100
-- sm90
-languages:
-- '*'
-tags:
-- pipeline-stages
-- double-buffering
-- tma
-- mbarrier
-confidence: source-reported
-evidence_level: spec
-reproducibility: snippet
-prerequisites:
-- hw-tma
-- hw-tmem
-related:
-- technique-warp-specialization
-- technique-double-buffering
-- hw-tma
-sources:
-- blog-tcgen05-tutorial
-- blog-modular-blackwell
-- doc-nvidia-tuning-guide
----
-## Overview
-
-Software pipelining overlaps data loading (TMA copies from global to shared memory) with computation (tcgen05.mma or wgmma) by maintaining multiple in-flight tile buffers. A circular buffer of 3-5 stages allows the TMA producer to fill stage N+2 while the MMA consumer processes stage N, hiding the global memory latency entirely. This technique is critical for achieving high utilization on both Hopper and Blackwell.
-
-## Pipeline Progression
-
-The tcgen05-tutorial demonstrates the performance impact of pipelining:
-
-```
-No pipelining (load, then compute):    695 TFLOPS  (46%)
-3-stage pipeline (TMA + MMA overlap):  940 TFLOPS  (62%)
-+ warp specialization:                1476 TFLOPS  (98%)
-```
-
-The 35% improvement from pipelining alone (695 to 940 TFLOPS) comes from hiding global memory latency behind MMA computation.
-
-## Multi-Stage Circular Buffer Pattern
-
-The fundamental pattern allocates `NUM_STAGES` copies of each SMEM buffer and cycles through them:
-
-```cuda
-// 3-stage circular buffer with mbarrier synchronization
-// Stages: [0] loading, [1] ready for MMA, [2] being consumed by MMA
-#define NUM_STAGES 3
-
-__global__ void __launch_bounds__(512)
-pipelined_gemm(const __grid_constant__ GemmParams params)
-{
-    extern __shared__ char smem[];
-
-    // Circular buffer layout in shared memory
-    // Each stage has its own A and B tile buffers
-    half* smem_A[NUM_STAGES];
-    half* smem_B[NUM_STAGES];
-    for (int s = 0; s < NUM_STAGES; s++) {
-        smem_A[s] = reinterpret_cast<half*>(
-            smem + s * (TILE_A_BYTES + TILE_B_BYTES));
-        smem_B[s] = reinterpret_cast<half*>(
-            smem + s * (TILE_A_BYTES + TILE_B_BYTES) + TILE_A_BYTES);
-    }
-
-    // mbarrier arrays for producer-consumer sync
-    __shared__ uint64_t mbar_load_complete[NUM_STAGES];
-    __shared__ uint64_t mbar_mma_complete[NUM_STAGES];
-
-    int warp_id = threadIdx.x / 32;
-    int lane_id = threadIdx.x % 32;
-
-    // Initialize barriers
-    if (threadIdx.x == 0) {
-        for (int s = 0; s < NUM_STAGES; s++) {
-            mbarrier_init(&mbar_load_complete[s], 1);
-            mbarrier_init(&mbar_mma_complete[s], 1);
-        }
-    }
-    __syncthreads();
-
-    int num_k_tiles = params.K / TILE_K;
-
-    if (warp_id == 0) {
-        // ===== TMA PRODUCER =====
-        // Prologue: fill the first NUM_STAGES buffers
-        for (int s = 0; s < NUM_STAGES && s < num_k_tiles; s++) {
-            if (lane_id == 0) {
-                // Set expected TX bytes on mbarrier BEFORE issuing TMA.
-                // TMA hardware will arrive on the mbarrier when transfer completes.
-                uint32_t tx_bytes = TILE_A_BYTES + TILE_B_BYTES;
-                mbarrier_arrive_expect_tx(&mbar_load_complete[s], tx_bytes);
-                tma_load_tile_A(smem_A[s], params, s, &mbar_load_complete[s]);
-                tma_load_tile_B(smem_B[s], params, s, &mbar_load_complete[s]);
-                // NOTE: Do NOT manually arrive after TMA issue — the TMA
-                // hardware signals the mbarrier upon transfer completion.
-            }
-        }
-
-        // Steady state: load stage s while MMA processes stage s-NUM_STAGES
-        for (int k = NUM_STAGES; k < num_k_tiles; k++) {
-            int stage = k % NUM_STAGES;
-            // Wait for MMA to finish with this buffer
-            mbarrier_wait(&mbar_mma_complete[stage]);
-            if (lane_id == 0) {
-                uint32_t tx_bytes = TILE_A_BYTES + TILE_B_BYTES;
-                mbarrier_arrive_expect_tx(&mbar_load_complete[stage], tx_bytes);
-                tma_load_tile_A(smem_A[stage], params, k, &mbar_load_complete[stage]);
-                tma_load_tile_B(smem_B[stage], params, k, &mbar_load_complete[stage]);
-            }
-        }
-
-    } else if (warp_id == 1) {
-        // ===== MMA CONSUMER =====
-        for (int k = 0; k < num_k_tiles; k++) {
-            int stage = k % NUM_STAGES;
-            // Wait for TMA to fill this buffer
-            mbarrier_wait(&mbar_load_complete[stage]);
-
-            // Issue MMA on the filled buffer
-            if (lane_id == 0) {
-                tcgen05_mma(smem_A[stage], smem_B[stage]);
-            }
-            __syncwarp();
-
-            // Signal that this buffer is free for reuse
-            if (lane_id == 0) {
-                mbarrier_arrive(&mbar_mma_complete[stage]);
-            }
-        }
-    }
-    // Epilogue warps omitted for clarity
-}
-```
-
-## Stage Count Selection
-
-The optimal number of pipeline stages depends on the ratio of memory latency to compute time per tile:
-
-| Stages | SMEM Usage | Latency Hiding | Best For |
-|--------|-----------|----------------|----------|
-| 2 | 2x base | Partial | Small tiles, limited SMEM |
-| 3 | 3x base | Full for most GEMMs | Standard choice on Blackwell |
-| 4-5 | 4-5x base | Full with margin | Large K, high memory latency |
-| >5 | Excessive | Diminishing returns | Rarely justified |
-
-The SMEM budget on Blackwell is 228 KB per SM. For a BF16 GEMM with TILE_M=128, TILE_N=256, TILE_K=64:
-- A tile: 128 x 64 x 2B = 16 KB
-- B tile: 64 x 256 x 2B = 32 KB
-- Per stage: 48 KB
-- 3 stages: 144 KB (63% of SMEM, leaves room for barriers and epilogue)
-- 5 stages: 240 KB (exceeds SMEM capacity; must use smaller tiles)
-
-## mbarrier-Based Synchronization
-
-The mbarrier (memory barrier) is the hardware primitive that makes pipelining efficient. Unlike `__syncthreads()`, mbarrier supports asymmetric producer-consumer synchronization where only the relevant warp participates:
-
-```ptx
-// Phase-based mbarrier protocol for 3-stage pipeline
-//
-// Each mbarrier tracks a "phase" (0 or 1). The producer flips the phase
-// on arrive; the consumer waits for the expected phase.
-
-// Producer: arrive on stage %s (flips phase)
-mbarrier.arrive.shared.b64  %state, [%mbar_load + %s * 8];
-
-// Consumer: wait for phase %expected_phase on stage %s
-// try_wait is non-blocking; the warp can spin-wait or do other work
-WAIT_LOOP:
-    mbarrier.try_wait.parity.shared.b64  %ready, [%mbar_load + %s * 8], %phase;
-    @!%ready bra WAIT_LOOP;
-
-// TMA can also arrive directly on an mbarrier:
-// The TMA unit signals completion without CPU thread involvement
-cp.async.bulk.tensor.2d.shared::cluster.global.mbarrier::complete_tx::bytes
-    [%smem_addr], [%tensor_map, {%coord0, %coord1}], [%mbar_load + %s * 8];
-```
-
-The key advantage is that TMA can arrive on an mbarrier autonomously. The producer warp only needs to initiate the TMA; the TMA hardware signals completion directly, removing the producer from the critical path.
-
-## Modular's 5-Stage Implementation
-
-The Modular blog series describes a 5-stage circular buffer reaching 85% of SOTA performance on Blackwell:
-
-```cuda
-// Modular-style 5-stage pipeline constants
-// Chosen to fully hide B200 HBM latency (~400 cycles)
-// while fitting within 228 KB SMEM budget
-
-constexpr int NUM_STAGES = 5;
-constexpr int TILE_M = 128;
-constexpr int TILE_N = 128;  // Smaller N to fit 5 stages
-constexpr int TILE_K = 64;
-
-// Per stage: A (128*64*2=16KB) + B (64*128*2=16KB) = 32 KB
-// 5 stages: 160 KB + barriers + metadata < 228 KB
-
-// The pipeline timing diagram for steady state:
-//
-// Stage:  0         1         2         3         4
-// TMA:   [load k5] [done]    [done]    [load k8] [load k9]
-// MMA:   [done]    [done]    [comp k7] [done]    [done]
-//
-// At any point: 1 stage being loaded, 1 being computed, 3 in transit or done
-```
-
-## When to Use
-
-- **All memory-bound and compute-bound GEMM kernels**: Pipelining is never harmful and always improves utilization by hiding latency.
-- **Attention kernels**: The K-dimension loop in attention benefits from pipelining the KV tile loads.
-- **Combined with warp specialization**: Pipelining provides the buffer structure; warp specialization assigns the producer/consumer roles. The two techniques are complementary and almost always used together.
-
-## Caveats
-
-- More stages increase SMEM usage linearly. On Blackwell's fixed 228 KB, this constrains tile size choices.
-- Barrier initialization overhead is negligible but must happen before the first TMA. Place init in a `__syncthreads()` block at kernel start.
-- Incorrect phase tracking in mbarrier causes deadlocks. The phase alternates with each arrive/wait cycle; off-by-one errors are common during development.
-- For very short K dimensions (fewer iterations than stages), the prologue/epilogue overhead may dominate. Guard the loop bounds accordingly.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/techniques/register-budgeting.md

@@ -1,72 +0,0 @@
----
-id: technique-register-budgeting
-title: Register Budgeting for Occupancy
-type: technique
-architectures:
-- sm100
-- sm90
-languages:
-- '*'
-tags:
-- register-budgeting
-- register-reuse
-confidence: source-reported
-evidence_level: spec
-reproducibility: snippet
-prerequisites: []
-related:
-- pattern-memory-bound
-- pattern-register-pressure
-- kernel-nvfp4-gemv
-sources:
-- blog-yue-nvfp4
-- blog-amandeep-nvfp4
-- blog-simon-nvfp4-gemv
----
-# Register Budgeting
-
-## Overview
-
-SM occupancy is inversely proportional to registers-per-thread. For memory-bound kernels, higher occupancy = more warps to hide memory latency. `-maxrregcount` and `__launch_bounds__` force the compiler to stay within a budget.
-
-## Pattern
-
-```cuda
-// Aggressive: 32 registers/thread → ~4 blocks per SM at 256 threads/block
-__launch_bounds__(256, 4)
-__global__ void gemv_memory_bound(...) {
-    // Compiler will spill to local memory if needed
-}
-
-// Or via nvcc flag:
-// nvcc -maxrregcount=32 -arch=sm_100a ...
-```
-
-## Compiler Tradeoffs
-
-Lower register count → compiler may:
-- Spill frequently-used values to local memory (bad)
-- Recompute values instead of storing them (neutral)
-- Use fewer unrolled iterations (bad for compute-bound)
-
-For memory-bound kernels, spills can be hidden by memory latency anyway, so aggressive budgeting often wins.
-
-## GPU Mode NVFP4 GEMV Results
-
-| Rank | Register count | Latency |
-|------|---------------|---------|
-| 1 | 32 | 18.5μs |
-| 3 | 45 | ~20μs |
-
-The measurable difference between 32 and 45 registers shows occupancy dominates for memory-bound NVFP4 GEMV.
-
-## When To Use
-
-- Memory-bound kernels (first priority: occupancy)
-- Kernels where register pressure comes from inner loop, not accumulators (TMEM handles accumulators)
-- Sub-byte types with heavy decode/scale computation
-
-## When NOT To Use
-
-- Compute-bound GEMM (let compiler use what it needs)
-- Kernels where spills to local memory would serialize