@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/techniques/software-exp.md

@@ -1,217 +0,0 @@
----
-id: technique-software-exp
-title: "Software-Emulated Exponential"
-type: technique
-architectures: [sm100]
-languages:
-- '*'
-tags: [software-exp, attention]
-confidence: source-reported
-evidence_level: spec
-reproducibility: snippet
-prerequisites: []
-related: [kernel-flash-attention-4, technique-warp-specialization]
-sources: [blog-flash-attention-4, doc-flash-attention-4, doc-ptx-isa-sm100]
----
-
-## Overview
-
-FlashAttention-4 replaces the hardware Special Function Unit (SFU) exponential (`ex2.approx`) with a software-emulated 2^x function that distributes computation across the SM's FMA (fused multiply-add) units. On Blackwell, tensor core throughput doubled compared to Hopper while SFU count remained the same, making the SFU the throughput bottleneck for attention's softmax operation. The software exponential uses Cody-Waite range reduction followed by a Horner-form polynomial evaluation, achieving sufficient accuracy for attention while bypassing the SFU entirely.
-
-## Why SFU Is a Bottleneck on Blackwell
-
-The softmax in attention requires computing `exp(x - max)` for every element of the score matrix. On previous generations, the SFU's `ex2.approx` instruction was fast enough relative to the MMA throughput. On Blackwell:
-
-| Resource | Hopper (SM90) | Blackwell (SM100) | Ratio |
-|----------|--------------|-------------------|-------|
-| Tensor core TFLOPS (BF16) | ~990 | ~2250 | 2.27x |
-| SFU units per SM | 16 | 16 | 1.0x |
-| SFU throughput (exp per cycle) | 16 | 16 | 1.0x |
-| FMA units per SM | 128 | 128 | 1.0x |
-
-The tensor cores produce 2x more score elements per cycle, but the SFU can only process exp() at the same rate as before. This makes the SFU the bottleneck for any kernel that needs exp() proportional to the number of MMA outputs.
-
-FlashAttention-4's approach: distribute the exp() workload across FMA units (128 per SM) instead of SFU units (16 per SM), achieving 8x the throughput for the exponential computation.
-
-## Cody-Waite Range Reduction
-
-Range reduction transforms the input `x` into a small residual that a polynomial can accurately approximate. The Cody-Waite method splits the input into an integer part (for exact power-of-two scaling) and a fractional part (for polynomial approximation):
-
-```cuda
-// Cody-Waite range reduction for 2^x
-// Goal: decompose x = n + r where n is integer, r in [-0.5, 0.5]
-// Then 2^x = 2^n * 2^r, and 2^r is approximated by polynomial
-//
-// The Cody-Waite trick: subtract n using two constants (C1 + C2)
-// to maintain precision when x is large.
-//
-// C1 is the nearest representable float to log2(e) with low-order bits zeroed
-// C2 is the correction: log2(e) - C1
-// This avoids catastrophic cancellation in x - n
-
-__device__ float software_exp2(float x) {
-    // Step 1: Range reduction (Cody-Waite)
-    // Round x to nearest integer
-    float n = rintf(x);
-    // High-precision subtraction using two constants
-    // C1 and C2 together represent 1.0 in extended precision
-    const float C1 = 1.0f;    // Exact in float
-    const float C2 = 0.0f;    // Correction term (zero for 2^x, non-zero for e^x)
-    // For 2^x, range reduction is simpler: r = x - n
-    float r = x - n;          // r in [-0.5, 0.5]
-
-    // Step 2: Polynomial approximation of 2^r via Horner's method
-    // Minimax polynomial coefficients for 2^r on [-0.5, 0.5]
-    // Degree-4 polynomial: sufficient for ~22 bits of accuracy
-    const float c0 = 1.0f;
-    const float c1 = 0.6931471805599453f;   // ln(2)
-    const float c2 = 0.2402265069591007f;   // ln(2)^2 / 2
-    const float c3 = 0.05550410866482158f;  // ln(2)^3 / 6
-    const float c4 = 0.009618129107628477f; // ln(2)^4 / 24
-
-    // Horner evaluation: c0 + r*(c1 + r*(c2 + r*(c3 + r*c4)))
-    // Each step is one FMA instruction
-    float poly = c4;
-    poly = fmaf(poly, r, c3);   // FMA 1
-    poly = fmaf(poly, r, c2);   // FMA 2
-    poly = fmaf(poly, r, c1);   // FMA 3
-    poly = fmaf(poly, r, c0);   // FMA 4
-
-    // Step 3: Reconstruct 2^x = 2^n * poly
-    // Use integer addition to the float exponent field
-    int n_int = (int)n;
-    // ldexpf multiplies by 2^n by adjusting the exponent bits
-    float result = ldexpf(poly, n_int);
-
-    return result;
-}
-```
-
-## Distributing Across FMA Units
-
-The key insight is that Horner polynomial evaluation is a chain of FMA operations. With the softmax warp executing on CUDA cores while the MMA warp uses tensor cores, the FMA throughput is fully available:
-
-```cuda
-// FlashAttention-4 softmax with software exp2
-// Executed by dedicated softmax warpgroups (part of warp specialization)
-//
-// For each row of the score matrix S[i,:]:
-//   1. Find row max: m_new = max(S[i,:])
-//   2. Compute exp2((S[i,j] - m_new) * log2(e)) for each j
-//   3. Sum for normalization denominator
-//   4. Conditionally rescale previous output if max changed
-
-__device__ void softmax_with_software_exp(
-    float* scores,       // Input: S[i, 0..N-1] (one row)
-    float* output,       // Output: softmax(S[i,:])
-    int N,
-    float* row_max,      // Running max (for online softmax)
-    float* row_sum)      // Running sum
-{
-    int lane = threadIdx.x % 32;
-
-    // Step 1: Find max across the row (warp reduction)
-    float local_max = -INFINITY;
-    for (int j = lane; j < N; j += 32) {
-        local_max = fmaxf(local_max, scores[j]);
-    }
-    // Warp-level max reduction
-    for (int offset = 16; offset > 0; offset >>= 1) {
-        local_max = fmaxf(local_max, __shfl_xor_sync(0xFFFFFFFF, local_max, offset));
-    }
-    float m_new = local_max;
-
-    // Step 2: Compute software exp2 and sum
-    float local_sum = 0.0f;
-    const float LOG2E = 1.4426950408889634f;
-
-    for (int j = lane; j < N; j += 32) {
-        float x = (scores[j] - m_new) * LOG2E;
-        float exp_val = software_exp2(x);  // 4 FMAs instead of 1 SFU op
-        output[j] = exp_val;
-        local_sum += exp_val;
-    }
-
-    // Warp-level sum reduction
-    for (int offset = 16; offset > 0; offset >>= 1) {
-        local_sum += __shfl_xor_sync(0xFFFFFFFF, local_sum, offset);
-    }
-
-    // Step 3: Conditional rescaling (FlashAttention online softmax)
-    float m_old = *row_max;
-    if (m_new > m_old) {
-        // Rescale previous accumulated output
-        float scale = software_exp2((m_old - m_new) * LOG2E);
-        *row_sum = (*row_sum) * scale + local_sum;
-        *row_max = m_new;
-        // The output accumulator must also be rescaled by `scale`
-    } else {
-        float scale = software_exp2((m_new - m_old) * LOG2E);
-        *row_sum += local_sum * scale;
-        // Rescale current exp values, not the accumulator
-    }
-}
-```
-
-## PTX-Level FMA Chain
-
-At the PTX level, the Horner polynomial compiles to a tight chain of `fma.rn.f32` instructions:
-
-```ptx
-// Software exp2 via Horner polynomial in PTX
-// Input: %x (float, range-reduced to [-0.5, 0.5])
-// Output: %result (float, approximation of 2^x)
-
-.reg .f32 %x, %r, %n, %poly, %result;
-.reg .f32 %c0, %c1, %c2, %c3, %c4;
-
-// Load polynomial coefficients
-mov.f32 %c0, 0f3F800000;    // 1.0
-mov.f32 %c1, 0f3F317218;    // 0.6931471805599453  (ln2)
-mov.f32 %c2, 0f3E75FDF0;    // 0.2402265069591007
-mov.f32 %c3, 0f3D635847;    // 0.05550410866482158
-mov.f32 %c4, 0f3C1D9539;    // 0.009618129107628477
-
-// Range reduction: n = rintf(x), r = x - n
-cvt.rni.f32.f32 %n, %x;     // Round to nearest int
-sub.f32         %r, %x, %n; // Fractional part
-
-// Horner evaluation: 4 dependent FMAs
-//   poly = c4
-//   poly = poly * r + c3
-//   poly = poly * r + c2
-//   poly = poly * r + c1
-//   poly = poly * r + c0
-mov.f32         %poly, %c4;
-fma.rn.f32      %poly, %poly, %r, %c3;  // FMA 1
-fma.rn.f32      %poly, %poly, %r, %c2;  // FMA 2
-fma.rn.f32      %poly, %poly, %r, %c1;  // FMA 3
-fma.rn.f32      %poly, %poly, %r, %c0;  // FMA 4
-
-// Reconstruct: result = poly * 2^n
-// Convert n to int and use ex2 scaling via bit manipulation
-cvt.rzi.s32.f32 %ni, %n;
-ex2.approx.f32  %scale, %n;    // Or use integer exponent manipulation
-mul.f32         %result, %poly, %scale;
-```
-
-## Accuracy Considerations
-
-The degree-4 polynomial provides approximately 22 bits of mantissa accuracy, which is more than sufficient for attention softmax where:
-- The input `x = (S[i,j] - max) * log2(e)` is always non-positive
-- The softmax output is normalized, so small absolute errors cancel out
-- BF16 output has only 7 mantissa bits anyway
-
-For applications requiring higher accuracy, a degree-6 polynomial (6 FMAs) achieves near-ULP accuracy across the full float range.
-
-## When to Use
-
-- **Attention kernels on Blackwell**: Whenever the SFU is the bottleneck for softmax computation. FlashAttention-4 measured 1.1-1.3x speedup over cuDNN from this technique alone on B200.
-- **Any kernel limited by transcendental function throughput**: If profiling shows SFU utilization near 100% while FMA utilization is low, software emulation can rebalance the workload.
-- **Not recommended on Hopper**: The SFU-to-MMA throughput ratio is better balanced on SM90. The overhead of 4 FMAs vs 1 SFU instruction is not justified unless the SFU is proven to be the bottleneck.
-
-## Caveats
-
-- The 4-FMA chain has a latency of ~16 cycles (4 dependent FMAs at ~4 cycles each), vs ~20 cycles for SFU `ex2.approx`. Latency is comparable; the win comes from throughput (128 FMA units vs 16 SFU units).
-- Polynomial coefficients are for 2^x on [-0.5, 0.5]. For e^x, multiply the input by log2(e) first.
-- The `ldexpf` or exponent bit-manipulation step for 2^n must handle overflow/underflow (very large/small x). In attention, `x <= 0` always holds, so only underflow toward zero is possible.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/techniques/swizzling.md

@@ -1,179 +0,0 @@
----
-id: technique-swizzling
-title: Shared Memory Swizzling
-type: technique
-architectures:
-- sm100
-- sm90
-languages:
-- '*'
-tags:
-- swizzling
-- shared-memory-optimization
-- tma
-confidence: source-reported
-evidence_level: spec
-reproducibility: snippet
-prerequisites:
-- hw-tma
-related:
-- hw-tma
-- technique-pipeline-stages
-- pattern-memory-bound
-sources:
-- doc-nvidia-tuning-guide
-- blog-tcgen05-tutorial
-- blog-modular-blackwell
----
-## Overview
-
-Shared memory swizzling remaps the linear address layout of a matrix tile in SMEM so that threads accessing consecutive columns (or rows) hit different 32-byte banks rather than the same bank. This eliminates bank conflicts that would otherwise serialize concurrent accesses. On Blackwell (SM100), 128-byte swizzling is mandatory for TMA loads and tcgen05.mma operands. Without it, performance drops to 46% of the achievable throughput for GEMM workloads.
-
-## Why 128-Byte Swizzling is Mandatory on Blackwell
-
-Shared memory has 32 banks, each 4 bytes wide (128 bytes total per bank cycle). When a warp accesses a matrix stored in row-major layout, threads in the same warp reading elements from consecutive rows in the same column hit the same bank, causing a 32-way bank conflict.
-
-The TMA unit on both Hopper and Blackwell encodes the swizzle pattern as part of the tensor descriptor. The tcgen05.mma instruction expects its SMEM operands to already be swizzled in the 128-byte pattern. Using unswizzled data produces incorrect MMA results.
-
-The tcgen05-tutorial benchmark progression shows the impact:
-
-```
-Naive (no swizzle):   255 TFLOPS  (17% of cuBLAS)
-128B swizzle applied: 695 TFLOPS  (46% of cuBLAS)
-                      ---- 2.7x improvement from swizzling alone ----
-```
-
-## How 128-Byte Swizzling Works
-
-The swizzle function XORs a portion of the column address with the row address to scatter accesses across banks:
-
-```cuda
-// 128-byte swizzle: XOR bits [4:6] of the byte offset with the row index
-// This ensures that consecutive rows accessing the same logical column
-// map to different physical SMEM banks.
-//
-// For a tile stored in SMEM with TILE_N columns of 2-byte elements:
-//   byte_offset = row * (TILE_N * sizeof(half)) + col * sizeof(half)
-//   swizzled_offset = byte_offset ^ ((row & 0x7) << 4)
-//
-// The mask 0x7 = 3 bits, shift 4 = bits [4:6], giving 8-row periodicity
-// across the 128-byte bank group.
-
-__device__ int swizzle_128B(int row, int col, int stride_bytes) {
-    int byte_offset = row * stride_bytes + col * sizeof(half);
-    // XOR bits [4:6] of byte offset with low 3 bits of row
-    int swizzled = byte_offset ^ ((row & 0x7) << 4);
-    return swizzled;
-}
-```
-
-Visually, for an 8-row x 64-column half-precision tile (128 bytes per row):
-
-```
-Without swizzle (row-major):
-  Row 0: bank 0,1,2,...,31  bank 0,1,2,...,31
-  Row 1: bank 0,1,2,...,31  bank 0,1,2,...,31
-  Row 2: bank 0,1,2,...,31  bank 0,1,2,...,31
-  -> Column access = 8-way bank conflict
-
-With 128B swizzle (XOR pattern):
-  Row 0: bank 0,1,2,...,31  bank 0,1,2,...,31
-  Row 1: bank 1,2,3,...,0   bank 1,2,3,...,0    (rotated by 1)
-  Row 2: bank 2,3,4,...,1   bank 2,3,4,...,1    (rotated by 2)
-  -> Column access = conflict-free
-```
-
-## TMA Swizzle Encoding
-
-The TMA descriptor encodes the swizzle mode when creating a tensor map. The swizzle mode must match what the consumer (tcgen05.mma or wgmma) expects:
-
-```cuda
-// Creating a TMA descriptor with 128-byte swizzle
-#include <cuda.h>
-
-CUtensorMap tensor_map;
-
-// Swizzle mode: CU_TENSOR_MAP_SWIZZLE_128B
-// This tells TMA to apply the 128-byte XOR swizzle pattern
-// when writing data into shared memory
-cuTensorMapEncodeTiled(
-    &tensor_map,
-    CU_TENSOR_MAP_DATA_TYPE_FLOAT16,
-    2,                                    // 2D tensor
-    global_ptr,                           // global memory base
-    global_dims,                          // {N, M} dimensions
-    global_strides,                       // {N * sizeof(half), sizeof(half)}
-    tile_dims,                            // {TILE_N, TILE_M}
-    element_strides,                      // {1, 1}
-    CU_TENSOR_MAP_INTERLEAVE_NONE,
-    CU_TENSOR_MAP_SWIZZLE_128B,           // 128-byte swizzle
-    CU_TENSOR_MAP_L2_PROMOTION_NONE,
-    CU_TENSOR_MAP_FLOAT_OOB_FILL_NONE
-);
-```
-
-The available swizzle modes and their use cases:
-
-| Swizzle Mode | Bank Spread | Use Case |
-|-------------|-------------|----------|
-| `SWIZZLE_NONE` | No remapping | Non-MMA data (flags, scales) |
-| `SWIZZLE_32B` | 32-byte groups | Narrow tiles, small data types |
-| `SWIZZLE_64B` | 64-byte groups | Medium tiles |
-| `SWIZZLE_128B` | 128-byte groups | Standard for BF16/FP16 MMA operands |
-
-## CuTe Swizzle Layout
-
-In CuTe/CUTLASS, swizzle is expressed as a layout composition:
-
-```cuda
-// CuTe swizzle layout for 128-byte swizzle pattern
-// Swizzle<B, M, S> where:
-//   B = number of bits in the base (non-swizzled) portion
-//   M = number of bits in the mask
-//   S = shift amount
-//
-// Swizzle<3, 4, 3> encodes the 128B swizzle:
-//   3 base bits (8-byte alignment)
-//   4 mask bits (16 rows)
-//   3 shift bits (8-column groups)
-
-using SmemLayoutAtom = decltype(
-    composition(
-        Swizzle<3, 4, 3>{},
-        Layout<Shape<_8, _64>,
-               Stride<_64, _1>>{}
-    )
-);
-
-// Tile the atom across the full SMEM tile
-using SmemLayoutA = decltype(
-    tile_to_shape(SmemLayoutAtom{}, Shape<Int<TILE_M>, Int<TILE_K>>{})
-);
-```
-
-## Verification: Detecting Bank Conflicts
-
-Use `nvprof` or Nsight Compute to verify that swizzling eliminates conflicts:
-
-```python
-# Nsight Compute command to check shared memory bank conflicts
-# Look for "Shared Memory Bank Conflicts" metric
-# ncu --metrics l1tex__data_bank_conflicts_pipe_lsu_mem_shared_op_ld.sum \
-#     ./my_kernel
-
-# Expected results:
-# Without swizzle: bank_conflicts >> 0
-# With 128B swizzle: bank_conflicts == 0
-```
-
-## When to Use
-
-- **All Blackwell tensor core kernels**: 128-byte swizzling is not optional. Both TMA and tcgen05.mma require it for correct results and peak performance.
-- **Hopper wgmma kernels**: Same requirement applies; wgmma expects swizzled SMEM operands.
-- **Non-MMA shared memory access**: If multiple warps access the same SMEM tile in a column pattern (e.g., reduction), swizzling prevents serialization.
-
-## Caveats
-
-- The swizzle mode in the TMA descriptor must exactly match the access pattern of the consumer. A mismatch produces silently incorrect results, not a runtime error.
-- Swizzled layouts make SMEM address computation non-trivial. Using CuTe's layout algebra avoids manual indexing errors.
-- For data types wider than 2 bytes (e.g., FP32 accumulators), the optimal swizzle mode may differ. TMEM accumulators avoid this issue since they use a separate address space.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/techniques/tile-scheduling.md

@@ -1,223 +0,0 @@
----
-id: technique-tile-scheduling
-title: Tile Scheduling Strategies
-type: technique
-architectures:
-- sm100
-- sm90
-languages:
-- '*'
-tags:
-- tile-scheduling
-- clc
-- persistent-kernel
-confidence: source-reported
-evidence_level: spec
-reproducibility: snippet
-prerequisites:
-- hw-clc
-related:
-- hw-clc
-- technique-persistent-kernels
-- pattern-low-sm-utilization
-sources:
-- doc-nvidia-tuning-guide
-- doc-cutlass-blackwell
-- pr-cutlass-2161
----
-## Overview
-
-Tile scheduling determines the order in which output tiles of a GEMM (or attention) kernel are assigned to CTAs. The scheduling order affects L2 cache hit rates, tail-effect severity, and overall GPU utilization. On Blackwell, the CLC hardware unit supports dynamic scheduling policies including swizzled raster, while Hopper relies on software-based static stride or swizzled patterns computed at launch time.
-
-## Scheduling Strategies
-
-### Linear Raster (Naive)
-
-Tiles are assigned in row-major order. Simple but poor L2 locality: consecutive tiles share no B-matrix data until the entire M-dimension is traversed.
-
-```cuda
-// Linear raster: tile_idx maps directly to (tile_m, tile_n)
-__device__ void linear_raster(int tile_idx, int tiles_n,
-                               int& tile_m, int& tile_n) {
-    tile_m = tile_idx / tiles_n;
-    tile_n = tile_idx % tiles_n;
-}
-
-// Access pattern for a 4x4 tile grid:
-//  0  1  2  3
-//  4  5  6  7
-//  8  9 10 11
-// 12 13 14 15
-//
-// Problem: tiles 0,1,2,3 all load different B columns.
-// By tile 4, B column 0 has been evicted from L2.
-```
-
-### Swizzled Raster
-
-Tiles are assigned in a blocked pattern that groups nearby M and N tiles together, maximizing reuse of both A rows and B columns in L2 cache:
-
-```cuda
-// Swizzled raster: group tiles into blocks that share A and B data
-// swizzle_size controls the block width (typically 4-8)
-__device__ void swizzled_raster(int tile_idx, int tiles_m, int tiles_n,
-                                 int swizzle_size, int& tile_m, int& tile_n)
-{
-    // Number of tile columns per swizzle group
-    int group_cols = min(swizzle_size, tiles_n);
-    int tiles_per_group = tiles_m * group_cols;
-
-    // Which swizzle group
-    int group_idx = tile_idx / tiles_per_group;
-    int within_group = tile_idx % tiles_per_group;
-
-    // Within the group, iterate in column-major order
-    tile_m = within_group / group_cols;
-    tile_n = group_idx * group_cols + within_group % group_cols;
-}
-
-// Access pattern with swizzle_size=2 on a 4x4 grid:
-//  0  1 |  8  9
-//  2  3 | 10 11
-//  4  5 | 12 13
-//  6  7 | 14 15
-//
-// Tiles 0,1,2,3 share the same B columns (0,1).
-// Tiles 0,2,4,6 share the same A rows.
-// Much better L2 reuse.
-```
-
-### Static Stride (Hopper Persistent)
-
-Each CTA processes tiles at fixed intervals equal to the grid size:
-
-```cuda
-// Static stride: CTA i processes tiles i, i+gridDim.x, i+2*gridDim.x, ...
-__device__ void static_stride(int cta_id, int total_ctas,
-                               int iteration, int tiles_n,
-                               int& tile_m, int& tile_n) {
-    int tile_idx = cta_id + iteration * total_ctas;
-    tile_m = tile_idx / tiles_n;
-    tile_n = tile_idx % tiles_n;
-}
-```
-
-### CLC Dynamic Scheduling (Blackwell)
-
-The CLC hardware scheduler assigns tiles at runtime, combining the benefits of dynamic load balancing with configurable scheduling policies:
-
-```cuda
-// CLC-based scheduling on Blackwell
-// The scheduling policy is set once during CLC initialization
-enum class ClcSchedulePolicy {
-    LinearRaster,       // Simple row-major order
-    SwizzledRaster,     // Blocked pattern for L2 locality
-    ColumnFirst,        // Column-major for specific workloads
-    Hilbert             // Space-filling curve (experimental)
-};
-
-__device__ void clc_init_scheduler(
-    void* clc_buffer,
-    int tiles_m, int tiles_n,
-    ClcSchedulePolicy policy)
-{
-    if (threadIdx.x == 0) {
-        // Program the CLC with tile grid dimensions and policy
-        // CLC handles the swizzle mapping internally
-        uint32_t config = encode_clc_config(tiles_m, tiles_n, policy);
-        asm volatile(
-            "clusterctl.init.shared [%0], %1, %2, %3;"
-            : : "r"((uint32_t)clc_buffer),
-                "r"(tiles_m), "r"(tiles_n), "r"(config)
-        );
-    }
-    __syncwarp();
-}
-```
-
-## CUTLASS Tile Schedulers
-
-CUTLASS provides several tile schedulers that abstract these strategies:
-
-```cuda
-// CUTLASS tile scheduler selection for SM100
-// All persistent schedulers inherit from PersistentTileSchedulerSm100
-
-// 1. Default CLC scheduler with swizzled raster
-using Scheduler_Default = cutlass::gemm::PersistentTileSchedulerSm100;
-
-// 2. Stream-K scheduler for better tail handling
-//    Splits K-dimension across CTAs for the last wave
-using Scheduler_StreamK = cutlass::gemm::StreamKSchedulerSm100;
-
-// 3. Grouped GEMM scheduler for MoE workloads
-//    Each group has different M, shared N and K
-using Scheduler_Grouped = cutlass::gemm::GroupedTileSchedulerSm100;
-
-// Usage in CUTLASS kernel definition:
-using GemmKernel = cutlass::gemm::kernel::GemmUniversal<
-    cute::Shape<Int<128>, Int<256>, Int<64>>,   // Tile shape
-    ElementA, LayoutA,
-    ElementB, LayoutB,
-    ElementC, LayoutC,
-    TiledMma,
-    CollectiveMainloop,
-    CollectiveEpilogue,
-    Scheduler_Default                            // Tile scheduler
->;
-```
-
-## L2 Cache Locality Analysis
-
-The choice of scheduling strategy directly impacts L2 cache hit rates. On B200 with 126 MB L2:
-
-```python
-# L2 cache reuse analysis for different schedulers
-# Problem: M=8192, N=8192, K=4096, BF16
-# Tile: 128x256, giving 64x32 = 2048 tiles
-# B200: 142 SMs, 126 MB L2
-
-tile_bytes_A = 128 * 4096 * 2  # 1 MB per tile row of A
-tile_bytes_B = 4096 * 256 * 2  # 2 MB per tile column of B
-
-# Linear raster: first wave loads 142 tiles across 142/32 = 4.4 column groups
-# B data for 5 different column groups = 5 * 2 MB = 10 MB (fits in L2)
-# But A data for 142/32 = 4.4 row groups * 4.4 col groups = ~20 distinct A rows
-# 20 * 1 MB = 20 MB -> some L2 eviction
-
-# Swizzled raster (swizzle=4): first wave covers 142 tiles in ~36 groups of 4
-# Each group uses 1 A row + 4 B columns = 1 + 8 = 9 MB per group
-# But groups share A rows: total unique A = ~36 rows * 1 MB = 36 MB
-# L2 pressure: 36 MB + 8 MB = 44 MB (fits in B200's 126 MB L2)
-
-# Conclusion: swizzled raster reduces L2 misses by ~2x vs linear for large problems
-```
-
-## Tail Effect Mitigation
-
-The "tail effect" occurs when the last wave of tiles does not fully occupy all SMs. Different schedulers handle this differently:
-
-| Scheduler | Tail Handling | SM Utilization (Last Wave) |
-|-----------|---------------|---------------------------|
-| Linear raster | None | `(total_tiles % num_SMs) / num_SMs` |
-| Static stride | None | Same as linear |
-| CLC dynamic | Automatic | Fast CTAs steal from slow ones |
-| Stream-K | K-splitting | Near 100% (splits partial tiles across SMs) |
-
-For a problem with 150 tiles on 142 SMs:
-- Static: last wave has 8 tiles on 8 SMs, 134 SMs idle (5.6% utilization)
-- CLC: fast-finishing CTAs from wave 1 absorb the 8 extra tiles
-- Stream-K: the 8 remaining tiles are split across all 142 SMs
-
-## When to Use
-
-- **Swizzled raster**: Default choice for large GEMMs. Always better than linear for L2 locality.
-- **CLC dynamic**: Recommended on Blackwell for all persistent kernels. Combines dynamic load balancing with swizzled ordering.
-- **Stream-K**: Best for small-to-medium problems where the tail effect dominates. Adds complexity for K-dimension synchronization.
-- **Grouped scheduler**: Essential for MoE and batched GEMM where problem sizes vary across groups.
-
-## Caveats
-
-- Swizzle size must be tuned per problem shape. Too large a swizzle group exceeds L2 capacity; too small loses the locality benefit.
-- CLC scheduling adds a small latency per tile acquisition (~10s of cycles). For extremely small tiles, this overhead is proportionally larger.
-- Stream-K requires atomic accumulation where K-splits meet, adding synchronization overhead. Only worthwhile when tail utilization is a proven bottleneck.