@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/hardware/wgmma/skill-wgmma.md

@@ -1,222 +0,0 @@
----
-title: 'wgmma: Warpgroup MMA on Hopper via cutlass-cute'
-status: verified
-evidence_level: measured
-applies_to_pattern_class:
-- tensor-core
-applies_to_ops:
-- gemm
-- attention
-requires_sm: '>=9.0a'
-requires_features:
-- wgmma
-- tma
-- mbarrier
-single_kernel_useful: true
-cuda_version_tested: 12.9.86
-driver_version_tested: 570.124.06
-toolchain: nvcc 12.9 + ptxas 12.9
-measured_on: H200-SXM | sm_90a | cuda 12.9.86 | driver 570.124.06
-source:
-- path: spec
-  anchor: Reference
-artifacts:
-  code: artifacts/experience/api-probes/gemm/gemm_compare.cu
-  build: artifacts/experience/api-probes/gemm/build.sh
-  introspection: artifacts/experience/api-probes/gemm/device.json
-  profile: artifacts/experience/api-probes/gemm/2026-04-28-wgmma-counters.csv
-upstream_repo: cutlass@f74fea9c
-related_apis: []
-related_skills:
-- tma
-- warp-specialization
-id: skill-wgmma
-type: skill
-vendor: nvidia
-tags:
-- cuda-cpp
-- wgmma
-- tma
-- mbarrier
-- cluster
-- fp8
-- warp-specialization
-- persistent-kernel
-- pipeline-stages
-- register-budgeting
-- shared-memory-optimization
-- swizzling
-- tile-scheduling
-- tma-multicast
-- gemm
-- attention
-- quantization
-- grouped-gemm
-- ptx
-- cute-dsl
-applies_to:
-- general
-source_refs:
-- source_id: source-code/cutlass
-  path: include/cute/atom/mma_traits_sm90_gmma.hpp
-  anchor: MMA_64x{N}x{K}_F32{TF32|BF16|FP16|FP8}_SS_TN
-- source_id: source-code/cutlass
-  path: examples/48_hopper_warp_specialized_gemm/48_hopper_warp_specialized_gemm.cu
-  anchor: Lall
-- source_id: blogs/colfax
-  path: cutlass-tutorial-fast-matrix-multiplication-with-wgmma-on-nvidia-hopper-gpus
-  anchor: wgmma atom + warpgroup synchronization
-architectures:
-- sm90
-- sm90a
-languages:
-- ptx
-- cuda-cpp
-- cute-dsl
-hardware_features:
-- wgmma
-- tma
-- mbarrier
-- cluster
-- fp8
-techniques:
-- warp-specialization
-- persistent-kernel
-- pipeline-stages
-- register-budgeting
-- shared-memory-optimization
-- swizzling
-- tile-scheduling
-- tma-multicast
-kernel_types:
-- gemm
-- attention
-- quantization
-- grouped-gemm
-confidence: experimental
-artifact_dir: artifacts/experience/api-probes/gemm
----
-# wgmma — Warpgroup MMA on Hopper
-
-## What it is
-
-`wgmma` is the Hopper instruction family that lets one **warpgroup** (4 warps = 128 threads) issue a single matrix-multiply-accumulate covering an `M64 × N × K` tile in a single asynchronous PTX call. The atom shape is `M=64`, `N ∈ {8, 16, 32, 64, 96, 128, 192, 256}`, `K = 16` for bf16/fp16 / `K = 8` for tf32 / `K = 32` for fp8. A `wgmma.fence` + `wgmma.commit_group` + `wgmma.wait_group` triple separates issue from completion, and the F32 accumulator lives in the consumer warpgroup's registers throughout — no smem round-trip.
-
-In cutlass-cute the entire pattern lives in `MMA_Atom<MMA_64xNxK_F32{tf32|bf16|fp16|fp8}_{SS|RS}_TN<..>>` plus a `TiledMMA<...>` wrapper.
-
-## Why it helps
-
-- **One instruction per warpgroup tile** vs. dozens of `mma.sync` / `wmma` issues on Ampere. Issue bandwidth on the warp scheduler stops being the bottleneck for compute-bound GEMM.
-- **Asynchronous**. `wgmma` issues, then `wgmma.commit_group` releases the warpgroup; the warpgroup can do other work (or just spin on `wgmma.wait_group`) until the result lands in registers.
-- **Pairs naturally with TMA**. Together the producer/consumer pattern (TMA producer → mbarrier → wgmma consumer) keeps both pipes saturated.
-
-## When to use it
-
-- Any single-kernel gemm/attention compute path on Hopper. wgmma is the only path to ≥80 % of H200 peak TFLOPS at gemm.
-- Mixed-dtype mainloops (cutlass `MainloopSm90TmaGmmaWarpSpecialized*MixedInput…`) where the input dtype differs from the accumulator dtype.
-
-## When **not** to use it
-
-- Targets older than sm_90a (Ampere/Turing — use `mma.sync` or `wmma`).
-- Operators dominated by memory traffic, where the wgmma issue rate is irrelevant.
-- Kernels that cannot afford a 128-thread minimum block — wgmma fundamentally consumes one warpgroup.
-
-## Atom shapes supported on H200 (from `mma_traits_sm90_gmma.hpp`)
-
-| Family | M | N range | K |
-|---|---|---|---|
-| F32 ← BF16 × BF16 | 64 | 8, 16, 32, 64, 96, 128, 192, 256 | 16 |
-| F32 ← FP16 × FP16 | 64 | 8, 16, 32, 64, 96, 128, 192, 256 | 16 |
-| F32 ← TF32 × TF32 | 64 | 8, 16, 32, 64, 96, 128, 192, 256 | 8 |
-| F32 ← FP8(E4M3/E5M2) × FP8(E4M3/E5M2) | 64 | 8, 16, 32, 64, 96, 128, 192, 256 | 32 |
-
-A shape-selection guide (which N to pick for which gemm size) appears below; this skill currently pins the F32-TF32 64×128×8 shape that was measured at the canonical anchor.
-
-## Pre-conditions to issue wgmma (measured)
-
-1. `nvcc -arch=sm_90a` (NOT plain `sm_90`).
-2. Block size that includes ≥1 warpgroup of 128 threads (the example uses 384 threads = 3 warpgroups: 1 producer + 2 consumers in the cooperative pattern).
-3. SmemLayoutAtom that is wgmma-compatible — typically the `Swizzle<3,4,3>` × 32-bit layout cute provides via `cute::GMMA_64x128x8_Major<cute::GMMA::Major::K>` etc.
-4. F32 accumulator in registers; no smem-resident accumulator path is taken on Hopper.
-
-## Measured Characteristics
-
-End-to-end wgmma-driven GEMM via `examples/48_hopper_warp_specialized_gemm` at the cutlass commit pin (see `artifacts.code`):
-
-- 5120 × 4096 × 4096 GEMM (TF32 inputs, F32 acc) on the canonical `MMA_64x128x8_F32TF32TF32_SS_TN` atom: **0.608 ms / launch ≈ 282.6 TFLOPS** on H200-SXM, `Disposition: Passed` against the cutlass reference.
-- `sm__cycles_active.avg.pct_of_peak_sustained_elapsed` = **87.77 %** (decoded from the profiled kernel signature).
-- `sm__warps_active.avg.pct_of_peak_sustained_active` = 14.07 % — low warp-occupancy is the **expected** signature of warp-specialized wgmma: most warps spin on `wgmma.wait_group`, not on the warp scheduler.
-- 384 threads/CTA × 4×30 grid = 120 CTAs = one wave on H200's 132 SMs.
-
-Full probe record: sources/experience/api-probes/gemm.md.
-
-For the wgmma instruction in isolation (cutlass-free 11-config zoo across N-shape × dtype × layout × A-source on H200), see sources/experience/hw-probes/wgmma-ptx.md and sources/experience/hw-probes/wgmma-ptx.md.
-
-## Minimum repro
-
-`artifacts/experience/api-probes/gemm/gemm_compare.cu` (vendored from cutlass example 48 at commit `f74fea9c`), plus `build.sh` (nvcc invocation), `run.sh` (build → run → ncu → compute-sanitizer → CSV), `run-shape-sweep.sh` (atom-shape sweep), and `device.json` (H200 nvidia-smi snapshot) in the same `artifacts/` directory.
-
-## Shape-selection guide
-
-Measured on H200-SXM at problem size 4096 × 4096 × 4096, TF32 input / F32 accumulator, varying only the wgmma atom selected by cute's `CollectiveBuilder` from `TileShape.N`. Three atoms from the `MMA_64xNx8_F32TF32TF32_SS_TN` family. Cluster shape adjusted for the largest atom so the cluster-N × tile-N product still tiles the 4096-wide grid.
-
-| Atom | TileShape (M×N×K) | Cluster | Kernel μs | TFLOPS | SM cycles active | Warps active | Use when |
-|---|---|---|---|---|---|---|---|
-| `MMA_64x64x8_F32TF32TF32_SS_TN`  (small)  | 128×64×32  | (4,2,1) | 765.3 | **179.6** | 85.45 % | 14.07 % | Smaller per-CTA register footprint; useful when the kernel must coexist with a large epilogue, with persistent-kernel state, or when the operator's N dimension is itself small. About 58 % of the medium-atom rate at this problem size. |
-| `MMA_64x128x8_F32TF32TF32_SS_TN` (medium) | 128×128×32 | (4,2,1) | 497.8 | **276.1** | 85.81 % | 14.07 % | The cutlass-example-48 default. Best general-purpose choice for square TF32 GEMM in the 4K–8K range. Well within the H200 cooperative kernel's compute-bound sweet spot. |
-| `MMA_64x256x8_F32TF32TF32_SS_TN` (large)  | 128×256×32 | (2,2,1) | 444.8 | **309.0** | 77.23 % | 14.07 % | Largest atom in the family. Best peak at large square shapes, but the smaller cluster (`<2,2,1>` instead of `<4,2,1>`) cuts multicast efficiency, so the gain over medium is shape-dependent and shrinks for smaller problems. The grid drops to (2, 60, 1) = 120 CTAs ≈ one wave, hence the lower SM-cycles-active. |
-
-**Heuristic for picking N:**
-
-- Default is **medium (N=128)** unless you have reason otherwise. It pairs cleanly with the standard `<4,2,1>` cluster and gives ~85 % SM time at typical LLM GEMM shapes.
-- Pick **large (N=256)** only when (a) the problem is square at ≥4K and (b) you can afford the smaller cluster (or you can rebalance ClusterShape to keep multicast). At 4K² the gain is ~12 % over medium; at smaller shapes the gain disappears or reverses because the grid drops below one wave.
-- Pick **small (N=64)** only when register pressure or epilogue-co-resident state forces it. Throughput drops sharply (~35 % below medium) because the kernel issues 2× as many wgmma atoms per CTA tile and the wgmma issue pipeline becomes the bottleneck.
-- The `sm__warps_active` 14.07 % across all three atoms is the warp-specialized-wgmma signature, not a regression. Don't try to "improve" it; warps in the consumer wg are *meant* to spend most of their time on `wgmma.wait_group`.
-
-For complementary guidance on **problem-size scaling at a fixed atom** (small/medium/large GEMM under the medium atom), see the supplemental record sources/experience/api-probes/gemm.md. Headline: at the medium atom the kernel goes 21.4 → 189.5 → 276.2 → 282.5 → 292.3 TFLOPS as MNK grows from 512³ through 8192³; below 2K total problem size the kernel-launch/wave-amortization overheads dominate, so adjust CTA tile or grouped-GEMM rather than the wgmma atom.
-
-**Authoritative evidence:**
-- Atom-shape sweep (this section): sources/experience/api-probes/gemm.md. Reproducible from `artifacts/experience/api-probes/gemm/run_atom_sweep.sh`.
-- Problem-size sweep at the medium atom: sources/experience/api-probes/gemm.md. Reproducible from `artifacts/experience/api-probes/gemm/run_problem_size_sweep.sh`.
-
-## Atomic Usage (cute)
-
-Each row maps a cute `MMA_Atom` specialization (from `mma_traits_sm90_gmma.hpp`) to the PTX probe config that exercises the same instruction. Every atom listed has a 1:1 correspondence with a measured config in the wgmma-ptx probe zoo at `sources/experience/hw-probes/wgmma-ptx.md`.
-
-| cute MMA_Atom struct | PTX mnemonic shape | Probe config | Probe record |
-|---|---|---|---|
-| `SM90_64x8x16_F32BF16BF16_SS<GMMA::Major::K, GMMA::Major::K>` | `m64n8k16.f32.bf16.bf16` | `ss_tn_bf16_n8` | `sources/experience/hw-probes/wgmma-ptx.md` |
-| `SM90_64x16x16_F32BF16BF16_SS<GMMA::Major::K, GMMA::Major::K>` | `m64n16k16.f32.bf16.bf16` | `ss_tn_bf16_n16` | same |
-| `SM90_64x32x16_F32BF16BF16_SS<GMMA::Major::K, GMMA::Major::K>` | `m64n32k16.f32.bf16.bf16` | `ss_tn_bf16_n32` | same |
-| `SM90_64x64x16_F32BF16BF16_SS<GMMA::Major::K, GMMA::Major::K>` | `m64n64k16.f32.bf16.bf16` | `ss_tn_bf16_n64` | same |
-| `SM90_64x128x16_F32BF16BF16_SS<GMMA::Major::K, GMMA::Major::K>` | `m64n128k16.f32.bf16.bf16` | `ss_tn_bf16_n128` | same |
-| `SM90_64x256x16_F32BF16BF16_SS<GMMA::Major::K, GMMA::Major::K>` | `m64n256k16.f32.bf16.bf16` | `ss_tn_bf16_n256` | same |
-| `SM90_64x64x16_F32F16F16_SS<GMMA::Major::K, GMMA::Major::K>` | `m64n64k16.f32.f16.f16` | `ss_tn_fp16_n64` | same |
-| `SM90_64x64x8_F32TF32TF32_SS_TN<>` | `m64n64k8.f32.tf32.tf32` | `ss_tn_tf32_n64` | same |
-| `SM90_64x64x32_S32S8S8S32_SS_TN<>` | `m64n64k32.s32.s8.s8` | `ss_tn_s8_n64` | same |
-| `SM90_64x64x16_F32BF16BF16_SS<GMMA::Major::MN, GMMA::Major::MN>` | `m64n64k16.f32.bf16.bf16` (NT) | `ss_nt_bf16_n64` | same |
-| `SM90_64x64x16_F32BF16BF16_RS<GMMA::Major::K, GMMA::Major::K>` | `m64n64k16.f32.bf16.bf16` (RS) | `rs_tn_bf16_n64` | same |
-
-Minimal cute-API snippet (the pattern used by cutlass example 48 and the `gemm_compare.cu` harness):
-
-```cpp
-#include <cute/atom/mma_atom.hpp>
-#include <cute/atom/mma_traits_sm90_gmma.hpp>
-
-// Choose an atom (this is the medium N=128 TF32 default):
-using MMA_Op = SM90_64x128x8_F32TF32TF32_SS_TN<>;
-using MMA = MMA_Atom<MMA_Traits<MMA_Op>>;
-// Tile it across the CTA with TiledMMA:
-using TiledMma = TiledMMA<MMA, Layout<Shape<_2,_1,_1>>>;  // 2 warpgroups consumer
-```
-
-Families NOT exercised in the probe zoo but present in `mma_traits_sm90_gmma.hpp`:
-- fp8 atoms (`SM90_64xNx32_F32E4M3E4M3_SS_TN`, `SM90_64xNx32_F32E5M2E4M3_SS_TN`, etc.) — K=32, same issue port as s8.
-- Sparse atoms (`SM90_64xNx64_F32E4M3E4M3_SS_TN<SparseConfig::StructuredN>`) — requires structured sparsity metadata.
-- Mixed-input atoms (e.g. `SM90_64xNx32_F32E4M3E5M2_RS_TN`) — A and B may differ in dtype.
-
-## How it connects to the rest of the KB
-
-- Pairs with `wiki/nvidia/hardware/tma/skill.md` — the cooperative kernel exercises both. wgmma without TMA falls back to `cp.async`, which collapses the issue-rate advantage.
-- Pre-condition for `wiki/nvidia/techniques/warp-specialization/` and `wiki/nvidia/techniques/persistent-kernel/`.
-- Shape-selection guide is the dedicated section above, backed by the atom-shape sweep at sources/experience/api-probes/gemm.md.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/kernels/deepgemm.md

@@ -1,212 +0,0 @@
----
-id: kernel-deepgemm
-title: DeepGEMM — FP8 GEMM with Fine-Grained Scaling
-type: kernel
-architectures:
-- sm100
-- sm90
-tags:
-- gemm
-- fp8
-- fine-grained-quantization
-- block-scale
-confidence: source-reported
-evidence_level: spec
-reproducibility: snippet
-kernel_types:
-- gemm
-- grouped-gemm
-languages:
-- cuda-cpp
-- ptx
-related:
-- technique-fine-grained-quantization
-- hw-tcgen05-mma
-- hw-nvfp4
-sources:
-- blog-deepgemm
-- pr-deepgemm-304
-- pr-cutlass-2139
-- pr-vllm-23696
-performance_claims:
-- gpu: H800
-  dtype: fp8
-  shape: M=4096, N=4096, K=4096
-  metric: TFLOPS
-  value: 1550
-  utilization: ~90%
-  source_id: blog-deepgemm
-artifact_dir: artifacts/kernels/deepgemm
----
-# DeepGEMM -- FP8 GEMM with Fine-Grained Scaling
-
-## Overview
-
-DeepGEMM is DeepSeek's open-source FP8 GEMM library providing high-performance matrix multiplication with fine-grained per-tile/per-block scaling. The core kernel is remarkably compact (~300 lines), yet achieves approximately 90% utilization on H800. It supports both Hopper (SM90 via WGMMA) and Blackwell (SM100 via tcgen05.mma) architectures, and includes specialized MoE grouped GEMM layouts.
-
-The key innovation is the fine-grained quantization scheme: tile-wise 1x128 scaling for activations and block-wise 128x128 scaling for weights, which prevents outlier values from destroying quantization precision.
-
-## Fine-Grained Quantization Scheme
-
-```
-Activations (tile-wise 1x128):
-  +-----------+-----------+-----------+
-  | tile 0    | tile 1    | tile 2    |  <-- each tile: 1 row x 128 cols
-  | scale: s0 | scale: s1 | scale: s2 |  <-- one FP32 scale per tile
-  +-----------+-----------+-----------+
-
-Weights (block-wise 128x128):
-  +---------------+---------------+
-  |  block (0,0)  |  block (0,1)  |  <-- each block: 128 rows x 128 cols
-  |  scale: s_00  |  scale: s_01  |  <-- one FP32 scale per block
-  +---------------+---------------+
-  |  block (1,0)  |  block (1,1)  |
-  |  scale: s_10  |  scale: s_11  |
-  +---------------+---------------+
-```
-
-## FP8 Accumulation with Nc=128 CUDA Core Promotion
-
-On Hopper, the Tensor Core accumulator has limited precision (~FP22, not true FP32). DeepGEMM mitigates this by promoting partial sums to a separate FP32 accumulator on CUDA Cores every Nc=128 columns (4 consecutive WGMMA operations).
-
-```cpp
-// SM90 path: WGMMA with Nc=128 CUDA Core promotion
-// Every 4 WGMMAs, promote accumulated result to FP32 CUDA core accumulator
-constexpr int Nc = 128;  // Promotion interval (4 WGMMAs of n=32 each)
-constexpr int WGMMA_N = 32;
-
-float cuda_core_acc[TILE_M][TILE_N] = {0};  // FP32 accumulator on CUDA Cores
-
-for (int k = 0; k < K; k += Nc) {
-    // Run 4 consecutive WGMMAs with TC-limited precision accumulation
-    __half2 tc_acc[TILE_M][WGMMA_N];  // Tensor Core accumulator (~FP22)
-    memset(tc_acc, 0, sizeof(tc_acc));
-
-    for (int sub_k = 0; sub_k < Nc; sub_k += WGMMA_K) {
-        wgmma_mma_async(tc_acc, A_smem + sub_k, B_smem + sub_k);
-    }
-    wgmma_wait();
-
-    // Promote: add TC result to CUDA Core FP32 accumulator
-    // This prevents precision loss from repeated FP22 accumulation
-    for (int m = 0; m < TILE_M; m++)
-        for (int n = 0; n < TILE_N; n++)
-            cuda_core_acc[m][n] += (float)tc_acc[m][n] * scale_a[m] * scale_b[n];
-}
-```
-
-On Blackwell (SM100), the tcgen05.mma instruction with TMEM accumulation uses native UE8M0 block scaling, which eliminates the need for explicit CUDA core promotion.
-
-```cpp
-// SM100 path: tcgen05.mma with native block scaling
-// Scaling factors packed as UE8M0 (4 values per uint32)
-// No explicit CUDA core promotion needed -- TMEM accumulates in full precision
-
-// Pack 4 UE8M0 scale factors into a single uint32
-uint32_t packed_scales = pack_ue8m0(sf[0], sf[1], sf[2], sf[3]);
-
-// tcgen05.mma reads A/B from SMEM, accumulates into TMEM
-// Block scale applied natively during MMA
-asm volatile(
-    "tcgen05.mma.cta_group::1.kind::f8f6f4"
-    " [%0], %1, %2, %3, %4;"
-    :
-    : "l"(tmem_addr), "l"(a_smem_addr), "l"(b_smem_addr),
-      "r"(packed_scales), "n"(SCALE_D_ENABLED)
-);
-```
-
-## MoE Grouped GEMM Layouts
-
-DeepGEMM provides three grouped GEMM layouts tailored for MoE workloads, where only the M-axis varies (different token counts per expert) while N and K remain fixed:
-
-```
-Layout 1: Contiguous (prefill)
-  Expert 0: M0 tokens  ──┐
-  Expert 1: M1 tokens  ──┤── All packed contiguously in memory
-  Expert 2: M2 tokens  ──┘   Index array stores cumulative offsets
-
-Layout 2: Masked (decode with CUDA graphs)
-  Fixed-size M_max allocation per expert
-  Binary mask indicates valid tokens
-  Compatible with CUDA graph capture (no dynamic shapes)
-
-Layout 3: K-grouped (weight gradients)
-  Groups along K-axis instead of M-axis
-  Used for computing dW in MoE training backward
-```
-
-```cpp
-// Contiguous grouped GEMM dispatch
-// problem_sizes[i] = {M_i, N, K} for expert i
-void grouped_gemm_contiguous(
-    const fp8_t* A,         // All expert inputs packed
-    const fp8_t* B,         // Expert weights [num_experts, N, K]
-    float* C,               // Output packed
-    const int* offsets,     // Cumulative M offsets per expert
-    int num_experts
-) {
-    // Each thread block picks an expert via tile scheduling
-    // M-axis tiles distributed across experts using offset lookup
-    int expert_id = binary_search(offsets, num_experts, tile_m_start);
-    int local_m = tile_m_start - offsets[expert_id];
-
-    // Standard GEMM tile with expert-specific B matrix
-    compute_tile(A + offsets[expert_id] * K,
-                 B + expert_id * N * K,
-                 C + offsets[expert_id] * N,
-                 local_m, N, K);
-}
-```
-
-## JIT Compilation
-
-DeepGEMM uses JIT compilation via NVRTC to specialize kernels per problem shape at runtime. This avoids the combinatorial explosion of pre-compiled template instantiations while still achieving optimal register allocation and loop unrolling.
-
-```cpp
-// Lightweight JIT module: compile per-shape kernel at first call
-auto kernel = jit_compile(
-    "deepgemm_fp8",
-    {{"M", M}, {"N", N}, {"K", K},
-     {"BLOCK_M", 128}, {"BLOCK_N", 128}, {"BLOCK_K", 64},
-     {"NUM_STAGES", 4}}
-);
-kernel.launch(A, B, C, scales_a, scales_b, stream);
-```
-
-## Memory Layout
-
-SM90 kernels use NT (non-transposed A, transposed B) layout exclusively. SM100 kernels support all four layout combinations (NT, TN, NN, TT), enabled by tcgen05.mma's flexible operand addressing.
-
-## Performance
-
-| GPU | Dtype | Shape | TFLOPS | Utilization |
-|-----|-------|-------|--------|-------------|
-| H800 | FP8 | M=4096, N=4096, K=4096 | 1550 | ~90% |
-
-## When to Use
-
-- FP8 inference and training where per-tensor quantization loses too much precision
-- MoE expert computation with variable token counts per expert
-- Situations where fine-grained (1x128 / 128x128) scale granularity is needed
-
-## Caveats
-
-- SM90 path is NT layout only
-- JIT compilation adds first-call latency (amortized over repeated calls)
-- Fine-grained scaling adds overhead vs. per-tensor scaling -- only beneficial when outlier sensitivity matters
-
-## Sources
-
-- [DeepGEMM GitHub](https://github.com/deepseek-ai/DeepGEMM)
-- [DeepSeek-V3 Technical Report](https://arxiv.org/abs/2412.19437)
-
-## Full Reference Implementation
-
-Local verbatim upstream code lives in [`artifacts/kernels/deepgemm/full/`](../../../artifacts/kernels/deepgemm/full/) (see its `PROVENANCE.yaml` for the pinned upstream SHA and byte-verified SHA-256). Labeled derived variants — including a naive/teaching skeleton — live in [`artifacts/kernels/deepgemm/variants/`](../../../artifacts/kernels/deepgemm/variants/).
-
-Query via:
-
-```bash
-python3 scripts/get_page.py kernel-deepgemm --include-code
-```

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/kernels/flash-attention-4.md

@@ -1,202 +0,0 @@
----
-id: kernel-flash-attention-4
-title: FlashAttention-4
-type: kernel
-architectures:
-- sm100
-tags:
-- attention
-- flash-attention
-- tcgen05
-- tmem
-- 2sm-cooperative
-- software-exp
-confidence: source-reported
-evidence_level: spec
-reproducibility: snippet
-kernel_types:
-- attention
-- flash-attention
-languages:
-- cute-dsl
-related:
-- technique-warp-specialization
-- technique-software-exp
-- hw-tcgen05-mma
-- hw-tmem
-sources:
-- doc-flash-attention-4
-- blog-flash-attention-4
-- pr-flashinfer-1850
-performance_claims:
-- gpu: B200
-  dtype: bf16
-  shape: seqlen=8192, headdim=128
-  metric: TFLOPS
-  value: 1605
-  utilization: 71%
-  source_id: doc-flash-attention-4
-artifact_dir: artifacts/kernels/flash-attention-4
----
-
-# FlashAttention-4
-
-## Overview
-
-FlashAttention-4 is the Blackwell-native evolution of the FlashAttention family, designed to exploit SM100 architectural features that break the Hopper-era bottleneck: tensor core throughput doubles on Blackwell, but SFU (Special Function Unit) count and shared memory bandwidth remain unchanged. FA4 addresses this asymmetry through software-emulated exponentials, ping-pong scheduling across TMEM, and 2-CTA cooperative backward passes.
-
-Written entirely in CuTe DSL (Python), FA4 compiles 20-30x faster than equivalent CUTLASS C++ template code while matching or exceeding cuDNN performance.
-
-## Key Techniques
-
-### Ping-Pong Scheduling
-
-Two 128-token query tiles are assigned to each CTA. While one tile's matmul runs on the tensor cores, the other tile's softmax rescaling runs on dedicated warpgroups accessing TMEM. This hides the softmax latency behind MMA compute.
-
-```python
-# CuTe DSL: Ping-pong tile scheduling (simplified)
-# Two query tiles per CTA, alternating MMA and softmax phases
-
-@cute.kernel
-def flash_attention_4_fwd(Q, K, V, O, L):
-    # Each CTA processes 2 query tiles of 128 tokens
-    TILE_Q = 128
-    NUM_TILES = 2
-
-    # Phase A: tile_0 does MMA(Q0, K), tile_1 does softmax rescale
-    # Phase B: tile_1 does MMA(Q1, K), tile_0 does softmax rescale
-
-    for kv_block in range(num_kv_blocks):
-        # Ping: MMA on tile 0, softmax on tile 1
-        with warpgroup(mma_wg):
-            S0 = cute.mma(Q_tile[0], K_block)  # tcgen05.mma -> TMEM
-        with warpgroup(softmax_wg):
-            O_tile[1], L_tile[1] = rescale_and_accumulate(
-                O_tile[1], L_tile[1], S1, V_prev
-            )
-
-        # Pong: MMA on tile 1, softmax on tile 0
-        with warpgroup(mma_wg):
-            S1 = cute.mma(Q_tile[1], K_block)
-        with warpgroup(softmax_wg):
-            O_tile[0], L_tile[0] = rescale_and_accumulate(
-                O_tile[0], L_tile[0], S0, V_block
-            )
-```
-
-### Software-Emulated Exponential (Cody-Waite)
-
-The SFU `ex2` instruction is the bottleneck on Blackwell -- its throughput does not scale with the doubled tensor core rate. FA4 replaces it with a software exponential distributed across FMA units using Cody-Waite range reduction and Horner polynomial evaluation.
-
-```python
-# Software exp2 via Cody-Waite range reduction + Horner polynomial
-# Distributes across FMA units instead of using scarce SFU hardware
-
-def software_exp2(x):
-    """
-    Compute 2^x using FMA units instead of SFU ex2.
-    Cody-Waite range reduction splits x into integer + fraction.
-    Horner polynomial approximates 2^frac.
-    """
-    # Range reduction: x = n + f, where n is integer, f in [-0.5, 0.5]
-    n = round(x)
-    f = x - n  # Cody-Waite: use extended precision subtraction
-
-    # Horner polynomial for 2^f on [-0.5, 0.5]
-    # Coefficients chosen for bf16 precision target
-    p = f * (C5 + f * (C4 + f * (C3 + f * (C2 + f * C1))))
-    p = 1.0 + p
-
-    # Reconstruct: 2^x = 2^n * 2^f
-    return ldexp(p, n)  # Integer exponent via bit manipulation
-```
-
-This gives roughly 4x throughput improvement over the hardware SFU path by utilizing FMA units that would otherwise be idle during softmax phases.
-
-### Conditional Softmax Rescaling
-
-Standard FlashAttention rescales the output accumulator every KV block. FA4 only rescales when the running maximum changes significantly (large jump), reducing non-matmul operations.
-
-```python
-# Only rescale when max changes substantially
-def conditional_rescale(O_acc, lse_old, lse_new, threshold=2.0):
-    diff = lse_new - lse_old
-    if abs(diff) > threshold:
-        # Full rescale: O_acc *= exp(lse_old - lse_new)
-        scale = software_exp2((lse_old - lse_new) * LOG2E)
-        O_acc = O_acc * scale
-    # Otherwise: skip rescale, accumulate normally
-    return O_acc
-```
-
-### 2-CTA Backward Pass
-
-The backward pass spans two paired CTAs in a cluster, sharing TMEM across both SMs. This halves shared memory traffic for the dQ/dK/dV gradient computation.
-
-```python
-# 2-CTA backward: paired CTAs share TMEM via 2-SM cooperative mode
-@cute.kernel
-def flash_attention_4_bwd(Q, K, V, O, dO, dQ, dK, dV):
-    # Two CTAs cooperate: CTA_0 and CTA_1 in same cluster
-    # tcgen05.mma shape: m256 x n256 x k16 (2-SM cooperative)
-
-    cta_id = cute.cluster_rank()  # 0 or 1
-
-    for q_block in range(num_q_blocks):
-        # Both CTAs load shared KV block via TMA
-        K_smem = tma_load(K, kv_offset)
-        V_smem = tma_load(V, kv_offset)
-
-        # CTA_0: compute dK contribution
-        # CTA_1: compute dV contribution
-        if cta_id == 0:
-            # dK += dS^T @ Q  (accumulated in TMEM)
-            dS = compute_dS(O, dO, S)
-            cute.mma(dS.T, Q_tile, accumulator=dK_tmem)
-        else:
-            # dV += S^T @ dO  (accumulated in TMEM)
-            cute.mma(S.T, dO_tile, accumulator=dV_tmem)
-```
-
-## Performance
-
-| Configuration | GPU | Dtype | TFLOPS | Utilization | vs cuDNN | vs Triton |
-|---------------|-----|-------|--------|-------------|----------|-----------|
-| seqlen=8192, headdim=128 | B200 | BF16 | 1605 | 71% | 1.1-1.3x | 2.1-2.7x |
-
-The 71% MMA utilization represents the state of the art for attention kernels on Blackwell. The remaining 29% is consumed by softmax, rescaling, and memory transfers.
-
-## Implementation Notes
-
-- Written entirely in CuTe DSL (Python), not C++ templates
-- 20-30x faster compilation than equivalent CUTLASS C++ code
-- Uses `SM100_MMA_SS` atoms for tcgen05 MMA from shared memory
-- TMEM locality via `TMEM` locale in CuTe layout
-- TMA bulk loads for Q, K, V tiles into shared memory
-
-## When to Use
-
-- Standard multi-head attention on Blackwell with sequence lengths >= 1024
-- Both forward and backward passes
-- BF16 precision (FP8 support planned)
-
-## Caveats
-
-- SM100 only -- no fallback to SM90
-- Requires CuTe DSL toolchain (CUTLASS 4.5.0 + Python frontend)
-- Ping-pong scheduling most effective for headdim=128; smaller headdims may not fully overlap
-
-## Sources
-
-- [FlashAttention-4 paper](https://arxiv.org/abs/2603.05451)
-- [Tri Dao's blog](https://tridao.me/blog/2026/flash4/)
-
-## Full Reference Implementation
-
-Local verbatim upstream code lives in [`artifacts/kernels/flash-attention-4/full/`](../../../artifacts/kernels/flash-attention-4/full/) (see its `PROVENANCE.yaml` for the pinned upstream SHA and byte-verified SHA-256). Labeled derived variants — including a naive/teaching skeleton — live in [`artifacts/kernels/flash-attention-4/variants/`](../../../artifacts/kernels/flash-attention-4/variants/).
-
-Query via:
-
-```bash
-python3 scripts/get_page.py kernel-flash-attention-4 --include-code
-```