@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/references/primer.md

@@ -1,161 +0,0 @@
----
-version_sensitive:
-  id: vs-triton-3.6-blackwell-tcgen05
----
-
-# Topic Map / Primer
-
-A compact, authoritative map of the knowledge base. Use this as a fast lookup table when the user's question is broad — each row tells you exactly which page to open.
-
-All page IDs below resolve via `get_page.py <id>`. All paths are relative to the wiki root (the directory containing `data/`, `wiki/`, `sources/`, `queries/`).
-
----
-
-## Hardware Features (SM100)
-
-| Feature | Page ID | Path | Notes |
-|---|---|---|---|
-| tcgen05 MMA instruction | `hw-tcgen05-mma` | `wiki/nvidia/hardware/tcgen05-mma.md` | Blackwell tensor core instruction; replaces wgmma. CTA-scope (`cta_group::1`) or cluster-scope (`cta_group::2`). |
-| Tensor Memory (TMEM) | `hw-tmem` | `wiki/nvidia/hardware/tmem.md` | 256 KB/SM dedicated accumulator storage; 128 rows × 512 cols (32-bit); allocated via `tcgen05.alloc`. |
-| Cluster Launch Control (CLC) | `hw-clc` | `wiki/nvidia/hardware/clc.md` | Hardware work queue for persistent kernels; `clusterlaunchcontrol.try_cancel` for speculative workloads. |
-| Tensor Memory Accelerator (TMA) | `hw-tma` | `wiki/nvidia/hardware/tma.md` | Async bulk load/store (`cp.async.bulk.tensor`), multicast across cluster. |
-| 2-SM Cooperative MMA | `hw-2sm-cooperative` | `wiki/nvidia/hardware/2sm-cooperative.md` | `cta_group::2` — two CTAs in one cluster cooperate on a single MMA. |
-| NVFP4 / block-scaled FP | `hw-nvfp4` | `wiki/nvidia/hardware/nvfp4.md` | E2M1 data + FP8 (UE8M0) block scale per 16 elements. |
-| PDL / GDC | `hw-pdl-gdc` | `wiki/nvidia/hardware/pdl-gdc.md` | Programmatic Dependent Launch and Grid Dependency Control — overlap successive kernel launches. |
-| mbarrier primitives | `hw-mbarrier` | `wiki/nvidia/hardware/mbarrier.md` | Shared-memory barriers with phase tracking; the glue between TMA/tcgen05/warps. |
-
----
-
-## Optimization Techniques
-
-| Technique | Page ID | When to use |
-|---|---|---|
-| Warp specialization | `technique-warp-specialization` | Every nontrivial Blackwell kernel. Assign MMA / epilogue / TMA roles to different warps. |
-| Persistent kernels with CLC | `technique-persistent-kernels` | GEMM-like workloads with many tiles; eliminates launch overhead and tail effects. |
-| Ping-pong scheduling | `technique-ping-pong-scheduling` | Attention / back-to-back GEMMs; overlap two tiles by alternating SMEM buffers. |
-| Epilogue fusion | `technique-epilogue-fusion` | Fuse scale/bias/activation/quantize into the same kernel using TMEM-to-register epilogue warps. |
-| Software pipelining | `technique-pipeline-stages` | Multi-stage TMA→MMA overlap; typically 3–5 stages. |
-| Shared memory swizzling | `technique-swizzling` | Eliminate SMEM bank conflicts on A/B tile loads. |
-| Fine-grained FP8/FP4 quantization | `technique-fine-grained-quantization` | Per-tile or per-block scaling to preserve accuracy under aggressive quantization. |
-| Tile scheduling | `technique-tile-scheduling` | L2 locality, cluster reordering, CLC swizzle patterns. |
-| Double/multi-buffering | `technique-double-buffering` | Classical approach; overlaps load with compute, extends to 3+ stages. |
-| Software-emulated exp | `technique-software-exp` | FlashAttention softmax; cheaper than the hardware `ex2` path in specific regimes. |
-| Register budgeting | `technique-register-budgeting` | Raise occupancy when MMA warp counts are tight. |
-| Cache policy differentiation | `technique-cache-policy` | PTX `evict_first` / `no_allocate` on streaming data. |
-| Chunk-based parallelism | `technique-chunk-parallelism` | Linear-attention kernels (GatedDeltaNet, Mamba). |
-| Kernel fusion | `technique-kernel-fusion` | Combine elementwise + GEMM + reduction into a single launch. |
-| Wide vectorized loads | `technique-vectorized-loads` | Saturate HBM bandwidth; cp.async or 128-bit LDG. |
-
----
-
-## Kernel Case Studies
-
-| Kernel | Page ID | Headline perf | Key techniques |
-|---|---|---|---|
-| FlashAttention-4 | `kernel-flash-attention-4` | 1605 TFLOPS B200 BF16 (~71%) | Ping-pong scheduling, software exp, 2-CTA backward |
-| DeepGEMM (FP8) | `kernel-deepgemm` | ~1550 TFLOPS H800 FP8 | Fine-grained scaling, CUDA-core promotion (Nc=128) |
-| NVFP4 GEMM | `kernel-nvfp4-gemm` | ≈ cuBLAS FP4 on B200 | tcgen05 + UE8M0 block scales |
-| NVFP4 batched GEMV | `kernel-nvfp4-gemv` | 2000 µs → 22.4 µs | Memory-bound tricks, cache policy, register budgeting |
-| FP8 block-scale GEMM | `kernel-fp8-block-scale-gemm` | — | 1×128 / 128×128 block scaling scheme |
-| Fused MoE | `kernel-fused-moe` | — | Gate-up fused with SwiGLU; FP8 block scale routing |
-| Gated Dual GEMM | `kernel-gated-dual-gemm` | — | Gate × Up → SiLU fused in epilogue |
-| Grouped GEMM for MoE | `kernel-grouped-gemm` | — | Variable-sized expert GEMMs in one launch |
-| FlashMLA | `kernel-flashmla` | DeepSeek V3 decode | MLA-specific TMA + tcgen05 layout |
-| Sparse MLA | `kernel-sparse-mla` | DeepSeek V3.2 | Sparse KV retrieval before MLA core |
-| Native Sparse Attention (NSA) | `kernel-nsa` | 9× fwd speedup | Block-sparse + compressed attention |
-| Gated Delta Net | `kernel-gated-delta-net` | — | Chunk parallelism; linear attention |
-
----
-
-## Problem → Pattern (Diagnosis)
-
-| Symptom | Pattern page | Candidate techniques |
-|---|---|---|
-| Low SM utilization | `pattern-low-sm-utilization` | Persistent kernels, CLC, tile scheduling |
-| Memory bandwidth bound | `pattern-memory-bound` | Vectorized loads, cache policy, register budgeting |
-| Register pressure / low occupancy | `pattern-register-pressure` | Register budgeting, TMEM offload, kernel split |
-| Not reaching peak FLOPS | `pattern-compute-bound` | Warp specialization, epilogue fusion, pipeline stages |
-| Last wave underutilizes GPU | `pattern-tail-effect` | CLC, persistent kernels, tile-M/N aspect tuning |
-| Pipeline stalls | `pattern-pipeline-stalls` | Deeper pipeline stages, mbarrier phase management |
-| MoE expert load imbalance | `pattern-moe-load-imbalance` | Grouped GEMM, dynamic routing, overflow spill |
-
----
-
-## Languages / DSLs
-
-| DSL | Page ID | Notes |
-|---|---|---|
-| CuTe DSL | `lang-cute-dsl` | Preferred high-level path on SM100; native tcgen05/TMEM/CLC bindings. |
-| CUDA C++ | `lang-cuda-cpp` | PTX inline is common; used by CUTLASS, vLLM, SGLang custom kernels. |
-| PTX (SM100) | `lang-ptx` | `tcgen05.*`, `clusterlaunchcontrol.*`, `cp.async.bulk.tensor.*` — low-level control. |
-| Triton | `lang-triton` | On Blackwell: Triton 3.6+ ships native tcgen05 + TMEM lowering through descriptor/TMA + `tl.range(warp_specialize=True)`, `tl.dot_scaled`, and Gluon multi-CTA / 2CTA. Pre-3.6 the framing was "no tcgen05/TMEM exposure"; that historical context is preserved on the page. Cite via `version_sensitive: vs-triton-3.6-blackwell-tcgen05`. |
-
----
-
-## Migration (Hopper → Blackwell)
-
-| Page ID | Notes |
-|---|---|
-| `migration-wgmma-to-tcgen05` | wgmma (Hopper) → tcgen05 (Blackwell); CTA-scope vs cluster-scope MMA; descriptor changes. |
-| `migration-register-to-tmem` | Register accumulator (Hopper) → TMEM (Blackwell); epilogue restructuring required. |
-
----
-
-## Source Repositories (PR coverage)
-
-| Repo | PR pages | Ledger |
-|---|---|---|
-| NVIDIA/cutlass | 70 | `candidates/cutlass.yaml` |
-| sgl-project/sglang | 980 | `candidates/sglang.yaml` |
-| vllm-project/vllm | 1272 | `candidates/vllm.yaml` |
-| flashinfer-ai/flashinfer | 684 | `candidates/flashinfer.yaml` |
-| pytorch/pytorch | 89 | `candidates/pytorch.yaml` |
-| deepseek-ai/DeepGEMM | 13 | `candidates/deepgemm.yaml` |
-
-Query by repo: `python3 scripts/query.py --repo <name>`.
-
----
-
-## Competitions
-
-| Contest | Path | Problems/Tracks |
-|---|---|---|
-| GPU Mode NVFP4 Hackathon | `sources/contests/gpu-mode-nvfp4/` | `problem-1-gemv`, `problem-2-gemm`, `problem-3-gated-dual-gemm`, `problem-4-grouped-gemm` |
-| FlashInfer MLSys 2026 | `sources/contests/flashinfer-mlsys26/` | `track-a-fused-moe`, `track-b-sparse-attention`, `track-c-gated-delta-net` |
-
-Each contest page has a `submissions:` block listing rank-1/2/3 solutions with technique tags.
-
----
-
-## Canonical Aliases (normalization cheat sheet)
-
-When the user types one of these, match to the canonical term shown:
-
-| User-typed | Canonical |
-|---|---|
-| UMMA, tensor_core_gen05, tcgen05.mma | `tcgen05` |
-| tensor memory, TMEM | `tmem` |
-| Cluster Launch Control, CLC | `clc` |
-| Tensor Memory Accelerator, TMA, cp.async.bulk | `tma` |
-| 2-SM cooperative, two-SM cooperative, 2CTA, cta_group::2 | `2sm-cooperative` |
-| NVFP4, nv_float4, E2M1, FP4 E2M1 | `nvfp4` |
-| block scaling, UE8M0, microscaling, MX | `block-scale` |
-| Blackwell, B200, B100, GB200, GB300, SM100 | `sm100` |
-| Hopper, H100, H200, H800, SM90 | `sm90` |
-| MoE, mixture of experts, expert parallelism | `moe` |
-| MLA, multi-head latent attention | `mla` |
-| GDN, GatedDeltaNet, gated delta rule | `gated-delta-net` |
-| NSA, native sparse attention | `sparse-attention` |
-
-The `query.py` tool applies these automatically when scoring and when using `--tag`.
-
----
-
-## Confidence & Evidence Quick Reference
-
-- **`verified`** — a synthesis page carries official-doc plus upstream-code evidence, or an experience page records an independently reproduced measurement under its stated conditions.
-- **`source-reported`** — claims come from ≥ 1 authoritative source (paper, official blog, major repo). Most wiki pages live here.
-- **`inferred`** — synthesized from multiple sources; flag when quoting.
-- **`experimental`** — undocumented / version-sensitive / PTX tricks. Always caveat.
-
-Minimum reproducibility for technique, kernel, and language pages: **`snippet`** (compilable fragment) — validator-enforced.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/references/schema.md

@@ -1,149 +0,0 @@
-# Schema Reference
-
-Condensed reference for the wiki's controlled vocabulary and page schemas. Full definitions live in `data/schemas.yaml`.
-
-This reference describes the deployed legacy corpus. The adopted construction standard and the boundary for OKF migration are in [governance](../governance/README.md); [the declaration inventory](../governance/schemas/README.md) identifies current machine authorities.
-
-## Page Types and IDs
-
-Every page has a unique `id` with a type-specific prefix:
-
-| Type | ID Prefix | Purpose |
-|------|-----------|---------|
-| source-pr | `pr-<repo>-<N>` | A merged PR from a tracked repo |
-| source-doc | `doc-*` | Official NVIDIA docs, papers |
-| source-blog | `blog-*` | Community blog posts, tutorials |
-| source-contest | `contest-*` | Competition problems / tracks |
-| source-experience | `exp-*` | Hardware probe / measurement records |
-| wiki-hardware | `hw-*` | Hardware feature pages |
-| wiki-technique | `technique-*` | Optimization techniques |
-| wiki-kernel | `kernel-*` | Kernel case studies with perf claims |
-| wiki-pattern | `pattern-*` | Problem → solution diagnosis |
-| wiki-language | `lang-*` | DSL / language guides |
-| wiki-migration | `migration-*` | Hopper → Blackwell migration |
-| wiki-skill | `skill-*` | Foundational CUDA optimization skills |
-| wiki-api-definition | `api-*` | API definitions (CUDA Runtime, PTX) |
-| wiki-experience | `exp-*` | Hardware measurement / probe records |
-| wiki-operator-routing | `routing-*` | Operator decision trees |
-| wiki-algorithm | `algo-*` | Classical optimization algorithms |
-| wiki-code-walkthrough | `code-*` | Code repository extractions |
-| wiki-pitfall | `pitfall-*` | Common pitfalls and gotchas |
-
-## Vendor Field
-
-All wiki pages under `wiki/{vendor}/` carry an optional `vendor` field. When present, it must match the path vendor prefix. Pages may also carry `source_refs` for MANIFEST-backed external repository references (coexists with `sources` which holds wiki/source page IDs).
-
-## Required Frontmatter by Type
-
-### source-pr
-```yaml
-id: pr-cutlass-2472
-repo: NVIDIA/cutlass
-pr: 2472
-title: "Add Blackwell MLA forward"
-author: username
-date: 2025-07-16
-url: https://github.com/NVIDIA/cutlass/pull/2472
-source_category: upstream-code
-architectures: [sm100]
-tags: [mla, attention, prefill]
-techniques: [warp-specialization, pipeline-stages]
-hardware_features: [tcgen05, tmem, tma]
-kernel_types: [mla, attention, prefill]
-languages: [cute-dsl]
-captured_at: 2026-04-17
-status: merged
-merge_sha: abc12345
-inclusion_reason: "kernel file changes"
-changed_paths: [...]
-```
-
-### wiki-kernel (must have `performance_claims`)
-```yaml
-id: kernel-flash-attention-4
-title: "FlashAttention-4"
-type: kernel
-architectures: [sm100]
-tags: [attention, flash-attention, tcgen05, tmem, 2sm-cooperative]
-confidence: source-reported
-reproducibility: snippet
-kernel_types: [attention, flash-attention]
-languages: [cute-dsl]
-related: [technique-warp-specialization, technique-software-exp, hw-tcgen05-mma]
-sources: [doc-flash-attention-4, blog-flash-attention-4, pr-...]
-performance_claims:
-  - gpu: B200
-    dtype: bf16
-    shape: "seqlen=8192, headdim=128"
-    metric: TFLOPS
-    value: 1605
-    utilization: "71%"
-    source_id: doc-flash-attention-4
-```
-
-### wiki-pattern (diagnostic flow)
-```yaml
-id: pattern-memory-bound
-title: "Memory Bandwidth Bound"
-type: pattern
-tags: [vectorized-loads, cache-policy, shared-memory-optimization]
-symptoms: [memory-bound, low-compute-utilization, high-memory-throughput]
-candidate_techniques: [technique-vectorized-loads, technique-swizzling, technique-pipeline-stages]
-related: [pattern-compute-bound]
-sources: [...]
-```
-
-## Confidence Levels
-
-- **`verified`**: Either backed by official documentation plus upstream code, or independently reproduced for the exact conditions stated by an experience page.
-- **`source-reported`**: A traceable measurement reported by the original run or source, but not independently reproduced in the current verification workflow.
-- **`inferred`**: An observation, causal inference, or Agent/source report without corresponding direct measurement output.
-- **`experimental`**: Exploratory or legacy knowledge whose evidence is not yet stable enough for the other levels.
-
-## Reproducibility Levels
-
-For `wiki-technique`, `wiki-kernel`, `wiki-language`, must be ≥ `snippet`.
-
-| Level | Meaning |
-|-------|---------|
-| `concept` | Text only |
-| `pseudocode` | Language-agnostic algorithm |
-| `snippet` | Compilable code fragment (verified by validator) |
-| `runnable` | Self-contained buildable example |
-| `benchmarked` | Runnable + perf numbers with env metadata |
-
-## Controlled Vocabulary
-
-All values in these fields must appear in `data/tags.yaml`:
-
-- **architectures**: sm100, sm100a, sm90, sm90a, sm120
-- **hardware_features**: tcgen05, tmem, tma, clc, 2sm-cooperative, pdl, gdc, nvfp4, fp8, fp6, fp4, block-scale, wgmma, cluster, mbarrier
-- **techniques**: warp-specialization, persistent-kernel, swizzling, pipeline-stages, double-buffering, register-reuse, epilogue-fusion, tile-scheduling, tma-multicast, software-exp, fine-grained-quantization, cuda-core-promotion, jit-compilation, vectorized-loads, cache-policy, kernel-fusion, chunk-parallelism, loop-unrolling, register-budgeting, shared-memory-optimization, ping-pong-scheduling, conditional-rescaling, data-reuse, per-k-specialization
-- **kernel_types**: gemm, attention, moe, sparse-attention, gemv, grouped-gemm, gated-delta-net, fused-kernel, decode, prefill, quantization, flash-attention, mla, linear-attention, gated-dual-gemm, batched-gemv
-- **languages**: cuda-cpp, cute-dsl, triton, tilelang, cutile, ptx, python, jax-pallas
-- **source_category**: official-doc, upstream-code, paper, benchmark-blog, contest-report, community-note
-
-## Canonical Aliases (from data/aliases.yaml)
-
-When asking about:
-- UMMA → canonical tag is `tcgen05`
-- Tensor Memory / TMEM → `tmem`
-- Cluster Launch Control / CLC → `clc`
-- Blackwell / B200 → architecture `sm100`
-- Hopper / H100 → architecture `sm90`
-- MoE / Mixture of Experts → `moe`
-- MLA / Multi-head Latent Attention → `mla`
-- GDN / Gated Delta Net → `gated-delta-net`
-- NSA / Native Sparse Attention → `sparse-attention`
-- WGMMA / wgmma.mma_async → `wgmma`
-
-## Cross-Reference Fields
-
-- `sources`: list of source IDs whose content backs this wiki page
-- `related`: list of wiki page IDs that are topically related
-- `prerequisites`: list of wiki page IDs the reader should read first
-- `candidate_techniques` (pattern only): list of technique/hw/migration IDs that address the symptoms
-
-## Architecture-Neutral Scope
-
-SM90/Hopper, SM100/Blackwell, and general CUDA pages are all first-class. Architecture applicability is expressed with `architectures`; no architecture requires a special relevance-justification field.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/requirements.txt

@@ -1,2 +0,0 @@
-PyYAML>=6.0
-click>=8.0

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/research-contests.md

@@ -1,261 +0,0 @@
-# Research Results: GPU Mode NVFP4 Hackathon + FlashInfer MLSys 2026 Contest
-
-> Raw research data collected 2026-04-16. To be processed into source/wiki pages during implementation.
-
----
-
-# 1. GPU Mode Blackwell NVFP4 Kernel Hackathon
-
-## 1.1 Overview
-
-Four-part performance challenge hosted by NVIDIA + GPU MODE (Nov 2025 - Feb 2026), targeting B200 GPUs. Individual competition. Sesterce provided DGX B200 compute. Submissions via [Popcorn CLI](https://github.com/gpu-mode/popcorn-cli).
-
-**Reference Kernels Repository**: `gpu-mode/reference-kernels` → `/problems/nvidia/` contains: `nvfp4_gemv/`, `nvfp4_gemm/`, `nvfp4_dual_gemm/`, `modal_nvfp4_dual_gemm/`, `nvfp4_group_gemm/`.
-
-## 1.2 NVFP4 Data Format
-
-- 4-bit floating-point (E2M1): 1 sign, 2 exponent, 1 mantissa
-- Representable values: 0, 0.5, 1, 1.5, 2, 3, 4, 6 (positive and negative)
-- Block scaling: every 16 FP4 elements share one FP8 E4M3 scale factor
-- Two-level scaling: per-block E4M3 + per-tensor FP32 global scale
-- Dequantization: `x_hat_i = s_global * s_block * deq_FP4(q_i)`
-- E4M3 block scale (vs UE8M0 in MXFP4): non-power-of-two scales, smaller block (16 vs 32)
-
-## 1.3 Timeline and Prizes
-
-| Problem | Dates |
-|---|---|
-| Kernel 1: NVFP4 Batched GEMV | Nov 10 -- Nov 28, 2025 |
-| Kernel 2: NVFP4 GEMM | Nov 29 -- Dec 19, 2025 |
-| Kernel 3: NVFP4 Gated Dual GEMM | Dec 20, 2025 -- Jan 16, 2026 |
-| Kernel 4: NVFP4 Grouped GEMM | Jan 17 -- Feb 13, 2026 |
-
-Prizes per problem: 1st DGX Spark + GTC, 2nd RTX 5090 + GTC, 3rd RTX 5080.
-Grand Prize: Dell Pro Max with GB300. Weighted scoring: 10%/20%/30%/40% for problems 1-4.
-
-## 1.4 The Four Problems
-
-### Problem 1: NVFP4 Batched GEMV
-- Inputs: Matrix `a` (M×K×L, NVFP4), vector `b` (1×K×L, NVFP4), scales `sfa` (M×K/16×L, FP8), `sfb` (1×K/16×L, FP8)
-- Output: `c` (M×1×L, FP16)
-- Nature: Memory-bound (low arithmetic intensity)
-- Speed of light: ~8.6us for largest config
-- Benchmark configs: (M:7168, K:16384, L:1), (M:4096, K:7168, L:8), (M:7168, K:2048, L:4)
-
-### Problem 2: NVFP4 GEMM
-- Standard NVFP4 block-scaled GEMM
-- Compute-bound, uses tensor cores
-- Top: Simon 10.807us, yue 10.914us, currybab 10.931us (geometric mean)
-
-### Problem 3: NVFP4 Gated Dual GEMM
-- Two GEMMs (gate + up) → SiLU → element-wise multiply
-- Standard MLP gate-up pattern in modern LLMs
-- Key: fusion of GEMM + SiLU + multiply
-
-### Problem 4: NVFP4 Grouped GEMM
-- Multiple GEMMs with variable M, shared N and K
-- Directly relevant to MoE inference
-- Heaviest weight (40%) in grand prize
-
-## 1.5 Key Optimization Techniques
-
-### Memory-Bound (Problem 1 GEMV)
-
-**PTX-level control**:
-- Raw PTX `cvt.rn.f16x2.e2m1x2`, `ld.global` instead of C intrinsics
-- PTX byte unpacking `mov.b32 {tmp0, tmp1, tmp2, tmp3}` instead of bitwise extraction
-
-**Cache policy differentiation**:
-- Matrix A (streamed once): `L1::no_allocate` to avoid L1 pollution
-- Vector B (reused): `L1::evict_last` to keep hot
-- Rank 1 used different `ld.global` qualifiers per K-dimension variant
-
-**Register budgeting**:
-- Rank 1: `-maxrregcount=32`; Rank 3: `-maxrregcount=45`
-- Lower registers → higher occupancy → critical for memory-bound
-
-**Wider vectorized loads**:
-- 128-bit (`ld.global.v2.u64`) and 256-bit (`ld.global.v4.u64`)
-- Only effective when PTX byte unpacking avoids bitwise overhead
-
-**Per-K specialization**:
-- Separate kernel compilations per K-dimension with full loop unrolling
-- Different optimal configs for K=1024, K=3584, K=8192
-
-**Data reuse**:
-- Rank 2 shared B vector reads across BLOCK_M rows per thread block
-
-### Compute-Bound (Problems 2-4 GEMM)
-
-- Warp specialization with dedicated TMA and tensor core warps
-- CUTLASS `KernelPtrArrayTmaWarpSpecialized1SmNvf4Sm100` schedule
-- TMA for async bulk loads to shared memory
-- TMEM for MMA results (128×512 matrix, 32-bit elements)
-- 128-byte alignment for TMA
-
-## 1.6 Participant Blogs
-
-1. **[Yue's Journey](https://yue-zhang-2025.github.io/2025/12/02/blackwell-nvfp4-kernel-hackathon-journey.html)**: Problem 1. CuTe DSL (100us) → optimized CUDA (22.392us). Steps: coalesced access (2000→443us), hardware intrinsics (443→39us), PTX assembly (39→27us), ILP (27→22.9us).
-
-2. **[Twelve Attempts (Amandeep)](https://amandeepsp.github.io/blog/nvfp4-blackwell-gemv/)**: Problem 1. 12 approaches, final ~26.7us (3.1x off SOL). Key lesson: "Run Nsight Compute to confirm memory-bound behavior."
-
-3. **[Simon's posts](https://veitner.bearblog.dev/nvfp4-gemv/)**: CuTe DSL tutorial approach with block parallelization, shared memory reduction, atomic ops.
-
-## 1.7 The Reward Hack
-
-Submission to `nvfp4_group_gemm` reporting 11.191us (~2us ahead of next):
-- Correctness phase: harness clones data → real kernel runs correctly
-- Timing phase: harness reuses same objects → first call fires 120-group super-batch (all 15 problems fused), calls 2-15 skip
-- Led to improvements in FlashInfer-Bench evaluation methodology
-
-## 1.8 B200 Specs (Competition Context)
-- sm_100a, 142 SMs
-- 8 TB/s memory bandwidth
-- Native FP4 (E2M1) tensor core instructions
-- TMA, TMEM (128×512 × 32-bit per SM)
-- tcgen05.mma operates directly on shared memory
-
----
-
-# 2. FlashInfer AI Kernel Generation Contest (MLSys 2026)
-
-## 2.1 Overview
-
-One of three MLSys 2026 competitions. Targets B200 GPUs. Human, AI, or hybrid submissions.
-
-**URLs**:
-- Contest: [mlsys26.flashinfer.ai](https://mlsys26.flashinfer.ai/)
-- Starter kit: [flashinfer-ai/flashinfer-bench-starter-kit](https://github.com/flashinfer-ai/flashinfer-bench-starter-kit)
-- Agent baseline: [flashinfer-ai/mlsys26-agent-baseline](https://github.com/flashinfer-ai/mlsys26-agent-baseline)
-- Dataset: [flashinfer-ai/mlsys26-contest on HuggingFace](https://huggingface.co/datasets/flashinfer-ai/mlsys26-contest) (1.88 GB)
-
-## 2.2 Timeline
-
-| Date | Milestone |
-|---|---|
-| Jan 22, 2026 | Launch, registration opens |
-| Feb 5, 2026 | Full dataset released |
-| Feb 9, 2026 | Baselines released |
-| Feb 15, 2026 | Registration deadline (teams up to 5) |
-| Apr 24, 2026 | Kernel submission deadline |
-| May 1, 2026 | Technical writeup due (4 pages max) |
-| May 11, 2026 | Winners notified |
-| May 17-22, 2026 | Award ceremony, Bellevue, WA |
-
-## 2.3 Submission Format
-
-- Fork starter kit per track
-- Implement in `solution/triton/kernel.py` or `solution/cuda/kernel.cu` + `binding.py`
-- `python scripts/pack_solution.py` → `solution.json`
-- Tag commits for biweekly evaluations
-- DPS (Destination Passing Style) by default
-- CUDA binding via TVM FFI
-
-## 2.4 Track A: Fused MoE (FP8)
-
-**Benchmark**: `fp8_moe_block_scale_ds_routing_topk8_ng8_kg4_e32_h7168_i2048`
-
-**What**: Fused MoE with FP8 block-scale — routing, dispatch, dual GEMM (gate-up + down), SwiGLU, combine.
-
-**Parameters**: topk=8, num_groups=8, topk_group=4, num_experts=32, hidden=7168, intermediate=2048
-
-**Challenges**:
-1. No pre-tuned FP8 MoE config for B200
-2. FP8 numerical overflow requires careful block scaling (block size 128)
-3. Performance varies drastically by batch size
-4. Multiple kernel launches (7 in vLLM, 5 in SGLang fused)
-5. Variable expert load balancing
-6. TMA 128-byte alignment requirements
-
-**Baseline performance**:
-| Framework | Batch 4096 TFLOPS | Batch 1 Latency |
-|---|---|---|
-| SGLang | 1262 | 206.9us |
-| FlashInfer CuTeDSL | 1225 | 481.9us |
-| vLLM | 1117 | 369.5us |
-
-**API**: `flashinfer.fused_moe.trtllm_fp8_block_scale_moe()`
-
-## 2.5 Track B: DeepSeek V3.2 Sparse Attention
-
-**Benchmarks**: Indexer `fp8_h64_d128_topk2048_ps64`, Attention `h16_ckv512_kpe64_topk2048_ps64`
-
-**Two-stage**:
-1. **Lightning Indexer**: FP8 scorer, selects top-2048 tokens per query
-2. **Sparse Attention**: MLA over selected 2048 tokens only
-
-**Architecture**: MLA cache = 656 bytes/token (512 FP8 + 16 scales + 128 RoPE). Block size 64.
-
-**Challenges**:
-1. Continuous batching with varying sequence lengths
-2. Kernel padding overhead (TP=8 → 16 heads padded to 64)
-3. Variable sparsity patterns
-4. Two-kernel coordination
-5. "Nobody has fully unlocked performance potential" (vLLM blog)
-
-**References**: DeepGEMM for indexer, FlashMLA for sparse attention, TileLang for research.
-
-## 2.6 Track C: Gated Delta Net (Qwen3-Next)
-
-**Benchmarks**: Decode `qk4_v8_d128_k_last`, Prefill `qk4_v8_d128_k_last`
-
-**What**: Linear attention with delta rule mechanism (ICLR 2025, NVlabs). Used in Qwen3-Next-80B with 3:1 hybrid ratio (3 GDN : 1 full attention).
-
-**Parameters**: qk_dim=4, v_dim=8, d=128
-
-**Unique challenges**:
-1. O(n) linear complexity — different optimization targets than O(n²) attention
-2. Recurrent state management with learned decay
-3. Gating mechanism adds branching complexity
-4. Dual-mode: chunk-based prefill, streaming decode
-5. Variable-length support needed for production
-
-**Status** (FlashInfer issue #1690):
-- Prefill: Done on Hopper, in progress for Blackwell
-- Decode: Done for both Hopper and Blackwell
-
-**Implementations**: NVlabs/GatedDeltaNet (reference), FLA optimized (recommended, faster, varlen support).
-
-## 2.7 Agent Baseline
-
-[mlsys26-agent-baseline](https://github.com/flashinfer-ai/mlsys26-agent-baseline):
-- Iterative Agent: propose → refine via `str_replace`
-- Evolve Agent: multiple proposals → elite pool → evolution
-- Supports OpenAI + Claude models
-
-## 2.8 FlashInfer-Bench Leaderboard (Current)
-
-| Model | Avg Speedup | Resolved % |
-|---|---|---|
-| Gemini 2.5 Pro | 0.628x | 73.1% |
-| GPT-5 (2025-08-07) | 0.467x | 92.3% |
-| Claude Opus 4.1 | 0.456x | 73.1% |
-| GPT-O3 | 0.450x | 92.3% |
-
-All below 1.0x vs FlashInfer baselines — AI-generated kernels still lag behind human-optimized.
-
-## 2.9 K-Search
-
-Automated kernel generation framework using co-evolving world model to guide LLM optimization. 2.10x average improvement over OpenEvolve, up to 14.3x on complex MoE kernels.
-
----
-
-## Key Sources
-
-- [GPU MODE Hackathon (Luma)](https://luma.com/9n27uem4)
-- [NVIDIA Forums Announcement](https://forums.developer.nvidia.com/t/join-us-for-the-blackwell-nvfp4-kernel-hackathon-with-nvidia-and-gpu-mode/350092)
-- [Yue's Blog](https://yue-zhang-2025.github.io/2025/12/02/blackwell-nvfp4-kernel-hackathon-journey.html)
-- [Twelve Attempts (Amandeep)](https://amandeepsp.github.io/blog/nvfp4-blackwell-gemv/)
-- [Simon's Blog](https://veitner.bearblog.dev/nvfp4-gemv/)
-- [Reward Hack Writeup](https://www.gpumode.com/news/reward-hacking-nvfp4)
-- [TFLOPS Gap Blog](https://huggingface.co/blog/apsys/blackwell-nvfp4-comparison)
-- [NVFP4 Format Details](https://haroldbenoit.com/notes/ml/engineering/precision/nvfp4-format)
-- [NVIDIA NVFP4 Blog](https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/)
-- [gpu-mode/reference-kernels](https://github.com/gpu-mode/reference-kernels)
-- [MLSys 2026 Contest](https://mlsys26.flashinfer.ai/)
-- [FlashInfer Starter Kit](https://github.com/flashinfer-ai/flashinfer-bench-starter-kit)
-- [FlashInfer Agent Baseline](https://github.com/flashinfer-ai/mlsys26-agent-baseline)
-- [FlashInfer-Bench Paper](https://arxiv.org/abs/2601.00227)
-- [FlashInfer-Bench Leaderboard](https://bench.flashinfer.ai/)
-- [mlsys26-contest Dataset](https://huggingface.co/datasets/flashinfer-ai/mlsys26-contest)
-- [K-Search Paper](https://arxiv.org/abs/2602.19128)