@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/hardware/clc.md

@@ -1,307 +0,0 @@
----
-id: hw-clc
-title: "Cluster Launch Control (CLC)"
-type: hardware
-architectures: [sm100, sm100a]
-languages:
-- '*'
-tags: [clc, persistent-kernel, tile-scheduling]
-confidence: source-reported
-evidence_level: spec
-related: [technique-persistent-kernels, technique-tile-scheduling, pattern-tail-effect]
-sources: [doc-nvidia-tuning-guide, doc-cutlass-blackwell, pr-cutlass-2161]
-aliases: [CLC, "cluster launch control"]
----
-
-# Cluster Launch Control (CLC)
-
-## Overview
-
-Cluster Launch Control (CLC) is a Blackwell hardware mechanism for **dynamic tile scheduling** in persistent kernels. It replaces the static grid scheduling model where the CUDA runtime pre-assigns tile coordinates to CTAs at launch time.
-
-With CLC, persistent CTAs dynamically request work from a hardware queue, enabling:
-
-- **Better load balancing**: No fixed CTA-to-tile mapping; busy SMs consume tiles as they become available.
-- **Tail-effect mitigation**: The "tail" of a GEMM (when remaining tiles < number of SMs) is handled efficiently because idle CTAs pick up remaining work.
-- **Dynamic cancellation**: Tiles can be cancelled via `try_cancel` when the output is no longer needed (e.g., speculative decoding).
-
-## Static Scheduling vs CLC
-
-### Static Scheduling (Hopper and earlier)
-
-```
-Launch grid: 256 CTAs for 256 tiles
-CTA 0 -> tile (0,0)     [fixed at launch]
-CTA 1 -> tile (0,1)     [fixed at launch]
-CTA 2 -> tile (0,2)     [fixed at launch]
-...
-CTA 255 -> tile (15,15)  [fixed at launch]
-
-Problem: If SM count = 132, first wave = 132 CTAs.
-         Second wave = 124 CTAs -> 8 SMs idle = 6% waste.
-         For small GEMMs, tail effect dominates.
-```
-
-### CLC Dynamic Scheduling (Blackwell)
-
-```
-Launch grid: 132 persistent CTAs (= SM count)
-CTA 0: request tile -> get (0,0) -> compute -> request tile -> get (2,4) -> ...
-CTA 1: request tile -> get (0,1) -> compute -> request tile -> get (2,5) -> ...
-...
-CTA 131: request tile -> get (0,131) -> compute -> request tile -> ...
-
-All CTAs stay busy until the tile queue is empty.
-Tail: last few tiles distributed to first-available CTAs.
-```
-
-## How CLC Works
-
-### Hardware Queue
-
-CLC maintains a hardware-managed work queue. Each entry represents a tile coordinate (or cluster coordinate in multi-CTA setups). The queue is populated at kernel launch and drained by CTAs calling `clusterlaunchcontrol.try_acquire`.
-
-### CLC Programming Model
-
-```cuda
-__global__ void persistent_gemm_clc(
-    const half* A, const half* B, half* C,
-    int M, int N, int K,
-    int num_tiles_m, int num_tiles_n
-) {
-    // Allocate persistent resources (TMEM, pipeline state)
-    uint32_t tmem_acc = tmem_alloc(256);
-
-    // Shared storage for CLC results (visible to all threads in CTA)
-    __shared__ uint2 clc_tile_coord;
-    __shared__ int clc_has_tile;
-
-    // CLC tile loop: keep requesting tiles until none remain
-    while (true) {
-        // Thread 0 acquires the next tile; result goes to shared memory
-        if (threadIdx.x == 0) {
-            uint2 result;
-            int acquired = 0;
-            asm volatile(
-                "{\n"
-                "  .reg .pred p;\n"
-                "  clusterlaunchcontrol.try_cancel {%0, %1}, p;\n"
-                "  selp.s32 %2, 1, 0, p;\n"
-                "}\n"
-                : "=r"(result.x), "=r"(result.y), "=r"(acquired)
-            );
-            clc_tile_coord = result;
-            clc_has_tile = acquired;
-        }
-        __syncthreads();  // All threads see the shared result
-
-        // Exit if no more tiles
-        if (!clc_has_tile) break;
-
-        int tile_m = clc_tile_coord.x;
-        int tile_n = clc_tile_coord.y;
-
-        // Zero accumulator
-        tmem_zero(tmem_acc, 256);
-
-        // Mainloop: iterate over K dimension
-        for (int k = 0; k < K / TILE_K; ++k) {
-            // TMA load A and B tiles to SMEM
-            tma_load_a(smem_a, A, tile_m, k);
-            tma_load_b(smem_b, B, k, tile_n);
-            wait_barrier();
-
-            // Issue MMA
-            if (threadIdx.x == 0) {
-                asm volatile(
-                    "tcgen05.mma.cta_group::1.kind::f16 "
-                    "[%0], %1, %2, %3, 1;"
-                    :
-                    : "r"(tmem_acc), "l"(desc_a), "l"(desc_b), "r"(0)
-                );
-            }
-        }
-
-        // Epilogue
-        asm volatile("tcgen05.mma.fence::before_thread_sync;");
-        __syncthreads();
-        store_output(tmem_acc, C, tile_m, tile_n);
-    }
-
-    // Cleanup
-    tmem_dealloc(tmem_acc, 256);
-}
-```
-
-## try_cancel API
-
-CLC provides a `try_cancel` mechanism to cancel pending tiles. This is useful for speculative execution where some outputs may not be needed.
-
-```cuda
-// Cancel a specific tile if it hasn't started execution yet
-__device__ bool clc_try_cancel(uint2 tile_coord) {
-    bool cancelled = false;
-    if (threadIdx.x == 0) {
-        asm volatile(
-            "clusterlaunchcontrol.try_cancel.async.shared::cta "
-            "%0, [%1];"
-            : "=r"(cancelled)
-            : "r"(&tile_coord)
-        );
-    }
-    return cancelled;
-}
-```
-
-Use cases for `try_cancel`:
-- **Speculative decoding**: Cancel tiles for rejected draft tokens.
-- **Early termination**: If an attention mask makes certain output tiles zero, cancel them.
-- **Dynamic batching**: Cancel tiles for sequences that have finished.
-
-## CUTLASS Integration
-
-CUTLASS 4.5.0 for SM100 provides CLC support through the `PersistentScheduler` class:
-
-```cuda
-// CUTLASS SM100 persistent GEMM with CLC scheduling
-using Gemm = cutlass::gemm::device::GemmUniversal<
-    cutlass::half_t,              // ElementA
-    cutlass::layout::RowMajor,    // LayoutA
-    cutlass::half_t,              // ElementB
-    cutlass::layout::ColumnMajor, // LayoutB
-    cutlass::half_t,              // ElementC
-    cutlass::layout::RowMajor,    // LayoutC
-    float,                        // ElementAccumulator
-    cutlass::arch::OpClassTensorOp,
-    cutlass::arch::Sm100,         // Blackwell
-    // Tile shape: 128x256x64
-    cutlass::gemm::GemmShape<128, 256, 64>,
-    // Cluster shape
-    cutlass::gemm::GemmShape<2, 1, 1>,
-    // Use CLC persistent scheduler
-    cutlass::gemm::PersistentScheduler
->;
-
-// Launch: CUTLASS handles CLC internally
-Gemm gemm_op;
-gemm_op(args, workspace, stream);
-```
-
-### CUTLASS CLC Tile Scheduler
-
-```cpp
-// Simplified CUTLASS CLC scheduler logic
-struct ClcTileScheduler {
-    CUTLASS_DEVICE
-    WorkTileInfo get_next_work() {
-        WorkTileInfo work;
-        // Acquire next tile from CLC hardware
-        bool valid = clc_try_acquire(&work.tile_coord);
-        work.is_valid = valid;
-
-        if (valid) {
-            // Convert linear tile index to 2D coordinates
-            work.tile_m = work.tile_coord.x;
-            work.tile_n = work.tile_coord.y;
-
-            // Apply swizzle for L2 locality
-            apply_l2_swizzle(work.tile_m, work.tile_n);
-        }
-        return work;
-    }
-};
-```
-
-## Performance Impact
-
-CLC delivers significant performance gains, especially for small-to-medium GEMMs where tail effects dominate:
-
-### Tail Effect Mitigation
-
-| GEMM Size | Static Scheduler | CLC Scheduler | Improvement |
-|---|---|---|---|
-| 2048x2048 (small) | 86% SM utilization | 98% SM utilization | +14% |
-| 4096x4096 (medium) | 92% SM utilization | 98% SM utilization | +6.5% |
-| 8192x8192 (large) | 97% SM utilization | 99% SM utilization | +2% |
-
-The canonical benchmark from the "tcgen05 for dummies" tutorial shows the jump from 940 TFLOPS (pipelined, static scheduling) to **1476 TFLOPS** (persistent + CLC), approaching 98% of reference GEMM (1507 TFLOPS).
-
-### Why CLC Matters for Inference
-
-Production LLM inference typically hits shapes where tail effects are severe:
-
-```python
-# Typical LLM GEMM shapes during decode (batch_size=1-64)
-# M is small (batch * seq_len for decode), N and K are large (model dim)
-# Example: Llama-70B decode, batch=32
-M = 32     # small!
-N = 8192   # hidden dim
-K = 8192   # hidden dim
-
-# Tile = 128x256 -> tiles_m = 1, tiles_n = 32 -> only 32 tiles total
-# On B200 (132 SMs): 100 SMs idle with static scheduling
-# CLC: 32 persistent CTAs handle all 32 tiles efficiently
-```
-
-## CLC with 2-SM Cooperative Mode
-
-When using 2-SM cooperative MMA (`cta_group::2`), CLC distributes work in **cluster-sized units**:
-
-```cuda
-// 2-SM cooperative CLC: each acquisition gets a cluster-sized tile
-__device__ void cooperative_clc_loop() {
-    while (true) {
-        // Acquire tile for the 2-CTA cluster
-        ClusterTile tile;
-        bool valid = clc_try_acquire_cluster(&tile);
-        if (!valid) break;
-
-        // Both CTAs in the cluster share the tile
-        // CTA 0 handles rows 0-127, CTA 1 handles rows 128-255
-        int my_row_start = (blockIdx.x % 2) * 128;
-
-        // Issue cooperative MMA
-        if (threadIdx.x == 0) {
-            asm volatile(
-                "tcgen05.mma.cta_group::2.kind::f16 "
-                "[%0], %1, %2, %3, 1;"
-                :
-                : "r"(tmem_acc), "l"(desc_a), "l"(desc_b), "r"(0)
-            );
-        }
-        // ...
-    }
-}
-```
-
-## L2 Cache Swizzling with CLC
-
-CLC tile ordering can be customized with swizzle patterns to improve L2 cache hit rates:
-
-```cuda
-// Swizzle tile coordinates for better L2 locality
-// Tiles are visited in a Z-order (Morton) curve pattern
-__device__ void apply_l2_swizzle(int& tile_m, int& tile_n, int swizzle_bits) {
-    // Convert linear tile index to swizzled 2D coordinates
-    // This groups spatially adjacent tiles together, improving
-    // L2 reuse for the B matrix (shared across M tiles)
-    int linear = tile_m * num_tiles_n + tile_n;
-    int swizzle_mask = (1 << swizzle_bits) - 1;
-    int group = linear >> swizzle_bits;
-    int within = linear & swizzle_mask;
-
-    tile_m = group / num_tiles_n;
-    tile_n = (group % num_tiles_n) ^ (tile_m & swizzle_mask);
-}
-```
-
-## Comparison: CLC vs Software Persistent Scheduling
-
-| Feature | CLC (Hardware) | Software Atomics |
-|---|---|---|
-| Scheduling overhead | Near zero (hardware) | atomicAdd contention |
-| Tail-effect handling | Optimal | Good with careful design |
-| Cancellation | try_cancel API | Complex (flags + barriers) |
-| L2 swizzle | Configurable at launch | Manual implementation |
-| Portability | SM100+ only | SM70+ |
-| CUTLASS support | Built-in | Manual scheduler |

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/hardware/decompression-engine.md

@@ -1,52 +0,0 @@
----
-id: hw-decompression-engine
-title: "Hardware Decompression Engine (Blackwell)"
-type: hardware
-vendor: nvidia
-architectures: [sm100, sm100a]
-languages:
-- '*'
-tags: [nvfp4, cache-policy, decode]
-confidence: source-reported
-evidence_level: spec
-related: [hw-foundation-b200-specs, hw-tmem, hw-nvfp4]
-sources: []
-aliases: [decompression engine, nvCOMP, "hardware decompression"]
----
-# Hardware Decompression Engine (Blackwell)
-
-B200 contains a **hardware decompression engine that runs alongside the SMs**.
-It decompresses data on the way out of HBM, so a memory-bound kernel can trade
-HBM bytes for engine throughput without spending SM cycles on the unpacking.
-
-> ⚠️ **Unverified.** Everything on this page is transcribed from a vendor-derived
-> guideline document. We have not run nvCOMP on our own B200, so the throughput
-> figure in particular should be treated as a headline number, not a budget input.
-
-## What it does
-
-| Property | Value |
-|---|---|
-| Formats | LZ4, Snappy, Zstandard, GZIP, Bitcomp, ANS |
-| Throughput | ~539 GB/s (100 MB blocks, sub-millisecond latency) ⚠️ unverified |
-| Programming interface | NVIDIA **nvCOMP** library |
-| SM cost | none -- the engine runs in parallel with the SMs |
-
-## When it is worth using
-
-The engine only pays when **HBM read bandwidth is the binding constraint**. Two
-cases where that is typically true:
-
-- **Large-model inference.** Store quantized weights compressed in HBM and let the
-  engine expand them during the load. The win is in bytes moved, so it compounds
-  with low-precision formats rather than competing with them -- see
-  [NVFP4](nvfp4.md).
-- **Sparse operands.** Store sparse data RLE/Bitcomp-compressed and stream it
-  decompressed into TMEM for the MMA -- see [TMEM](tmem.md).
-
-## When it does not
-
-If the kernel is compute-bound, or if the working set already fits in the 126 MB
-L2, decompression adds a dependency without removing a bottleneck. Establish that
-`dram__bytes` is the limiter **before** reaching for it -- the profiling side of
-that question belongs to the `gpu-profiling` skill, not here.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/hardware/foundation/README.md

@@ -1,67 +0,0 @@
----
-id: hw-foundation-runtime-introspection
-title: Hardware Runtime Introspection
-type: hardware
-vendor: nvidia
-architectures:
-- sm90
-- sm90a
-languages:
-- '*'
-tags:
-- cuda-cpp
-confidence: source-reported
-evidence_level: spec
-related: []
-sources: []
-aliases: []
----
-# Runtime Introspection (MVP subset of booklet 07 §R)
-
-> This directory holds the contract and sample outputs for
-> `tools/kp_introspect.py`. The tool is implemented in M2; this README is the
-> stub contract it will populate.
-
-## MVP scope
-
-MVP implements **R1-R4 minus NVML**, which means four CLI subcommands, two of which are non-cached:
-
-| Subcommand | Cached? | Purpose |
-| --- | --- | --- |
-| `device-static`   | by `(gpu_uuid, driver)` | compute-capability / sm_count / shmem limits / reg limits / L2 / memory bus |
-| `kernel-static`   | by cubin sha256         | regs/thread / shmem_static / local_mem / ptx_version (from `cuFuncGetAttribute`) |
-| `kernel-dynamic`  | no                      | occupancy / limit_factor for a given (block_size, smem) tuple |
-| `bundle`          | no                      | aggregate of the above into one json |
-
-**Deferred** (M5+):
-- `device-dynamic` with NVML (clock/temp/power, `clock_policy` inference)
-- cubin loader fallbacks (cuobjdump)
-- multi-GPU bundle
-- `regen-spec-md` auto-write
-
-## Schema version
-
-MVP starts at `schema_version: 0.1.0`. Anything under this schema is forward-compatible only — fields may be added, never removed or renamed, until `0.2.0`.
-
-## Example bundle (filled in M3)
-
-`examples/bundle-h200.json` will be produced by running:
-
-```bash
-python -m tools.kp_introspect bundle \
-    --device 0 \
-    --binary artifacts/warp_reduce/probe \
-    --symbol warp_reduce_kernel \
-    --block 256 \
-    --smem 0 \
-    --output 00-foundation/hardware-spec/runtime-introspection/bundle-h200.json
-```
-
-After M3 this file will be the canonical reference for what a "good" bundle looks like and what every field should contain on H200. The key H200 facts from these JSON examples are also mirrored into `../h200-specs.md` for easier reading.
-
-## Consumed by
-
-- `benchmark-protocol.md` — every measured entry must bind a bundle.
-- `api-probing.md` Step 4 — every probe runs `kernel-static` at minimum.
-- `bottleneck-triage.md` Q2 — reads `kernel.dynamic.limit_factor` when available.
-- `tools/lint_knowledge.py` — verifies `artifacts.introspection` paths resolve to a bundle that contains the declared `measured_on` facts.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/hardware/foundation/b200-specs.md

@@ -1,146 +0,0 @@
----
-id: hw-foundation-b200-specs
-title: B200 Specs
-type: hardware
-vendor: nvidia
-architectures:
-- sm100
-- sm100a
-languages:
-- '*'
-tags:
-- cuda-cpp
-- tmem
-- tcgen05
-- nvfp4
-confidence: source-reported
-evidence_level: spec
-related:
-- hw-foundation-h200-specs
-- hw-tmem
-- hw-tcgen05-mma
-- hw-nvfp4
-- hw-2sm-cooperative
-sources: []
-aliases:
-- B200
-- GB200
-- Blackwell specs
-- sm_100a
----
-# NVIDIA B200 Hardware Reference Card
-
-Quick-reference for roofline analysis and kernel optimization decisions on Blackwell.
-Companion to [H200 Specs](h200-specs.md) — the two tables use the same row order so
-they can be read side by side.
-
-> ⚠️ **Provenance and verification status.** The numbers below are vendor-published
-> specifications transcribed from an internal `gpu-profiling-skill` guideline document.
-> **None of them has been re-measured on our own B200.** Treat every row as
-> `evidence_level: spec` — good enough to size a roofline, not good enough to
-> contradict a measurement. Rows we have measured will be moved to a
-> `## Measured Characteristics` section with machine, CUDA version and date,
-> the way [`foundations/memory/coalescing.md`](../../foundations/memory/coalescing.md) does it.
-
-## GPU Compute Architecture
-
-| Parameter | B200 | H200 |
-|---|---|---|
-| Architecture / Compute Capability | Blackwell / sm_100a (CC 10.0) | Hopper / sm_90a |
-| SMs | 148 (2 dies x 74 SM) | 132 |
-| Max Warps per SM | 64 | 64 |
-| Max Thread Blocks per SM | 32 | 32 |
-| Register File per SM | 64K x 32-bit | 64K x 32-bit |
-| Max Registers per Thread | 255 | 255 |
-| Warp Size | 32 threads | 32 threads |
-| Max Cluster Size (portable / non-portable) | 8 / 16 | 8 / 16 |
-
-**Compile target:** `-arch=sm_100a`, e.g.
-`nvcc -arch=sm_100a -lineinfo -O3 -o my_kernel my_kernel.cu`.
-The `a` suffix is required for `tcgen05` and the other architecture-specific
-features on this page; plain `sm_100` does not expose them.
-
-## Memory Hierarchy
-
-| Level | B200 | H200 |
-|---|---|---|
-| Shared Memory per SM (configurable) | up to 228 KB (227 KB usable) | up to 228 KB |
-| **Tensor Memory (TMEM) per SM** | **256 KB (512 columns x 128 lanes x 32-bit)** | **none** |
-| L2 Cache | 126 MB (GB200) | 50 MB |
-| HBM Capacity | 192 GB HBM3e | 141 GB HBM3e |
-| HBM Bandwidth | 8 TB/s | 4.8 TB/s |
-| NVLink Bandwidth | 1.8 TB/s (NVLink 5) | 900 GB/s (NVLink 4) |
-
-TMEM is the structural change with the widest blast radius: `tcgen05.mma`
-accumulators live there rather than in registers, which is why register pressure
-behaves differently on Blackwell — see [TMEM](../tmem.md) and
-[tcgen05 MMA](../tcgen05-mma.md).
-
-## Peak Compute Throughput
-
-Tensor Core, dense / sparse:
-
-| Precision | B200 dense | B200 sparse | H200 dense |
-|---|---|---|---|
-| FP4 | 9 PFLOPS | 18 PFLOPS | not supported |
-| FP8 | 4.5 PFLOPS | 9 PFLOPS | 1.98 PFLOPS |
-| FP16 / BF16 | 2.25 PFLOPS | 4.5 PFLOPS | 0.99 PFLOPS |
-| TF32 | 1.13 PFLOPS | 2.25 PFLOPS | 0.49 PFLOPS |
-| FP64 | 45 TFLOPS | -- | 66.9 TFLOPS |
-
-**FP4 is 4x the FP16 dense rate**, which is the whole argument for NVFP4 weights
-in LLM inference — see [NVFP4](../nvfp4.md).
-
-### FP64 relative to FP32 -- why the ratio matters more than the absolute
-
-The general rule a kernel author needs is **FP64 throughput is a fraction of FP32,
-and the fraction is architecture-dependent** -- roughly 1/2 on datacenter parts down
-to 1/64 on consumer parts. On B200 the Tensor Core FP64 rate (45 TFLOPS) is
-**lower than H200's** (66.9 TFLOPS): Blackwell spent its transistor budget on the
-low-precision path. A kernel that is FP64-bound does not automatically get faster
-by moving to B200.
-
-> ⚠️ **Unverified.** The 1/2-to-1/64 span is transcribed from the guideline
-> document and has not been checked against a per-architecture table. The two
-> absolute numbers are vendor-published.
-
-## Blackwell-specific features
-
-Each has its own page; this card only says which ones exist so a roofline
-calculation does not silently assume a Hopper feature set.
-
-| Feature | Page |
-|---|---|
-| 5th-gen Tensor Core, `tcgen05` | [tcgen05-mma](../tcgen05-mma.md) |
-| Tensor Memory | [tmem](../tmem.md) |
-| CTA Pair (2CTA, `cta_group::2`) | [2sm-cooperative](../2sm-cooperative.md) |
-| FP4/FP6/FP8 with block scaling | [nvfp4](../nvfp4.md) |
-| Hardware decompression engine | [decompression-engine](../decompression-engine.md) |
-
-## CUDA 13.2 toolchain notes
-
-Not hardware, but they change what the hardware above is reachable *through*.
-Kept as a short list here rather than as their own pages: we have not built or
-benchmarked any of them, and **a page that claims to be an API reference without
-verified code is worse than a pointer**.
-
-- **cuTile (CUDA Tile).** A tile-level programming model, `sm_80` and newer, with
-  Python and C++ interfaces. It generates the `tcgen05` + TMA + pipeline
-  scaffolding instead of making the author write inline PTX. For a new Blackwell
-  kernel this is the shortest route to using those units at all -- a starting
-  point, not a ceiling: a generated pipeline still has to be profiled and its
-  stage count tuned, see [pipeline-stages](../../techniques/pipeline-stages.md).
-- **cuBLAS grouped GEMM** gained experimental MXFP8 support. With CUDA Graphs it
-  allows device-side shape handling and no host round-trip, which is exactly the
-  cost described in
-  [host-device-sync](../../foundations/sync/host-device-sync.md).
-  NVIDIA quotes roughly a **4x** end-to-end gain for MoE-shaped workloads from
-  this combination. ⚠️ *Vendor-claimed, not reproduced here, and the baseline is
-  not stated -- treat it as an order of magnitude, not a target.*
-- **cuSOLVER FP64 emulation** on INT8 Tensor Cores accelerates QR / LU / Cholesky.
-  It is attractive here precisely because B200's FP64 rate is *lower* than H200's
-  while its INT8 rate is far higher.
-- **Nsight Compute** added report clustering, register-dependency visualization
-  and PM Sampling improvements.
-
-> ⚠️ **All four items are transcribed from a guideline document and unverified.**

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/hardware/foundation/h200-specs.md

@@ -1,111 +0,0 @@
----
-id: hw-foundation-h200-specs
-title: H200 Specs
-type: hardware
-vendor: nvidia
-architectures:
-- sm90
-- sm90a
-languages:
-- '*'
-tags:
-- cuda-cpp
-confidence: source-reported
-evidence_level: spec
-related: []
-sources: []
-aliases: []
----
-# NVIDIA H200 SXM Hardware Reference Card
-
-Quick-reference for roofline analysis and kernel optimization decisions.
-The H200 uses the same GH100 Hopper die as the H100 SXM5, with upgraded HBM3e memory.
-
-## GPU Compute Architecture
-
-| Parameter | Value |
-|---|---|
-| Architecture / Compute Capability | Hopper / sm_90a |
-| SMs | 132 (8 GPCs, 66 TPCs, 2 SMs/TPC) |
-| FP32 CUDA Cores per SM | 128 |
-| FP32 CUDA Cores total | 16,896 |
-| Tensor Cores per SM | 4 (4th-generation) |
-| Tensor Cores total | 528 |
-| GPU Boost Clock | 1830 MHz |
-| Max Warps per SM | 64 |
-| Max Thread Blocks per SM | 32 |
-| Warp Size | 32 threads |
-
-## Memory Hierarchy
-
-| Parameter | Value |
-|---|---|
-| HBM3e Capacity | 141 GB |
-| HBM3e Bandwidth | 4.8 TB/s |
-| L2 Cache | 50 MB |
-| L1 / Shared Memory per SM (combined) | 256 KB |
-| Max Shared Memory per SM | 228 KB |
-| Max Shared Memory per Thread Block | 227 KB |
-| Shared Memory Carveout Options | 0, 8, 16, 32, 64, 100, 132, 164, 196, 228 KB |
-| Register File per SM | 64K 32-bit registers (256 KB) |
-| Max Registers per Thread | 255 |
-
-## Peak Compute Throughput
-
-All values are for the H200 SXM (identical to H100 SXM5 compute). "Sparse" column uses 2:4 structured sparsity.
-
-### CUDA Core (non-Tensor)
-
-| Precision | Dense |
-|---|---|
-| FP64 | 33.5 TFLOPS |
-| FP32 | 66.9 TFLOPS |
-| FP16 | 133.8 TFLOPS |
-| BF16 | 133.8 TFLOPS |
-
-### Tensor Core
-
-| Precision | Dense | Sparse (2:4) |
-|---|---|---|
-| FP64 | 66.9 TFLOPS | -- |
-| TF32 | 494.7 TFLOPS | 989.4 TFLOPS |
-| FP16 | 989.4 TFLOPS | 1,978.9 TFLOPS |
-| BF16 | 989.4 TFLOPS | 1,978.9 TFLOPS |
-| FP8 (E4M3/E5M2) | 1,978.9 TFLOPS | 3,957.8 TFLOPS |
-| INT8 | 1,978.9 TOPS | 3,957.8 TOPS |
-
-## Roofline Key Numbers
-
-These are the numbers to plug directly into roofline calculations:
-
-| Metric | Value |
-|---|---|
-| HBM Bandwidth (B_mem) | 4.8 TB/s = 4,800 GB/s |
-| L2 Bandwidth (approx) | ~12 TB/s (estimated from H100) |
-| Peak FP16 Tensor FLOPS (P) | 989.4 TFLOPS |
-| Peak FP8 Tensor FLOPS | 1,978.9 TFLOPS |
-| Arithmetic Intensity Crossover FP16 TC | 989.4 / 4.8 = ~206 FLOP/Byte |
-| Arithmetic Intensity Crossover FP8 TC | 1,978.9 / 4.8 = ~412 FLOP/Byte |
-
-## Differences from H100 SXM5
-
-| Parameter | H100 SXM5 | H200 SXM |
-|---|---|---|
-| HBM Type | HBM3 | HBM3e |
-| HBM Capacity | 80 GB | 141 GB |
-| HBM Bandwidth | 3.35 TB/s | 4.8 TB/s |
-| Compute (all precisions) | Same | Same |
-| TDP | 700W | 700W |
-
-## Interconnect
-
-| Parameter | Value |
-|---|---|
-| NVLink (4th gen) | 18 links, 900 GB/s bidirectional |
-| PCIe | Gen 5, 128 GB/s bidirectional |
-
-## Sources
-
-- NVIDIA H200 Tensor Core GPU Datasheet (SC24)
-- NVIDIA H100 Tensor Core GPU Architecture Whitepaper
-- CUDA Hopper Tuning Guide (CUDA Toolkit 13.2)