@kernaut/kernaut-resources 0.1.17-beta.1 → 0.1.18

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/compute/fast-math/apis.md

@@ -1,222 +0,0 @@
----
-func_name: __expf
-namespace: math
-header: math_functions.h
-signature: float __expf(float x)
-status: documented
-has_end_to_end_example: true
-source:
-- path: spec
-  anchor: Reference
-id: api-fast-math-ref
-type: api-definition
-vendor: nvidia
-title: Apis
-architectures:
-- '*'
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L26993-L26995
-languages:
-- ptx
-- cuda-cpp
-techniques:
-- kernel-fusion
-- software-exp
-kernel_types:
-- fused-kernel
-confidence: inferred
-evidence_level: inferred
-tags:
-- kernel-fusion
-- software-exp
-- fused-kernel
-- ptx
-- cuda-cpp
----
-# Fast Math Intrinsics API Reference
-
-This file lists the APIs touched by the fast-math skill. Each entry records the function name, namespace, signature, semantics, ULP error bound, and upstream source reference.
-
-## Approximate Transcendental Intrinsics (Single-Precision Only)
-
-### `__expf`
-
-- **Namespace**: math (CUDA intrinsic)
-- **Header**: `math_functions.h` (included via `cuda_runtime.h`)
-- **Signature**: `float __expf(float x)`
-- **PTX**: compiles to `ex2.approx.f32` with range-reduction scaling
-- **Semantics**: Computes `e^x` using the SFU approximate exponential.
-- **Max ULP error**: `2 + floor(abs(1.173 * x))`
-- **Source**: Programming guide Table 58, L26993-L26995
-- **Probed**: expf-vs-fast-expf
-
-### `__exp10f`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __exp10f(float x)`
-- **Semantics**: Computes `10^x`.
-- **Max ULP error**: `2 + floor(abs(2.97 * x))`
-- **Source**: Programming guide Table 58, L26997-L26999
-
-### `__logf`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __logf(float x)`
-- **PTX**: compiles to `lg2.approx.f32` with base-conversion scaling
-- **Semantics**: Computes `ln(x)`.
-- **Max ULP error**: `2^-21.41` absolute error for `x in [0.5, 2]`; 3 ULP otherwise
-- **Source**: Programming guide Table 58, L27005-L27007
-
-### `__log2f`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __log2f(float x)`
-- **PTX**: `lg2.approx.f32`
-- **Semantics**: Computes `log2(x)`.
-- **Max ULP error**: `2^-22` absolute error for `x in [0.5, 2]`; 2 ULP otherwise
-- **Source**: Programming guide Table 58, L27009-L27011
-
-### `__log10f`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __log10f(float x)`
-- **Semantics**: Computes `log10(x)`.
-- **Max ULP error**: `2^-24` absolute error for `x in [0.5, 2]`; 3 ULP otherwise
-- **Source**: Programming guide Table 58, L27013-L27015
-
-### `__sinf`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __sinf(float x)`
-- **PTX**: `sin.approx.f32`
-- **Semantics**: Computes `sin(x)`.
-- **Max ULP error**: `2^-21.41` absolute error for `x in [-pi, pi]`; larger otherwise
-- **Source**: Programming guide Table 58, L27017-L27019
-
-### `__cosf`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __cosf(float x)`
-- **PTX**: `cos.approx.f32`
-- **Semantics**: Computes `cos(x)`.
-- **Max ULP error**: `2^-21.41` absolute error for `x in [-pi, pi]`; larger otherwise
-- **Source**: Programming guide Table 58, L27021-L27023
-
-### `__sincosf`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `void __sincosf(float x, float* sptr, float* cptr)`
-- **Semantics**: Computes `sin(x)` and `cos(x)` simultaneously.
-- **Max ULP error**: Component-wise, same as `__sinf` and `__cosf`.
-- **Source**: Programming guide Table 58, L27025-L27027
-
-### `__tanf`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __tanf(float x)`
-- **Semantics**: Computes `tan(x)`.
-- **Max ULP error**: Derived from `__sinf(x) * (1 / __cosf(x))`
-- **Source**: Programming guide Table 58, L27029-L27031
-
-### `__tanhf`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __tanhf(float x)`
-- **Semantics**: Computes `tanh(x)`.
-- **Max ULP error**: Max relative error `2^-11`. Subnormal results are not flushed even under `-ftz=true`.
-- **Source**: Programming guide Table 58, L27033-L27035
-
-### `__powf`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __powf(float x, float y)`
-- **Semantics**: Computes `x^y`.
-- **Max ULP error**: Derived from `exp2f(y * __log2f(x))` -- compounds errors.
-- **Source**: Programming guide Table 58, L27001-L27003
-
-### `__fdividef`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __fdividef(float x, float y)`
-- **Semantics**: Computes `x / y`.
-- **Max ULP error**: 2 ULP for `|y| in [2^-126, 2^126]`. Returns 0 for `|y| > 2^126`.
-- **Source**: Programming guide Table 58, L26985-L26988
-
-## Rounded Arithmetic Intrinsics (IEEE-754 Exact, 0 ULP)
-
-### `__fadd_rn`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __fadd_rn(float x, float y)`
-- **Semantics**: `x + y` with round-to-nearest-even. Prevents FFMA contraction.
-- **Max ULP error**: 0 (IEEE-754 compliant)
-- **Source**: Programming guide Table 57, L26937-L26938
-
-### `__fmul_rn`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __fmul_rn(float x, float y)`
-- **Semantics**: `x * y` with round-to-nearest-even. Prevents FFMA contraction.
-- **Max ULP error**: 0
-- **Source**: Programming guide Table 57, L26945-L26947
-
-### `__fmaf_rn`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __fmaf_rn(float x, float y, float z)`
-- **Semantics**: `x * y + z` as a single fused multiply-add. Round-to-nearest-even.
-- **Max ULP error**: 0
-- **Source**: Programming guide Table 57, L26949-L26951
-
-### `__frcp_rn`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __frcp_rn(float x)`
-- **Semantics**: `1 / x` with round-to-nearest-even.
-- **Max ULP error**: 0
-- **Source**: Programming guide Table 57, L26957-L26959
-
-### `__fsqrt_rn`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __fsqrt_rn(float x)`
-- **Semantics**: `sqrt(x)` with round-to-nearest-even.
-- **Max ULP error**: 0
-- **Source**: Programming guide Table 57, L26961-L26963
-
-### `__frsqrt_rn`
-
-- **Namespace**: math
-- **Header**: `math_functions.h`
-- **Signature**: `float __frsqrt_rn(float x)`
-- **Semantics**: `1 / sqrt(x)` with round-to-nearest-even.
-- **Max ULP error**: 0
-- **Source**: Programming guide Table 58, L26989-L26991
-
-### `rsqrtf`
-
-- **Namespace**: math (CUDA Math API, not prefixed with `__`)
-- **Header**: `math_functions.h`
-- **Signature**: `float rsqrtf(float x)`
-- **Semantics**: Fast reciprocal square root. Maps to SFU `rsqrt.approx.f32`.
-- **Max ULP error**: 2 ULP
-- **Source**: Programming guide standard functions table, L26781; Best-practices guide L1372
-- **Note**: Recommended to invoke directly rather than writing `1.0f / sqrtf(x)`, which the compiler only optimizes to rsqrt under specific flags.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/compute/fast-math/pitfalls.md

@@ -1,126 +0,0 @@
----
-title: Fast Math Intrinsics - Pitfalls
-status: draft
-evidence_level: spec
-applies_to_pattern_class:
-- cuda-core
-applies_to_ops:
-- elementwise
-- activation
-- normalization
-- softmax
-requires_sm: '>=3.0'
-single_kernel_useful: true
-source:
-- path: spec
-  anchor: Reference
-id: pitfall-fast-math
-type: pitfall
-vendor: nvidia
-architectures:
-- '*'
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L26993-L26995
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L27041-L27045
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L25507
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1561
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1370-L1374
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L27017-L27023
-languages:
-- cuda-cpp
-techniques:
-- pipeline-stages
-- software-exp
-kernel_types:
-- fused-kernel
-- attention
-confidence: source-reported
-tags:
-- pipeline-stages
-- software-exp
-- fused-kernel
-- attention
-- cuda-cpp
----
-## P1: Input-dependent ULP error for __expf
-
-**Symptom**: Precision degrades progressively as `|x|` increases, eventually producing results with tens or hundreds of ULP error.
-
-**Cause**: The documented max ULP error for `__expf(x)` is `2 + floor(abs(1.173 * x))` (programming guide Table 58, L26993-L26995). This is not a fixed bound like standard `expf` (which has 2 ULP max). For `|x| = 10`, the theoretical worst case is 13 ULP; for `|x| = 80` (near overflow), it is 95 ULP.
-
-**Fix**: Restrict `__expf` to inputs with bounded magnitude. For activation functions like sigmoid (`expf(-x)` where `x` is a pre-activation value), inputs are typically in `[-10, 10]` where the error stays below ~13 ULP. For larger ranges, use standard `expf`.
-
-**Detection**: Compare kernel output against a double-precision reference on representative inputs covering the full expected range.
-
-## P2: --use_fast_math is global and includes hidden side effects
-
-**Symptom**: Enabling `--use_fast_math` changes not only transcendental function precision but also division precision, subnormal handling, and FMA contraction behavior throughout the entire compilation unit.
-
-**Cause**: `--use_fast_math` implies three additional flags (programming guide L25507, L25993-L25996):
-- `-ftz=true`: flushes all subnormal (denormalized) fp32 values to zero.
-- `-prec-div=false`: replaces `/` with `__fdividef` (2 ULP).
-- `-fmad=true`: allows contraction of `a*b + c` into FFMA (changes rounding behavior).
-
-These side effects affect every floating-point operation in the file, not just transcendentals.
-
-**Fix**: Do not use `--use_fast_math` globally. Instead, call fast intrinsics explicitly (`__expf`, `__sinf`, etc.) only in the specific code paths where reduced precision is acceptable (programming guide L27045: "A more robust approach is to selectively replace mathematical function calls with intrinsic versions only where the performance gains justify it").
-
-## P3: __sinf / __cosf lose accuracy outside [-pi, pi]
-
-**Symptom**: Large absolute errors for trigonometric functions called with large arguments.
-
-**Cause**: `__sinf(x)` and `__cosf(x)` have `2^-21.41` absolute error for `x in [-pi, pi]`, but the error is documented as "larger otherwise" (programming guide Table 58, L27017-L27023). Standard `sinf` performs multi-precision argument reduction to handle large arguments correctly, but `__sinf` skips this reduction entirely. For large `|x|`, the argument- reduction error dominates, and the result can be arbitrarily wrong.
-
-**Fix**: Ensure inputs to `__sinf` / `__cosf` are reduced to `[-pi, pi]` before calling the intrinsic. If the application naturally produces arguments in this range (e.g., phase angles), the intrinsic is safe. For unbounded inputs, use standard `sinf` / `cosf`.
-
-## P4: __fdividef produces 0 for large denominators
-
-**Symptom**: Division returns 0 instead of a small but finite result.
-
-**Cause**: `__fdividef(x, y)` has 2 ULP accuracy only for `|y| in [2^-126, 2^126]` (programming guide Table 58, L26985-L26988). For `|y| > 2^126`, the function returns 0 regardless of `x`. Standard division correctly handles these cases.
-
-**Fix**: Guard `__fdividef` calls against extreme denominator magnitudes, or use standard division when the denominator range is not known.
-
-## P5: rsqrtf not generated automatically
-
-**Symptom**: `1.0f / sqrtf(x)` does not compile to a single `rsqrt` instruction, resulting in slower code than expected.
-
-**Cause**: The compiler can only optimize `1.0f / sqrtf(x)` into `rsqrtf(x)` when both `-prec-div=false` and `-prec-sqrt=false` are active (best-practices guide L1372). With default compiler flags, the reciprocal and square root remain separate operations.
-
-**Fix**: Call `rsqrtf(x)` directly instead of writing `1.0f / sqrtf(x)`. This gives 2 ULP accuracy in a single SFU operation regardless of compiler flags.
-
-## P6: -ftz=true silently changes results near zero
-
-**Symptom**: Computations involving very small (subnormal) floating-point values produce zero instead of a tiny nonzero result.
-
-**Cause**: When `-ftz=true` is active (either explicitly or via `--use_fast_math`), all subnormal fp32 values are flushed to zero. This affects any computation whose intermediate or final result falls in the subnormal range `(0, 1.175e-38)`.
-
-**Fix**: If the application requires correct subnormal behavior (e.g., computing log-probabilities of rare events, where `expf(x)` for very negative `x` produces a subnormal), do not use `-ftz=true` or `--use_fast_math`. Use explicit `__expf` calls instead, which respect the `-ftz` flag but can be controlled separately.
-
-## P7: No speedup in memory-bound kernels
-
-**Symptom**: Switching from `expf` to `__expf` produces no measurable throughput improvement.
-
-**Cause**: When a kernel is bottlenecked by global memory bandwidth (e.g., a single `expf` call per element loaded from HBM), the compute pipeline is underutilized and both `expf` and `__expf` complete within the memory latency window. The probe confirmed this: with 1 call per element on a 4M-element array, both versions ran at ~0.015 ms with no speedup.
-
-**Fix**: The fast-math speedup materializes only in compute-bound regions where transcendental operations dominate the instruction mix. Confirm that the kernel is compute-bound (e.g., via NCU's `sm__throughput` metric) before applying this optimization.
-
-## P8: __powf accumulates errors from __log2f and exp2f
-
-**Symptom**: `__powf(x, y)` produces larger errors than expected for a single intrinsic call.
-
-**Cause**: `__powf(x, y)` is implemented as `exp2f(y * __log2f(x))` (programming guide Table 58, L27001-L27003). It compounds errors from both `__log2f` (2 ULP) and `exp2f`, so the resulting error can exceed what users expect from a "fast" single function call.
-
-**Fix**: For small integer exponents, use explicit multiplication (`x*x`, `x*x*x`) which is faster and exact. For base-2 or base-10 exponentiation, use `exp2f` / `exp10f` directly (best-practices guide L1572). Only use `__powf` when the exponent is a non-integer runtime value and precision is not critical.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/compute/fast-math.md

@@ -1,216 +0,0 @@
----
-title: Fast Math Intrinsics
-status: draft
-evidence_level: measured
-applies_to_pattern_class:
-- cuda-core
-applies_to_ops:
-- elementwise
-- activation
-- normalization
-- softmax
-requires_sm: '>=3.0'
-requires_features: []
-single_kernel_useful: true
-cuda_version_tested: '12.9'
-driver_version_tested: 570.124.06
-toolchain: nvcc 12.9 + ptxas 12.9
-measured_on: H200-SXM
-source:
-- path: spec
-  anchor: Reference
-artifacts:
-  code: artifacts/experience/hw-probes/fast-math/expf_probe.cu
-  build: nvcc -arch=sm_90a -O3 -std=c++17 -lineinfo -o expf_probe expf_probe.cu
-  introspection: ''
-  profile: ''
-related_apis:
-- __expf
-- __logf
-- __sinf
-- __cosf
-- __powf
-- __fdividef
-- __fmaf_rn
-related_skills:
-- warp-primitives
-- coalescing
-id: skill-fast-math
-type: skill
-vendor: nvidia
-tags:
-- cuda-cpp
-- pipeline-stages
-- software-exp
-- fused-kernel
-- attention
-applies_to:
-- general
-source_refs:
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L26909-L26916
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L26969-L27038
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L27041-L27094
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1243-L1244
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1556-L1581
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L26917-L26967
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-programming-guide/cuda_cuda-programming-guide_index.html.md
-  anchor: L25993-L25996
-- source_id: cuda-official/toolkit-docs-13.2
-  path: CUDA Programming Guides/cuda-c-best-practices-guide/cuda_cuda-c-best-practices-guide_index.html.md
-  anchor: L1370-L1374
-architectures:
-- sm90
-- sm90a
-languages:
-- cuda-cpp
-techniques:
-- pipeline-stages
-- software-exp
-kernel_types:
-- fused-kernel
-- attention
-confidence: experimental
-artifact_dir: artifacts/experience/hw-probes/fast-math
----
-## What
-
-CUDA fast math intrinsics are hardware-accelerated approximate versions of standard C math functions, available only in device code. They trade precision for throughput by mapping more directly to the GPU's Special Function Unit (SFU) hardware.
-
-There are two families:
-
-**Approximate transcendentals** (single-precision only): `__sinf`, `__cosf`, `__expf`, `__logf`, `__log2f`, `__log10f`, `__exp10f`, `__powf`, `__sincosf`, `__tanf`, `__tanhf`, `__fdividef`. These have reduced precision (documented ULP error bounds in the programming guide Table 58) but are faster because they use fewer native instructions. The SFU processes approximate operations at 16 ops/clock/SM on all compute capabilities (best-practices guide Table 5).
-
-**Rounded arithmetic intrinsics** (IEEE-754 exact, 0 ULP): `__fadd_rn`, `__fmul_rn`, `__fmaf_rn`, `__fsub_rn`, `__fdiv_rn`, `__frcp_rn`, `__fsqrt_rn`, `__frsqrt_rn` (and `_rz`, `_ru`, `_rd` variants). These have zero ULP error but prevent the compiler from fusing operations into FMA instructions, which is useful when strict IEEE-754 rounding semantics are needed. `__frsqrt_rn` gives 0 ULP reciprocal square root.
-
-The nvcc compiler flag `--use_fast_math` globally replaces standard math functions with their `__` intrinsic counterparts for all single-precision calls. It also implies `-ftz=true` (flush denormals to zero), `-prec-div=false` (imprecise division), and `-fmad=true` (enable FMA contraction). The programming guide (Section 5.5.9.3, Table 59) lists the complete set of affected functions.
-
-## Precision has a throughput ratio, and it is architecture-dependent
-
-Before reaching for a faster *function*, check whether the *type* is the cost.
-**FP64 throughput is a fraction of FP32, and the fraction moves by an order of
-magnitude across parts** -- roughly 1/2 on datacenter silicon down to about 1/64
-on consumer silicon. Two concrete points from the spec cards:
-[H200](../../hardware/foundation/h200-specs.md) 33.5 TFLOPS FP64 CUDA-core, and
-[B200](../../hardware/foundation/b200-specs.md) 45 TFLOPS FP64 Tensor -- **lower
-than H200's 66.9**, because Blackwell spent its budget on the low-precision path.
-A kernel that is FP64-bound does not get faster by moving to a newer datacenter part.
-
-⚠️ *The 1/2-to-1/64 span is transcribed from a guideline document and has not been
-checked against a per-architecture table. The two absolute figures are vendor-published.*
-
-## Why
-
-Standard math library functions like `expf` and `sinf` are implemented as multi-instruction software sequences. They perform range reduction, polynomial approximation, and IEEE-754 rounding correction to achieve 1-2 ULP accuracy. This costs many instructions per call.
-
-Fast math intrinsics like `__expf` bypass most of these correction steps and map more directly to the SFU `ex2.approx.f32` (or `lg2.approx.f32`, `sin.approx.f32`, etc.) instructions. The SFU has a dedicated pipeline with throughput of 16 ops/clock/SM on sm_9.0, compared to 128 ops/clock/SM for basic fp32 add/multiply. Standard `expf` issues multiple instructions including SFU calls plus correction arithmetic, so its effective throughput is lower than the SFU's raw 16 ops/clock rate. `__expf` reduces that instruction overhead.
-
-Measured on H200 (sm_90a), `__expf` achieves 1.27x higher throughput than `expf` in a compute-bound loop (3318 Gop/s vs 2604 Gop/s). See Measured Characteristics below for the full probe record.
-
-## When to use
-
-- **Activation functions** (GeLU, SiLU/Swish, sigmoid, softmax-exp): these apply expf per element and tolerate the ~8 ULP error of `__expf` because the downstream operations (softmax normalization, gradient computation at reduced precision) absorb the small inaccuracy.
-
-- **Normalization inverse-root** (LayerNorm, RMSNorm): `rsqrtf` (fast reciprocal square root, 2 ULP from Table 58) is the canonical single-instruction path. The compiler will use `rsqrtf` automatically only when `-prec-div=false` and `-prec-sqrt=false` are both active; the best-practices guide (L1372) recommends invoking `rsqrtf()` explicitly.
-
-- **Softmax denominator**: the exp inside softmax is typically followed by a reduction sum, so per-element errors of 2-8 ULP are negligible relative to the sum magnitude.
-
-- **Fast division**: `__fdividef(x, y)` gives 2 ULP error for `|y| in [2^-126, 2^126]` (Table 58, L26985-L26988). Faster than the default `/` operator when IEEE-754 precision is not required.
-
-- **Trigonometric functions in signal processing kernels**: `__sinf` / `__cosf` are accurate to `2^-21.41` absolute error within `[-pi, pi]` (Table 58, L27017-L27023). Outside this range, error grows because the intrinsic skips the multi-precision argument reduction that the standard `sinf` performs.
-
-## When NOT to use
-
-- **Loss computation**: Cross-entropy and similar losses involve `log` of softmax outputs. Precision errors in `__logf` (3 ULP outside `[0.5, 2]`) can produce incorrect gradients and destabilize training.
-
-- **Gradient accumulation**: FP32 gradient accumulators are used precisely to maintain precision. Substituting `__expf` or `__logf` in the backward pass undermines this goal.
-
-- **Numerically sensitive reductions**: any code path where small per-element errors compound (e.g., prefix scan of log-probabilities, Kahan summation) should use standard library functions.
-
-- **Large-argument trigonometry**: `__sinf(x)` for `|x| >> pi` gives unbounded error because it skips argument reduction. Standard `sinf` handles this correctly (at the cost of local memory for argument reduction when `|x|` is large, per best-practices guide L1561).
-
-- **Mixed-precision training master weights**: The fp32 master copy of weights is updated in full precision. Fast math intrinsics in the update step corrupt the precision advantage of maintaining fp32 weights.
-
-- **`--use_fast_math` on the entire compilation unit**: This flag globally downgrades every single-precision function. A more robust approach is to selectively call `__expf` / `__sinf` / etc. only where the precision loss is acceptable (programming guide L27045).
-
-## Classical example: fast sigmoid
-
-```cuda
-// Standard sigmoid: uses expf (multi-instruction)
-__device__ __forceinline__ float sigmoid_std(float x) {
-    return 1.0f / (1.0f + expf(-x));
-}
-
-// Fast sigmoid: uses __expf (SFU-direct)
-__device__ __forceinline__ float sigmoid_fast(float x) {
-    return 1.0f / (1.0f + __expf(-x));
-}
-
-// Even faster: use __fdividef to also speed up the division
-__device__ __forceinline__ float sigmoid_fastest(float x) {
-    return __fdividef(1.0f, 1.0f + __expf(-x));
-}
-
-__global__ void apply_sigmoid(const float* __restrict__ in,
-                              float*       __restrict__ out, int n) {
-    int i = blockIdx.x * blockDim.x + threadIdx.x;
-    if (i < n)
-        out[i] = sigmoid_fast(in[i]);
-}
-```
-
-For softmax:
-
-```cuda
-// Fast softmax numerator: __expf for the exponential
-__device__ float fast_softmax_exp(float x, float row_max) {
-    return __expf(x - row_max);  // subtract max for numerical stability
-}
-
-// Fast reciprocal for normalization
-__device__ float fast_softmax_norm(float x, float sum) {
-    return __fdividef(x, sum);
-}
-```
-
-For LayerNorm inverse root:
-
-```cuda
-// Direct rsqrtf call (2 ULP, recommended over 1.0f/sqrtf(x))
-__device__ float fast_inv_sqrt(float variance_plus_eps) {
-    return rsqrtf(variance_plus_eps);
-}
-```
-
-## Measured Characteristics
-
-- expf-vs-fast-expf probe: On H200 (sm_90a, CUDA 12.9), a compute-bound loop of 128 chained expf calls per thread (1M threads) measured `__expf` at **0.0404 ms** (3318 Gop/s) vs standard `expf` at **0.0516 ms** (2604 Gop/s), a **1.27x speedup**. Single-call precision: `expf` max ULP error = 2, `__expf` max ULP error = 8, both measured against double-precision reference over 1M random inputs in [-10, 10]. The speedup vanishes in memory-bound kernels where both versions are bottlenecked by global memory bandwidth.
-
-### Refuted: "transcendentals are 10x slower than their intrinsics"
-
-A widely repeated guideline says FP32 transcendental functions (`sinf`, `cosf`,
-`expf`) are **more than 10x slower** than the fast-math intrinsics. On H200 the
-measured gap is **1.27x** (`__expf` 3318 Gop/s vs `expf` 2604 Gop/s in a
-compute-bound loop -- see *Measured Characteristics*).
-
-The 10x figure most likely comes from conflating two different comparisons: the
-SFU pipeline runs at 16 ops/clock/SM against 128 ops/clock/SM for basic FP32
-add/multiply (**8x**), but that is *transcendental vs arithmetic*, not
-*`expf` vs `__expf`*. Both forms use the SFU; the intrinsic only skips the
-correction arithmetic around it.
-
-This matters for where the effort goes: at 1.27x, swapping in `__expf` is worth
-doing in an elementwise activation kernel and is not worth an accuracy argument
-anywhere else.

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/compute/gemm/pitfalls.md

@@ -1,52 +0,0 @@
----
-id: pitfall-gemm
-type: pitfall
-vendor: nvidia
-title: Pitfalls
-architectures:
-- sm90
-- sm90a
-languages:
-- cuda-cpp
-hardware_features:
-- wgmma
-- tma
-- cluster
-techniques:
-- warp-specialization
-- tma-multicast
-kernel_types:
-- gemm
-confidence: inferred
-evidence_level: inferred
-tags:
-- wgmma
-- tma
-- cluster
-- warp-specialization
-- tma-multicast
-- gemm
-- cuda-cpp
----
-# Non-aligned GEMM tail handling — pitfalls
-
-## 1. Sub-CtaTile shapes are REJECTED, not silently padded
-
-The cooperative kernel's `gemm.can_implement(arguments)` rejects `(M, N) < CtaTile` (here, smaller than 128 × 128). At 30³ the kernel returns `cutlass::Status::kErrorInvalidProblem` and example 48 prints `Got cutlass error: Invalid status at: 415`. This is the **explicit** rejection path; the kernel never silently runs at a sub-tile shape. The "silent padding without recording the assumption" anti-pattern is therefore not violated by this kernel — but if you build your own kernel you must replicate this rejection or you risk silent miscompute. Always check `can_implement` before launching.
-
-## 2. The cooperative tail path is not a runtime toggle
-
-`MainloopSm90TmaGmmaWarpSpecialized` always runs the predicate-based tail path at boundary tiles when the problem is not a CtaTile multiple. There is no flag to disable it; the only way to "remove" the tail strategy is to pick a different mainloop template (`MainloopSm90TmaGmma` — the non-warpspecialized path) or to manually pad the input tensors before launching. Manual padding is the silent-padding anti-pattern unless documented.
-
-## 3. The "tail cost" at moderate sizes is real but not catastrophic
-
-At 1440³ the cooperative kernel runs at 129.4 TFLOPS vs. 188.7 TFLOPS at 2048³ (aligned). That is a ~32% throughput cost for a shape that is just one CtaTile away from being fully aligned. The cost is **not** purely the predicate-tail logic; it is the combined effect of (a) the boundary CTA tile having reduced effective work, (b) the cluster-multicast TMA losing some efficiency at the boundary, and (c) the wgmma atom not fully utilized at the tail. Don't try to attribute the full 32% to a single cause.
-
-## 4. cuBLAS auto-selects different kernels for different non-aligned shapes
-
-cuBLAS at 200³ achieves 2,621 GFLOPS; at 1440³ it achieves 153,708 GFLOPS. The ratio of cutlass to cuBLAS varies by shape (0.71× at 200³ vs 0.84× at 1440³) because cuBLAS's selector picks per-shape kernels. cutlass example 48 uses the same cooperative template across all shapes; the gap is expected and is not a regression. If your operator must always beat cuBLAS, you either match cuBLAS's per-shape kernel choice or accept the gap.
-
-## 5. ClusterShape constrains which non-aligned shapes work
-
-The cooperative kernel uses ClusterShape `<4, 2, 1>`. The cluster covers 4 CTAs in M and 2 in N, so the kernel's effective minimum problem size in (M-CTAs, N-CTAs) is at least one full cluster. At CtaTile 128 in M and N, that is 4 × 128 = 512 in M and 2 × 128 = 256 in N. If the problem is smaller than this in either dimension (e.g. 200×200×200), the cluster-multicast still fires but launches fewer-than-one-cluster's worth of CTAs in the boundary direction; this is fine as long as `can_implement` accepts the shape, but it reduces multicast efficiency. At 30×30×30 the shape is below the cluster minimum and `can_implement` rejects.
-

package/resources/kop-2026-10-23-0.1.17-beta.1/product_capabilities/kernel_optflow/skills/KernelWiki-private/wiki/nvidia/foundations/compute/gemm/tuning.md

@@ -1,50 +0,0 @@
----
-id: skill-gemm-tuning
-type: skill
-vendor: nvidia
-title: Tuning
-tags:
-- cuda-cpp
-- wgmma
-- warp-specialization
-- gemm
-applies_to:
-- general
-source:
-- path: spec
-  anchor: CUDA Programming Guide
-evidence_level: spec
-architectures:
-- sm90
-- sm90a
-languages:
-- cuda-cpp
-hardware_features:
-- wgmma
-techniques:
-- warp-specialization
-kernel_types:
-- gemm
-confidence: source-reported
----
-# Non-aligned GEMM Tuning Parameters
-
-## Compile-time parameters
-
-All aligned-GEMM tuning knobs apply (see `wiki/nvidia/foundations/compute/gemm/aligned/tuning.md`). Additional knobs specific to non-aligned shapes:
-
-| Knob | cutlass template param | Default | Range | Effect |
-|---|---|---|---|---|
-| Tail strategy | `MainloopSm90TmaGmmaWarpSpecialized*` | predicate-based | predicate, split-k tail | How boundary CTAs handle partial tiles. Predicate: wgmma atoms mask out-of-bounds elements. Split-K: decompose the K dimension into aligned chunks + remainder. |
-| Fallback tile shape | `CtaTile` (auto-selected by CollectiveBuilder) | varies | (64,64,32), (128,128,32) | For shapes far smaller than the primary tile, cutlass may select a smaller tile. The tail strategy applies to the executing kernel's tile, not the canonical tile. |
-
-## Key interaction: shape vs executing kernel
-
-For non-aligned shapes, the kernel that actually executes may differ from the canonical kernel (cutlass's `can_implement` check selects the first compatible kernel from a priority list). The tail cost is measured against the executing kernel's tile, not the canonical tile.
-
-See BitLesson `BL-20260428-shape-non-aligned-on-executing-kernel` for the diagnostic that established this distinction.
-
-## Measured sweep reference
-
-Non-aligned shapes: `sources/experience/api-probes/gemm.md`
-Adjacent-shape proxy measurement (1440 vs 1536): `artifacts/experience/api-probes/gemm/2026-04-28-gemm-tail.csv`