kernelport 0.1.0__py3-none-any.whl

kernelport/__init__.py

@@ -0,0 +1,3 @@
+"""kernelport: port CUDA kernels to other backends and prove they are correct."""
+
+__version__ = "0.1.0"

kernelport/__main__.py

@@ -0,0 +1,5 @@
+"""`python -m kernelport` == the `kernelport` CLI."""
+
+from kernelport.cli import app
+
+app()

kernelport/_skills/kernelport/SKILL.md

@@ -0,0 +1,200 @@
+---
+name: kernelport
+description: Port CUDA kernels to Triton, Metal (MLX), WebGPU (WGSL), CPU (Numba) or HIP and PROVE each port correct with the kernelport MCP tools. Use when the user asks to port, convert, translate or verify CUDA kernels on other hardware.
+---
+
+# kernelport: port CUDA kernels and prove they work
+
+You write the code. kernelport runs it on real hardware and checks it against a
+trusted reference. **A port only counts as working when `verify_port` returns
+`PASS`.** Nothing else (compiling, looking right, passing your own test) counts.
+
+## Hard rules
+
+1. The **original CUDA kernel is the ground truth for behaviour**. Never edit it.
+2. The **reference is the source of truth for ports**. If the reference and the
+   original disagree, fix the reference, never the original.
+3. Never say a port works unless `verify_port` returned `PASS` for its current code.
+   Report `FAIL`, `UNVERIFIED` and `SKIPPED` exactly as returned.
+4. Before every retry, write one or two sentences explaining the bug you suspect,
+   based on the failure report. Then fix and verify again.
+5. Stop after **6 failed `verify_port` attempts** for a kernel/backend and report it as FAIL
+   (the tool tells you `attempts_remaining`).
+6. Do not try to change tolerances or inputs to make a port pass. Tolerances are
+   fixed by kernelport. If you believe a tolerance or a test case is wrong, stop
+   and explain to the user instead.
+7. Keep ports readable: short comments mapping each part to the CUDA original.
+
+## Workflow (call the tools in this order)
+
+| Step | Do this | Tool |
+|---|---|---|
+| 0 | Check which backends can run here | `kernelport_doctor()` |
+| 1 SCAN | Find kernels | `scan_kernels(path)` |
+| 2 READ | Get source, parsed params, launch sites, suggested input spec | `read_kernel(kernel_id)` |
+| 3 REFERENCE | Write a plain-PyTorch reference + complete input spec | `write_reference(kernel_id, code, spec)` |
+| 4 VALIDATE | Run the ORIGINAL CUDA kernel vs your reference. PASS freezes it | `validate_reference(kernel_id)` |
+| 5 TRANSLATE | Read the backend guide, write the port | `read_skill("<backend>")`, `write_port(kernel_id, backend, code)` |
+| 6 VERIFY | Port vs frozen reference on every case | `verify_port(kernel_id, backend)` |
+| 7 REPAIR | Explain suspected bug, fix, verify again (max 6 failures) | `write_port`, `verify_port` |
+| 8 BENCHMARK | Only after PASS | `bench_port(kernel_id, backend)` |
+| 8b TUNE (optional) | Python ports with a `TUNE` dict: verify + time each variant, keep the fastest passing one | `tune_port(kernel_id, backend)` |
+| 9 REPORT | Show the user the status table and the report path | `list_status()`, `make_report()` |
+
+Do the steps for one kernel at a time (validate it, port it, verify it) before
+moving to the next; it keeps context small. Read `read_skill("common_bugs")` once
+before your first port.
+
+If `validate_reference` returns `UNAVAILABLE` (no NVIDIA GPU here), the reference
+cannot be frozen on this machine. Still write the references and specs for every
+kernel, then tell the user how to freeze them all at once on an NVIDIA machine:
+commit `.kernelport/` and run `kernelport validate --all` there (a free Colab T4
+works: `notebooks/freeze_references_colab.ipynb` in the kernelport repo). Once the
+frozen `.kernelport/` is back in the repo, porting and verifying continue here.
+Do not write ports for a kernel whose reference is not frozen: `verify_port` refuses.
+
+If the target backend is not available here, still write the port; `verify_port`
+returns `UNVERIFIED` and the port can be verified later with `kernelport verify` on
+the right hardware. Say clearly that it is unverified.
+
+## Step 3 in detail: the input spec (launch.json)
+
+`read_kernel` returns `input_spec`: a suggestion with `"TODO"` wherever it could not
+infer a value. Fill every TODO; `write_reference` rejects a spec with TODOs.
+
+```json
+{
+  "name": "softmax",
+  "source": "softmax.cu",
+  "entry": "softmax<{T}>",
+  "dtypes": ["float32", "float16", "bfloat16"],
+  "params": [
+    {"name": "x",    "kind": "input",  "dtype": "T", "shape": ["rows", "cols"]},
+    {"name": "y",    "kind": "output", "dtype": "T", "shape": ["rows", "cols"]},
+    {"name": "rows", "kind": "scalar", "dtype": "int32", "value": "rows"},
+    {"name": "cols", "kind": "scalar", "dtype": "int32", "value": "cols"}
+  ],
+  "launch": {"grid": ["rows"], "block": [256], "shared_mem": 0},
+  "cases": [
+    {"name": "tiny_1x1",   "dims": {"rows": 1, "cols": 1}},
+    {"name": "odd_5x1023", "dims": {"rows": 5, "cols": 1023}},
+    {"name": "large_4096x4096", "dims": {"rows": 4096, "cols": 4096}}
+  ],
+  "init": {"x": {"dist": "normal", "std": 3.0, "edges": true}},
+  "bench_case": "large_4096x4096"
+}
+```
+
+- **params**: in the exact order of the CUDA signature. `kind` is `input`, `output`
+  (kernel only writes it), `inout` (kernel reads and writes it, e.g. `y = a*x + y`,
+  `atomicAdd(out, ...)`), or `scalar`. Check the suggested kinds against the source.
+- **dtype**: `"T"` means "each entry of `dtypes`" (for templated kernels). Fixed types:
+  `float32 float16 bfloat16 float64 int8 uint8 int16 int32 int64`.
+- **entry**: kernel name; for templates use `{T}` for the type, e.g. `"rmsnorm<{T}, 256>"`.
+  Only list `dtypes` the original kernel actually supports (bfloat16 needs `__nv_bfloat16`
+  instantiation to make sense).
+- **source**: informational in a repo; kernelport compiles the file the kernel was
+  scanned from (`file` in `read_kernel`), with host code stripped, so `main()`, cuBLAS
+  calls and `#include <stdio.h>` in that file are fine.
+- **shape / value / grid / block / shared_mem**: expressions over the case `dims`:
+  `+ - * / // % **`, `cdiv(a, b)`, `min`, `max`, `next_pow2`. `grid`/`block` must reproduce
+  exactly how the host code launches the kernel (see `launches` from `read_kernel`; it
+  shows the expressions, so read the host code for the values). If the host picks the
+  block size at run time (e.g. benchmarks a list of sizes), choose one it uses, 256 if
+  present, and keep grid consistent with it.
+- **dims** may be floats for float scalars: `{"n": 1000, "lr": 0.001, "eps": 1e-8}` with
+  `{"name": "lr", "kind": "scalar", "dtype": "float32", "value": "lr"}`. Every reader of
+  the case (original, reference, ports) gets the value rounded to the param's dtype, the
+  way the kernel receives it (0.999 as float32 is 0.99900001...).
+- **cases**: always include tiny (1 to 7 elements), odd sizes (1000, 1023, 4097),
+  a power of two, and one large case. Use sizes that hit every code path (e.g. a row
+  longer than one block, a dimension not divisible by the tile size). Keep every case
+  inside the kernel's own limits: 32-bit `int` index math (`b*T*V` overflows past
+  2^31), vectorised loads that need sizes divisible by 4 or 8, a block size that must
+  not exceed some dimension. The reference runs on the CPU in float64, so a large case
+  of ~50M elements is plenty. If the original fails only outside such a limit, say so
+  in your report and keep the cases inside it; never edit the original.
+- **init** must respect the kernel's input domain. The suggestion from `read_kernel` is
+  `normal` + `edges` everywhere; change it where that is invalid: variances and second
+  moments `>= 0` (`uniform`, `low: 0`), probabilities in (0, 1] before a `log`, indices
+  with `randint` in range.
+- **init** per input param: `dist` = `normal` (`mean`, `std`), `uniform` (`low`, `high`),
+  `randint` (`low`, `high` exclusive, for index arrays: keep indices in range!),
+  `zeros`, `ones`, `arange`. `edges: true` puts 0, -0, ±1, ±big, ±1/big first
+  (`edge_big`, default 1000). Values may be expressions over dims: `"high": "vocab"`.
+  Per-case overrides go in the case's own `"init"`.
+- **step** (accumulating kernels: sums, matmuls, scans, atomics): add `"step": 0.0625`
+  to every float input. Values become multiples of 1/16, so float32 sums are exact in
+  any order and the comparison is not polluted by summation-order rounding.
+- Outputs are pre-filled with NaN so unwritten elements are detected; inouts get
+  generated values. Keep `"name"` values valid Python identifiers (`in` -> `in_`).
+
+## Step 3 in detail: the reference
+
+```python
+import torch
+
+def reference(x, rows, cols):          # names = launch.json param names
+    return torch.softmax(x, dim=-1)     # one output: return the tensor
+```
+
+- Arguments: any subset of the `input`/`inout`/`scalar` params, by name.
+- Float inputs arrive as **float64 CPU tensors** (integers keep their dtype). Write
+  plain math; do not cast to the kernel dtype; kernelport rounds your result to the
+  output dtype before comparing. Scalars arrive as Python numbers, already rounded to
+  their declared dtype.
+- Return one tensor (single output) or a dict `{output_name: tensor}`. For `inout`
+  params return the **final** value (e.g. `{"y": a * x + y}`).
+- Model what the kernel computes, including its quirks (e.g. biased vs unbiased
+  variance, where eps is added, integer division, clamping).
+
+`validate_reference` runs the original on every case. On FAIL, read the report,
+fix the **reference** (or the spec: wrong kind, wrong shape, wrong grid), and retry.
+
+## Step 6-7: reading a failure report
+
+```
+softmax/triton on <GPU>: FAIL (18/24 cases passed)
+- case odd_3x4097 dtype float32:
+  y shape [3, 4097]: 4368/12291 values outside tolerance ...; worst at [2, 3075]
+  flat[11265:] expected [...]
+  flat[11265:] actual   [...]
+```
+
+Use it: which cases fail (only wide rows? only fp16? only tiny?), where the worst
+index is (tail of a row = masking or loop bounds; every element = formula), and
+whether actual is NaN ("unexpected NaN" usually means an element was never written
+or a 0/0). Errors are compact; an `error:` line means a crash or compile error with
+the offending line of your port.
+
+## Step 8b: tuning (optional)
+
+Only worth it for kernels where speed matters, and only after PASS. Lift the knobs
+you would tune (block size, warps, rows per program) to module-level constants and
+list values to try in a literal `TUNE` dict (at most 32 combinations):
+
+```python
+BLOCK = 1024
+NUM_WARPS = 4
+TUNE = {"BLOCK": [256, 512, 1024, 2048], "NUM_WARPS": [4, 8]}
+```
+
+`write_port`, `verify_port` (the port must PASS as written), then `tune_port(kernel_id, backend)`.
+Every variant is verified on every case before it is timed; a variant that FAILs is
+just reported, it does not use your repair budget. If one is at least 3% faster, it
+replaces the port, but only if each of its 5 timing rounds also beat each round of the
+current port; otherwise the summary says the difference was within timing noise and the
+port is kept. Report the result it returns, not your own estimate.
+
+`bench_port` times port and original in 3 alternating runs and reports the median with
+the range of the runs. On laptops and shared GPUs the runs can differ a lot; when the
+summary says the runs differed by more than 20%, report the speedup as approximate (with
+the range), or run `bench_port` again on a quieter machine.
+
+## Step 9: what to tell the user
+
+Call `make_report(format="markdown")` and show its `content` (the status matrix, benchmarks,
+and what needs attention), then give the path of the HTML report from `make_report()`. For each kernel and
+backend state PASS / FAIL / UNVERIFIED / SKIPPED exactly as reported, with the
+hardware it was verified on and the speedup if benchmarked. For FAILs, include the
+last suspected bug. Never round UNVERIFIED up to "should work".

kernelport/_skills/kernelport/backends/cpu.md

@@ -0,0 +1,138 @@
+# Porting CUDA to the CPU (Numba)
+
+A cpu port is plain Python whose hot loops are compiled by Numba (`@njit`).
+`prange` loops are split across CPU threads, so one `prange` iteration plays the
+role of one CUDA thread block (or one thread, for element-wise kernels). There is
+no shared memory, no `__syncthreads`, no warps: code inside one iteration runs
+sequentially, which makes most block-level CUDA tricks unnecessary.
+
+Runs on any machine with Numba (verified so far on Windows x86-64).
+Timings are wall-clock and **not comparable** to GPU kernel times; the report says so.
+
+## Port file format
+
+```python
+"""CPU (Numba) port of <kernel>: <what it computes>."""
+
+import math
+
+import numpy as np
+from numba import njit, prange
+
+
+@njit(parallel=True)
+def _softmax(x, y, rows, cols):
+    for r in prange(rows):            # CUDA: one block per row (blockIdx.x)
+        m = -math.inf
+        for c in range(cols):         # CUDA: threads striding over c, then a block reduction
+            m = max(m, float(x[r, c]))
+        s = 0.0
+        for c in range(cols):
+            s += math.exp(float(x[r, c]) - m)
+        for c in range(cols):
+            y[r, c] = math.exp(float(x[r, c]) - m) / s
+
+
+def run(x, y, rows, cols):
+    _softmax(x, y, rows, cols)
+```
+
+- `run(...)` takes **the launch.json parameter names** (any subset, any order).
+- Tensors arrive as **C-contiguous NumPy arrays with the launch.json shapes**
+  (`["rows", "cols"]` gives a 2-D array; index it as `x[r, c]`, or `x.ravel()` for
+  the CUDA flat index). Integer params (`int64` ids, positions) keep their dtype.
+- **Write outputs in place** (`y[i] = ...`, `y[:] = ...`). Outputs are pre-filled
+  with NaN, so an element you never write shows up as "unexpected NaN".
+  `y = np.maximum(x, 0)` only rebinds the local name, and **returning an array is
+  rejected** with an error.
+- `inout` params hold their initial values: add to them, do not overwrite
+  (`out[0] += total`).
+- `run` itself is ordinary Python and may call several `@njit` functions. The
+  kernel's work belongs in `@njit` loops; use NumPy in `run` only for glue
+  (views, `ravel`, zero-filling scratch), never `np.cumsum`/`np.matmul`-style
+  calls that replace the kernel.
+- Supported dtypes: **float32, float64**. float16/bfloat16 cases are skipped and
+  named in the result (NumPy has no bfloat16; Numba does not compile float16 math).
+
+## CUDA → Numba mapping
+
+| CUDA | Numba (CPU) |
+|---|---|
+| `__global__ void k(...)` + `<<<grid, block>>>` | `@njit(parallel=True) def _k(...)` called from `run` |
+| `i = blockIdx.x * blockDim.x + threadIdx.x; if (i < n)` | `for i in prange(n):` (no guard needed, the range is exact) |
+| one block per row (`blockIdx.x` = row) | `for r in prange(rows):` with sequential inner loops |
+| 2-D grid (`blockIdx.y`, `blockIdx.x`) | `prange` over the outer index, `range` over the inner one |
+| threads striding over a row + shared-memory reduction | a plain `for c in range(cols): acc += ...` inside the iteration |
+| `__shfl_xor_sync` / warp reductions | same: a sequential loop; no warps exist |
+| `__syncthreads()` between phases | nothing: phases inside one iteration already run in order |
+| `__shared__ float tile[32][33]` | not needed; for cache-friendly access loop over tiles (see transpose) |
+| per-block scratch (`__shared__` buffer) | `buf = np.empty(n)` inside the `prange` body (one per iteration) |
+| `atomicAdd(out, partial)` from every block | prange **scalar** reduction, then one write: `acc += v` in the loop, `out[0] += acc` after |
+| `atomicAdd(&hist[bin], 1)` (scatter) | **no atomics**: serial loop, or per-thread private histograms merged after |
+| `expf`, `logf`, `sqrtf`, `rsqrtf(x)` | `math.exp`, `math.log`, `math.sqrt`, `1.0 / math.sqrt(x)` |
+| `fmaxf(a, b)` / `fminf` | `max(a, b)` / `min(a, b)` |
+| `(float)x[i]` on half inputs | `float(x[i])` (computes in float64) |
+| `long long` index / `size_t` offsets | Python ints in Numba are int64; no overflow at these sizes |
+| `__launch_bounds__`, block size, `#pragma unroll` | ignore; Numba/LLVM vectorises inner loops itself |
+
+## Patterns
+
+**Element-wise** (vector_add, relu, saxpy): one `prange` over `n`.
+
+**Row-wise reductions** (softmax, layernorm, rmsnorm): `prange` over rows; each
+iteration does the kernel's phases (max, sum, normalise) as sequential loops over
+the row. Keep the kernel's math: two-pass mean/variance stays two-pass, biased
+variance stays biased (`/ cols`, not `/ (cols - 1)`).
+
+**Whole-array reduction** (reduce_sum): scalar reduction across a `prange`:
+
+```python
+@njit(parallel=True)
+def _sum(x, n):
+    acc = 0.0
+    for i in prange(n):
+        acc += x[i]          # Numba recognises scalar +=, *=, and m = max(m, ...) as reductions
+    return acc
+
+def run(x, out, n):
+    out[0] += _sum(x, n)     # out is inout: add
+```
+
+**Scans** (cumsum): `prange` over rows, a running sum along the row. A
+Hillis-Steele or Blelloch scan is a GPU technique; on the CPU the sequential scan
+is both simpler and faster. Inclusive vs exclusive must match the kernel.
+
+**Matmul**: `prange` over output rows; put `k` outside `j` so `B[k, :]` is read
+contiguously and the `j` loop vectorises; accumulate a row in a float64 buffer.
+
+**Transpose / strided access**: tile both loops (e.g. 32×32) so reads and writes
+stay in cache. Correctness does not depend on it, only speed.
+
+**Gathers** (embedding, rope): `prange` over tokens; index with the int64 ids
+directly (`table[ids[t], d]`).
+
+## Precision notes
+
+- Float literals are float64: `0.5 * x[i]` with a float32 `x` computes in float64.
+  `float32 * float32` stays float32; `math.exp` of a float32 returns float32.
+- Computing in float64 (`float(x[i])`, `acc = 0.0`) is **always allowed and
+  recommended**: it costs little on a CPU and is at least as accurate as the
+  kernel's float math. The result is rounded to float32 on store.
+- Leave `fastmath` off (the default). `fastmath=True` lets LLVM reorder sums and
+  assume no NaN/inf; the tolerances are not loosened for it.
+- Scalar Python args (`a`, `eps`) arrive as Python floats, i.e. float64.
+
+## CPU-specific bugs checklist
+
+- [ ] Every output element written in place; nothing returned from `run`.
+- [ ] `inout` params added to (or updated), not overwritten.
+- [ ] No writes to the **same array element from different `prange` iterations**
+      (`hist[0] += 1` or `out[0] += v` inside `prange` is a data race; Numba has no
+      atomics on the CPU and silently loses updates). Reduce into a scalar instead.
+- [ ] Inner loops use `range`, not `prange` (nested `prange` runs serially anyway,
+      and races are easy to introduce).
+- [ ] Loop bounds match the kernel's guards (`range(n)`, not `range(n - 1)`).
+- [ ] No `import` inside an `@njit` function (unsupported); import at module top.
+- [ ] Only NumPy/`math` calls Numba supports inside `@njit` (no torch, no Python
+      objects, no `print` of arrays in hot loops).
+- [ ] No `cache=True`: it writes cache files next to the port inside `.kernelport/`.

kernelport/_skills/kernelport/backends/hip.md

@@ -0,0 +1,53 @@
+# Porting CUDA to HIP (AMD GPUs)
+
+HIP is AMD's CUDA-like C++ dialect: most CUDA kernels port by swapping headers and a
+few intrinsics. A hip port is a `.hip` source file that kernelport compiles at runtime
+with hipRTC (through CuPy's ROCm build) and **launches exactly like the original**:
+same entry name (`launch.json` `entry`, `{T}` filled with HIP type names), same
+parameter order, same grid, block and shared memory.
+
+Verification needs an AMD GPU with ROCm torch and ROCm CuPy (e.g. `cupy-rocm-6-0`). On
+other machines `verify_port` returns UNVERIFIED. **Status: the runner's launch path is
+exercised on NVIDIA in tests, but hipRTC compilation and the zoo ports have not yet been
+verified on AMD hardware.**
+
+If you only need AMD support, a Triton port also runs on ROCm unchanged; HIP is for
+staying close to the CUDA source.
+
+## Port file format
+
+```cpp
+// HIP port of vector_add.
+#include <hip/hip_fp16.h>
+#include <hip/hip_bf16.h>
+
+template <typename T>
+__global__ void vector_add(const T* __restrict__ a, const T* __restrict__ b, T* __restrict__ out, int n) {
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    if (i < n) out[i] = a[i] + b[i];
+}
+```
+
+- Keep the kernel name, template parameters and parameter order of the original.
+  The launch comes from launch.json, so the kernel must work with the original
+  grid/block (loop inside the kernel if you need a different mapping).
+- `{T}` becomes `float`, `__half` or `__hip_bfloat16`.
+
+## CUDA → HIP mapping
+
+| CUDA | HIP |
+|---|---|
+| `#include <cuda_fp16.h>` / `<cuda_bf16.h>` | `#include <hip/hip_fp16.h>` / `<hip/hip_bf16.h>` |
+| `__nv_bfloat16` | `__hip_bfloat16` |
+| `blockIdx`, `threadIdx`, `__syncthreads`, `__shared__`, `atomicAdd` | same |
+| `__shfl_xor_sync(0xffffffff, v, off)` | `__shfl_xor(v, off)`. **Wavefronts are 64 lanes on CDNA (MI-series), 32 on RDNA**: use `warpSize`, never a hard-coded 32 |
+| warp-level reductions sized for 32 lanes | a shared-memory tree reduction (wave-size independent), or loops over `warpSize` |
+| `__ballot_sync` / masks (32-bit) | `__ballot` returns a 64-bit mask |
+| `cudaStream_t`, host API | not used: kernelport does the launch |
+
+## Bugs checklist
+
+- [ ] No `0xffffffff`-masked `*_sync` intrinsics and no hard-coded warp size 32.
+- [ ] Shared arrays sized by the block size, not by warp counts that assume 32 lanes.
+- [ ] Same entry name, template parameters and parameter order as launch.json.
+- [ ] HIP headers and type names (`__hip_bfloat16`).

kernelport/_skills/kernelport/backends/metal.md

@@ -0,0 +1,76 @@
+# Porting CUDA to Metal (MLX custom kernels)
+
+A metal port is the **body** of a Metal compute kernel. MLX (`mx.fast.metal_kernel`)
+writes the signature, compiles it, and launches it on the Apple GPU. A CUDA thread
+block is a Metal **threadgroup**, `__shared__` is `threadgroup`, `__syncthreads()` is
+`threadgroup_barrier(mem_flags::mem_threadgroup)`.
+
+Verification needs an Apple Silicon Mac with MLX. On other machines `verify_port`
+returns UNVERIFIED. Status: all 12 zoo metal ports PASS every case in fp32, fp16 and
+bf16 on Apple Silicon (an M1 on GitHub's macOS runners, checked in CI on every push).
+Treat the zoo ports as working examples of the patterns below.
+
+## Port file format
+
+```python
+"""Metal (MLX) port of saxpy: y = a * x + y."""
+
+SOURCE = """
+    uint i = thread_position_in_grid.x;
+    if (i < uint(n[0])) {
+        y_out[i] = static_cast<T>(a[0] * static_cast<float>(x[i]) + static_cast<float>(y[i]));
+    }
+"""
+THREADGROUP = (256, 1, 1)
+
+
+def grid(n):
+    return (n, 1, 1)          # TOTAL threads, not threadgroups
+```
+
+- **`grid()` returns total threads per dimension** (Metal `dispatchThreads`), unlike
+  CUDA's block counts. One threadgroup per row of 256 threads: `(rows * 256, 1, 1)`.
+- Names (launch.json order), generated by MLX:
+  - inputs: every `input` and `inout` tensor, then **every scalar as a one-element
+    array**: read `n[0]`, `eps[0]`;
+  - outputs: every `output` by name; **every `inout` as `<name>_out`**. MLX outputs are
+    fresh buffers, so read the old value from the input `<name>` and write `<name>_out`.
+- `T` is a template parameter when launch.json uses dtype "T" (`float`, `half`,
+  `bfloat16_t`). Do arithmetic in float: `static_cast<float>(x[i])`, store with
+  `static_cast<T>(v)`.
+- Outputs start as NaN (`INIT_VALUE`, override if the algorithm needs another start).
+- `ATOMIC_OUTPUTS = True` declares every output `device atomic<...>*` (for atomicAdd).
+- `HEADER = """..."""` holds helper functions (placed before the kernel).
+- Do not name a local variable like a param (`rows`, `cols`): params are already in scope.
+
+## CUDA → Metal mapping
+
+| CUDA | Metal (MLX body) |
+|---|---|
+| `blockIdx.x` | `threadgroup_position_in_grid.x` |
+| `threadIdx.x` | `thread_position_in_threadgroup.x` |
+| `blockIdx.x * blockDim.x + threadIdx.x` | `thread_position_in_grid.x` |
+| `blockDim.x` / `gridDim.x` | `threads_per_threadgroup.x` / `threadgroups_per_grid.x` |
+| `__shared__ float red[256];` | `threadgroup float red[256];` |
+| `__syncthreads()` | `threadgroup_barrier(mem_flags::mem_threadgroup);` |
+| warp shuffles (32 lanes) | `simd_sum`, `simd_max`, `simd_shuffle_xor` (SIMD width 32 on Apple GPUs), or a threadgroup tree |
+| `atomicAdd(float*, v)` | `ATOMIC_OUTPUTS = True` + `atomic_fetch_add_explicit(&out_out[0], v, memory_order_relaxed)` |
+| `expf`, `sqrtf`, `rsqrtf` | `metal::precise::exp`, `metal::precise::sqrt`, `metal::precise::rsqrt` |
+| `fmaxf` | `metal::max` |
+| `long long` | `int64_t` |
+| scalar param `n` | `n[0]` |
+
+## Precision notes
+
+- Metal may compile with fast-math, so use **`metal::precise::`** for exp, log, sqrt
+  and rsqrt. Tolerances are not loosened.
+- Accumulate in float for half/bfloat16 data.
+
+## Metal-specific bugs checklist
+
+- [ ] `grid()` is in threads (rows × threadgroup width), not threadgroups.
+- [ ] Scalars read as `name[0]`; inouts written to `name_out`, with the old value read from `name`.
+- [ ] Every `threadgroup_barrier` is reached by all threads (not inside `if (tid < s)`).
+- [ ] Atomic outputs: `INIT_VALUE` set to the identity (0 for sums), and an inout's
+      initial value added exactly once.
+- [ ] `metal::precise::` math functions.

kernelport/_skills/kernelport/backends/triton.md

@@ -0,0 +1,117 @@
+# Porting CUDA to Triton
+
+Triton programs operate on **blocks of values**, not single threads. One Triton
+*program* (`tl.program_id`) does the work of one CUDA *thread block*; the block's
+threads become the lanes of `tl.arange` vectors. You never write `__syncthreads`
+or shared memory by hand: block-wide operations (`tl.sum`, `tl.max`, `tl.cumsum`)
+do it for you.
+
+Verified on NVIDIA (triton / triton-windows) and AMD ROCm (same port file).
+
+## Port file format
+
+```python
+"""Triton port of <kernel>: <what it computes>."""
+
+import triton
+import triton.language as tl
+
+
+@triton.jit
+def _kernel(x_ptr, y_ptr, n, BLOCK: tl.constexpr):
+    offs = tl.program_id(0) * BLOCK + tl.arange(0, BLOCK)   # CUDA: blockIdx.x * blockDim.x + threadIdx.x
+    mask = offs < n                                          # CUDA: if (i < n)
+    x = tl.load(x_ptr + offs, mask=mask)
+    tl.store(y_ptr + offs, x, mask=mask)
+
+
+def run(x, y, n):
+    BLOCK = 1024
+    _kernel[(triton.cdiv(n, BLOCK),)](x, y, n, BLOCK=BLOCK)
+```
+
+- `run(...)` takes the **launch.json param names** (any subset). Tensors are GPU
+  torch tensors; outputs are pre-filled with NaN and must be written **in place**.
+  Scalars are plain Python numbers. Return value is ignored.
+- Choose your own grid; it does not have to match the CUDA grid.
+- The output dtype is fixed by the tensor you store into: `tl.store` casts a float32
+  value to the pointer's element type (fp16/bf16) automatically.
+
+## CUDA → Triton mapping
+
+| CUDA | Triton |
+|---|---|
+| `blockIdx.x`, `gridDim.x` | `tl.program_id(0)`, `tl.num_programs(0)` |
+| `threadIdx.x` over `blockDim.x` | `tl.arange(0, BLOCK)` (BLOCK must be a power of 2 `tl.constexpr`) |
+| `if (i < n)` bounds check | `mask = offs < n`, pass `mask=` to every `tl.load`/`tl.store` |
+| 2D tile `(ty, tx)` | `rows = r0 + tl.arange(0, BM)[:, None]`, `cols = c0 + tl.arange(0, BN)[None, :]`, `ptr + rows * stride + cols` |
+| grid-stride / chunk loop over a row | `for start in range(0, cols, BLOCK): offs = start + tl.arange(0, BLOCK)` |
+| `__shared__` buffers + `__syncthreads` | keep values in block tensors; no sync needed |
+| block reduction (shared-memory tree) | `tl.sum(x, axis=0)`, `tl.max(x, axis=0)`, `tl.min`, `tl.reduce(x, 0, combine_fn)` |
+| warp shuffles `__shfl_xor_sync` reduce | same as block reduction: `tl.sum` / `tl.max` over the block tensor |
+| prefix sum / scan | `tl.cumsum(x, axis=0)` or `tl.associative_scan(x, 0, combine_fn)`; carry the running total across chunks yourself |
+| `atomicAdd(&p[i], v)` | `tl.atomic_add(p_ptr + i, v, mask=m)`; also `tl.atomic_max/min/xchg/cas` |
+| `float4` vector loads | not needed; contiguous `tl.arange` loads are vectorised automatically |
+| `(float)h` for `__half`/bf16 | `x.to(tl.float32)` |
+| `(T)acc` store | just `tl.store(ptr, acc)`; it is cast to the pointer dtype |
+| `expf`, `rsqrtf`, `sqrtf`, `fmaxf` | `tl.exp`, `tl.rsqrt`, `tl.sqrt_rn` (exact) / `tl.sqrt` (approx), `tl.maximum` |
+| `-INFINITY` | `float("-inf")` |
+| `size_t` index math | cast before multiplying: `row = tl.program_id(0).to(tl.int64)` |
+| gather `table[ids[t] * dim + d]` | `idx = tl.load(ids_ptr + t)`; `tl.load(table_ptr + idx * dim + offs, mask=...)` |
+| matmul inner loop | `acc += tl.dot(a, b, input_precision="ieee")` (see TF32 warning) |
+| `template <typename T>` | nothing to do: Triton specialises on the pointer dtype |
+| `#define BLOCK 256` | `BLOCK: tl.constexpr` argument |
+
+## Patterns
+
+**Row-wise op on a [rows, cols] matrix (softmax, layernorm, rmsnorm):**
+one program per row. If `cols` may exceed one block, loop over the row in chunks;
+for softmax use an online max/sum (rescale the running sum when the max grows),
+or do one pass for the max and a second for the sum.
+
+```python
+row = tl.program_id(0).to(tl.int64)
+acc = tl.zeros((BLOCK,), tl.float32)
+for start in range(0, cols, BLOCK):
+    offs = start + tl.arange(0, BLOCK)
+    x = tl.load(x_ptr + row * cols + offs, mask=offs < cols, other=0.0).to(tl.float32)
+    acc += x * x
+mean_sq = tl.sum(acc, axis=0) / cols
+```
+
+**Masked loads feeding a reduction:** choose `other=` so masked lanes are neutral:
+`0.0` for sums, `float("-inf")` for max, `float("inf")` for min.
+
+**Full reduction to one value:** either `tl.atomic_add` each program's partial into
+the output (the output must start at the value the CUDA kernel expects; inout
+params arrive with their initial values), or one program looping over everything.
+
+**Scan across chunks:** `tl.cumsum` within the chunk, add the carry, then
+`carry += tl.sum(chunk)`.
+
+**Matmul:** tile with `tl.dot` on `[BM, BK] x [BK, BN]` blocks, accumulate in a
+float32 `tl.zeros((BM, BN), tl.float32)`, mask the K tail with `other=0.0`.
+`tl.dot` needs every block dim ≥ 16.
+
+## Precision notes
+
+- **TF32:** `tl.dot` on float32 inputs uses TF32 by default (~1e-3 relative error).
+  That FAILS verification. Always pass `input_precision="ieee"` for float32.
+- Accumulate in float32 (`.to(tl.float32)`) like the CUDA kernel does; accumulating
+  in fp16/bf16 will fail.
+- `tl.sqrt` and `/` use fast approximate instructions (≤ 2 ulp); that is within
+  tolerance. If a result must be correctly rounded, use `tl.sqrt_rn` / `tl.div_rn`.
+
+## Triton-specific bugs checklist
+
+- [ ] Every `tl.load`/`tl.store` that can go out of bounds has a `mask`.
+- [ ] `tl.arange` size is a power of two `constexpr`; use `triton.next_power_of_2(n)`
+      and mask the excess, or loop in chunks.
+- [ ] Masked lanes use a neutral `other=` value for the reduction they feed.
+- [ ] Large offsets cast to int64 before multiplying (rows × cols > 2^31).
+- [ ] `run()` writes outputs in place; it does not allocate and return new tensors.
+- [ ] `run()` parameter names match launch.json exactly.
+- [ ] float32 `tl.dot` uses `input_precision="ieee"`.
+- [ ] Integer outputs store integer values (cast with `.to(tl.int32)` etc. if needed).
+- [ ] No Python-side work on tensor values inside `run()` (no `.item()`, no torch
+      math replacing the kernel); the port must compute on the GPU in Triton.