kernelport 0.1.0__py3-none-any.whl

kernelport/_skills/kernelport/backends/webgpu.md

@@ -0,0 +1,146 @@
+# Porting CUDA to WebGPU (WGSL)
+
+A WebGPU port is a WGSL compute shader. The mapping from CUDA is close to
+one-to-one: a CUDA thread block is a WGSL **workgroup**, `__shared__` is
+`var<workgroup>`, `__syncthreads()` is `workgroupBarrier()`.
+
+kernelport runs ports through **wgpu-native** (Vulkan / Metal / D3D12) with only
+core WebGPU features enabled (plus the standard `shader-f16`), so WGSL that passes
+here is WGSL a browser accepts. Timings are wall-clock including submission and are
+**not comparable** to native kernel times.
+
+## Port file format
+
+```python
+"""WebGPU (WGSL) port of relu: y[i] = max(x[i], 0)."""
+
+WGSL = """
+@compute @workgroup_size(256)
+fn main(@builtin(global_invocation_id) gid: vec3<u32>,
+        @builtin(num_workgroups) nwg: vec3<u32>) {
+    let n = u32(params.n);
+    for (var i = gid.x; i < n; i += nwg.x * 256u) {    // grid-stride: see the 65535 limit
+        let v = x[i];
+        y[i] = select(T(0.0), v, v > T(0.0));
+    }
+}
+"""
+
+
+def grid(n):                         # launch.json names; arrays are NumPy (use .shape)
+    return (min((n + 255) // 256, 65535),)   # workgroups per dimension
+```
+
+- The entry point is **`fn main`**. `grid(...)` returns 1 to 3 workgroup counts
+  (CUDA: the grid); each must be **≤ 65535**.
+- **Do not declare the bindings.** The runner appends them:
+  - `alias T = f32;` (`f16` for float16 cases, with `enable f16;`), so write
+    `array<T>` code once for every dtype;
+  - one storage buffer per tensor param **named exactly like the launch.json param**,
+    in launch.json order: `var<storage, read>` for `input`, `read_write` for
+    `output`/`inout`. Element type: `T` for dtype "T"; else `f32`, `f16`, `i32`, `u32`;
+  - **int64 arrives as `vec2<u32>`** (low word in `.x`); core WGSL has no 64-bit ints.
+    Index arrays are small and non-negative: use `ids[t].x`;
+  - `params`: a uniform struct with every scalar param (`params.n`, `params.eps`).
+    int32 → `i32`, float32 → `f32`. Convert before comparing: `u32(params.n)`.
+- To change a buffer's declared type (atomics), set
+  `BINDINGS = {"out": "array<atomic<u32>>"}` in the port.
+- Shader errors are reported with line numbers **inside the WGSL string**.
+- Supported dtypes: float32, float16 (when the adapter has `shader-f16`). bfloat16
+  cases are skipped and named in the result (no bf16 type in WGSL).
+
+## CUDA → WGSL mapping
+
+| CUDA | WGSL |
+|---|---|
+| `__global__ void k(...)` | `@compute @workgroup_size(256) fn main(...)` |
+| `blockDim.x` (256) | the constant in `@workgroup_size` |
+| `blockIdx.x` | `@builtin(workgroup_id) wid: vec3<u32>` → `wid.x` |
+| `threadIdx.x` | `@builtin(local_invocation_id) lid` or `@builtin(local_invocation_index) tid` |
+| `blockIdx.x * blockDim.x + threadIdx.x` | `@builtin(global_invocation_id) gid` → `gid.x` |
+| `gridDim.x` | `@builtin(num_workgroups) nwg` → `nwg.x` |
+| `if (i < n)` | `if (i < u32(params.n)) { ... }` |
+| `__shared__ float red[256];` | `var<workgroup> red: array<f32, 256>;` (module scope) |
+| `__syncthreads()` | `workgroupBarrier()` (must be reached by the whole workgroup: not inside a divergent `if`) |
+| `__shfl_xor_sync` warp reductions | tree reduction in `var<workgroup>` memory (subgroups are optional in WebGPU) |
+| `atomicAdd(int*)` | `atomicAdd(&buf[i], v)` on `array<atomic<i32>>` / `atomic<u32>` |
+| `atomicAdd(float*)` | **no float atomics in core WGSL**: compare-exchange loop on the bits (below) |
+| `expf, logf, sqrtf, rsqrtf, fmaxf` | `exp, log, sqrt, inverseSqrt, max` |
+| `-INFINITY` | no inf literals in WGSL: use `-3.4028235e38` (-FLT_MAX) as the max identity |
+| `(float)x[i]` / `(T)v` | `f32(x[i])` / `T(v)` |
+| `c ? a : b` | `select(b, a, c)` (note the argument order) |
+| `size_t` / `long long` offsets | `u32` (fine below 4G elements) |
+| template `<typename T>` | `alias T` provided by the runner |
+
+## Patterns
+
+**Element-wise, large n**: one invocation per element only works up to
+65535 × workgroup_size elements. Use a grid-stride loop with the grid capped at
+65535 (shown above), or a 2-D grid.
+
+**One workgroup per row** (softmax, layernorm, rmsnorm, cumsum): `grid(rows) =
+(rows,)`; invocations stride over the row (`for (var c = tid; c < cols; c += 256u)`),
+then a workgroup tree reduction:
+
+```wgsl
+var<workgroup> red: array<f32, 256>;
+
+fn block_sum(v: f32, tid: u32) -> f32 {
+    red[tid] = v;
+    workgroupBarrier();
+    for (var s = 128u; s > 0u; s >>= 1u) {
+        if (tid < s) { red[tid] += red[tid + s]; }
+        workgroupBarrier();                // outside the if: every invocation reaches it
+    }
+    let r = red[0];
+    workgroupBarrier();                    // before red is reused
+    return r;
+}
+```
+
+**Float atomicAdd** (reduce_sum): reduce per workgroup, then one invocation adds
+the partial with a compare-exchange loop (`BINDINGS = {"out": "array<atomic<u32>>"}`):
+
+```wgsl
+var old = atomicLoad(&out[0]);
+loop {
+    let r = atomicCompareExchangeWeak(&out[0], old, bitcast<u32>(bitcast<f32>(old) + partial));
+    if (r.exchanged) { break; }
+    old = r.old_value;
+}
+```
+
+**Tiles** (transpose, tiled matmul): `var<workgroup> tile: array<array<T, 33>, 32>;`,
+a 2-D `@workgroup_size(32, 8)`, `workgroupBarrier()` between load and store, exactly
+like the CUDA tile.
+
+**Scans**: port the CUDA scan as is (Hillis-Steele in workgroup memory + a
+`var<workgroup> carry: f32`), with barriers where CUDA has `__syncthreads`.
+
+## Buffer size limits
+
+One storage binding is limited by the adapter: browsers default to 128 MiB, software
+Vulkan (llvmpipe) allows 128 MiB, discrete GPUs allow up to 2 GiB here. A case whose
+tensor is larger than the adapter's limit is not run at all: the result names it as
+"cases this device cannot run, not tested" with the sizes. That is a device limit, not
+a bug in your port, so do not change the port for it. Say in your report which cases
+were not tested and why.
+
+## Precision notes
+
+- Compute in f32 for f16 data (`f32(x[i])`, store with `T(v)`), like CUDA's float
+  accumulators.
+- WGSL's `exp`, `log` and `sqrt` precision is implementation-defined (exp: about
+  3 + 2·|x| ulp). That is within tolerance when the argument is small, as in
+  softmax's `exp(x - max)`. Nothing is loosened for it.
+- There is no `fma` contraction control; plain `a * b + c` is fine.
+
+## WebGPU-specific bugs checklist
+
+- [ ] No binding declarations in the WGSL (the runner adds them); names match launch.json.
+- [ ] `grid()` returns ≤ 65535 per dimension; large 1-D inputs use a grid-stride loop.
+- [ ] Every `workgroupBarrier()` is in uniform control flow (not inside `if (tid < s)`).
+- [ ] Scalars converted before use: `u32(params.n)`, `f32(params.cols)`.
+- [ ] No plain `out[0] = out[0] + v` from several workgroups (a race): use atomics.
+- [ ] int64 inputs read as `.x` of `vec2<u32>`.
+- [ ] Workgroup arrays sized by the constant workgroup size, not by a runtime value.

kernelport/_skills/kernelport/common_bugs.md

@@ -0,0 +1,40 @@
+# Common porting bugs (all backends)
+
+Read this once before your first port. When `verify_port` fails, match the
+failure report against these symptoms first.
+
+| Symptom in the report | Likely cause |
+|---|---|
+| `unexpected NaN` at the end of the tensor, odd sizes only | Tail not covered: grid rounded down, or loop stops at `n - n % BLOCK` |
+| `unexpected NaN` everywhere | Port never writes the output (wrote to a copy, returned a new tensor, wrong param name) |
+| `unexpected NaN` in a few rows | 0/0 or inf-inf: missing max-subtraction (softmax), uninitialised accumulator, masked lanes with a non-neutral `other=` |
+| Fails only for rows longer than one block (e.g. cols=4097, not 1024) | Chunk loop missing or reduction only covers the first chunk |
+| Small errors (1e-4 to 1e-2) only when the size is not a multiple of the block (7, 1000, 4097 fail; 1, 512, 1024 pass) | Masked lanes feed a reduction: they load `other=0.0` and then contribute (e.g. `(0 - mean)**2` in a variance). Wrap the term: `tl.where(mask, d * d, 0.0)` |
+| Fails only for tiny sizes (1, 7) | Block size larger than data without a mask, or division by a size that becomes 0 |
+| Fails only for fp16 / bf16 | Accumulating in half precision; missing `.to(float32)` before math |
+| Fails only for float32 with ~1e-3 error | TF32 / fast-math / reduced-precision instructions |
+| Every value off by a constant factor | Wrong normalisation: divide by `n` vs `n-1`, eps inside vs outside sqrt |
+| Values correct but in the wrong place | Transposed indexing: row/col swapped, wrong stride, wrong output layout |
+| Off by exactly one element / shifted | Inclusive vs exclusive scan, off-by-one in offsets |
+| `inout` output wrong | Overwrote instead of updating (`y = a*x` instead of `y = a*x + y`), or ignored the initial value |
+| Integer outputs differ | Wrong rounding/truncation, int32 overflow in index math |
+| Different wrong values on each run, or only at large sizes | Data race: several threads/iterations write one element without an atomic (missing `__syncthreads` / `tl.atomic_add`; on cpu, `out[0] += v` inside `prange`) |
+| `error: TypeError: run() returned a value` (cpu) | Outputs must be written in place; `run` must return None |
+| Crash / illegal address | Unmasked out-of-bounds load, index array value used without bounds |
+
+## Checklist before calling verify_port
+
+- [ ] Same math as the CUDA original, including its exact formula quirks (where eps goes,
+      biased variance, clamping, rounding mode of integer division).
+- [ ] Every element of every output is written exactly once (or accumulated, for inouts/atomics).
+- [ ] Bounds handled for every size in the spec: 1, 7, odd, power of two, large.
+- [ ] Reductions accumulate in float32 for fp16/bf16 inputs.
+- [ ] Index math cannot overflow 32 bits for the large case.
+- [ ] No shortcuts that skip the kernel (calling torch / numpy library ops that do the kernel's work
+      instead of writing it for the backend).
+
+## When you are stuck
+
+After 2 failures with the same symptom, change strategy: simplify the port (one
+element per lane, no tiling), get it to PASS, then optimise and verify again.
+After 6 failed attempts total, stop and report FAIL with your best explanation.

kernelport/agent.py

@@ -0,0 +1,178 @@
+"""Optional built-in agent: drives the kernelport tools with Claude through the Anthropic API.
+
+For people who do not use a coding agent. It needs `pip install kernelport[agent]` and the
+user's own Anthropic credentials (ANTHROPIC_API_KEY, or an `ant auth login` profile); every
+run is billed to that account. The agent can only act through the 12 kernelport tools, so
+all writes stay inside the repo's .kernelport/ workspace and it cannot run shell commands.
+
+The loop is written by hand (not the SDK's beta tool runner) so the turn budget, the tool
+surface and the client are fully under kernelport's control and testable with a fake client.
+"""
+
+from __future__ import annotations
+
+import inspect
+import json
+import types
+import typing
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any, Callable
+
+from kernelport import tools as kp_tools
+from kernelport.mcp_server import INSTRUCTIONS
+
+DEFAULT_MODEL = "claude-opus-5-5"
+DEFAULT_EFFORT = "high"
+DEFAULT_MAX_TURNS = 80
+MAX_TOKENS = 16000
+MAX_RESULT_CHARS = 60_000
+
+SYSTEM_PREAMBLE = """You are kernelport's built-in agent. You port CUDA kernels to other backends and prove
+each port correct, using only the kernelport tools. Follow the workflow skill below exactly.
+Work autonomously: do not ask the user questions; make reasonable choices and state them.
+When finished, reply with the markdown report from make_report(format="markdown") and a short
+summary of anything that is not PASS and why. Never describe a port as working unless
+verify_port returned PASS."""
+
+
+def _json_type(annotation: Any) -> dict:
+    """JSON schema for a tool parameter annotation (str, dict, and their Optional forms)."""
+    if isinstance(annotation, str):  # from __future__ annotations
+        annotation = eval(annotation, {"dict": dict, "str": str, "Any": Any})  # noqa: S307 (own source)
+    origin = typing.get_origin(annotation)
+    if origin in (typing.Union, types.UnionType):
+        args = [a for a in typing.get_args(annotation) if a is not type(None)]
+        return _json_type(args[0]) if args else {}
+    if annotation is dict or origin is dict:
+        return {"type": "object"}
+    return {"type": "string"}
+
+
+def tool_definitions() -> list[dict]:
+    """Anthropic tool definitions for every kernelport tool (deterministic order, no `root`)."""
+    defs = []
+    for fn in kp_tools.TOOLS:
+        props, required = {}, []
+        for name, p in inspect.signature(fn).parameters.items():
+            if name == "root":
+                continue  # the workspace is fixed by the agent, never chosen by the model
+            props[name] = _json_type(p.annotation)
+            if p.default is inspect.Parameter.empty:
+                required.append(name)
+        doc = inspect.getdoc(fn) or fn.__name__
+        defs.append({"name": fn.__name__, "description": doc,
+                     "input_schema": {"type": "object", "properties": props, "required": required}})
+    return defs
+
+
+def system_prompt() -> str:
+    """Stable system prompt (cached): preamble + MCP instructions + the workflow skill."""
+    skill = kp_tools.read_skill("workflow").get("content", "")
+    return f"{SYSTEM_PREAMBLE}\n\n{INSTRUCTIONS}\n\n<workflow_skill>\n{skill}\n</workflow_skill>"
+
+
+def run_tool(name: str, args: dict, root: Path) -> tuple[str, bool, str]:
+    """Execute one tool call. Returns (result JSON text, is_error, one-line summary)."""
+    fns = {fn.__name__: fn for fn in kp_tools.TOOLS}
+    if name not in fns:
+        return json.dumps({"error": f"unknown tool {name}"}), True, f"unknown tool {name}"
+    fn = fns[name]
+    accepted = set(inspect.signature(fn).parameters) - {"root"}
+    bad = set(args) - accepted
+    if bad:
+        msg = f"unknown arguments {sorted(bad)} for {name}; accepted: {sorted(accepted)}"
+        return json.dumps({"error": msg}), True, msg
+    kwargs = dict(args)
+    if "root" in inspect.signature(fn).parameters:
+        kwargs["root"] = str(root)
+    try:
+        result = fn(**kwargs)
+    except Exception as exc:  # a tool bug must not kill the loop; the model sees the error
+        msg = f"{type(exc).__name__}: {exc}"
+        return json.dumps({"error": msg}), True, msg
+    text = json.dumps(result, default=str)
+    if len(text) > MAX_RESULT_CHARS:
+        text = text[:MAX_RESULT_CHARS] + ' ...[truncated]"}'
+    return text, not result.get("ok", True) and "error" in result, str(result.get("summary", ""))
+
+
+@dataclass
+class AgentResult:
+    """Outcome of one agent run."""
+
+    final_text: str
+    stop_reason: str
+    turns: int
+    tool_calls: int
+    input_tokens: int = 0
+    output_tokens: int = 0
+    cache_read_tokens: int = 0
+    log: list[str] = field(default_factory=list)
+
+
+def make_client():
+    """The real Anthropic client (imported only here, so kernelport works without the SDK)."""
+    try:
+        import anthropic
+    except ImportError as exc:
+        raise RuntimeError("the agent needs the Anthropic SDK: pip install kernelport[agent]") from exc
+    return anthropic.Anthropic()
+
+
+def run_agent(task: str, root: Path, *, client=None, model: str = DEFAULT_MODEL,
+              effort: str = DEFAULT_EFFORT, max_turns: int = DEFAULT_MAX_TURNS,
+              fallbacks: bool = True, on_event: Callable[[str], None] = print) -> AgentResult:
+    """Run the tool-use loop until Claude finishes, refuses, or the turn budget is spent."""
+    client = client or make_client()
+    tools = tool_definitions()
+    system = [{"type": "text", "text": system_prompt(), "cache_control": {"type": "ephemeral"}}]
+    messages: list[dict] = [{"role": "user", "content": task}]
+    res = AgentResult(final_text="", stop_reason="", turns=0, tool_calls=0)
+
+    def emit(line: str) -> None:
+        res.log.append(line)
+        on_event(line)
+
+    extra: dict[str, Any] = {}
+    if fallbacks:  # on a safety decline, the API retries on a fallback model in the same call
+        extra = {"betas": ["server-side-fallback-2026-07-01"], "fallbacks": "default"}
+
+    while res.turns < max_turns:
+        res.turns += 1
+        response = client.beta.messages.create(
+            model=model, max_tokens=MAX_TOKENS, system=system, tools=tools, messages=messages,
+            output_config={"effort": effort}, cache_control={"type": "ephemeral"}, **extra)
+        usage = getattr(response, "usage", None)
+        if usage is not None:
+            res.input_tokens += getattr(usage, "input_tokens", 0) or 0
+            res.output_tokens += getattr(usage, "output_tokens", 0) or 0
+            res.cache_read_tokens += getattr(usage, "cache_read_input_tokens", 0) or 0
+        res.stop_reason = response.stop_reason
+        for block in response.content:
+            if block.type == "text" and block.text.strip():
+                res.final_text = block.text
+                emit(block.text)
+        messages.append({"role": "assistant", "content": response.content})
+
+        if response.stop_reason == "refusal":
+            emit("Claude declined to continue this request (stop_reason: refusal).")
+            return res
+        if response.stop_reason == "max_tokens":
+            emit("response hit max_tokens; stopping.")
+            return res
+        calls = [b for b in response.content if b.type == "tool_use"]
+        if response.stop_reason != "tool_use" or not calls:
+            return res
+        results = []
+        for call in calls:
+            res.tool_calls += 1
+            text, is_error, summary = run_tool(call.name, dict(call.input or {}), root)
+            emit(f"  [{call.name}] {summary}")
+            results.append({"type": "tool_result", "tool_use_id": call.id, "content": text,
+                            **({"is_error": True} if is_error else {})})
+        messages.append({"role": "user", "content": results})  # all results in one message
+
+    res.stop_reason = "max_turns"
+    emit(f"turn budget of {max_turns} used up; stopping. Run `kernelport status` to see where it got to.")
+    return res