nbxtensor 0.0.1__tar.gz → 0.0.2__tar.gz

nbxtensor-0.0.2/.github/PULL_REQUEST_TEMPLATE/certificate.md

@@ -0,0 +1,28 @@
+## Certificates for a GPU class
+
+**GPU model** (as `nvidia-smi -L` or the system reports it):
+**Driver version:**
+**Operating system / Python:**
+**Versions** (`pip show nbxtensor neurobrix triton`):
+
+**Models run** (the `neurobrix run` commands whose keys were certified):
+
+```
+neurobrix run ...
+```
+
+**Certification command:**
+
+```
+neurobrix autotune certify --profile <profile> --keys <...> --out <dir>
+```
+
+**Check — the certified-only run, served from these certificates** (paste its `[autotune] certified directory:` line and
+the output's sha256):
+
+```
+NEUROBRIX_AUTOTUNE_CERTIFIED_DIR=<dir> neurobrix run ... --certified-only
+```
+
+- [ ] the files are added under `src/neurobrix/config/autotune/<vendor>/<profile>/` of the NeuroBrix repository
+- [ ] the certified-only run above served every key (0 swept at runtime) and its output matches the uncertified run

{nbxtensor-0.0.1 → nbxtensor-0.0.2}/CHANGELOG.md

@@ -7,6 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.0.2] - 2026-11-06
+
+### Added
+- **`backend.launcher.prewarm_compiler()`**: computes Triton's cache key (`triton_key`, the sha256 of libtriton and of Triton's sources, about 1.3 s on a V100 host) in a daemon thread, so a process's first kernel compile no longer waits for it; an application calls it at start, while it loads weights.
+- **Intel GPUs through Level Zero** (`backend.level_zero`, `backend.level_zero_driver`; the `xpu` backend, chosen when the system's Level Zero loader reports a GPU): a runtime over Level Zero that serves the allocator (device and page-locked host memory, in-order command lists as streams, timed events), and a launch driver for the SPIR-V kernels Intel's Triton build compiles, with Intel's own argument ABI; the compile target is built without torch or SYCL (Level Zero's counts, the OpenCL ICD's extensions, the device file's architecture name). Proven against stand-ins of libze_loader and the OpenCL ICD: no Intel card has run it yet.
+- **What to install when a GPU is present and unusable** (`backend.install_help`, `python -m nbxtensor doctor`): for every NVIDIA, AMD or Intel GPU on the PCI bus, whether its runtime library and its Triton backend are installed, with the vendor's install commands when one is not; the refusal when no GPU runtime answers carries the same steps instead of a bare error. The README's "Install on AMD / Intel GPUs" section gives them.
+- **Device files** for AMD CDNA4 (gfx950), RDNA3 (gfx11xx) and RDNA4 (gfx120x), and Intel Data Center GPU Max (pvc), Arc A-series and Flex (dg2), Arc B-series (bmg) and Lunar Lake (lnl), every value with its source.
+- **A HIP launch driver for AMD GPUs** (`backend.launcher.HipDriver`, chosen when the runtime detects HIP): the HIP module API through the HIP runtime library the runtime already loaded, the launch Triton's own AMD launcher performs (hsaco code objects, `hipModuleLaunchKernel`, the kernel's arguments then the scratch pointers, a block of the code object's wavefront times the kernel's warps), one module per device and code object, the argument count read from the code object's own metadata and a list of another length refused, shared memory past the device's per-block maximum refused as Triton's own AMD launcher refuses it, the wavefront read from the code object and the device file (never defaulted), the launcher contract's `compile` surface shared with the CUDA driver, every missing symbol refused by name; page-locked host memory is asked of `hipHostMalloc` with its flags word. Its tests compile real Triton kernels for gfx90a, gfx942, gfx950, gfx1100 and gfx1201 on a machine without an AMD card and launch them through a stand-in HIP library: no AMD card has run it yet.
+
+### Removed
+- **Nine environment switches that changed a kernel**: `NBX_MM_MAX_OUTPUT_ELEMS` (the matrix product's output split, fixed by an int32 element offset), `NBX_FLASH_BLOCK_M_VOLTA` (Volta's flash Q tile, 64, the measured floor of correctness), `NBX_CONV3D_EAGER_FREE_BYTES` (when a 3-D convolution frees its buffers early), `NBX_FLASH_NUM_WARPS` and `NBX_FLASH_NUM_STAGES` (the flash kernel's launch, which its device file's flash rows state), `NBX_DGEMV_ADAPT` (the dequantising GEMV's width adaptation, set by `dequant_gemv.min_programs`), `NBX_D128_DETOUR` (the power-of-two head-dim detour, decided by `flash.pow2_head_dim_correct`; `launch_keys.flash_headdim_detour` loses its `enabled` argument), `NBX_DEPTHWISE_DISABLE` (the depthwise convolution route) and `NBX_GC_ON_OOM` (a garbage-collecting retry of a refused allocation, which is raised, never rescued). A value a kernel needs is a fact of its device file or of a certificate, never a host setting.
+
+### Changed
+- **A certified file is parsed and validated once, then read from its compiled index** (`ops.certified_index`): the directory stays JSON, the source contributors edit; the first launch after a change validates a file and writes, per memory class, a binary index of the entries that class is served (length-prefixed keys and certifications, a SHA-256 trailer, never a pickle), named by the fingerprint of everything the validation read; later launches read the index and parse a certification only when its key is asked. A corrupt index is refused by name.
+- **The launch recorder receives each pointer's extent** (`backend.launcher.set_launch_recorder`: `fn(prepared, grid, params, extents)`): the byte extent of each pointer's tensor that a driver binding by extent is handed, None for a driver that binds by address — a replay can now launch exactly what was launched.
+- **The README opens with what the library is and `pip install nbxtensor`**: the 0.0.1 README began its installation with `pip install -e nbxtensor` from the engine's repository, which PyPI showed as the package's description; it now opens with its title, what it does, the install from PyPI (the repository install is below, for contributors), a GPU example its tests run, why the library exists (no PyTorch, any Triton GPU, the V100's tensor cores back in use with their measured gains, planned memory, certified tuning) and the hardware tested and expected, and it links its API and backend references in the NeuroBrix repository, where they live (they were relative links to files the library's repository does not hold). The package's one-line description no longer names a CPU backend, which the library does not have.
+
+### Fixed
+- **CDNA3's shared memory per block is 64 KB**, not 160 KB (the device file had CDNA4's figure; Triton 3.8's AMD target features state 64 KB for gfx942): an autotune config needing more is no longer kept for an MI300.
+
+
 ## [0.0.1] - 2026-11-05
 
 The first release of nbxtensor: an inference-first tensor library whose kernels are written in Triton and which never
@@ -61,6 +82,7 @@ memory; the fix is in test).
 - **nbxtensor names the device in force publicly**: `nbxtensor.backend.device_files.in_force()`, `for_arch(arch)` and `forget()`, and `nbxtensor.backend.launcher.bind_target(target)` for a derivation that launches nothing; the engine's key census and paged KV cache read them instead of the library's private configuration module.
 
 ### Changed
+- **The launch recorder receives each pointer's extent** (`backend.launcher.set_launch_recorder`: `fn(prepared, grid, params, extents)`): the byte extent of each pointer's tensor that a driver binding by extent is handed, None for a driver that binds by address — a replay can now launch exactly what was launched.
 - **Every device file states the tolerance an integer kernel output is certified at**: `autotune_screen_rtol.int64: 0.0` — met exactly (the MoE alignment's routing tables); the certifier invents no tolerance and refused every alignment key without it.
 - **The MoE alignment's three stages take their chunk widths and warps from their certificates**: `moe_align_stage{1,2,3}_kernel` ran per-backend literal widths (128 or the experts' count on CUDA, 32-64 on Metal) at 4 warps; each is now certified per key (the block, the padded expert count, the layer's pairs in the profile's `elements` classes), its candidates pruned to the blocks that fit the profile's `memory.max_shared_memory_per_block`. Every candidate computes the same routing; the choice is speed only.
 - **`nbxtensor.memory.pointer_tables.build_ptr_tables` is public**: the expert pointer tables a MoE kernel reads (on Metal captured under a pinned scope), built by the engine's MoE layer and by its certifier alike; it was the private `_build_ptr_tables`.
@@ -115,5 +137,6 @@ memory; the fix is in test).
 - **`flash_decode_split_kernel` and its route**: armed only by an environment switch (`NBX_FLASH_DECODE`, with `NBX_FD_BLOCK_N` and `NBX_FD_SEG`) and measured slower than the vector decode kernels; `flash_decode.py` keeps the fixed-order reduce they share.
 - **`conv1d_forward_kernel`**: launched by no route (a 1-D convolution runs through `conv2d`).
 
-[Unreleased]: https://github.com/NeuroBrix/nbxtensor/compare/v0.0.1...HEAD
+[Unreleased]: https://github.com/NeuroBrix/nbxtensor/compare/v0.0.2...HEAD
+[0.0.2]: https://github.com/NeuroBrix/nbxtensor/compare/v0.0.1...v0.0.2
 [0.0.1]: https://github.com/NeuroBrix/nbxtensor/releases/tag/v0.0.1

nbxtensor-0.0.2/CONTRIBUTING.md

@@ -0,0 +1,37 @@
+# Contributing to nbxtensor
+
+nbxtensor is open source under the Apache 2.0 license. Issues, fixes, new device files and certificates are welcome.
+
+## Issues
+
+Found a bug on your hardware? [Open an issue](https://github.com/NeuroBrix/nbxtensor/issues) with:
+
+- your GPU model, its driver version, your operating system and Python version;
+- `pip show nbxtensor triton` (the versions installed);
+- the smallest code or command that shows the problem, and its full output.
+
+## Pull requests
+
+1. Fork the repository and branch from `main`.
+2. Keep a change complete: the code, its test (seen failing before the fix, passing after), and one line under
+   `## [Unreleased]` in `CHANGELOG.md` when a user would notice the change (a bold title, the cause and the effect).
+3. Run the library's tests: `pip install -e ".[test]"` then `pytest tests`. On a machine with a GPU the examples of the
+   README that launch a kernel run too.
+4. Open the pull request.
+
+The library's source lives in `nbxtensor/` of the [NeuroBrix repository](https://github.com/NeuroBrix/neurobrix) and is
+exported here; a pull request on either repository is fine — we carry it across.
+
+## Style
+
+- **No torch.** nbxtensor never imports torch, not even optionally or inside a `try`.
+- **No guessed value.** A fact the library needs (a tile, a tolerance, a limit of the device) is read from the device's
+  file (`src/nbxtensor/backend/devices/<vendor>/<name>.yml`) or measured; a missing fact is an error that names it,
+  never a default.
+- **No model knowledge.** The library knows tensors, kernels and devices, never a model by name.
+- English in code, comments and documentation.
+
+## Certificates
+
+Certificates are the proven kernel settings of a GPU class: see [Certify your own hardware](README.md#certify-your-own-hardware)
+for how to make them and share them. Open the pull request with the `certificate` template.

{nbxtensor-0.0.1 → nbxtensor-0.0.2}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.5
 Name: nbxtensor
-Version: 0.0.1
-Summary: An inference-first tensor library, torch-free: planned memory, Triton kernels that declare their cost, CUDA / Apple Metal / CPU backends.
+Version: 0.0.2
+Summary: Run AI models without PyTorch: tensors, Triton kernels, planned memory and certified tuning, on NVIDIA and Apple GPUs.
 Project-URL: Homepage, https://github.com/NeuroBrix/nbxtensor
 Project-URL: Source, https://github.com/NeuroBrix/nbxtensor
 Project-URL: Issues, https://github.com/NeuroBrix/nbxtensor/issues
@@ -18,33 +18,106 @@ Provides-Extra: test
 Requires-Dist: pytest; extra == 'test'
 Description-Content-Type: text/markdown
 
-# nbxtensor
+<p align="center"><img src="https://raw.githubusercontent.com/NeuroBrix/nbxtensor/main/assets/logo.png" alt="NeuroBrix" width="240"></p>
 
-An inference-first tensor library, torch-free.
+# nbxtensor — the torch-free tensor library for inference
 
-PyTorch is a general framework built for research and training in eager mode: dynamic graphs, autograd, a generic
-dispatcher, a caching allocator that discovers memory as the program runs. Inference is a narrower problem, and a
-better-informed one: the graph and the shapes are known before the first run. nbxtensor is built for that case.
+Run AI models without PyTorch, on any GPU Triton supports: tensors, Triton kernels, planned memory and certified tuning.
 
-- **Memory is planned, not discovered.** Allocators per device behind one interface, and a static planner that
-  assigns every buffer before the first run.
-- **Every kernel declares its cost.** Outputs, workspace and scratch with their lifetimes, bytes moved, flops. A
-  planner prices a model from these declarations instead of guessing what the allocator will do.
-- **Capture and replay are first-class.** A repeated sequence of launches is recorded, verified byte-equal and
-  replayed.
-- **One table of facts.** One dtype table, one view rule, one conversion per pair of dtypes, one capability table per
-  backend — declared, never guessed.
-- **Backends:** NVIDIA CUDA, Apple Metal (through triton-ext), CPU.
-- **No torch, ever.** nbxtensor never imports torch, not even optionally. Exchange with any tensor library goes
-  through the DLPack protocol (`__dlpack__`, `from_dlpack`), so a torch user writes `torch.from_dlpack(t)` on their
-  side.
+Runs wherever Triton runs — NVIDIA, AMD, Intel and Apple Silicon GPUs. Inference first; training is not in this
+release. Found a bug on your hardware? [Open an issue](https://github.com/NeuroBrix/nbxtensor/issues) — we fix fast.
 
-nbxtensor is extracted from NeuroBrix, the universal inference engine whose Triton engine is its first client. It
-lives in `nbxtensor/` of the NeuroBrix repository until the extraction is complete, then in its own repository.
+[![PyPI](https://img.shields.io/pypi/v/nbxtensor)](https://pypi.org/project/nbxtensor/)
+[![License: Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue)](https://github.com/NeuroBrix/nbxtensor/blob/main/LICENSE)
+[![Python](https://img.shields.io/pypi/pyversions/nbxtensor)](https://pypi.org/project/nbxtensor/)
+![torch-free](https://img.shields.io/badge/torch-free-success)
+![Triton](https://img.shields.io/badge/kernels-Triton-orange)
+
+```
+pip install nbxtensor
+```
+
+<!-- needs-gpu -->
+```python
+import numpy as np
+from nbxtensor.tensor import NBXTensor
+from nbxtensor.ops.linalg_wrappers import mm
+
+rng = np.random.default_rng(0)
+a = rng.standard_normal((256, 512)).astype(np.float16)
+b = rng.standard_normal((512, 128)).astype(np.float16)
+c = mm(NBXTensor.from_numpy(a), NBXTensor.from_numpy(b))      # a Triton kernel on the GPU, no torch
+ref = a.astype(np.float32) @ b.astype(np.float32)
+assert c.shape == (256, 128) and np.allclose(c.numpy().astype(np.float32), ref, rtol=2e-2, atol=2e-1)
+```
+
+## Why nbxtensor
+
+- **No PyTorch.** nbxtensor never imports torch, not even optionally; its tests, kernels and NeuroBrix's models run in
+  an environment where torch is not installed. Exchange with any tensor library goes through DLPack (`__dlpack__`,
+  `from_dlpack`), so a torch user writes `torch.from_dlpack(t)` on their side.
+- **Runs wherever Triton runs.** The kernels are Triton; a device is described by a file of facts (tiles, shared
+  memory, the matrix unit, the autotune ladders), never by code.
+- **Fast.** Faster than PyTorch on many models (in NeuroBrix's gate of 2026-11-05, the Triton engine on nbxtensor beat
+  the PyTorch engine on 18 of 49 models on a 16 GB V100 and 17 of 49 on a 32 GB one). Still running V100s? Triton leaves
+  their tensor cores idle (`tl.dot` compiles to FMA on sm_70); nbxtensor turns them back on with its own `mma.m8n8k4`
+  kernels (matrix products, convolutions, flash attention, the fused mixture-of-experts GEMM), chosen by the certificate
+  only where they measured faster — an 80 x 4096 x 4096 fp16 product 0.404 -> 0.191 ms, the fused MoE prefill 2.49 ->
+  0.96 ms (DeepSeek, 20 tokens) and 7.24 -> 1.40 ms (Qwen3, 512 tokens), fp32 1 x 1 convolutions 1.26-1.95 times faster
+  (V100, the measures recorded in [the 0.0.1 changelog](https://github.com/NeuroBrix/nbxtensor/blob/main/CHANGELOG.md#001---2026-11-05)).
+- **Lightweight.** nbxtensor itself is 1 MB; the whole install, Triton included, is about 1 GB — PyTorch's is 5.6 GB.
+- **Certified tuning.** Kernel settings are measured once per GPU class against a float64 oracle and shipped — no
+  autotuning at run time for the shapes we cover. We are certifying every model we can, so you never have to autotune.
+  Certified your GPU? [Share your certificates](#certify-your-own-hardware).
+- **Memory is planned, not discovered.** Allocators per device behind one interface, a static planner that assigns every
+  buffer before the first run, and every kernel declares its cost (outputs, workspace, scratch, bytes moved, flops) so a
+  planner prices a model before running it. Capture and replay of a repeated sequence of launches, verified byte-equal.
+
+## Certify your own hardware
+
+A kernel setting is certified per launch key (the kernel, its shape class and dtypes) and per GPU class. The keys a GPU
+class has no certificate for are swept at run time and kept on your machine; certifying them makes them permanent and
+shareable. Certification runs through [NeuroBrix](https://github.com/NeuroBrix/neurobrix), the engine whose models form
+the keys (`pip install neurobrix`), with these commands (run as written on a clean machine, 2026-11-06):
+
+1. Run the models you use. Every key no certificate covers is swept and kept in `~/.neurobrix/replay_cache/`:
+
+   ```
+   neurobrix run --model TinyLlama-1.1B-Chat-v1.0 --triton --prompt "Hello" --max-tokens 32
+   ```
+
+2. Certify the keys your card formed — every candidate setting is run against a float64 oracle and timed; the proofs are
+   written into `--out`, one file per kernel and dtype under `<vendor>/<profile>/` (the profile is your GPU's device
+   file, e.g. `volta`):
+
+   ```
+   neurobrix autotune certify --profile volta \
+       --keys ~/.neurobrix/replay_cache/autotune_configs_cuda-70-triton_3.8.0.json --out ./my-certificates
+   ```
+
+3. Check that they are served: a certified-only run uses only certificates and fails, naming the key, if one is missing:
+
+   ```
+   NEUROBRIX_AUTOTUNE_CERTIFIED_DIR=./my-certificates neurobrix run --model TinyLlama-1.1B-Chat-v1.0 --triton \
+       --prompt "Hello" --max-tokens 32 --certified-only
+   ```
+
+   Its `[autotune] certified directory:` line says `0 swept at runtime`, and its output is the same as step 1's.
+
+4. Share them: fork [NeuroBrix](https://github.com/NeuroBrix/neurobrix), add the files of `./my-certificates/<vendor>/<profile>/`
+   under `src/neurobrix/config/autotune/<vendor>/<profile>/`, and open a pull request with the `certificate` template:
+   your GPU model, driver, the commands above and the output of step 3. Certified your GPU? Share your certificates —
+   every class certified is a class nobody autotunes again.
+
+
+**Links:** [API reference](https://github.com/NeuroBrix/neurobrix/blob/main/docs/nbxtensor/API.md) ·
+[release notes](https://github.com/NeuroBrix/nbxtensor/blob/main/CHANGELOG.md) ·
+[NeuroBrix](https://github.com/NeuroBrix/neurobrix), the inference engine whose Triton engine is nbxtensor's first
+client; nbxtensor's source lives in its `nbxtensor/` directory and is exported here.
 
 ## Status
 
-Every area below is in the library, and every public name is documented in `docs/nbxtensor/API.md` (the list of
+Every area below is in the library, and every public name is documented in [`docs/nbxtensor/API.md`](https://github.com/NeuroBrix/neurobrix/blob/main/docs/nbxtensor/API.md) (the list of
 undocumented names is empty and a test keeps it so). This README documents what is in the library TODAY; every example below is executed by
 `nbxtensor/tests/test_readme.py`, so an example that stops working fails a test. The target API is
 `docs/nbxtensor/API.md`.
@@ -67,10 +140,20 @@ attempt to import torch, caught ones included, and recorded none). Python 3.10 o
 pyyaml and Triton.
 
 ```
-pip install -e nbxtensor          # from the NeuroBrix repository
+pip install nbxtensor
 ```
 
-**Linux, NVIDIA (CUDA).** Triton comes from PyPI with the install above.
+**Linux, NVIDIA (CUDA).** Triton and the CUDA runtime library come from PyPI with the install above; the NVIDIA driver
+is the only other requirement.
+
+**Linux, AMD (ROCm) and Intel (Level Zero).** The system's GPU runtime and a Triton that compiles for the card are
+needed: see [Install on AMD / Intel GPUs](#install-on-amd--intel-gpus).
+
+**Which GPU, and what is missing.** `python -m nbxtensor doctor` names the backend in force and, for every NVIDIA, AMD
+or Intel GPU on the PCI bus, whether its runtime library and its Triton backend are installed — with the vendor's
+install commands when one is not.
+
+**From the NeuroBrix repository** (contributors): `pip install -e nbxtensor`.
 
 **macOS, Apple silicon (Metal).** PyPI publishes no Triton for macOS, so Triton and its Metal backend are built from
 source, then nbxtensor is installed into that environment:
@@ -83,6 +166,46 @@ source, then nbxtensor is installed into that environment:
 
 `nbxtensor.dtype` and `nbxtensor.shape` need neither a device nor Triton.
 
+## Install on AMD / Intel GPUs
+
+The library speaks to the card through the vendor's own runtime library, loaded at run time, and compiles its kernels
+with the Triton backend for that card. When a card is present and either is missing, the library refuses with the
+steps below (the same text `python -m nbxtensor doctor` prints). The commands are the vendors' (Ubuntu 24.04).
+
+**AMD (ROCm, HIP).** The ROCm runtime, from AMD's
+[quick start](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/quick-start.html):
+
+```
+wget https://repo.radeon.com/amdgpu-install/6.4.3/ubuntu/noble/amdgpu-install_6.4.60403-1_all.deb
+sudo apt install ./amdgpu-install_6.4.60403-1_all.deb
+sudo apt update && sudo apt install rocm
+sudo usermod -a -G render,video $LOGNAME     # then log out and in (or reboot)
+pip install nbxtensor                        # the Triton wheel on PyPI carries the AMD backend
+```
+
+Targets: CDNA (gfx908, gfx90a, gfx942, gfx950) and RDNA 3 / 4 (gfx11xx, gfx120x); the hardware profile states the
+card's gfx arch. Triton 3.8 compiles the library's kernels for these targets on a machine without the card (the
+library's tests do it).
+
+**Intel (Level Zero).** Intel's GPU driver, from Intel's
+[package guide](https://dgpu-docs.intel.com/installation-guides/installing-packages-from-the-intel-ppa.html), then
+Intel's Triton build (the PyPI Triton has no Intel backend):
+
+```
+sudo add-apt-repository -y ppa:kobuk-team/intel-graphics
+sudo apt-get install -y libze-intel-gpu1 libze1 intel-opencl-icd clinfo intel-gsc
+sudo usermod -a -G render $LOGNAME           # then log out and in
+pip install triton-xpu --index-url https://download.pytorch.org/whl/xpu
+pip install nbxtensor
+```
+
+Devices: Data Center GPU Max (Ponte Vecchio), Arc A-series and Data Center GPU Flex (Alchemist), Arc B-series
+(Battlemage), Core Ultra 200V graphics (Lunar Lake); the card's PCI id selects its device file.
+
+**Status.** The AMD and Intel drivers are written to the level of the NVIDIA and Apple ones and proven against
+stand-ins of the vendors' libraries; no AMD or Intel card has run them yet. [Open an
+issue](https://github.com/NeuroBrix/nbxtensor/issues) with what your card does — we fix fast.
+
 ## Concepts
 
 **Plain data at the edges.** A dtype is named by its table name (`"float16"`, `"bfloat16"` …), a shape is a tuple of
@@ -337,22 +460,21 @@ The host conversions of a weight at load are split the same way, across the load
 
 ## Hardware
 
-| hardware | backend | state |
-|---|---|---|
-| NVIDIA Tesla V100 (Volta, sm_70) | CUDA (`CudaDriver`, `AbiMemory`) | **proven**: the library's tests and the reference models' outputs identical to their judged sha, on a 4 x V100 rack, 2026-10-29 |
-| Apple M4 Pro (Apple silicon, 24 GB unified) | Metal through triton-ext (`TritonExtDriver`, `MetalMemory`) | **proven**: the library's tests, the reference models' outputs identical to their judged sha, in an environment where torch is not installed, 2026-10-29 |
-| NVIDIA Ampere, Hopper | CUDA | **written, not proven here**: device files exist (`backend/devices/nvidia/`), no such card on our machines |
-| Apple M1, M2, M3, M5 families and the other M4 chips | Metal | **written, not proven here**: device files exist (`backend/devices/apple/`); only the M4 Pro is measured |
-| AMD Instinct (CDNA, CDNA2, CDNA3) | HIP through the CUDA-style C ABI (`AbiMemory`) | **interface ready, not tested for lack of hardware**: memory primitives and device files exist; the HIP launch driver is still to write (`docs/nbxtensor/BACKENDS.md` section 6) |
-| Intel GPUs (XPU) | Level Zero (not the CUDA ABI) | **interface ready, not tested for lack of hardware**: the backend contract is documented; its `DeviceMemory` over Level Zero and its launch driver are to write (BACKENDS.md section 6) |
-| a CPU | none yet | **not supported** for the kernels: host work (loading, conversions) runs with numpy on every core (`backend.cpu.pool`); a CPU kernel backend is not part of 0.0.1 |
-| hardware with no Triton backend (NPUs, phones) | none | **not supported**: covering it needs a Triton backend for it and the five pieces of `docs/nbxtensor/BACKENDS.md` |
+| vendor | backend | devices | state |
+|---|---|---|---|
+| NVIDIA | `cuda` (the CUDA driver and runtime) | Volta to Hopper (sm_70 to sm_90) | V100 16 GB and 32 GB: every release's gate |
+| Apple | `metal` (Metal through triton-ext) | M1 to M5 | M4 Pro: every release's gate |
+| AMD | `hip` (ROCm's HIP runtime) | CDNA 1 to 4, RDNA 3 and 4 | written and proven against a stand-in; no card has run it |
+| Intel | `xpu` (Level Zero) | Data Center GPU Max, Arc A and B, Lunar Lake | written and proven against a stand-in; no card has run it |
+
+Each device has a device file (`nbxtensor/backend/devices/<vendor>/*.yml`): its shared memory, warp size, tiles and
+tuning screens, every value with its source.
 
 ## Backends
 
-CUDA (NVIDIA) and Apple Metal through triton-ext; what a new backend provides is `docs/nbxtensor/BACKENDS.md`, and
+CUDA (NVIDIA), HIP (AMD), Level Zero (Intel) and Apple Metal through triton-ext; what a new backend provides is [`docs/nbxtensor/BACKENDS.md`](https://github.com/NeuroBrix/neurobrix/blob/main/docs/nbxtensor/BACKENDS.md), and
 `tests/nbxtensor_backend_conformance` is what it must pass. The backend in force is public API:
-`nbxtensor.backend.runtime.gpu_backend()` names it ("cuda", "hip", "metal"), `gpu_runtime()` is its runtime,
+`nbxtensor.backend.runtime.gpu_backend()` names it ("cuda", "hip", "xpu", "metal"), `gpu_runtime()` is its runtime,
 `backend_table()` its entry points, `device_memory()` its memory primitives, and `set_device(tensor)` binds the
 driver to a tensor's device before a launch.
 
@@ -370,7 +492,7 @@ with the tensor class (API.md §4).
 
 ## Versions
 
-The first publication is **0.0.1** (this version); each fix published after it increments the number (0.0.2, 0.0.3,
+The first publication was **0.0.1**; this is **0.0.2**. Each fix published after it increments the number (0.0.3,
 …), with no alpha or beta label. The rewrite of the core in Rust will be **2.0.0**. `nbxtensor.__version__` is the
 installed version.
 

{nbxtensor-0.0.1 → nbxtensor-0.0.2}/README.md

@@ -1,30 +1,103 @@
-# nbxtensor
-
-An inference-first tensor library, torch-free.
-
-PyTorch is a general framework built for research and training in eager mode: dynamic graphs, autograd, a generic
-dispatcher, a caching allocator that discovers memory as the program runs. Inference is a narrower problem, and a
-better-informed one: the graph and the shapes are known before the first run. nbxtensor is built for that case.
-
-- **Memory is planned, not discovered.** Allocators per device behind one interface, and a static planner that
-  assigns every buffer before the first run.
-- **Every kernel declares its cost.** Outputs, workspace and scratch with their lifetimes, bytes moved, flops. A
-  planner prices a model from these declarations instead of guessing what the allocator will do.
-- **Capture and replay are first-class.** A repeated sequence of launches is recorded, verified byte-equal and
-  replayed.
-- **One table of facts.** One dtype table, one view rule, one conversion per pair of dtypes, one capability table per
-  backend — declared, never guessed.
-- **Backends:** NVIDIA CUDA, Apple Metal (through triton-ext), CPU.
-- **No torch, ever.** nbxtensor never imports torch, not even optionally. Exchange with any tensor library goes
-  through the DLPack protocol (`__dlpack__`, `from_dlpack`), so a torch user writes `torch.from_dlpack(t)` on their
-  side.
-
-nbxtensor is extracted from NeuroBrix, the universal inference engine whose Triton engine is its first client. It
-lives in `nbxtensor/` of the NeuroBrix repository until the extraction is complete, then in its own repository.
+<p align="center"><img src="https://raw.githubusercontent.com/NeuroBrix/nbxtensor/main/assets/logo.png" alt="NeuroBrix" width="240"></p>
+
+# nbxtensor — the torch-free tensor library for inference
+
+Run AI models without PyTorch, on any GPU Triton supports: tensors, Triton kernels, planned memory and certified tuning.
+
+Runs wherever Triton runs — NVIDIA, AMD, Intel and Apple Silicon GPUs. Inference first; training is not in this
+release. Found a bug on your hardware? [Open an issue](https://github.com/NeuroBrix/nbxtensor/issues) — we fix fast.
+
+[![PyPI](https://img.shields.io/pypi/v/nbxtensor)](https://pypi.org/project/nbxtensor/)
+[![License: Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue)](https://github.com/NeuroBrix/nbxtensor/blob/main/LICENSE)
+[![Python](https://img.shields.io/pypi/pyversions/nbxtensor)](https://pypi.org/project/nbxtensor/)
+![torch-free](https://img.shields.io/badge/torch-free-success)
+![Triton](https://img.shields.io/badge/kernels-Triton-orange)
+
+```
+pip install nbxtensor
+```
+
+<!-- needs-gpu -->
+```python
+import numpy as np
+from nbxtensor.tensor import NBXTensor
+from nbxtensor.ops.linalg_wrappers import mm
+
+rng = np.random.default_rng(0)
+a = rng.standard_normal((256, 512)).astype(np.float16)
+b = rng.standard_normal((512, 128)).astype(np.float16)
+c = mm(NBXTensor.from_numpy(a), NBXTensor.from_numpy(b))      # a Triton kernel on the GPU, no torch
+ref = a.astype(np.float32) @ b.astype(np.float32)
+assert c.shape == (256, 128) and np.allclose(c.numpy().astype(np.float32), ref, rtol=2e-2, atol=2e-1)
+```
+
+## Why nbxtensor
+
+- **No PyTorch.** nbxtensor never imports torch, not even optionally; its tests, kernels and NeuroBrix's models run in
+  an environment where torch is not installed. Exchange with any tensor library goes through DLPack (`__dlpack__`,
+  `from_dlpack`), so a torch user writes `torch.from_dlpack(t)` on their side.
+- **Runs wherever Triton runs.** The kernels are Triton; a device is described by a file of facts (tiles, shared
+  memory, the matrix unit, the autotune ladders), never by code.
+- **Fast.** Faster than PyTorch on many models (in NeuroBrix's gate of 2026-11-05, the Triton engine on nbxtensor beat
+  the PyTorch engine on 18 of 49 models on a 16 GB V100 and 17 of 49 on a 32 GB one). Still running V100s? Triton leaves
+  their tensor cores idle (`tl.dot` compiles to FMA on sm_70); nbxtensor turns them back on with its own `mma.m8n8k4`
+  kernels (matrix products, convolutions, flash attention, the fused mixture-of-experts GEMM), chosen by the certificate
+  only where they measured faster — an 80 x 4096 x 4096 fp16 product 0.404 -> 0.191 ms, the fused MoE prefill 2.49 ->
+  0.96 ms (DeepSeek, 20 tokens) and 7.24 -> 1.40 ms (Qwen3, 512 tokens), fp32 1 x 1 convolutions 1.26-1.95 times faster
+  (V100, the measures recorded in [the 0.0.1 changelog](https://github.com/NeuroBrix/nbxtensor/blob/main/CHANGELOG.md#001---2026-11-05)).
+- **Lightweight.** nbxtensor itself is 1 MB; the whole install, Triton included, is about 1 GB — PyTorch's is 5.6 GB.
+- **Certified tuning.** Kernel settings are measured once per GPU class against a float64 oracle and shipped — no
+  autotuning at run time for the shapes we cover. We are certifying every model we can, so you never have to autotune.
+  Certified your GPU? [Share your certificates](#certify-your-own-hardware).
+- **Memory is planned, not discovered.** Allocators per device behind one interface, a static planner that assigns every
+  buffer before the first run, and every kernel declares its cost (outputs, workspace, scratch, bytes moved, flops) so a
+  planner prices a model before running it. Capture and replay of a repeated sequence of launches, verified byte-equal.
+
+## Certify your own hardware
+
+A kernel setting is certified per launch key (the kernel, its shape class and dtypes) and per GPU class. The keys a GPU
+class has no certificate for are swept at run time and kept on your machine; certifying them makes them permanent and
+shareable. Certification runs through [NeuroBrix](https://github.com/NeuroBrix/neurobrix), the engine whose models form
+the keys (`pip install neurobrix`), with these commands (run as written on a clean machine, 2026-11-06):
+
+1. Run the models you use. Every key no certificate covers is swept and kept in `~/.neurobrix/replay_cache/`:
+
+   ```
+   neurobrix run --model TinyLlama-1.1B-Chat-v1.0 --triton --prompt "Hello" --max-tokens 32
+   ```
+
+2. Certify the keys your card formed — every candidate setting is run against a float64 oracle and timed; the proofs are
+   written into `--out`, one file per kernel and dtype under `<vendor>/<profile>/` (the profile is your GPU's device
+   file, e.g. `volta`):
+
+   ```
+   neurobrix autotune certify --profile volta \
+       --keys ~/.neurobrix/replay_cache/autotune_configs_cuda-70-triton_3.8.0.json --out ./my-certificates
+   ```
+
+3. Check that they are served: a certified-only run uses only certificates and fails, naming the key, if one is missing:
+
+   ```
+   NEUROBRIX_AUTOTUNE_CERTIFIED_DIR=./my-certificates neurobrix run --model TinyLlama-1.1B-Chat-v1.0 --triton \
+       --prompt "Hello" --max-tokens 32 --certified-only
+   ```
+
+   Its `[autotune] certified directory:` line says `0 swept at runtime`, and its output is the same as step 1's.
+
+4. Share them: fork [NeuroBrix](https://github.com/NeuroBrix/neurobrix), add the files of `./my-certificates/<vendor>/<profile>/`
+   under `src/neurobrix/config/autotune/<vendor>/<profile>/`, and open a pull request with the `certificate` template:
+   your GPU model, driver, the commands above and the output of step 3. Certified your GPU? Share your certificates —
+   every class certified is a class nobody autotunes again.
+
+
+**Links:** [API reference](https://github.com/NeuroBrix/neurobrix/blob/main/docs/nbxtensor/API.md) ·
+[release notes](https://github.com/NeuroBrix/nbxtensor/blob/main/CHANGELOG.md) ·
+[NeuroBrix](https://github.com/NeuroBrix/neurobrix), the inference engine whose Triton engine is nbxtensor's first
+client; nbxtensor's source lives in its `nbxtensor/` directory and is exported here.
 
 ## Status
 
-Every area below is in the library, and every public name is documented in `docs/nbxtensor/API.md` (the list of
+Every area below is in the library, and every public name is documented in [`docs/nbxtensor/API.md`](https://github.com/NeuroBrix/neurobrix/blob/main/docs/nbxtensor/API.md) (the list of
 undocumented names is empty and a test keeps it so). This README documents what is in the library TODAY; every example below is executed by
 `nbxtensor/tests/test_readme.py`, so an example that stops working fails a test. The target API is
 `docs/nbxtensor/API.md`.
@@ -47,10 +120,20 @@ attempt to import torch, caught ones included, and recorded none). Python 3.10 o
 pyyaml and Triton.
 
 ```
-pip install -e nbxtensor          # from the NeuroBrix repository
+pip install nbxtensor
 ```
 
-**Linux, NVIDIA (CUDA).** Triton comes from PyPI with the install above.
+**Linux, NVIDIA (CUDA).** Triton and the CUDA runtime library come from PyPI with the install above; the NVIDIA driver
+is the only other requirement.
+
+**Linux, AMD (ROCm) and Intel (Level Zero).** The system's GPU runtime and a Triton that compiles for the card are
+needed: see [Install on AMD / Intel GPUs](#install-on-amd--intel-gpus).
+
+**Which GPU, and what is missing.** `python -m nbxtensor doctor` names the backend in force and, for every NVIDIA, AMD
+or Intel GPU on the PCI bus, whether its runtime library and its Triton backend are installed — with the vendor's
+install commands when one is not.
+
+**From the NeuroBrix repository** (contributors): `pip install -e nbxtensor`.
 
 **macOS, Apple silicon (Metal).** PyPI publishes no Triton for macOS, so Triton and its Metal backend are built from
 source, then nbxtensor is installed into that environment:
@@ -63,6 +146,46 @@ source, then nbxtensor is installed into that environment:
 
 `nbxtensor.dtype` and `nbxtensor.shape` need neither a device nor Triton.
 
+## Install on AMD / Intel GPUs
+
+The library speaks to the card through the vendor's own runtime library, loaded at run time, and compiles its kernels
+with the Triton backend for that card. When a card is present and either is missing, the library refuses with the
+steps below (the same text `python -m nbxtensor doctor` prints). The commands are the vendors' (Ubuntu 24.04).
+
+**AMD (ROCm, HIP).** The ROCm runtime, from AMD's
+[quick start](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/quick-start.html):
+
+```
+wget https://repo.radeon.com/amdgpu-install/6.4.3/ubuntu/noble/amdgpu-install_6.4.60403-1_all.deb
+sudo apt install ./amdgpu-install_6.4.60403-1_all.deb
+sudo apt update && sudo apt install rocm
+sudo usermod -a -G render,video $LOGNAME     # then log out and in (or reboot)
+pip install nbxtensor                        # the Triton wheel on PyPI carries the AMD backend
+```
+
+Targets: CDNA (gfx908, gfx90a, gfx942, gfx950) and RDNA 3 / 4 (gfx11xx, gfx120x); the hardware profile states the
+card's gfx arch. Triton 3.8 compiles the library's kernels for these targets on a machine without the card (the
+library's tests do it).
+
+**Intel (Level Zero).** Intel's GPU driver, from Intel's
+[package guide](https://dgpu-docs.intel.com/installation-guides/installing-packages-from-the-intel-ppa.html), then
+Intel's Triton build (the PyPI Triton has no Intel backend):
+
+```
+sudo add-apt-repository -y ppa:kobuk-team/intel-graphics
+sudo apt-get install -y libze-intel-gpu1 libze1 intel-opencl-icd clinfo intel-gsc
+sudo usermod -a -G render $LOGNAME           # then log out and in
+pip install triton-xpu --index-url https://download.pytorch.org/whl/xpu
+pip install nbxtensor
+```
+
+Devices: Data Center GPU Max (Ponte Vecchio), Arc A-series and Data Center GPU Flex (Alchemist), Arc B-series
+(Battlemage), Core Ultra 200V graphics (Lunar Lake); the card's PCI id selects its device file.
+
+**Status.** The AMD and Intel drivers are written to the level of the NVIDIA and Apple ones and proven against
+stand-ins of the vendors' libraries; no AMD or Intel card has run them yet. [Open an
+issue](https://github.com/NeuroBrix/nbxtensor/issues) with what your card does — we fix fast.
+
 ## Concepts
 
 **Plain data at the edges.** A dtype is named by its table name (`"float16"`, `"bfloat16"` …), a shape is a tuple of
@@ -317,22 +440,21 @@ The host conversions of a weight at load are split the same way, across the load
 
 ## Hardware
 
-| hardware | backend | state |
-|---|---|---|
-| NVIDIA Tesla V100 (Volta, sm_70) | CUDA (`CudaDriver`, `AbiMemory`) | **proven**: the library's tests and the reference models' outputs identical to their judged sha, on a 4 x V100 rack, 2026-10-29 |
-| Apple M4 Pro (Apple silicon, 24 GB unified) | Metal through triton-ext (`TritonExtDriver`, `MetalMemory`) | **proven**: the library's tests, the reference models' outputs identical to their judged sha, in an environment where torch is not installed, 2026-10-29 |
-| NVIDIA Ampere, Hopper | CUDA | **written, not proven here**: device files exist (`backend/devices/nvidia/`), no such card on our machines |
-| Apple M1, M2, M3, M5 families and the other M4 chips | Metal | **written, not proven here**: device files exist (`backend/devices/apple/`); only the M4 Pro is measured |
-| AMD Instinct (CDNA, CDNA2, CDNA3) | HIP through the CUDA-style C ABI (`AbiMemory`) | **interface ready, not tested for lack of hardware**: memory primitives and device files exist; the HIP launch driver is still to write (`docs/nbxtensor/BACKENDS.md` section 6) |
-| Intel GPUs (XPU) | Level Zero (not the CUDA ABI) | **interface ready, not tested for lack of hardware**: the backend contract is documented; its `DeviceMemory` over Level Zero and its launch driver are to write (BACKENDS.md section 6) |
-| a CPU | none yet | **not supported** for the kernels: host work (loading, conversions) runs with numpy on every core (`backend.cpu.pool`); a CPU kernel backend is not part of 0.0.1 |
-| hardware with no Triton backend (NPUs, phones) | none | **not supported**: covering it needs a Triton backend for it and the five pieces of `docs/nbxtensor/BACKENDS.md` |
+| vendor | backend | devices | state |
+|---|---|---|---|
+| NVIDIA | `cuda` (the CUDA driver and runtime) | Volta to Hopper (sm_70 to sm_90) | V100 16 GB and 32 GB: every release's gate |
+| Apple | `metal` (Metal through triton-ext) | M1 to M5 | M4 Pro: every release's gate |
+| AMD | `hip` (ROCm's HIP runtime) | CDNA 1 to 4, RDNA 3 and 4 | written and proven against a stand-in; no card has run it |
+| Intel | `xpu` (Level Zero) | Data Center GPU Max, Arc A and B, Lunar Lake | written and proven against a stand-in; no card has run it |
+
+Each device has a device file (`nbxtensor/backend/devices/<vendor>/*.yml`): its shared memory, warp size, tiles and
+tuning screens, every value with its source.
 
 ## Backends
 
-CUDA (NVIDIA) and Apple Metal through triton-ext; what a new backend provides is `docs/nbxtensor/BACKENDS.md`, and
+CUDA (NVIDIA), HIP (AMD), Level Zero (Intel) and Apple Metal through triton-ext; what a new backend provides is [`docs/nbxtensor/BACKENDS.md`](https://github.com/NeuroBrix/neurobrix/blob/main/docs/nbxtensor/BACKENDS.md), and
 `tests/nbxtensor_backend_conformance` is what it must pass. The backend in force is public API:
-`nbxtensor.backend.runtime.gpu_backend()` names it ("cuda", "hip", "metal"), `gpu_runtime()` is its runtime,
+`nbxtensor.backend.runtime.gpu_backend()` names it ("cuda", "hip", "xpu", "metal"), `gpu_runtime()` is its runtime,
 `backend_table()` its entry points, `device_memory()` its memory primitives, and `set_device(tensor)` binds the
 driver to a tensor's device before a launch.
 
@@ -350,7 +472,7 @@ with the tensor class (API.md §4).
 
 ## Versions
 
-The first publication is **0.0.1** (this version); each fix published after it increments the number (0.0.2, 0.0.3,
+The first publication was **0.0.1**; this is **0.0.2**. Each fix published after it increments the number (0.0.3,
 …), with no alpha or beta label. The rewrite of the core in Rust will be **2.0.0**. `nbxtensor.__version__` is the
 installed version.