tensoracle 0.4.0__tar.gz → 0.5.0__tar.gz

{tensoracle-0.4.0 → tensoracle-0.5.0}/.gitignore

@@ -2,5 +2,5 @@
 .nf/logs/
 .venv/
 __pycache__/
-src/tensoracle/_version.py
+python/src/tensoracle/_version.py
 site/

{tensoracle-0.4.0 → tensoracle-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tensoracle
-Version: 0.4.0
+Version: 0.5.0
 Summary: Build, save, reuse, and execute TensorRT engines from safetensors weights
 Project-URL: documentation, https://sync-dev-org.github.io/tensoracle/
 Project-URL: repository, https://github.com/sync-dev-org/tensoracle
@@ -44,7 +44,7 @@ Choose the path that matches your work:
 | --- | --- | --- |
 | Inference application | [Reopen a saved engine](#use-a-saved-engine-for-inference) and enqueue it with `handle.create_execution()`. | The application supplies device buffers and CUDA streams, for example through CuPy or `cuda-python`. |
 | Standard graph developer | [Define and build a graph](#define-and-build-a-standard-graph), [save and reopen](#save-and-reopen-an-engine), and [compare outputs](#compare-output-arrays). | None for standard TensorRT layers. |
-| Cubin plugin developer | [Build a cubin plugin](#build-a-cubin-plugin) and package it with the model. | `tensoracle[producer]`, CUDA toolkit, TensorRT SDK, CMake, and a C++17 compiler. |
+| Cubin plugin developer | [Build a cubin plugin](#build-a-cubin-plugin) and package it with the model. | `tensoracle[producer]`, CUDA toolkit, TensorRT SDK, CMake 3.20 or later, and a C++17 compiler. |
 
 | Requirement | Inference application | Standard graph developer | Cubin plugin developer |
 | --- | --- | --- | --- |
@@ -55,7 +55,7 @@ Choose the path that matches your work:
 | Triton 3.8.0, `ptxas` 12.9.86, `cuobjdump` 13.3 | Not needed | Not needed | Installed with `tensoracle[producer]` for cubin production |
 | CUDA toolkit | Not needed | Not needed | Native plugin build |
 | TensorRT 11 SDK | Not needed | Not needed | Native plugin build |
-| CMake and C++17 compiler | Not needed | Not needed | Native plugin build |
+| CMake 3.20 or later and C++17 compiler | Not needed | Not needed | Native plugin build |
 
 The supported targets are Linux x86_64 (including WSL2), TensorRT 11, and GPU generations sm86, sm89, and sm120. The producer and native build tools are separate from the `tensoracle` runtime dependencies. Engine building, engine reuse, output comparison, and cubin production do not require torch. Converting `.pth` checkpoints requires the `convert` extra described below.
 
@@ -104,7 +104,6 @@ def graph(network: trt.INetworkDefinition) -> None:
 
 
 declaration = EngineBuild(
-    precision="fp32",
     inputs={"features": InputShape.ranged((1, 1, 2), (1, 64, 2), (1, 448, 2))},
     settings=BuilderSettings(workspace_limit=1 << 30, optimization_level=3),
 )
@@ -113,7 +112,7 @@ plan: bytes = build_serialized_engine(graph, declaration)
 
 Use `InputShape.fixed((1, 64, 2))` for one input size; the graph may declare that size directly or use `-1` for a dimension fixed by the profile. For a range, put `-1` in each varying dimension of the graph input and declare its minimum, optimum, and maximum sizes. Every graph input needs a matching entry in `EngineBuild.inputs`.
 
-Set `precision="fp16"` to declare an fp16 engine. The graph function chooses each tensor's actual type, including any fp32 tensors inside an fp16 graph. TF32 is always disabled; `BuilderSettings(tf32=True)` raises `ValueError`. The function returns serialized plan bytes. Invalid declarations raise `ValueError`, and a failed TensorRT build raises `RuntimeError`.
+The engine's precision is determined by the tensor types that the graph function places in the network. For an fp16 graph, use `trt.float16` tensors and `np.float16` weights where needed; the graph can also contain fp32 tensors. `EngineBuild` declares input shapes and builder settings. TF32 is always disabled; `BuilderSettings(tf32=True)` raises `ValueError`. The function returns serialized plan bytes. Invalid declarations raise `ValueError`, and a failed TensorRT build raises `RuntimeError`.
 
 ## Save and reopen an engine
 
@@ -138,7 +137,7 @@ handle = open_engine(
 engine = handle.engine  # Keep handle.runtime alive while using the engine.
 ```
 
-Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under `~/.cache` when that variable is unset. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, precision, input shapes, builder settings, GPU product name, OS, CPU architecture, and TensorRT version. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact.
+Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under `~/.cache` when that variable is unset. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, builder settings, GPU product name, OS, CPU architecture, and TensorRT version. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact.
 
 `plugins` names native TensorRT plugin libraries. Both `open_engine` and `build_serialized_engine` accept it. Each function checks that every library exports `getCreators`, loads it through TensorRT's registry, and retains it for the process before building the graph or deserializing an engine. Repeating the same library does not register it twice. `open_engine` also hashes the loaded library bytes into the engine identity; a changed library selects a different saved engine in a new process. A missing, unreadable, or incompatible library, or a library changed after loading in the same process, raises `PluginLibraryError` before a cached plan is opened. Keep the library and its cubin bundle together in the caller's wheel.
 
@@ -265,6 +264,18 @@ Install the producer extra when you need to compile cubins from your own kernels
 pip install 'tensoracle[producer]'
 ```
 
+Every repository that adopts tensoracle must use this top-level layout:
+
+```text
+cpp/
+  CMakeLists.txt
+python/
+  pyproject.toml
+  src/
+```
+
+Keep C++ sources in `cpp/`, with a CMake project that builds and runs its tests through CTest without Python. Keep the Python project, its src-layout package, and its uv environment in `python/`; run the project's uv commands from that directory. Tensoracle itself follows this layout, and the installed C++ helper paths described below remain inside its Python package.
+
 `tensoracle.producer.compile_bundle` uses the extra's **Triton 3.8.0**, **ptxas 12.9.86**, and **cuobjdump 13.3**. It finds the NVIDIA wheel executables by default; explicit `ptxas` and `cuobjdump` paths can override them. The normal import, engine build, and engine reuse paths do not import the producer or Triton. Cubin production does not need a GPU or import torch; run it without importing torch first. The caller supplies Triton functions and `KernelSpec` records with their public argument types, compile time constants, and any divisibility assumptions. The producer writes sm86, sm89, and sm120 cubins, `bundle.json`, and a C++ translation unit that embeds exactly those cubins. The generated C++ entry records carry each public argument's order, type, count, and divisibility rule into the common launcher. It rejects an unexpected parameter ABI or nonzero scratch requirement. The included [example](https://github.com/sync-dev-org/tensoracle/blob/main/examples/cubin_plugin/README.md#rebuild-the-cubins) has the command to rebuild its cubins from a repository clone.
 
 The wheel also contains `tensoracle/cpp/include`, `tensoracle/cpp/src`, and `tensoracle/cpp/cmake`. Pass the installed `tensoracle/cpp` directory to CMake as `TENSORACLE_CPP_ROOT` and include `TensoracleCubinPlugin.cmake`. Its `tensoracle_add_cubin_plugin` function compiles the common C++ source, the caller's plugin source, and the generated `bundle.cpp` into the caller's library. The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common source owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common source releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.

{tensoracle-0.4.0 → tensoracle-0.5.0}/README.md

@@ -14,7 +14,7 @@ Choose the path that matches your work:
 | --- | --- | --- |
 | Inference application | [Reopen a saved engine](#use-a-saved-engine-for-inference) and enqueue it with `handle.create_execution()`. | The application supplies device buffers and CUDA streams, for example through CuPy or `cuda-python`. |
 | Standard graph developer | [Define and build a graph](#define-and-build-a-standard-graph), [save and reopen](#save-and-reopen-an-engine), and [compare outputs](#compare-output-arrays). | None for standard TensorRT layers. |
-| Cubin plugin developer | [Build a cubin plugin](#build-a-cubin-plugin) and package it with the model. | `tensoracle[producer]`, CUDA toolkit, TensorRT SDK, CMake, and a C++17 compiler. |
+| Cubin plugin developer | [Build a cubin plugin](#build-a-cubin-plugin) and package it with the model. | `tensoracle[producer]`, CUDA toolkit, TensorRT SDK, CMake 3.20 or later, and a C++17 compiler. |
 
 | Requirement | Inference application | Standard graph developer | Cubin plugin developer |
 | --- | --- | --- | --- |
@@ -25,7 +25,7 @@ Choose the path that matches your work:
 | Triton 3.8.0, `ptxas` 12.9.86, `cuobjdump` 13.3 | Not needed | Not needed | Installed with `tensoracle[producer]` for cubin production |
 | CUDA toolkit | Not needed | Not needed | Native plugin build |
 | TensorRT 11 SDK | Not needed | Not needed | Native plugin build |
-| CMake and C++17 compiler | Not needed | Not needed | Native plugin build |
+| CMake 3.20 or later and C++17 compiler | Not needed | Not needed | Native plugin build |
 
 The supported targets are Linux x86_64 (including WSL2), TensorRT 11, and GPU generations sm86, sm89, and sm120. The producer and native build tools are separate from the `tensoracle` runtime dependencies. Engine building, engine reuse, output comparison, and cubin production do not require torch. Converting `.pth` checkpoints requires the `convert` extra described below.
 
@@ -74,7 +74,6 @@ def graph(network: trt.INetworkDefinition) -> None:
 
 
 declaration = EngineBuild(
-    precision="fp32",
     inputs={"features": InputShape.ranged((1, 1, 2), (1, 64, 2), (1, 448, 2))},
     settings=BuilderSettings(workspace_limit=1 << 30, optimization_level=3),
 )
@@ -83,7 +82,7 @@ plan: bytes = build_serialized_engine(graph, declaration)
 
 Use `InputShape.fixed((1, 64, 2))` for one input size; the graph may declare that size directly or use `-1` for a dimension fixed by the profile. For a range, put `-1` in each varying dimension of the graph input and declare its minimum, optimum, and maximum sizes. Every graph input needs a matching entry in `EngineBuild.inputs`.
 
-Set `precision="fp16"` to declare an fp16 engine. The graph function chooses each tensor's actual type, including any fp32 tensors inside an fp16 graph. TF32 is always disabled; `BuilderSettings(tf32=True)` raises `ValueError`. The function returns serialized plan bytes. Invalid declarations raise `ValueError`, and a failed TensorRT build raises `RuntimeError`.
+The engine's precision is determined by the tensor types that the graph function places in the network. For an fp16 graph, use `trt.float16` tensors and `np.float16` weights where needed; the graph can also contain fp32 tensors. `EngineBuild` declares input shapes and builder settings. TF32 is always disabled; `BuilderSettings(tf32=True)` raises `ValueError`. The function returns serialized plan bytes. Invalid declarations raise `ValueError`, and a failed TensorRT build raises `RuntimeError`.
 
 ## Save and reopen an engine
 
@@ -108,7 +107,7 @@ handle = open_engine(
 engine = handle.engine  # Keep handle.runtime alive while using the engine.
 ```
 
-Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under `~/.cache` when that variable is unset. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, precision, input shapes, builder settings, GPU product name, OS, CPU architecture, and TensorRT version. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact.
+Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under `~/.cache` when that variable is unset. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, builder settings, GPU product name, OS, CPU architecture, and TensorRT version. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact.
 
 `plugins` names native TensorRT plugin libraries. Both `open_engine` and `build_serialized_engine` accept it. Each function checks that every library exports `getCreators`, loads it through TensorRT's registry, and retains it for the process before building the graph or deserializing an engine. Repeating the same library does not register it twice. `open_engine` also hashes the loaded library bytes into the engine identity; a changed library selects a different saved engine in a new process. A missing, unreadable, or incompatible library, or a library changed after loading in the same process, raises `PluginLibraryError` before a cached plan is opened. Keep the library and its cubin bundle together in the caller's wheel.
 
@@ -235,6 +234,18 @@ Install the producer extra when you need to compile cubins from your own kernels
 pip install 'tensoracle[producer]'
 ```
 
+Every repository that adopts tensoracle must use this top-level layout:
+
+```text
+cpp/
+  CMakeLists.txt
+python/
+  pyproject.toml
+  src/
+```
+
+Keep C++ sources in `cpp/`, with a CMake project that builds and runs its tests through CTest without Python. Keep the Python project, its src-layout package, and its uv environment in `python/`; run the project's uv commands from that directory. Tensoracle itself follows this layout, and the installed C++ helper paths described below remain inside its Python package.
+
 `tensoracle.producer.compile_bundle` uses the extra's **Triton 3.8.0**, **ptxas 12.9.86**, and **cuobjdump 13.3**. It finds the NVIDIA wheel executables by default; explicit `ptxas` and `cuobjdump` paths can override them. The normal import, engine build, and engine reuse paths do not import the producer or Triton. Cubin production does not need a GPU or import torch; run it without importing torch first. The caller supplies Triton functions and `KernelSpec` records with their public argument types, compile time constants, and any divisibility assumptions. The producer writes sm86, sm89, and sm120 cubins, `bundle.json`, and a C++ translation unit that embeds exactly those cubins. The generated C++ entry records carry each public argument's order, type, count, and divisibility rule into the common launcher. It rejects an unexpected parameter ABI or nonzero scratch requirement. The included [example](https://github.com/sync-dev-org/tensoracle/blob/main/examples/cubin_plugin/README.md#rebuild-the-cubins) has the command to rebuild its cubins from a repository clone.
 
 The wheel also contains `tensoracle/cpp/include`, `tensoracle/cpp/src`, and `tensoracle/cpp/cmake`. Pass the installed `tensoracle/cpp` directory to CMake as `TENSORACLE_CPP_ROOT` and include `TensoracleCubinPlugin.cmake`. Its `tensoracle_add_cubin_plugin` function compiles the common C++ source, the caller's plugin source, and the generated `bundle.cpp` into the caller's library. The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common source owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common source releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.

tensoracle-0.5.0/build_backend.py

@@ -0,0 +1,94 @@
+"""Build from the Python project using the repository's canonical shared sources."""
+
+import shutil
+import tempfile
+import tomllib
+from contextlib import contextmanager
+from pathlib import Path
+
+from hatchling import build as hatchling_build
+from hatchling.builders.sdist import SdistBuilder
+from hatchling.builders.wheel import WheelBuilder
+
+get_requires_for_build_editable = hatchling_build.get_requires_for_build_editable
+get_requires_for_build_sdist = hatchling_build.get_requires_for_build_sdist
+get_requires_for_build_wheel = hatchling_build.get_requires_for_build_wheel
+
+
+@contextmanager
+def _builder(builder_class):
+    project = Path(__file__).resolve().parent
+    config = tomllib.loads((project / "pyproject.toml").read_text(encoding="utf-8"))
+    # Source archives already contain the shared files next to pyproject.toml.
+    source_archive = (project / "PKG-INFO").is_file()
+    repository = project if source_archive else project.parent
+    shared = ("README.md", "LICENSE", "cpp", "docs", "examples", "mkdocs.yml", ".gitignore")
+    for name in shared:
+        if not (repository / name).exists():
+            raise FileNotFoundError(f"Required build source is missing: {repository / name}")
+    if source_archive:
+        yield builder_class(str(project), config=config)
+        return
+
+    with tempfile.TemporaryDirectory(prefix="tensoracle-build-") as directory:
+        staged = Path(directory)
+        for base, names in (
+            (project, ("pyproject.toml", "uv.lock", "build_backend.py", "src", "tests")),
+            (repository, shared),
+        ):
+            for name in names:
+                source, destination = base / name, staged / name
+                if source.is_dir():
+                    shutil.copytree(
+                        source,
+                        destination,
+                        ignore=shutil.ignore_patterns("__pycache__", "*.pyc", ".pytest_cache"),
+                    )
+                else:
+                    shutil.copyfile(source, destination)
+        hatch = config["tool"]["hatch"]
+        hatch["version"]["raw-options"]["root"] = str(repository)
+        hatch["build"]["dev-mode-dirs"] = [str(project / "src")]
+        yield builder_class(str(staged), config=config)
+
+
+def _build(builder_class, directory, version):
+    output = str(Path(directory).resolve())
+    with _builder(builder_class) as builder:
+        artifact = next(builder.build(directory=output, versions=[version]))
+        if version == "editable":
+            generated = Path(builder.root) / "src/tensoracle/_version.py"
+            target = Path(__file__).resolve().parent / "src/tensoracle/_version.py"
+            if generated != target:
+                shutil.copyfile(generated, target)
+        return Path(artifact).name
+
+
+def build_sdist(sdist_directory, config_settings=None):
+    return _build(SdistBuilder, sdist_directory, "standard")
+
+
+def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
+    return _build(WheelBuilder, wheel_directory, "standard")
+
+
+def build_editable(wheel_directory, config_settings=None, metadata_directory=None):
+    return _build(WheelBuilder, wheel_directory, "editable")
+
+
+def prepare_metadata_for_build_wheel(metadata_directory, config_settings=None):
+    with _builder(WheelBuilder) as builder:
+        builder.metadata.validate_fields()
+        directory = Path(metadata_directory) / f"{builder.artifact_project_id}.dist-info"
+        directory.mkdir(parents=True, exist_ok=True)
+        metadata = builder.config.core_metadata_constructor(builder.metadata)
+        (directory / "METADATA").write_text(metadata, encoding="utf-8")
+        for relative in builder.metadata.core.license_files:
+            target = directory / "licenses" / relative
+            target.parent.mkdir(parents=True, exist_ok=True)
+            shutil.copyfile(Path(builder.root) / relative, target)
+        return directory.name
+
+
+def prepare_metadata_for_build_editable(metadata_directory, config_settings=None):
+    return prepare_metadata_for_build_wheel(metadata_directory, config_settings)

tensoracle-0.5.0/cpp/CMakeLists.txt

@@ -0,0 +1,23 @@
+cmake_minimum_required(VERSION 3.20)
+project(tensoracle_cubin LANGUAGES CXX)
+
+set(TENSORRT_ROOT "" CACHE PATH "TensorRT 11 SDK root containing include/NvInfer.h")
+if(NOT EXISTS "${TENSORRT_ROOT}/include/NvInfer.h")
+    message(FATAL_ERROR "TensorRT SDK headers are missing; set TENSORRT_ROOT to the TensorRT 11 SDK")
+endif()
+find_package(CUDAToolkit REQUIRED)
+
+add_library(tensoracle_cubin STATIC src/cubin_plugin.cpp)
+target_compile_features(tensoracle_cubin PUBLIC cxx_std_17)
+target_include_directories(tensoracle_cubin PUBLIC
+    "${CMAKE_CURRENT_SOURCE_DIR}/include" "${TENSORRT_ROOT}/include")
+target_link_libraries(tensoracle_cubin PUBLIC CUDA::cuda_driver)
+target_compile_options(tensoracle_cubin PRIVATE
+    $<$<CXX_COMPILER_ID:GNU,Clang>:-Wno-deprecated-declarations>)
+
+include(CTest)
+if(BUILD_TESTING)
+    add_executable(tensoracle_core_test tests/core.cpp)
+    target_link_libraries(tensoracle_core_test PRIVATE tensoracle_cubin)
+    add_test(NAME tensoracle_fields_and_sha256 COMMAND tensoracle_core_test)
+endif()

tensoracle-0.5.0/cpp/tests/core.cpp

@@ -0,0 +1,31 @@
+// SP-TENSORACLE-004: retain typed fields in order and compute SHA-256.
+#include "tensoracle/cubin_plugin.hpp"
+#include <cstring>
+using namespace tensoracle::cubin;
+using namespace nvinfer1;
+int main() {
+    if (sha256("abc", 3) !=
+        "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad") return 1;
+    float weight[2] = {1.25f, 2.5f};
+    int version = 7;
+    PluginField input[] = {
+        {"weight", weight, PluginFieldType::kFLOAT32, 2},
+        {"version", &version, PluginFieldType::kINT32, 1},
+    };
+    PluginFieldCollection source{2, input};
+    Fields fields;
+    if (!Fields::parse(&source, fields)) return 2;
+    auto const* saved = fields.collection();
+    if (saved->nbFields != 2 ||
+        std::strcmp(saved->fields[0].name, "weight") != 0 ||
+        std::strcmp(saved->fields[1].name, "version") != 0 ||
+        saved->fields[0].type != PluginFieldType::kFLOAT32 ||
+        saved->fields[0].length != 2 ||
+        std::memcmp(saved->fields[0].data, weight, sizeof(weight)) != 0) return 3;
+    weight[0] = 9.0f;
+    if (std::memcmp(saved->fields[0].data, weight, sizeof(weight)) == 0) return 4;
+    Fields copy = fields;
+    if (std::memcmp(copy.collection()->fields[0].data, saved->fields[0].data,
+                    sizeof(weight)) != 0) return 5;
+    return 0;
+}

{tensoracle-0.4.0 → tensoracle-0.5.0}/docs/index.md

@@ -16,7 +16,7 @@ Follow the [four-stage optimization workflow](optimization.md) to establish outp
 | --- | --- | --- |
 | Inference application | Use `open_engine` to reopen a matching saved engine, then enqueue it with `handle.create_execution()`. | The application supplies buffers and streams, for example through CuPy or `cuda-python`. |
 | Standard graph developer | Define a TensorRT graph, build it with `build_serialized_engine` or `open_engine`, and measure outputs with `compare_outputs`. | None for standard TensorRT layers. |
-| Cubin plugin developer | Produce cubins, build a native TensorRT plugin, and package it with the model. | `tensoracle[producer]`, CUDA toolkit, TensorRT SDK, CMake, and a C++17 compiler. |
+| Cubin plugin developer | Produce cubins, build a native TensorRT plugin, and package it with the model. | `tensoracle[producer]`, CUDA toolkit, TensorRT SDK, CMake 3.20 or later, and a C++17 compiler. |
 
 | Requirement | Inference application | Standard graph developer | Cubin plugin developer |
 | --- | --- | --- | --- |
@@ -27,7 +27,7 @@ Follow the [four-stage optimization workflow](optimization.md) to establish outp
 | Triton 3.8.0, `ptxas` 12.9.86, `cuobjdump` 13.3 | Not needed | Not needed | Installed with `tensoracle[producer]` for cubin production |
 | CUDA toolkit | Not needed | Not needed | Native plugin build |
 | TensorRT 11 SDK | Not needed | Not needed | Native plugin build |
-| CMake and C++17 compiler | Not needed | Not needed | Native plugin build |
+| CMake 3.20 or later and C++17 compiler | Not needed | Not needed | Native plugin build |
 
 The supported platform is Linux x86_64 (including WSL2) with sm86, sm89, or sm120 GPUs and TensorRT 11. The producer and native build tools are separate from the `tensoracle` runtime dependencies. Engine building, engine reuse, output comparison, and cubin production do not require torch. Converting `.pth` checkpoints requires the `convert` extra.
 
@@ -161,12 +161,14 @@ Networks with outputs marked as shape tensors emit `tensoracle.ShapeTensorOutput
 
 ### Build a standard graph
 
-Define `graph(network)` with TensorRT's Python network API and load model weights from safetensors. Describe each graph input with `InputShape.fixed` or `InputShape.ranged`, and pass an `EngineBuild` declaration to `build_serialized_engine`. `open_engine` uses the same declaration plus a name, graph version, weight files, and `EngineIO` to save and reuse a checked engine. The supported precisions are fp32 (with TF32 disabled) and fp16. The [graph example](https://github.com/sync-dev-org/tensoracle#define-and-build-a-standard-graph) shows the exact calls.
+Define `graph(network)` with TensorRT's Python network API and load model weights from safetensors. Describe each graph input with `InputShape.fixed` or `InputShape.ranged`, and pass an `EngineBuild` declaration to `build_serialized_engine`. `EngineBuild` declares input shapes and builder settings; the graph function determines precision through the tensor types it places in the network. The supported precisions are fp32 (with TF32 disabled) and fp16. `open_engine` uses the same declaration plus a name, graph version, weight files, and `EngineIO` to save and reuse a checked engine. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. The [graph example](https://github.com/sync-dev-org/tensoracle#define-and-build-a-standard-graph) shows the exact calls.
 
 The application runs the engine and passes a CPU NumPy candidate output and a corresponding reference output to `compare_outputs`. The application chooses the acceptance limits. See [Compare output arrays](https://github.com/sync-dev-org/tensoracle#compare-output-arrays).
 
 ### Build a cubin plugin
 
+Repositories that adopt tensoracle must follow the [repository layout in the plugin guide](https://github.com/sync-dev-org/tensoracle#build-a-cubin-plugin).
+
 Install `tensoracle[producer]` when you need to compile cubins from your own kernels:
 
 ```bash

{tensoracle-0.4.0 → tensoracle-0.5.0}/docs/optimization.md

@@ -4,13 +4,13 @@ Use the same representative inputs, shapes, and corresponding outputs throughout
 
 ## 1. Establish the fp32 reference
 
-- **tensoracle tools:** Build an fp32 TensorRT engine with `EngineBuild(precision="fp32")` and `build_serialized_engine` or `open_engine`. TF32 is disabled for the engine. Run the engine in the application, then use `compare_outputs(candidate, reference, atol=..., rtol=...)` on corresponding CPU NumPy output arrays.
+- **tensoracle tools:** Build an fp32 TensorRT engine from a graph whose tensor types are fp32, using an `EngineBuild` declaration for input shapes and builder settings with `build_serialized_engine` or `open_engine`. TF32 is disabled for the engine. Run the engine in the application, then use `compare_outputs(candidate, reference, atol=..., rtol=...)` on corresponding CPU NumPy output arrays.
 - **You provide:** Run the official PyTorch implementation in fp32 outside `tensoracle` and fix its outputs as the reference. Supply the TensorRT graph, weights, representative inputs, and an acceptance limit for each output. Keep the reference and candidate inputs and preprocessing identical.
 - **Gate:** Reject non-finite outputs and any output that misses your limits. Advance only when every fp32 TensorRT output passes against its fixed PyTorch reference. Correct the graph or inputs before trying lower precision if this gate fails.
 
 ## 2. Accept lower precision by quality
 
-- **tensoracle tools:** Build an fp16 TensorRT engine with `EngineBuild(precision="fp16")`. Use `compare_outputs` with each fp16 output as the candidate and its accepted fp32 TensorRT output as the reference. It reports non-finite counts and, for finite floating point arrays, maximum absolute error, relative L2 error, cosine similarity, and, when both tolerances are supplied, counts and fractions outside tolerance.
+- **tensoracle tools:** Build an fp16 TensorRT engine from a graph whose tensor types are fp16, using the same `EngineBuild` declaration for input shapes and builder settings. The graph function determines precision through its tensor types. When saving with `open_engine`, use a different `name` or `graph_version` from the fp32 engine. Use `compare_outputs` with each fp16 output as the candidate and its accepted fp32 TensorRT output as the reference. It reports non-finite counts and, for finite floating point arrays, maximum absolute error, relative L2 error, cosine similarity, and, when both tolerances are supplied, counts and fractions outside tolerance.
 - **You provide:** Choose limits for those model-independent measurements. Compute any model-specific measurements separately, such as PSNR for images or application-specific time-series quality measures, and choose their limits. Run both engines on the same inputs.
 - **Gate:** Accept the fp16 engine only when its outputs meet all applicable limits, including the model-specific ones. Otherwise keep the accepted fp32 engine or revise the candidate and repeat this stage.
 

{tensoracle-0.4.0 → tensoracle-0.5.0}/docs/tensorrt-sdk.md

@@ -102,7 +102,7 @@ Keep `TENSORRT_ROOT` set to the absolute path of the verified `sdk` directory. F
 
 ```bash
 CMAKE_ARGS="-DTENSORRT_ROOT=$TENSORRT_ROOT" uv build examples/cubin_plugin --wheel
-TENSORACLE_SDK_ROOT="$TENSORRT_ROOT" uv run pytest -q tests/test_example_plugin.py
+TENSORACLE_SDK_ROOT="$TENSORRT_ROOT" uv run --project python pytest -q python/tests/test_example_plugin.py
 ```
 
 The tests also need the CUDA toolkit; tests marked `gpu` require an NVIDIA GPU and driver. The SDK is a build input. The application still selects the TensorRT runtime and verifies its own plugin with that runtime.

{tensoracle-0.4.0 → tensoracle-0.5.0}/examples/cubin_plugin/README.md

@@ -9,7 +9,7 @@ The first plugin computes an fp32 32 by 16 dot product and then scales it in a s
 From the tensoracle repository root, run:
 
 ```bash
-uv run --extra producer python -c 'from pathlib import Path; from examples.cubin_plugin.kernels import kernel_specs; from tensoracle.producer import compile_bundle; compile_bundle(kernel_specs(), Path("examples/cubin_plugin/bundle"))'
+uv run --project python --extra producer python -c 'from pathlib import Path; from examples.cubin_plugin.kernels import kernel_specs; from tensoracle.producer import compile_bundle; compile_bundle(kernel_specs(), Path("examples/cubin_plugin/bundle"))'
 ```
 
 This installs `tensoracle[producer]` with Triton 3.8.0, ptxas 12.9.86, and cuobjdump 13.3, and recreates `bundle/` without using a GPU. `compile_bundle` finds ptxas and cuobjdump in their installed NVIDIA wheels. The producer checks the cubin ELF SM, public and hidden argument layout, register count, and zero scratch sizes before writing the manifest. Run the tests below with the producer extra alone so torch is absent; `compile_bundle` rejects an already imported torch.
@@ -17,7 +17,7 @@ This installs `tensoracle[producer]` with Triton 3.8.0, ptxas 12.9.86, and cuobj
 Check that the regenerated files match the committed example without using a GPU or setting producer environment variables:
 
 ```bash
-uv run --extra producer pytest -m "not gpu" tests/test_producer.py
+uv run --project python --extra producer pytest -m "not gpu" python/tests/test_producer.py
 ```
 
 Without the producer extra, the compilation tests skip with an explanation of the missing tools.
@@ -39,7 +39,7 @@ Install the tensoracle wheel and this example wheel into a runtime Python enviro
 Run from the tensoracle repository with the SDK and CUDA toolkit paths set. The GPU tests build the example wheel, install it in a temporary runtime environment, and run dynamic shapes, direct builds, saved engine reuse, simultaneous execution in two distinct CUDA contexts, and an fp32 versus TF32 control. Each GPU subprocess has a 50 second limit.
 
 ```bash
-TENSORACLE_SDK_ROOT="$TENSORRT_ROOT" uv run pytest -q tests/test_example_plugin.py
+TENSORACLE_SDK_ROOT="$TENSORRT_ROOT" uv run --project python pytest -q python/tests/test_example_plugin.py
 ```
 
 Record the GPU name, SM, TensorRT runtime version, test result, and wheel digest on sm86, sm89, and sm120 separately. The code and wheel contain all three cubin generations; successful execution on one generation does not establish the other two.

{tensoracle-0.4.0 → tensoracle-0.5.0}/examples/cubin_plugin/example_cubin_plugin/__init__.py

@@ -27,7 +27,6 @@ def weights_path() -> Path:
 
 def declaration() -> EngineBuild:
     return EngineBuild(
-        "fp32",
         {"features": InputShape.ranged((1, 32), (16, 32), (32, 32))},
         BuilderSettings(optimization_level=0),
     )

{tensoracle-0.4.0 → tensoracle-0.5.0}/mkdocs.yml

@@ -17,6 +17,8 @@ plugins:
       handlers:
         python:
           paths:
+            - python/src
+            # Standalone source archives keep src next to pyproject.toml.
             - src
           options:
             docstring_style: google

{tensoracle-0.4.0 → tensoracle-0.5.0}/pyproject.toml

@@ -38,13 +38,16 @@ producer = [
 
 [build-system]
 requires = ["hatchling", "hatch-vcs"]
-build-backend = "hatchling.build"
+build-backend = "build_backend"
+backend-path = ["."]
 
 [tool.hatch.version]
 source = "vcs"
 fallback-version = "0.0.0.dev0"
 
 [tool.hatch.version.raw-options]
+root = ".."
+fallback_root = "."
 version_scheme = "only-version"
 local_scheme = "no-local-version"
 
@@ -52,20 +55,34 @@ local_scheme = "no-local-version"
 version-file = "src/tensoracle/_version.py"
 
 [tool.hatch.build.targets.sdist]
-exclude = ["/.nf/", "/.github/", "/AGENTS.md", "/CLAUDE.md", "/tools/"]
+exclude = [
+    "/.nf/",
+    "/.github/",
+    "/AGENTS.md",
+    "/CLAUDE.md",
+    "/tools/",
+    "/noren.yaml",
+    "/noren-words.txt",
+]
 
 [tool.hatch.build.targets.wheel]
 packages = ["src/tensoracle"]
 
 [tool.hatch.build.targets.wheel.force-include]
-cpp = "tensoracle/cpp"
+"cpp/cmake" = "tensoracle/cpp/cmake"
+"cpp/include" = "tensoracle/cpp/include"
+"cpp/src" = "tensoracle/cpp/src"
 
 [dependency-groups]
+docs = [
+    "mkdocs>=1.6.1",
+    "mkdocs-material>=9.7.7",
+    "mkdocstrings[python]>=1.0.6",
+]
 dev = [
+    { include-group = "docs" },
     "cuda-python>=13.0",
     "cupy-cuda13x>=14.2.0",
-    "mkdocs-material>=9.7.7",
-    "mkdocstrings[python]>=1.0.6",
     "pytest>=8.0",
     "ruff>=0.8",
 ]
@@ -75,7 +92,7 @@ testpaths = ["tests"]
 addopts = ["--strict-markers"]
 markers = [
     "gpu: tests that build or load a TensorRT engine on a CUDA GPU",
-    "req(id): parent requirement ID in .nf/docs/requirements.md",
+    "req(id): ID of the adopted spec item that the test verifies",
 ]
 
 [tool.ruff]

{tensoracle-0.4.0 → tensoracle-0.5.0}/src/tensoracle/_build.py

@@ -4,7 +4,6 @@ import warnings
 from collections.abc import Callable, Mapping, Sequence
 from dataclasses import dataclass, field
 from pathlib import Path
-from typing import Literal
 
 import tensorrt as trt
 
@@ -77,9 +76,8 @@ class BuilderSettings:
 
 @dataclass(frozen=True)
 class EngineBuild:
-    """One engine's precision, named input shapes, and builder settings."""
+    """One engine's named input shapes and builder settings."""
 
-    precision: Literal["fp32", "fp16"]
     inputs: Mapping[str, InputShape]
     settings: BuilderSettings = field(default_factory=BuilderSettings)
 
@@ -103,8 +101,6 @@ def _build_serialized_engine(
     plugins: Sequence[Path] = (),
     logger: trt.ILogger | None = None,
 ) -> tuple[bytes, tuple[str, ...]]:
-    if declaration.precision not in ("fp32", "fp16"):
-        raise ValueError(f"unsupported precision: {declaration.precision}")
     if declaration.settings.tf32:
         raise ValueError("TF32 cannot be enabled")
     if declaration.settings.workspace_limit <= 0:

{tensoracle-0.4.0 → tensoracle-0.5.0}/src/tensoracle/_cache.py

@@ -277,7 +277,6 @@ def _conditions(
         "graph_version": graph_version,
         "weights": [_weight_sha256(Path(path), force_rebuild=force_rebuild) for path in weights],
         "plugins": [_file_sha256(Path(path)) for path in plugins],
-        "precision": declaration.precision,
         "inputs": {
             name: {
                 "minimum": list(shape.minimum),

{tensoracle-0.4.0 → tensoracle-0.5.0}/src/tensoracle/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '0.4.0'
-__version_tuple__ = version_tuple = (0, 4, 0)
+__version__ = version = '0.5.0'
+__version_tuple__ = version_tuple = (0, 5, 0)
 
 __commit_id__ = commit_id = None

{tensoracle-0.4.0 → tensoracle-0.5.0}/tests/build_memory_runtime.py

@@ -47,7 +47,6 @@ def measure(entry: str, fail: bool, directory: Path) -> dict[str, int]:
             network.mark_output(value)
 
     declaration = EngineBuild(
-        "fp32",
         {"features": InputShape.fixed((1, 512))},
         BuilderSettings(optimization_level=0),
     )

{tensoracle-0.4.0 → tensoracle-0.5.0}/tests/test_build.py

@@ -13,7 +13,14 @@ from safetensors.numpy import load_file, save_file
 from tensoracle import BuilderSettings, EngineBuild, InputShape, build_serialized_engine
 
 
-@pytest.mark.req("REQ-TENSORACLE-004")
+@pytest.mark.req("SP-TENSORACLE-020")
+def test_build_declaration_rejects_precision_keyword():
+    """The build declaration rejects precision because the graph owns tensor types."""
+    with pytest.raises(TypeError, match="precision"):
+        EngineBuild(precision="fp32", inputs={"features": InputShape.fixed((1, 3, 2))})
+
+
+@pytest.mark.req("SP-TENSORACLE-005")
 def test_package_requires_torch_only_for_checkpoint_conversion():
     """Only the conversion extra declares torch, leaving inference dependencies unchanged."""
     dependencies = requires("tensoracle") or []
@@ -114,13 +121,12 @@ def _run(plan: bytes, value: np.ndarray) -> np.ndarray:
 
 
 @pytest.mark.gpu
-@pytest.mark.req("REQ-TENSORACLE-005")
+@pytest.mark.req("SP-TENSORACLE-007")
 def test_fixed_shape_build_runs_at_its_declared_size(tmp_path):
     """A fixed input shape produces a plan that computes the declared graph."""
     path = tmp_path / "weights.safetensors"
     _weights(path)
     declaration = EngineBuild(
-        precision="fp32",
         inputs={"features": InputShape.fixed((1, 3, 2))},
         settings=BuilderSettings(optimization_level=0),
     )
@@ -131,13 +137,12 @@ def test_fixed_shape_build_runs_at_its_declared_size(tmp_path):
 
 
 @pytest.mark.gpu
-@pytest.mark.req("REQ-TENSORACLE-005")
+@pytest.mark.req("SP-TENSORACLE-007")
 def test_range_build_runs_at_minimum_optimum_and_maximum_and_rejects_outside(tmp_path):
     """A ranged input accepts its three declared sizes and rejects an outside size."""
     path = tmp_path / "weights.safetensors"
     _weights(path)
     declaration = EngineBuild(
-        precision="fp32",
         inputs={"features": InputShape.ranged((1, 1, 2), (1, 3, 2), (1, 5, 2))},
         settings=BuilderSettings(optimization_level=0),
     )
@@ -154,13 +159,13 @@ def test_range_build_runs_at_minimum_optimum_and_maximum_and_rejects_outside(tmp
 
 
 @pytest.mark.gpu
-@pytest.mark.req("REQ-TENSORACLE-009")
+@pytest.mark.req("SP-TENSORACLE-013")
+@pytest.mark.req("SP-TENSORACLE-020")
 def test_fp16_build_runs_a_graph_with_fp16_tensors(tmp_path):
-    """A declared fp16 plan runs the graph's fp16 tensors."""
+    """An engine built without a precision declaration runs the graph's fp16 tensors."""
     path = tmp_path / "weights.safetensors"
     _weights(path)
     declaration = EngineBuild(
-        precision="fp16",
         inputs={"features": InputShape.ranged((1, 1, 2), (1, 2, 2), (1, 3, 2))},
         settings=BuilderSettings(optimization_level=0),
     )
@@ -170,11 +175,10 @@ def test_fp16_build_runs_a_graph_with_fp16_tensors(tmp_path):
     np.testing.assert_allclose(_run(plan, value), expected, rtol=2e-3, atol=2e-3)
 
 
-@pytest.mark.req("REQ-TENSORACLE-009")
+@pytest.mark.req("SP-TENSORACLE-013")
 def test_tf32_request_is_rejected_before_graph_is_built():
     """An explicit request to enable TF32 fails without invoking the graph."""
     declaration = EngineBuild(
-        precision="fp32",
         inputs={"features": InputShape.fixed((1, 3, 2))},
         settings=BuilderSettings(tf32=True),
     )
@@ -182,7 +186,7 @@ def test_tf32_request_is_rejected_before_graph_is_built():
         build_serialized_engine(lambda network: pytest.fail("graph was built"), declaration)
 
 
-@pytest.mark.req("REQ-TENSORACLE-009")
+@pytest.mark.req("SP-TENSORACLE-013")
 def test_tf32_is_disabled_and_other_builder_settings_are_applied(monkeypatch):
     """The builder clears TF32 while applying the declared workspace and optimization level."""
     import tensoracle._build as module
@@ -224,7 +228,6 @@ def test_tf32_is_disabled_and_other_builder_settings_are_applied(monkeypatch):
     observed = []
     monkeypatch.setattr(module.trt, "Builder", Builder)
     declaration = EngineBuild(
-        precision="fp32",
         inputs={"features": InputShape.fixed((1, 3, 2))},
         settings=BuilderSettings(workspace_limit=4096, optimization_level=2),
     )