tensoracle 0.5.0__tar.gz → 1.0.0__tar.gz

{tensoracle-0.5.0 → tensoracle-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tensoracle
-Version: 0.5.0
+Version: 1.0.0
 Summary: Build, save, reuse, and execute TensorRT engines from safetensors weights
 Project-URL: documentation, https://sync-dev-org.github.io/tensoracle/
 Project-URL: repository, https://github.com/sync-dev-org/tensoracle
@@ -12,6 +12,7 @@ Classifier: Development Status :: 4 - Beta
 Classifier: Environment :: GPU :: NVIDIA CUDA :: 13
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: Microsoft :: Windows
 Classifier: Operating System :: POSIX :: Linux
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
@@ -57,9 +58,11 @@ Choose the path that matches your work:
 | TensorRT 11 SDK | Not needed | Not needed | Native plugin build |
 | CMake 3.20 or later and C++17 compiler | Not needed | Not needed | Native plugin build |
 
-The supported targets are Linux x86_64 (including WSL2), TensorRT 11, and GPU generations sm86, sm89, and sm120. The producer and native build tools are separate from the `tensoracle` runtime dependencies. Engine building, engine reuse, output comparison, and cubin production do not require torch. Converting `.pth` checkpoints requires the `convert` extra described below.
+The supported targets are Linux x86_64 (including WSL2) and Windows x86_64, TensorRT 11, and GPU generations sm86, sm89, and sm120. The same `py3-none-any` wheel installs on both operating systems. Cubin production with `tensoracle[producer]` runs on Linux; cubins do not depend on the OS, so a plugin DLL built on Windows can embed cubins produced on Linux. The producer and native build tools are separate from the `tensoracle` runtime dependencies. Engine building, engine reuse, output comparison, and cubin production do not require torch. Converting `.pth` checkpoints requires the `convert` extra described below.
 
-Follow the [four-stage optimization workflow](https://github.com/sync-dev-org/tensoracle/blob/main/docs/optimization.md) to establish output quality, evaluate fp16 and plugins, and measure speed.
+Follow the [four-stage optimization workflow](https://github.com/sync-dev-org/tensoracle/blob/main/docs/optimization.md) to establish output quality, evaluate lower precision and plugins, and measure speed.
+
+Follow [Build a C++ and Python repository](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) for source layout, development environments, bindings, and Linux wheel packaging.
 
 ## Convert a PyTorch checkpoint
 
@@ -114,6 +117,8 @@ Use `InputShape.fixed((1, 64, 2))` for one input size; the graph may declare tha
 
 The engine's precision is determined by the tensor types that the graph function places in the network. For an fp16 graph, use `trt.float16` tensors and `np.float16` weights where needed; the graph can also contain fp32 tensors. `EngineBuild` declares input shapes and builder settings. TF32 is always disabled; `BuilderSettings(tf32=True)` raises `ValueError`. The function returns serialized plan bytes. Invalid declarations raise `ValueError`, and a failed TensorRT build raises `RuntimeError`.
 
+The same API builds bf16, int8, and fp8 graphs. Cast tensors to `trt.bfloat16` for bf16; for int8/fp8, place `add_quantize` / `add_dequantize` layers and load their scales from safetensors. The application determines scales with an external quantization library and chooses output quality limits with `compare_outputs`. fp8 targets sm89 and sm120; bf16 and int8 also target sm86. Follow [Build bf16, int8, and fp8 engines](https://github.com/sync-dev-org/tensoracle/blob/main/docs/precision.md) for a complete example, NumPy dtype constraints, and comparison against an fp32 engine. Use different engine names or graph versions for different precisions.
+
 ## Save and reopen an engine
 
 `open_engine` builds on the first call and reuses the saved engine on later calls, including calls from another process. It invokes `graph` only when building. Declare the expected input and output names, TensorRT types, and build-time shapes:
@@ -137,7 +142,7 @@ handle = open_engine(
 engine = handle.engine  # Keep handle.runtime alive while using the engine.
 ```
 
-Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under `~/.cache` when that variable is unset. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, builder settings, GPU product name, OS, CPU architecture, and TensorRT version. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact.
+Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under the home directory's `.cache` when that variable is unset. The same rule applies on every OS; on Windows the home directory is `%USERPROFILE%`, so the default is `%USERPROFILE%\.cache\tensoracle`. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, builder settings, GPU product name, OS, CPU architecture, and TensorRT version. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact. tensoracle closes a saved engine file after reading it, so another process using an opened engine does not block replacement. On Windows, a process that keeps the saved file itself open prevents replacement or removal with `PermissionError`; after that process closes the file, a file left empty by the failed rebuild is rejected as `CorruptEngineError` and can be rebuilt with `force_rebuild=True`.
 
 `plugins` names native TensorRT plugin libraries. Both `open_engine` and `build_serialized_engine` accept it. Each function checks that every library exports `getCreators`, loads it through TensorRT's registry, and retains it for the process before building the graph or deserializing an engine. Repeating the same library does not register it twice. `open_engine` also hashes the loaded library bytes into the engine identity; a changed library selects a different saved engine in a new process. A missing, unreadable, or incompatible library, or a library changed after loading in the same process, raises `PluginLibraryError` before a cached plan is opened. Keep the library and its cubin bundle together in the caller's wheel.
 
@@ -317,7 +322,17 @@ The example was also built with an older compiler and CUDA toolkit:
 | --- | --- | --- | --- |
 | gcc 13.3.0 / CMake 4.2.3 / CUDA toolkit 12.9.86 / TensorRT SDK 11.2.1.2 | `build` and a separate-process `reuse` with TensorRT runtime 11.2.1.2 on sm86 succeeded with zero outputs outside tolerance and both variants launched | 3.13.7 | WSL2 / glibc 2.39 |
 
-Each GPU subprocess had a 50-second limit. Speed was not measured. Native Linux, other kernels, and TensorRT 11 versions not listed here have not been tested.
+Each GPU subprocess had a 50-second limit. Speed was not measured.
+
+On 2026-10-18, the test suite and a native plugin build were run on Windows:
+
+| GPU | Result | Python | OS / NVIDIA driver / build tools / TensorRT |
+| --- | --- | --- | --- |
+| NVIDIA RTX 6000 Ada Generation (sm89) | `pytest`: 246 passed, 20 skipped (Linux-only memory checks and optional `convert` and `producer` dependencies) | 3.12.11 | Windows 11 Pro x64 build 26200 / 596.72 / Visual Studio 2022 (MSVC 19.44), CMake 3.31.6, Ninja, CUDA toolkit 12.8.93 / runtime and SDK 11.4.0.106 |
+
+On Windows, the standalone `cpp/` build and CTest passed, the CMake helper built the example plugin DLL against `nvinfer_11.lib`, and the plugin engine was built and reused in a separate process. The `py3-none-any` wheel built on Linux was installed unchanged on Windows. With `PATH` reduced to the Windows system directories and no CUDA or TensorRT SDK paths, the driver's `nvcuda.dll` and the pip-installed TensorRT runtime were sufficient to build and reuse engines.
+
+Native Linux, other kernels, Windows with sm86 or sm120, and TensorRT 11 versions not listed here have not been tested.
 
 ## Compare output arrays
 

{tensoracle-0.5.0 → tensoracle-1.0.0}/README.md

@@ -27,9 +27,11 @@ Choose the path that matches your work:
 | TensorRT 11 SDK | Not needed | Not needed | Native plugin build |
 | CMake 3.20 or later and C++17 compiler | Not needed | Not needed | Native plugin build |
 
-The supported targets are Linux x86_64 (including WSL2), TensorRT 11, and GPU generations sm86, sm89, and sm120. The producer and native build tools are separate from the `tensoracle` runtime dependencies. Engine building, engine reuse, output comparison, and cubin production do not require torch. Converting `.pth` checkpoints requires the `convert` extra described below.
+The supported targets are Linux x86_64 (including WSL2) and Windows x86_64, TensorRT 11, and GPU generations sm86, sm89, and sm120. The same `py3-none-any` wheel installs on both operating systems. Cubin production with `tensoracle[producer]` runs on Linux; cubins do not depend on the OS, so a plugin DLL built on Windows can embed cubins produced on Linux. The producer and native build tools are separate from the `tensoracle` runtime dependencies. Engine building, engine reuse, output comparison, and cubin production do not require torch. Converting `.pth` checkpoints requires the `convert` extra described below.
 
-Follow the [four-stage optimization workflow](https://github.com/sync-dev-org/tensoracle/blob/main/docs/optimization.md) to establish output quality, evaluate fp16 and plugins, and measure speed.
+Follow the [four-stage optimization workflow](https://github.com/sync-dev-org/tensoracle/blob/main/docs/optimization.md) to establish output quality, evaluate lower precision and plugins, and measure speed.
+
+Follow [Build a C++ and Python repository](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) for source layout, development environments, bindings, and Linux wheel packaging.
 
 ## Convert a PyTorch checkpoint
 
@@ -84,6 +86,8 @@ Use `InputShape.fixed((1, 64, 2))` for one input size; the graph may declare tha
 
 The engine's precision is determined by the tensor types that the graph function places in the network. For an fp16 graph, use `trt.float16` tensors and `np.float16` weights where needed; the graph can also contain fp32 tensors. `EngineBuild` declares input shapes and builder settings. TF32 is always disabled; `BuilderSettings(tf32=True)` raises `ValueError`. The function returns serialized plan bytes. Invalid declarations raise `ValueError`, and a failed TensorRT build raises `RuntimeError`.
 
+The same API builds bf16, int8, and fp8 graphs. Cast tensors to `trt.bfloat16` for bf16; for int8/fp8, place `add_quantize` / `add_dequantize` layers and load their scales from safetensors. The application determines scales with an external quantization library and chooses output quality limits with `compare_outputs`. fp8 targets sm89 and sm120; bf16 and int8 also target sm86. Follow [Build bf16, int8, and fp8 engines](https://github.com/sync-dev-org/tensoracle/blob/main/docs/precision.md) for a complete example, NumPy dtype constraints, and comparison against an fp32 engine. Use different engine names or graph versions for different precisions.
+
 ## Save and reopen an engine
 
 `open_engine` builds on the first call and reuses the saved engine on later calls, including calls from another process. It invokes `graph` only when building. Declare the expected input and output names, TensorRT types, and build-time shapes:
@@ -107,7 +111,7 @@ handle = open_engine(
 engine = handle.engine  # Keep handle.runtime alive while using the engine.
 ```
 
-Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under `~/.cache` when that variable is unset. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, builder settings, GPU product name, OS, CPU architecture, and TensorRT version. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact.
+Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under the home directory's `.cache` when that variable is unset. The same rule applies on every OS; on Windows the home directory is `%USERPROFILE%`, so the default is `%USERPROFILE%\.cache\tensoracle`. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, builder settings, GPU product name, OS, CPU architecture, and TensorRT version. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact. tensoracle closes a saved engine file after reading it, so another process using an opened engine does not block replacement. On Windows, a process that keeps the saved file itself open prevents replacement or removal with `PermissionError`; after that process closes the file, a file left empty by the failed rebuild is rejected as `CorruptEngineError` and can be rebuilt with `force_rebuild=True`.
 
 `plugins` names native TensorRT plugin libraries. Both `open_engine` and `build_serialized_engine` accept it. Each function checks that every library exports `getCreators`, loads it through TensorRT's registry, and retains it for the process before building the graph or deserializing an engine. Repeating the same library does not register it twice. `open_engine` also hashes the loaded library bytes into the engine identity; a changed library selects a different saved engine in a new process. A missing, unreadable, or incompatible library, or a library changed after loading in the same process, raises `PluginLibraryError` before a cached plan is opened. Keep the library and its cubin bundle together in the caller's wheel.
 
@@ -287,7 +291,17 @@ The example was also built with an older compiler and CUDA toolkit:
 | --- | --- | --- | --- |
 | gcc 13.3.0 / CMake 4.2.3 / CUDA toolkit 12.9.86 / TensorRT SDK 11.2.1.2 | `build` and a separate-process `reuse` with TensorRT runtime 11.2.1.2 on sm86 succeeded with zero outputs outside tolerance and both variants launched | 3.13.7 | WSL2 / glibc 2.39 |
 
-Each GPU subprocess had a 50-second limit. Speed was not measured. Native Linux, other kernels, and TensorRT 11 versions not listed here have not been tested.
+Each GPU subprocess had a 50-second limit. Speed was not measured.
+
+On 2026-10-18, the test suite and a native plugin build were run on Windows:
+
+| GPU | Result | Python | OS / NVIDIA driver / build tools / TensorRT |
+| --- | --- | --- | --- |
+| NVIDIA RTX 6000 Ada Generation (sm89) | `pytest`: 246 passed, 20 skipped (Linux-only memory checks and optional `convert` and `producer` dependencies) | 3.12.11 | Windows 11 Pro x64 build 26200 / 596.72 / Visual Studio 2022 (MSVC 19.44), CMake 3.31.6, Ninja, CUDA toolkit 12.8.93 / runtime and SDK 11.4.0.106 |
+
+On Windows, the standalone `cpp/` build and CTest passed, the CMake helper built the example plugin DLL against `nvinfer_11.lib`, and the plugin engine was built and reused in a separate process. The `py3-none-any` wheel built on Linux was installed unchanged on Windows. With `PATH` reduced to the Windows system directories and no CUDA or TensorRT SDK paths, the driver's `nvcuda.dll` and the pip-installed TensorRT runtime were sufficient to build and reuse engines.
+
+Native Linux, other kernels, Windows with sm86 or sm120, and TensorRT 11 versions not listed here have not been tested.
 
 ## Compare output arrays
 

{tensoracle-0.5.0 → tensoracle-1.0.0}/cpp/cmake/TensoracleCubinPlugin.cmake

@@ -10,7 +10,11 @@ function(tensoracle_add_cubin_plugin)
     find_package(CUDAToolkit REQUIRED)
     find_path(TENSORACLE_NVINFER_INCLUDE NvInfer.h
         PATHS "${ARG_TENSORRT_ROOT}/include" NO_DEFAULT_PATH REQUIRED)
-    find_library(TENSORACLE_NVINFER_LIBRARY NAMES nvinfer
+    set(_tensoracle_nvinfer_names nvinfer)
+    if(WIN32)
+        list(PREPEND _tensoracle_nvinfer_names nvinfer_11)
+    endif()
+    find_library(TENSORACLE_NVINFER_LIBRARY NAMES ${_tensoracle_nvinfer_names}
         PATHS "${ARG_TENSORRT_ROOT}/lib" "${ARG_TENSORRT_ROOT}/lib64"
         NO_DEFAULT_PATH REQUIRED)
 

tensoracle-1.0.0/docs/cpp-python-repositories.md

@@ -0,0 +1,199 @@
+# Build a C++ and Python repository
+
+Use this guide to organize a repository with a standalone C++ library and a Python package, then build and inspect its Linux wheels. Keep the C++ build independent of Python, put the Python project in its own directory, and make the package build collect the files it needs from those sources. The binding and wheel examples below illustrate a package named `example`; adapt the target names and dependencies to your project.
+
+## 1. Separate the language projects
+
+Keep `cpp/` and `python/` at the repository root:
+
+```text
+README.md
+LICENSE
+docs/
+cpp/
+  CMakeLists.txt
+  include/
+  src/
+  tests/
+  bindings.cpp          # Optional Python binding source
+python/
+  pyproject.toml
+  uv.lock
+  CMakeLists.txt        # Only when the Python build compiles an extension
+  src/
+    example/
+      __init__.py
+  tests/
+```
+
+`cpp/` owns the C++ sources, headers, CMake targets, and CTest tests. Its standalone configuration builds and tests the C++ library without discovering Python or nanobind. The Python build can compile `bindings.cpp` separately and link it to that library.
+
+`python/` is the Python project root. It owns package metadata, runtime dependencies, development dependencies, the lockfile, and the src-layout package. Run the project's uv commands there. Keep repository-wide documentation, the license, and examples at the repository root; keep `pyproject.toml`, Python tests, and the Python environment under `python/`.
+
+Keep one editable copy of each source file. Configure the package build to collect C++ sources or compiled libraries into the package as appropriate. A build that uses sibling `cpp/` files must also make those files available when building from a source distribution: stage a complete source tree or arrange an equivalent archive layout, then verify a build from the extracted archive.
+
+Tensoracle is an example of source packaging: its wheel includes the shared C++ headers, sources, and CMake helpers under `tensoracle/cpp/`, and the caller compiles its own plugin. Tensoracle's wheel contains no native binaries and uses `py3-none-any`. A repository that builds a Python extension needs the additional steps in sections 4–6.
+
+## 2. Keep development tools in the appropriate environment
+
+Create the Python environment at `python/.venv`, alongside `python/pyproject.toml` and `python/uv.lock`; keep the repository root free of a second `.venv`. uv uses a project-local [virtual environment](https://docs.astral.sh/uv/concepts/projects/layout/#the-project-environment). For this layout, start from the repository root and run:
+
+```bash
+cd python
+uv sync
+```
+
+Declare Python development tools such as pytest, Ruff, and a type checker in the Python project's development dependency group. Keep runtime dependencies in the `[project]` table's `dependencies` field, optional runtime features in extras, and isolated build requirements in the `[build-system]` table's `requires` field.
+
+Keep one CMake and one Ninja installation on the host's `PATH`, shared by standalone C++ builds and tests that invoke CMake. Do not add `cmake` or `ninja` to each project's Python development dependency group. Their installation method can follow the developer's platform and package manager.
+
+Each CMake entry point declares its minimum supported version with [`cmake_minimum_required`](https://cmake.org/cmake/help/latest/command/cmake_minimum_required.html). The host installation must meet all projects' minimums. Tensoracle's standalone C++ project requires CMake 3.20; the binding example below uses CMake 3.26 for `Development.SABIModule`. Verify affected C++ builds before upgrading a shared host installation.
+
+An isolated wheel build has separate requirements. Keep any CMake requirement it needs in `[build-system] requires`; that environment does not replace the host tools. [scikit-build-core](https://scikit-build-core.readthedocs.io/en/stable/configuration/index.html#cmake-and-ninja-minimum-versions) can obtain CMake or Ninja when the available versions do not satisfy its configuration.
+
+## 3. Test each language and identify GPU tests
+
+Run pytest from `python/` so that the Python project's configuration and environment apply:
+
+```bash
+uv run pytest
+uv run pytest -m "not gpu"
+```
+
+Register a `gpu` marker in the pytest configuration and apply `@pytest.mark.gpu` to tests that need a GPU. Use strict marker checking to catch misspelled markers. The second command selects tests that can run without a GPU; those tests must also avoid GPU initialization during collection. See [pytest's marker documentation](https://docs.pytest.org/en/stable/how-to/mark.html).
+
+Register the standalone C++ tests with CTest, using `include(CTest)` and `add_test`. From the repository root, configure and run them without activating the Python environment:
+
+```bash
+cmake -S cpp -B build/cpp -G Ninja -DBUILD_TESTING=ON
+cmake --build build/cpp
+ctest --test-dir build/cpp --output-on-failure
+ctest --test-dir build/cpp --output-on-failure -LE gpu
+```
+
+Supply any required SDK or toolchain paths at configuration time. Python independence does not remove C++ library, header, or CUDA toolkit prerequisites. For TensorRT plugins, follow [Prepare a TensorRT SDK](tensorrt-sdk.md).
+
+Give GPU-dependent CTest tests a [`gpu` label](https://cmake.org/cmake/help/latest/prop_test/LABELS.html); the final command excludes that label. A machine without a CUDA toolkit may also need a configuration that omits GPU targets. Test C++ library behavior through CTest and Python imports, bindings, and integration through pytest. Before distributing a wheel, install it into a clean environment and test the installed package outside the source tree.
+
+## 4. Install nanobind extensions inside the package
+
+Keep the standalone library's CMake configuration independent of the Python bindings. The Python project's CMake configuration discovers Python and nanobind, adds the C++ library, and installs the extension into the Python package. For a C++ target named `example_core`, these fragments illustrate that arrangement.
+
+In `python/pyproject.toml`, alongside the project's metadata:
+
+```toml
+[build-system]
+requires = ["scikit-build-core", "nanobind"]
+build-backend = "scikit_build_core.build"
+
+[tool.scikit-build]
+wheel.packages = ["src/example"]
+wheel.py-api = "cp312"
+```
+
+In `python/CMakeLists.txt`:
+
+```cmake
+cmake_minimum_required(VERSION 3.26)
+project(example_python LANGUAGES CXX)
+
+find_package(Python 3.12 REQUIRED COMPONENTS
+    Interpreter Development.Module Development.SABIModule)
+find_package(nanobind CONFIG REQUIRED)
+
+add_subdirectory(../cpp cpp)
+set_target_properties(example_core PROPERTIES POSITION_INDEPENDENT_CODE ON)
+nanobind_add_module(_native STABLE_ABI ../cpp/bindings.cpp)
+target_link_libraries(_native PRIVATE example_core)
+install(TARGETS _native LIBRARY DESTINATION example)
+```
+
+The binding source declares `NB_MODULE(_native, m)`, and `python/src/example/__init__.py` imports from `._native`. The installed wheel contains `example/__init__.py` and the extension, such as `example/_native.abi3.so`. The snippets assume that `cpp/CMakeLists.txt` defines `example_core` and that the package build supplies the sibling C++ sources as described in section 1. See [nanobind packaging](https://nanobind.readthedocs.io/en/latest/packaging.html) for the binding build and installation mechanism.
+
+For nanobind's linked build mode, [`STABLE_ABI`](https://nanobind.readthedocs.io/en/latest/api_cmake.html#command:nanobind_add_module) requires Python 3.12 or later. Pair it with `wheel.py-api = "cp312"` so that the binary and wheel metadata agree. This produces a CPython stable-ABI extension for compatible GIL-enabled Python versions from that floor, subject to the package's `requires-python`. It does not remove platform, C++ ABI, or runtime library constraints. Free-threaded Python needs its own build and compatibility decisions.
+
+Read the three parts of the [wheel compatibility tag](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/) separately:
+
+| Wheel contents | Example Python–ABI–platform tag |
+| --- | --- |
+| Python and source files, without native binaries | `py3-none-any` |
+| Native libraries that do not use the Python C API | `py3-none-manylinux_2_28_x86_64` |
+| A CPython stable-ABI extension with a Python 3.12 floor | `cp312-abi3-manylinux_2_28_x86_64` |
+| A CPython 3.12 extension using its version-specific ABI | `cp312-cp312-manylinux_2_28_x86_64` |
+
+These are illustrative targets, not tags to assign before inspecting the binaries. `none` describes the Python ABI; a wheel carrying a shared library still needs a platform tag. `abi3` retains a minimum Python version and does not mean every Python implementation or build is supported.
+
+## 5. Check manylinux compatibility before uploading to PyPI
+
+For glibc-based Linux x86_64 wheels, use a supported manylinux platform tag. PyPI's [platform validation](https://github.com/pypi/warehouse/blob/main/warehouse/utils/wheel.py) rejects the generic `linux_x86_64` tag. Under [PEP 600](https://peps.python.org/pep-0600/), `manylinux_2_28_x86_64` promises compatibility with mainstream x86_64 Linux systems using glibc 2.28 or later. It does not describe CUDA or GPU driver requirements; document those separately.
+
+The actual minimum follows the symbols and libraries used by the built artifacts. Linking on a newer system can introduce newer glibc requirements even when the source code has not changed. The host's glibc version alone is not a measurement of the wheel's minimum, and changing the filename cannot lower that minimum. Inspect required `GLIBC_*` versions, along with `GLIBCXX_*` and `CXXABI_*` requirements for the C++ runtime.
+
+Run [`auditwheel show`](https://github.com/pypa/auditwheel) in the build container. Extract the wheel and inspect its ELF files with [`readelf`](https://sourceware.org/binutils/docs/binutils/readelf.html). For example, after setting `WHEEL` to the exact wheel path and choosing a new extraction directory:
+
+```bash
+auditwheel show "$WHEEL"
+python -m zipfile -e "$WHEEL" wheel-inspect
+LC_ALL=C readelf --version-info --wide wheel-inspect/example/_native.abi3.so
+LC_ALL=C readelf --dynamic --wide wheel-inspect/example/_native.abi3.so
+```
+
+Repeat the ELF checks for every shared library and executable in the wheel, including nested dependencies. Inspect the version requirements and `NEEDED`, `RPATH`, and `RUNPATH` entries. Use `LC_ALL=C` when tools or scripts rely on English output headings, and compare numeric version components rather than sorting version strings lexically. Rebuild in an appropriate environment if a required symbol exceeds the intended baseline; auditwheel cannot repair an overly new glibc or libstdc++ dependency.
+
+### Shared libraries and search paths
+
+Assign each shipped shared library to one package. If another wheel supplies a library, declare that package as a runtime dependency and arrange a documented runtime lookup path. Do not copy its library into the consuming wheel. RAPIDS publishes separate [Python binding](https://pypi.org/project/pylibcudf-cu12/) and [C++ library](https://pypi.org/project/libcudf-cu12/) distributions as `pylibcudf-cu12` and `libcudf-cu12`; their release files illustrate the distinction between Python ABI tags and platform tags.
+
+`auditwheel repair` normally copies non-policy shared libraries into the wheel and changes their search paths. Review that list before repair, and exclude libraries owned by dependency wheels and libraries provided by the GPU driver. An exclusion does not establish manylinux compliance. Resolve and test those dependencies separately, and inspect the final wheel again. Also test libraries loaded through `dlopen` or `ctypes`, which auditwheel cannot discover from `NEEDED` entries alone.
+
+Use `$ORIGIN`-relative ELF search paths, where `$ORIGIN` is the directory containing the referencing binary. For an extension whose own package contains a `lib/` subdirectory, a CMake target can use:
+
+```cmake
+set_target_properties(_native PROPERTIES
+    BUILD_RPATH_USE_ORIGIN ON
+    INSTALL_RPATH "$ORIGIN/lib")
+```
+
+[`INSTALL_RPATH`](https://cmake.org/cmake/help/latest/prop_tgt/INSTALL_RPATH.html) controls the installed artifact; `BUILD_RPATH_USE_ORIGIN` applies to build-tree paths. Select the relative path for the actual installed layout, including dependency packages where needed, and check every library's own dependencies. Remove absolute build-machine paths, including paths introduced by imported CMake targets. Keep `$ORIGIN` literal when passing a value through a shell.
+
+Keep NVIDIA driver libraries such as `libcuda.so.1`, `libnvidia-encode.so.1`, and `libnvcuvid.so.1` outside the wheel. The target system supplies the compatible driver. Link-time SDK stubs must also stay out of the runtime package and its search paths; NVIDIA's [Video Codec SDK instructions](https://docs.nvidia.com/video-technologies/video-codec-sdk/13.0/read-me/index.html) distinguish these stubs from the installed driver. Treat CUDA runtime libraries separately, using an appropriate static runtime or declared runtime package dependencies for the chosen toolkit.
+
+Check artifact size before upload. PyPI's [default individual file limit](https://docs.pypi.org/project-management/storage-limits/) is 100 MB; projects can have an approved increase. Installing and testing the final wheel with only its declared dependencies is still necessary after tag and ELF inspection.
+
+## 6. Record the wheel build environment
+
+Build release wheels in a [manylinux container](https://github.com/pypa/manylinux) for the intended architecture and compatibility floor. For example, `quay.io/pypa/manylinux_2_28_x86_64` provides a glibc 2.28 baseline. Select compiler, SDK, CUDA toolkit, and dependency versions that can build and run against that baseline. A recent compiler does not require building against a recent host glibc.
+
+Pin the image by an immutable [image digest](https://docs.docker.com/reference/cli/docker/image/pull/#pull-an-image-by-digest-immutable-identifier), using `image@sha256:<digest>` with the actual digest of the selected image. A moving tag is useful for discovering updates but does not identify a repeatable build input. Pin any derived image as well, and record additional package installation inputs.
+
+Keep a build record with:
+
+- The source revision, patches, and source manifest.
+- The image reference, manifest digest, image configuration digest, and architecture.
+- The compiler, linker, CMake, Ninja, Python interpreter, and build backend versions.
+- The CUDA toolkit and SDK versions, when used, and relevant compiler or CMake options.
+- The resolved build and runtime dependencies, resulting wheel hashes, ELF inspection results, and test results.
+
+Build from clean inputs inside that container, inspect the final artifacts there, and test their installation in a separate runtime environment. When native C++ objects cross extension boundaries, align the C++ library ABI, compiler compatibility, debug mode, symbol visibility, and nanobind ABI. Python's stable ABI does not make incompatible C++ objects interchangeable. [nanobind's compatibility requirements](https://nanobind.readthedocs.io/en/latest/faq.html#how-can-i-avoid-conflicts-with-other-projects-using-nanobind) describe those constraints.
+
+## 7. Record and reconstruct upstream C++ sources
+
+When development needs a local copy of an upstream C++ project, copy the selected files into a dedicated directory and keep the source inputs explicit. Record the upstream URL and immutable revision, the selected paths, the ordered local patches, and the license information. Keep a SHA-256 manifest covering every file in the resulting source tree, including files added by patches.
+
+For example, a provenance directory can contain:
+
+```text
+upstream/library/
+  PROVENANCE.md
+  patches/
+  MANIFEST.sha256
+  fetch-and-verify.py
+```
+
+`PROVENANCE.md` identifies the upstream revision and explains the selection and patch order. `MANIFEST.sha256` records the expected relative paths and hashes after patching. The acquisition script reads these recorded inputs, retrieves the exact revision, selects the required files, applies the patches, and verifies the resulting tree. Keep the revision in one authoritative location that the script reads.
+
+Use ordinary copied files for the development tree, rather than a Git submodule. A submodule records a commit reference instead of ordinary source files, so a publication process that copies tracked files into a snapshot does not materialize that source tree. An explicit acquisition script also makes the build independent of recursive Git checkout behavior.
+
+Keep the copied upstream source tree out of the public repository. Publish the revision record, patches, complete file manifest, acquisition and verification script, and instructions for building and testing from the reconstructed sources. Preserve required license notices in the appropriate distributed artifacts. The public checkout must let another developer reconstruct the inputs and build the native components without access to the development copy.
+
+Verify both file contents and the complete set of paths: missing, changed, and unexpected files must fail verification. A checksum command that only checks listed files will not detect extra files by itself. Verify before configuring or compiling, and stop on a download, patch, or manifest failure. When updating upstream, regenerate the manifest from the reviewed source selection and rerun the native build and tests. If publishing a source distribution, ensure its documented acquisition process makes the same verified inputs available for its build.

{tensoracle-0.5.0 → tensoracle-1.0.0}/docs/index.md

@@ -2,7 +2,9 @@
 
 `tensoracle` builds TensorRT engines from model graphs supplied through TensorRT's Python network API. A model package owns its graph, safetensors weights, and output quality limits. `tensoracle` supplies engine building, saved engine reuse, execution, output comparison, and tools for cubin plugins. These workflows do not require torch. Only the optional `.pth` checkpoint converter requires torch, installed with `tensoracle[convert]`.
 
-Follow the [four-stage optimization workflow](optimization.md) to establish output quality, evaluate fp16 and plugins, and measure speed.
+Follow the [four-stage optimization workflow](optimization.md) to establish output quality, evaluate lower precision and plugins, and measure speed.
+
+Follow [Build a C++ and Python repository](cpp-python-repositories.md) for source layout, development environments, bindings, and Linux wheel packaging.
 
 ## Terms
 
@@ -29,7 +31,7 @@ Follow the [four-stage optimization workflow](optimization.md) to establish outp
 | TensorRT 11 SDK | Not needed | Not needed | Native plugin build |
 | CMake 3.20 or later and C++17 compiler | Not needed | Not needed | Native plugin build |
 
-The supported platform is Linux x86_64 (including WSL2) with sm86, sm89, or sm120 GPUs and TensorRT 11. The producer and native build tools are separate from the `tensoracle` runtime dependencies. Engine building, engine reuse, output comparison, and cubin production do not require torch. Converting `.pth` checkpoints requires the `convert` extra.
+The supported platforms are Linux x86_64 (including WSL2) and Windows x86_64 with sm86, sm89, or sm120 GPUs and TensorRT 11. The same `py3-none-any` wheel installs on both operating systems. Cubin production with `tensoracle[producer]` runs on Linux; a plugin DLL built on Windows can embed cubins produced on Linux. The producer and native build tools are separate from the `tensoracle` runtime dependencies. Engine building, engine reuse, output comparison, and cubin production do not require torch. Converting `.pth` checkpoints requires the `convert` extra.
 
 ### Convert a PyTorch checkpoint
 
@@ -70,7 +72,7 @@ handle = open_engine(
 )
 ```
 
-`my_model` represents the application's model package. On a valid cache hit, `open_engine` does not call `graph`; on a miss, it builds and saves a new engine. Raise `graph_version` when the graph code changes or when any other caller value that changes the engine contents but is not part of the cache identity changes, such as a plugin manifest or a build recipe. `open_engine` itself does not execute the engine or manage GPU buffers. Create an execution unit from its returned handle to enqueue inference. Saved engines live under `XDG_CACHE_HOME/tensoracle`, or `~/.cache/tensoracle` when `XDG_CACHE_HOME` is unset. See the [complete cache example](https://github.com/sync-dev-org/tensoracle#save-and-reopen-an-engine) for the declaration and expected input and output types.
+`my_model` represents the application's model package. On a valid cache hit, `open_engine` does not call `graph`; on a miss, it builds and saves a new engine. Raise `graph_version` when the graph code changes or when any other caller value that changes the engine contents but is not part of the cache identity changes, such as a plugin manifest or a build recipe. `open_engine` itself does not execute the engine or manage GPU buffers. Create an execution unit from its returned handle to enqueue inference. Saved engines live under `XDG_CACHE_HOME/tensoracle`, or `~/.cache/tensoracle` when `XDG_CACHE_HOME` is unset. On Windows, `~` is `%USERPROFILE%`. See the [complete cache example](https://github.com/sync-dev-org/tensoracle#save-and-reopen-an-engine) for the declaration and expected input and output types.
 
 ### Execute an engine
 
@@ -161,7 +163,7 @@ Networks with outputs marked as shape tensors emit `tensoracle.ShapeTensorOutput
 
 ### Build a standard graph
 
-Define `graph(network)` with TensorRT's Python network API and load model weights from safetensors. Describe each graph input with `InputShape.fixed` or `InputShape.ranged`, and pass an `EngineBuild` declaration to `build_serialized_engine`. `EngineBuild` declares input shapes and builder settings; the graph function determines precision through the tensor types it places in the network. The supported precisions are fp32 (with TF32 disabled) and fp16. `open_engine` uses the same declaration plus a name, graph version, weight files, and `EngineIO` to save and reuse a checked engine. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. The [graph example](https://github.com/sync-dev-org/tensoracle#define-and-build-a-standard-graph) shows the exact calls.
+Define `graph(network)` with TensorRT's Python network API and load model weights from safetensors. Describe each graph input with `InputShape.fixed` or `InputShape.ranged`, and pass an `EngineBuild` declaration to `build_serialized_engine`. `EngineBuild` declares input shapes and builder settings; the graph function determines precision through the tensor types it places in the network. The supported precisions are fp32 (with TF32 disabled), fp16, bf16, int8, and fp8. The [bf16, int8, and fp8 guide](precision.md) shows tensor casts, explicit Q/DQ layers, and scales supplied through safetensors; fp8 targets sm89 and sm120. `open_engine` uses the same declaration plus a name, graph version, weight files, and `EngineIO` to save and reuse a checked engine. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. The [graph example](https://github.com/sync-dev-org/tensoracle#define-and-build-a-standard-graph) shows the exact calls.
 
 The application runs the engine and passes a CPU NumPy candidate output and a corresponding reference output to `compare_outputs`. The application chooses the acceptance limits. See [Compare output arrays](https://github.com/sync-dev-org/tensoracle#compare-output-arrays).
 

{tensoracle-0.5.0 → tensoracle-1.0.0}/docs/optimization.md

@@ -10,9 +10,9 @@ Use the same representative inputs, shapes, and corresponding outputs throughout
 
 ## 2. Accept lower precision by quality
 
-- **tensoracle tools:** Build an fp16 TensorRT engine from a graph whose tensor types are fp16, using the same `EngineBuild` declaration for input shapes and builder settings. The graph function determines precision through its tensor types. When saving with `open_engine`, use a different `name` or `graph_version` from the fp32 engine. Use `compare_outputs` with each fp16 output as the candidate and its accepted fp32 TensorRT output as the reference. It reports non-finite counts and, for finite floating point arrays, maximum absolute error, relative L2 error, cosine similarity, and, when both tolerances are supplied, counts and fractions outside tolerance.
+- **tensoracle tools:** Build an fp16, bf16, int8, or fp8 TensorRT engine using the same `EngineBuild` declaration for input shapes and builder settings. The graph function determines precision through its tensor types. For int8 and fp8, the graph places explicit quantize/dequantize layers and reads their scales from safetensors; the application prepares those scales with an external quantization library. Follow [Build bf16, int8, and fp8 engines](precision.md) for a complete graph example and the GPU targets (fp8 requires sm89 or sm120). When saving with `open_engine`, use a different `name` or `graph_version` from the fp32 engine. Use `compare_outputs` with each lower precision output as the candidate and its accepted fp32 TensorRT output as the reference. It reports non-finite counts and, for finite floating point arrays, maximum absolute error, relative L2 error, cosine similarity, and, when both tolerances are supplied, counts and fractions outside tolerance.
 - **You provide:** Choose limits for those model-independent measurements. Compute any model-specific measurements separately, such as PSNR for images or application-specific time-series quality measures, and choose their limits. Run both engines on the same inputs.
-- **Gate:** Accept the fp16 engine only when its outputs meet all applicable limits, including the model-specific ones. Otherwise keep the accepted fp32 engine or revise the candidate and repeat this stage.
+- **Gate:** Accept the lower precision engine only when its outputs meet all applicable limits, including the model-specific ones. Otherwise keep the accepted fp32 engine or revise the candidate and repeat this stage.
 
 ## 3. Replace a remaining bottleneck with a plugin
 

tensoracle-1.0.0/docs/precision.md

@@ -0,0 +1,154 @@
+# Build bf16, int8, and fp8 engines
+
+The graph function selects tensor types. Use the same `EngineBuild`, `open_engine`, and execution API for fp32, fp16, bf16, int8, and fp8. For bf16, cast tensors to `trt.bfloat16`. For int8 and fp8, place explicit quantize/dequantize (Q/DQ) layers and their scales in the graph. TensorRT 11's [`add_quantize` and `add_dequantize` operators](https://docs.nvidia.com/deeplearning/tensorrt/11.4.0/_static/operators/Quantize.html) accept the quantized type and dequantized type explicitly.
+
+| Graph precision | Tensor placement in this example | Supported tensoracle GPU targets |
+| --- | --- | --- |
+| fp32 | fp32 matrix multiplication and bias, with TF32 disabled | sm86, sm89, sm120 |
+| bf16 | Cast features, matrix, and bias to bf16; cast the result to fp32 | sm86, sm89, sm120 |
+| int8 | Q/DQ the features and matrix with `trt.int8` and explicit scales | sm86, sm89, sm120 |
+| fp8 | Q/DQ the features and matrix with `trt.fp8` (E4M3) and explicit scales | sm89, sm120 |
+
+Do not build the fp8 graph on sm86. The graph keeps fp32 input and output buffers for every precision; the internal tensors determine its arithmetic. TensorRT chooses implementations for the typed graph, so an int8 or fp8 graph can retain higher precision operations such as the bias addition. This example does not establish a speed improvement.
+
+## Prepare weights and scales outside tensoracle
+
+For a real model, use an external quantization library, such as [NVIDIA Model Optimizer](https://github.com/NVIDIA/Model-Optimizer), to determine scales from representative data or quantization-aware training. The model package translates those results into its safetensors file and graph. tensoracle does not calculate scales, calibrate a model, or translate a quantizer's export format.
+
+Store the fp32 weights and positive fp32 scales together in safetensors. In this example, `matrix` has shape `(32, 16)`, `bias` has shape `(1, 16)`, and each scale has shape `(1,)`. The keys `int8.input_scale`, `int8.weight_scale`, `fp8.input_scale`, and `fp8.weight_scale` distinguish the two quantization recipes. Both Q and DQ use the same scale for each tensor.
+
+The following small fixture uses maximum absolute values to choose scales. Its inputs and weights match the GPU test; this scale calculation is only a demonstration for that fixture, not a calibration recipe for other models:
+
+```python
+from pathlib import Path
+
+import numpy as np
+from safetensors.numpy import save_file
+
+path = Path("linear.safetensors")
+features = np.random.default_rng(11).uniform(-1, 1, (16, 32)).astype(np.float32)
+rng = np.random.default_rng(29)
+matrix = rng.uniform(-0.25, 0.25, (32, 16)).astype(np.float32)
+bias = rng.uniform(-0.1, 0.1, (1, 16)).astype(np.float32)
+weights = {"matrix": matrix, "bias": bias}
+for precision, maximum in (("int8", 127), ("fp8", 448)):
+    weights[f"{precision}.input_scale"] = np.array(
+        [np.max(np.abs(features)) / maximum], dtype=np.float32
+    )
+    weights[f"{precision}.weight_scale"] = np.array(
+        [np.max(np.abs(matrix)) / maximum], dtype=np.float32
+    )
+save_file(weights, path)
+```
+
+[`safetensors.numpy` 0.8.0](https://github.com/huggingface/safetensors/blob/v0.8.0/bindings/python/py_src/safetensors/numpy.py) with NumPy 2.5 does not directly load bf16 or fp8 tensors into NumPy arrays: bf16 loading raises `TypeError`, and FP8 E4M3 loading raises `AttributeError`. A safetensors file can store those types, but its NumPy reader cannot represent them in this environment. Keep this example's weights and scales in fp32, then cast or quantize in TensorRT. If an existing checkpoint stores bf16 or fp8 weights, convert them with a reader that supports that dtype outside this NumPy graph path. Installing torch is not required for this example.
+
+## Define the typed graph
+
+This graph computes `features @ matrix + bias`. The bf16 branch casts all three operands before arithmetic and returns fp32. The int8/fp8 branch applies per-tensor Q/DQ to the features and matrix; its bias stays fp32. Keeping the stored matrix in fp32 lets TensorRT quantize it with the supplied scale during building.
+
+```python
+import tensorrt as trt
+from safetensors.numpy import load_file
+
+
+def graph(path, precision):
+    def define(network):
+        weights = load_file(path)
+
+        def constant(array):
+            array = np.ascontiguousarray(array, dtype=np.float32)
+            return network.add_constant(array.shape, array).get_output(0)
+
+        features = network.add_input("features", trt.float32, (16, 32))
+        matrix = constant(weights["matrix"])
+        bias = constant(weights["bias"])
+        if precision == "bf16":
+            features = network.add_cast(features, trt.bfloat16).get_output(0)
+            matrix = network.add_cast(matrix, trt.bfloat16).get_output(0)
+            bias = network.add_cast(bias, trt.bfloat16).get_output(0)
+        elif precision in ("int8", "fp8"):
+            quantized_type = trt.int8 if precision == "int8" else trt.fp8
+
+            def qdq(tensor, key):
+                scale = constant(weights[f"{precision}.{key}_scale"])
+                quantized = network.add_quantize(tensor, scale, quantized_type).get_output(0)
+                return network.add_dequantize(quantized, scale, trt.float32).get_output(0)
+
+            features = qdq(features, "input")
+            matrix = qdq(matrix, "weight")
+        value = network.add_matrix_multiply(
+            features, trt.MatrixOperation.NONE, matrix, trt.MatrixOperation.NONE
+        ).get_output(0)
+        value = network.add_elementwise(value, bias, trt.ElementWiseOperation.SUM).get_output(0)
+        if precision == "bf16":
+            value = network.add_cast(value, trt.float32).get_output(0)
+        value.name = "result"
+        network.mark_output(value)
+
+    return define
+```
+
+## Build, save, and reuse each precision
+
+Use a distinct engine name or graph version whenever tensor types or the graph recipe change. The same weight file serves all four graphs here, so its digest cannot distinguish their precision. Scales are part of that file and therefore part of the saved engine identity.
+
+```python
+from tensoracle import (
+    BuilderSettings, EngineBuild, EngineIO, InputShape, TensorSpec, open_engine,
+)
+
+declaration = EngineBuild(
+    {"features": InputShape.fixed((16, 32))},
+    BuilderSettings(optimization_level=0),
+)
+io = EngineIO(
+    {"features": TensorSpec(trt.float32, (16, 32))},
+    {"result": TensorSpec(trt.float32, (16, 16))},
+)
+
+
+def open_linear(precision):
+    return open_engine(
+        graph(path, precision),
+        declaration,
+        name=f"linear-{precision}",
+        graph_version="1",
+        weights=(path,),
+        io=io,
+    )
+
+
+reference_handle = open_linear("fp32")
+candidate_handle = open_linear("bf16")  # Or "int8"; "fp8" on sm89/sm120.
+```
+
+TF32 remains disabled by tensoracle, including for the fp32 reference. Engines are saved under `XDG_CACHE_HOME/tensoracle`, or `~/.cache/tensoracle` (`%USERPROFILE%\.cache\tensoracle` on Windows). Repeat the same call with unchanged build conditions in another process to reuse the plan without invoking the graph function. Keep the weight file and the engine names/versions stable across the restart. Follow [Execute an engine](index.md#execute-an-engine) to allocate fp32 input/output buffers and enqueue each handle with the same inputs.
+
+## Compare outputs and choose limits
+
+First establish the fp32 engine against fixed fp32 outputs from the official model implementation, as described in [stage 1](optimization.md#1-establish-the-fp32-reference). Then run the candidate and accepted fp32 engine with identical inputs. Pass their CPU NumPy outputs to `compare_outputs` and choose quality limits in the application:
+
+```python
+from tensoracle import compare_outputs
+
+# candidate_output and fp32_output are the application's engine outputs.
+comparison = compare_outputs(candidate_output, fp32_output, atol=0.01, rtol=0)
+accepted = (
+    comparison.candidate_nonfinite_count == 0
+    and comparison.reference_nonfinite_count == 0
+    and comparison.outside_tolerance_count == 0
+    and comparison.relative_l2_error is not None
+    and comparison.relative_l2_error < 0.01
+)
+```
+
+These example limits belong to the small bf16 fixture. For the fixture above, a TensorRT 11.4 run on sm120 measured:
+
+| Candidate | Maximum absolute error | Relative L2 error | Test `atol` (`rtol=0`) | Test relative L2 limit |
+| --- | --- | --- | --- | --- |
+| bf16 | 0.00357 | 0.00294 | 0.01 | 0.01 |
+| int8 | 0.00710 | 0.00572 | 0.02 | 0.02 |
+| fp8 | 0.0501 | 0.0382 | 0.12 | 0.08 |
+
+The test allows at least twice the observed errors for differences in GPU tactics. These are not tensoracle defaults or quality guarantees for other models. Choose representative data, per-output limits, and model-specific quality measurements for the real application. The GPU test also checks that the two engines occupy separate cache entries and reproduce their outputs in another process without calling either graph.