tensoracle 1.4.0__tar.gz → 1.6.0__tar.gz

{tensoracle-1.4.0 → tensoracle-1.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tensoracle
-Version: 1.4.0
+Version: 1.6.0
 Summary: Build, save, reuse, and execute TensorRT engines from safetensors weights
 Project-URL: documentation, https://sync-dev-org.github.io/tensoracle/
 Project-URL: repository, https://github.com/sync-dev-org/tensoracle
@@ -64,7 +64,7 @@ The supported targets are Linux x86_64 (including WSL2) and Windows x86_64, Tens
 
 Follow the [four-stage optimization workflow](https://github.com/sync-dev-org/tensoracle/blob/main/docs/optimization.md) to establish output quality, evaluate lower precision and plugins, and measure speed.
 
-Follow [Build a C++ and Python repository](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) for source layout, development environments, bindings, and Linux wheel packaging.
+Follow [Build plugins with tensoracle's C++ sources](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) to locate the sources in the wheel and use the CMake helper in your plugin build.
 
 ## Convert a PyTorch checkpoint
 
@@ -110,7 +110,7 @@ def graph(network: trt.INetworkDefinition) -> None:
 
 declaration = EngineBuild(
     inputs={"features": InputShape.ranged((1, 1, 2), (1, 64, 2), (1, 448, 2))},
-    settings=BuilderSettings(workspace_limit=1 << 30, optimization_level=3),
+    settings=BuilderSettings(workspace_limit=None, optimization_level=3),
 )
 plan: bytes = build_serialized_engine(graph, declaration)
 ```
@@ -119,6 +119,8 @@ Use `InputShape.fixed((1, 64, 2))` for one input size; the graph may declare tha
 
 The engine's precision is determined by the tensor types that the graph function places in the network. For an fp16 graph, use `trt.float16` tensors and `np.float16` weights where needed; the graph can also contain fp32 tensors. `EngineBuild` declares input shapes and builder settings. TF32 is always disabled; `BuilderSettings(tf32=True)` raises `ValueError`. The function returns serialized plan bytes. Invalid declarations raise `ValueError`, and a failed TensorRT build raises `RuntimeError`.
 
+`BuilderSettings.workspace_limit` defaults to `None`, which leaves the workspace limit at TensorRT's default without calling `set_memory_pool_limit`. To impose a limit, pass a positive integer in bytes, such as `workspace_limit=4 << 30` for 4 GiB. Zero, negative values, booleans, and noninteger values raise `ValueError`.
+
 The same API builds bf16, int8, and fp8 graphs. Cast tensors to `trt.bfloat16` for bf16; for int8/fp8, place `add_quantize` / `add_dequantize` layers and load their scales from safetensors. The application determines scales with an external quantization library and chooses output quality limits with `compare_outputs`. fp8 targets sm89 and sm120; bf16 and int8 also target sm86. Follow [Build bf16, int8, and fp8 engines](https://github.com/sync-dev-org/tensoracle/blob/main/docs/precision.md) for a complete example, NumPy dtype constraints, and comparison against an fp32 engine. Use different engine names or graph versions for different precisions.
 
 ## Save and reopen an engine
@@ -144,7 +146,9 @@ handle = open_engine(
 engine = handle.engine  # Keep handle.runtime alive while using the engine.
 ```
 
-Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under the home directory's `.cache` when that variable is unset. The same rule applies on every OS; on Windows the home directory is `%USERPROFILE%`, so the default is `%USERPROFILE%\.cache\tensoracle`. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, builder settings, the GPU's compatibility properties, OS, CPU architecture, and TensorRT version. The GPU is the device of the current CUDA context, or device 0 when no context is current. Its properties are the product name, compute capability, memory bus width, L2 cache size, shared memory limits, texture alignment, multiprocessor count, whether the GPU is integrated, ECC mode, and total memory; another device with the same properties reuses the saved engine. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact. tensoracle closes a saved engine file after reading it, so another process using an opened engine does not block replacement. On Windows, a process that keeps the saved file itself open prevents replacement or removal with `PermissionError`; after that process closes the file, a file left empty by the failed rebuild is rejected as `CorruptEngineError` and can be rebuilt with `force_rebuild=True`.
+Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under the home directory's `.cache` when that variable is unset. The same rule applies on every OS; on Windows the home directory is `%USERPROFILE%`, so the default is `%USERPROFILE%\.cache\tensoracle`. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, the builder's `optimization_level` and `tf32` settings, the GPU's compatibility properties, OS, CPU architecture, and TensorRT version. The GPU is the device of the current CUDA context, or device 0 when no context is current. Its properties are the product name, compute capability, memory bus width, L2 cache size, shared memory limits, texture alignment, multiprocessor count, whether the GPU is integrated, ECC mode, and total memory; another device with the same properties reuses the saved engine. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact. tensoracle closes a saved engine file after reading it, so another process using an opened engine does not block replacement. On Windows, a process that keeps the saved file itself open prevents replacement or removal with `PermissionError`; after that process closes the file, a file left empty by the failed rebuild is rejected as `CorruptEngineError` and can be rebuilt with `force_rebuild=True`.
+
+`workspace_limit` is excluded from the cache identity. Changing it reuses a matching saved engine; pass `force_rebuild=True` to `open_engine` to build again with the new limit. The saved header records the workspace limit used for that build, including `None`, as a reference outside the conditions used for matching. Cache reads leave this reference unchanged. `tensoracle cache list` shows the recorded limit.
 
 `plugins` names native TensorRT plugin libraries. Both `open_engine` and `build_serialized_engine` accept it. Each function checks that every library exports `getCreators`, loads it through TensorRT's registry, and retains it for the process before building the graph or deserializing an engine. Repeating the same library does not register it twice. `open_engine` also hashes the loaded library bytes into the engine identity; a changed library selects a different saved engine in a new process. A missing, unreadable, or incompatible library, or a library changed after loading in the same process, raises `PluginLibraryError` before a cached plan is opened. Keep the library and its cubin bundle together in the caller's wheel.
 
@@ -156,6 +160,30 @@ For each weight file, `open_engine` stores its SHA-256 alongside the saved engin
 
 `CorruptEngineError` reports incomplete or damaged saved files and plans that TensorRT cannot deserialize. When deserialization fails, its message asks the caller to check the required plugin libraries in `plugins=`. `EngineConditionError` reports a saved header with different build conditions. These types and `EngineIOError` derive from `EngineCacheError`. The caller chooses whether to rebuild after an error.
 
+## Manage saved engines
+
+Inspect saved engines and preview or perform cleanup with the installed command:
+
+```bash
+tensoracle cache list
+tensoracle cache prune --unused-for 30 --dry-run
+tensoracle cache prune --unused-for 30
+tensoracle cache prune --max-size 10GiB --dry-run
+tensoracle cache prune --max-size 10GiB
+```
+
+The commands use the same `XDG_CACHE_HOME` / home `.cache` directory as `open_engine`. They read only filesystem metadata, engine headers, and saved weight digest records. They do not read or checksum the plan body, load the original weights, deserialize engines, import TensorRT, or query a GPU. Listing recognizes the header format; it does not establish that an engine can be loaded.
+
+`cache list` prints one JSON object with `directory`, `entries`, and `total_size_bytes`. Each entry contains the file name, size in bytes, recorded time in UTC (`recorded_at`) and exact nanoseconds (`recorded_at_ns`), and the complete header conditions. This distinguishes engines with the same name and graph version but different weights or plugins. Each recognized entry also contains `workspace_limit`: a positive integer in bytes for an explicit build limit, JSON `null` when the build left the limit at TensorRT's default, or `"unknown"` when the header has no reference value. `weight_paths` associates each weight digest with paths from valid, checksum-verified `weight-*.json` records whose SHA-256 matches that digest; an empty `paths` array means unknown. These paths are references and do not establish which application uses an engine or whether the weight file still exists. Empty files have status `building or leftover`; unreadable or invalid headers have status `unrecognized` and a reason.
+
+An engine's `.engine` modification time records its initial save or the last successful `open_engine` cache read. Failed loads and these management commands leave that time unchanged. A timestamp write failure, including a read-only cache, does not prevent a successful load, and a disappeared file is never recreated to record its time. The record describes the time of `open_engine`, not whether inference continues afterward. Files saved before timestamp recording was introduced, and files whose timestamp could not be written, may not reflect the actual last successful read.
+
+`cache prune` requires exactly one of `--unused-for DAYS` and `--max-size SIZE`. Days are finite nonnegative numbers, may be fractional, and mean 24 hours each; only timestamps strictly earlier than the cutoff are selected. Sizes accept nonnegative bytes or the case-insensitive units `B`, `KiB`, `MiB`, `GiB`, and `TiB` (powers of 1024). Fractions such as `1.5GiB` are allowed and fractional bytes are rounded down. Size pruning selects the oldest recorded times first, breaking ties by file name, until the total `.engine` bytes would fit. Empty and unrecognized files count toward that total but are always excluded from deletion. Weight digest records are kept.
+
+Before deletion, pruning prints a JSON `plan` containing the selected engines, excluded files, and a notice that removed engines will be rebuilt the next time they are used. It then prints a JSON `result` separating removed, excluded, failed, and already missing files, the bytes actually removed, and the remaining total. `--dry-run` shows the same targets without changing file contents or recorded times; its result also gives the projected total. Exit status is 0 for success, 1 for filesystem failures, failed deletions, or an unattainable size limit, and 2 for invalid usage. Files that disappear before removal are reported in `missing` and do not count as bytes removed. Observed file changes are reported as failed removals with a reason.
+
+Run pruning during maintenance when no engine is being built or opened in the same cache directory. Cleanup provides no lock or guarantee of protection against concurrent builds and reads. Inspect the selection with `--dry-run` before performing cleanup. `tensoracle cache prune --help` describes the options.
+
 ## Use a saved engine for inference
 
 An inference application can call `open_engine` with the same build declaration and files used to save the engine. For example, assume the model package exports `graph`, `declaration`, and `io` matching the example above:
@@ -336,23 +364,13 @@ Install the producer extra when you need to compile cubins from your own kernels
 pip install 'tensoracle[producer]'
 ```
 
-Every repository that adopts tensoracle must use this top-level layout:
-
-```text
-cpp/
-  CMakeLists.txt
-python/
-  pyproject.toml
-  src/
-```
-
-Keep C++ sources in `cpp/`, with a CMake project that builds and runs its tests through CTest without Python. Keep the Python project, its src-layout package, and its uv environment in `python/`; run the project's uv commands from that directory. Tensoracle itself follows this layout, and the installed C++ helper paths described below remain inside its Python package.
+Use the [C++ sources and CMake helper supplied in the tensoracle wheel](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) to build your native plugin.
 
 `tensoracle.producer.compile_bundle` uses the extra's **Triton 3.8.0** with **ptxas >=13.2,<14** and **cuobjdump** from the PyPI `nvidia-cuda-nvcc` and `nvidia-cuda-cuobjdump` wheels. The extra pins those tools, `nvidia-cuda-crt`, and `nvidia-nvvm` to the 13.2 series. It does not read `CUDA_HOME`; explicit `ptxas` and `cuobjdump` paths override the wheel defaults independently. A missing tool identifies its package and searched path and explains how to install the extra or pass a path. The normal import, engine build, and engine reuse paths do not import the producer or Triton. Cubin production does not need a GPU or import torch; run it without importing torch first. The caller supplies Triton functions and `KernelSpec` records with their public argument types, compile time constants, and any divisibility assumptions. The producer writes sm86, sm89, and sm120 cubins, `bundle.json`, and a C++ translation unit that embeds exactly those cubins. The generated C++ entry records carry each public argument's order, type, count, and divisibility rule into the common launcher. It rejects an unexpected parameter ABI or nonzero scratch requirement. The included [example](https://github.com/sync-dev-org/tensoracle/blob/main/examples/cubin_plugin/README.md#rebuild-the-cubins) has the command to rebuild its cubins from a repository clone.
 
 Neither cubin production nor native plugin building requires an OS-installed CUDA Toolkit. For the native build, install PyPI `nvidia-cuda-nvcc==13.2.*` and `nvidia-cuda-runtime==13.2.*`, then pass their `site-packages/nvidia/cu13` directory as `CUDAToolkit_ROOT` (a CMake variable or environment variable). The helper uses its `cuda.h` and links the NVIDIA driver's `libcuda.so` on Linux, including `/usr/lib/wsl/lib/libcuda.so` on WSL, or the wheel's `lib/x64/cuda.lib` on Windows. TensorRT headers still come from the separately prepared SDK. The [SDK guide](https://github.com/sync-dev-org/tensoracle/blob/main/docs/tensorrt-sdk.md#5-pass-the-verified-root-to-the-build-and-tests) gives the setup commands. An explicit OS Toolkit root and the existing default locations remain supported for native builds.
 
-The wheel also contains `tensoracle/cpp/include`, `tensoracle/cpp/src`, and `tensoracle/cpp/cmake`. Pass the installed `tensoracle/cpp` directory to CMake as `TENSORACLE_CPP_ROOT` and include `TensoracleCubinPlugin.cmake`. Its `tensoracle_add_cubin_plugin` function compiles the common C++ source, the caller's plugin source, and the generated `bundle.cpp` into the caller's library. The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common source owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common source releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.
+The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common C++ implementation owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common implementation releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.
 
 To record the CUDA inputs used by the plugin build, call `tensoracle_find_cuda` from `TensoracleCuda.cmake` (also included by `TensoracleCubinPlugin.cmake`):
 

{tensoracle-1.4.0 → tensoracle-1.6.0}/README.md

@@ -31,7 +31,7 @@ The supported targets are Linux x86_64 (including WSL2) and Windows x86_64, Tens
 
 Follow the [four-stage optimization workflow](https://github.com/sync-dev-org/tensoracle/blob/main/docs/optimization.md) to establish output quality, evaluate lower precision and plugins, and measure speed.
 
-Follow [Build a C++ and Python repository](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) for source layout, development environments, bindings, and Linux wheel packaging.
+Follow [Build plugins with tensoracle's C++ sources](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) to locate the sources in the wheel and use the CMake helper in your plugin build.
 
 ## Convert a PyTorch checkpoint
 
@@ -77,7 +77,7 @@ def graph(network: trt.INetworkDefinition) -> None:
 
 declaration = EngineBuild(
     inputs={"features": InputShape.ranged((1, 1, 2), (1, 64, 2), (1, 448, 2))},
-    settings=BuilderSettings(workspace_limit=1 << 30, optimization_level=3),
+    settings=BuilderSettings(workspace_limit=None, optimization_level=3),
 )
 plan: bytes = build_serialized_engine(graph, declaration)
 ```
@@ -86,6 +86,8 @@ Use `InputShape.fixed((1, 64, 2))` for one input size; the graph may declare tha
 
 The engine's precision is determined by the tensor types that the graph function places in the network. For an fp16 graph, use `trt.float16` tensors and `np.float16` weights where needed; the graph can also contain fp32 tensors. `EngineBuild` declares input shapes and builder settings. TF32 is always disabled; `BuilderSettings(tf32=True)` raises `ValueError`. The function returns serialized plan bytes. Invalid declarations raise `ValueError`, and a failed TensorRT build raises `RuntimeError`.
 
+`BuilderSettings.workspace_limit` defaults to `None`, which leaves the workspace limit at TensorRT's default without calling `set_memory_pool_limit`. To impose a limit, pass a positive integer in bytes, such as `workspace_limit=4 << 30` for 4 GiB. Zero, negative values, booleans, and noninteger values raise `ValueError`.
+
 The same API builds bf16, int8, and fp8 graphs. Cast tensors to `trt.bfloat16` for bf16; for int8/fp8, place `add_quantize` / `add_dequantize` layers and load their scales from safetensors. The application determines scales with an external quantization library and chooses output quality limits with `compare_outputs`. fp8 targets sm89 and sm120; bf16 and int8 also target sm86. Follow [Build bf16, int8, and fp8 engines](https://github.com/sync-dev-org/tensoracle/blob/main/docs/precision.md) for a complete example, NumPy dtype constraints, and comparison against an fp32 engine. Use different engine names or graph versions for different precisions.
 
 ## Save and reopen an engine
@@ -111,7 +113,9 @@ handle = open_engine(
 engine = handle.engine  # Keep handle.runtime alive while using the engine.
 ```
 
-Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under the home directory's `.cache` when that variable is unset. The same rule applies on every OS; on Windows the home directory is `%USERPROFILE%`, so the default is `%USERPROFILE%\.cache\tensoracle`. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, builder settings, the GPU's compatibility properties, OS, CPU architecture, and TensorRT version. The GPU is the device of the current CUDA context, or device 0 when no context is current. Its properties are the product name, compute capability, memory bus width, L2 cache size, shared memory limits, texture alignment, multiprocessor count, whether the GPU is integrated, ECC mode, and total memory; another device with the same properties reuses the saved engine. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact. tensoracle closes a saved engine file after reading it, so another process using an opened engine does not block replacement. On Windows, a process that keeps the saved file itself open prevents replacement or removal with `PermissionError`; after that process closes the file, a file left empty by the failed rebuild is rejected as `CorruptEngineError` and can be rebuilt with `force_rebuild=True`.
+Saved files live in the `tensoracle` directory under `XDG_CACHE_HOME`, or under the home directory's `.cache` when that variable is unset. The same rule applies on every OS; on Windows the home directory is `%USERPROFILE%`, so the default is `%USERPROFILE%\.cache\tensoracle`. The cache identity includes the contents of all declared safetensors weight files and optional `plugins` files, the engine name, graph version, input shapes, the builder's `optimization_level` and `tf32` settings, the GPU's compatibility properties, OS, CPU architecture, and TensorRT version. The GPU is the device of the current CUDA context, or device 0 when no context is current. Its properties are the product name, compute capability, memory bus width, L2 cache size, shared memory limits, texture alignment, multiprocessor count, whether the GPU is integrated, ECC mode, and total memory; another device with the same properties reuses the saved engine. Raise `graph_version` when graph code changes; without a version change, the previous engine is reused. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. Include in `graph_version` any other caller value that changes the engine contents but is not part of the cache identity, such as a plugin manifest or a build recipe. Pass `force_rebuild=True` to replace a cached engine even when it is intact. tensoracle closes a saved engine file after reading it, so another process using an opened engine does not block replacement. On Windows, a process that keeps the saved file itself open prevents replacement or removal with `PermissionError`; after that process closes the file, a file left empty by the failed rebuild is rejected as `CorruptEngineError` and can be rebuilt with `force_rebuild=True`.
+
+`workspace_limit` is excluded from the cache identity. Changing it reuses a matching saved engine; pass `force_rebuild=True` to `open_engine` to build again with the new limit. The saved header records the workspace limit used for that build, including `None`, as a reference outside the conditions used for matching. Cache reads leave this reference unchanged. `tensoracle cache list` shows the recorded limit.
 
 `plugins` names native TensorRT plugin libraries. Both `open_engine` and `build_serialized_engine` accept it. Each function checks that every library exports `getCreators`, loads it through TensorRT's registry, and retains it for the process before building the graph or deserializing an engine. Repeating the same library does not register it twice. `open_engine` also hashes the loaded library bytes into the engine identity; a changed library selects a different saved engine in a new process. A missing, unreadable, or incompatible library, or a library changed after loading in the same process, raises `PluginLibraryError` before a cached plan is opened. Keep the library and its cubin bundle together in the caller's wheel.
 
@@ -123,6 +127,30 @@ For each weight file, `open_engine` stores its SHA-256 alongside the saved engin
 
 `CorruptEngineError` reports incomplete or damaged saved files and plans that TensorRT cannot deserialize. When deserialization fails, its message asks the caller to check the required plugin libraries in `plugins=`. `EngineConditionError` reports a saved header with different build conditions. These types and `EngineIOError` derive from `EngineCacheError`. The caller chooses whether to rebuild after an error.
 
+## Manage saved engines
+
+Inspect saved engines and preview or perform cleanup with the installed command:
+
+```bash
+tensoracle cache list
+tensoracle cache prune --unused-for 30 --dry-run
+tensoracle cache prune --unused-for 30
+tensoracle cache prune --max-size 10GiB --dry-run
+tensoracle cache prune --max-size 10GiB
+```
+
+The commands use the same `XDG_CACHE_HOME` / home `.cache` directory as `open_engine`. They read only filesystem metadata, engine headers, and saved weight digest records. They do not read or checksum the plan body, load the original weights, deserialize engines, import TensorRT, or query a GPU. Listing recognizes the header format; it does not establish that an engine can be loaded.
+
+`cache list` prints one JSON object with `directory`, `entries`, and `total_size_bytes`. Each entry contains the file name, size in bytes, recorded time in UTC (`recorded_at`) and exact nanoseconds (`recorded_at_ns`), and the complete header conditions. This distinguishes engines with the same name and graph version but different weights or plugins. Each recognized entry also contains `workspace_limit`: a positive integer in bytes for an explicit build limit, JSON `null` when the build left the limit at TensorRT's default, or `"unknown"` when the header has no reference value. `weight_paths` associates each weight digest with paths from valid, checksum-verified `weight-*.json` records whose SHA-256 matches that digest; an empty `paths` array means unknown. These paths are references and do not establish which application uses an engine or whether the weight file still exists. Empty files have status `building or leftover`; unreadable or invalid headers have status `unrecognized` and a reason.
+
+An engine's `.engine` modification time records its initial save or the last successful `open_engine` cache read. Failed loads and these management commands leave that time unchanged. A timestamp write failure, including a read-only cache, does not prevent a successful load, and a disappeared file is never recreated to record its time. The record describes the time of `open_engine`, not whether inference continues afterward. Files saved before timestamp recording was introduced, and files whose timestamp could not be written, may not reflect the actual last successful read.
+
+`cache prune` requires exactly one of `--unused-for DAYS` and `--max-size SIZE`. Days are finite nonnegative numbers, may be fractional, and mean 24 hours each; only timestamps strictly earlier than the cutoff are selected. Sizes accept nonnegative bytes or the case-insensitive units `B`, `KiB`, `MiB`, `GiB`, and `TiB` (powers of 1024). Fractions such as `1.5GiB` are allowed and fractional bytes are rounded down. Size pruning selects the oldest recorded times first, breaking ties by file name, until the total `.engine` bytes would fit. Empty and unrecognized files count toward that total but are always excluded from deletion. Weight digest records are kept.
+
+Before deletion, pruning prints a JSON `plan` containing the selected engines, excluded files, and a notice that removed engines will be rebuilt the next time they are used. It then prints a JSON `result` separating removed, excluded, failed, and already missing files, the bytes actually removed, and the remaining total. `--dry-run` shows the same targets without changing file contents or recorded times; its result also gives the projected total. Exit status is 0 for success, 1 for filesystem failures, failed deletions, or an unattainable size limit, and 2 for invalid usage. Files that disappear before removal are reported in `missing` and do not count as bytes removed. Observed file changes are reported as failed removals with a reason.
+
+Run pruning during maintenance when no engine is being built or opened in the same cache directory. Cleanup provides no lock or guarantee of protection against concurrent builds and reads. Inspect the selection with `--dry-run` before performing cleanup. `tensoracle cache prune --help` describes the options.
+
 ## Use a saved engine for inference
 
 An inference application can call `open_engine` with the same build declaration and files used to save the engine. For example, assume the model package exports `graph`, `declaration`, and `io` matching the example above:
@@ -303,23 +331,13 @@ Install the producer extra when you need to compile cubins from your own kernels
 pip install 'tensoracle[producer]'
 ```
 
-Every repository that adopts tensoracle must use this top-level layout:
-
-```text
-cpp/
-  CMakeLists.txt
-python/
-  pyproject.toml
-  src/
-```
-
-Keep C++ sources in `cpp/`, with a CMake project that builds and runs its tests through CTest without Python. Keep the Python project, its src-layout package, and its uv environment in `python/`; run the project's uv commands from that directory. Tensoracle itself follows this layout, and the installed C++ helper paths described below remain inside its Python package.
+Use the [C++ sources and CMake helper supplied in the tensoracle wheel](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) to build your native plugin.
 
 `tensoracle.producer.compile_bundle` uses the extra's **Triton 3.8.0** with **ptxas >=13.2,<14** and **cuobjdump** from the PyPI `nvidia-cuda-nvcc` and `nvidia-cuda-cuobjdump` wheels. The extra pins those tools, `nvidia-cuda-crt`, and `nvidia-nvvm` to the 13.2 series. It does not read `CUDA_HOME`; explicit `ptxas` and `cuobjdump` paths override the wheel defaults independently. A missing tool identifies its package and searched path and explains how to install the extra or pass a path. The normal import, engine build, and engine reuse paths do not import the producer or Triton. Cubin production does not need a GPU or import torch; run it without importing torch first. The caller supplies Triton functions and `KernelSpec` records with their public argument types, compile time constants, and any divisibility assumptions. The producer writes sm86, sm89, and sm120 cubins, `bundle.json`, and a C++ translation unit that embeds exactly those cubins. The generated C++ entry records carry each public argument's order, type, count, and divisibility rule into the common launcher. It rejects an unexpected parameter ABI or nonzero scratch requirement. The included [example](https://github.com/sync-dev-org/tensoracle/blob/main/examples/cubin_plugin/README.md#rebuild-the-cubins) has the command to rebuild its cubins from a repository clone.
 
 Neither cubin production nor native plugin building requires an OS-installed CUDA Toolkit. For the native build, install PyPI `nvidia-cuda-nvcc==13.2.*` and `nvidia-cuda-runtime==13.2.*`, then pass their `site-packages/nvidia/cu13` directory as `CUDAToolkit_ROOT` (a CMake variable or environment variable). The helper uses its `cuda.h` and links the NVIDIA driver's `libcuda.so` on Linux, including `/usr/lib/wsl/lib/libcuda.so` on WSL, or the wheel's `lib/x64/cuda.lib` on Windows. TensorRT headers still come from the separately prepared SDK. The [SDK guide](https://github.com/sync-dev-org/tensoracle/blob/main/docs/tensorrt-sdk.md#5-pass-the-verified-root-to-the-build-and-tests) gives the setup commands. An explicit OS Toolkit root and the existing default locations remain supported for native builds.
 
-The wheel also contains `tensoracle/cpp/include`, `tensoracle/cpp/src`, and `tensoracle/cpp/cmake`. Pass the installed `tensoracle/cpp` directory to CMake as `TENSORACLE_CPP_ROOT` and include `TensoracleCubinPlugin.cmake`. Its `tensoracle_add_cubin_plugin` function compiles the common C++ source, the caller's plugin source, and the generated `bundle.cpp` into the caller's library. The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common source owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common source releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.
+The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common C++ implementation owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common implementation releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.
 
 To record the CUDA inputs used by the plugin build, call `tensoracle_find_cuda` from `TensoracleCuda.cmake` (also included by `TensoracleCubinPlugin.cmake`):
 

tensoracle-1.6.0/docs/cpp-python-repositories.md

@@ -0,0 +1,65 @@
+# Build plugins with tensoracle's C++ sources
+
+Tensoracle supplies the shared C++ implementation and CMake helpers for cubin plugins. Your model package supplies the kernels and model-specific plugin behavior, compiles the native plugin, and packages it for the application.
+
+## C++ sources in the wheel
+
+The tensoracle wheel includes:
+
+```text
+tensoracle/cpp/
+  include/tensoracle/cubin_plugin.hpp
+  src/cubin_plugin.cpp
+  cmake/TensoracleCubinPlugin.cmake
+  cmake/TensoracleCuda.cmake
+```
+
+The wheel contains no native binaries and uses the `py3-none-any` tag. Your plugin's native library is compiled separately and belongs to your model package.
+
+## Prepare the build inputs
+
+Use CMake 3.20 or later and a C++17 compiler. Follow [Prepare a TensorRT SDK](tensorrt-sdk.md) to obtain and verify the TensorRT 11 headers and library and prepare the PyPI CUDA headers and driver link library. That guide covers Linux, WSL2, and [Windows](tensorrt-sdk.md#windows-x86_64); no OS-installed CUDA Toolkit is required.
+
+Generate a cubin bundle with the [cubin producer](api/producer.md). The bundle directory must contain `bundle.cpp`, `bundle.json`, and the cubin files. Cubin production runs on Linux; the same bundle can be used to build a Windows plugin DLL. Follow [Build a cubin plugin](index.md#build-a-cubin-plugin) for the model-specific plugin interface and tensor declarations.
+
+## Use the packaged CMake helper
+
+In the Python environment where tensoracle is installed, locate its C++ source directory:
+
+```bash
+python -c 'from importlib.metadata import distribution; print(distribution("tensoracle").locate_file("tensoracle/cpp"))'
+```
+
+Pass that directory as `TENSORACLE_CPP_ROOT` to your plugin's CMake project. Include the helper and give it your plugin source, verified SDK root, and generated bundle directory:
+
+```cmake
+cmake_minimum_required(VERSION 3.20)
+project(my_model_plugin LANGUAGES CXX)
+
+include("${TENSORACLE_CPP_ROOT}/cmake/TensoracleCubinPlugin.cmake")
+tensoracle_add_cubin_plugin(
+    TARGET my_model_plugin
+    TENSORRT_ROOT "${TENSORRT_ROOT}"
+    BUNDLE_ROOT "${TENSORACLE_BUNDLE_ROOT}"
+    SOURCES "${CMAKE_CURRENT_SOURCE_DIR}/plugin.cpp")
+```
+
+`tensoracle_add_cubin_plugin` builds a shared library from your `plugin.cpp`, tensoracle's common `cubin_plugin.cpp`, and the bundle's generated `bundle.cpp`. It adds the required include directories and links TensorRT and the CUDA driver.
+
+For a Bash build, replace the absolute paths below with your plugin project, installed tensoracle sources, verified SDK, PyPI CUDA root, and generated bundle:
+
+```bash
+cmake -S /absolute/path/to/plugin-project -B build/plugin \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DTENSORACLE_CPP_ROOT=/absolute/path/to/tensoracle/cpp \
+  -DTENSORRT_ROOT=/absolute/path/to/TensorRT-SDK \
+  -DCUDAToolkit_ROOT=/absolute/path/to/site-packages/nvidia/cu13 \
+  -DTENSORACLE_BUNDLE_ROOT=/absolute/path/to/bundle
+cmake --build build/plugin --config Release
+```
+
+The [plugin guide](https://github.com/sync-dev-org/tensoracle#build-a-cubin-plugin) documents the required `getCreators` export and the optional `tensoracle_find_cuda` function and `tensoracle_cuda_driver` target for consumers that also need the resolved CUDA inputs.
+
+## Package and inspect the plugin
+
+Follow the [cubin plugin example](https://github.com/sync-dev-org/tensoracle/blob/main/examples/cubin_plugin/README.md#build-and-inspect-the-wheel) for packaging the native library and bundle in your model wheel and testing it in a runtime environment. Use the [bundle inspection API](api/bundle.md) to check the bundle and its match with the compiled plugin before distribution.

{tensoracle-1.4.0 → tensoracle-1.6.0}/docs/index.md

@@ -4,7 +4,7 @@
 
 Follow the [four-stage optimization workflow](optimization.md) to establish output quality, evaluate lower precision and plugins, and measure speed.
 
-Follow [Build a C++ and Python repository](cpp-python-repositories.md) for source layout, development environments, bindings, and Linux wheel packaging.
+Follow [Build plugins with tensoracle's C++ sources](cpp-python-repositories.md) to locate the sources in the wheel and use the CMake helper in your plugin build.
 
 ## Terms
 
@@ -74,6 +74,32 @@ handle = open_engine(
 
 `my_model` represents the application's model package. On a valid cache hit, `open_engine` does not call `graph`; on a miss, it builds and saves a new engine. Raise `graph_version` when the graph code changes or when any other caller value that changes the engine contents but is not part of the cache identity changes, such as a plugin manifest or a build recipe. `open_engine` itself does not execute the engine or manage GPU buffers. Create an execution unit from its returned handle to enqueue inference. Saved engines live under `XDG_CACHE_HOME/tensoracle`, or `~/.cache/tensoracle` when `XDG_CACHE_HOME` is unset. On Windows, `~` is `%USERPROFILE%`. See the [complete cache example](https://github.com/sync-dev-org/tensoracle#save-and-reopen-an-engine) for the declaration and expected input and output types.
 
+`workspace_limit` is excluded from the cache identity; `optimization_level` and `tf32` remain in the conditions. Changing only the workspace limit reuses a matching saved engine. Pass `force_rebuild=True` to `open_engine` to build again with the new limit. The saved header records the workspace limit used for that build, including `None`, as a reference outside the conditions used for matching. Cache reads leave this reference unchanged. `tensoracle cache list` shows the recorded limit.
+
+### Manage saved engines
+
+Inspect saved engines and preview or perform cleanup with the installed command:
+
+```bash
+tensoracle cache list
+tensoracle cache prune --unused-for 30 --dry-run
+tensoracle cache prune --unused-for 30
+tensoracle cache prune --max-size 10GiB --dry-run
+tensoracle cache prune --max-size 10GiB
+```
+
+The commands use the same `XDG_CACHE_HOME` / home `.cache` directory as `open_engine`. They read only filesystem metadata, engine headers, and saved weight digest records. They do not read or checksum the plan body, load the original weights, deserialize engines, import TensorRT, or query a GPU. Listing recognizes the header format; it does not establish that an engine can be loaded.
+
+`cache list` prints one JSON object with `directory`, `entries`, and `total_size_bytes`. Each entry contains the file name, size in bytes, recorded time in UTC (`recorded_at`) and exact nanoseconds (`recorded_at_ns`), and the complete header conditions. This distinguishes engines with the same name and graph version but different weights or plugins. Each recognized entry also contains `workspace_limit`: a positive integer in bytes for an explicit build limit, JSON `null` when the build left the limit at TensorRT's default, or `"unknown"` when the header has no reference value. `weight_paths` associates each weight digest with paths from valid, checksum-verified `weight-*.json` records whose SHA-256 matches that digest; an empty `paths` array means unknown. These paths are references and do not establish which application uses an engine or whether the weight file still exists. Empty files have status `building or leftover`; unreadable or invalid headers have status `unrecognized` and a reason.
+
+An engine's `.engine` modification time records its initial save or the last successful `open_engine` cache read. Failed loads and these management commands leave that time unchanged. A timestamp write failure, including a read-only cache, does not prevent a successful load, and a disappeared file is never recreated to record its time. The record describes the time of `open_engine`, not whether inference continues afterward. Files saved before timestamp recording was introduced, and files whose timestamp could not be written, may not reflect the actual last successful read.
+
+`cache prune` requires exactly one of `--unused-for DAYS` and `--max-size SIZE`. Days are finite nonnegative numbers, may be fractional, and mean 24 hours each; only timestamps strictly earlier than the cutoff are selected. Sizes accept nonnegative bytes or the case-insensitive units `B`, `KiB`, `MiB`, `GiB`, and `TiB` (powers of 1024). Fractions such as `1.5GiB` are allowed and fractional bytes are rounded down. Size pruning selects the oldest recorded times first, breaking ties by file name, until the total `.engine` bytes would fit. Empty and unrecognized files count toward that total but are always excluded from deletion. Weight digest records are kept.
+
+Before deletion, pruning prints a JSON `plan` containing the selected engines, excluded files, and a notice that removed engines will be rebuilt the next time they are used. It then prints a JSON `result` separating removed, excluded, failed, and already missing files, the bytes actually removed, and the remaining total. `--dry-run` shows the same targets without changing file contents or recorded times; its result also gives the projected total. Exit status is 0 for success, 1 for filesystem failures, failed deletions, or an unattainable size limit, and 2 for invalid usage. Files that disappear before removal are reported in `missing` and do not count as bytes removed. Observed file changes are reported as failed removals with a reason.
+
+Run pruning during maintenance when no engine is being built or opened in the same cache directory. Cleanup provides no lock or guarantee of protection against concurrent builds and reads. Inspect the selection with `--dry-run` before performing cleanup. `tensoracle cache prune --help` describes the options.
+
 ### Execute an engine
 
 The execution flow is: open the engine, create an execution unit, set all ranged input shapes together, read concrete output shapes and dtypes, allocate output buffers, then enqueue the device addresses on a caller-owned stream. Each `EngineExecution` keeps the handle alive and owns an independent TensorRT context. Fixed inputs and single-point profiles are initialized when the unit is created. Ranged inputs retain their shapes until the next successful `set_input_shapes` call. A partial TensorRT setting failure blocks queries for concrete shapes and enqueue until a complete setting succeeds.
@@ -228,13 +254,13 @@ Engines with loops or conditionals, or layers or plugins that synchronize with t
 
 ### Build a standard graph
 
-Define `graph(network)` with TensorRT's Python network API and load model weights from safetensors. Describe each graph input with `InputShape.fixed` or `InputShape.ranged`, and pass an `EngineBuild` declaration to `build_serialized_engine`. `EngineBuild` declares input shapes and builder settings; the graph function determines precision through the tensor types it places in the network. The supported precisions are fp32 (with TF32 disabled), fp16, bf16, int8, and fp8. The [bf16, int8, and fp8 guide](precision.md) shows tensor casts, explicit Q/DQ layers, and scales supplied through safetensors; fp8 targets sm89 and sm120. `open_engine` uses the same declaration plus a name, graph version, weight files, and `EngineIO` to save and reuse a checked engine. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. The [graph example](https://github.com/sync-dev-org/tensoracle#define-and-build-a-standard-graph) shows the exact calls.
+Define `graph(network)` with TensorRT's Python network API and load model weights from safetensors. Describe each graph input with `InputShape.fixed` or `InputShape.ranged`, and pass an `EngineBuild` declaration to `build_serialized_engine`. `EngineBuild` declares input shapes and builder settings; the graph function determines precision through the tensor types it places in the network. `BuilderSettings.workspace_limit` defaults to `None`, leaving the workspace limit at TensorRT's default. To impose a limit, pass a positive integer in bytes, such as `workspace_limit=4 << 30` for 4 GiB. Zero, negative values, booleans, and noninteger values raise `ValueError`. The supported precisions are fp32 (with TF32 disabled), fp16, bf16, int8, and fp8. The [bf16, int8, and fp8 guide](precision.md) shows tensor casts, explicit Q/DQ layers, and scales supplied through safetensors; fp8 targets sm89 and sm120. `open_engine` uses the same declaration plus a name, graph version, weight files, and `EngineIO` to save and reuse a checked engine. Use a different `name` or `graph_version` for engines with different tensor types, such as fp32 and fp16. The [graph example](https://github.com/sync-dev-org/tensoracle#define-and-build-a-standard-graph) shows the exact calls.
 
 The application runs the engine and passes a CPU NumPy candidate output and a corresponding reference output to `compare_outputs`. The application chooses the acceptance limits. See [Compare output arrays](https://github.com/sync-dev-org/tensoracle#compare-output-arrays).
 
 ### Build a cubin plugin
 
-Repositories that adopt tensoracle must follow the [repository layout in the plugin guide](https://github.com/sync-dev-org/tensoracle#build-a-cubin-plugin).
+Use the [C++ sources and CMake helper supplied in the tensoracle wheel](cpp-python-repositories.md) to build your native plugin.
 
 Install `tensoracle[producer]` when you need to compile cubins from your own kernels:
 

{tensoracle-1.4.0 → tensoracle-1.6.0}/mkdocs.yml

@@ -35,7 +35,7 @@ nav:
   - Optimization workflow: optimization.md
   - bf16, int8, and fp8 engines: precision.md
   - TensorRT SDK setup: tensorrt-sdk.md
-  - C++ and Python repositories: cpp-python-repositories.md
+  - C++ plugin builds: cpp-python-repositories.md
   - API reference:
       - Engine and comparison: api/index.md
       - Cubin bundle inspection: api/bundle.md

{tensoracle-1.4.0 → tensoracle-1.6.0}/pyproject.toml

@@ -24,6 +24,9 @@ dependencies = [
     "tensorrt-cu13>=11.0,<12",
 ]
 
+[project.scripts]
+tensoracle = "tensoracle._cli:main"
+
 [project.urls]
 documentation = "https://sync-dev-org.github.io/tensoracle/"
 repository = "https://github.com/sync-dev-org/tensoracle"
@@ -63,6 +66,7 @@ exclude = [
     "/.github/",
     "/AGENTS.md",
     "/CLAUDE.md",
+    "/intro/",
     "/tools/",
     "/noren.yaml",
     "/noren-words.txt",

{tensoracle-1.4.0 → tensoracle-1.6.0}/src/tensoracle/_build.py

@@ -69,10 +69,16 @@ class InputShape:
 class BuilderSettings:
     """TensorRT builder options that affect the serialized engine."""
 
-    workspace_limit: int = 1 << 30
+    workspace_limit: int | None = None
     optimization_level: int = 3
     tf32: bool = False
 
+    def __post_init__(self) -> None:
+        if self.workspace_limit is not None and (
+            type(self.workspace_limit) is not int or self.workspace_limit <= 0
+        ):
+            raise ValueError("workspace_limit must be a positive integer or None")
+
 
 @dataclass(frozen=True)
 class EngineBuild:
@@ -103,8 +109,6 @@ def _build_serialized_engine(
 ) -> tuple[bytes, tuple[str, ...]]:
     if declaration.settings.tf32:
         raise ValueError("TF32 cannot be enabled")
-    if declaration.settings.workspace_limit <= 0:
-        raise ValueError("workspace_limit must be positive")
     if not 0 <= declaration.settings.optimization_level <= 5:
         raise ValueError("optimization_level must be between 0 and 5")
 
@@ -125,9 +129,10 @@ def _build_serialized_engine(
         _warn_shape_tensor_outputs(output_names, stacklevel=3)
         config = builder.create_builder_config()
         config.clear_flag(trt.BuilderFlag.TF32)
-        config.set_memory_pool_limit(
-            trt.MemoryPoolType.WORKSPACE, declaration.settings.workspace_limit
-        )
+        if declaration.settings.workspace_limit is not None:
+            config.set_memory_pool_limit(
+                trt.MemoryPoolType.WORKSPACE, declaration.settings.workspace_limit
+            )
         config.builder_optimization_level = declaration.settings.optimization_level
 
         inputs = {