tensoracle 1.4.0__tar.gz → 1.5.0__tar.gz

{tensoracle-1.4.0 → tensoracle-1.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tensoracle
-Version: 1.4.0
+Version: 1.5.0
 Summary: Build, save, reuse, and execute TensorRT engines from safetensors weights
 Project-URL: documentation, https://sync-dev-org.github.io/tensoracle/
 Project-URL: repository, https://github.com/sync-dev-org/tensoracle
@@ -64,7 +64,7 @@ The supported targets are Linux x86_64 (including WSL2) and Windows x86_64, Tens
 
 Follow the [four-stage optimization workflow](https://github.com/sync-dev-org/tensoracle/blob/main/docs/optimization.md) to establish output quality, evaluate lower precision and plugins, and measure speed.
 
-Follow [Build a C++ and Python repository](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) for source layout, development environments, bindings, and Linux wheel packaging.
+Follow [Build plugins with tensoracle's C++ sources](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) to locate the sources in the wheel and use the CMake helper in your plugin build.
 
 ## Convert a PyTorch checkpoint
 
@@ -156,6 +156,30 @@ For each weight file, `open_engine` stores its SHA-256 alongside the saved engin
 
 `CorruptEngineError` reports incomplete or damaged saved files and plans that TensorRT cannot deserialize. When deserialization fails, its message asks the caller to check the required plugin libraries in `plugins=`. `EngineConditionError` reports a saved header with different build conditions. These types and `EngineIOError` derive from `EngineCacheError`. The caller chooses whether to rebuild after an error.
 
+## Manage saved engines
+
+Inspect saved engines and preview or perform cleanup with the installed command:
+
+```bash
+tensoracle cache list
+tensoracle cache prune --unused-for 30 --dry-run
+tensoracle cache prune --unused-for 30
+tensoracle cache prune --max-size 10GiB --dry-run
+tensoracle cache prune --max-size 10GiB
+```
+
+The commands use the same `XDG_CACHE_HOME` / home `.cache` directory as `open_engine`. They read only filesystem metadata, engine headers, and saved weight digest records. They do not read or checksum the plan body, load the original weights, deserialize engines, import TensorRT, or query a GPU. Listing recognizes the header format; it does not establish that an engine can be loaded.
+
+`cache list` prints one JSON object with `directory`, `entries`, and `total_size_bytes`. Each entry contains the file name, size in bytes, recorded time in UTC (`recorded_at`) and exact nanoseconds (`recorded_at_ns`), and the complete header conditions. This distinguishes engines with the same name and graph version but different weights or plugins. `weight_paths` associates each weight digest with paths from valid, checksum-verified `weight-*.json` records whose SHA-256 matches that digest; an empty `paths` array means unknown. These paths are references and do not establish which application uses an engine or whether the weight file still exists. Empty files have status `building or leftover`; unreadable or invalid headers have status `unrecognized` and a reason.
+
+An engine's `.engine` modification time records its initial save or the last successful `open_engine` cache read. Failed loads and these management commands leave that time unchanged. A timestamp write failure, including a read-only cache, does not prevent a successful load, and a disappeared file is never recreated to record its time. The record describes the time of `open_engine`, not whether inference continues afterward. Files saved before timestamp recording was introduced, and files whose timestamp could not be written, may not reflect the actual last successful read.
+
+`cache prune` requires exactly one of `--unused-for DAYS` and `--max-size SIZE`. Days are finite nonnegative numbers, may be fractional, and mean 24 hours each; only timestamps strictly earlier than the cutoff are selected. Sizes accept nonnegative bytes or the case-insensitive units `B`, `KiB`, `MiB`, `GiB`, and `TiB` (powers of 1024). Fractions such as `1.5GiB` are allowed and fractional bytes are rounded down. Size pruning selects the oldest recorded times first, breaking ties by file name, until the total `.engine` bytes would fit. Empty and unrecognized files count toward that total but are always excluded from deletion. Weight digest records are kept.
+
+Before deletion, pruning prints a JSON `plan` containing the selected engines, excluded files, and a notice that removed engines will be rebuilt the next time they are used. It then prints a JSON `result` separating removed, excluded, failed, and already missing files, the bytes actually removed, and the remaining total. `--dry-run` shows the same targets without changing file contents or recorded times; its result also gives the projected total. Exit status is 0 for success, 1 for filesystem failures, failed deletions, or an unattainable size limit, and 2 for invalid usage. Files that disappear before removal are reported in `missing` and do not count as bytes removed. Observed file changes are reported as failed removals with a reason.
+
+Run pruning during maintenance when no engine is being built or opened in the same cache directory. Cleanup provides no lock or guarantee of protection against concurrent builds and reads. Inspect the selection with `--dry-run` before performing cleanup. `tensoracle cache prune --help` describes the options.
+
 ## Use a saved engine for inference
 
 An inference application can call `open_engine` with the same build declaration and files used to save the engine. For example, assume the model package exports `graph`, `declaration`, and `io` matching the example above:
@@ -336,23 +360,13 @@ Install the producer extra when you need to compile cubins from your own kernels
 pip install 'tensoracle[producer]'
 ```
 
-Every repository that adopts tensoracle must use this top-level layout:
-
-```text
-cpp/
-  CMakeLists.txt
-python/
-  pyproject.toml
-  src/
-```
-
-Keep C++ sources in `cpp/`, with a CMake project that builds and runs its tests through CTest without Python. Keep the Python project, its src-layout package, and its uv environment in `python/`; run the project's uv commands from that directory. Tensoracle itself follows this layout, and the installed C++ helper paths described below remain inside its Python package.
+Use the [C++ sources and CMake helper supplied in the tensoracle wheel](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) to build your native plugin.
 
 `tensoracle.producer.compile_bundle` uses the extra's **Triton 3.8.0** with **ptxas >=13.2,<14** and **cuobjdump** from the PyPI `nvidia-cuda-nvcc` and `nvidia-cuda-cuobjdump` wheels. The extra pins those tools, `nvidia-cuda-crt`, and `nvidia-nvvm` to the 13.2 series. It does not read `CUDA_HOME`; explicit `ptxas` and `cuobjdump` paths override the wheel defaults independently. A missing tool identifies its package and searched path and explains how to install the extra or pass a path. The normal import, engine build, and engine reuse paths do not import the producer or Triton. Cubin production does not need a GPU or import torch; run it without importing torch first. The caller supplies Triton functions and `KernelSpec` records with their public argument types, compile time constants, and any divisibility assumptions. The producer writes sm86, sm89, and sm120 cubins, `bundle.json`, and a C++ translation unit that embeds exactly those cubins. The generated C++ entry records carry each public argument's order, type, count, and divisibility rule into the common launcher. It rejects an unexpected parameter ABI or nonzero scratch requirement. The included [example](https://github.com/sync-dev-org/tensoracle/blob/main/examples/cubin_plugin/README.md#rebuild-the-cubins) has the command to rebuild its cubins from a repository clone.
 
 Neither cubin production nor native plugin building requires an OS-installed CUDA Toolkit. For the native build, install PyPI `nvidia-cuda-nvcc==13.2.*` and `nvidia-cuda-runtime==13.2.*`, then pass their `site-packages/nvidia/cu13` directory as `CUDAToolkit_ROOT` (a CMake variable or environment variable). The helper uses its `cuda.h` and links the NVIDIA driver's `libcuda.so` on Linux, including `/usr/lib/wsl/lib/libcuda.so` on WSL, or the wheel's `lib/x64/cuda.lib` on Windows. TensorRT headers still come from the separately prepared SDK. The [SDK guide](https://github.com/sync-dev-org/tensoracle/blob/main/docs/tensorrt-sdk.md#5-pass-the-verified-root-to-the-build-and-tests) gives the setup commands. An explicit OS Toolkit root and the existing default locations remain supported for native builds.
 
-The wheel also contains `tensoracle/cpp/include`, `tensoracle/cpp/src`, and `tensoracle/cpp/cmake`. Pass the installed `tensoracle/cpp` directory to CMake as `TENSORACLE_CPP_ROOT` and include `TensoracleCubinPlugin.cmake`. Its `tensoracle_add_cubin_plugin` function compiles the common C++ source, the caller's plugin source, and the generated `bundle.cpp` into the caller's library. The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common source owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common source releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.
+The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common C++ implementation owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common implementation releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.
 
 To record the CUDA inputs used by the plugin build, call `tensoracle_find_cuda` from `TensoracleCuda.cmake` (also included by `TensoracleCubinPlugin.cmake`):
 

{tensoracle-1.4.0 → tensoracle-1.5.0}/README.md

@@ -31,7 +31,7 @@ The supported targets are Linux x86_64 (including WSL2) and Windows x86_64, Tens
 
 Follow the [four-stage optimization workflow](https://github.com/sync-dev-org/tensoracle/blob/main/docs/optimization.md) to establish output quality, evaluate lower precision and plugins, and measure speed.
 
-Follow [Build a C++ and Python repository](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) for source layout, development environments, bindings, and Linux wheel packaging.
+Follow [Build plugins with tensoracle's C++ sources](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) to locate the sources in the wheel and use the CMake helper in your plugin build.
 
 ## Convert a PyTorch checkpoint
 
@@ -123,6 +123,30 @@ For each weight file, `open_engine` stores its SHA-256 alongside the saved engin
 
 `CorruptEngineError` reports incomplete or damaged saved files and plans that TensorRT cannot deserialize. When deserialization fails, its message asks the caller to check the required plugin libraries in `plugins=`. `EngineConditionError` reports a saved header with different build conditions. These types and `EngineIOError` derive from `EngineCacheError`. The caller chooses whether to rebuild after an error.
 
+## Manage saved engines
+
+Inspect saved engines and preview or perform cleanup with the installed command:
+
+```bash
+tensoracle cache list
+tensoracle cache prune --unused-for 30 --dry-run
+tensoracle cache prune --unused-for 30
+tensoracle cache prune --max-size 10GiB --dry-run
+tensoracle cache prune --max-size 10GiB
+```
+
+The commands use the same `XDG_CACHE_HOME` / home `.cache` directory as `open_engine`. They read only filesystem metadata, engine headers, and saved weight digest records. They do not read or checksum the plan body, load the original weights, deserialize engines, import TensorRT, or query a GPU. Listing recognizes the header format; it does not establish that an engine can be loaded.
+
+`cache list` prints one JSON object with `directory`, `entries`, and `total_size_bytes`. Each entry contains the file name, size in bytes, recorded time in UTC (`recorded_at`) and exact nanoseconds (`recorded_at_ns`), and the complete header conditions. This distinguishes engines with the same name and graph version but different weights or plugins. `weight_paths` associates each weight digest with paths from valid, checksum-verified `weight-*.json` records whose SHA-256 matches that digest; an empty `paths` array means unknown. These paths are references and do not establish which application uses an engine or whether the weight file still exists. Empty files have status `building or leftover`; unreadable or invalid headers have status `unrecognized` and a reason.
+
+An engine's `.engine` modification time records its initial save or the last successful `open_engine` cache read. Failed loads and these management commands leave that time unchanged. A timestamp write failure, including a read-only cache, does not prevent a successful load, and a disappeared file is never recreated to record its time. The record describes the time of `open_engine`, not whether inference continues afterward. Files saved before timestamp recording was introduced, and files whose timestamp could not be written, may not reflect the actual last successful read.
+
+`cache prune` requires exactly one of `--unused-for DAYS` and `--max-size SIZE`. Days are finite nonnegative numbers, may be fractional, and mean 24 hours each; only timestamps strictly earlier than the cutoff are selected. Sizes accept nonnegative bytes or the case-insensitive units `B`, `KiB`, `MiB`, `GiB`, and `TiB` (powers of 1024). Fractions such as `1.5GiB` are allowed and fractional bytes are rounded down. Size pruning selects the oldest recorded times first, breaking ties by file name, until the total `.engine` bytes would fit. Empty and unrecognized files count toward that total but are always excluded from deletion. Weight digest records are kept.
+
+Before deletion, pruning prints a JSON `plan` containing the selected engines, excluded files, and a notice that removed engines will be rebuilt the next time they are used. It then prints a JSON `result` separating removed, excluded, failed, and already missing files, the bytes actually removed, and the remaining total. `--dry-run` shows the same targets without changing file contents or recorded times; its result also gives the projected total. Exit status is 0 for success, 1 for filesystem failures, failed deletions, or an unattainable size limit, and 2 for invalid usage. Files that disappear before removal are reported in `missing` and do not count as bytes removed. Observed file changes are reported as failed removals with a reason.
+
+Run pruning during maintenance when no engine is being built or opened in the same cache directory. Cleanup provides no lock or guarantee of protection against concurrent builds and reads. Inspect the selection with `--dry-run` before performing cleanup. `tensoracle cache prune --help` describes the options.
+
 ## Use a saved engine for inference
 
 An inference application can call `open_engine` with the same build declaration and files used to save the engine. For example, assume the model package exports `graph`, `declaration`, and `io` matching the example above:
@@ -303,23 +327,13 @@ Install the producer extra when you need to compile cubins from your own kernels
 pip install 'tensoracle[producer]'
 ```
 
-Every repository that adopts tensoracle must use this top-level layout:
-
-```text
-cpp/
-  CMakeLists.txt
-python/
-  pyproject.toml
-  src/
-```
-
-Keep C++ sources in `cpp/`, with a CMake project that builds and runs its tests through CTest without Python. Keep the Python project, its src-layout package, and its uv environment in `python/`; run the project's uv commands from that directory. Tensoracle itself follows this layout, and the installed C++ helper paths described below remain inside its Python package.
+Use the [C++ sources and CMake helper supplied in the tensoracle wheel](https://sync-dev-org.github.io/tensoracle/cpp-python-repositories/) to build your native plugin.
 
 `tensoracle.producer.compile_bundle` uses the extra's **Triton 3.8.0** with **ptxas >=13.2,<14** and **cuobjdump** from the PyPI `nvidia-cuda-nvcc` and `nvidia-cuda-cuobjdump` wheels. The extra pins those tools, `nvidia-cuda-crt`, and `nvidia-nvvm` to the 13.2 series. It does not read `CUDA_HOME`; explicit `ptxas` and `cuobjdump` paths override the wheel defaults independently. A missing tool identifies its package and searched path and explains how to install the extra or pass a path. The normal import, engine build, and engine reuse paths do not import the producer or Triton. Cubin production does not need a GPU or import torch; run it without importing torch first. The caller supplies Triton functions and `KernelSpec` records with their public argument types, compile time constants, and any divisibility assumptions. The producer writes sm86, sm89, and sm120 cubins, `bundle.json`, and a C++ translation unit that embeds exactly those cubins. The generated C++ entry records carry each public argument's order, type, count, and divisibility rule into the common launcher. It rejects an unexpected parameter ABI or nonzero scratch requirement. The included [example](https://github.com/sync-dev-org/tensoracle/blob/main/examples/cubin_plugin/README.md#rebuild-the-cubins) has the command to rebuild its cubins from a repository clone.
 
 Neither cubin production nor native plugin building requires an OS-installed CUDA Toolkit. For the native build, install PyPI `nvidia-cuda-nvcc==13.2.*` and `nvidia-cuda-runtime==13.2.*`, then pass their `site-packages/nvidia/cu13` directory as `CUDAToolkit_ROOT` (a CMake variable or environment variable). The helper uses its `cuda.h` and links the NVIDIA driver's `libcuda.so` on Linux, including `/usr/lib/wsl/lib/libcuda.so` on WSL, or the wheel's `lib/x64/cuda.lib` on Windows. TensorRT headers still come from the separately prepared SDK. The [SDK guide](https://github.com/sync-dev-org/tensoracle/blob/main/docs/tensorrt-sdk.md#5-pass-the-verified-root-to-the-build-and-tests) gives the setup commands. An explicit OS Toolkit root and the existing default locations remain supported for native builds.
 
-The wheel also contains `tensoracle/cpp/include`, `tensoracle/cpp/src`, and `tensoracle/cpp/cmake`. Pass the installed `tensoracle/cpp` directory to CMake as `TENSORACLE_CPP_ROOT` and include `TensoracleCubinPlugin.cmake`. Its `tensoracle_add_cubin_plugin` function compiles the common C++ source, the caller's plugin source, and the generated `bundle.cpp` into the caller's library. The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common source owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common source releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.
+The caller implements the graph's meaning, field meanings, weight placement, launch sequence, and variant selection. The common C++ implementation owns typed fields in their original order, handles `IPluginV3` lifecycle methods, checks cubin digests and resources, loads one module per CUDA context, checks the generated argument count and specialization rules, appends the two supported zero scratch pointers, and launches kernels. The caller selects a variant using `Kernel::accepts` and derives the count passed to `Kernel::launch` from its argument array. The common implementation releases modules and device allocations in their owning contexts. The library must export TensorRT's `getCreators`; a static registration constructor is not supported.
 
 To record the CUDA inputs used by the plugin build, call `tensoracle_find_cuda` from `TensoracleCuda.cmake` (also included by `TensoracleCubinPlugin.cmake`):
 

tensoracle-1.5.0/docs/cpp-python-repositories.md

@@ -0,0 +1,65 @@
+# Build plugins with tensoracle's C++ sources
+
+Tensoracle supplies the shared C++ implementation and CMake helpers for cubin plugins. Your model package supplies the kernels and model-specific plugin behavior, compiles the native plugin, and packages it for the application.
+
+## C++ sources in the wheel
+
+The tensoracle wheel includes:
+
+```text
+tensoracle/cpp/
+  include/tensoracle/cubin_plugin.hpp
+  src/cubin_plugin.cpp
+  cmake/TensoracleCubinPlugin.cmake
+  cmake/TensoracleCuda.cmake
+```
+
+The wheel contains no native binaries and uses the `py3-none-any` tag. Your plugin's native library is compiled separately and belongs to your model package.
+
+## Prepare the build inputs
+
+Use CMake 3.20 or later and a C++17 compiler. Follow [Prepare a TensorRT SDK](tensorrt-sdk.md) to obtain and verify the TensorRT 11 headers and library and prepare the PyPI CUDA headers and driver link library. That guide covers Linux, WSL2, and [Windows](tensorrt-sdk.md#windows-x86_64); no OS-installed CUDA Toolkit is required.
+
+Generate a cubin bundle with the [cubin producer](api/producer.md). The bundle directory must contain `bundle.cpp`, `bundle.json`, and the cubin files. Cubin production runs on Linux; the same bundle can be used to build a Windows plugin DLL. Follow [Build a cubin plugin](index.md#build-a-cubin-plugin) for the model-specific plugin interface and tensor declarations.
+
+## Use the packaged CMake helper
+
+In the Python environment where tensoracle is installed, locate its C++ source directory:
+
+```bash
+python -c 'from importlib.metadata import distribution; print(distribution("tensoracle").locate_file("tensoracle/cpp"))'
+```
+
+Pass that directory as `TENSORACLE_CPP_ROOT` to your plugin's CMake project. Include the helper and give it your plugin source, verified SDK root, and generated bundle directory:
+
+```cmake
+cmake_minimum_required(VERSION 3.20)
+project(my_model_plugin LANGUAGES CXX)
+
+include("${TENSORACLE_CPP_ROOT}/cmake/TensoracleCubinPlugin.cmake")
+tensoracle_add_cubin_plugin(
+    TARGET my_model_plugin
+    TENSORRT_ROOT "${TENSORRT_ROOT}"
+    BUNDLE_ROOT "${TENSORACLE_BUNDLE_ROOT}"
+    SOURCES "${CMAKE_CURRENT_SOURCE_DIR}/plugin.cpp")
+```
+
+`tensoracle_add_cubin_plugin` builds a shared library from your `plugin.cpp`, tensoracle's common `cubin_plugin.cpp`, and the bundle's generated `bundle.cpp`. It adds the required include directories and links TensorRT and the CUDA driver.
+
+For a Bash build, replace the absolute paths below with your plugin project, installed tensoracle sources, verified SDK, PyPI CUDA root, and generated bundle:
+
+```bash
+cmake -S /absolute/path/to/plugin-project -B build/plugin \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DTENSORACLE_CPP_ROOT=/absolute/path/to/tensoracle/cpp \
+  -DTENSORRT_ROOT=/absolute/path/to/TensorRT-SDK \
+  -DCUDAToolkit_ROOT=/absolute/path/to/site-packages/nvidia/cu13 \
+  -DTENSORACLE_BUNDLE_ROOT=/absolute/path/to/bundle
+cmake --build build/plugin --config Release
+```
+
+The [plugin guide](https://github.com/sync-dev-org/tensoracle#build-a-cubin-plugin) documents the required `getCreators` export and the optional `tensoracle_find_cuda` function and `tensoracle_cuda_driver` target for consumers that also need the resolved CUDA inputs.
+
+## Package and inspect the plugin
+
+Follow the [cubin plugin example](https://github.com/sync-dev-org/tensoracle/blob/main/examples/cubin_plugin/README.md#build-and-inspect-the-wheel) for packaging the native library and bundle in your model wheel and testing it in a runtime environment. Use the [bundle inspection API](api/bundle.md) to check the bundle and its match with the compiled plugin before distribution.

{tensoracle-1.4.0 → tensoracle-1.5.0}/docs/index.md

@@ -4,7 +4,7 @@
 
 Follow the [four-stage optimization workflow](optimization.md) to establish output quality, evaluate lower precision and plugins, and measure speed.
 
-Follow [Build a C++ and Python repository](cpp-python-repositories.md) for source layout, development environments, bindings, and Linux wheel packaging.
+Follow [Build plugins with tensoracle's C++ sources](cpp-python-repositories.md) to locate the sources in the wheel and use the CMake helper in your plugin build.
 
 ## Terms
 
@@ -74,6 +74,30 @@ handle = open_engine(
 
 `my_model` represents the application's model package. On a valid cache hit, `open_engine` does not call `graph`; on a miss, it builds and saves a new engine. Raise `graph_version` when the graph code changes or when any other caller value that changes the engine contents but is not part of the cache identity changes, such as a plugin manifest or a build recipe. `open_engine` itself does not execute the engine or manage GPU buffers. Create an execution unit from its returned handle to enqueue inference. Saved engines live under `XDG_CACHE_HOME/tensoracle`, or `~/.cache/tensoracle` when `XDG_CACHE_HOME` is unset. On Windows, `~` is `%USERPROFILE%`. See the [complete cache example](https://github.com/sync-dev-org/tensoracle#save-and-reopen-an-engine) for the declaration and expected input and output types.
 
+### Manage saved engines
+
+Inspect saved engines and preview or perform cleanup with the installed command:
+
+```bash
+tensoracle cache list
+tensoracle cache prune --unused-for 30 --dry-run
+tensoracle cache prune --unused-for 30
+tensoracle cache prune --max-size 10GiB --dry-run
+tensoracle cache prune --max-size 10GiB
+```
+
+The commands use the same `XDG_CACHE_HOME` / home `.cache` directory as `open_engine`. They read only filesystem metadata, engine headers, and saved weight digest records. They do not read or checksum the plan body, load the original weights, deserialize engines, import TensorRT, or query a GPU. Listing recognizes the header format; it does not establish that an engine can be loaded.
+
+`cache list` prints one JSON object with `directory`, `entries`, and `total_size_bytes`. Each entry contains the file name, size in bytes, recorded time in UTC (`recorded_at`) and exact nanoseconds (`recorded_at_ns`), and the complete header conditions. This distinguishes engines with the same name and graph version but different weights or plugins. `weight_paths` associates each weight digest with paths from valid, checksum-verified `weight-*.json` records whose SHA-256 matches that digest; an empty `paths` array means unknown. These paths are references and do not establish which application uses an engine or whether the weight file still exists. Empty files have status `building or leftover`; unreadable or invalid headers have status `unrecognized` and a reason.
+
+An engine's `.engine` modification time records its initial save or the last successful `open_engine` cache read. Failed loads and these management commands leave that time unchanged. A timestamp write failure, including a read-only cache, does not prevent a successful load, and a disappeared file is never recreated to record its time. The record describes the time of `open_engine`, not whether inference continues afterward. Files saved before timestamp recording was introduced, and files whose timestamp could not be written, may not reflect the actual last successful read.
+
+`cache prune` requires exactly one of `--unused-for DAYS` and `--max-size SIZE`. Days are finite nonnegative numbers, may be fractional, and mean 24 hours each; only timestamps strictly earlier than the cutoff are selected. Sizes accept nonnegative bytes or the case-insensitive units `B`, `KiB`, `MiB`, `GiB`, and `TiB` (powers of 1024). Fractions such as `1.5GiB` are allowed and fractional bytes are rounded down. Size pruning selects the oldest recorded times first, breaking ties by file name, until the total `.engine` bytes would fit. Empty and unrecognized files count toward that total but are always excluded from deletion. Weight digest records are kept.
+
+Before deletion, pruning prints a JSON `plan` containing the selected engines, excluded files, and a notice that removed engines will be rebuilt the next time they are used. It then prints a JSON `result` separating removed, excluded, failed, and already missing files, the bytes actually removed, and the remaining total. `--dry-run` shows the same targets without changing file contents or recorded times; its result also gives the projected total. Exit status is 0 for success, 1 for filesystem failures, failed deletions, or an unattainable size limit, and 2 for invalid usage. Files that disappear before removal are reported in `missing` and do not count as bytes removed. Observed file changes are reported as failed removals with a reason.
+
+Run pruning during maintenance when no engine is being built or opened in the same cache directory. Cleanup provides no lock or guarantee of protection against concurrent builds and reads. Inspect the selection with `--dry-run` before performing cleanup. `tensoracle cache prune --help` describes the options.
+
 ### Execute an engine
 
 The execution flow is: open the engine, create an execution unit, set all ranged input shapes together, read concrete output shapes and dtypes, allocate output buffers, then enqueue the device addresses on a caller-owned stream. Each `EngineExecution` keeps the handle alive and owns an independent TensorRT context. Fixed inputs and single-point profiles are initialized when the unit is created. Ranged inputs retain their shapes until the next successful `set_input_shapes` call. A partial TensorRT setting failure blocks queries for concrete shapes and enqueue until a complete setting succeeds.
@@ -234,7 +258,7 @@ The application runs the engine and passes a CPU NumPy candidate output and a co
 
 ### Build a cubin plugin
 
-Repositories that adopt tensoracle must follow the [repository layout in the plugin guide](https://github.com/sync-dev-org/tensoracle#build-a-cubin-plugin).
+Use the [C++ sources and CMake helper supplied in the tensoracle wheel](cpp-python-repositories.md) to build your native plugin.
 
 Install `tensoracle[producer]` when you need to compile cubins from your own kernels:
 

{tensoracle-1.4.0 → tensoracle-1.5.0}/mkdocs.yml

@@ -35,7 +35,7 @@ nav:
   - Optimization workflow: optimization.md
   - bf16, int8, and fp8 engines: precision.md
   - TensorRT SDK setup: tensorrt-sdk.md
-  - C++ and Python repositories: cpp-python-repositories.md
+  - C++ plugin builds: cpp-python-repositories.md
   - API reference:
       - Engine and comparison: api/index.md
       - Cubin bundle inspection: api/bundle.md

{tensoracle-1.4.0 → tensoracle-1.5.0}/pyproject.toml

@@ -24,6 +24,9 @@ dependencies = [
     "tensorrt-cu13>=11.0,<12",
 ]
 
+[project.scripts]
+tensoracle = "tensoracle._cli:main"
+
 [project.urls]
 documentation = "https://sync-dev-org.github.io/tensoracle/"
 repository = "https://github.com/sync-dev-org/tensoracle"
@@ -63,6 +66,7 @@ exclude = [
     "/.github/",
     "/AGENTS.md",
     "/CLAUDE.md",
+    "/intro/",
     "/tools/",
     "/noren.yaml",
     "/noren-words.txt",

{tensoracle-1.4.0 → tensoracle-1.5.0}/src/tensoracle/_cache.py

@@ -3,7 +3,6 @@
 import ctypes
 import ctypes.util
 import hashlib
-import json
 import os
 import platform
 import tempfile
@@ -16,14 +15,22 @@ from typing import TYPE_CHECKING
 import tensorrt as trt
 
 from ._build import EngineBuild, _build_serialized_engine
+from ._cache_format import (
+    _MAGIC,
+    _WEIGHT_RECORD_VERSION,
+    CorruptEngineError,
+    EngineCacheError,
+    EngineConditionError,
+    _cache_directory,
+    _json_bytes,
+    _read_header,
+    _read_weight_record,
+)
 from ._plugins import register_libraries
 
 if TYPE_CHECKING:
     from ._execution import EngineExecution
 
-_MAGIC = b"TENSORACLE\x01"
-_WEIGHT_RECORD_VERSION = 1
-
 
 @dataclass(frozen=True)
 class TensorSpec:
@@ -59,18 +66,6 @@ class EngineHandle:
         return EngineExecution(self)
 
 
-class EngineCacheError(ValueError):
-    """A stored engine cannot be used for this declaration."""
-
-
-class CorruptEngineError(EngineCacheError):
-    """A stored engine is incomplete or corrupt."""
-
-
-class EngineConditionError(EngineCacheError):
-    """A stored engine belongs to different build conditions."""
-
-
 class EngineIOError(EngineCacheError):
     """The engine's input or output signature differs from the declaration."""
 
@@ -106,7 +101,7 @@ def open_engine(
             fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
         except FileExistsError:
             plan, shape_tensor_outputs = _read_plan(path, conditions)
-            return _open_plan(
+            handle = _open_plan(
                 plan,
                 path,
                 declaration,
@@ -114,6 +109,11 @@ def open_engine(
                 actual_logger,
                 shape_tensor_outputs=shape_tensor_outputs,
             )
+            try:
+                os.utime(path, None)
+            except OSError:
+                pass
+            return handle
         else:
             os.close(fd)
 
@@ -145,10 +145,6 @@ def open_engine(
         raise
 
 
-def _json_bytes(value: object) -> bytes:
-    return json.dumps(value, sort_keys=True, separators=(",", ":")).encode("utf-8")
-
-
 def _file_sha256(path: Path) -> str:
     digest = hashlib.sha256()
     with path.open("rb") as stream:
@@ -157,11 +153,6 @@ def _file_sha256(path: Path) -> str:
     return digest.hexdigest()
 
 
-def _cache_directory() -> Path:
-    cache_root = Path(os.environ.get("XDG_CACHE_HOME") or Path.home() / ".cache")
-    return cache_root / "tensoracle"
-
-
 def _weight_sha256(path: Path, *, force_rebuild: bool) -> str:
     absolute_path = Path(os.path.abspath(path))
     before = absolute_path.stat()
@@ -175,27 +166,8 @@ def _weight_sha256(path: Path, *, force_rebuild: bool) -> str:
     record_path = _cache_directory() / f"weight-{path_digest}.json"
 
     if not force_rebuild:
-        try:
-            record = json.loads(record_path.read_bytes())
-        except (OSError, UnicodeDecodeError, json.JSONDecodeError):
-            record = None
-        if (
-            isinstance(record, dict)
-            and record.keys() == identity.keys() | {"version", "sha256", "checksum"}
-            and record["version"] == _WEIGHT_RECORD_VERSION
-            and type(record["version"]) is int
-            and all(
-                type(record[key]) is type(value) and record[key] == value
-                for key, value in identity.items()
-            )
-            and isinstance(record["sha256"], str)
-            and len(record["sha256"]) == 64
-            and all(character in "0123456789abcdef" for character in record["sha256"])
-            and record["checksum"]
-            == hashlib.sha256(
-                _json_bytes({key: value for key, value in record.items() if key != "checksum"})
-            ).hexdigest()
-        ):
+        record = _read_weight_record(record_path)
+        if record is not None and all(record[key] == value for key, value in identity.items()):
             return record["sha256"]
 
     digest = _file_sha256(absolute_path)
@@ -323,31 +295,18 @@ def _conditions(
 
 
 def _read_plan(path: Path, conditions: dict) -> tuple[memoryview, tuple[str, ...]]:
-    data = path.read_bytes()
-    offset = len(_MAGIC)
-    if not data.startswith(_MAGIC) or len(data) < offset + 8:
-        raise CorruptEngineError(f"incomplete TensorRT engine file: {path}")
-    header_size = int.from_bytes(data[offset : offset + 8], "big")
-    header_end = offset + 8 + header_size
-    if header_size == 0 or header_end >= len(data):
-        raise CorruptEngineError(f"invalid TensorRT engine header: {path}")
-    try:
-        header = json.loads(data[offset + 8 : header_end])
-    except (UnicodeDecodeError, json.JSONDecodeError) as error:
-        raise CorruptEngineError(f"invalid TensorRT engine header: {path}") from error
-    if not isinstance(header, dict) or not isinstance(header.get("conditions"), dict):
-        raise CorruptEngineError(f"invalid TensorRT engine header: {path}")
-    shape_tensor_outputs = header.get("shape_tensor_outputs", [])
-    if not isinstance(shape_tensor_outputs, list) or any(
-        not isinstance(name, str) for name in shape_tensor_outputs
-    ):
-        raise CorruptEngineError(f"invalid TensorRT engine header: {path}")
-    if header["conditions"] != conditions:
-        raise EngineConditionError(f"TensorRT engine conditions differ: {path}")
-    plan = memoryview(data)[header_end:]
+    # A buffered read after the header can concatenate a buffered prefix with a
+    # large plan while holding the GIL. FileIO.readall avoids that concatenation
+    # and still handles plans larger than a single OS read can return.
+    with path.open("rb", buffering=0) as stream:
+        file_size = os.fstat(stream.fileno()).st_size
+        header = _read_header(stream, path, file_size)
+        if header["conditions"] != conditions:
+            raise EngineConditionError(f"TensorRT engine conditions differ: {path}")
+        plan = memoryview(stream.read())
     if header.get("plan_sha256") != hashlib.sha256(plan).hexdigest():
         raise CorruptEngineError(f"TensorRT engine checksum differs: {path}")
-    return plan, tuple(shape_tensor_outputs)
+    return plan, tuple(header.get("shape_tensor_outputs", []))
 
 
 def _open_plan(

tensoracle-1.5.0/src/tensoracle/_cache_format.py

@@ -0,0 +1,78 @@
+"""Shared on-disk cache format and weight records, without TensorRT imports."""
+
+import hashlib
+import json
+import os
+from pathlib import Path
+from typing import BinaryIO
+
+_MAGIC = b"TENSORACLE\x01"
+_WEIGHT_RECORD_VERSION = 1
+
+
+class EngineCacheError(ValueError):
+    """A stored engine cannot be used for this declaration."""
+
+
+class CorruptEngineError(EngineCacheError):
+    """A stored engine is incomplete or corrupt."""
+
+
+class EngineConditionError(EngineCacheError):
+    """A stored engine belongs to different build conditions."""
+
+
+def _json_bytes(value: object) -> bytes:
+    return json.dumps(value, sort_keys=True, separators=(",", ":")).encode("utf-8")
+
+
+def _cache_directory() -> Path:
+    cache_root = Path(os.environ.get("XDG_CACHE_HOME") or Path.home() / ".cache")
+    return cache_root / "tensoracle"
+
+
+def _read_header(stream: BinaryIO, path: Path, file_size: int) -> dict:
+    """Read and validate only the header, leaving the stream at the plan body."""
+    prefix = stream.read(len(_MAGIC) + 8)
+    if not prefix.startswith(_MAGIC) or len(prefix) != len(_MAGIC) + 8:
+        raise CorruptEngineError(f"incomplete TensorRT engine file: {path}")
+    header_size = int.from_bytes(prefix[len(_MAGIC) :], "big")
+    if header_size == 0 or len(prefix) + header_size >= file_size:
+        raise CorruptEngineError(f"invalid TensorRT engine header: {path}")
+    try:
+        header = json.loads(stream.read(header_size))
+    except (UnicodeDecodeError, ValueError, RecursionError) as error:
+        raise CorruptEngineError(f"invalid TensorRT engine header: {path}") from error
+    if not isinstance(header, dict) or not isinstance(header.get("conditions"), dict):
+        raise CorruptEngineError(f"invalid TensorRT engine header: {path}")
+    shape_tensor_outputs = header.get("shape_tensor_outputs", [])
+    if not isinstance(shape_tensor_outputs, list) or any(
+        not isinstance(name, str) for name in shape_tensor_outputs
+    ):
+        raise CorruptEngineError(f"invalid TensorRT engine header: {path}")
+    return header
+
+
+def _read_weight_record(path: Path) -> dict | None:
+    """Validate a digest record without accessing the original weight file."""
+    try:
+        record = json.loads(path.read_bytes())
+    except (OSError, UnicodeDecodeError, ValueError, RecursionError):
+        return None
+    if (
+        not isinstance(record, dict)
+        or record.keys() != {"version", "path", "size", "mtime_ns", "inode", "sha256", "checksum"}
+        or type(record["version"]) is not int
+        or record["version"] != _WEIGHT_RECORD_VERSION
+        or not isinstance(record["path"], str)
+        or any(type(record[key]) is not int for key in ("size", "mtime_ns", "inode"))
+        or not isinstance(record["sha256"], str)
+        or len(record["sha256"]) != 64
+        or any(character not in "0123456789abcdef" for character in record["sha256"])
+        or record["checksum"]
+        != hashlib.sha256(
+            _json_bytes({key: value for key, value in record.items() if key != "checksum"})
+        ).hexdigest()
+    ):
+        return None
+    return record