tensorstudio 1.6.0__tar.gz → 1.8.0__tar.gz

{tensorstudio-1.6.0 → tensorstudio-1.8.0}/CHANGELOG.md

@@ -2,6 +2,44 @@
 
 ## Unreleased
 
+## 1.8.0 - 2026-07-07
+
+- Completed the ordered Autograd Coverage And Hardening roadmap section as one
+  release batch.
+- Added `retain_graph` support to `Tensor.backward()` and
+  `tensorstudio.autograd.backward()`.
+- Added graph lifecycle hardening: normal backward frees non-leaf graph history,
+  repeated backward through a freed graph raises a clear error, and retained
+  graphs clear intermediate gradients between backward passes.
+- Added Tensor `is_leaf`, `clear_history()`, and `detach_()` controls for
+  explicit graph lifecycle management.
+- Added guarded public in-place methods `zero_()`, `fill_()`, and `add_()`.
+  They reject gradient-tracked mutation while grad mode is enabled and work
+  inside `tensorstudio.no_grad()`.
+- Expanded non-scalar backward and finite-difference gradient tests for stable
+  probability ops, statistics, norms, and batched matrix multiplication.
+- Expanded autograd documentation with a coverage matrix, lifecycle notes, and
+  explicit higher-order-gradient limitations.
+
+## 1.7.0 - 2026-07-07
+
+- Completed the ordered Core Math Expansion roadmap section as one release
+  batch.
+- Added native C++ `logsumexp`, `softmax`, and `log_softmax` operations with
+  max-shifted stable numerics and autograd support.
+- Added native C++ batched matrix multiplication through `bmm` and 3D `@`
+  dispatch, including reverse-mode gradients for both operands.
+- Added native C++ `var`, `variance`, `std`, `all`, and `any` operations, plus
+  Tensor methods and top-level Python exports where appropriate.
+- Added Tensor-level `norm()` and expanded `tensorstudio.math` with
+  `logsumexp`, `softmax`, `log_softmax`, boolean reductions, and a documented
+  practical `einsum` subset.
+- Added seeded native random distributions: `uniform`, `normal`, `randint`,
+  and `bernoulli`, with Python `*_like` helpers where useful.
+- Switched neural-network functional softmax/log-softmax and cross entropy to
+  the native stable kernels.
+- Added NumPy parity and autograd tests for the expanded math surface.
+
 ## 1.6.0 - 2026-07-07
 
 - Completed the ordered CPU Performance Core roadmap section as one release

{tensorstudio-1.6.0 → tensorstudio-1.8.0}/CMakeLists.txt

@@ -1,6 +1,6 @@
 cmake_minimum_required(VERSION 3.18)
 
-project(tensorstudio VERSION 1.6.0 LANGUAGES CXX)
+project(tensorstudio VERSION 1.8.0 LANGUAGES CXX)
 
 find_package(Python COMPONENTS Interpreter Development.Module REQUIRED)
 set(PYBIND11_FINDPYTHON ON)

{tensorstudio-1.6.0 → tensorstudio-1.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: tensorstudio
-Version: 1.6.0
+Version: 1.8.0
 Summary: TensorStudio is a compact C++ tensor and autograd engine with a Python API for learning, experimentation, and lightweight ML workloads.
 Keywords: tensor,autograd,machine-learning,cpp,pybind11
 Author: TensorStudio contributors
@@ -71,10 +71,13 @@ Description-Content-Type: text/markdown
 TensorStudio is a compact C++ tensor and autograd engine with a Python API for
 learning, experimentation, and lightweight ML workloads.
 
-TensorStudio `1.6.0` is a CPU-only stable API foundation with native C++
+TensorStudio `1.8.0` is a CPU-only stable API foundation with native C++
 threading, storage reuse, SIMD-friendly typed kernels, and optional
-CBLAS/Accelerate matrix multiplication when available. It is eager-only,
-intentionally compact, and not a replacement for mature ML frameworks.
+CBLAS/Accelerate matrix multiplication when available. It adds native stable
+softmax/logsumexp, batched matrix multiplication, statistical reductions,
+boolean reductions, seeded random distributions, and a hardened eager autograd
+lifecycle. It is eager-only, intentionally compact, and not a replacement for
+mature ML frameworks.
 
 ## Install
 
@@ -165,12 +168,15 @@ a = ts.zeros((2, 3))
 b = ts.rand((2, 3))
 c = ts.eye(3)
 d = ts.linspace(0.0, 1.0, 5)
+labels = ts.randint((4,), low=0, high=3, seed=3)
+mask = ts.bernoulli((2, 3), probability=0.25, seed=5)
 
 print(a.shape, a.strides, a.device, a.is_contiguous)
 print((b.clamp(0.2, 0.8) + 1).mean().item())
 print(b.sum(axis=1).tolist())
 print(ts.concat([b, b], axis=0).shape, b.astype("float64").dtype)
 print(c.tolist(), d.tolist())
+print(labels.tolist(), mask.any(axis=1).tolist())
 print(ts.zeros_like(b).shape, ts.randn_like(b, seed=11).dtype)
 ```
 
@@ -199,14 +205,29 @@ print(loss.item())
 print(x.grad.tolist())
 ```
 
-Higher-level helpers live in `tensorstudio.math`:
+Stable reductions and normalized probabilities are available as Tensor methods,
+functional ops, and `tensorstudio.math` helpers:
 
 ```python
-values = ts.tensor([[1.0, 2.0], [3.0, 4.0]])
+values = ts.tensor([[1000.0, 1001.0, 999.0], [1.0, 2.0, 3.0]])
 
 print(ts.math.variance(values).item())
 print(ts.math.std(values, axis=0).tolist())
 print(ts.math.norm(values, ord=2).item())
+print(values.softmax(axis=1).tolist())
+print(ts.logsumexp(values, axis=1).tolist())
+```
+
+Batched matrix multiplication and a small documented `einsum` subset cover
+common model and scientific-programming patterns:
+
+```python
+left = ts.randn((2, 3, 4), seed=1)
+right = ts.randn((2, 4, 5), seed=2)
+
+print((left @ right).shape)
+print(ts.bmm(left, right).shape)
+print(ts.einsum("bij,bjk->bik", left, right).shape)
 ```
 
 ## Autograd
@@ -221,11 +242,20 @@ loss.backward()
 print(x.grad.tolist())
 ```
 
+Reuse a graph explicitly when needed:
+
+```python
+loss = (x * x).sum()
+loss.backward(retain_graph=True)
+loss.backward()
+```
+
 Use `no_grad()` when you want eager computation without recording a graph:
 
 ```python
 with ts.no_grad():
     y = x * 2
+    x.zero_()
 ```
 
 ## Neural Networks
@@ -361,11 +391,11 @@ Run the loose local regression thresholds with:
 python benchmark_all.py --check-thresholds
 ```
 
-On one Windows CPython 3.10 development run reporting `1.6.0`, with
+On one Windows CPython 3.10 development run reporting `1.8.0`, with
 TensorStudio threads enabled, storage pooling enabled, SSE2 autovectorization
 reported, and no BLAS provider found, TensorStudio beat NumPy on 7 local
 benchmark cases and lost on 96 NumPy-comparable cases. JAX CPU dispatch was
-available on that machine; TensorStudio won 39 local cases and lost 59. The
+available on that machine; TensorStudio won 47 local cases and lost 51. The
 strongest local wins were the simple NumPy convolution/pooling reference loops
 and some small JAX-dispatch-heavy eager cases. NumPy and JAX were faster for
 many elementwise, reduction, matrix multiplication, larger activation, and
@@ -377,15 +407,15 @@ Snapshot from that local run:
 
 | operation | shape | TensorStudio | NumPy | JAX CPU dispatch | TS vs NumPy | TS vs JAX |
 |---|---:|---:|---:|---:|---:|---:|
-| `sigmoid` | `(32,)` | 0.0173 ms | 0.0047 ms | 0.0719 ms | 0.2724x | 4.1608x |
-| `mean` | `(32,)` | 0.0157 ms | 0.0082 ms | 0.0119 ms | 0.5208x | 0.7587x |
-| `sum_axis1` | `(16, 16)` | 0.0159 ms | 0.0031 ms | 0.0131 ms | 0.1924x | 0.8231x |
-| `chain_relu` | `(128,)` | 0.0992 ms | 0.0063 ms | 0.1113 ms | 0.0636x | 1.1219x |
-| `matmul` | `(256, 256)` | 2.5261 ms | 0.3848 ms | 0.2288 ms | 0.1524x | 0.0906x |
-| `conv2d_3x3_padding1` | `(1, 1, 8, 8)` | 0.2120 ms | 1.2975 ms | 0.0959 ms | 6.1191x | 0.4524x |
-| `max_pool2d_2x2` | `(1, 1, 16, 16)` | 0.0286 ms | 0.1613 ms | n/a | 5.6321x | n/a |
-| `avg_pool2d_2x2` | `(1, 1, 16, 16)` | 0.0304 ms | 0.5601 ms | n/a | 18.4457x | n/a |
-| `elementwise_backward` | `(1024,)` | 2.7150 ms | n/a | n/a | n/a | n/a |
+| `sigmoid` | `(32,)` | 0.0155 ms | 0.0044 ms | 0.0761 ms | 0.2860x | 4.9227x |
+| `mean` | `(32,)` | 0.0158 ms | 0.0082 ms | 0.0126 ms | 0.5215x | 0.7968x |
+| `sum_axis1` | `(16, 16)` | 0.0161 ms | 0.0028 ms | 0.0136 ms | 0.1714x | 0.8417x |
+| `chain_relu` | `(128,)` | 0.0861 ms | 0.0056 ms | 0.0944 ms | 0.0646x | 1.0971x |
+| `matmul` | `(256, 256)` | 2.1900 ms | 0.4380 ms | 0.2610 ms | 0.2000x | 0.1192x |
+| `conv2d_3x3_padding1` | `(1, 1, 8, 8)` | 0.1995 ms | 1.2390 ms | 0.1086 ms | 6.2118x | 0.5443x |
+| `max_pool2d_2x2` | `(1, 1, 16, 16)` | 0.0286 ms | 0.1684 ms | n/a | 5.8921x | n/a |
+| `avg_pool2d_2x2` | `(1, 1, 16, 16)` | 0.0286 ms | 0.5569 ms | n/a | 19.4803x | n/a |
+| `elementwise_backward` | `(1024,)` | 2.7758 ms | n/a | n/a | n/a | n/a |
 
 Speedup is `competitor median / TensorStudio median`, so values above `1.0x`
 favor TensorStudio.

{tensorstudio-1.6.0 → tensorstudio-1.8.0}/README.md

@@ -7,10 +7,13 @@
 TensorStudio is a compact C++ tensor and autograd engine with a Python API for
 learning, experimentation, and lightweight ML workloads.
 
-TensorStudio `1.6.0` is a CPU-only stable API foundation with native C++
+TensorStudio `1.8.0` is a CPU-only stable API foundation with native C++
 threading, storage reuse, SIMD-friendly typed kernels, and optional
-CBLAS/Accelerate matrix multiplication when available. It is eager-only,
-intentionally compact, and not a replacement for mature ML frameworks.
+CBLAS/Accelerate matrix multiplication when available. It adds native stable
+softmax/logsumexp, batched matrix multiplication, statistical reductions,
+boolean reductions, seeded random distributions, and a hardened eager autograd
+lifecycle. It is eager-only, intentionally compact, and not a replacement for
+mature ML frameworks.
 
 ## Install
 
@@ -101,12 +104,15 @@ a = ts.zeros((2, 3))
 b = ts.rand((2, 3))
 c = ts.eye(3)
 d = ts.linspace(0.0, 1.0, 5)
+labels = ts.randint((4,), low=0, high=3, seed=3)
+mask = ts.bernoulli((2, 3), probability=0.25, seed=5)
 
 print(a.shape, a.strides, a.device, a.is_contiguous)
 print((b.clamp(0.2, 0.8) + 1).mean().item())
 print(b.sum(axis=1).tolist())
 print(ts.concat([b, b], axis=0).shape, b.astype("float64").dtype)
 print(c.tolist(), d.tolist())
+print(labels.tolist(), mask.any(axis=1).tolist())
 print(ts.zeros_like(b).shape, ts.randn_like(b, seed=11).dtype)
 ```
 
@@ -135,14 +141,29 @@ print(loss.item())
 print(x.grad.tolist())
 ```
 
-Higher-level helpers live in `tensorstudio.math`:
+Stable reductions and normalized probabilities are available as Tensor methods,
+functional ops, and `tensorstudio.math` helpers:
 
 ```python
-values = ts.tensor([[1.0, 2.0], [3.0, 4.0]])
+values = ts.tensor([[1000.0, 1001.0, 999.0], [1.0, 2.0, 3.0]])
 
 print(ts.math.variance(values).item())
 print(ts.math.std(values, axis=0).tolist())
 print(ts.math.norm(values, ord=2).item())
+print(values.softmax(axis=1).tolist())
+print(ts.logsumexp(values, axis=1).tolist())
+```
+
+Batched matrix multiplication and a small documented `einsum` subset cover
+common model and scientific-programming patterns:
+
+```python
+left = ts.randn((2, 3, 4), seed=1)
+right = ts.randn((2, 4, 5), seed=2)
+
+print((left @ right).shape)
+print(ts.bmm(left, right).shape)
+print(ts.einsum("bij,bjk->bik", left, right).shape)
 ```
 
 ## Autograd
@@ -157,11 +178,20 @@ loss.backward()
 print(x.grad.tolist())
 ```
 
+Reuse a graph explicitly when needed:
+
+```python
+loss = (x * x).sum()
+loss.backward(retain_graph=True)
+loss.backward()
+```
+
 Use `no_grad()` when you want eager computation without recording a graph:
 
 ```python
 with ts.no_grad():
     y = x * 2
+    x.zero_()
 ```
 
 ## Neural Networks
@@ -297,11 +327,11 @@ Run the loose local regression thresholds with:
 python benchmark_all.py --check-thresholds
 ```
 
-On one Windows CPython 3.10 development run reporting `1.6.0`, with
+On one Windows CPython 3.10 development run reporting `1.8.0`, with
 TensorStudio threads enabled, storage pooling enabled, SSE2 autovectorization
 reported, and no BLAS provider found, TensorStudio beat NumPy on 7 local
 benchmark cases and lost on 96 NumPy-comparable cases. JAX CPU dispatch was
-available on that machine; TensorStudio won 39 local cases and lost 59. The
+available on that machine; TensorStudio won 47 local cases and lost 51. The
 strongest local wins were the simple NumPy convolution/pooling reference loops
 and some small JAX-dispatch-heavy eager cases. NumPy and JAX were faster for
 many elementwise, reduction, matrix multiplication, larger activation, and
@@ -313,15 +343,15 @@ Snapshot from that local run:
 
 | operation | shape | TensorStudio | NumPy | JAX CPU dispatch | TS vs NumPy | TS vs JAX |
 |---|---:|---:|---:|---:|---:|---:|
-| `sigmoid` | `(32,)` | 0.0173 ms | 0.0047 ms | 0.0719 ms | 0.2724x | 4.1608x |
-| `mean` | `(32,)` | 0.0157 ms | 0.0082 ms | 0.0119 ms | 0.5208x | 0.7587x |
-| `sum_axis1` | `(16, 16)` | 0.0159 ms | 0.0031 ms | 0.0131 ms | 0.1924x | 0.8231x |
-| `chain_relu` | `(128,)` | 0.0992 ms | 0.0063 ms | 0.1113 ms | 0.0636x | 1.1219x |
-| `matmul` | `(256, 256)` | 2.5261 ms | 0.3848 ms | 0.2288 ms | 0.1524x | 0.0906x |
-| `conv2d_3x3_padding1` | `(1, 1, 8, 8)` | 0.2120 ms | 1.2975 ms | 0.0959 ms | 6.1191x | 0.4524x |
-| `max_pool2d_2x2` | `(1, 1, 16, 16)` | 0.0286 ms | 0.1613 ms | n/a | 5.6321x | n/a |
-| `avg_pool2d_2x2` | `(1, 1, 16, 16)` | 0.0304 ms | 0.5601 ms | n/a | 18.4457x | n/a |
-| `elementwise_backward` | `(1024,)` | 2.7150 ms | n/a | n/a | n/a | n/a |
+| `sigmoid` | `(32,)` | 0.0155 ms | 0.0044 ms | 0.0761 ms | 0.2860x | 4.9227x |
+| `mean` | `(32,)` | 0.0158 ms | 0.0082 ms | 0.0126 ms | 0.5215x | 0.7968x |
+| `sum_axis1` | `(16, 16)` | 0.0161 ms | 0.0028 ms | 0.0136 ms | 0.1714x | 0.8417x |
+| `chain_relu` | `(128,)` | 0.0861 ms | 0.0056 ms | 0.0944 ms | 0.0646x | 1.0971x |
+| `matmul` | `(256, 256)` | 2.1900 ms | 0.4380 ms | 0.2610 ms | 0.2000x | 0.1192x |
+| `conv2d_3x3_padding1` | `(1, 1, 8, 8)` | 0.1995 ms | 1.2390 ms | 0.1086 ms | 6.2118x | 0.5443x |
+| `max_pool2d_2x2` | `(1, 1, 16, 16)` | 0.0286 ms | 0.1684 ms | n/a | 5.8921x | n/a |
+| `avg_pool2d_2x2` | `(1, 1, 16, 16)` | 0.0286 ms | 0.5569 ms | n/a | 19.4803x | n/a |
+| `elementwise_backward` | `(1024,)` | 2.7758 ms | n/a | n/a | n/a | n/a |
 
 Speedup is `competitor median / TensorStudio median`, so values above `1.0x`
 favor TensorStudio.