modelstudio 0.1.0__tar.gz → 0.2.0__tar.gz

{modelstudio-0.1.0 → modelstudio-0.2.0}/MANIFEST.in

@@ -1,5 +1,8 @@
 include CMakeLists.txt
+include python/modelstudio/py.typed
 recursive-include csrc *
+recursive-include docs *.md
 recursive-include examples *.py
 recursive-include benchmarks *.py
+recursive-include scripts *.py
 recursive-include tests *.py

modelstudio-0.2.0/PKG-INFO

@@ -0,0 +1,196 @@
+Metadata-Version: 2.4
+Name: modelstudio
+Version: 0.2.0
+Summary: An early-stage AI tensor framework with CPU tensors, autograd, and backend extension scaffolding.
+Author: ModelStudio Contributors
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/imattas/modelstudio
+Project-URL: Repository, https://github.com/imattas/modelstudio
+Project-URL: Issues, https://github.com/imattas/modelstudio/issues
+Keywords: ai,autograd,deep-learning,neural-networks,tensor
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.26
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: twine>=5; extra == "dev"
+Dynamic: license-file
+
+# ModelStudio
+
+ModelStudio is an early-stage AI tensor framework. Version `0.2.0` provides a
+CPU tensor/autograd MVP with neural-network modules, optimizers, serialization,
+basic data loading, and small LLM-oriented building blocks.
+
+It is not a PyTorch or TensorFlow replacement. CPU is the only working backend.
+CUDA, ROCm, and oneAPI remain explicit scaffolds until real kernels are built
+and tested.
+
+## Installation
+
+From PyPI:
+
+```bash
+python -m pip install modelstudio
+```
+
+For development:
+
+```bash
+python -m pip install -e ".[dev]"
+```
+
+## Feature Table
+
+| Area | Status |
+| --- | --- |
+| CPU tensors | Working MVP |
+| Autograd | Reverse-mode for core CPU ops |
+| Reductions | `sum`, `mean`, `max` with axis and keepdims; `max` is value-only |
+| Activations | ReLU, GELU, exp, log, tanh, sigmoid, SiLU, softmax, log-softmax |
+| Losses | MSE and cross entropy |
+| Modules | Parameters, buffers, state dicts, save/load |
+| Layers | Linear, Embedding, LayerNorm, RMSNorm, TransformerBlock |
+| Data | Dataset, TensorDataset, DataLoader |
+| Compiler | Placeholder IR and passes |
+
+## Backend Status
+
+| Backend | Status |
+| --- | --- |
+| CPU | working MVP |
+| CUDA | scaffold only |
+| ROCm | scaffold only |
+| oneAPI | scaffold only |
+
+Unsupported accelerator devices fail with `ModelStudioBackendUnavailable`.
+
+## Tensor Example
+
+```python
+import modelstudio as ms
+
+x = ms.randn((32, 784), requires_grad=True)
+w = ms.randn((784, 10), requires_grad=True)
+loss = (x @ w).mean()
+loss.backward()
+print(w.grad)
+```
+
+## MLP Example
+
+```python
+import modelstudio as ms
+from modelstudio import nn
+
+
+class MLP(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.fc1 = nn.Linear(784, 256)
+        self.fc2 = nn.Linear(256, 10)
+
+    def forward(self, x):
+        return self.fc2(ms.gelu(self.fc1(x)))
+
+
+model = MLP()
+optimizer = ms.optim.AdamW(model.parameters(), lr=3e-4)
+x = ms.randn((16, 784))
+target = ms.randn((16, 10))
+loss = ms.mse_loss(model(x), target)
+optimizer.zero_grad()
+loss.backward()
+optimizer.step()
+```
+
+## State Dict and Save/Load
+
+```python
+model = nn.Linear(4, 2)
+ms.save(model.state_dict(), "model.ms")
+state = ms.load("model.ms")
+model.load_state_dict(state)
+```
+
+## DataLoader
+
+```python
+from modelstudio import data
+
+dataset = data.TensorDataset(ms.randn((8, 4)), ms.arange(8))
+loader = data.DataLoader(dataset, batch_size=2, shuffle=False)
+for xb, yb in loader:
+    print(xb.shape, yb.shape)
+```
+
+## Embedding
+
+```python
+emb = nn.Embedding(num_embeddings=100, embedding_dim=32)
+tokens = ms.tensor([[1, 2, 3]], dtype=ms.int64)
+print(emb(tokens).shape)
+```
+
+## Cross Entropy
+
+```python
+logits = ms.randn((4, 10), requires_grad=True)
+targets = ms.tensor([1, 2, 3, 4], dtype=ms.int64)
+loss = ms.cross_entropy(logits, targets)
+loss.backward()
+```
+
+## TransformerBlock
+
+```python
+block = nn.TransformerBlock(embed_dim=16, num_heads=4)
+x = ms.randn((2, 8, 16), requires_grad=True)
+y = block(x)
+print(y.shape)
+```
+
+## Commands
+
+```bash
+python -m pytest
+python scripts/smoke_test.py
+python examples/train_mlp.py
+python examples/train_classifier.py
+python examples/tiny_transformer.py
+python examples/save_load.py
+python benchmarks/bench_matmul.py
+python benchmarks/bench_mlp.py
+python benchmarks/bench_attention.py
+python benchmarks/bench_dataloader.py
+```
+
+## Documentation
+
+- [Tensor API](docs/tensor-api.md)
+- [Neural network API](docs/nn.md)
+- [Data utilities](docs/data.md)
+- [Backend architecture](docs/backend-architecture.md)
+- [Autograd design](docs/autograd.md)
+- [Releasing](docs/releasing.md)
+- [Contributing](CONTRIBUTING.md)
+
+## Roadmap
+
+- Expand tensor and autograd coverage.
+- Wire native CPU kernels into Python bindings.
+- Add tested CUDA, ROCm, and oneAPI packages when hardware-backed CI exists.
+- Improve compiler graph capture and lowering.

modelstudio-0.2.0/README.md

@@ -0,0 +1,165 @@
+# ModelStudio
+
+ModelStudio is an early-stage AI tensor framework. Version `0.2.0` provides a
+CPU tensor/autograd MVP with neural-network modules, optimizers, serialization,
+basic data loading, and small LLM-oriented building blocks.
+
+It is not a PyTorch or TensorFlow replacement. CPU is the only working backend.
+CUDA, ROCm, and oneAPI remain explicit scaffolds until real kernels are built
+and tested.
+
+## Installation
+
+From PyPI:
+
+```bash
+python -m pip install modelstudio
+```
+
+For development:
+
+```bash
+python -m pip install -e ".[dev]"
+```
+
+## Feature Table
+
+| Area | Status |
+| --- | --- |
+| CPU tensors | Working MVP |
+| Autograd | Reverse-mode for core CPU ops |
+| Reductions | `sum`, `mean`, `max` with axis and keepdims; `max` is value-only |
+| Activations | ReLU, GELU, exp, log, tanh, sigmoid, SiLU, softmax, log-softmax |
+| Losses | MSE and cross entropy |
+| Modules | Parameters, buffers, state dicts, save/load |
+| Layers | Linear, Embedding, LayerNorm, RMSNorm, TransformerBlock |
+| Data | Dataset, TensorDataset, DataLoader |
+| Compiler | Placeholder IR and passes |
+
+## Backend Status
+
+| Backend | Status |
+| --- | --- |
+| CPU | working MVP |
+| CUDA | scaffold only |
+| ROCm | scaffold only |
+| oneAPI | scaffold only |
+
+Unsupported accelerator devices fail with `ModelStudioBackendUnavailable`.
+
+## Tensor Example
+
+```python
+import modelstudio as ms
+
+x = ms.randn((32, 784), requires_grad=True)
+w = ms.randn((784, 10), requires_grad=True)
+loss = (x @ w).mean()
+loss.backward()
+print(w.grad)
+```
+
+## MLP Example
+
+```python
+import modelstudio as ms
+from modelstudio import nn
+
+
+class MLP(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.fc1 = nn.Linear(784, 256)
+        self.fc2 = nn.Linear(256, 10)
+
+    def forward(self, x):
+        return self.fc2(ms.gelu(self.fc1(x)))
+
+
+model = MLP()
+optimizer = ms.optim.AdamW(model.parameters(), lr=3e-4)
+x = ms.randn((16, 784))
+target = ms.randn((16, 10))
+loss = ms.mse_loss(model(x), target)
+optimizer.zero_grad()
+loss.backward()
+optimizer.step()
+```
+
+## State Dict and Save/Load
+
+```python
+model = nn.Linear(4, 2)
+ms.save(model.state_dict(), "model.ms")
+state = ms.load("model.ms")
+model.load_state_dict(state)
+```
+
+## DataLoader
+
+```python
+from modelstudio import data
+
+dataset = data.TensorDataset(ms.randn((8, 4)), ms.arange(8))
+loader = data.DataLoader(dataset, batch_size=2, shuffle=False)
+for xb, yb in loader:
+    print(xb.shape, yb.shape)
+```
+
+## Embedding
+
+```python
+emb = nn.Embedding(num_embeddings=100, embedding_dim=32)
+tokens = ms.tensor([[1, 2, 3]], dtype=ms.int64)
+print(emb(tokens).shape)
+```
+
+## Cross Entropy
+
+```python
+logits = ms.randn((4, 10), requires_grad=True)
+targets = ms.tensor([1, 2, 3, 4], dtype=ms.int64)
+loss = ms.cross_entropy(logits, targets)
+loss.backward()
+```
+
+## TransformerBlock
+
+```python
+block = nn.TransformerBlock(embed_dim=16, num_heads=4)
+x = ms.randn((2, 8, 16), requires_grad=True)
+y = block(x)
+print(y.shape)
+```
+
+## Commands
+
+```bash
+python -m pytest
+python scripts/smoke_test.py
+python examples/train_mlp.py
+python examples/train_classifier.py
+python examples/tiny_transformer.py
+python examples/save_load.py
+python benchmarks/bench_matmul.py
+python benchmarks/bench_mlp.py
+python benchmarks/bench_attention.py
+python benchmarks/bench_dataloader.py
+```
+
+## Documentation
+
+- [Tensor API](docs/tensor-api.md)
+- [Neural network API](docs/nn.md)
+- [Data utilities](docs/data.md)
+- [Backend architecture](docs/backend-architecture.md)
+- [Autograd design](docs/autograd.md)
+- [Releasing](docs/releasing.md)
+- [Contributing](CONTRIBUTING.md)
+
+## Roadmap
+
+- Expand tensor and autograd coverage.
+- Wire native CPU kernels into Python bindings.
+- Add tested CUDA, ROCm, and oneAPI packages when hardware-backed CI exists.
+- Improve compiler graph capture and lowering.

modelstudio-0.2.0/benchmarks/bench_attention.py

@@ -0,0 +1,39 @@
+from __future__ import annotations
+
+import platform
+import time
+
+import modelstudio as ms
+import numpy as np
+
+
+def timeit(fn, iterations: int = 20, warmup: int = 5) -> float:
+    for _ in range(warmup):
+        fn()
+    start = time.perf_counter()
+    for _ in range(iterations):
+        fn()
+    return (time.perf_counter() - start) / iterations
+
+
+def main() -> None:
+    shape = (2, 4, 16, 8)
+    iterations = 20
+    warmup = 5
+    q = ms.randn(shape)
+    k = ms.randn(shape)
+    v = ms.randn(shape)
+
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print(f"Shape:       {shape}")
+    print(f"Warmup:      {warmup}")
+    print(f"Iterations:  {iterations}")
+    print("Backend:     CPU MVP")
+    elapsed = timeit(lambda: ms.scaled_dot_product_attention(q, k, v, causal=True), iterations, warmup)
+    print(f"Attention avg: {elapsed * 1_000:.3f} ms")
+
+
+if __name__ == "__main__":
+    main()

modelstudio-0.2.0/benchmarks/bench_dataloader.py

@@ -0,0 +1,46 @@
+from __future__ import annotations
+
+import platform
+import time
+
+import modelstudio as ms
+import numpy as np
+from modelstudio import data
+
+
+def main() -> None:
+    samples = 1024
+    features = 32
+    batch_size = 64
+    iterations = 20
+    warmup = 5
+    dataset = data.TensorDataset(ms.randn((samples, features)), ms.arange(samples))
+    loader = data.DataLoader(dataset, batch_size=batch_size, shuffle=False)
+
+    def consume() -> int:
+        count = 0
+        for xb, _ in loader:
+            count += xb.shape[0]
+        return count
+
+    for _ in range(warmup):
+        consume()
+    start = time.perf_counter()
+    for _ in range(iterations):
+        consume()
+    elapsed = (time.perf_counter() - start) / iterations
+
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print(f"Samples:     {samples}")
+    print(f"Features:    {features}")
+    print(f"Batch size:  {batch_size}")
+    print(f"Warmup:      {warmup}")
+    print(f"Iterations:  {iterations}")
+    print("Backend:     CPU MVP")
+    print(f"DataLoader avg: {elapsed * 1_000:.3f} ms")
+
+
+if __name__ == "__main__":
+    main()

{modelstudio-0.1.0 → modelstudio-0.2.0}/benchmarks/bench_matmul.py

@@ -1,12 +1,15 @@
 from __future__ import annotations
 
+import platform
 import time
 
 import modelstudio as ms
 import numpy as np
 
 
-def timeit(fn, iterations: int = 20) -> float:
+def timeit(fn, iterations: int = 30, warmup: int = 5) -> float:
+    for _ in range(warmup):
+        fn()
     start = time.perf_counter()
     for _ in range(iterations):
         fn()
@@ -20,11 +23,17 @@ def main() -> None:
     a_ms = ms.tensor(a_np)
     b_ms = ms.tensor(b_np)
 
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print(f"Shape:       {shape} x {shape}")
+    print("Backend:     CPU MVP")
+
     numpy_time = timeit(lambda: a_np @ b_np)
     modelstudio_time = timeit(lambda: a_ms @ b_ms)
 
-    print(f"NumPy matmul:       {numpy_time * 1_000:.3f} ms")
-    print(f"ModelStudio CPU:   {modelstudio_time * 1_000:.3f} ms")
+    print(f"NumPy matmul avg:       {numpy_time * 1_000:.3f} ms")
+    print(f"ModelStudio CPU avg:    {modelstudio_time * 1_000:.3f} ms")
 
 
 if __name__ == "__main__":

{modelstudio-0.1.0 → modelstudio-0.2.0}/benchmarks/bench_mlp.py

@@ -1,8 +1,10 @@
 from __future__ import annotations
 
+import platform
 import time
 
 import modelstudio as ms
+import numpy as np
 from modelstudio import nn
 
 
@@ -21,11 +23,19 @@ def main() -> None:
     x = ms.randn((64, 128), requires_grad=True)
     target = ms.randn((64, 64))
 
-    # Warm up dynamic dispatch and Python imports.
-    loss = ms.mse_loss(model(x), target)
-    loss.backward()
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print("Backend:     CPU MVP")
 
     iterations = 20
+    warmup = 5
+    for _ in range(warmup):
+        model.zero_grad()
+        x.zero_grad()
+        loss = ms.mse_loss(model(x), target)
+        loss.backward()
+
     start = time.perf_counter()
     for _ in range(iterations):
         model.zero_grad()
@@ -33,7 +43,7 @@ def main() -> None:
         loss = ms.mse_loss(model(x), target)
         loss.backward()
     elapsed = (time.perf_counter() - start) / iterations
-    print(f"ModelStudio MLP forward/backward: {elapsed * 1_000:.3f} ms")
+    print(f"ModelStudio MLP forward/backward avg: {elapsed * 1_000:.3f} ms")
 
 
 if __name__ == "__main__":

{modelstudio-0.1.0 → modelstudio-0.2.0}/csrc/backends/cuda/cuda_backend.cu

@@ -5,14 +5,17 @@
 namespace modelstudio::cuda {
 
 Tensor CUDABackend::empty(const Shape&, DType) {
+  // TODO: Implement CUDA allocation and register this backend only when built and tested.
   throw Error("CUDA backend scaffold is present, but CUDA kernels are not implemented");
 }
 
 Tensor CUDABackend::add(const Tensor&, const Tensor&) {
+  // TODO: Add a tested CUDA elementwise add kernel.
   throw Error("CUDA add kernel is not implemented");
 }
 
 Tensor CUDABackend::matmul(const Tensor&, const Tensor&) {
+  // TODO: Add a tested CUDA matmul kernel or cuBLAS integration.
   throw Error("CUDA matmul kernel is not implemented");
 }
 

{modelstudio-0.1.0 → modelstudio-0.2.0}/csrc/backends/oneapi/oneapi_backend.cpp

@@ -5,14 +5,17 @@
 namespace modelstudio::oneapi {
 
 Tensor OneAPIBackend::empty(const Shape&, DType) {
+  // TODO: Implement SYCL allocation and register this backend only when built and tested.
   throw Error("oneAPI backend scaffold is present, but SYCL kernels are not implemented");
 }
 
 Tensor OneAPIBackend::add(const Tensor&, const Tensor&) {
+  // TODO: Add a tested SYCL elementwise add kernel.
   throw Error("oneAPI add kernel is not implemented");
 }
 
 Tensor OneAPIBackend::matmul(const Tensor&, const Tensor&) {
+  // TODO: Add a tested SYCL matmul kernel or oneMKL integration.
   throw Error("oneAPI matmul kernel is not implemented");
 }
 

{modelstudio-0.1.0 → modelstudio-0.2.0}/csrc/backends/rocm/rocm_backend.cpp

@@ -5,14 +5,17 @@
 namespace modelstudio::rocm {
 
 Tensor ROCmBackend::empty(const Shape&, DType) {
+  // TODO: Implement HIP allocation and register this backend only when built and tested.
   throw Error("ROCm backend scaffold is present, but HIP kernels are not implemented");
 }
 
 Tensor ROCmBackend::add(const Tensor&, const Tensor&) {
+  // TODO: Add a tested HIP elementwise add kernel.
   throw Error("ROCm add kernel is not implemented");
 }
 
 Tensor ROCmBackend::matmul(const Tensor&, const Tensor&) {
+  // TODO: Add a tested HIP matmul kernel or rocBLAS integration.
   throw Error("ROCm matmul kernel is not implemented");
 }
 

modelstudio-0.2.0/docs/autograd.md

@@ -0,0 +1,55 @@
+# Autograd
+
+ModelStudio implements a small reverse-mode autograd engine for CPU tensors.
+
+## Graph Construction
+
+When gradient mode is enabled, tensor operations create result tensors with:
+
+- parent tensor references
+- a `grad_fn` label
+- a backward closure that receives the upstream gradient
+
+Operations skip graph construction if no input requires gradients or if the code
+is running under `ms.no_grad()`.
+
+## Backward Pass
+
+`Tensor.backward()` performs a topological sort from the output tensor, seeds the
+output gradient, then walks the graph in reverse topological order. Each backward
+closure accumulates gradients into parent tensors.
+
+Scalar outputs default to a gradient of one. Non-scalar outputs require an
+explicit gradient.
+
+## Gradient Accumulation
+
+Gradients accumulate by default, matching the common neural-network training
+workflow. Call `zero_grad()` on tensors, modules, or optimizers before a new
+training step when accumulation is not desired.
+
+## Broadcasting
+
+Forward broadcasting can expand dimensions. During backward, ModelStudio reduces
+the upstream gradient back to each input's original shape with `unbroadcast`.
+This handles extra leading dimensions and dimensions of size one.
+
+## `no_grad`
+
+`ms.no_grad()` temporarily disables graph recording:
+
+```python
+with ms.no_grad():
+    y = x * 2
+```
+
+This is used by optimizers and inference code.
+
+## Limitations
+
+- CPU tensors only
+- Dynamic graph only
+- No higher-order gradients
+- No in-place autograd correctness checks
+- Limited operation coverage
+- No graph pruning beyond Python reference lifetimes