modelstudio 0.1.1__tar.gz → 0.2.0__tar.gz

{modelstudio-0.1.1 → modelstudio-0.2.0}/MANIFEST.in

@@ -1,6 +1,8 @@
 include CMakeLists.txt
 include python/modelstudio/py.typed
 recursive-include csrc *
+recursive-include docs *.md
 recursive-include examples *.py
 recursive-include benchmarks *.py
+recursive-include scripts *.py
 recursive-include tests *.py

modelstudio-0.2.0/PKG-INFO

@@ -0,0 +1,196 @@
+Metadata-Version: 2.4
+Name: modelstudio
+Version: 0.2.0
+Summary: An early-stage AI tensor framework with CPU tensors, autograd, and backend extension scaffolding.
+Author: ModelStudio Contributors
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/imattas/modelstudio
+Project-URL: Repository, https://github.com/imattas/modelstudio
+Project-URL: Issues, https://github.com/imattas/modelstudio/issues
+Keywords: ai,autograd,deep-learning,neural-networks,tensor
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.26
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: twine>=5; extra == "dev"
+Dynamic: license-file
+
+# ModelStudio
+
+ModelStudio is an early-stage AI tensor framework. Version `0.2.0` provides a
+CPU tensor/autograd MVP with neural-network modules, optimizers, serialization,
+basic data loading, and small LLM-oriented building blocks.
+
+It is not a PyTorch or TensorFlow replacement. CPU is the only working backend.
+CUDA, ROCm, and oneAPI remain explicit scaffolds until real kernels are built
+and tested.
+
+## Installation
+
+From PyPI:
+
+```bash
+python -m pip install modelstudio
+```
+
+For development:
+
+```bash
+python -m pip install -e ".[dev]"
+```
+
+## Feature Table
+
+| Area | Status |
+| --- | --- |
+| CPU tensors | Working MVP |
+| Autograd | Reverse-mode for core CPU ops |
+| Reductions | `sum`, `mean`, `max` with axis and keepdims; `max` is value-only |
+| Activations | ReLU, GELU, exp, log, tanh, sigmoid, SiLU, softmax, log-softmax |
+| Losses | MSE and cross entropy |
+| Modules | Parameters, buffers, state dicts, save/load |
+| Layers | Linear, Embedding, LayerNorm, RMSNorm, TransformerBlock |
+| Data | Dataset, TensorDataset, DataLoader |
+| Compiler | Placeholder IR and passes |
+
+## Backend Status
+
+| Backend | Status |
+| --- | --- |
+| CPU | working MVP |
+| CUDA | scaffold only |
+| ROCm | scaffold only |
+| oneAPI | scaffold only |
+
+Unsupported accelerator devices fail with `ModelStudioBackendUnavailable`.
+
+## Tensor Example
+
+```python
+import modelstudio as ms
+
+x = ms.randn((32, 784), requires_grad=True)
+w = ms.randn((784, 10), requires_grad=True)
+loss = (x @ w).mean()
+loss.backward()
+print(w.grad)
+```
+
+## MLP Example
+
+```python
+import modelstudio as ms
+from modelstudio import nn
+
+
+class MLP(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.fc1 = nn.Linear(784, 256)
+        self.fc2 = nn.Linear(256, 10)
+
+    def forward(self, x):
+        return self.fc2(ms.gelu(self.fc1(x)))
+
+
+model = MLP()
+optimizer = ms.optim.AdamW(model.parameters(), lr=3e-4)
+x = ms.randn((16, 784))
+target = ms.randn((16, 10))
+loss = ms.mse_loss(model(x), target)
+optimizer.zero_grad()
+loss.backward()
+optimizer.step()
+```
+
+## State Dict and Save/Load
+
+```python
+model = nn.Linear(4, 2)
+ms.save(model.state_dict(), "model.ms")
+state = ms.load("model.ms")
+model.load_state_dict(state)
+```
+
+## DataLoader
+
+```python
+from modelstudio import data
+
+dataset = data.TensorDataset(ms.randn((8, 4)), ms.arange(8))
+loader = data.DataLoader(dataset, batch_size=2, shuffle=False)
+for xb, yb in loader:
+    print(xb.shape, yb.shape)
+```
+
+## Embedding
+
+```python
+emb = nn.Embedding(num_embeddings=100, embedding_dim=32)
+tokens = ms.tensor([[1, 2, 3]], dtype=ms.int64)
+print(emb(tokens).shape)
+```
+
+## Cross Entropy
+
+```python
+logits = ms.randn((4, 10), requires_grad=True)
+targets = ms.tensor([1, 2, 3, 4], dtype=ms.int64)
+loss = ms.cross_entropy(logits, targets)
+loss.backward()
+```
+
+## TransformerBlock
+
+```python
+block = nn.TransformerBlock(embed_dim=16, num_heads=4)
+x = ms.randn((2, 8, 16), requires_grad=True)
+y = block(x)
+print(y.shape)
+```
+
+## Commands
+
+```bash
+python -m pytest
+python scripts/smoke_test.py
+python examples/train_mlp.py
+python examples/train_classifier.py
+python examples/tiny_transformer.py
+python examples/save_load.py
+python benchmarks/bench_matmul.py
+python benchmarks/bench_mlp.py
+python benchmarks/bench_attention.py
+python benchmarks/bench_dataloader.py
+```
+
+## Documentation
+
+- [Tensor API](docs/tensor-api.md)
+- [Neural network API](docs/nn.md)
+- [Data utilities](docs/data.md)
+- [Backend architecture](docs/backend-architecture.md)
+- [Autograd design](docs/autograd.md)
+- [Releasing](docs/releasing.md)
+- [Contributing](CONTRIBUTING.md)
+
+## Roadmap
+
+- Expand tensor and autograd coverage.
+- Wire native CPU kernels into Python bindings.
+- Add tested CUDA, ROCm, and oneAPI packages when hardware-backed CI exists.
+- Improve compiler graph capture and lowering.

modelstudio-0.2.0/README.md

@@ -0,0 +1,165 @@
+# ModelStudio
+
+ModelStudio is an early-stage AI tensor framework. Version `0.2.0` provides a
+CPU tensor/autograd MVP with neural-network modules, optimizers, serialization,
+basic data loading, and small LLM-oriented building blocks.
+
+It is not a PyTorch or TensorFlow replacement. CPU is the only working backend.
+CUDA, ROCm, and oneAPI remain explicit scaffolds until real kernels are built
+and tested.
+
+## Installation
+
+From PyPI:
+
+```bash
+python -m pip install modelstudio
+```
+
+For development:
+
+```bash
+python -m pip install -e ".[dev]"
+```
+
+## Feature Table
+
+| Area | Status |
+| --- | --- |
+| CPU tensors | Working MVP |
+| Autograd | Reverse-mode for core CPU ops |
+| Reductions | `sum`, `mean`, `max` with axis and keepdims; `max` is value-only |
+| Activations | ReLU, GELU, exp, log, tanh, sigmoid, SiLU, softmax, log-softmax |
+| Losses | MSE and cross entropy |
+| Modules | Parameters, buffers, state dicts, save/load |
+| Layers | Linear, Embedding, LayerNorm, RMSNorm, TransformerBlock |
+| Data | Dataset, TensorDataset, DataLoader |
+| Compiler | Placeholder IR and passes |
+
+## Backend Status
+
+| Backend | Status |
+| --- | --- |
+| CPU | working MVP |
+| CUDA | scaffold only |
+| ROCm | scaffold only |
+| oneAPI | scaffold only |
+
+Unsupported accelerator devices fail with `ModelStudioBackendUnavailable`.
+
+## Tensor Example
+
+```python
+import modelstudio as ms
+
+x = ms.randn((32, 784), requires_grad=True)
+w = ms.randn((784, 10), requires_grad=True)
+loss = (x @ w).mean()
+loss.backward()
+print(w.grad)
+```
+
+## MLP Example
+
+```python
+import modelstudio as ms
+from modelstudio import nn
+
+
+class MLP(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.fc1 = nn.Linear(784, 256)
+        self.fc2 = nn.Linear(256, 10)
+
+    def forward(self, x):
+        return self.fc2(ms.gelu(self.fc1(x)))
+
+
+model = MLP()
+optimizer = ms.optim.AdamW(model.parameters(), lr=3e-4)
+x = ms.randn((16, 784))
+target = ms.randn((16, 10))
+loss = ms.mse_loss(model(x), target)
+optimizer.zero_grad()
+loss.backward()
+optimizer.step()
+```
+
+## State Dict and Save/Load
+
+```python
+model = nn.Linear(4, 2)
+ms.save(model.state_dict(), "model.ms")
+state = ms.load("model.ms")
+model.load_state_dict(state)
+```
+
+## DataLoader
+
+```python
+from modelstudio import data
+
+dataset = data.TensorDataset(ms.randn((8, 4)), ms.arange(8))
+loader = data.DataLoader(dataset, batch_size=2, shuffle=False)
+for xb, yb in loader:
+    print(xb.shape, yb.shape)
+```
+
+## Embedding
+
+```python
+emb = nn.Embedding(num_embeddings=100, embedding_dim=32)
+tokens = ms.tensor([[1, 2, 3]], dtype=ms.int64)
+print(emb(tokens).shape)
+```
+
+## Cross Entropy
+
+```python
+logits = ms.randn((4, 10), requires_grad=True)
+targets = ms.tensor([1, 2, 3, 4], dtype=ms.int64)
+loss = ms.cross_entropy(logits, targets)
+loss.backward()
+```
+
+## TransformerBlock
+
+```python
+block = nn.TransformerBlock(embed_dim=16, num_heads=4)
+x = ms.randn((2, 8, 16), requires_grad=True)
+y = block(x)
+print(y.shape)
+```
+
+## Commands
+
+```bash
+python -m pytest
+python scripts/smoke_test.py
+python examples/train_mlp.py
+python examples/train_classifier.py
+python examples/tiny_transformer.py
+python examples/save_load.py
+python benchmarks/bench_matmul.py
+python benchmarks/bench_mlp.py
+python benchmarks/bench_attention.py
+python benchmarks/bench_dataloader.py
+```
+
+## Documentation
+
+- [Tensor API](docs/tensor-api.md)
+- [Neural network API](docs/nn.md)
+- [Data utilities](docs/data.md)
+- [Backend architecture](docs/backend-architecture.md)
+- [Autograd design](docs/autograd.md)
+- [Releasing](docs/releasing.md)
+- [Contributing](CONTRIBUTING.md)
+
+## Roadmap
+
+- Expand tensor and autograd coverage.
+- Wire native CPU kernels into Python bindings.
+- Add tested CUDA, ROCm, and oneAPI packages when hardware-backed CI exists.
+- Improve compiler graph capture and lowering.

modelstudio-0.2.0/benchmarks/bench_attention.py

@@ -0,0 +1,39 @@
+from __future__ import annotations
+
+import platform
+import time
+
+import modelstudio as ms
+import numpy as np
+
+
+def timeit(fn, iterations: int = 20, warmup: int = 5) -> float:
+    for _ in range(warmup):
+        fn()
+    start = time.perf_counter()
+    for _ in range(iterations):
+        fn()
+    return (time.perf_counter() - start) / iterations
+
+
+def main() -> None:
+    shape = (2, 4, 16, 8)
+    iterations = 20
+    warmup = 5
+    q = ms.randn(shape)
+    k = ms.randn(shape)
+    v = ms.randn(shape)
+
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print(f"Shape:       {shape}")
+    print(f"Warmup:      {warmup}")
+    print(f"Iterations:  {iterations}")
+    print("Backend:     CPU MVP")
+    elapsed = timeit(lambda: ms.scaled_dot_product_attention(q, k, v, causal=True), iterations, warmup)
+    print(f"Attention avg: {elapsed * 1_000:.3f} ms")
+
+
+if __name__ == "__main__":
+    main()

modelstudio-0.2.0/benchmarks/bench_dataloader.py

@@ -0,0 +1,46 @@
+from __future__ import annotations
+
+import platform
+import time
+
+import modelstudio as ms
+import numpy as np
+from modelstudio import data
+
+
+def main() -> None:
+    samples = 1024
+    features = 32
+    batch_size = 64
+    iterations = 20
+    warmup = 5
+    dataset = data.TensorDataset(ms.randn((samples, features)), ms.arange(samples))
+    loader = data.DataLoader(dataset, batch_size=batch_size, shuffle=False)
+
+    def consume() -> int:
+        count = 0
+        for xb, _ in loader:
+            count += xb.shape[0]
+        return count
+
+    for _ in range(warmup):
+        consume()
+    start = time.perf_counter()
+    for _ in range(iterations):
+        consume()
+    elapsed = (time.perf_counter() - start) / iterations
+
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print(f"Samples:     {samples}")
+    print(f"Features:    {features}")
+    print(f"Batch size:  {batch_size}")
+    print(f"Warmup:      {warmup}")
+    print(f"Iterations:  {iterations}")
+    print("Backend:     CPU MVP")
+    print(f"DataLoader avg: {elapsed * 1_000:.3f} ms")
+
+
+if __name__ == "__main__":
+    main()

modelstudio-0.2.0/docs/autograd.md

@@ -0,0 +1,55 @@
+# Autograd
+
+ModelStudio implements a small reverse-mode autograd engine for CPU tensors.
+
+## Graph Construction
+
+When gradient mode is enabled, tensor operations create result tensors with:
+
+- parent tensor references
+- a `grad_fn` label
+- a backward closure that receives the upstream gradient
+
+Operations skip graph construction if no input requires gradients or if the code
+is running under `ms.no_grad()`.
+
+## Backward Pass
+
+`Tensor.backward()` performs a topological sort from the output tensor, seeds the
+output gradient, then walks the graph in reverse topological order. Each backward
+closure accumulates gradients into parent tensors.
+
+Scalar outputs default to a gradient of one. Non-scalar outputs require an
+explicit gradient.
+
+## Gradient Accumulation
+
+Gradients accumulate by default, matching the common neural-network training
+workflow. Call `zero_grad()` on tensors, modules, or optimizers before a new
+training step when accumulation is not desired.
+
+## Broadcasting
+
+Forward broadcasting can expand dimensions. During backward, ModelStudio reduces
+the upstream gradient back to each input's original shape with `unbroadcast`.
+This handles extra leading dimensions and dimensions of size one.
+
+## `no_grad`
+
+`ms.no_grad()` temporarily disables graph recording:
+
+```python
+with ms.no_grad():
+    y = x * 2
+```
+
+This is used by optimizers and inference code.
+
+## Limitations
+
+- CPU tensors only
+- Dynamic graph only
+- No higher-order gradients
+- No in-place autograd correctness checks
+- Limited operation coverage
+- No graph pruning beyond Python reference lifetimes

modelstudio-0.2.0/docs/backend-architecture.md

@@ -0,0 +1,72 @@
+# Backend Architecture
+
+ModelStudio keeps the Python tensor API separate from concrete kernel
+implementations.
+
+```text
+Tensor API
+  -> ops
+  -> dispatcher
+  -> backend implementation
+  -> CPU / CUDA / ROCm / oneAPI runtime
+```
+
+## Current Backend Status
+
+| Backend | Status |
+| --- | --- |
+| CPU | Working MVP backed by NumPy |
+| CUDA | Scaffold only |
+| ROCm | Scaffold only |
+| oneAPI | Scaffold only |
+
+Unsupported accelerator backends are registered as unavailable placeholders.
+They fail at runtime instead of import time so CPU users can import ModelStudio
+on machines without CUDA, ROCm, or oneAPI installed.
+
+## Python Backend Interface
+
+Backends implement methods such as:
+
+```python
+empty(shape, dtype)
+add(lhs, rhs)
+matmul(lhs, rhs)
+sum(value, axis, keepdims)
+```
+
+Tensor operations route through the dispatcher using the tensor device type.
+Device checks stay near dispatch boundaries instead of being scattered through
+the public API.
+
+## Native Extension Direction
+
+The native scaffolding under `csrc/` mirrors the Python runtime:
+
+- `csrc/core`: dtype, device, shape, tensor metadata, storage
+- `csrc/dispatcher`: backend interface and operator registry
+- `csrc/backends/cpu`: native CPU backend and kernels
+- `csrc/backends/cuda`: CUDA placeholders
+- `csrc/backends/rocm`: ROCm/HIP placeholders
+- `csrc/backends/oneapi`: oneAPI/SYCL placeholders
+
+Future native kernels should be registered behind the backend interface so the
+Python API stays stable.
+
+## Memory and Streams
+
+The CPU MVP stores arrays in NumPy. Future native backends should introduce:
+
+- CPU allocator abstractions for native storage
+- CUDA device allocator and stream support
+- HIP allocator and stream support
+- SYCL allocator and queue support
+
+Streams and queues should be backend-owned details exposed through explicit
+runtime APIs only after they are tested.
+
+## Backend Rule
+
+Do not claim backend support until kernels, memory management, packaging, and
+tests exist for that backend. Placeholder backends must raise clear
+`ModelStudioBackendUnavailable` errors.

modelstudio-0.2.0/docs/data.md

@@ -0,0 +1,32 @@
+# Data Utilities
+
+ModelStudio includes small CPU-only data utilities for tests, examples, and
+simple experiments.
+
+## Dataset
+
+Subclass `Dataset` and implement `__len__` and `__getitem__`.
+
+## TensorDataset
+
+```python
+from modelstudio import data
+
+dataset = data.TensorDataset(x, y)
+```
+
+All tensors must have the same first dimension. Items are returned as tuples.
+
+## DataLoader
+
+```python
+loader = data.DataLoader(dataset, batch_size=32, shuffle=True, drop_last=False)
+```
+
+The current `DataLoader` is intentionally simple:
+
+- CPU only
+- no multiprocessing
+- no workers
+- supports tensors and tuples of tensors
+- batches along axis 0

modelstudio-0.2.0/docs/nn.md

@@ -0,0 +1,51 @@
+# Neural Network API
+
+ModelStudio exposes neural-network building blocks through `modelstudio.nn`.
+
+```python
+import modelstudio as ms
+from modelstudio import nn
+
+model = nn.Linear(4, 2)
+```
+
+## Modules
+
+`nn.Module` supports:
+
+- `parameters()`
+- `named_parameters()`
+- `register_buffer()`
+- `named_buffers()`
+- `state_dict()`
+- `load_state_dict()`
+- `train()` / `eval()`
+- `to("cpu")`
+
+State dictionaries use dotted keys for nested modules, such as
+`encoder.fc1.weight`.
+
+## Layers
+
+Current CPU layers include:
+
+- `Linear`
+- `Embedding`
+- `LayerNorm`
+- `RMSNorm`
+- `TransformerBlock`
+- `ReLU`
+- `GELU`
+- `MSELoss`
+- `CrossEntropyLoss`
+
+`TransformerBlock` is intentionally minimal and CPU-only. Nonzero dropout raises
+a clear error because dropout is not implemented yet.
+
+## Save and Load
+
+```python
+ms.save(model.state_dict(), "model.ms")
+state = ms.load("model.ms")
+model.load_state_dict(state)
+```