modelstudio 0.2.0__tar.gz → 0.4.0__tar.gz

{modelstudio-0.2.0/python/modelstudio.egg-info → modelstudio-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: modelstudio
-Version: 0.2.0
+Version: 0.4.0
 Summary: An early-stage AI tensor framework with CPU tensors, autograd, and backend extension scaffolding.
 Author: ModelStudio Contributors
 License-Expression: MIT
@@ -31,7 +31,7 @@ Dynamic: license-file
 
 # ModelStudio
 
-ModelStudio is an early-stage AI tensor framework. Version `0.2.0` provides a
+ModelStudio is an early-stage AI tensor framework. Version `0.4.0` provides a
 CPU tensor/autograd MVP with neural-network modules, optimizers, serialization,
 basic data loading, and small LLM-oriented building blocks.
 
@@ -60,11 +60,15 @@ python -m pip install -e ".[dev]"
 | CPU tensors | Working MVP |
 | Autograd | Reverse-mode for core CPU ops |
 | Reductions | `sum`, `mean`, `max` with axis and keepdims; `max` is value-only |
-| Activations | ReLU, GELU, exp, log, tanh, sigmoid, SiLU, softmax, log-softmax |
-| Losses | MSE and cross entropy |
-| Modules | Parameters, buffers, state dicts, save/load |
-| Layers | Linear, Embedding, LayerNorm, RMSNorm, TransformerBlock |
-| Data | Dataset, TensorDataset, DataLoader |
+| Activations | ReLU, GELU, LeakyReLU, ELU, Softplus, exp, log, tanh, sigmoid, SiLU, softmax, log-softmax |
+| Losses | MSE and cross entropy with `none`, `mean`, and `sum` reductions |
+| Modules | Parameters, buffers, child traversal, state dicts, save/load |
+| Layers | Linear, Embedding, LayerNorm, RMSNorm, BatchNorm1d, Dropout, Conv1d, Conv2d, pooling, TransformerBlock |
+| Optimizers | SGD and AdamW with state serialization, parameter groups, and LR schedulers |
+| Data | Dataset, TensorDataset, random_split, DataLoader with deterministic seeded shuffle |
+| Randomness | `manual_seed`, RNG-backed `randn`, dropout, and init helpers |
+| Interop | `asarray`, `from_numpy`, `to_numpy`, and `ms.numpy` |
+| Metrics | accuracy and top-k accuracy |
 | Compiler | Placeholder IR and passes |
 
 ## Backend Status
@@ -163,6 +167,49 @@ y = block(x)
 print(y.shape)
 ```
 
+## 0.4.0 Training Utilities
+
+```python
+ms.manual_seed(123)
+model = nn.Linear(4, 2)
+optimizer = ms.optim.AdamW(model.parameters(), lr=1e-3)
+state = {"model": model.state_dict(), "optimizer": optimizer.state_dict()}
+ms.save(state, "checkpoint.ms")
+```
+
+New CPU-only helpers include `ms.concat`, `ms.stack`, `Tensor.flatten`,
+`Tensor.squeeze`, `Tensor.unsqueeze`, `nn.init`, `nn.Dropout`,
+`nn.BatchNorm1d`, `nn.Conv1d`, `nn.Conv2d`, `nn.AvgPool2d`, `nn.MaxPool2d`,
+and `nn.utils` gradient clipping.
+
+## NumPy Interop
+
+```python
+x = ms.asarray([[1, 2, 3], [4, 5, 6]], dtype=ms.float32)
+arr = ms.to_numpy(x)
+y = ms.from_numpy(arr)
+```
+
+CPU uses NumPy internally. Normal examples prefer ModelStudio APIs; `ms.numpy`
+is exposed for advanced users who explicitly want NumPy access.
+
+## Schedulers and Metrics
+
+```python
+optimizer = ms.optim.AdamW(model.parameters(), lr=1e-3)
+scheduler = ms.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=0.5)
+scheduler.step()
+
+acc = ms.metrics.accuracy(logits, targets)
+```
+
+## Checkpointing
+
+```python
+ms.save_checkpoint("checkpoint.ms", model=model, optimizer=optimizer, scheduler=scheduler, extra={"epoch": 1})
+checkpoint = ms.load_checkpoint("checkpoint.ms", model=model, optimizer=optimizer, scheduler=scheduler)
+```
+
 ## Commands
 
 ```bash
@@ -172,10 +219,21 @@ python examples/train_mlp.py
 python examples/train_classifier.py
 python examples/tiny_transformer.py
 python examples/save_load.py
+python examples/train_cnn_toy.py
+python examples/dropout_batchnorm.py
+python examples/checkpoint_training.py
+python examples/numpy_interop.py
+python examples/scheduler_training.py
+python examples/checkpoint_resume.py
+python examples/metrics_demo.py
 python benchmarks/bench_matmul.py
 python benchmarks/bench_mlp.py
 python benchmarks/bench_attention.py
 python benchmarks/bench_dataloader.py
+python benchmarks/bench_conv.py
+python benchmarks/bench_dropout.py
+python benchmarks/bench_creation.py
+python benchmarks/bench_manipulation.py
 ```
 
 ## Documentation
@@ -183,6 +241,17 @@ python benchmarks/bench_dataloader.py
 - [Tensor API](docs/tensor-api.md)
 - [Neural network API](docs/nn.md)
 - [Data utilities](docs/data.md)
+- [Training](docs/training.md)
+- [Modules](docs/modules.md)
+- [Serialization](docs/serialization.md)
+- [Randomness](docs/randomness.md)
+- [Native backend roadmap](docs/native-backend-roadmap.md)
+- [NumPy interop](docs/numpy-interop.md)
+- [Tensor creation](docs/tensor-creation.md)
+- [Tensor manipulation](docs/tensor-manipulation.md)
+- [Optimizers](docs/optimizers.md)
+- [Checkpointing](docs/checkpointing.md)
+- [Metrics](docs/metrics.md)
 - [Backend architecture](docs/backend-architecture.md)
 - [Autograd design](docs/autograd.md)
 - [Releasing](docs/releasing.md)

{modelstudio-0.2.0 → modelstudio-0.4.0}/README.md

@@ -1,6 +1,6 @@
 # ModelStudio
 
-ModelStudio is an early-stage AI tensor framework. Version `0.2.0` provides a
+ModelStudio is an early-stage AI tensor framework. Version `0.4.0` provides a
 CPU tensor/autograd MVP with neural-network modules, optimizers, serialization,
 basic data loading, and small LLM-oriented building blocks.
 
@@ -29,11 +29,15 @@ python -m pip install -e ".[dev]"
 | CPU tensors | Working MVP |
 | Autograd | Reverse-mode for core CPU ops |
 | Reductions | `sum`, `mean`, `max` with axis and keepdims; `max` is value-only |
-| Activations | ReLU, GELU, exp, log, tanh, sigmoid, SiLU, softmax, log-softmax |
-| Losses | MSE and cross entropy |
-| Modules | Parameters, buffers, state dicts, save/load |
-| Layers | Linear, Embedding, LayerNorm, RMSNorm, TransformerBlock |
-| Data | Dataset, TensorDataset, DataLoader |
+| Activations | ReLU, GELU, LeakyReLU, ELU, Softplus, exp, log, tanh, sigmoid, SiLU, softmax, log-softmax |
+| Losses | MSE and cross entropy with `none`, `mean`, and `sum` reductions |
+| Modules | Parameters, buffers, child traversal, state dicts, save/load |
+| Layers | Linear, Embedding, LayerNorm, RMSNorm, BatchNorm1d, Dropout, Conv1d, Conv2d, pooling, TransformerBlock |
+| Optimizers | SGD and AdamW with state serialization, parameter groups, and LR schedulers |
+| Data | Dataset, TensorDataset, random_split, DataLoader with deterministic seeded shuffle |
+| Randomness | `manual_seed`, RNG-backed `randn`, dropout, and init helpers |
+| Interop | `asarray`, `from_numpy`, `to_numpy`, and `ms.numpy` |
+| Metrics | accuracy and top-k accuracy |
 | Compiler | Placeholder IR and passes |
 
 ## Backend Status
@@ -132,6 +136,49 @@ y = block(x)
 print(y.shape)
 ```
 
+## 0.4.0 Training Utilities
+
+```python
+ms.manual_seed(123)
+model = nn.Linear(4, 2)
+optimizer = ms.optim.AdamW(model.parameters(), lr=1e-3)
+state = {"model": model.state_dict(), "optimizer": optimizer.state_dict()}
+ms.save(state, "checkpoint.ms")
+```
+
+New CPU-only helpers include `ms.concat`, `ms.stack`, `Tensor.flatten`,
+`Tensor.squeeze`, `Tensor.unsqueeze`, `nn.init`, `nn.Dropout`,
+`nn.BatchNorm1d`, `nn.Conv1d`, `nn.Conv2d`, `nn.AvgPool2d`, `nn.MaxPool2d`,
+and `nn.utils` gradient clipping.
+
+## NumPy Interop
+
+```python
+x = ms.asarray([[1, 2, 3], [4, 5, 6]], dtype=ms.float32)
+arr = ms.to_numpy(x)
+y = ms.from_numpy(arr)
+```
+
+CPU uses NumPy internally. Normal examples prefer ModelStudio APIs; `ms.numpy`
+is exposed for advanced users who explicitly want NumPy access.
+
+## Schedulers and Metrics
+
+```python
+optimizer = ms.optim.AdamW(model.parameters(), lr=1e-3)
+scheduler = ms.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=0.5)
+scheduler.step()
+
+acc = ms.metrics.accuracy(logits, targets)
+```
+
+## Checkpointing
+
+```python
+ms.save_checkpoint("checkpoint.ms", model=model, optimizer=optimizer, scheduler=scheduler, extra={"epoch": 1})
+checkpoint = ms.load_checkpoint("checkpoint.ms", model=model, optimizer=optimizer, scheduler=scheduler)
+```
+
 ## Commands
 
 ```bash
@@ -141,10 +188,21 @@ python examples/train_mlp.py
 python examples/train_classifier.py
 python examples/tiny_transformer.py
 python examples/save_load.py
+python examples/train_cnn_toy.py
+python examples/dropout_batchnorm.py
+python examples/checkpoint_training.py
+python examples/numpy_interop.py
+python examples/scheduler_training.py
+python examples/checkpoint_resume.py
+python examples/metrics_demo.py
 python benchmarks/bench_matmul.py
 python benchmarks/bench_mlp.py
 python benchmarks/bench_attention.py
 python benchmarks/bench_dataloader.py
+python benchmarks/bench_conv.py
+python benchmarks/bench_dropout.py
+python benchmarks/bench_creation.py
+python benchmarks/bench_manipulation.py
 ```
 
 ## Documentation
@@ -152,6 +210,17 @@ python benchmarks/bench_dataloader.py
 - [Tensor API](docs/tensor-api.md)
 - [Neural network API](docs/nn.md)
 - [Data utilities](docs/data.md)
+- [Training](docs/training.md)
+- [Modules](docs/modules.md)
+- [Serialization](docs/serialization.md)
+- [Randomness](docs/randomness.md)
+- [Native backend roadmap](docs/native-backend-roadmap.md)
+- [NumPy interop](docs/numpy-interop.md)
+- [Tensor creation](docs/tensor-creation.md)
+- [Tensor manipulation](docs/tensor-manipulation.md)
+- [Optimizers](docs/optimizers.md)
+- [Checkpointing](docs/checkpointing.md)
+- [Metrics](docs/metrics.md)
 - [Backend architecture](docs/backend-architecture.md)
 - [Autograd design](docs/autograd.md)
 - [Releasing](docs/releasing.md)

modelstudio-0.4.0/benchmarks/bench_conv.py

@@ -0,0 +1,40 @@
+from __future__ import annotations
+
+import platform
+import time
+
+import modelstudio as ms
+import numpy as np
+from modelstudio import nn
+
+
+def timeit(fn, iterations: int, warmup: int) -> float:
+    for _ in range(warmup):
+        fn()
+    start = time.perf_counter()
+    for _ in range(iterations):
+        fn()
+    return (time.perf_counter() - start) / iterations
+
+
+def main() -> None:
+    warmup = 3
+    iterations = 10
+    shape = (8, 3, 16, 16)
+    ms.manual_seed(1)
+    x = ms.randn(shape)
+    conv = nn.Conv2d(3, 8, kernel_size=3, padding=1)
+
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print(f"Operation:   Conv2d input={shape} out_channels=8 kernel=3 padding=1")
+    print(f"Warmup:      {warmup}")
+    print(f"Iterations:  {iterations}")
+    print("Backend:     CPU only")
+    print(f"Conv2d avg:  {timeit(lambda: conv(x), iterations, warmup) * 1_000:.3f} ms")
+
+
+if __name__ == "__main__":
+    main()
+

modelstudio-0.4.0/benchmarks/bench_creation.py

@@ -0,0 +1,38 @@
+from __future__ import annotations
+
+import platform
+import time
+
+import modelstudio as ms
+import numpy as np
+
+
+def timeit(fn, iterations: int, warmup: int) -> float:
+    for _ in range(warmup):
+        fn()
+    start = time.perf_counter()
+    for _ in range(iterations):
+        fn()
+    return (time.perf_counter() - start) / iterations
+
+
+def main() -> None:
+    warmup = 5
+    iterations = 50
+    shape = (512, 512)
+
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print(f"Operation:   rand/full/eye shape={shape}")
+    print(f"Warmup:      {warmup}")
+    print(f"Iterations:  {iterations}")
+    print("Backend:     CPU only")
+    print(f"rand avg:    {timeit(lambda: ms.rand(shape), iterations, warmup) * 1_000:.3f} ms")
+    print(f"full avg:    {timeit(lambda: ms.full(shape, 1.25), iterations, warmup) * 1_000:.3f} ms")
+    print(f"eye avg:     {timeit(lambda: ms.eye(512), iterations, warmup) * 1_000:.3f} ms")
+
+
+if __name__ == "__main__":
+    main()
+

modelstudio-0.4.0/benchmarks/bench_dropout.py

@@ -0,0 +1,37 @@
+from __future__ import annotations
+
+import platform
+import time
+
+import modelstudio as ms
+import numpy as np
+
+
+def timeit(fn, iterations: int, warmup: int) -> float:
+    for _ in range(warmup):
+        fn()
+    start = time.perf_counter()
+    for _ in range(iterations):
+        fn()
+    return (time.perf_counter() - start) / iterations
+
+
+def main() -> None:
+    warmup = 5
+    iterations = 50
+    shape = (512, 512)
+    x = ms.randn(shape)
+
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print(f"Operation:   dropout shape={shape} p=0.5")
+    print(f"Warmup:      {warmup}")
+    print(f"Iterations:  {iterations}")
+    print("Backend:     CPU only")
+    print(f"Dropout avg: {timeit(lambda: ms.dropout(x, p=0.5), iterations, warmup) * 1_000:.3f} ms")
+
+
+if __name__ == "__main__":
+    main()
+

modelstudio-0.4.0/benchmarks/bench_manipulation.py

@@ -0,0 +1,38 @@
+from __future__ import annotations
+
+import platform
+import time
+
+import modelstudio as ms
+import numpy as np
+
+
+def timeit(fn, iterations: int, warmup: int) -> float:
+    for _ in range(warmup):
+        fn()
+    start = time.perf_counter()
+    for _ in range(iterations):
+        fn()
+    return (time.perf_counter() - start) / iterations
+
+
+def main() -> None:
+    warmup = 5
+    iterations = 50
+    x = ms.randn((64, 64, 8))
+
+    print(f"Python:      {platform.python_version()}")
+    print(f"NumPy:       {np.__version__}")
+    print(f"ModelStudio: {ms.__version__}")
+    print("Operation:   permute/repeat/tile")
+    print(f"Warmup:      {warmup}")
+    print(f"Iterations:  {iterations}")
+    print("Backend:     CPU only")
+    print(f"permute avg: {timeit(lambda: ms.permute(x, (2, 0, 1)), iterations, warmup) * 1_000:.3f} ms")
+    print(f"repeat avg:  {timeit(lambda: ms.repeat(x, 2, axis=0), iterations, warmup) * 1_000:.3f} ms")
+    print(f"tile avg:    {timeit(lambda: ms.tile(x, (2, 1, 1)), iterations, warmup) * 1_000:.3f} ms")
+
+
+if __name__ == "__main__":
+    main()
+

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/CMakeLists.txt

@@ -2,6 +2,8 @@ add_library(modelstudio_native STATIC
   backends/cpu/cpu_backend.cpp
   backends/cpu/kernels/add.cpp
   backends/cpu/kernels/matmul.cpp
+  backends/cpu/kernels/mul.cpp
+  backends/cpu/kernels/relu.cpp
 )
 
 target_include_directories(modelstudio_native PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/backends/cpu/cpu_backend.cpp

@@ -10,8 +10,16 @@ Tensor CPUBackend::add(const Tensor& lhs, const Tensor& rhs) {
   return add_kernel(lhs, rhs);
 }
 
+Tensor CPUBackend::mul(const Tensor& lhs, const Tensor& rhs) {
+  return mul_kernel(lhs, rhs);
+}
+
 Tensor CPUBackend::matmul(const Tensor& lhs, const Tensor& rhs) {
   return matmul_kernel(lhs, rhs);
 }
 
+Tensor CPUBackend::relu(const Tensor& input) {
+  return relu_kernel(input);
+}
+
 }  // namespace modelstudio::cpu

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/backends/cpu/cpu_backend.hpp

@@ -5,14 +5,18 @@
 namespace modelstudio::cpu {
 
 Tensor add_kernel(const Tensor& lhs, const Tensor& rhs);
+Tensor mul_kernel(const Tensor& lhs, const Tensor& rhs);
 Tensor matmul_kernel(const Tensor& lhs, const Tensor& rhs);
+Tensor relu_kernel(const Tensor& input);
 
 class CPUBackend final : public Backend {
  public:
   std::string_view name() const override { return "cpu"; }
   Tensor empty(const Shape& shape, DType dtype) override;
   Tensor add(const Tensor& lhs, const Tensor& rhs) override;
+  Tensor mul(const Tensor& lhs, const Tensor& rhs) override;
   Tensor matmul(const Tensor& lhs, const Tensor& rhs) override;
+  Tensor relu(const Tensor& input) override;
 };
 
 }  // namespace modelstudio::cpu

modelstudio-0.4.0/csrc/backends/cpu/kernels/mul.cpp

@@ -0,0 +1,43 @@
+#include "backends/cpu/cpu_backend.hpp"
+
+namespace modelstudio::cpu {
+namespace {
+
+template <typename T>
+void mul_typed(const Tensor& lhs, const Tensor& rhs, Tensor& out) {
+  const auto* lhs_ptr = static_cast<const T*>(lhs.data());
+  const auto* rhs_ptr = static_cast<const T*>(rhs.data());
+  auto* out_ptr = static_cast<T*>(out.data());
+  for (std::int64_t i = 0; i < lhs.numel(); ++i) {
+    out_ptr[i] = lhs_ptr[i] * rhs_ptr[i];
+  }
+}
+
+}  // namespace
+
+Tensor mul_kernel(const Tensor& lhs, const Tensor& rhs) {
+  if (lhs.shape() != rhs.shape() || lhs.dtype() != rhs.dtype()) {
+    throw Error("native CPU mul currently requires identical shape and dtype");
+  }
+  Tensor out(lhs.shape(), lhs.dtype(), lhs.device());
+  switch (lhs.dtype()) {
+    case DType::Float32:
+      mul_typed<float>(lhs, rhs, out);
+      break;
+    case DType::Float64:
+      mul_typed<double>(lhs, rhs, out);
+      break;
+    case DType::Int32:
+      mul_typed<std::int32_t>(lhs, rhs, out);
+      break;
+    case DType::Int64:
+      mul_typed<std::int64_t>(lhs, rhs, out);
+      break;
+    case DType::Bool:
+      mul_typed<bool>(lhs, rhs, out);
+      break;
+  }
+  return out;
+}
+
+}  // namespace modelstudio::cpu

modelstudio-0.4.0/csrc/backends/cpu/kernels/relu.cpp

@@ -0,0 +1,40 @@
+#include "backends/cpu/cpu_backend.hpp"
+
+#include <algorithm>
+
+namespace modelstudio::cpu {
+namespace {
+
+template <typename T>
+void relu_typed(const Tensor& input, Tensor& out) {
+  const auto* input_ptr = static_cast<const T*>(input.data());
+  auto* out_ptr = static_cast<T*>(out.data());
+  for (std::int64_t i = 0; i < input.numel(); ++i) {
+    out_ptr[i] = std::max<T>(input_ptr[i], T{});
+  }
+}
+
+}  // namespace
+
+Tensor relu_kernel(const Tensor& input) {
+  Tensor out(input.shape(), input.dtype(), input.device());
+  switch (input.dtype()) {
+    case DType::Float32:
+      relu_typed<float>(input, out);
+      break;
+    case DType::Float64:
+      relu_typed<double>(input, out);
+      break;
+    case DType::Int32:
+      relu_typed<std::int32_t>(input, out);
+      break;
+    case DType::Int64:
+      relu_typed<std::int64_t>(input, out);
+      break;
+    case DType::Bool:
+      throw Error("native CPU bool relu is not implemented");
+  }
+  return out;
+}
+
+}  // namespace modelstudio::cpu

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/backends/cuda/cuda_backend.cu

@@ -14,9 +14,19 @@ Tensor CUDABackend::add(const Tensor&, const Tensor&) {
   throw Error("CUDA add kernel is not implemented");
 }
 
+Tensor CUDABackend::mul(const Tensor&, const Tensor&) {
+  // TODO: Add a tested CUDA elementwise multiply kernel.
+  throw Error("CUDA mul kernel is not implemented");
+}
+
 Tensor CUDABackend::matmul(const Tensor&, const Tensor&) {
   // TODO: Add a tested CUDA matmul kernel or cuBLAS integration.
   throw Error("CUDA matmul kernel is not implemented");
 }
 
+Tensor CUDABackend::relu(const Tensor&) {
+  // TODO: Add a tested CUDA ReLU kernel.
+  throw Error("CUDA relu kernel is not implemented");
+}
+
 }  // namespace modelstudio::cuda

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/backends/cuda/cuda_backend.hpp

@@ -9,7 +9,9 @@ class CUDABackend final : public Backend {
   std::string_view name() const override { return "cuda"; }
   Tensor empty(const Shape& shape, DType dtype) override;
   Tensor add(const Tensor& lhs, const Tensor& rhs) override;
+  Tensor mul(const Tensor& lhs, const Tensor& rhs) override;
   Tensor matmul(const Tensor& lhs, const Tensor& rhs) override;
+  Tensor relu(const Tensor& input) override;
 };
 
 }  // namespace modelstudio::cuda

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/backends/oneapi/oneapi_backend.cpp

@@ -14,9 +14,19 @@ Tensor OneAPIBackend::add(const Tensor&, const Tensor&) {
   throw Error("oneAPI add kernel is not implemented");
 }
 
+Tensor OneAPIBackend::mul(const Tensor&, const Tensor&) {
+  // TODO: Add a tested SYCL elementwise multiply kernel.
+  throw Error("oneAPI mul kernel is not implemented");
+}
+
 Tensor OneAPIBackend::matmul(const Tensor&, const Tensor&) {
   // TODO: Add a tested SYCL matmul kernel or oneMKL integration.
   throw Error("oneAPI matmul kernel is not implemented");
 }
 
+Tensor OneAPIBackend::relu(const Tensor&) {
+  // TODO: Add a tested SYCL ReLU kernel.
+  throw Error("oneAPI relu kernel is not implemented");
+}
+
 }  // namespace modelstudio::oneapi

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/backends/oneapi/oneapi_backend.hpp

@@ -9,7 +9,9 @@ class OneAPIBackend final : public Backend {
   std::string_view name() const override { return "oneapi"; }
   Tensor empty(const Shape& shape, DType dtype) override;
   Tensor add(const Tensor& lhs, const Tensor& rhs) override;
+  Tensor mul(const Tensor& lhs, const Tensor& rhs) override;
   Tensor matmul(const Tensor& lhs, const Tensor& rhs) override;
+  Tensor relu(const Tensor& input) override;
 };
 
 }  // namespace modelstudio::oneapi

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/backends/rocm/rocm_backend.cpp

@@ -14,9 +14,19 @@ Tensor ROCmBackend::add(const Tensor&, const Tensor&) {
   throw Error("ROCm add kernel is not implemented");
 }
 
+Tensor ROCmBackend::mul(const Tensor&, const Tensor&) {
+  // TODO: Add a tested HIP elementwise multiply kernel.
+  throw Error("ROCm mul kernel is not implemented");
+}
+
 Tensor ROCmBackend::matmul(const Tensor&, const Tensor&) {
   // TODO: Add a tested HIP matmul kernel or rocBLAS integration.
   throw Error("ROCm matmul kernel is not implemented");
 }
 
+Tensor ROCmBackend::relu(const Tensor&) {
+  // TODO: Add a tested HIP ReLU kernel.
+  throw Error("ROCm relu kernel is not implemented");
+}
+
 }  // namespace modelstudio::rocm

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/backends/rocm/rocm_backend.hpp

@@ -9,7 +9,9 @@ class ROCmBackend final : public Backend {
   std::string_view name() const override { return "rocm"; }
   Tensor empty(const Shape& shape, DType dtype) override;
   Tensor add(const Tensor& lhs, const Tensor& rhs) override;
+  Tensor mul(const Tensor& lhs, const Tensor& rhs) override;
   Tensor matmul(const Tensor& lhs, const Tensor& rhs) override;
+  Tensor relu(const Tensor& input) override;
 };
 
 }  // namespace modelstudio::rocm

{modelstudio-0.2.0 → modelstudio-0.4.0}/csrc/dispatcher/backend.hpp

@@ -12,7 +12,9 @@ class Backend {
   virtual std::string_view name() const = 0;
   virtual Tensor empty(const Shape& shape, DType dtype) = 0;
   virtual Tensor add(const Tensor& lhs, const Tensor& rhs) = 0;
+  virtual Tensor mul(const Tensor& lhs, const Tensor& rhs) = 0;
   virtual Tensor matmul(const Tensor& lhs, const Tensor& rhs) = 0;
+  virtual Tensor relu(const Tensor& input) = 0;
 };
 
 }  // namespace modelstudio

modelstudio-0.4.0/docs/checkpointing.md

@@ -0,0 +1,30 @@
+# Checkpointing
+
+High-level checkpoint helpers wrap `ms.save` and `ms.load`.
+
+```python
+ms.save_checkpoint(
+    "checkpoint.ms",
+    model=model,
+    optimizer=optimizer,
+    scheduler=scheduler,
+    extra={"epoch": 3},
+)
+
+checkpoint = ms.load_checkpoint(
+    "checkpoint.ms",
+    model=model,
+    optimizer=optimizer,
+    scheduler=scheduler,
+)
+```
+
+The checkpoint dictionary contains:
+
+- `model`
+- optional `optimizer`
+- optional `scheduler`
+- `extra`
+
+The file format is currently ModelStudio's internal pickle-backed format.
+