synapgrad 1.0.3__tar.gz

synapgrad-1.0.3/LICENSE

@@ -0,0 +1,7 @@
+The MIT License (MIT) Copyright (c) 2020 Andrej Karpathy
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

synapgrad-1.0.3/PKG-INFO

@@ -0,0 +1,137 @@
+Metadata-Version: 2.4
+Name: synapgrad
+Version: 1.0.3
+Summary: A lightweight autograd engine with a deep learning library and a Torch-like API
+Author-email: Pablo García Mesa <pgmesa.sm@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/pgmesa/synapgrad
+Keywords: Autograd,Tensor,Machine Learning,Deep Learning,Python
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy<3.0.0,>=1.23.0
+Provides-Extra: train
+Requires-Dist: tqdm<5.0.0,>=4.0.0; extra == "train"
+Requires-Dist: matplotlib<4.0.0,>=3.0.0; extra == "train"
+Requires-Dist: scikit-learn<2.0.0,>=1.2.1; extra == "train"
+Dynamic: license-file
+
+#  SynapGrad
+
+A lightweight autograd engine with a deep learning library and a Torch-like API, built from scratch.
+
+[![Downloads](https://static.pepy.tech/personalized-badge/synapgrad?period=total&units=international_system&left_color=black&right_color=blue&left_text=Downloads)](https://pepy.tech/project/synapgrad)
+
+## Installation
+_Python ≥ `3.9`_
+```bash 
+pip install synapgrad
+```
+## Technical Description
+This project implements a completely functional engine for tracking operations between Tensors, by dynamically building a Directed Acyclic Graph (DAG), and an automatic gradient calculation and backpropagation algorithm (reverse-mode autodiff) over this DAG.
+
+Built on top of the engine, the deep learning library implements the most common functions, layers, losses and optimizers in order to create AI models able to solve real problems. This library mimics PyTorch in a simplified way, with the same API style and function names. 
+
+## Aim of the project
+The objective of this project is to develop a lightweight deep learning library entirely from scratch, without relying on any existing frameworks (such as Keras, PyTorch, TensorFlow, scikit-learn, etc.), using only the `numpy` library.
+
+> At the moment, supporting GPU execution is not within the scope of the project
+
+## Autograd Engine
+Automatic gradient calculation and backpropagation algorithm
+
+### Requirements
+```r
+numpy>=1.23.0,<3.0.0 # Core
+graphviz==0.20.1 # (Optional) Visualize DAG (https://graphviz.org/download/)
+```
+
+In the `examples/visualize_graph.ipynb` notebook there is an example of how to display the graph that synapgrad creates in the background as operations are chained:
+
+```python
+import synapgrad
+
+with synapgrad.retain_grads():
+    x1 = synapgrad.tensor([[-5.0, 3.0], [2.0, -4.0]], requires_grad=True)
+    x2 = synapgrad.tensor([[6.0, 0.4], [1.9,  2.0]], requires_grad=True)
+    x3 = synapgrad.tensor(3.0, requires_grad=True)
+    y = synapgrad.addmm(x3, x1, x2) # x3 + x1 @ x2
+    z = x2.sqrt() @ y
+    z.backward(synapgrad.ones(z.shape))
+    
+# graphviz is required to draw the graph
+z.draw_graph()
+```
+![Graph Image](/.github/graph_example.svg)
+
+## Deep learning library
+Built on top of the engine, synapgrad has a deep learning library that implements the following features:
+
+- `Weight initialization`: Xavier Glorot uniform, Xavier Glorot normal, He Kaiming uniform, He Kaiming normal
+- `Activations`: ReLU, LeakyReLU, SELU, Tanh, Sigmoid, Softmax, LogSoftmax
+- `Layers`: Linear, Unfold, Fold, Flatten, Dropout
+- `Convolutions`: MaxPool1d, MaxPool2d, AvgPool1d, AvgPool2d, Conv1d, Conv2d
+- `Normalizations`: BatchNorm1d, BatchNorm2d
+- `Optimizers`: SGD, Adam, AdamW
+- `Losses`: MSELoss, NLLLoss, BCELoss, BCEWithLogitsLoss, CrossEntropyLoss
+
+This project includes three Jupyter notebooks (located in `examples/`) that tackle three beginner-level AI problems:
+
+- [x] 1. Basic MLP for binary classification (sklearn 'make_moons' toy dataset)
+- [x] 2. MLP for handwritten digits classification (MNIST dataset) 
+- [x] 3. CNN for handwritten digits classification (MNIST dataset)
+
+### Extra requirements
+To use the SynapGrad training utilities, install the optional requirements by running:
+```bash
+pip install synapgrad[train]
+```
+
+Additionally to run the notebooks in `/examples` you will need to install:
+```r
+torchvision>=0.13.1
+# **** torch required only if you want to use torch engine instead of synapgrad's *****
+torch>=1.12.1,<3.0.0 # Install following the instructions in https://pytorch.org/
+```
+
+### Comparisons with PyTorch
+To measure the efficiency of synapgrad, all three examples were compared to PyTorch 2.14. All training sessions were conducted on a Ryzen 7 9700X (8 cores). The times are those of `trainer.fit`, training plus the validation pass of every epoch.
+
+| Notebook | torch | synapgrad | Model params | Dataset size | Batch size | Epochs |
+|     :---:     |  :---:  |  :---:  | :---:  | :---:  | :---:  | :---:  |
+| basic_mlp.ipynb | 1.4 s | 0.26 s | 337 | [150, 2] | 4 | 50 |
+| mnist_mlp.ipynb | 4.5 s | 9.7 s | 178_110 |  [48_000, 28, 28]  | 64 | 10 | 
+| mnist_cnn.ipynb |  37.8 s  |  4 min 55 s |  20_538  | [48_000, 1, 28, 28]  |  128  | 5 |
+
+## Running tests
+
+To run the unit tests you will have to install Pytest and PyTorch, which is used to check whether the gradients are calculated correctly
+```r
+pytest>=8.0.0,<9.0.0
+```
+
+Run all the tests:
+```bash
+python -m pytest
+```
+To compare the speed of `torch` and `synapgrad` in each operation, run the command below:
+```
+python -m pytest ./tests/test_ops.py -s
+```

synapgrad-1.0.3/README.md

@@ -0,0 +1,102 @@
+#  SynapGrad
+
+A lightweight autograd engine with a deep learning library and a Torch-like API, built from scratch.
+
+[![Downloads](https://static.pepy.tech/personalized-badge/synapgrad?period=total&units=international_system&left_color=black&right_color=blue&left_text=Downloads)](https://pepy.tech/project/synapgrad)
+
+## Installation
+_Python ≥ `3.9`_
+```bash 
+pip install synapgrad
+```
+## Technical Description
+This project implements a completely functional engine for tracking operations between Tensors, by dynamically building a Directed Acyclic Graph (DAG), and an automatic gradient calculation and backpropagation algorithm (reverse-mode autodiff) over this DAG.
+
+Built on top of the engine, the deep learning library implements the most common functions, layers, losses and optimizers in order to create AI models able to solve real problems. This library mimics PyTorch in a simplified way, with the same API style and function names. 
+
+## Aim of the project
+The objective of this project is to develop a lightweight deep learning library entirely from scratch, without relying on any existing frameworks (such as Keras, PyTorch, TensorFlow, scikit-learn, etc.), using only the `numpy` library.
+
+> At the moment, supporting GPU execution is not within the scope of the project
+
+## Autograd Engine
+Automatic gradient calculation and backpropagation algorithm
+
+### Requirements
+```r
+numpy>=1.23.0,<3.0.0 # Core
+graphviz==0.20.1 # (Optional) Visualize DAG (https://graphviz.org/download/)
+```
+
+In the `examples/visualize_graph.ipynb` notebook there is an example of how to display the graph that synapgrad creates in the background as operations are chained:
+
+```python
+import synapgrad
+
+with synapgrad.retain_grads():
+    x1 = synapgrad.tensor([[-5.0, 3.0], [2.0, -4.0]], requires_grad=True)
+    x2 = synapgrad.tensor([[6.0, 0.4], [1.9,  2.0]], requires_grad=True)
+    x3 = synapgrad.tensor(3.0, requires_grad=True)
+    y = synapgrad.addmm(x3, x1, x2) # x3 + x1 @ x2
+    z = x2.sqrt() @ y
+    z.backward(synapgrad.ones(z.shape))
+    
+# graphviz is required to draw the graph
+z.draw_graph()
+```
+![Graph Image](/.github/graph_example.svg)
+
+## Deep learning library
+Built on top of the engine, synapgrad has a deep learning library that implements the following features:
+
+- `Weight initialization`: Xavier Glorot uniform, Xavier Glorot normal, He Kaiming uniform, He Kaiming normal
+- `Activations`: ReLU, LeakyReLU, SELU, Tanh, Sigmoid, Softmax, LogSoftmax
+- `Layers`: Linear, Unfold, Fold, Flatten, Dropout
+- `Convolutions`: MaxPool1d, MaxPool2d, AvgPool1d, AvgPool2d, Conv1d, Conv2d
+- `Normalizations`: BatchNorm1d, BatchNorm2d
+- `Optimizers`: SGD, Adam, AdamW
+- `Losses`: MSELoss, NLLLoss, BCELoss, BCEWithLogitsLoss, CrossEntropyLoss
+
+This project includes three Jupyter notebooks (located in `examples/`) that tackle three beginner-level AI problems:
+
+- [x] 1. Basic MLP for binary classification (sklearn 'make_moons' toy dataset)
+- [x] 2. MLP for handwritten digits classification (MNIST dataset) 
+- [x] 3. CNN for handwritten digits classification (MNIST dataset)
+
+### Extra requirements
+To use the SynapGrad training utilities, install the optional requirements by running:
+```bash
+pip install synapgrad[train]
+```
+
+Additionally to run the notebooks in `/examples` you will need to install:
+```r
+torchvision>=0.13.1
+# **** torch required only if you want to use torch engine instead of synapgrad's *****
+torch>=1.12.1,<3.0.0 # Install following the instructions in https://pytorch.org/
+```
+
+### Comparisons with PyTorch
+To measure the efficiency of synapgrad, all three examples were compared to PyTorch 2.14. All training sessions were conducted on a Ryzen 7 9700X (8 cores). The times are those of `trainer.fit`, training plus the validation pass of every epoch.
+
+| Notebook | torch | synapgrad | Model params | Dataset size | Batch size | Epochs |
+|     :---:     |  :---:  |  :---:  | :---:  | :---:  | :---:  | :---:  |
+| basic_mlp.ipynb | 1.4 s | 0.26 s | 337 | [150, 2] | 4 | 50 |
+| mnist_mlp.ipynb | 4.5 s | 9.7 s | 178_110 |  [48_000, 28, 28]  | 64 | 10 | 
+| mnist_cnn.ipynb |  37.8 s  |  4 min 55 s |  20_538  | [48_000, 1, 28, 28]  |  128  | 5 |
+
+## Running tests
+
+To run the unit tests you will have to install Pytest and PyTorch, which is used to check whether the gradients are calculated correctly
+```r
+pytest>=8.0.0,<9.0.0
+```
+
+Run all the tests:
+```bash
+python -m pytest
+```
+To compare the speed of `torch` and `synapgrad` in each operation, run the command below:
+```
+python -m pytest ./tests/test_ops.py -s
+```

synapgrad-1.0.3/pyproject.toml

@@ -0,0 +1,58 @@
+[build-system]
+requires = ["setuptools>=64.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "synapgrad"
+version = "1.0.3"
+description = "A lightweight autograd engine with a deep learning library and a Torch-like API"
+readme = "README.md"
+requires-python = ">=3.9"
+license = {text = "MIT"}
+urls = {Homepage = "https://github.com/pgmesa/synapgrad"}
+keywords = ["Autograd", "Tensor", "Machine Learning", "Deep Learning", "Python"]
+authors = [
+    {name = "Pablo García Mesa", email = "pgmesa.sm@gmail.com"}
+]
+
+dependencies = [
+    "numpy>=1.23.0,<3.0.0",
+]
+
+classifiers = [
+    "Development Status :: 5 - Production/Stable",
+    
+    "Intended Audience :: Developers",
+    "Intended Audience :: Education",
+    "Intended Audience :: Science/Research",
+
+    "License :: OSI Approved :: MIT License",
+    
+    "Operating System :: OS Independent",
+    
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    
+    "Topic :: Scientific/Engineering",
+    "Topic :: Scientific/Engineering :: Mathematics",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Software Development",
+    "Topic :: Software Development :: Libraries",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+
+[project.optional-dependencies]
+train = [
+    "tqdm>=4.0.0,<5.0.0",
+    "matplotlib>=3.0.0,<4.0.0",
+    "scikit-learn>=1.2.1,<2.0.0"
+]
+
+[tool.setuptools]
+packages = { find = { where = ["."], include = ["synapgrad", "synapgrad.*"]} }
+
+[tool.test]
+framework = "pytest"

synapgrad-1.0.3/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

synapgrad-1.0.3/synapgrad/__init__.py

@@ -0,0 +1,23 @@
+from synapgrad.tensor import (
+    Tensor, tensor, empty, ones, ones_like, zeros, zeros_like, 
+    arange, rand, randn, normal, randint, eye, no_grad,
+    retain_grads, no_grad
+)
+from synapgrad.functional import (
+    add, mul, matmul, addmm, pow, rpow, neg, slice,
+    concat, stack, unbind,
+    clone, exp, log, sqrt, sum, mean, max, min, squeeze, unsqueeze,
+    reshape, movedim, transpose, flatten, unfold_dim
+)
+
+from synapgrad.utils import manual_seed
+
+from synapgrad.nn.functional import (
+    relu, leaky_relu, selu, tanh, sigmoid, softmax, log_softmax,
+    mse_loss, nll_loss, binary_cross_entropy, binary_cross_entropy_with_logits, cross_entropy,
+    linear, unfold, fold, max_pool1d, max_pool2d, avg_pool1d, avg_pool2d, conv1d, conv2d, batch_norm
+)
+
+from synapgrad import (
+    cpu_ops, conv_tools, device, functional, nn, optim, utils, visual
+)