froog 0.3.2__tar.gz → 0.4.2__tar.gz

{froog-0.3.2 → froog-0.4.2}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.1
 Name: froog
-Version: 0.3.2
-Summary: a beautifully simplistic tensor library
+Version: 0.4.2
+Summary: a toy tensor library with opencl support
 Author: Kevin Buhler
 License: MIT
 Classifier: Programming Language :: Python :: 3
@@ -9,10 +9,6 @@ Classifier: License :: OSI Approved :: MIT License
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: numpy
-Requires-Dist: requests
-Requires-Dist: matplotlib
-Requires-Dist: urllib
 
 # froog <img src="https://github.com/kevbuh/froog/actions/workflows/test.yml/badge.svg" alt="unit test badge" > <img src="https://static.pepy.tech/badge/froog" alt="num downloads badge">
 <div align="center" >
@@ -27,9 +23,10 @@ Requires-Dist: urllib
   <br/>
 </div>
 
-```froog``` is an easy-to-read tensor library (<a href="https://www.pepy.tech/projects/froog">16k pip installs!</a>) meant for those looking to get into machine learning and who want to understand how the underlying machine learning framework's code works before they are ultra-optimized (which all modern ml libraries are).
+```froog``` is an easy-to-read tensor library (<a href="https://www.pepy.tech/projects/froog">25k pip installs!</a>) with OpenCL support for GPU acceleration. Inspired by pytorch, tinygrad, and micrograd.
 
-```froog``` encapsulates everything from <a href="https://github.com/kevbuh/froog/blob/main/models/linear_regression.py">linear regression</a> to <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">convolutional neural networks </a> in under 1000 lines.
+
+<!-- ```froog``` encapsulates everything from <a href="https://github.com/kevbuh/froog/blob/main/models/linear_regression.py">linear regression</a> to <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">convolutional neural networks </a> in under 2000 lines. -->
 
 # Installation
 ```bash
@@ -85,7 +82,7 @@ from froog.tensor import Tensor
 my_tensor = Tensor([1,2,3])
 ```
 
-Notice how we had to import numpy. If you want to create a Tensor manually, make sure that it is a Numpy array!
+Notice how we had to import NumPy. If you want to create a Tensor manually, make sure that it is a NumPy array!
 
 <!-- Learn more about ```froog``` Tensors <a href="https://github.com/kevbuh/froog/blob/main/docs/tensors.md">here</a>. -->
 
@@ -95,13 +92,10 @@ Tensors are the fundamental datatype in froog, and one of the two main classes.
 
 - ```def __init__(self, data)```:
 
-  - Tensor takes in one param, which is the data. Since froog has a numpy backend, the input data into tensors has to be a numpy array.
-
+  - Tensor takes in one param, which is the data. Since ```froog``` has a NumPy backend, the input data into tensors has to be a NumPy array.
   - Tensor has a ```self.data``` state that it holds. this contains the data inside of the tensor.
-
   - In addition, it has ```self.grad```. this is to hold what the gradients of the tensor is. 
-
-  - Lastly, it has ```self._ctx```. theser are the internal vairables used for autograd graph construction. put more simply, this is where the backward gradient computations are saved. 
+  - Lastly, it has ```self._ctx```. These are the internal variables used for autograd graph construction. This is where the backward gradient computations are saved. 
 
 *Properties*
 
@@ -109,38 +103,34 @@ Tensors are the fundamental datatype in froog, and one of the two main classes.
 
 *Methods*
 - ```def zeros(*shape)```: this returns a tensor full of zeros with any shape that you pass in. Defaults to np.float32
-
 - ```def ones(*shape)```: this returns a tensor full of ones with any shape that you pass in. Defaults to np.float32
-
 - ```def randn(*shape):```: this returns a randomly initialized Tensor of *shape
 
 *Gradient calculations*
 
-- ```froog``` computes gradients automatically through a process called automatic differentiation. it has a variable ```_ctx```, which stores the chain of operations. it will take the current operation, lets say a dot product, and go to the dot product definition in ```froog/ops.py```, which contains a backward pass specfically for dot products. all methods, from add to 2x2 maxpools, have this backward pass implemented.
+- ```froog``` computes gradients automatically through a process called automatic differentiation. it has a variable ```_ctx```, which stores the chain of operations. It will take the current operation, let's say a dot product, and go to the dot product definition in ```froog/ops.py```, which contains a backward pass specifically for dot products. all methods, from add to 2x2 maxpools, have this backward pass implemented.
 
 *Functions*
 
 The other base class in froog is the class ```Function```. It keeps track of input tensors and tensors that need to be saved for backward passes
 
 - ```def __init__(self, *tensors)```: takes in an argument of tensors, which are then saved. 
-
 - ```def save_for_backward(self, *x)```: saves Tensors that are necessary to compute for the computation of gradients in the backward pass. 
-
-- ```def apply(self, arg, *x)```: This is what makes everything work. The apply() method takes care of the forward pass, applying the operation to the inputs.
+- ```def apply(self, arg, *x)```: takes care of the forward pass, applying the operation to the inputs.
 
 *Register*
 
-```def register(name, fxn)```: this function allows you to add a method to a Tensor. This allows you to chain any operations, e.g. x.dot(w).relu(), where w is a tensor
+- ```def register(name, fxn)```: allows you to add a method to a Tensor. This allows you to chain any operations, e.g. x.dot(w).relu(), where w is a tensor
 
 # Creating a model
 
 Okay cool, so now you know that ```froog```'s main datatype is a Tensor and uses NumPy in the background. How do I actually build a model? 
 
-Here's an example of how to create an MNIST multi-layer perceptron (MLP). We wanted to make it as simple as possible for you to do so so it resembles very basic python concepts like classes. There's really only two methods you need to define: 
+Here's an example of how to create an MNIST multi-layer perceptron (MLP). We wanted to make it as simple as possible for you to do so it resembles very basic Python concepts like classes. There are really only two methods you need to define: 
 1. ```__init__``` that defines layers of the model (here we use ```Linear```) 
 2. ```forward``` which defines how the input should flow through your model. We use a simple dot product with a ```Linear``` layer with a <a href="https://en.wikipedia.org/wiki/Rectifier_(neural_networks)">```ReLU```</a> activation.
 
-In order to create an instance of the ```mnistMLP``` model, do the same as you would in python: ```model = mnistMLP()``` . 
+To create an instance of the ```mnistMLP``` model, do the same as you would in Python: ```model = mnistMLP()```. 
 
 We support a few different optimizers, <a href="https://github.com/kevbuh/froog/blob/main/froog/optim.py">here</a> which include:
 - <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent">Stochastic Gradient Descent (SGD)</a>
@@ -199,9 +189,9 @@ So there are two quick examples to get you up and running. You might have notice
 - ```.max_pool2d()```
 - ```.avg_pool2d()```
 
-## GPU Support
+# GPU Support
 
-Have a GPU and need a speedup? You're in good luck because we have GPU support from for our operations defined in <a href="https://github.com/kevbuh/froog/blob/main/froog/ops_gpu.py">```ops_gpu.py```</a>. In order to do this we have a backend built on <a href="https://en.wikipedia.org/wiki/OpenGL">OpenGL</a> that invokes kernel functions that work on the GPU.
+Have a GPU and need a speedup? You're in good luck because we have GPU support via OpenCL for our operations defined in <a href="https://github.com/kevbuh/froog/blob/main/froog/ops_gpu.py">```ops_gpu.py```</a>.
 
 Here's how you can send data to the GPU during a forward pass and bring it back to the CPU.
 
@@ -212,75 +202,19 @@ if GPU:
   out = model.forward(Tensor(img).to_gpu()).cpu()
 ```
 
-## EfficientNet in froog!
+# EfficientNet in froog!
+
+<img src="assets/efficientnet_pug.png" alt="pug" height="300">
 
 We have a really cool finished implementation of EfficientNet built entirely in ```froog```!
 
 In order to run EfficientNet inference:
 
 ```bash
-VIZ=1 python models/efficientnet.py <https://put_your_image_url_here>
+VIZ=1 python3 models/efficientnet.py <https://put_your_image_url_here>
 ```
 
-I would recommend checking out the <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">code</a>, it's highly documented and pretty cool. Here's some of the documentation
-```
-Paper           : https://arxiv.org/abs/1905.11946
-PyTorch version : https://github.com/lukemelas/EfficientNet-PyTorch/blob/master/efficientnet_pytorch/model.py
-
-ConvNets are commonly developed at a fixed resource cost, and then scaled up in order to achieve better accuracy when more resources are made available
-The scaling method was found by performing a grid search to find the relationship between different scaling dimensions of the baseline network under a fixed resource constraint
-"SE" stands for "Squeeze-and-Excitation." Introduced by the "Squeeze-and-Excitation Networks" paper by Jie Hu, Li Shen, and Gang Sun (CVPR 2018).
-
-Environment Variables:
-  VIZ=1 --> plots processed image and output probabilities
-
-How to Run:
-  'VIZ=1 python models/efficientnet.py https://your_image_url'
-
-EfficientNet Hyper-Parameters and Weights:
-url_map = {
-    'efficientnet-b0': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b0-355c32eb.pth',
-    'efficientnet-b1': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b1-f1951068.pth',
-    'efficientnet-b2': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b2-8bb594d6.pth',
-    'efficientnet-b3': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b3-5fb5a3c3.pth',
-    'efficientnet-b4': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b4-6ed6700e.pth',
-    'efficientnet-b5': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b5-b6417697.pth',
-    'efficientnet-b6': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b6-c76e70fd.pth',
-    'efficientnet-b7': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b7-dcc49843.pth',
-}
-
-params_dict = {
-        # Coefficients:   width,depth,res,dropout
-        'efficientnet-b0': (1.0, 1.0, 224, 0.2),
-        'efficientnet-b1': (1.0, 1.1, 240, 0.2),
-        'efficientnet-b2': (1.1, 1.2, 260, 0.3),
-        'efficientnet-b3': (1.2, 1.4, 300, 0.3),
-        'efficientnet-b4': (1.4, 1.8, 380, 0.4),
-        'efficientnet-b5': (1.6, 2.2, 456, 0.4),
-        'efficientnet-b6': (1.8, 2.6, 528, 0.5),
-        'efficientnet-b7': (2.0, 3.1, 600, 0.5),
-        'efficientnet-b8': (2.2, 3.6, 672, 0.5),
-        'efficientnet-l2': (4.3, 5.3, 800, 0.5),
-    }
-
-blocks_args = [
-        'r1_k3_s11_e1_i32_o16_se0.25',
-        'r2_k3_s22_e6_i16_o24_se0.25',
-        'r2_k5_s22_e6_i24_o40_se0.25',
-        'r3_k3_s22_e6_i40_o80_se0.25',
-        'r3_k5_s11_e6_i80_o112_se0.25',
-        'r4_k5_s22_e6_i112_o192_se0.25',
-        'r1_k3_s11_e6_i192_o320_se0.25',
-    ]
-```
-
-## Linear regression 
-
-Doing linear regression in ```froog``` is pretty easy, check out the entire <a href="https://github.com/kevbuh/froog/blob/main/models/linear_regression.py">code</a>.
-
-```bash
-VIZ=1 python3 linear_regression.py
-```
+I would recommend checking out the <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">code</a>, it's highly documented and pretty cool.
 
 # Contributing
 <!-- THERES LOT OF STUFF TO WORK ON! VISIT THE <a href="https://github.com/kevbuh/froog/blob/main/docs/bounties.md">BOUNTY SHOP</a>  -->
@@ -290,12 +224,7 @@ Pull requests will be merged if they:
 * increase functionality
 * increase efficiency
 
-More info on <a href="https://github.com/kevbuh/froog/blob/main/docs/contributing.md">contributing</a>.
-
-# Documentation
-
-Need more information about how ```froog``` works? Visit the <a href="https://github.com/kevbuh/froog/tree/main/docs">documentation</a>.
-
-# Interested in more?
+More info on <a href="https://github.com/kevbuh/froog/blob/main/docs/contributing.md">contributing</a>. Make sure to run ```python -m pytest``` before creating a PR.
 
-If you thought ```froog``` was cool, check out the inspirations for this project: pytorch, tinygrad, and https://github.com/karpathy/micrograd/blob/master/micrograd/engine.py
+<!-- # Documentation
+Need more information about how ```froog``` works? Visit the <a href="https://github.com/kevbuh/froog/tree/main/docs">documentation</a>. -->

{froog-0.3.2 → froog-0.4.2}/README.md

@@ -11,9 +11,10 @@
   <br/>
 </div>
 
-```froog``` is an easy-to-read tensor library (<a href="https://www.pepy.tech/projects/froog">16k pip installs!</a>) meant for those looking to get into machine learning and who want to understand how the underlying machine learning framework's code works before they are ultra-optimized (which all modern ml libraries are).
+```froog``` is an easy-to-read tensor library (<a href="https://www.pepy.tech/projects/froog">25k pip installs!</a>) with OpenCL support for GPU acceleration. Inspired by pytorch, tinygrad, and micrograd.
 
-```froog``` encapsulates everything from <a href="https://github.com/kevbuh/froog/blob/main/models/linear_regression.py">linear regression</a> to <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">convolutional neural networks </a> in under 1000 lines.
+
+<!-- ```froog``` encapsulates everything from <a href="https://github.com/kevbuh/froog/blob/main/models/linear_regression.py">linear regression</a> to <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">convolutional neural networks </a> in under 2000 lines. -->
 
 # Installation
 ```bash
@@ -69,7 +70,7 @@ from froog.tensor import Tensor
 my_tensor = Tensor([1,2,3])
 ```
 
-Notice how we had to import numpy. If you want to create a Tensor manually, make sure that it is a Numpy array!
+Notice how we had to import NumPy. If you want to create a Tensor manually, make sure that it is a NumPy array!
 
 <!-- Learn more about ```froog``` Tensors <a href="https://github.com/kevbuh/froog/blob/main/docs/tensors.md">here</a>. -->
 
@@ -79,13 +80,10 @@ Tensors are the fundamental datatype in froog, and one of the two main classes.
 
 - ```def __init__(self, data)```:
 
-  - Tensor takes in one param, which is the data. Since froog has a numpy backend, the input data into tensors has to be a numpy array.
-
+  - Tensor takes in one param, which is the data. Since ```froog``` has a NumPy backend, the input data into tensors has to be a NumPy array.
   - Tensor has a ```self.data``` state that it holds. this contains the data inside of the tensor.
-
   - In addition, it has ```self.grad```. this is to hold what the gradients of the tensor is. 
-
-  - Lastly, it has ```self._ctx```. theser are the internal vairables used for autograd graph construction. put more simply, this is where the backward gradient computations are saved. 
+  - Lastly, it has ```self._ctx```. These are the internal variables used for autograd graph construction. This is where the backward gradient computations are saved. 
 
 *Properties*
 
@@ -93,38 +91,34 @@ Tensors are the fundamental datatype in froog, and one of the two main classes.
 
 *Methods*
 - ```def zeros(*shape)```: this returns a tensor full of zeros with any shape that you pass in. Defaults to np.float32
-
 - ```def ones(*shape)```: this returns a tensor full of ones with any shape that you pass in. Defaults to np.float32
-
 - ```def randn(*shape):```: this returns a randomly initialized Tensor of *shape
 
 *Gradient calculations*
 
-- ```froog``` computes gradients automatically through a process called automatic differentiation. it has a variable ```_ctx```, which stores the chain of operations. it will take the current operation, lets say a dot product, and go to the dot product definition in ```froog/ops.py```, which contains a backward pass specfically for dot products. all methods, from add to 2x2 maxpools, have this backward pass implemented.
+- ```froog``` computes gradients automatically through a process called automatic differentiation. it has a variable ```_ctx```, which stores the chain of operations. It will take the current operation, let's say a dot product, and go to the dot product definition in ```froog/ops.py```, which contains a backward pass specifically for dot products. all methods, from add to 2x2 maxpools, have this backward pass implemented.
 
 *Functions*
 
 The other base class in froog is the class ```Function```. It keeps track of input tensors and tensors that need to be saved for backward passes
 
 - ```def __init__(self, *tensors)```: takes in an argument of tensors, which are then saved. 
-
 - ```def save_for_backward(self, *x)```: saves Tensors that are necessary to compute for the computation of gradients in the backward pass. 
-
-- ```def apply(self, arg, *x)```: This is what makes everything work. The apply() method takes care of the forward pass, applying the operation to the inputs.
+- ```def apply(self, arg, *x)```: takes care of the forward pass, applying the operation to the inputs.
 
 *Register*
 
-```def register(name, fxn)```: this function allows you to add a method to a Tensor. This allows you to chain any operations, e.g. x.dot(w).relu(), where w is a tensor
+- ```def register(name, fxn)```: allows you to add a method to a Tensor. This allows you to chain any operations, e.g. x.dot(w).relu(), where w is a tensor
 
 # Creating a model
 
 Okay cool, so now you know that ```froog```'s main datatype is a Tensor and uses NumPy in the background. How do I actually build a model? 
 
-Here's an example of how to create an MNIST multi-layer perceptron (MLP). We wanted to make it as simple as possible for you to do so so it resembles very basic python concepts like classes. There's really only two methods you need to define: 
+Here's an example of how to create an MNIST multi-layer perceptron (MLP). We wanted to make it as simple as possible for you to do so it resembles very basic Python concepts like classes. There are really only two methods you need to define: 
 1. ```__init__``` that defines layers of the model (here we use ```Linear```) 
 2. ```forward``` which defines how the input should flow through your model. We use a simple dot product with a ```Linear``` layer with a <a href="https://en.wikipedia.org/wiki/Rectifier_(neural_networks)">```ReLU```</a> activation.
 
-In order to create an instance of the ```mnistMLP``` model, do the same as you would in python: ```model = mnistMLP()``` . 
+To create an instance of the ```mnistMLP``` model, do the same as you would in Python: ```model = mnistMLP()```. 
 
 We support a few different optimizers, <a href="https://github.com/kevbuh/froog/blob/main/froog/optim.py">here</a> which include:
 - <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent">Stochastic Gradient Descent (SGD)</a>
@@ -183,9 +177,9 @@ So there are two quick examples to get you up and running. You might have notice
 - ```.max_pool2d()```
 - ```.avg_pool2d()```
 
-## GPU Support
+# GPU Support
 
-Have a GPU and need a speedup? You're in good luck because we have GPU support from for our operations defined in <a href="https://github.com/kevbuh/froog/blob/main/froog/ops_gpu.py">```ops_gpu.py```</a>. In order to do this we have a backend built on <a href="https://en.wikipedia.org/wiki/OpenGL">OpenGL</a> that invokes kernel functions that work on the GPU.
+Have a GPU and need a speedup? You're in good luck because we have GPU support via OpenCL for our operations defined in <a href="https://github.com/kevbuh/froog/blob/main/froog/ops_gpu.py">```ops_gpu.py```</a>.
 
 Here's how you can send data to the GPU during a forward pass and bring it back to the CPU.
 
@@ -196,75 +190,19 @@ if GPU:
   out = model.forward(Tensor(img).to_gpu()).cpu()
 ```
 
-## EfficientNet in froog!
+# EfficientNet in froog!
+
+<img src="assets/efficientnet_pug.png" alt="pug" height="300">
 
 We have a really cool finished implementation of EfficientNet built entirely in ```froog```!
 
 In order to run EfficientNet inference:
 
 ```bash
-VIZ=1 python models/efficientnet.py <https://put_your_image_url_here>
+VIZ=1 python3 models/efficientnet.py <https://put_your_image_url_here>
 ```
 
-I would recommend checking out the <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">code</a>, it's highly documented and pretty cool. Here's some of the documentation
-```
-Paper           : https://arxiv.org/abs/1905.11946
-PyTorch version : https://github.com/lukemelas/EfficientNet-PyTorch/blob/master/efficientnet_pytorch/model.py
-
-ConvNets are commonly developed at a fixed resource cost, and then scaled up in order to achieve better accuracy when more resources are made available
-The scaling method was found by performing a grid search to find the relationship between different scaling dimensions of the baseline network under a fixed resource constraint
-"SE" stands for "Squeeze-and-Excitation." Introduced by the "Squeeze-and-Excitation Networks" paper by Jie Hu, Li Shen, and Gang Sun (CVPR 2018).
-
-Environment Variables:
-  VIZ=1 --> plots processed image and output probabilities
-
-How to Run:
-  'VIZ=1 python models/efficientnet.py https://your_image_url'
-
-EfficientNet Hyper-Parameters and Weights:
-url_map = {
-    'efficientnet-b0': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b0-355c32eb.pth',
-    'efficientnet-b1': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b1-f1951068.pth',
-    'efficientnet-b2': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b2-8bb594d6.pth',
-    'efficientnet-b3': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b3-5fb5a3c3.pth',
-    'efficientnet-b4': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b4-6ed6700e.pth',
-    'efficientnet-b5': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b5-b6417697.pth',
-    'efficientnet-b6': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b6-c76e70fd.pth',
-    'efficientnet-b7': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b7-dcc49843.pth',
-}
-
-params_dict = {
-        # Coefficients:   width,depth,res,dropout
-        'efficientnet-b0': (1.0, 1.0, 224, 0.2),
-        'efficientnet-b1': (1.0, 1.1, 240, 0.2),
-        'efficientnet-b2': (1.1, 1.2, 260, 0.3),
-        'efficientnet-b3': (1.2, 1.4, 300, 0.3),
-        'efficientnet-b4': (1.4, 1.8, 380, 0.4),
-        'efficientnet-b5': (1.6, 2.2, 456, 0.4),
-        'efficientnet-b6': (1.8, 2.6, 528, 0.5),
-        'efficientnet-b7': (2.0, 3.1, 600, 0.5),
-        'efficientnet-b8': (2.2, 3.6, 672, 0.5),
-        'efficientnet-l2': (4.3, 5.3, 800, 0.5),
-    }
-
-blocks_args = [
-        'r1_k3_s11_e1_i32_o16_se0.25',
-        'r2_k3_s22_e6_i16_o24_se0.25',
-        'r2_k5_s22_e6_i24_o40_se0.25',
-        'r3_k3_s22_e6_i40_o80_se0.25',
-        'r3_k5_s11_e6_i80_o112_se0.25',
-        'r4_k5_s22_e6_i112_o192_se0.25',
-        'r1_k3_s11_e6_i192_o320_se0.25',
-    ]
-```
-
-## Linear regression 
-
-Doing linear regression in ```froog``` is pretty easy, check out the entire <a href="https://github.com/kevbuh/froog/blob/main/models/linear_regression.py">code</a>.
-
-```bash
-VIZ=1 python3 linear_regression.py
-```
+I would recommend checking out the <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">code</a>, it's highly documented and pretty cool.
 
 # Contributing
 <!-- THERES LOT OF STUFF TO WORK ON! VISIT THE <a href="https://github.com/kevbuh/froog/blob/main/docs/bounties.md">BOUNTY SHOP</a>  -->
@@ -274,12 +212,7 @@ Pull requests will be merged if they:
 * increase functionality
 * increase efficiency
 
-More info on <a href="https://github.com/kevbuh/froog/blob/main/docs/contributing.md">contributing</a>.
-
-# Documentation
-
-Need more information about how ```froog``` works? Visit the <a href="https://github.com/kevbuh/froog/tree/main/docs">documentation</a>.
-
-# Interested in more?
+More info on <a href="https://github.com/kevbuh/froog/blob/main/docs/contributing.md">contributing</a>. Make sure to run ```python -m pytest``` before creating a PR.
 
-If you thought ```froog``` was cool, check out the inspirations for this project: pytorch, tinygrad, and https://github.com/karpathy/micrograd/blob/master/micrograd/engine.py
+<!-- # Documentation
+Need more information about how ```froog``` works? Visit the <a href="https://github.com/kevbuh/froog/tree/main/docs">documentation</a>. -->

{froog-0.3.2 → froog-0.4.2}/froog/ops.py

@@ -9,6 +9,7 @@
 import numpy as np
 from froog.tensor import Function, register
 from froog.utils import im2col, col2im
+from froog.tensor import Tensor
 
 # *****************************************************
 #     ____  ___   _____ __________   ____  ____  _____
@@ -142,6 +143,29 @@ class Sigmoid(Function):
     return grad_input
 register("sigmoid", Sigmoid)
 
+# class Dropout(Function):
+#   """
+#   Randomly zeroes some of the elements of the input tensor with probability p during training.
+#   The elements to zero are randomized on every forward call.
+#   During inference, dropout is disabled and the input is scaled by (1-p) to maintain the expected value.
+#   """
+#   @staticmethod
+#   def forward(ctx, input, p=0.5, training=True):
+#     if training:
+#       # Create a binary mask with probability (1-p) of being 1
+#       mask = (np.random.random(input.shape) > p).astype(np.float32)
+#       ctx.save_for_backward(mask)
+#       return input * mask
+#     else:
+#       # during inference, scale the input by (1-p)
+#       return input * (1-p)
+
+#   @staticmethod
+#   def backward(ctx, grad_output):
+#     mask, = ctx.saved_tensors
+#     return grad_output * mask
+# register("dropout", Dropout)
+
 class Reshape(Function):
   @staticmethod
   def forward(ctx, x, shape):
@@ -358,3 +382,53 @@ class AvgPool2D(Function):
         ret[:, :, Y:my:py, X:mx:px] = grad_output / py / px   # divide by avg of pool, e.g. for 2x2 pool /= 4
     return ret
 register('avg_pool2d', AvgPool2D)
+
+# *************************************
+#     _   ___   __   ____  ____  _____
+#    / | / / | / /  / __ \/ __ \/ ___/
+#   /  |/ /  |/ /  / / / / /_/ /\__ \ 
+#  / /|  / /|  /  / /_/ / ____/___/ / 
+# /_/ |_/_/ |_/   \____/_/    /____/  
+#
+# ************* nn ops ************   
+
+def Linear(*x):
+  # random Glorot initialization
+  ret = np.random.uniform(-1., 1., size=x)/np.sqrt(np.prod(x))
+  return ret.astype(np.float32)
+
+def swish(x):
+  return x.mul(x.sigmoid())
+
+class BatchNorm2D:
+  """
+  __call__ follows the formula from the link below
+  pytorch version: https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html
+
+  self.weight       = γ
+  self.bias         = β
+  self.running_mean = E[x] 
+  self.running_var  = Var[x]
+
+  the reshaping step ensures that each channel of the input has its 
+  own separate set of parameters (mean, variance, weight, and bias)
+
+  self.running_mean has shape [num_channels].
+  self.running_mean.reshape(shape=[1, -1, 1, 1]) reshapes it to [1, num_channels, 1, 1]
+  """
+  def __init__(self, sz, eps=0.001):
+    self.eps = eps
+    self.weight = Tensor.zeros(sz)
+    self.bias = Tensor.zeros(sz)
+
+    # TODO: need running_mean and running_var
+    self.running_mean = Tensor.zeros(sz)
+    self.running_var = Tensor.zeros(sz)
+    self.num_batches_tracked = Tensor.zeros(1)
+
+  def __call__(self, x):
+    x = x.sub(self.running_mean.reshape(shape=[1, -1, 1, 1]))
+    x = x.mul(self.weight.reshape(shape=[1, -1, 1, 1]))
+    x = x.div(self.running_var.add(Tensor([self.eps], gpu=x.gpu)).reshape(shape=[1, -1, 1, 1]).sqrt())
+    x = x.add(self.bias.reshape(shape=[1, -1, 1, 1]))
+    return x

{froog-0.3.2 → froog-0.4.2}/froog/ops_gpu.py

@@ -303,23 +303,42 @@ register('relu', ReLU, gpu=True)
 class LogSoftmax(Function):
   @staticmethod
   def forward(ctx, input):
+    # first find max values for numerical stability
+    max_vals = buffer_new(ctx, (input.shape[0],))
+    prg = clbuild(ctx.cl_ctx, """
+    __kernel void max_vals(
+        __global const float *a_g, int sz, __global float *res_g)
+    {
+      int gid = get_global_id(0);
+      int gidsz = gid*sz;
+      float max_val = -INFINITY;
+      for (int x = 0; x < sz; x++) {
+        max_val = max(max_val, a_g[gidsz+x]);
+      }
+      res_g[gid] = max_val;
+    }
+    """)
+    prg.max_vals(ctx.cl_queue, [input.shape[0]], None, input, np.int32(input.shape[1]), max_vals)
+
+    # compute exp(x - max) and sum
     lsum = buffer_new(ctx, (input.shape[0],))
     prg = clbuild(ctx.cl_ctx, """
     __kernel void logsoftmax(
-        __global const float *a_g, int sz, __global float *res_g)
+        __global const float *a_g, __global const float *max_vals, int sz, __global float *res_g)
     {
       int gid = get_global_id(0);
       int gidsz = gid*sz;
-      // TODO: stability with max
+      float max_val = max_vals[gid];
       float out = 0.0;
       for (int x = 0; x < sz; x++) {
-        out += exp(a_g[gidsz+x]);
+        out += exp(a_g[gidsz+x] - max_val);
       }
-      res_g[gid] = log(out);
+      res_g[gid] = log(out) + max_val;
     }
     """)
-    prg.logsoftmax(ctx.cl_queue, [input.shape[0]], None, input, np.int32(input.shape[1]), lsum)
+    prg.logsoftmax(ctx.cl_queue, [input.shape[0]], None, input, max_vals, np.int32(input.shape[1]), lsum)
 
+    # compute final output
     output = buffer_like(ctx, input)
     prg = clbuild(ctx.cl_ctx, """
     __kernel void lsmsub(
@@ -475,8 +494,38 @@ class AvgPool2D(Function):
 
   @staticmethod
   def backward(ctx, grad_output):
-    # TODO Finish this
-    pass
+    # for average pooling, we need to distribute the gradient evenly across all elements in the pooling window
+    input_shape = ctx.data.shape
+    N, C, Y, X = input_shape
+    py, px = ctx.kernel_size
+    ret = buffer_zeros(ctx, input_shape)
+    
+    prg = clbuild(ctx.cl_ctx, """
+    __kernel void avgpool_backward(
+      __global float *grad_input, __global const float *grad_output,
+      uint2 osize, uint2 isize, uint2 kernel_size, int nelem
+    ) {
+      int3 gid = (int3)(get_global_id(2), get_global_id(1), get_global_id(0));
+      int oid = gid.x + osize.x*(gid.y + osize.y*gid.z);
+      float grad = grad_output[oid] / (kernel_size.x * kernel_size.y);
+      
+      for (uint j=0; j<kernel_size.y; ++j) {
+        for (uint i=0; i<kernel_size.x; ++i) {
+          int iid = (gid.x*kernel_size.x+i) + isize.x*((gid.y*kernel_size.y+j) + isize.y*gid.z);
+          if (iid < nelem)
+            grad_input[iid] += grad;
+        }
+      }
+    }
+    """)
+    
+    osize = np.array((X//px, Y//py), dtype=cl.cltypes.uint2)
+    isize = np.array((X, Y), dtype=cl.cltypes.uint2)
+    ksize = np.array((px,py), dtype=cl.cltypes.uint2)
+    
+    prg.avgpool_backward(ctx.cl_queue, (N*C, Y//py, X//px), None, ret, grad_output, osize, isize, ksize, np.int32(input_shape.size))
+    
+    return ret
 register('avg_pool2d', AvgPool2D, gpu=True)
 
 class MaxPool2D(Function):
@@ -485,10 +534,65 @@ class MaxPool2D(Function):
     init_val = "FLT_MIN"
     iter_op = "group_res = max(group_res, input[iid])"
     result_op = "group_res"
-    return pooling_op(ctx, input, kernel_size, iter_op, result_op, init_val=init_val)
+    ret = pooling_op(ctx, input, kernel_size, iter_op, result_op, init_val=init_val)
+    
+    # save indices of max elements for backward pass
+    indices = buffer_new(ctx, ret.shape)
+    prg = clbuild(ctx.cl_ctx, """
+    __kernel void maxpool_indices(
+      __global const float *input, __global float *output, __global int *indices,
+      uint2 osize, uint2 isize, uint2 kernel_size, int nelem
+    ) {
+      int3 gid = (int3)(get_global_id(2), get_global_id(1), get_global_id(0));
+      int oid = gid.x + osize.x*(gid.y + osize.y*gid.z);
+      float max_val = -INFINITY;
+      int max_idx = 0;
+      
+      for (uint j=0; j<kernel_size.y; ++j) {
+        for (uint i=0; i<kernel_size.x; ++i) {
+          int iid = (gid.x*kernel_size.x+i) + isize.x*((gid.y*kernel_size.y+j) + isize.y*gid.z);
+          if (iid < nelem) {
+            float val = input[iid];
+            if (val > max_val) {
+              max_val = val;
+              max_idx = iid;
+            }
+          }
+        }
+      }
+      indices[oid] = max_idx;
+    }
+    """)
+    
+    N, C, Y, X = input.shape
+    py, px = kernel_size
+    osize = np.array((X//px, Y//py), dtype=cl.cltypes.uint2)
+    isize = np.array((X, Y), dtype=cl.cltypes.uint2)
+    ksize = np.array((px,py), dtype=cl.cltypes.uint2)
+    
+    prg.maxpool_indices(ctx.cl_queue, (N*C, Y//py, X//px), None, input, ret, indices, osize, isize, ksize, np.int32(input.size))
+    
+    ctx.save_for_backward(indices)
+    return ret
 
   @staticmethod
   def backward(ctx, grad_output):
-    # TODO Finish this
-    pass
+    indices, = ctx.saved_tensors
+    input_shape = ctx.data.shape
+    ret = buffer_zeros(ctx, input_shape)
+    prg = clbuild(ctx.cl_ctx, """
+    __kernel void maxpool_backward(
+      __global float *grad_input, __global const float *grad_output,
+      __global const int *indices, int nelem
+    ) {
+      int gid = get_global_id(0);
+      if (gid < nelem) {
+        int idx = indices[gid];
+        grad_input[idx] += grad_output[gid];
+      }
+    }
+    """)
+  
+    prg.maxpool_backward(ctx.cl_queue, [np.prod(grad_output.shape)], None, ret, grad_output, indices, np.int32(grad_output.size))
+    return ret
 register('max_pool2d', MaxPool2D, gpu=True)

{froog-0.3.2 → froog-0.4.2}/froog/optim.py

@@ -57,7 +57,7 @@ class RMSprop(Optimizer):
   RMSprop divides the learning rate by an exponentially decaying average of squared gradients.
 
   Notes: 
-  The reason RPROP doesn’t work is that it violates the central idea behind stochastic gradient descent, 
+  The reason RPROP doesn't work is that it violates the central idea behind stochastic gradient descent, 
   which is when we have small enough learning rate, it averages the gradients over successive mini-batches.
   """
   def __init__(self, params, decay=0.9, lr=0.001, eps=1e-8):

{froog-0.3.2 → froog-0.4.2}/froog.egg-info/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.1
 Name: froog
-Version: 0.3.2
-Summary: a beautifully simplistic tensor library
+Version: 0.4.2
+Summary: a toy tensor library with opencl support
 Author: Kevin Buhler
 License: MIT
 Classifier: Programming Language :: Python :: 3
@@ -9,10 +9,6 @@ Classifier: License :: OSI Approved :: MIT License
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: numpy
-Requires-Dist: requests
-Requires-Dist: matplotlib
-Requires-Dist: urllib
 
 # froog <img src="https://github.com/kevbuh/froog/actions/workflows/test.yml/badge.svg" alt="unit test badge" > <img src="https://static.pepy.tech/badge/froog" alt="num downloads badge">
 <div align="center" >
@@ -27,9 +23,10 @@ Requires-Dist: urllib
   <br/>
 </div>
 
-```froog``` is an easy-to-read tensor library (<a href="https://www.pepy.tech/projects/froog">16k pip installs!</a>) meant for those looking to get into machine learning and who want to understand how the underlying machine learning framework's code works before they are ultra-optimized (which all modern ml libraries are).
+```froog``` is an easy-to-read tensor library (<a href="https://www.pepy.tech/projects/froog">25k pip installs!</a>) with OpenCL support for GPU acceleration. Inspired by pytorch, tinygrad, and micrograd.
 
-```froog``` encapsulates everything from <a href="https://github.com/kevbuh/froog/blob/main/models/linear_regression.py">linear regression</a> to <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">convolutional neural networks </a> in under 1000 lines.
+
+<!-- ```froog``` encapsulates everything from <a href="https://github.com/kevbuh/froog/blob/main/models/linear_regression.py">linear regression</a> to <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">convolutional neural networks </a> in under 2000 lines. -->
 
 # Installation
 ```bash
@@ -85,7 +82,7 @@ from froog.tensor import Tensor
 my_tensor = Tensor([1,2,3])
 ```
 
-Notice how we had to import numpy. If you want to create a Tensor manually, make sure that it is a Numpy array!
+Notice how we had to import NumPy. If you want to create a Tensor manually, make sure that it is a NumPy array!
 
 <!-- Learn more about ```froog``` Tensors <a href="https://github.com/kevbuh/froog/blob/main/docs/tensors.md">here</a>. -->
 
@@ -95,13 +92,10 @@ Tensors are the fundamental datatype in froog, and one of the two main classes.
 
 - ```def __init__(self, data)```:
 
-  - Tensor takes in one param, which is the data. Since froog has a numpy backend, the input data into tensors has to be a numpy array.
-
+  - Tensor takes in one param, which is the data. Since ```froog``` has a NumPy backend, the input data into tensors has to be a NumPy array.
   - Tensor has a ```self.data``` state that it holds. this contains the data inside of the tensor.
-
   - In addition, it has ```self.grad```. this is to hold what the gradients of the tensor is. 
-
-  - Lastly, it has ```self._ctx```. theser are the internal vairables used for autograd graph construction. put more simply, this is where the backward gradient computations are saved. 
+  - Lastly, it has ```self._ctx```. These are the internal variables used for autograd graph construction. This is where the backward gradient computations are saved. 
 
 *Properties*
 
@@ -109,38 +103,34 @@ Tensors are the fundamental datatype in froog, and one of the two main classes.
 
 *Methods*
 - ```def zeros(*shape)```: this returns a tensor full of zeros with any shape that you pass in. Defaults to np.float32
-
 - ```def ones(*shape)```: this returns a tensor full of ones with any shape that you pass in. Defaults to np.float32
-
 - ```def randn(*shape):```: this returns a randomly initialized Tensor of *shape
 
 *Gradient calculations*
 
-- ```froog``` computes gradients automatically through a process called automatic differentiation. it has a variable ```_ctx```, which stores the chain of operations. it will take the current operation, lets say a dot product, and go to the dot product definition in ```froog/ops.py```, which contains a backward pass specfically for dot products. all methods, from add to 2x2 maxpools, have this backward pass implemented.
+- ```froog``` computes gradients automatically through a process called automatic differentiation. it has a variable ```_ctx```, which stores the chain of operations. It will take the current operation, let's say a dot product, and go to the dot product definition in ```froog/ops.py```, which contains a backward pass specifically for dot products. all methods, from add to 2x2 maxpools, have this backward pass implemented.
 
 *Functions*
 
 The other base class in froog is the class ```Function```. It keeps track of input tensors and tensors that need to be saved for backward passes
 
 - ```def __init__(self, *tensors)```: takes in an argument of tensors, which are then saved. 
-
 - ```def save_for_backward(self, *x)```: saves Tensors that are necessary to compute for the computation of gradients in the backward pass. 
-
-- ```def apply(self, arg, *x)```: This is what makes everything work. The apply() method takes care of the forward pass, applying the operation to the inputs.
+- ```def apply(self, arg, *x)```: takes care of the forward pass, applying the operation to the inputs.
 
 *Register*
 
-```def register(name, fxn)```: this function allows you to add a method to a Tensor. This allows you to chain any operations, e.g. x.dot(w).relu(), where w is a tensor
+- ```def register(name, fxn)```: allows you to add a method to a Tensor. This allows you to chain any operations, e.g. x.dot(w).relu(), where w is a tensor
 
 # Creating a model
 
 Okay cool, so now you know that ```froog```'s main datatype is a Tensor and uses NumPy in the background. How do I actually build a model? 
 
-Here's an example of how to create an MNIST multi-layer perceptron (MLP). We wanted to make it as simple as possible for you to do so so it resembles very basic python concepts like classes. There's really only two methods you need to define: 
+Here's an example of how to create an MNIST multi-layer perceptron (MLP). We wanted to make it as simple as possible for you to do so it resembles very basic Python concepts like classes. There are really only two methods you need to define: 
 1. ```__init__``` that defines layers of the model (here we use ```Linear```) 
 2. ```forward``` which defines how the input should flow through your model. We use a simple dot product with a ```Linear``` layer with a <a href="https://en.wikipedia.org/wiki/Rectifier_(neural_networks)">```ReLU```</a> activation.
 
-In order to create an instance of the ```mnistMLP``` model, do the same as you would in python: ```model = mnistMLP()``` . 
+To create an instance of the ```mnistMLP``` model, do the same as you would in Python: ```model = mnistMLP()```. 
 
 We support a few different optimizers, <a href="https://github.com/kevbuh/froog/blob/main/froog/optim.py">here</a> which include:
 - <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent">Stochastic Gradient Descent (SGD)</a>
@@ -199,9 +189,9 @@ So there are two quick examples to get you up and running. You might have notice
 - ```.max_pool2d()```
 - ```.avg_pool2d()```
 
-## GPU Support
+# GPU Support
 
-Have a GPU and need a speedup? You're in good luck because we have GPU support from for our operations defined in <a href="https://github.com/kevbuh/froog/blob/main/froog/ops_gpu.py">```ops_gpu.py```</a>. In order to do this we have a backend built on <a href="https://en.wikipedia.org/wiki/OpenGL">OpenGL</a> that invokes kernel functions that work on the GPU.
+Have a GPU and need a speedup? You're in good luck because we have GPU support via OpenCL for our operations defined in <a href="https://github.com/kevbuh/froog/blob/main/froog/ops_gpu.py">```ops_gpu.py```</a>.
 
 Here's how you can send data to the GPU during a forward pass and bring it back to the CPU.
 
@@ -212,75 +202,19 @@ if GPU:
   out = model.forward(Tensor(img).to_gpu()).cpu()
 ```
 
-## EfficientNet in froog!
+# EfficientNet in froog!
+
+<img src="assets/efficientnet_pug.png" alt="pug" height="300">
 
 We have a really cool finished implementation of EfficientNet built entirely in ```froog```!
 
 In order to run EfficientNet inference:
 
 ```bash
-VIZ=1 python models/efficientnet.py <https://put_your_image_url_here>
+VIZ=1 python3 models/efficientnet.py <https://put_your_image_url_here>
 ```
 
-I would recommend checking out the <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">code</a>, it's highly documented and pretty cool. Here's some of the documentation
-```
-Paper           : https://arxiv.org/abs/1905.11946
-PyTorch version : https://github.com/lukemelas/EfficientNet-PyTorch/blob/master/efficientnet_pytorch/model.py
-
-ConvNets are commonly developed at a fixed resource cost, and then scaled up in order to achieve better accuracy when more resources are made available
-The scaling method was found by performing a grid search to find the relationship between different scaling dimensions of the baseline network under a fixed resource constraint
-"SE" stands for "Squeeze-and-Excitation." Introduced by the "Squeeze-and-Excitation Networks" paper by Jie Hu, Li Shen, and Gang Sun (CVPR 2018).
-
-Environment Variables:
-  VIZ=1 --> plots processed image and output probabilities
-
-How to Run:
-  'VIZ=1 python models/efficientnet.py https://your_image_url'
-
-EfficientNet Hyper-Parameters and Weights:
-url_map = {
-    'efficientnet-b0': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b0-355c32eb.pth',
-    'efficientnet-b1': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b1-f1951068.pth',
-    'efficientnet-b2': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b2-8bb594d6.pth',
-    'efficientnet-b3': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b3-5fb5a3c3.pth',
-    'efficientnet-b4': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b4-6ed6700e.pth',
-    'efficientnet-b5': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b5-b6417697.pth',
-    'efficientnet-b6': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b6-c76e70fd.pth',
-    'efficientnet-b7': 'https://github.com/lukemelas/EfficientNet-PyTorch/releases/download/1.0/efficientnet-b7-dcc49843.pth',
-}
-
-params_dict = {
-        # Coefficients:   width,depth,res,dropout
-        'efficientnet-b0': (1.0, 1.0, 224, 0.2),
-        'efficientnet-b1': (1.0, 1.1, 240, 0.2),
-        'efficientnet-b2': (1.1, 1.2, 260, 0.3),
-        'efficientnet-b3': (1.2, 1.4, 300, 0.3),
-        'efficientnet-b4': (1.4, 1.8, 380, 0.4),
-        'efficientnet-b5': (1.6, 2.2, 456, 0.4),
-        'efficientnet-b6': (1.8, 2.6, 528, 0.5),
-        'efficientnet-b7': (2.0, 3.1, 600, 0.5),
-        'efficientnet-b8': (2.2, 3.6, 672, 0.5),
-        'efficientnet-l2': (4.3, 5.3, 800, 0.5),
-    }
-
-blocks_args = [
-        'r1_k3_s11_e1_i32_o16_se0.25',
-        'r2_k3_s22_e6_i16_o24_se0.25',
-        'r2_k5_s22_e6_i24_o40_se0.25',
-        'r3_k3_s22_e6_i40_o80_se0.25',
-        'r3_k5_s11_e6_i80_o112_se0.25',
-        'r4_k5_s22_e6_i112_o192_se0.25',
-        'r1_k3_s11_e6_i192_o320_se0.25',
-    ]
-```
-
-## Linear regression 
-
-Doing linear regression in ```froog``` is pretty easy, check out the entire <a href="https://github.com/kevbuh/froog/blob/main/models/linear_regression.py">code</a>.
-
-```bash
-VIZ=1 python3 linear_regression.py
-```
+I would recommend checking out the <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">code</a>, it's highly documented and pretty cool.
 
 # Contributing
 <!-- THERES LOT OF STUFF TO WORK ON! VISIT THE <a href="https://github.com/kevbuh/froog/blob/main/docs/bounties.md">BOUNTY SHOP</a>  -->
@@ -290,12 +224,7 @@ Pull requests will be merged if they:
 * increase functionality
 * increase efficiency
 
-More info on <a href="https://github.com/kevbuh/froog/blob/main/docs/contributing.md">contributing</a>.
-
-# Documentation
-
-Need more information about how ```froog``` works? Visit the <a href="https://github.com/kevbuh/froog/tree/main/docs">documentation</a>.
-
-# Interested in more?
+More info on <a href="https://github.com/kevbuh/froog/blob/main/docs/contributing.md">contributing</a>. Make sure to run ```python -m pytest``` before creating a PR.
 
-If you thought ```froog``` was cool, check out the inspirations for this project: pytorch, tinygrad, and https://github.com/karpathy/micrograd/blob/master/micrograd/engine.py
+<!-- # Documentation
+Need more information about how ```froog``` works? Visit the <a href="https://github.com/kevbuh/froog/tree/main/docs">documentation</a>. -->

{froog-0.3.2 → froog-0.4.2}/froog.egg-info/SOURCES.txt

@@ -3,7 +3,6 @@ README.md
 setup.py
 froog/__init__.py
 froog/gradcheck.py
-froog/nn.py
 froog/ops.py
 froog/ops_gpu.py
 froog/optim.py

{froog-0.3.2 → froog-0.4.2}/froog.egg-info/requires.txt

@@ -1,4 +1,3 @@
 numpy
 requests
 matplotlib
-urllib

{froog-0.3.2 → froog-0.4.2}/setup.py

@@ -1,4 +1,3 @@
-
 #!/usr/bin/env python3
 # this file specifies how the froog package is installed, including any necessary dependencies required to run
 
@@ -10,8 +9,8 @@ with open(os.path.join(directory, 'README.md'), encoding='utf-8') as f:
   long_description = f.read()
 
 setup(name='froog',
-      version='0.3.2',
-      description='a beautifully simplistic tensor library',
+      version='0.4.2',
+      description='a toy tensor library with opencl support',
       author='Kevin Buhler',
       license='MIT',
       long_description=long_description,
@@ -21,6 +20,6 @@ setup(name='froog',
         "Programming Language :: Python :: 3",
         "License :: OSI Approved :: MIT License"
       ],
-      install_requires=['numpy', 'requests', 'matplotlib', 'urllib'],
+      install_requires=['numpy', 'requests', 'matplotlib'],
       python_requires='>=3.8',
       include_package_data=True)

{froog-0.3.2 → froog-0.4.2}/tests/test_models.py

@@ -2,7 +2,7 @@ import numpy as np
 from tqdm import trange
 from froog.tensor import Tensor, GPU
 from froog.utils import fetch_mnist
-from froog.nn import Linear
+from froog.ops import Linear
 import froog.optim as optim
 import unittest
 import os

froog-0.3.2/froog/nn.py

@@ -1,60 +0,0 @@
-#  _______  ______    _______  _______  _______ 
-# |       ||    _ |  |       ||       ||       |
-# |    ___||   | ||  |   _   ||   _   ||    ___|
-# |   |___ |   |_||_ |  | |  ||  | |  ||   | __ 
-# |    ___||    __  ||  |_|  ||  |_|  ||   ||  |
-# |   |    |   |  | ||       ||       ||   |_| |
-# |___|    |___|  |_||_______||_______||_______|
-
-from froog.tensor import Tensor
-import numpy as np
-
-def Linear(*x):
-  # random Glorot initialization
-  ret = np.random.uniform(-1., 1., size=x)/np.sqrt(np.prod(x))
-  return ret.astype(np.float32)
-
-def swish(x):
-  return x.mul(x.sigmoid())
-
-# *************************************
-#     _   ___   __   ____  ____  _____
-#    / | / / | / /  / __ \/ __ \/ ___/
-#   /  |/ /  |/ /  / / / / /_/ /\__ \ 
-#  / /|  / /|  /  / /_/ / ____/___/ / 
-# /_/ |_/_/ |_/   \____/_/    /____/  
-#
-# ************* nn ops ************   
-
-class BatchNorm2D:
-  """
-  __call__ follows the formula from the link below
-  pytorch version: https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html
-
-  self.weight       = γ
-  self.bias         = β
-  self.running_mean = E[x] 
-  self.running_var  = Var[x]
-
-  the reshaping step ensures that each channel of the input has its 
-  own separate set of parameters (mean, variance, weight, and bias)
-
-  self.running_mean has shape [num_channels].
-  self.running_mean.reshape(shape=[1, -1, 1, 1]) reshapes it to [1, num_channels, 1, 1]
-  """
-  def __init__(self, sz, eps=0.001):
-    self.eps = eps
-    self.weight = Tensor.zeros(sz)
-    self.bias = Tensor.zeros(sz)
-
-    # TODO: need running_mean and running_var
-    self.running_mean = Tensor.zeros(sz)
-    self.running_var = Tensor.zeros(sz)
-    self.num_batches_tracked = Tensor.zeros(1)
-
-  def __call__(self, x):
-    x = x.sub(self.running_mean.reshape(shape=[1, -1, 1, 1]))
-    x = x.mul(self.weight.reshape(shape=[1, -1, 1, 1]))
-    x = x.div(self.running_var.add(Tensor([self.eps], gpu=x.gpu)).reshape(shape=[1, -1, 1, 1]).sqrt())
-    x = x.add(self.bias.reshape(shape=[1, -1, 1, 1]))
-    return x