froog 0.4.2__tar.gz → 0.5.0__tar.gz

froog-0.5.0/PKG-INFO

@@ -0,0 +1,202 @@
+Metadata-Version: 2.1
+Name: froog
+Version: 0.5.0
+Summary: tensor library with opencl and metal support
+Author: Kevin Buhler
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+
+# froog <img src="https://github.com/kevbuh/froog/actions/workflows/test.yml/badge.svg" alt="unit test badge" > <img src="https://static.pepy.tech/badge/froog" alt="num downloads badge">
+<div align="center" >
+  <img src="https://raw.githubusercontent.com/kevbuh/froog/main/assets/froog.png" alt="froog the frog" height="200">
+  <br/>
+  froog: fast real-time optimization of gradients 
+  <br/>
+  a beautifully compact tensor library
+  <br/>
+  <a href="https://github.com/kevbuh/froog">homepage</a> | <a href="https://github.com/kevbuh/froog/tree/main/DOCS.md">documentation</a> | <a href="https://pypi.org/project/froog/">pip</a>
+  <br/>
+  <br/>
+</div>
+
+```froog``` is an easy-to-read tensor library (<a href="https://www.pepy.tech/projects/froog">25k pip installs!</a>) with OpenCL support for GPU acceleration. Inspired by pytorch, tinygrad, and micrograd.
+
+# Installation
+```bash
+pip install froog
+```
+
+# Features
+- <a href="https://github.com/kevbuh/froog/blob/main/froog/tensor.py">Custom Tensors</a> 
+  - Backpropagation
+  - Automatic Differentiation (autograd)
+      - Forward and backward passes
+- <a href="https://github.com/kevbuh/froog/blob/main/froog/ops.py">ML Operations</a> 
+  - 2D Convolutions (im2col)
+  - Numerical gradient checking
+  - Acceleration methods (Adam)
+  - Avg & Max pooling
+- <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">EfficientNet</a> inference
+- <a href="https://github.com/kevbuh/froog/blob/main/froog/gpu/ops_gpu.py">GPU Support</a> 
+- <a href="https://github.com/kevbuh/froog/blob/main/docs/env.md">Configuration via Environment Variables</a>
+- and a bunch <a href="https://github.com/kevbuh/froog/tree/main/froog">more</a> 
+
+# Example
+
+Here's how you set up a simple multilayer perceptron for classification on MNIST. Looks pretty similar to pytorch, right?
+
+```python
+from froog.tensor import Tensor
+from froog.nn import Linear
+import froog.optim as optim
+
+class mnistMLP:
+  def __init__(self):
+    self.l1 = Tensor(Linear(784, 128)) # layer 1
+    self.l2 = Tensor(Linear(128, 10))  # layer 2
+
+  def forward(self, x):
+    # forward pass through both layers and softmax for output probabilities
+    return x.dot(self.l1).relu().dot(self.l2).logsoftmax() 
+
+model = mnistMLP() # create model
+optim = optim.SGD([model.l1, model.l2], lr=0.001) # stochastic gradient descent optimizer
+```
+
+# GPU Support
+
+Device management is handled transparently and will automatically select one of ```[METAL, OPENCL, CPU]```. To use the GPU:
+
+```python
+from froog.tensor import Tensor
+from froog import get_device
+# Check if GPU is available
+has_gpu = get_device() is not None and get_device().name != "CPU"
+# Create a tensor
+x = Tensor([1, 2, 3])
+# Push to GPU if available
+if has_gpu: x = x.to_gpu()
+# Operations run on GPU automatically
+y = x + x
+z = y * y
+# Bring back to CPU when needed
+result = z.to_cpu()
+print(result.data)
+```
+
+You can also check what devices are available:
+
+```python
+from froog import get_available_devices
+available_devices = get_available_devices()
+print(f"Available devices: {available_devices}")
+```
+
+Or set a specific device:
+
+```python
+from froog import set_device
+set_device("METAL")  # or "OPENCL"
+```
+
+# EfficientNet in froog!
+
+<img src="assets/efficientnet_pug.png" alt="pug" height="300">
+
+We have an implementation of [EfficientNet v2](https://arxiv.org/abs/2104.00298) built entirely in ```froog``` using the official PyTorch weights! Run inference with:
+
+```bash
+python3 models/efficientnet.py <https://optional_image_url>
+
+***********output*************
+inference 4.34 s
+
+imagenet class: 254
+prediction    : pug, pug-dog
+probability   : 0.8902361
+******************************
+```
+
+I would recommend checking out the <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">code</a>, it's highly documented and pretty cool.
+
+# Contributing
+<!-- THERES LOT OF STUFF TO WORK ON! VISIT THE <a href="https://github.com/kevbuh/froog/blob/main/docs/bounties.md">BOUNTY SHOP</a>  -->
+
+Pull requests will be merged if they:
+* increase simplicity
+* increase functionality
+* increase efficiency
+
+More info on <a href="https://github.com/kevbuh/froog/blob/main/docs/contributing.md">contributing</a>. Make sure to run ```python -m pytest``` before creating a PR.
+
+# API
+
+## Basic Math Operations
+- ```.add(y)``` - Addition with y
+- ```.sub(y)``` - Subtraction with y
+- ```.mul(y)``` - Multiplication with y
+- ```.div(y)``` - Division by y
+- ```.pow(y)``` - Power function (raise to power y)
+- ```.sum()``` - Sum all elements
+- ```.mean()``` - Mean of all elements
+- ```.sqrt()``` - Square root
+
+## Linear Algebra Operations
+- ```.dot(y)``` - Matrix multiplication with y
+- ```.matmul(y)``` - Alias for dot
+
+## Neural Network Operations
+- ```.relu()``` - Rectified Linear Unit activation
+- ```.sigmoid()``` - Sigmoid activation
+- ```.dropout(p=0.5, training=True)``` - Dropout regularization
+- ```.logsoftmax()``` - Log softmax function
+- ```.swish()``` - Swish activation function (x * sigmoid(x))
+- ```.conv2d(w, stride=1, groups=1)``` - 2D convolution
+- ```.im2col2dconv(w)``` - Image to column for convolution
+
+## Pooling Operations
+- ```.max_pool2d(kernel_size=(2,2))``` - 2D max pooling
+- ```.avg_pool2d(kernel_size=(2,2))``` - 2D average pooling
+
+## Tensor Manipulation
+- ```.reshape(*shape)``` - Change tensor shape
+- ```.view(*shape)``` - Alternative to reshape
+- ```.pad2d(padding=None)``` - Pad 2D tensors
+- ```.flatten()``` - Returns a flattened 1D copy of the tensor
+- ```.unsqueeze(dim)``` - Add dimension of size 1 at specified position
+- ```.squeeze(dim=None)``` - Remove dimensions of size 1
+- ```.detach()``` - Returns a tensor detached from computation graph
+- ```.assign(x)``` - Assign values from tensor x to this tensor
+
+## Tensor Properties
+- ```.shape``` - The shape of the tensor as a tuple
+- ```.size``` - Total number of elements in the tensor
+- ```.ndim``` - Number of dimensions (rank) of the tensor
+- ```.transpose``` - Transpose of the tensor
+- ```.dtype``` - Data type of the tensor
+- ```.is_gpu``` - Whether tensor is on GPU
+- ```.grad``` - Gradient of tensor with respect to some scalar value
+- ```.data``` - Underlying NumPy array (or GPU buffer)
+
+## Device Management
+- ```.to_cpu()``` - Moves tensor to CPU
+- ```.to_gpu()``` - Moves tensor to GPU 
+- ```.gpu_()``` - In-place GPU conversion (modifies tensor)
+
+## Data Type Conversion
+- ```.to_float()``` - Converts tensor to float32 data type
+- ```.to_int()``` - Converts tensor to int32 data type
+- ```.to_bool()``` - Converts tensor to boolean data type
+
+## Autograd Operations
+- ```.backward(allow_fill=True)``` - Performs backpropagation
+
+## Tensor Creation Methods
+- ```Tensor.zeros(*shape)``` - Create tensor of zeros
+- ```Tensor.ones(*shape)``` - Create tensor of ones
+- ```Tensor.randn(*shape)``` - Create tensor with random normal values
+- ```Tensor.eye(dim)``` - Create identity matrix
+- ```Tensor.arange(start, stop=None, step=1)``` - Create tensor with evenly spaced values

froog-0.5.0/README.md

@@ -0,0 +1,191 @@
+# froog <img src="https://github.com/kevbuh/froog/actions/workflows/test.yml/badge.svg" alt="unit test badge" > <img src="https://static.pepy.tech/badge/froog" alt="num downloads badge">
+<div align="center" >
+  <img src="https://raw.githubusercontent.com/kevbuh/froog/main/assets/froog.png" alt="froog the frog" height="200">
+  <br/>
+  froog: fast real-time optimization of gradients 
+  <br/>
+  a beautifully compact tensor library
+  <br/>
+  <a href="https://github.com/kevbuh/froog">homepage</a> | <a href="https://github.com/kevbuh/froog/tree/main/DOCS.md">documentation</a> | <a href="https://pypi.org/project/froog/">pip</a>
+  <br/>
+  <br/>
+</div>
+
+```froog``` is an easy-to-read tensor library (<a href="https://www.pepy.tech/projects/froog">25k pip installs!</a>) with OpenCL support for GPU acceleration. Inspired by pytorch, tinygrad, and micrograd.
+
+# Installation
+```bash
+pip install froog
+```
+
+# Features
+- <a href="https://github.com/kevbuh/froog/blob/main/froog/tensor.py">Custom Tensors</a> 
+  - Backpropagation
+  - Automatic Differentiation (autograd)
+      - Forward and backward passes
+- <a href="https://github.com/kevbuh/froog/blob/main/froog/ops.py">ML Operations</a> 
+  - 2D Convolutions (im2col)
+  - Numerical gradient checking
+  - Acceleration methods (Adam)
+  - Avg & Max pooling
+- <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">EfficientNet</a> inference
+- <a href="https://github.com/kevbuh/froog/blob/main/froog/gpu/ops_gpu.py">GPU Support</a> 
+- <a href="https://github.com/kevbuh/froog/blob/main/docs/env.md">Configuration via Environment Variables</a>
+- and a bunch <a href="https://github.com/kevbuh/froog/tree/main/froog">more</a> 
+
+# Example
+
+Here's how you set up a simple multilayer perceptron for classification on MNIST. Looks pretty similar to pytorch, right?
+
+```python
+from froog.tensor import Tensor
+from froog.nn import Linear
+import froog.optim as optim
+
+class mnistMLP:
+  def __init__(self):
+    self.l1 = Tensor(Linear(784, 128)) # layer 1
+    self.l2 = Tensor(Linear(128, 10))  # layer 2
+
+  def forward(self, x):
+    # forward pass through both layers and softmax for output probabilities
+    return x.dot(self.l1).relu().dot(self.l2).logsoftmax() 
+
+model = mnistMLP() # create model
+optim = optim.SGD([model.l1, model.l2], lr=0.001) # stochastic gradient descent optimizer
+```
+
+# GPU Support
+
+Device management is handled transparently and will automatically select one of ```[METAL, OPENCL, CPU]```. To use the GPU:
+
+```python
+from froog.tensor import Tensor
+from froog import get_device
+# Check if GPU is available
+has_gpu = get_device() is not None and get_device().name != "CPU"
+# Create a tensor
+x = Tensor([1, 2, 3])
+# Push to GPU if available
+if has_gpu: x = x.to_gpu()
+# Operations run on GPU automatically
+y = x + x
+z = y * y
+# Bring back to CPU when needed
+result = z.to_cpu()
+print(result.data)
+```
+
+You can also check what devices are available:
+
+```python
+from froog import get_available_devices
+available_devices = get_available_devices()
+print(f"Available devices: {available_devices}")
+```
+
+Or set a specific device:
+
+```python
+from froog import set_device
+set_device("METAL")  # or "OPENCL"
+```
+
+# EfficientNet in froog!
+
+<img src="assets/efficientnet_pug.png" alt="pug" height="300">
+
+We have an implementation of [EfficientNet v2](https://arxiv.org/abs/2104.00298) built entirely in ```froog``` using the official PyTorch weights! Run inference with:
+
+```bash
+python3 models/efficientnet.py <https://optional_image_url>
+
+***********output*************
+inference 4.34 s
+
+imagenet class: 254
+prediction    : pug, pug-dog
+probability   : 0.8902361
+******************************
+```
+
+I would recommend checking out the <a href="https://github.com/kevbuh/froog/blob/main/models/efficientnet.py">code</a>, it's highly documented and pretty cool.
+
+# Contributing
+<!-- THERES LOT OF STUFF TO WORK ON! VISIT THE <a href="https://github.com/kevbuh/froog/blob/main/docs/bounties.md">BOUNTY SHOP</a>  -->
+
+Pull requests will be merged if they:
+* increase simplicity
+* increase functionality
+* increase efficiency
+
+More info on <a href="https://github.com/kevbuh/froog/blob/main/docs/contributing.md">contributing</a>. Make sure to run ```python -m pytest``` before creating a PR.
+
+# API
+
+## Basic Math Operations
+- ```.add(y)``` - Addition with y
+- ```.sub(y)``` - Subtraction with y
+- ```.mul(y)``` - Multiplication with y
+- ```.div(y)``` - Division by y
+- ```.pow(y)``` - Power function (raise to power y)
+- ```.sum()``` - Sum all elements
+- ```.mean()``` - Mean of all elements
+- ```.sqrt()``` - Square root
+
+## Linear Algebra Operations
+- ```.dot(y)``` - Matrix multiplication with y
+- ```.matmul(y)``` - Alias for dot
+
+## Neural Network Operations
+- ```.relu()``` - Rectified Linear Unit activation
+- ```.sigmoid()``` - Sigmoid activation
+- ```.dropout(p=0.5, training=True)``` - Dropout regularization
+- ```.logsoftmax()``` - Log softmax function
+- ```.swish()``` - Swish activation function (x * sigmoid(x))
+- ```.conv2d(w, stride=1, groups=1)``` - 2D convolution
+- ```.im2col2dconv(w)``` - Image to column for convolution
+
+## Pooling Operations
+- ```.max_pool2d(kernel_size=(2,2))``` - 2D max pooling
+- ```.avg_pool2d(kernel_size=(2,2))``` - 2D average pooling
+
+## Tensor Manipulation
+- ```.reshape(*shape)``` - Change tensor shape
+- ```.view(*shape)``` - Alternative to reshape
+- ```.pad2d(padding=None)``` - Pad 2D tensors
+- ```.flatten()``` - Returns a flattened 1D copy of the tensor
+- ```.unsqueeze(dim)``` - Add dimension of size 1 at specified position
+- ```.squeeze(dim=None)``` - Remove dimensions of size 1
+- ```.detach()``` - Returns a tensor detached from computation graph
+- ```.assign(x)``` - Assign values from tensor x to this tensor
+
+## Tensor Properties
+- ```.shape``` - The shape of the tensor as a tuple
+- ```.size``` - Total number of elements in the tensor
+- ```.ndim``` - Number of dimensions (rank) of the tensor
+- ```.transpose``` - Transpose of the tensor
+- ```.dtype``` - Data type of the tensor
+- ```.is_gpu``` - Whether tensor is on GPU
+- ```.grad``` - Gradient of tensor with respect to some scalar value
+- ```.data``` - Underlying NumPy array (or GPU buffer)
+
+## Device Management
+- ```.to_cpu()``` - Moves tensor to CPU
+- ```.to_gpu()``` - Moves tensor to GPU 
+- ```.gpu_()``` - In-place GPU conversion (modifies tensor)
+
+## Data Type Conversion
+- ```.to_float()``` - Converts tensor to float32 data type
+- ```.to_int()``` - Converts tensor to int32 data type
+- ```.to_bool()``` - Converts tensor to boolean data type
+
+## Autograd Operations
+- ```.backward(allow_fill=True)``` - Performs backpropagation
+
+## Tensor Creation Methods
+- ```Tensor.zeros(*shape)``` - Create tensor of zeros
+- ```Tensor.ones(*shape)``` - Create tensor of ones
+- ```Tensor.randn(*shape)``` - Create tensor with random normal values
+- ```Tensor.eye(dim)``` - Create identity matrix
+- ```Tensor.arange(start, stop=None, step=1)``` - Create tensor with evenly spaced values

froog-0.5.0/froog/__init__.py

@@ -0,0 +1,36 @@
+import froog.optim
+import froog.tensor
+import froog.utils
+
+# Import GPU packages
+import froog.gpu.cl.cl_utils
+
+# Try to import Metal utils if available
+try:
+    import froog.gpu.metal.metal_utils
+except ImportError:
+    pass
+
+# Import device management functions
+from froog.gpu import (
+    Device, OpenCLDevice, get_device, set_device,
+    upload_tensor, download_tensor, is_buffer,
+    allocate_buffer, synchronize, get_available_devices
+)
+
+# Try to import Metal device if available
+try:
+    from froog.gpu.metal import MetalDevice
+    __all__ = [
+        'Device', 'OpenCLDevice', 'MetalDevice', 
+        'get_device', 'set_device', 'upload_tensor', 
+        'download_tensor', 'is_buffer', 
+        'allocate_buffer', 'synchronize', 'get_available_devices'
+    ]
+except ImportError:
+    __all__ = [
+        'Device', 'OpenCLDevice', 
+        'get_device', 'set_device', 'upload_tensor', 
+        'download_tensor', 'is_buffer', 
+        'allocate_buffer', 'synchronize', 'get_available_devices'
+    ]

froog-0.4.2/froog/gradcheck.py → froog-0.5.0/froog/gradient.py

@@ -7,16 +7,15 @@
 # |___|    |___|  |_||_______||_______||_______|
 
 import numpy as np
+from typing import Callable, Union, Any, Tuple
 from froog.tensor import Tensor
 from froog.utils import mask_like
 
-def jacobian(model, input):
+def jacobian(model: Callable[[Tensor], Tensor], input: Tensor) -> np.ndarray:
   output = model(input)
-
   ji = input.data.reshape(-1).shape[-1]  # jacobian of input
   jo = output.data.reshape(-1).shape[-1] # jacobian of output
   J = np.zeros((jo, ji), dtype=np.float32)
-
   for o in range(jo):
     o_scalar = Tensor(mask_like(output.data, o, 1.)).mul(output).sum()
     o_scalar.backward()
@@ -24,7 +23,7 @@ def jacobian(model, input):
       J[o,i] = grad
   return J
 
-def numerical_jacobian(model, input, eps = 1e-6):
+def numerical_jacobian(model: Callable[[Tensor], Tensor], input: Tensor, eps: float = 1e-6) -> np.ndarray:
 #     """
 #     https://timvieira.github.io/blog/post/2017/04/21/how-to-test-gradient-implementations/
 #     Computes :
@@ -37,24 +36,18 @@ def numerical_jacobian(model, input, eps = 1e-6):
 #         NJ    : an approx. of the Jacobian
 #     """
   output = model(input)
-
   ji = input.data.reshape(-1).shape[-1]
   jo = output.data.reshape(-1).shape[-1]
   NJ = np.zeros((jo, ji), dtype=np.float32)
-
   for i in range(ji):
     eps_perturb = mask_like(input.data, i, mask_value = eps)
-
     output_perturb_add = model(Tensor(input.data + eps_perturb)).data.reshape(-1)
     output_perturb_sub = model(Tensor(input.data - eps_perturb)).data.reshape(-1)
-
     grad_approx = ((output_perturb_add) - (output_perturb_sub)) / (2*eps) # CDM: (f(x + h) - f(x - h)) / (2 * h)
-
     NJ[:,i] = grad_approx
-
   return NJ
 
-def gradcheck(model, input, eps = 1e-06, atol = 1e-5, rtol = 0.001):
+def gradcheck(model: Callable[[Tensor], Tensor], input: Tensor, eps: float = 1e-06, atol: float = 1e-5, rtol: float = 0.001) -> bool:
   """
   Checks whether computed gradient is close to numerical approximation of the Jacobian
   Params: