neutro 0.1.0__tar.gz

neutro-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

neutro-0.1.0/PKG-INFO

@@ -0,0 +1,126 @@
+Metadata-Version: 2.4
+Name: neutro
+Version: 0.1.0
+Summary: A Keras-style deep learning library using NumPy and SciPy
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: scipy
+Requires-Dist: joblib
+Requires-Dist: tqdm
+Requires-Dist: regex
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest-cov; extra == "test"
+Dynamic: license-file
+
+# 🧠 Neutro: The "Old School" Deep Learning Playground
+
+[![codecov](https://codecov.io/gh/sourcepirate/neutro/graph/badge.svg?token=8H4Q2Q2Q2Q)](https://codecov.io/gh/sourcepirate/neutro)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**Neutro** is a intentionally naive, NumPy-only implementation of modern deep learning architectures. It’s the Keras experience you love, powered by the NumPy you tolerate, built specifically for people who want to peek under the hood and actually *understand* how the gears turn.
+
+---
+
+## 👴 The Philosophy: Why Does This Exist?
+
+Let's be honest: modern DL frameworks are black boxes. You pip install 4GB of binaries and suddenly you're "doing AI." 
+
+**Neutro is for the curious, the learners, and the "old-school" folks like me** who believe that if you can't build it in a matrix, you don't really know it. 
+
+- **Learn, Don't just Run**: Every line of code is designed to be readable. We don't hide behind C++ kernels or CUDA kernels. If you want to know how FlashAttention *actually* tiles memory, you can just read the Python file.
+- **A Toy, not a Tool**: This isn't meant for production. It's a playground for learning advanced algorithms (MHA, GQA, FlashAttention, LSTM) in their purest form.
+- **For the Wisdom-Rich**: If you remember when 64MB of RAM was a flex and "vectorization" meant loop unrolling, this is for you. It's a fun way to play with cutting-edge 2024 algorithms using 1990s-era clarity.
+
+---
+
+## 🚀 What's Inside?
+
+- **"I can't believe it's not Keras!"**: Your muscle memory is safe here. `.compile()`, `.fit()`, `.predict()`—it’s all exactly where you left it.
+- **Pure NumPy Math**: We did the math so you don't have to. Every gradient, from Softmax to LSTM gates, is hand-derived and vectorized.
+- **Speed (for a CPU)**: We use `im2col` for convolutions and **FlashAttention** (yes, really) to keep your CPU fans humming in a way that sounds productive.
+- **Zero Heavy Dependencies**: Tired of downloading 4GB of CUDA binaries just to train on MNIST? We require exactly `numpy` and `scipy`. That’s it.
+
+---
+
+## 🛠 Features That'll Make You Say "Wait, You Implemented That?"
+
+| Category | The "Fancy" Stuff | Why You Should Care |
+| :--- | :--- | :--- |
+| **Attention** | `FlashAttention`, `MQA`, `GQA`, `RoPE` | We have more attention variants than a distracted toddler. |
+| **Tokenization** | `BPETokenizer`, `RegexTokenizer` | Byte-level BPE with regex splitting, just like the big kids. |
+| **Vision** | `AlexNet`, `VGG16`, `VGG19`, `im2col` | Classical and modern vision architectures, vectorized. |
+| **LLMs** | `Llama`, `Qwen`, `DeepSeek` (MoE) | Yes, you can run a (very tiny) MoE model on your CPU. |
+| **Modern Ops** | `RMSNorm`, `SiLU`, `SwiGLU` | The secret sauce of modern LLMs, hand-implemented. |
+| **Optimizers** | `AdamW`, `Adam`, `SGD+Momentum` | Keep your weights from exploding like a bad science fair project. |
+
+---
+
+## 🏆 The Hall of Fame: Pre-built Architectures
+
+Why build from scratch when we've already done the heavy lifting?
+
+- **The Visionaries**: `AlexNet`, `VGG16`, `VGG19`
+- **The Linguists**: `GPT-2`, `LlamaTiny`, `QwenTiny`, `DeepSeekTiny` (Mixture of Experts)
+
+---
+
+## 💻 Show Me The Code!
+
+If you know Keras, you already know Neutro. It's that simple.
+
+```python
+from neutro.models import Sequential
+from neutro.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout
+
+# Build a CNN that actually fits in your head
+model = Sequential([
+    Conv2D(32, kernel_size=3, activation='relu', input_shape=(28, 28, 1)),
+    MaxPooling2D(pool_size=2),
+    Flatten(),
+    Dense(128, activation='relu'),
+    Dropout(0.5),
+    Dense(10, activation='softmax')
+])
+
+# Compile it like it's 2015
+model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
+
+# Fit it like a tailored suit
+model.fit(train_flow, epochs=10)
+```
+
+---
+
+## 📂 Deep Dives & Nerdy Stuff
+
+We documented everything because we know you like to check the math:
+
+- [**Attention Mechanisms**](./docs/layers/attention/) - How we made FlashAttention work on a CPU.
+- [**Convolutional Magic**](./docs/layers/convolutional/) - The `im2col` deep dive.
+- [**Activations & Gradients**](./docs/activations/) - Proofs for the brave.
+- [**Optimizers**](./docs/optimizers/) - Why AdamW is better than your ex.
+
+---
+
+## 🧪 Examples to Flex Your CPU
+
+Check out the `examples/` folder for end-to-end scripts:
+- `mnist_cnn.py`: Standard digit classification with real-time augmentation.
+- `wikitext_llm.py`: A character-level Transformer that actually talks back.
+
+---
+
+## 🏗 Installation
+
+```bash
+git clone https://github.com/sourcepirate/neutro.git
+cd neutro
+pip install -e .
+```
+
+---
+
+**Disclaimer**: This is a hobby project for learning and exploration. It is intentionally naive, likely inefficient compared to compiled kernels, and 100% focused on the joy of understanding advanced algorithms. If you're looking to change the world with AGI, go to PyTorch. If you're looking to understand why your Transformer works while drinking a nice cup of tea, you're in the right place.

neutro-0.1.0/README.md

@@ -0,0 +1,109 @@
+# 🧠 Neutro: The "Old School" Deep Learning Playground
+
+[![codecov](https://codecov.io/gh/sourcepirate/neutro/graph/badge.svg?token=8H4Q2Q2Q2Q)](https://codecov.io/gh/sourcepirate/neutro)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**Neutro** is a intentionally naive, NumPy-only implementation of modern deep learning architectures. It’s the Keras experience you love, powered by the NumPy you tolerate, built specifically for people who want to peek under the hood and actually *understand* how the gears turn.
+
+---
+
+## 👴 The Philosophy: Why Does This Exist?
+
+Let's be honest: modern DL frameworks are black boxes. You pip install 4GB of binaries and suddenly you're "doing AI." 
+
+**Neutro is for the curious, the learners, and the "old-school" folks like me** who believe that if you can't build it in a matrix, you don't really know it. 
+
+- **Learn, Don't just Run**: Every line of code is designed to be readable. We don't hide behind C++ kernels or CUDA kernels. If you want to know how FlashAttention *actually* tiles memory, you can just read the Python file.
+- **A Toy, not a Tool**: This isn't meant for production. It's a playground for learning advanced algorithms (MHA, GQA, FlashAttention, LSTM) in their purest form.
+- **For the Wisdom-Rich**: If you remember when 64MB of RAM was a flex and "vectorization" meant loop unrolling, this is for you. It's a fun way to play with cutting-edge 2024 algorithms using 1990s-era clarity.
+
+---
+
+## 🚀 What's Inside?
+
+- **"I can't believe it's not Keras!"**: Your muscle memory is safe here. `.compile()`, `.fit()`, `.predict()`—it’s all exactly where you left it.
+- **Pure NumPy Math**: We did the math so you don't have to. Every gradient, from Softmax to LSTM gates, is hand-derived and vectorized.
+- **Speed (for a CPU)**: We use `im2col` for convolutions and **FlashAttention** (yes, really) to keep your CPU fans humming in a way that sounds productive.
+- **Zero Heavy Dependencies**: Tired of downloading 4GB of CUDA binaries just to train on MNIST? We require exactly `numpy` and `scipy`. That’s it.
+
+---
+
+## 🛠 Features That'll Make You Say "Wait, You Implemented That?"
+
+| Category | The "Fancy" Stuff | Why You Should Care |
+| :--- | :--- | :--- |
+| **Attention** | `FlashAttention`, `MQA`, `GQA`, `RoPE` | We have more attention variants than a distracted toddler. |
+| **Tokenization** | `BPETokenizer`, `RegexTokenizer` | Byte-level BPE with regex splitting, just like the big kids. |
+| **Vision** | `AlexNet`, `VGG16`, `VGG19`, `im2col` | Classical and modern vision architectures, vectorized. |
+| **LLMs** | `Llama`, `Qwen`, `DeepSeek` (MoE) | Yes, you can run a (very tiny) MoE model on your CPU. |
+| **Modern Ops** | `RMSNorm`, `SiLU`, `SwiGLU` | The secret sauce of modern LLMs, hand-implemented. |
+| **Optimizers** | `AdamW`, `Adam`, `SGD+Momentum` | Keep your weights from exploding like a bad science fair project. |
+
+---
+
+## 🏆 The Hall of Fame: Pre-built Architectures
+
+Why build from scratch when we've already done the heavy lifting?
+
+- **The Visionaries**: `AlexNet`, `VGG16`, `VGG19`
+- **The Linguists**: `GPT-2`, `LlamaTiny`, `QwenTiny`, `DeepSeekTiny` (Mixture of Experts)
+
+---
+
+## 💻 Show Me The Code!
+
+If you know Keras, you already know Neutro. It's that simple.
+
+```python
+from neutro.models import Sequential
+from neutro.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout
+
+# Build a CNN that actually fits in your head
+model = Sequential([
+    Conv2D(32, kernel_size=3, activation='relu', input_shape=(28, 28, 1)),
+    MaxPooling2D(pool_size=2),
+    Flatten(),
+    Dense(128, activation='relu'),
+    Dropout(0.5),
+    Dense(10, activation='softmax')
+])
+
+# Compile it like it's 2015
+model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
+
+# Fit it like a tailored suit
+model.fit(train_flow, epochs=10)
+```
+
+---
+
+## 📂 Deep Dives & Nerdy Stuff
+
+We documented everything because we know you like to check the math:
+
+- [**Attention Mechanisms**](./docs/layers/attention/) - How we made FlashAttention work on a CPU.
+- [**Convolutional Magic**](./docs/layers/convolutional/) - The `im2col` deep dive.
+- [**Activations & Gradients**](./docs/activations/) - Proofs for the brave.
+- [**Optimizers**](./docs/optimizers/) - Why AdamW is better than your ex.
+
+---
+
+## 🧪 Examples to Flex Your CPU
+
+Check out the `examples/` folder for end-to-end scripts:
+- `mnist_cnn.py`: Standard digit classification with real-time augmentation.
+- `wikitext_llm.py`: A character-level Transformer that actually talks back.
+
+---
+
+## 🏗 Installation
+
+```bash
+git clone https://github.com/sourcepirate/neutro.git
+cd neutro
+pip install -e .
+```
+
+---
+
+**Disclaimer**: This is a hobby project for learning and exploration. It is intentionally naive, likely inefficient compared to compiled kernels, and 100% focused on the joy of understanding advanced algorithms. If you're looking to change the world with AGI, go to PyTorch. If you're looking to understand why your Transformer works while drinking a nice cup of tea, you're in the right place.

neutro-0.1.0/neutro/__init__.py

@@ -0,0 +1,8 @@
+from . import layers
+from . import activations
+from . import initializers
+from . import losses
+from . import optimizers
+from . import metrics
+from . import callbacks
+from .models import Sequential, Model

neutro-0.1.0/neutro/activations/__init__.py

@@ -0,0 +1,15 @@
+from .base import Activation
+from .relu import ReLU
+from .sigmoid import Sigmoid
+from .tanh import Tanh
+from .softmax import Softmax
+from .silu import SiLU
+
+def get(identifier):
+    if identifier == 'relu': return ReLU()
+    if identifier == 'sigmoid': return Sigmoid()
+    if identifier == 'tanh': return Tanh()
+    if identifier == 'softmax': return Softmax()
+    if identifier == 'silu': return SiLU()
+    if isinstance(identifier, Activation): return identifier
+    return identifier

neutro-0.1.0/neutro/activations/base.py

@@ -0,0 +1,5 @@
+class Activation:
+    def __call__(self, x):
+        raise NotImplementedError
+    def gradient(self, x):
+        raise NotImplementedError

neutro-0.1.0/neutro/activations/relu.py

@@ -0,0 +1,9 @@
+import numpy as np
+from .base import Activation
+
+class ReLU(Activation):
+    def __call__(self, x):
+        self.last_x = x
+        return np.maximum(0, x)
+    def gradient(self, x):
+        return (x > 0).astype(float)

neutro-0.1.0/neutro/activations/sigmoid.py

@@ -0,0 +1,10 @@
+import numpy as np
+from .base import Activation
+
+class Sigmoid(Activation):
+    def __call__(self, x):
+        self.last_output = 1 / (1 + np.exp(-np.clip(x, -500, 500)))
+        return self.last_output
+    def gradient(self, x):
+        s = self.__call__(x)
+        return s * (1 - s)

neutro-0.1.0/neutro/activations/silu.py

@@ -0,0 +1,20 @@
+import numpy as np
+from .base import Activation
+
+class SiLU(Activation):
+    """
+    SiLU (Sigmoid Linear Unit) or Swish activation function: x * sigmoid(x).
+    Commonly used in Llama, Qwen, and DeepSeek.
+    """
+    def __call__(self, x):
+        self.sigmoid_x = 1 / (1 + np.exp(-x))
+        self.x = x
+        return x * self.sigmoid_x
+
+    def gradient(self, x):
+        # f'(x) = f(x) + sigmoid(x) * (1 - f(x))
+        f_x = x * self.sigmoid_x
+        return f_x + self.sigmoid_x * (1 - f_x)
+
+    def gradient_fast(self, x, grad_output):
+        return grad_output * self.gradient(x)

neutro-0.1.0/neutro/activations/softmax.py

@@ -0,0 +1,22 @@
+import numpy as np
+from .base import Activation
+
+class Softmax(Activation):
+    def __call__(self, x):
+        exps = np.exp(x - np.max(x, axis=-1, keepdims=True))
+        self.last_output = exps / np.sum(exps, axis=-1, keepdims=True)
+        return self.last_output
+    def gradient(self, x):
+        return self.last_output * (1 - self.last_output)
+    def gradient_fast(self, x, grad_output):
+        orig_shape = grad_output.shape
+        grad_flat = grad_output.reshape(-1, orig_shape[-1])
+        out_flat = self.last_output.reshape(-1, orig_shape[-1])
+        
+        n_samples, units = grad_flat.shape
+        res = np.zeros_like(grad_flat)
+        for i in range(n_samples):
+            s = out_flat[i].reshape(-1, 1)
+            jacobian = np.diagflat(s) - np.dot(s, s.T)
+            res[i] = np.dot(grad_flat[i], jacobian)
+        return res.reshape(orig_shape)

neutro-0.1.0/neutro/activations/tanh.py

@@ -0,0 +1,8 @@
+import numpy as np
+from .base import Activation
+
+class Tanh(Activation):
+    def __call__(self, x):
+        return np.tanh(x)
+    def gradient(self, x):
+        return 1 - np.tanh(x)**2

neutro-0.1.0/neutro/callbacks/__init__.py

@@ -0,0 +1,5 @@
+from .base import Callback
+from .checkpoint import ModelCheckpoint
+from .early_stopping import EarlyStopping
+from .history import History
+from .lr_scheduler import LearningRateScheduler, ReduceLROnPlateau

neutro-0.1.0/neutro/callbacks/base.py

@@ -0,0 +1,13 @@
+class Callback:
+    def __init__(self):
+        self.model = None
+
+    def set_model(self, model):
+        self.model = model
+
+    def on_epoch_begin(self, epoch, logs=None): pass
+    def on_epoch_end(self, epoch, logs=None): pass
+    def on_batch_begin(self, batch, logs=None): pass
+    def on_batch_end(self, batch, logs=None): pass
+    def on_train_begin(self, logs=None): pass
+    def on_train_end(self, logs=None): pass

neutro-0.1.0/neutro/callbacks/checkpoint.py

@@ -0,0 +1,25 @@
+import numpy as np
+from .base import Callback
+
+class ModelCheckpoint(Callback):
+    def __init__(self, filepath, monitor='val_loss', save_best_only=False, mode='auto'):
+        super().__init__()
+        self.filepath = filepath
+        self.monitor = monitor
+        self.save_best_only = save_best_only
+        self.best = -np.inf if mode == 'max' or (mode == 'auto' and 'acc' in monitor) else np.inf
+        self.mode = mode
+
+    def on_epoch_end(self, epoch, logs=None):
+        logs = logs or {}
+        current = logs.get(self.monitor)
+        if current is None: return
+
+        if self.save_best_only:
+            if (self.mode == 'min' and current < self.best) or \
+               (self.mode == 'max' and current > self.best) or \
+               (self.mode == 'auto' and (('acc' in self.monitor and current > self.best) or ('loss' in self.monitor and current < self.best))):
+                self.best = current
+                self.model.save(self.filepath)
+        else:
+            self.model.save(self.filepath.format(epoch=epoch + 1, **logs))

neutro-0.1.0/neutro/callbacks/early_stopping.py

@@ -0,0 +1,27 @@
+import numpy as np
+from .base import Callback
+
+class EarlyStopping(Callback):
+    def __init__(self, monitor='val_loss', patience=0, mode='auto'):
+        super().__init__()
+        self.monitor = monitor
+        self.patience = patience
+        self.wait = 0
+        self.best = -np.inf if mode == 'max' or (mode == 'auto' and 'acc' in monitor) else np.inf
+        self.mode = mode
+
+    def on_epoch_end(self, epoch, logs=None):
+        logs = logs or {}
+        current = logs.get(self.monitor)
+        if current is None: return
+
+        if (self.mode == 'min' and current < self.best) or \
+           (self.mode == 'max' and current > self.best) or \
+           (self.mode == 'auto' and (('acc' in self.monitor and current > self.best) or ('loss' in self.monitor and current < self.best))):
+            self.best = current
+            self.wait = 0
+        else:
+            self.wait += 1
+            if self.wait >= self.patience:
+                self.model.stop_training = True
+                print(f"Epoch {epoch+1}: early stopping")

neutro-0.1.0/neutro/callbacks/history.py

@@ -0,0 +1,10 @@
+from .base import Callback
+
+class History(Callback):
+    def on_train_begin(self, logs=None):
+        self.history = {'loss': [], 'epoch': []}
+
+    def on_epoch_end(self, epoch, logs=None):
+        self.history['epoch'].append(epoch)
+        for k, v in logs.items():
+            self.history.setdefault(k, []).append(v)

neutro-0.1.0/neutro/callbacks/lr_scheduler.py

@@ -0,0 +1,79 @@
+import numpy as np
+from .base import Callback
+
+class LearningRateScheduler(Callback):
+    """
+    Learning rate scheduler.
+    
+    Args:
+        schedule: a function that takes an epoch index (integer, indexed from 0) and current learning rate as inputs and returns a new learning rate as output (float).
+    """
+    def __init__(self, schedule, verbose=0):
+        super().__init__()
+        self.schedule = schedule
+        self.verbose = verbose
+
+    def on_epoch_begin(self, epoch, logs=None):
+        if not hasattr(self.model.optimizer, 'lr'):
+            raise ValueError('Optimizer must have a "lr" attribute.')
+        
+        lr = float(self.model.optimizer.lr)
+        lr = self.schedule(epoch, lr)
+        self.model.optimizer.lr = lr
+        if self.verbose > 0:
+            print(f'\nEpoch {epoch + 1}: LearningRateScheduler setting learning rate to {lr}.')
+
+class ReduceLROnPlateau(Callback):
+    """
+    Reduce learning rate when a metric has stopped improving.
+    """
+    def __init__(self, monitor='val_loss', factor=0.1, patience=10, verbose=0, mode='auto', min_delta=1e-4, cooldown=0, min_lr=0):
+        super().__init__()
+        self.monitor = monitor
+        self.factor = factor
+        self.patience = patience
+        self.verbose = verbose
+        self.mode = mode
+        self.min_delta = min_delta
+        self.cooldown = cooldown
+        self.min_lr = min_lr
+        self.wait = 0
+        self.best = np.inf if 'loss' in monitor else -np.inf
+        self.cooldown_counter = 0
+
+    def on_train_begin(self, logs=None):
+        self.wait = 0
+        self.best = np.inf if 'loss' in self.monitor else -np.inf
+        self.cooldown_counter = 0
+
+    def on_epoch_end(self, epoch, logs=None):
+        logs = logs or {}
+        current = logs.get(self.monitor)
+        if current is None:
+            return
+
+        if self.cooldown_counter > 0:
+            self.cooldown_counter -= 1
+            self.wait = 0
+
+        if self._is_improvement(current, self.best):
+            self.best = current
+            self.wait = 0
+        elif self.cooldown_counter <= 0:
+            self.wait += 1
+            if self.wait >= self.patience:
+                old_lr = float(self.model.optimizer.lr)
+                if old_lr > self.min_lr:
+                    new_lr = old_lr * self.factor
+                    new_lr = max(new_lr, self.min_lr)
+                    self.model.optimizer.lr = new_lr
+                    if self.verbose > 0:
+                        print(f'\nEpoch {epoch + 1}: ReduceLROnPlateau reducing learning rate to {new_lr}.')
+                    self.cooldown_counter = self.cooldown
+                    self.wait = 0
+
+    def _is_improvement(self, current, best):
+        if 'loss' in self.monitor or self.mode == 'min':
+            return current < best - self.min_delta
+        else:
+            return current > best + self.min_delta

neutro-0.1.0/neutro/data.py

@@ -0,0 +1,44 @@
+import numpy as np
+
+class DataLoader:
+    """
+    Data loader for batching and shuffling data.
+    
+    Args:
+        x: Input data (NumPy array).
+        y: Target data (NumPy array).
+        batch_size: Number of samples per batch.
+        shuffle: Whether to shuffle the data at the beginning of each epoch.
+    """
+    def __init__(self, x, y, batch_size=32, shuffle=True, augmenter=None):
+        self.x = x
+        self.y = y
+        self.batch_size = batch_size
+        self.shuffle = shuffle
+        self.augmenter = augmenter
+        self.indices = np.arange(len(x))
+        self.on_epoch_end()
+
+    def __len__(self):
+        return int(np.ceil(len(self.x) / self.batch_size))
+
+    def on_epoch_end(self):
+        if self.shuffle:
+            np.random.shuffle(self.indices)
+
+    def __getitem__(self, index):
+        indices = self.indices[index * self.batch_size : (index + 1) * self.batch_size]
+        batch_x, batch_y = self.x[indices], self.y[indices]
+        
+        if self.augmenter:
+            augmented_x = np.zeros_like(batch_x)
+            for i in range(len(batch_x)):
+                augmented_x[i] = self.augmenter.apply_transform(batch_x[i])
+            batch_x = augmented_x
+            
+        return batch_x, batch_y
+
+    def __iter__(self):
+        for i in range(len(self)):
+            yield self[i]
+        self.on_epoch_end()

neutro-0.1.0/neutro/initializers/__init__.py

@@ -0,0 +1,13 @@
+from .base import Initializer
+from .constant import Zeros, Ones
+from .random import RandomNormal
+from .glorot import GlorotUniform
+from .he import HeNormal
+
+def get(identifier):
+    if identifier == 'zeros': return Zeros()
+    if identifier == 'ones': return Ones()
+    if identifier == 'glorot_uniform': return GlorotUniform()
+    if identifier == 'he_normal': return HeNormal()
+    if isinstance(identifier, Initializer): return identifier
+    return identifier

neutro-0.1.0/neutro/initializers/base.py

@@ -0,0 +1,3 @@
+class Initializer:
+    def __call__(self, shape):
+        raise NotImplementedError

neutro-0.1.0/neutro/initializers/constant.py

@@ -0,0 +1,10 @@
+import numpy as np
+from .base import Initializer
+
+class Zeros(Initializer):
+    def __call__(self, shape):
+        return np.zeros(shape)
+
+class Ones(Initializer):
+    def __call__(self, shape):
+        return np.ones(shape)

neutro-0.1.0/neutro/initializers/glorot.py

@@ -0,0 +1,19 @@
+import numpy as np
+from .base import Initializer
+
+class GlorotUniform(Initializer):
+    def __call__(self, shape):
+        fan_in, fan_out = self._calculate_fan_in_and_fan_out(shape)
+        limit = np.sqrt(6 / (fan_in + fan_out))
+        return np.random.uniform(-limit, limit, shape)
+
+    def _calculate_fan_in_and_fan_out(self, shape):
+        if len(shape) < 2:
+            return shape[0], shape[0]
+        if len(shape) == 2:
+            return shape[0], shape[1]
+        else:
+            receptive_field_size = np.prod(shape[:-2])
+            fan_in = shape[-2] * receptive_field_size
+            fan_out = shape[-1] * receptive_field_size
+            return fan_in, fan_out

neutro-0.1.0/neutro/initializers/he.py

@@ -0,0 +1,19 @@
+import numpy as np
+from .base import Initializer
+
+class HeNormal(Initializer):
+    def __call__(self, shape):
+        fan_in, _ = self._calculate_fan_in_and_fan_out(shape)
+        std = np.sqrt(2 / fan_in)
+        return np.random.normal(0, std, shape)
+
+    def _calculate_fan_in_and_fan_out(self, shape):
+        if len(shape) < 2:
+            return shape[0], shape[0]
+        if len(shape) == 2:
+            return shape[0], shape[1]
+        else:
+            receptive_field_size = np.prod(shape[:-2])
+            fan_in = shape[-2] * receptive_field_size
+            fan_out = shape[-1] * receptive_field_size
+            return fan_in, fan_out

neutro-0.1.0/neutro/initializers/random.py

@@ -0,0 +1,9 @@
+import numpy as np
+from .base import Initializer
+
+class RandomNormal(Initializer):
+    def __init__(self, mean=0.0, stddev=0.05):
+        self.mean = mean
+        self.stddev = stddev
+    def __call__(self, shape):
+        return np.random.normal(self.mean, self.stddev, shape)