neural-lab 0.5.0__tar.gz

neural_lab-0.5.0/.gitignore

@@ -0,0 +1,163 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+*.pkl
+*.html

neural_lab-0.5.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 FAR Analytics & Research
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

neural_lab-0.5.0/PKG-INFO

@@ -0,0 +1,283 @@
+Metadata-Version: 2.5
+Name: neural-lab
+Version: 0.5.0
+Summary: A neural network implementation built as an explicit graph of neurons and weighted connections.
+Project-URL: Homepage, https://github.com/adamjpatterson/neural-lab
+Project-URL: Issues, https://github.com/adamjpatterson/neural-lab/issues
+Author-email: FAR Analytics & Research <adpatter2000@gmail.com>
+License-File: LICENSE
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+
+# Neural Lab
+
+[![PyPI version](https://img.shields.io/pypi/v/neural-lab.svg)](https://pypi.org/project/neural-lab/) [![Python versions](https://img.shields.io/pypi/pyversions/neural-lab.svg)](https://pypi.org/project/neural-lab/) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/adamjpatterson/neural-lab/blob/main/LICENSE)
+
+A neural network implementation built as an explicit graph of neurons and weighted connections.
+
+## Introduction
+
+**Neural Lab** represents individual neurons and their connections explicitly. Backpropagation is done using explicitly computed gradients. The object-oriented API consists of Network, Layer, and Neuron constructors. The networks can be easily [visualized](#visualizations-of-the-network-before-and-after-training) using a visualization library.
+
+## Table of contents
+
+- [Installation](#installation)
+- [How it works](#how-it-works)
+- [Examples](#examples)
+- [The nibble challenge](#the-nibble-challenge)
+- [The digits experiment](#the-digits-experiment)
+- [Tests](#tests)
+- [Documentation maintenance](#documentation-maintenance)
+
+## Installation
+
+To use the published package:
+
+```bash
+python -m pip install neural-lab
+```
+
+## How it works
+
+This implementation demonstrates each component of the API. A 3 layer neural network is constructed that has a 4 neuron input Layer and a 1 neuron output layer. The hidden layer has 8 neurons.
+
+### Implement a 3 layer neural network
+
+#### Import the Network, Layer, and Neuron classes.
+
+```python
+from neural_lab import Network, Layer, Neuron
+```
+
+#### Set a step.
+
+`step` is the learning rate: it controls the size of weight and bias updates. Each update subtracts `step` times the corresponding gradient. `Layer(step=...)` supplies an initial default only to neurons whose `step` is `None`. Changing `layer.step` later does not change the neurons; update each `neuron.step` directly to change its learning rate.
+
+```python
+STEP = 0.01
+```
+
+#### Construct a neural network by specifying the Neurons for each Layer and adding the Layers to a Network.
+
+```python
+l1 = Layer(neurons=[Neuron() for i in range(0, 4)], step=STEP)
+l2 = Layer(neurons=[Neuron() for i in range(0, 8)], step=STEP)
+l3 = Layer(neurons=[Neuron()], step=STEP)
+n1 = Network([l1, l2, l3], seed=42)
+```
+
+#### With the Network defined, you can train the network.
+
+Here the network is trained to recognize the nibble 1111 as the decimal number 15.
+
+```python
+n1.train([1,1,1,1], [15])
+```
+
+You can generate and print a prediction using the `predict` method. Because the network underwent just one iteration of training, the estimate will likely be inaccurate. The accuracy of the prediction can be improved by iteratively training the network. Please see [the nibble challenge](#the-nibble-challenge) for an example of how to iteratively train the network.
+
+```python
+prediction = n1.predict([1,1,1,1])
+print(prediction)
+```
+
+### How a neuron learns
+
+Consider the training call from the example above:
+
+```python
+n1.train([1, 1, 1, 1], [15])
+```
+
+The four inputs are the bits of `1111`; the target output is `15`. Here is how this one call changes the network. The excerpts below come from `Network` and `Neuron`.
+
+#### 1. Calculate a prediction
+
+`train` checks that there is one target per output neuron and that every non-input neuron has a finite, positive learning rate (`step`). It then calls:
+
+```python
+self.predict(X_train)
+```
+
+`predict` checks the inputs and connections, assigns the four input values, and calls `activate` on each hidden neuron, then the output neuron. Inside `Neuron.activate`:
+
+```python
+total = self.bias
+for source, weight in self.weights.items():
+    total += source.value * weight
+self.value = tanh(total) if self.activation == "tanh" else total
+```
+
+Each neuron adds its bias to the weighted inputs and stores its activation in `value`. This example uses linear activation, so `value` is simply `total`. Hidden neurons can also use `tanh`.
+
+#### 2. Clear old gradients and calculate the output error
+
+Back in `train`, gradients from the previous call are cleared before calculating the output gradients:
+
+```python
+for layer in self.layers:
+    for neuron in layer.neurons:
+        neuron.gradient = 0.0
+        neuron.delta = 0.0
+for neuron, target in zip(self.output_layer.neurons, y_train):
+    neuron.gradient = neuron.value - target
+```
+
+The loss is half the sum of squared output errors. A neuron's `gradient` is the derivative of that loss with respect to its output. For our output neuron, this starts as the prediction minus `15`.
+
+#### 3. Propagate gradients backward
+
+`train` visits the output layer first, then the hidden layers:
+
+```python
+for layer in reversed(self.layers[1:]):
+    for neuron in layer.neurons:
+        neuron.propagate()
+```
+
+Inside `Neuron.propagate`:
+
+```python
+derivative = 1 - self.value ** 2 if self.activation == "tanh" else 1.0
+self.delta = self.gradient * derivative
+for source, weight in self.weights.items():
+    source.gradient += self.delta * weight
+```
+
+`delta` is the derivative of the loss with respect to the weighted sum plus bias, before activation. For linear activation, the derivative is `1`, so `delta` equals `gradient`. Each neuron passes a contribution back through each incoming connection. The `+=` lets a source accumulate contributions from every neuron it feeds. Visiting layers in reverse ensures those contributions arrive before the source propagates its own gradient.
+
+#### 4. Update weights and biases
+
+Only after the backward pass finishes does `train` change the parameters:
+
+```python
+for layer in self.layers[1:]:
+    for neuron in layer.neurons:
+        for source in neuron.weights:
+            neuron.weights[source] -= neuron.step * neuron.delta * source.value
+        neuron.bias -= neuron.step * neuron.delta
+```
+
+A weight's gradient is `delta * source.value`; the bias gradient is `delta`. Subtracting `step` times each gradient updates the parameters. Waiting until this point means every backward calculation uses the same weights as the forward pass. The next call to `predict` uses the updated weights and biases.
+
+Each incoming connection has its own weight in `neuron.weights[source]`. `Neuron(bias=...)` sets the initial bias, and `Network(..., seed=42)` initializes weights reproducibly in [-0.1, 0.1]. Connections added later accept an explicit `weight`; repeated connection calls preserve existing weights.
+
+### Explore a trained network
+
+After training, remove a connection and compare predictions to see how it affects the network. This experiment is called an ablation.
+
+Using your trained network as `n1`:
+
+```python
+inputs = [1, 0, 1, 0]
+source = n1.layers[1].neurons[0]
+receiver = n1.output_layer.neurons[0]
+weight = receiver.weights[source]
+before = n1.predict(inputs)
+
+source.disconnectRHS(receiver)
+after = n1.predict(inputs)
+print("Original prediction:", before)
+print("Without the connection:", after)
+
+source.connectRHS(receiver, weight=weight)
+```
+
+Try several inputs: a connection's effect can depend on the input.
+
+You can also change an existing weight through `receiver.weights[source]`. Use the connection methods to add or remove edges so incoming weights and outgoing connections stay consistent.
+
+## Examples
+
+### The nibble challenge
+
+#### Setup
+
+Change directory into the example project from the repository root.
+
+```
+cd examples/the_nibble_challenge
+```
+
+Install the example dependencies.
+
+```
+pip install .
+```
+
+#### Explore the notebook
+
+Open [the nibble challenge notebook](https://github.com/adamjpatterson/neural-lab/blob/main/examples/the_nibble_challenge/main.ipynb). It introduces all sixteen four-bit inputs, constructs a 4 → 8 → 1 network, compares initial and trained predictions, and lets you change an input to inspect weights and activations. With seed 42, a step of 0.01, and 10,000 training examples, the displayed R² rounds to 1.000000000.
+
+The network learns a simple linear relationship. Training and evaluation both use all sixteen nibbles; hence, the experiment does not provide evidence of generalization. The notebook imports `visualize_network` from [its visualization module](https://github.com/adamjpatterson/neural-lab/blob/main/examples/the_nibble_challenge/visualizations.py).
+
+##### Visualizations of the network before and after training
+
+![Nibble network before and after training](https://raw.githubusercontent.com/adamjpatterson/neural-lab/main/images/nibble_multiweight.png)
+
+- Edge thickness is proportional to the absolute connection weight. Blue is positive; orange is negative. Zero-weight edges have zero width.
+- Node area above a small visibility floor is proportional to absolute activation. Green nodes are inputs, red nodes are hidden, and yellow is the output.
+- Both panels use the same input, positions, and scales. Node size represents activity, not importance, and does not encode the activation's sign.
+
+This visualization demonstrates that an irregular-looking network can learn a simple mapping almost perfectly: individual connections can reinforce or cancel one another.
+
+### The digits experiment
+
+#### Setup
+
+Change directory into the example project from the repository root.
+
+```bash
+cd examples/the_digits_experiment
+```
+
+Install the example dependencies.
+
+```bash
+pip install .
+```
+
+#### Explore the notebook
+
+Open [the digits experiment notebook](https://github.com/adamjpatterson/neural-lab/blob/main/examples/the_digits_experiment/main.ipynb). It starts by displaying the [8×8 digits dataset](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html), then normalizes pixels, creates a training/test split, constructs a 64 → 16 → 10 network, trains, and evaluates unseen images. You can inspect an individual prediction and its ten output scores. Scores are not probabilities.
+
+The reproducible settings use seed 42, 1,347 training images, 450 test images, 20 epochs, and a step of 0.001. The example achieves **90.22% test accuracy**, compared with **10.22%** for always predicting the most common training digit.
+
+The notebook imports `plot_predictions` and `plot_confusion_matrix` from [its visualization module](https://github.com/adamjpatterson/neural-lab/blob/main/examples/the_digits_experiment/visualizations.py).
+
+#### Digits predictions and confusion matrix
+
+The prediction figure shows the first twelve test images. The confusion matrix includes all 450 test images; rows show the true digit and columns show the predicted digit.
+
+![Digits predictions and confusion matrix](https://raw.githubusercontent.com/adamjpatterson/neural-lab/main/images/digits_results.png)
+
+## Tests
+
+The suite in `tests/test.py` checks forward calculations, manual gradients, parameter updates, connection removal and restoration, and input validation.
+
+#### Run all tests.
+
+```bash
+python -m unittest -v
+```
+
+## Documentation maintenance
+
+### Regenerating README figures
+
+The [documentation-image project](https://github.com/adamjpatterson/neural-lab/blob/main/docs/README.md) has its own dependencies and fixed seed-42 scenarios. From `docs/`, after following its environment setup:
+
+```bash
+python generate_images.py
+```
+
+It writes the nibble network image and the combined digits predictions and confusion matrix image to `images/`.
+
+## Support
+
+If you have a feature request or run into any issues, feel free to submit an [issue](https://github.com/adamjpatterson/neural-lab/issues). You’re also welcome to reach out to one of the authors.
+
+- [Adam Patterson](https://github.com/adamjpatterson)