pokepy-generator 1.0.1__tar.gz

pokepy_generator-1.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Anay Shekhar
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pokepy_generator-1.0.1/PKG-INFO

@@ -0,0 +1,411 @@
+Metadata-Version: 2.4
+Name: pokepy-generator
+Version: 1.0.1
+Summary: A character-level language model built from scratch using only numpy.
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: gradio>=4.0.0
+Requires-Dist: huggingface-hub<=0.24.0
+Dynamic: license-file
+
+# pokepy :)
+
+![Alt text](pokemon.jpg) </br>
+
+a character-level language model built completely from scratch using only numpy that generates pokémon sounding names. no PyTorch, no autograd, no deep learning frameworks — every forward pass, backward pass, and gradient update is manually implemented.
+
+this project started as a way to understand how neural networks actually work under the hood. I first built a simple MLP, then expanded it into a WaveNet-style architecture to explore how increasing context length changes what a model can learn.
+
+## demo
+
+try it live (local desktop):
+
+for mac users, double click PokepyLauncher.command to launch the application.
+for windows users, double click PokepyLauncher.bat to launch the application.
+
+once running, the terminal will spin up a local matrix-inference engine. open your browser and navigate to: http://localhost:10000
+
+---
+
+# what is this
+
+I wanted to understand the foundations behind language models, so I built a mini character-level text generator completely from scratch.
+
+instead of relying on existing machine learning libraries, I manually implemented:
+
+* embeddings
+* linear layers
+* batch normalization
+* tanh activations
+* softmax
+* cross entropy loss
+* backpropagation
+* gradient descent
+
+everything runs only with numpy.
+
+the model learns character patterns from pokémon names and generates new names based on the relationships it discovers.
+
+---
+
+# model 1 — MLP
+
+## architecture
+
+the first version was a simple multilayer perceptron:
+
+* character embedding layer (10-dimensional vectors)
+* linear layer
+* batch normalization
+* tanh activation
+* output linear layer
+* softmax + cross entropy loss
+
+hidden size:
+
+```
+200 neurons
+```
+
+context length:
+
+```
+3 characters
+```
+
+this means the model only looks at the previous 3 characters to predict the next one.
+
+example:
+
+```
+pik → a
+ika → next character
+```
+
+the context continuously shifts as the model generates.
+
+---
+
+## MLP results
+
+training:
+
+```
+train loss: 1.294
+validation loss: 3.504
+```
+
+generated names:
+
+```
+blipedeedo
+rosalini
+lect
+dartic
+star
+vigus
+swannon
+hippowdon
+the
+larvinerao
+```
+
+the MLP learned basic character relationships, but the limited context window made it difficult to understand longer patterns inside names.
+
+---
+
+# model 2 — WaveNet
+
+## why I built this
+
+the biggest limitation of the MLP was context length.
+
+with only 3 characters of context, the model could only see a small part of each name.
+
+for example:
+
+```
+charizard
+
+cha
+har
+ari
+riz
+```
+
+the model does not understand the larger structure of the word.
+
+WaveNet improves this by gradually combining groups of characters, allowing the model to build larger representations without massively increasing the number of parameters.
+
+---
+
+## architecture
+
+WaveNet-style architecture:
+
+* character embedding layer (10 dimensions)
+* FlattenConsecutive layers
+* multiple linear layers
+* batch normalization
+* tanh activations
+* final output layer
+* softmax + cross entropy loss
+
+context length:
+
+```
+8 characters
+```
+
+the model builds information hierarchically:
+
+```
+characters
+
+↓
+
+combined character groups
+
+↓
+
+higher level features
+
+↓
+
+next character prediction
+```
+
+---
+
+## WaveNet results
+
+training:
+
+```
+train loss: 1.949
+validation loss: 2.748
+```
+
+generated names:
+
+```
+gropinig
+pyghislacat
+poloun
+hoongel
+spuspiniyan
+ongtover
+kasato
+xel
+felspipon
+linmatie
+asherron
+beatdiqdule
+madstutf
+drudona
+rouzslra
+liwsywunk
+galeon
+magnoslaws
+araidono
+lickopt
+```
+
+WaveNet produced longer and more structured generations because it had access to a larger context window.
+
+---
+
+# MLP vs WaveNet
+
+|                  | MLP                 | WaveNet             |
+| ---------------- | ------------------- | ------------------- |
+| Context size     | 3 characters        | 8 characters        |
+| Hidden size      | 200                 | 32                  |
+| Training steps   | 300,000             | 10,000              |
+| Architecture     | Single hidden layer | Hierarchical layers |
+| Feature learning | Direct              | Progressive         |
+| Main advantage   | Simple baseline     | Larger context      |
+
+---
+
+# challenges
+
+## context length
+
+one of the biggest lessons from this project was understanding why context matters.
+
+a model with a smaller context window can only learn local patterns, while larger context allows it to understand longer relationships.
+
+the MLP used:
+
+```
+3 character context
+```
+
+while WaveNet increased this to:
+
+```
+8 character context
+```
+
+which allowed it to capture more structure from names.
+
+---
+
+## batch normalization
+
+implementing batch normalization manually was one of the hardest parts.
+
+I had to handle:
+
+* batch mean
+* batch variance
+* running mean
+* running variance
+
+training and inference use different statistics, so saving the running values was required for the deployed model to generate correctly.
+
+---
+
+## backpropagation
+
+instead of using:
+
+```python
+loss.backward()
+```
+
+I manually calculated gradients for:
+
+* embeddings
+* linear layers
+* batch normalization
+* tanh activations
+* softmax cross entropy
+
+this helped me understand how neural networks actually learn instead of treating them as black boxes.
+
+---
+
+## random generation
+
+generation is probabilistic.
+
+even with the same trained model, outputs change because the next character is sampled from the model's probability distribution.
+
+---
+
+# training details
+
+## MLP
+
+dataset:
+
+```
+pokemon names
+```
+
+training:
+
+* optimizer: SGD
+* batch size: 32
+* steps: 300,000
+* learning rate: `0.1 → 0.01 after 100k steps`
+
+parameters:
+
+```
+C
+W1
+W2
+b2
+bngain
+bnbias
+```
+
+---
+
+## WaveNet
+
+dataset:
+
+```
+pokemon names
+```
+
+training:
+
+* optimizer: SGD
+* batch size: 32
+* steps: 10,000
+* learning rate: `0.1 → 0.01 after 8000 steps`
+
+parameters:
+
+```
+embeddings
+linear layers
+batch normalization parameters
+```
+
+---
+
+# deployment
+
+the model is deployed using Hugging Face Spaces with Gradio.
+
+the demo loads the trained numpy weights and runs inference without PyTorch or external ML frameworks.
+
+the deployed model uses:
+
+* trained embeddings
+* linear layer weights
+* batch normalization parameters
+* vocabulary mappings
+
+the entire inference pipeline runs using manually implemented numpy layers.
+
+---
+
+# usage
+
+install dependencies:
+
+```bash
+pip install numpy
+```
+
+run:
+
+```bash
+python mlp.py
+python wavenet.py
+```
+
+you will need:
+
+```
+data/
+
+└── pokemon.txt
+```
+
+with one pokémon name per line.
+
+---
+
+# what I learned
+
+this project taught me how language models are built from the ground up.
+
+the biggest takeaway was that improving a model is not always about making it bigger. changing the architecture and giving the model better ways to understand context can have a larger impact than simply adding more parameters.
+
+---
+
+# license
+
+MIT
+
+heavily inspired by Andrej Karpathy's makemore series — highly recommend if you want to understand neural networks from the inside out :)