ai-snake-lab 0.4.8__tar.gz → 0.5.0__tar.gz

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ai-snake-lab
-Version: 0.4.8
+Version: 0.5.0
 Summary: Interactive reinforcement learning sandbox for experimenting with AI agents in a classic Snake Game environment.
 License: GPL-3.0
 License-File: LICENSE
@@ -35,6 +35,10 @@ Project-URL: Documentation, https://snakelab.osoyalce.com/
 Project-URL: Source, https://github.com/NadimGhaznavi/ai_snake_lab
 Description-Content-Type: text/markdown
 
+# AI Snake Lab
+
+---
+
 # Introduction
 
 **AI Snake Lab** is an interactive reinforcement learning sandbox for experimenting with AI agents in a classic Snake Game environment — featuring a live Textual TUI interface, flexible replay memory database, and modular model definitions.
@@ -95,10 +99,35 @@ ai-snake-lab
 
 ---
 
-# Links and Acknowledgements
+# Technical Docs
+
+- [Database Schema Documentation](/pages/db_schema.html)
+- [Project Layout](/pages/project_layout.html)
+
+---
+
+# Acknowledgements
 
-This code is based on a YouTube tutorial, [Python + PyTorch + Pygame Reinforcement Learning – Train an AI to Play Snake](https://www.youtube.com/watch?v=L8ypSXwyBds&t=1042s&ab_channel=freeCodeCamp.org) by Patrick Loeber. You can access his original code [here](https://github.com/patrickloeber/snake-ai-pytorch) on GitHub. Thank you Patrick!!! You are amazing!!!!
+The original code for this project was based on a YouTube tutorial, [Python + PyTorch + Pygame Reinforcement Learning – Train an AI to Play Snake](https://www.youtube.com/watch?v=L8ypSXwyBds&t=1042s&ab_channel=freeCodeCamp.org) by Patrick Loeber. You can access his original code [here](https://github.com/patrickloeber/snake-ai-pytorch) on GitHub. Thank you Patrick!!! You are amazing!!!! This project is a port of the pygame and matplotlib solution.
 
-Thanks also go out to Will McGugan and the [Textual](https://textual.textualize.io/) team. Textual is an amazing framework. Talk about *rapid Application Development*. Porting this took less than a day.
+Thanks also go out to Will McGugan and the [Textual](https://textual.textualize.io/) team. Textual is an amazing framework. Talk about *Rapid Application Development*. Porting this from a Pygame and MatPlotLib solution to Textual took less than a day.
 
 ---
+
+# Inspiration
+
+Creating an artificial intelligence agent, letting it loose and watching how it performs is an amazing process. It's not unlike having children, except on a much, much, much smaller scale, at least today! Watching the AI driven Snake Game is mesmerizing. I'm constantly thinking of ways I could improve it. I credit Patrick Loeber for giving me a fun project to explore the AI space.
+
+Much of my career has been as a Linux Systems administrator. My comfort zone is on the command line. I've never worked as a programmer and certainly not as a front end developer. [Textual](https://textual.textualize.io/), as a framework for building rich *Terminal User Interfaces* is exactly my speed and when I saw [Dolphie](https://github.com/charles-001/dolphie), I was blown away. Built-in, real-time plots of MySQL metrics: Amazing! 
+
+Richard S. Sutton is also an inspiration to me. His thoughts on *Reinforcement Learning* are a slow motion revolution. His criticisms of the existing AI landscape with it's focus on engineering a specific AI to do a specific task and then considering the job done is spot on. His vision for an AI agent that does continuous, non-linear learning remains the next frontier on the path to *General Artificial Intelligence*.
+
+---
+
+# Links
+
+- Patrick Loeber's [YouTube Tutorial](https://www.youtube.com/watch?v=L8ypSXwyBds&t=1042s&ab_channel=freeCodeCamp.org)
+- Will McGugan's [Textual](https://textual.textualize.io/) *Rapid Application Development* framework
+- [Dolphie](https://github.com/charles-001/dolphie): *A single pane of glass for real-time analytics into MySQL/MariaDB & ProxySQL*
+- Richard Sutton's [Homepage](http://www.incompleteideas.net/)
+- Richard Sutton [quotes](/pages/richard-sutton.html) and other materials.

ai_snake_lab-0.5.0/README.md

@@ -0,0 +1,96 @@
+# AI Snake Lab
+
+---
+
+# Introduction
+
+**AI Snake Lab** is an interactive reinforcement learning sandbox for experimenting with AI agents in a classic Snake Game environment — featuring a live Textual TUI interface, flexible replay memory database, and modular model definitions.
+
+---
+
+# 🚀 Features
+
+- 🐍 **Classic Snake environment** with customizable grid and rules
+- 🧠 **AI agent interface** supporting multiple architectures (Linear, RNN, CNN)
+- 🎮 **Textual-based simulator** for live visualization and metrics
+- 💾 **SQLite-backed replay memory** for storing frames, episodes, and runs
+- 🧩 **Experiment metadata tracking** — models, hyperparameters, state-map versions
+- 📊 **Built-in plotting** for hashrate, scores, and learning progress
+
+---
+
+# 🧰 Tech Stack
+
+| Component | Description |
+|------------|--------------|
+| **Python 3.11+** | Core language |
+| **Textual** | Terminal UI framework |
+| **SQLite3** | Lightweight replay memory + experiment store |
+| **PyTorch** *(optional)* | Deep learning backend for models |
+| **Plotext / Matplotlib** | Visualization tools |
+
+---
+
+# Installation
+
+This project is on [PyPI](https://pypi.org/project/ai-snake-lab/). You can install the *AI Snake Lab* software using `pip`.
+
+## Create a Sandbox 
+
+```shell
+python3 -m venv snake_venv
+. snake_venv/bin/activate
+```
+
+## Install the AI Snake Lab
+
+After you have activated your *venv* environment:
+
+```shell
+pip install ai-snake-lab
+```
+
+---
+
+# Running the AI Snake Lab
+
+From within your *venv* environment:
+
+```shell
+ai-snake-lab
+```
+
+---
+
+# Technical Docs
+
+- [Database Schema Documentation](/pages/db_schema.html)
+- [Project Layout](/pages/project_layout.html)
+
+---
+
+# Acknowledgements
+
+The original code for this project was based on a YouTube tutorial, [Python + PyTorch + Pygame Reinforcement Learning – Train an AI to Play Snake](https://www.youtube.com/watch?v=L8ypSXwyBds&t=1042s&ab_channel=freeCodeCamp.org) by Patrick Loeber. You can access his original code [here](https://github.com/patrickloeber/snake-ai-pytorch) on GitHub. Thank you Patrick!!! You are amazing!!!! This project is a port of the pygame and matplotlib solution.
+
+Thanks also go out to Will McGugan and the [Textual](https://textual.textualize.io/) team. Textual is an amazing framework. Talk about *Rapid Application Development*. Porting this from a Pygame and MatPlotLib solution to Textual took less than a day.
+
+---
+
+# Inspiration
+
+Creating an artificial intelligence agent, letting it loose and watching how it performs is an amazing process. It's not unlike having children, except on a much, much, much smaller scale, at least today! Watching the AI driven Snake Game is mesmerizing. I'm constantly thinking of ways I could improve it. I credit Patrick Loeber for giving me a fun project to explore the AI space.
+
+Much of my career has been as a Linux Systems administrator. My comfort zone is on the command line. I've never worked as a programmer and certainly not as a front end developer. [Textual](https://textual.textualize.io/), as a framework for building rich *Terminal User Interfaces* is exactly my speed and when I saw [Dolphie](https://github.com/charles-001/dolphie), I was blown away. Built-in, real-time plots of MySQL metrics: Amazing! 
+
+Richard S. Sutton is also an inspiration to me. His thoughts on *Reinforcement Learning* are a slow motion revolution. His criticisms of the existing AI landscape with it's focus on engineering a specific AI to do a specific task and then considering the job done is spot on. His vision for an AI agent that does continuous, non-linear learning remains the next frontier on the path to *General Artificial Intelligence*.
+
+---
+
+# Links
+
+- Patrick Loeber's [YouTube Tutorial](https://www.youtube.com/watch?v=L8ypSXwyBds&t=1042s&ab_channel=freeCodeCamp.org)
+- Will McGugan's [Textual](https://textual.textualize.io/) *Rapid Application Development* framework
+- [Dolphie](https://github.com/charles-001/dolphie): *A single pane of glass for real-time analytics into MySQL/MariaDB & ProxySQL*
+- Richard Sutton's [Homepage](http://www.incompleteideas.net/)
+- Richard Sutton [quotes](/pages/richard-sutton.html) and other materials.

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/ai/AIAgent.py

@@ -60,28 +60,22 @@ class AIAgent:
     def played_game(self, score):
         self.epsilon_algo.played_game()
 
-    def remember(self, state, action, reward, next_state, done):
+    def remember(self, state, action, reward, next_state, done, score=None):
         # Store the state, action, reward, next_state, and done in memory
-        self.memory.append((state, action, reward, next_state, done))
+        self.memory.append((state, action, reward, next_state, done, score))
 
     def set_optimizer(self, optimizer):
         self.trainer.set_optimizer(optimizer)
 
     def train_long_memory(self):
-        # Train on 5 games
-        max_games = 2
-        # Get a random full game
-        while max_games > 0:
-            max_games -= 1
-            game = self.memory.get_random_game()
-            if not game:
-                return  # no games to train on yet
-
-            for count, (state, action, reward, next_state, done) in enumerate(
-                game, start=1
-            ):
-                # print(f"Move #{count}: {action}")
-                self.trainer.train_step(state, action, reward, next_state, [done])
+        # Ask ReplayMemory for data
+        training_data = self.memory.get_training_data(n_games=1)
+        if not training_data:
+            return  # either no memory or user chose None
+
+        for state, action, reward, next_state, done, *_ in training_data:
+            self.trainer.train_step(state, action, reward, next_state, [done])
 
     def train_short_memory(self, state, action, reward, next_state, done):
+        # Always train on the current frame
         self.trainer.train_step(state, action, reward, next_state, [done])

ai_snake_lab-0.5.0/ai_snake_lab/ai/ReplayMemory.py

@@ -0,0 +1,254 @@
+"""
+ai/ReplayMemory.py
+
+    AI Snake Game Simulator
+    Author: Nadim-Daniel Ghaznavi
+    Copyright: (c) 2024-2025 Nadim-Daniel Ghaznavi
+    GitHub: https://github.com/NadimGhaznavi/ai
+    License: GPL 3.0
+
+This file contains the ReplayMemory class.
+"""
+
+import os
+import random
+import sqlite3, pickle
+import tempfile
+
+from ai_snake_lab.constants.DReplayMemory import MEM_TYPE
+from ai_snake_lab.constants.DDef import DDef
+
+
+class ReplayMemory:
+
+    def __init__(self, seed: int):
+        random.seed(seed)
+        self.batch_size = 250
+        # Valid options: shuffle, random_game or none
+        self._mem_type = MEM_TYPE.RANDOM_GAME
+        self.min_games = 1
+
+        # All of the states for a game are stored, in order.
+        self.cur_memory = []
+
+        # Get a temporary directory for the DB file
+        self._tmpfile = tempfile.NamedTemporaryFile(suffix=DDef.DOT_DB, delete=False)
+        self.db_file = self._tmpfile.name
+
+        # Connect to SQLite
+        self.conn = sqlite3.connect(self.db_file, check_same_thread=False)
+
+        # Get a cursor
+        self.cursor = self.conn.cursor()
+
+        # We don't need the file handle anymore
+        self._tmpfile.close()
+
+        # Intialize the schema
+        self.init_db()
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        self.close()
+
+    def __del__(self):
+        try:
+            self.close()
+        except Exception:
+            pass  # avoid errors on interpreter shutdown
+
+    def append(self, transition, final_score=None):
+        """Add a transition to the current game."""
+        old_state, move, reward, new_state, done, final_score = transition
+
+        self.cur_memory.append((old_state, move, reward, new_state, done))
+
+        if done:
+            if final_score is None:
+                raise ValueError("final_score must be provided when the game ends")
+
+            total_frames = len(self.cur_memory)
+
+            # Record the game
+            self.cursor.execute(
+                "INSERT INTO games (score, total_frames) VALUES (?, ?)",
+                (final_score, total_frames),
+            )
+            game_id = self.cursor.lastrowid
+
+            # Record the frames
+            for i, (state, action, reward, next_state, done) in enumerate(
+                self.cur_memory
+            ):
+                self.cursor.execute(
+                    """
+                    INSERT INTO frames (game_id, frame_index, state, action, reward, next_state, done)
+                    VALUES (?, ?, ?, ?, ?, ?, ?)
+                    """,
+                    (
+                        game_id,
+                        i,
+                        pickle.dumps(state),
+                        pickle.dumps(action),
+                        reward,
+                        pickle.dumps(next_state),
+                        done,
+                    ),
+                )
+
+            self.conn.commit()
+            self.cur_memory = []
+
+    def close(self):
+        """Close the database connection."""
+        if getattr(self, "conn", None):
+            self.conn.close()
+            self.conn = None
+        if getattr(self, "db_file", None) and os.path.exists(self.db_file):
+            os.remove(self.db_file)
+            self.db_file = None
+
+    def get_average_game_length(self):
+        self.cursor.execute("SELECT AVG(total_frames) FROM games")
+        avg = self.cursor.fetchone()[0]
+        return int(avg) if avg else 0
+
+    def get_random_frames(self, n=None):
+        if n is None:
+            n = self.get_average_game_length() or 32  # fallback if no data
+
+        self.cursor.execute(
+            "SELECT state, action, reward, next_state, done "
+            "FROM frames ORDER BY RANDOM() LIMIT ?",
+            (n,),
+        )
+        rows = self.cursor.fetchall()
+
+        frames = [
+            (
+                pickle.loads(state_blob),
+                pickle.loads(action),
+                float(reward),
+                pickle.loads(next_state_blob),
+                bool(done),
+            )
+            for state_blob, action, reward, next_state_blob, done in rows
+        ]
+        return frames
+
+    def get_random_game(self):
+        self.cursor.execute("SELECT id FROM games")
+        all_ids = [row[0] for row in self.cursor.fetchall()]
+        if not all_ids or len(all_ids) < self.min_games:
+            return False
+
+        rand_id = random.choice(all_ids)
+        self.cursor.execute(
+            "SELECT state, action, reward, next_state, done "
+            "FROM frames WHERE game_id = ? ORDER BY frame_index ASC",
+            (rand_id,),
+        )
+        rows = self.cursor.fetchall()
+        if not rows:
+            return False
+
+        game = [
+            (
+                pickle.loads(state_blob),
+                pickle.loads(action),
+                float(reward),
+                pickle.loads(next_state_blob),
+                bool(done),
+            )
+            for state_blob, action, reward, next_state_blob, done in rows
+        ]
+        return game
+
+    def get_num_games(self):
+        """Return number of games stored in the database."""
+        self.cursor.execute("SELECT COUNT(*) FROM games")
+        return self.cursor.fetchone()[0]
+
+    def get_training_data(self, n_games=None, n_frames=None):
+        """
+        Returns a list of transitions for training based on the current memory type.
+
+        - n_games: used for RANDOM_GAME (how many full games to sample)
+        - n_frames: used for SHUFFLE (how many frames to sample)
+        - Returns empty list if memory type is NONE or if database/memory is empty
+        """
+        mem_type = self.mem_type()
+
+        print(f"SELECTED memory type: {mem_type}")
+        if mem_type == MEM_TYPE.NONE:
+            return []
+
+        elif mem_type == MEM_TYPE.RANDOM_GAME:
+            n_games = n_games or 1
+            training_data = []
+            for _ in range(n_games):
+                game = self.get_random_game()
+                if game:
+                    training_data.extend(game)
+            return training_data
+
+        elif mem_type == MEM_TYPE.SHUFFLE:
+            n_frames = n_frames or self.get_average_game_length()
+            frames = self.get_random_frames(n=n_frames)
+            return frames
+
+        else:
+            raise ValueError(f"Unknown memory type: {mem_type}")
+
+    def init_db(self):
+        self.cursor.execute(
+            """
+            CREATE TABLE IF NOT EXISTS games (
+                id INTEGER PRIMARY KEY AUTOINCREMENT,
+                score INTEGER NOT NULL,
+                total_frames INTEGER NOT NULL
+            );
+            """
+        )
+        self.conn.commit()
+
+        self.cursor.execute(
+            """
+            CREATE TABLE IF NOT EXISTS frames (
+                id INTEGER PRIMARY KEY AUTOINCREMENT,
+                game_id INTEGER NOT NULL,
+                frame_index INTEGER NOT NULL,
+                state BLOB NOT NULL,
+                action BLOB NOT NULL,      
+                reward INTEGER NOT NULL,
+                next_state BLOB NOT NULL,
+                done INTEGER NOT NULL,        -- 0 or 1
+                FOREIGN KEY (game_id) REFERENCES games(id)
+            );
+            """
+        )
+        self.conn.commit()
+
+        self.cursor.execute(
+            """
+            CREATE UNIQUE INDEX IF NOT EXISTS idx_game_frame ON frames (game_id, frame_index);
+            """
+        )
+        self.conn.commit()
+
+        self.cursor.execute(
+            """
+            CREATE INDEX IF NOT EXISTS idx_frames_game_id ON frames (game_id);
+            """
+        )
+        self.conn.commit()
+
+    def mem_type(self, mem_type=None):
+        if mem_type is not None:
+            self._mem_type = mem_type
+        return self._mem_type
+
+    def set_memory(self, memory):
+        self.memory = memory

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/ai/models/ModelRNN.py

@@ -12,16 +12,19 @@ import torch
 import torch.nn as nn
 import torch.nn.functional as F
 
+from ai_snake_lab.constants.DSim import DSim
+from ai_snake_lab.constants.DModelLRNN import DModelRNN
+
 
 class ModelRNN(nn.Module):
     def __init__(self, seed: int):
         super(ModelRNN, self).__init__()
         torch.manual_seed(seed)
-        input_size = 30
-        hidden_size = 200
-        output_size = 3
-        rnn_layers = 4
-        rnn_dropout = 0.2
+        input_size = DSim.STATE_SIZE
+        hidden_size = DModelRNN.HIDDEN_SIZE
+        output_size = DSim.OUTPUT_SIZE
+        rnn_layers = DModelRNN.RNN_LAYERS
+        rnn_dropout = DModelRNN.RNN_DROPOUT
         self.m_in = nn.Sequential(
             nn.Linear(input_size, hidden_size),
             nn.ReLU(),
@@ -37,7 +40,7 @@ class ModelRNN(nn.Module):
 
     def forward(self, x):
         x = self.m_in(x)
-        inputs = x.view(1, -1, 200)
+        inputs = x.view(1, -1, DModelRNN.HIDDEN_SIZE)
         x, h_n = self.m_rnn(inputs)
         x = self.m_out(x)
         return x[len(x) - 1]

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/constants/DDef.py

@@ -14,6 +14,6 @@ from ai_snake_lab.utils.ConstGroup import ConstGroup
 class DDef(ConstGroup):
     """Defaults"""
 
-    APP_TITLE: str = "AI Snake Game Lab"
+    APP_TITLE: str = "AI Snake Lab"
     DOT_DB: str = ".db"
     MOVE_DELAY: float = 0.0

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/constants/DLabels.py

@@ -30,12 +30,14 @@ class DLabel(ConstGroup):
     GAME_SCORE: str = "Game Score"
     GAME_NUM: str = "Game Number"
     HIGHSCORE: str = "Highscore"
+    HIGHSCORES: str = "Highscores"
     MEM_TYPE: str = "Memory Type"
     MIN_EPSILON: str = "Minimum Epsilon"
     MODEL_LINEAR: str = "Linear"
     MODEL_RNN: str = "RNN"
     MODEL_TYPE: str = "Model Type"
     MOVE_DELAY: str = "Move Delay"
+    N_SLASH_A: str = "N/A"
     PAUSE: str = "Pause"
     QUIT: str = "Quit"
     RESTART: str = "Restart"

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/constants/DLayout.py

@@ -37,12 +37,17 @@ class DLayout(ConstGroup):
     GAME_BOX: str = "game_box"
     GAME_SCORE: str = "game_score"
     GAME_SCORE_PLOT: str = "game_score_plot"
+    HIGHSCORES: str = "highscores"
+    HIGHSCORES_BOX: str = "highscores_box"
+    HIGHSCORES_HEADER: str = "highscores_header"
     EPSILON_DECAY: str = "epsilon_decay"
     EPSILON_INITIAL: str = "initial_epsilon"
     EPSILON_MIN: str = "epsilon_min"
     INPUT_10: str = "input_10"
     LABEL: str = "label"
     LABEL_SETTINGS: str = "label_settings"
+    LABEL_SETTINGS_12: str = "label_settings_12"
+    MEM_TYPE: str = "memory_type"
     MOVE_DELAY: str = "move_delay"
     NUM_GAMES: str = "num_games"
     RUNTIME_BOX: str = "runtime_box"

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/constants/DModelLRNN.py

@@ -15,6 +15,6 @@ class DModelRNN(ConstGroup):
     """RNN Model Defaults"""
 
     LEARNING_RATE: float = 0.0007
-    INPUT_SIZE: int = 400
-    MAX_MEMORIES: int = 20
-    MAX_MEMORY: int = 100000
+    HIDDEN_SIZE: int = 200
+    RNN_LAYERS: int = 4
+    RNN_DROPOUT: float = 0.2

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/constants/DReplayMemory.py

@@ -14,12 +14,21 @@ from ai_snake_lab.utils.ConstGroup import ConstGroup
 class MEM_TYPE(ConstGroup):
     """Replay Memory Type"""
 
-    SHUFFLE: str = "shuffle"
-    SHUFFLE_LABEL: str = "Shuffled set"
+    NONE: str = "none"
+    NONE_LABEL: str = "None"
     RANDOM_GAME: str = "random_game"
-    RANDOM_GAME_LABEL: str = "Random game"
+    RANDOM_GAME_LABEL: str = "Random Game"
+    SHUFFLE: str = "shuffle"
+    SHUFFLE_LABEL: str = "Random Frames"
 
     MEM_TYPE_TABLE: dict = {
-        SHUFFLE: SHUFFLE_LABEL,
+        NONE: NONE_LABEL,
         RANDOM_GAME: RANDOM_GAME_LABEL,
+        SHUFFLE: SHUFFLE_LABEL,
     }
+
+    MEMORY_TYPES: list = [
+        (NONE_LABEL, NONE),
+        (RANDOM_GAME_LABEL, RANDOM_GAME),
+        (SHUFFLE_LABEL, SHUFFLE),
+    ]

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/constants/DSim.py

@@ -15,6 +15,6 @@ class DSim(ConstGroup):
     """Simulation Constants"""
 
     # Size of the statemap, this is from the GameBoard class
-    STATE_SIZE: int = 30
+    STATE_SIZE: int = 27
     # The number of "choices" the snake has: go forward, left or right.
     OUTPUT_SIZE: int = 3

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/game/GameBoard.py

@@ -82,6 +82,123 @@ class GameBoard(ScrollView):
         return out_list
 
     def get_state(self):
+        head = self.snake_head
+        direction = self.direction
+
+        # Adjacent points
+        point_l = Offset(head.x - 1, head.y)
+        point_r = Offset(head.x + 1, head.y)
+        point_u = Offset(head.x, head.y - 1)
+        point_d = Offset(head.x, head.y + 1)
+
+        # Direction flags
+        dir_l = direction == Direction.LEFT
+        dir_r = direction == Direction.RIGHT
+        dir_u = direction == Direction.UP
+        dir_d = direction == Direction.DOWN
+
+        # Length encoded in 7-bit binary
+        slb = self.get_binary(7, len(self.snake_body))
+
+        # Normalized distances to walls (0=touching, 1=center)
+        width = height = self.board_size()
+        dist_left = head.x / width
+        dist_right = (width - head.x - 1) / width
+        dist_up = head.y / height
+        dist_down = (height - head.y - 1) / height
+
+        # Relative food direction (normalized)
+        dx = self.food.x - head.x
+        dy = self.food.y - head.y
+        food_dx = dx / max(1, width)
+        food_dy = dy / max(1, height)
+
+        # Free space straight ahead
+        free_ahead = 0
+        probe = Offset(head.x, head.y)
+        while (
+            0 <= probe.x < width
+            and 0 <= probe.y < height
+            and not self.is_snake_collision(probe)
+        ):
+            free_ahead += 1
+            if dir_r:
+                probe = Offset(probe.x + 1, probe.y)
+            elif dir_l:
+                probe = Offset(probe.x - 1, probe.y)
+            elif dir_u:
+                probe = Offset(probe.x, probe.y - 1)
+            elif dir_d:
+                probe = Offset(probe.x, probe.y + 1)
+        free_ahead = free_ahead / max(width, height)  # normalize
+
+        # Local free cell count (0–4)
+        adjacent_points = [point_l, point_r, point_u, point_d]
+        local_free = (
+            sum(
+                1
+                for p in adjacent_points
+                if not self.is_wall_collision(p) and not self.is_snake_collision(p)
+            )
+            / 4.0
+        )
+
+        # Optional context (if tracked elsewhere)
+        recent_growth = getattr(self, "recent_growth", 0.0)
+        time_since_food = getattr(self, "steps_since_food", 0.0) / 100.0  # normalize
+
+        # --- EXISTING FEATURES ---
+        state = [
+            # 1-3. Snake collision directions
+            (dir_r and self.is_snake_collision(point_r))
+            or (dir_l and self.is_snake_collision(point_l))
+            or (dir_u and self.is_snake_collision(point_u))
+            or (dir_d and self.is_snake_collision(point_d)),
+            (dir_u and self.is_snake_collision(point_r))
+            or (dir_d and self.is_snake_collision(point_l))
+            or (dir_l and self.is_snake_collision(point_u))
+            or (dir_r and self.is_snake_collision(point_d)),
+            (dir_d and self.is_snake_collision(point_r))
+            or (dir_u and self.is_snake_collision(point_l))
+            or (dir_r and self.is_snake_collision(point_u))
+            or (dir_l and self.is_snake_collision(point_d)),
+            # 4-6. Wall collision directions
+            (dir_r and self.is_wall_collision(point_r))
+            or (dir_l and self.is_wall_collision(point_l))
+            or (dir_u and self.is_wall_collision(point_u))
+            or (dir_d and self.is_wall_collision(point_d)),
+            (dir_u and self.is_wall_collision(point_r))
+            or (dir_d and self.is_wall_collision(point_l))
+            or (dir_l and self.is_wall_collision(point_u))
+            or (dir_r and self.is_wall_collision(point_d)),
+            (dir_d and self.is_wall_collision(point_r))
+            or (dir_u and self.is_wall_collision(point_l))
+            or (dir_r and self.is_wall_collision(point_u))
+            or (dir_l and self.is_wall_collision(point_d)),
+            # 7-10. Direction flags
+            dir_l,
+            dir_r,
+            dir_u,
+            dir_d,
+            # 11-14. Food relative direction
+            food_dx,
+            food_dy,
+            # 15-21. Snake length bits
+            *slb,
+            # 22-26. Distances
+            dist_left,
+            dist_right,
+            dist_up,
+            dist_down,
+            free_ahead,
+            local_free,
+            recent_growth,
+            time_since_food,
+        ]
+
+        return [float(x) for x in state]
+
+    def get_state2(self):
 
         head = self.snake_head
         direction = self.direction

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/game/SnakeGame.py

@@ -155,9 +155,9 @@ class SnakeGame:
 
         ## 6. Set a negative reward if the snake head is adjacent to the snake body.
         # This is to discourage snake collisions.
-        for segment in self.snake[1:]:
-            if abs(self.head.x - segment.x) < 2 and abs(self.head.y - segment.y) < 2:
-                reward -= -2
+        # for segment in self.snake[1:]:
+        #    if abs(self.head.x - segment.x) < 2 and abs(self.head.y - segment.y) < 2:
+        #        reward -= -2
 
         self.game_reward += reward
         self.game_board.update_snake(snake=self.snake, direction=self.direction)

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/ui/AISim.py

@@ -14,9 +14,8 @@ import sys, os
 from datetime import datetime, timedelta
 
 from textual.app import App, ComposeResult
-from textual.widgets import Label, Input, Button, Static
+from textual.widgets import Label, Input, Button, Static, Log, Select
 from textual.containers import Vertical, Horizontal
-from textual.reactive import var
 from textual.theme import Theme
 
 from ai_snake_lab.constants.DDef import DDef
@@ -32,10 +31,13 @@ from ai_snake_lab.constants.DDb4EPlot import Plot
 
 from ai_snake_lab.ai.AIAgent import AIAgent
 from ai_snake_lab.ai.EpsilonAlgo import EpsilonAlgo
+
 from ai_snake_lab.game.GameBoard import GameBoard
 from ai_snake_lab.game.SnakeGame import SnakeGame
+
 from ai_snake_lab.ui.Db4EPlot import Db4EPlot
 
+
 RANDOM_SEED = 1970
 
 snake_lab_theme = Theme(
@@ -67,17 +69,15 @@ class AISim(App):
 
     ## Runtime values
     # Current epsilon value (degrades in real-time)
-    cur_epsilon_widget = Label("N/A", id=DLayout.CUR_EPSILON)
-    # Current memory type
-    cur_mem_type_widget = Label("N/A", id=DLayout.CUR_MEM_TYPE)
+    cur_epsilon_widget = Label(DLabel.N_SLASH_A, id=DLayout.CUR_EPSILON)
     # Current model type
-    cur_model_type_widget = Label("N/A", id=DLayout.CUR_MODEL_TYPE)
+    cur_model_type_widget = Label(DLabel.N_SLASH_A, id=DLayout.CUR_MODEL_TYPE)
     # Time delay between moves
     cur_move_delay = DDef.MOVE_DELAY
     # Number of stored games in the ReplayMemory
-    cur_num_games_widget = Label("N/A", id=DLayout.NUM_GAMES)
+    cur_num_games_widget = Label(DLabel.N_SLASH_A, id=DLayout.NUM_GAMES)
     # Elapsed time
-    cur_runtime_widget = Label("N/A", id=DLayout.RUNTIME)
+    cur_runtime_widget = Label(DLabel.N_SLASH_A, id=DLayout.RUNTIME)
 
     # Intial Settings for Epsilon
     initial_epsilon_input = Input(
@@ -185,6 +185,13 @@ class AISim(App):
                 ),
                 self.move_delay_input,
             ),
+            Horizontal(
+                Label(
+                    f"{DLabel.MEM_TYPE}",
+                    classes=DLayout.LABEL_SETTINGS_12,
+                ),
+                Select(MEM_TYPE.MEMORY_TYPES, compact=True, id=DLayout.MEM_TYPE),
+            ),
             id=DLayout.SETTINGS_BOX,
         )
 
@@ -202,7 +209,7 @@ class AISim(App):
             ),
             Horizontal(
                 Label(f"{DLabel.MEM_TYPE}", classes=DLayout.LABEL),
-                self.cur_mem_type_widget,
+                Label(DLabel.N_SLASH_A, id=DLayout.CUR_MEM_TYPE),
             ),
             Horizontal(
                 Label(f"{DLabel.STORED_GAMES}", classes=DLayout.LABEL),
@@ -236,7 +243,11 @@ class AISim(App):
         )
 
         # Empty fillers
-        yield Static(id=DLayout.FILLER_1)
+        yield Vertical(
+            Static(id=DLayout.HIGHSCORES_HEADER),
+            Log(highlight=False, auto_scroll=True, id=DLayout.HIGHSCORES),
+            id=DLayout.HIGHSCORES_BOX,
+        )
         yield Static(id=DLayout.FILLER_2)
         yield Static(id=DLayout.FILLER_3)
 
@@ -252,10 +263,14 @@ class AISim(App):
         settings_box.border_title = DLabel.SETTINGS
         runtime_box = self.query_one(f"#{DLayout.RUNTIME_BOX}", Vertical)
         runtime_box.border_title = DLabel.RUNTIME_VALUES
-        self.cur_mem_type_widget.update(
-            MEM_TYPE.MEM_TYPE_TABLE[self.agent.memory.mem_type()]
-        )
-        self.cur_num_games_widget.update(str(self.agent.memory.get_num_games()))
+        highscore_box = self.query_one(f"#{DLayout.HIGHSCORES_BOX}", Vertical)
+        highscore_box.border_title = DLabel.HIGHSCORES
+        cur_mem_type_widget = self.query_one(f"#{DLayout.CUR_MEM_TYPE}", Label)
+        cur_mem_type_widget.update(DLabel.N_SLASH_A)
+        highscores_header = self.query_one(f"#{DLayout.HIGHSCORES_HEADER}", Static)
+        highscores_header.update(f"   [b #3e99af]{DLabel.GAME:6s}{DLabel.SCORE:6s}[/]")
+        memory_type_widget = self.query_one(f"#{DLayout.MEM_TYPE}")
+        memory_type_widget.value = MEM_TYPE.RANDOM_GAME
         # Initial state is that the app is stopped
         self.add_class(DField.STOPPED)
         # Register the theme
@@ -305,6 +320,8 @@ class AISim(App):
             game_box = self.query_one(f"#{DLayout.GAME_BOX}", Vertical)
             game_box.border_title = ""
             game_box.border_subtitle = ""
+            highscores = self.query_one(f"#{DLayout.HIGHSCORES}", Log)
+            highscores.clear()
 
             # Recreate events and get a new thread
             self.stop_event = threading.Event()
@@ -324,13 +341,21 @@ class AISim(App):
             self.remove_class(DField.PAUSED)
             self.cur_move_delay = float(self.move_delay_input.value)
             self.cur_model_type_widget.update(self.agent.model_type())
+            memory_type_widget = self.query_one(f"#{DLayout.MEM_TYPE}")
+            self.agent.memory.mem_type(memory_type_widget.value)
+            cur_mem_type_widget = self.query_one(f"#{DLayout.CUR_MEM_TYPE}", Label)
+            cur_mem_type_widget.update(
+                MEM_TYPE.MEM_TYPE_TABLE[memory_type_widget.value]
+            )
 
-        # Reset button was pressed
+        # Defaults button was pressed
         elif button_id == DLayout.BUTTON_DEFAULTS:
             self.initial_epsilon_input.value = str(DEpsilon.EPSILON_INITIAL)
             self.epsilon_decay_input.value = str(DEpsilon.EPSILON_DECAY)
             self.epsilon_min_input.value = str(DEpsilon.EPSILON_MIN)
             self.move_delay_input.value = str(DDef.MOVE_DELAY)
+            memory_type_widget = self.query_one(f"#{DLayout.MEM_TYPE}")
+            memory_type_widget.value = MEM_TYPE.RANDOM_GAME
 
         # Quit button was pressed
         elif button_id == DLayout.BUTTON_QUIT:
@@ -339,6 +364,8 @@ class AISim(App):
         # Update button was pressed
         elif button_id == DLayout.BUTTON_UPDATE:
             self.cur_move_delay = float(self.move_delay_input.value)
+            memory_type_widget = self.query_one(f"#{DLayout.MEM_TYPE}")
+            self.agent.memory.mem_type(memory_type_widget.value)
 
     def start_sim(self):
         self.snake_game.reset()
@@ -351,6 +378,8 @@ class AISim(App):
         game_box = self.query_one(f"#{DLayout.GAME_BOX}", Vertical)
         game_box.border_title = f"{DLabel.GAME} #{self.epoch}"
         start_time = datetime.now()
+        self.cur_num_games_widget.update(str(self.agent.memory.get_num_games()))
+        highscores = self.query_one(f"#{DLayout.HIGHSCORES}", Log)
 
         while not self.stop_event.is_set():
             if self.pause_event.is_set():
@@ -363,6 +392,8 @@ class AISim(App):
             reward, game_over, score = snake_game.play_step(move)
             if score > highscore:
                 highscore = score
+                # Update the UI
+                highscores.write_line(f"{self.epoch:6d} {score:6d}")
             game_box.border_subtitle = (
                 f"{DLabel.HIGHSCORE}: {highscore}, {DLabel.SCORE}: {score}"
             )
@@ -378,7 +409,9 @@ class AISim(App):
                 game_box = self.query_one(f"#{DLayout.GAME_BOX}", Vertical)
                 game_box.border_title = f"{DLabel.GAME} #{self.epoch}"
                 # Remember the last move
-                agent.remember(old_state, move, reward, new_state, game_over)
+                agent.remember(
+                    old_state, move, reward, new_state, game_over, score=score
+                )
                 # Train long memory
                 agent.train_long_memory()
                 # Reset the game

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/ai_snake_lab/ui/AISim.tcss

@@ -2,7 +2,7 @@ Screen {
     layout: grid;
     grid-size: 3 4;
     grid-rows: 3 7 6 11 10;
-    grid-columns: 32 46 30;
+    grid-columns: 32 46 32;
 }
 
 #title {
@@ -34,6 +34,7 @@ Screen {
 }
 
 #runtime_box {
+    height: 100%;
     border-title-color: #5fc442;
     border-title-style: bold;
     border: round #0c323e;
@@ -41,7 +42,14 @@ Screen {
     background: black;
 }
 
-#filler_1 {
+#highscores_box {
+    row-span: 2;
+    border-title-color: #5fc442;
+    border-title-style: bold;
+    border: round #0c323e;
+    padding: 0 1;
+    background: black;
+
 }
 
 #filler_2 {
@@ -54,7 +62,7 @@ Screen {
     dock: bottom;
     border: round #0c323e;
     height: 15;
-    width: 108;
+    width: 110;
     background: black
 }
 
@@ -121,6 +129,11 @@ Button {
     width: 18;
 }
 
+.label_settings_12 {
+    color: #5fc442;
+    width: 12;
+}
+
 .paused #button_pause {
     display: none;
 }

{ai_snake_lab-0.4.8 → ai_snake_lab-0.5.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ai-snake-lab"
-version = "0.4.8"
+version = "0.5.0"
 description = "Interactive reinforcement learning sandbox for experimenting with AI agents in a classic Snake Game environment."
 authors = [{ name = "Nadim-Daniel Ghaznavi", email = "nghaznavi@gmail.com" }]
 license = { text = "GPL-3.0" }

ai_snake_lab-0.4.8/README.md

@@ -1,67 +0,0 @@
-# Introduction
-
-**AI Snake Lab** is an interactive reinforcement learning sandbox for experimenting with AI agents in a classic Snake Game environment — featuring a live Textual TUI interface, flexible replay memory database, and modular model definitions.
-
----
-
-# 🚀 Features
-
-- 🐍 **Classic Snake environment** with customizable grid and rules
-- 🧠 **AI agent interface** supporting multiple architectures (Linear, RNN, CNN)
-- 🎮 **Textual-based simulator** for live visualization and metrics
-- 💾 **SQLite-backed replay memory** for storing frames, episodes, and runs
-- 🧩 **Experiment metadata tracking** — models, hyperparameters, state-map versions
-- 📊 **Built-in plotting** for hashrate, scores, and learning progress
-
----
-
-# 🧰 Tech Stack
-
-| Component | Description |
-|------------|--------------|
-| **Python 3.11+** | Core language |
-| **Textual** | Terminal UI framework |
-| **SQLite3** | Lightweight replay memory + experiment store |
-| **PyTorch** *(optional)* | Deep learning backend for models |
-| **Plotext / Matplotlib** | Visualization tools |
-
----
-
-# Installation
-
-This project is on [PyPI](https://pypi.org/project/ai-snake-lab/). You can install the *AI Snake Lab* software using `pip`.
-
-## Create a Sandbox 
-
-```shell
-python3 -m venv snake_venv
-. snake_venv/bin/activate
-```
-
-## Install the AI Snake Lab
-
-After you have activated your *venv* environment:
-
-```shell
-pip install ai-snake-lab
-```
-
----
-
-# Running the AI Snake Lab
-
-From within your *venv* environment:
-
-```shell
-ai-snake-lab
-```
-
----
-
-# Links and Acknowledgements
-
-This code is based on a YouTube tutorial, [Python + PyTorch + Pygame Reinforcement Learning – Train an AI to Play Snake](https://www.youtube.com/watch?v=L8ypSXwyBds&t=1042s&ab_channel=freeCodeCamp.org) by Patrick Loeber. You can access his original code [here](https://github.com/patrickloeber/snake-ai-pytorch) on GitHub. Thank you Patrick!!! You are amazing!!!!
-
-Thanks also go out to Will McGugan and the [Textual](https://textual.textualize.io/) team. Textual is an amazing framework. Talk about *rapid Application Development*. Porting this took less than a day.
-
----

ai_snake_lab-0.4.8/ai_snake_lab/ai/ReplayMemory.py

@@ -1,148 +0,0 @@
-"""
-ai/ReplayMemory.py
-
-    AI Snake Game Simulator
-    Author: Nadim-Daniel Ghaznavi
-    Copyright: (c) 2024-2025 Nadim-Daniel Ghaznavi
-    GitHub: https://github.com/NadimGhaznavi/ai
-    License: GPL 3.0
-
-This file contains the ReplayMemory class.
-"""
-
-import os
-from collections import deque
-import random
-import sqlite3, pickle
-import tempfile
-import shutil
-
-from ai_snake_lab.constants.DReplayMemory import MEM_TYPE
-from ai_snake_lab.constants.DDef import DDef
-
-
-class ReplayMemory:
-
-    def __init__(self, seed: int):
-        random.seed(seed)
-        self.batch_size = 250
-        # Valid options: shuffle, random_game, targeted_score, random_targeted_score
-        self._mem_type = MEM_TYPE.RANDOM_GAME
-        self.min_games = 1
-        self.max_states = 15000
-        self.max_shuffle_games = 40
-        self.max_games = 500
-
-        if self._mem_type == MEM_TYPE.SHUFFLE:
-            # States are stored in a deque and a random sample will be returned
-            self.memories = deque(maxlen=self.max_states)
-
-        elif self._mem_type == MEM_TYPE.RANDOM_GAME:
-            # All of the states for a game are stored, in order, in a deque.
-            # A complete game will be returned
-            self.cur_memory = []
-
-        # Get a temporary directory for the DB file
-        self._tmpfile = tempfile.NamedTemporaryFile(suffix=DDef.DOT_DB, delete=False)
-        self.db_file = self._tmpfile.name
-
-        # Connect to SQLite
-        self.conn = sqlite3.connect(self.db_file, check_same_thread=False)
-
-        # Get a cursor
-        self.cursor = self.conn.cursor()
-
-        # We don't need the file handle anymore
-        self._tmpfile.close()
-
-        # Intialize the schema
-        self.init_db()
-
-    def __enter__(self):
-        return self
-
-    def __exit__(self, exc_type, exc_val, exc_tb):
-        self.close()
-
-    def __del__(self):
-        try:
-            self.close()
-        except Exception:
-            pass  # avoid errors on interpreter shutdown
-
-    def append(self, transition):
-        """Add a transition to the current game."""
-        if self._mem_type != MEM_TYPE.RANDOM_GAME:
-            raise NotImplementedError(
-                "Only RANDOM_GAME memory type is implemented for SQLite backend"
-            )
-
-        self.cur_memory.append(transition)
-        _, _, _, _, done = transition
-
-        if done:
-            # Serialize the full game to JSON
-            serialized = pickle.dumps(self.cur_memory)
-            self.cursor.execute(
-                "INSERT INTO games (transitions) VALUES (?)", (serialized,)
-            )
-            self.conn.commit()
-            self.cur_memory = []
-
-    def close(self):
-        """Close the database connection."""
-        if getattr(self, "conn", None):
-            self.conn.close()
-            self.conn = None
-        if getattr(self, "db_file", None) and os.path.exists(self.db_file):
-            os.remove(self.db_file)
-            self.db_file = None
-
-    def get_random_game(self):
-        """Return a random full game from the database."""
-        self.cursor.execute("SELECT id FROM games")
-        all_ids = [row[0] for row in self.cursor.fetchall()]
-        if len(all_ids) >= self.min_games:
-            rand_id = random.choice(all_ids)
-            self.cursor.execute("SELECT transitions FROM games WHERE id=?", (rand_id,))
-            row = self.cursor.fetchone()
-            if row:
-                return pickle.loads(row[0])
-        return False
-
-    def get_random_states(self):
-        mem_size = len(self.memories)
-        if mem_size < self.batch_size:
-            return self.memories
-        return random.sample(self.memories, self.batch_size)
-
-    def get_memory(self):
-        if self._mem_type == MEM_TYPE.SHUFFLE:
-            return self.get_random_states()
-
-        elif self._mem_type == MEM_TYPE.RANDOM_GAME:
-            return self.get_random_game()
-
-    def get_num_games(self):
-        """Return number of games stored in the database."""
-        self.cursor.execute("SELECT COUNT(*) FROM games")
-        return self.cursor.fetchone()[0]
-
-    def init_db(self):
-        self.cursor.execute(
-            """
-        CREATE TABLE IF NOT EXISTS games (
-            id INTEGER PRIMARY KEY AUTOINCREMENT,
-            transitions TEXT NOT NULL
-        )
-        """
-        )
-        self.conn.commit()
-
-    def mem_type(self, mem_type=None):
-        if mem_type is not None:
-            self._mem_type = mem_type
-        return self._mem_type
-
-    def set_memory(self, memory):
-        self.memory = memory