nanochat 0.1.0.pre

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 28b30d23549c308bf1b45d404c0ecd6f2308b5be936249e8a00b3bea53438e40
+  data.tar.gz: b9d2246356b5a8e3c67ac5c0a6c6b2bdedb2146737cf243ad73d128afce83f54
+SHA512:
+  metadata.gz: 7592265a945e4683317de282d941aa43dd24f36eda32e3e99c0d600d6f17229d7dad832db7a633c197f13c93ddf693bf3e07298cb1fb8088078d2157423ce1ac
+  data.tar.gz: 7d3eefcfa0a56dd1e1268dc85de9a1a5b7c161634b81f38fd759bca2b10313c001ba4805cda39906b59abd6766c1cd7c96304cabe179e861ab3496f2f13e169a

data/LICENSE

@@ -0,0 +1,25 @@
+MIT License
+
+Ruby implementation Copyright (c) 2025 Shannon Skipper
+Original Python implementation Copyright (c) 2025 Andrej Karpathy
+
+This Ruby port is based on nanochat by Andrej Karpathy
+(https://github.com/karpathy/nanochat)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,129 @@
+# Nanochat Ruby
+
+Ruby port of [nanochat](https://github.com/karpathy/nanochat) by Andrej Karpathy. Loads PyTorch checkpoints for inference and fine-tuning.
+
+[![Ruby](https://img.shields.io/badge/ruby-%3E%3D%203.4.0-red.svg)](https://www.ruby-lang.org/)
+[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+## Quick Start
+
+Train and chat with your own language model:
+
+```bash
+# Train a demo model (~30 mins on CPU)
+bash bin/speedrun.sh
+
+# Install Ruby gem (prerelease)
+gem install nanochat --pre
+
+# Chat with your model
+ruby examples/chat_cli.rb
+```
+
+The `bin/speedrun.sh` script trains a tiny checkpoint using [python-nanochat](https://github.com/karpathy/nanochat). Ruby nanochat loads it for inference.
+
+## Fine-Tuning
+
+```bash
+ruby examples/finetune.rb \
+  --data my_training_data.txt \
+  --epochs 3 \
+  --output custom_model.pt
+```
+
+Uses AdamW with gradient clipping. Saves best checkpoint. See `examples/sample_training_data.txt` for data format.
+
+## What Works
+
+**Complete:**
+- Inference: CLI chat, web UI, streaming, KV caching
+- Fine-tuning on single GPU
+- Tokenizer training
+- PyTorch checkpoint loading
+
+**Not implemented:**
+- Multi-GPU training (not planned)
+- Evaluation benchmarks (maybe later)
+- Base training from scratch (maybe later)
+- Reinforcement learning (maybe later)
+
+## How It Works
+
+GPT model with RoPE, GQA and causal attention. BPE tokenizer via HuggingFace. KV cache for fast inference. Temperature, top-k and top-p sampling. Auto-detects CUDA, MPS or CPU.
+
+## Requirements
+
+Ruby >= 3.4.0. LibTorch installs automatically via torch-rb.
+
+## Tokenizer Training
+
+```ruby
+require 'nanochat'
+
+tokenizer = Nanochat::Tokenizer.train_from_files(
+  ['data/train.txt'],
+  vocab_size: 50_257
+)
+tokenizer.save('~/.cache/nanochat/my_tokenizer')
+```
+
+Tokenizers trained in Python work in Ruby (both save as `.json`). RustBPE `.pkl` files don't work.
+
+## Benchmarks
+
+```bash
+ruby examples/benchmark.rb
+```
+
+Benchmarks tokenizer, forward pass, generation and sampling.
+
+## Development
+
+```bash
+# Clone repository
+git clone https://github.com/havenwood/nanochat
+cd nanochat
+
+# Install dependencies
+bundle install
+
+# Run tests
+bundle exec rake test
+
+# Run linter
+bundle exec rubocop
+
+# Run all checks
+bundle exec rake
+
+# Run performance benchmarks
+ruby examples/benchmark.rb
+```
+
+## Training Models
+
+`bin/speedrun.sh` trains a tiny d4 model (~30 mins on CPU).
+
+For production d20 models, use [python-nanochat](https://github.com/karpathy/nanochat) (~4 hours on 8×H100):
+
+```bash
+git clone https://github.com/karpathy/nanochat
+cd nanochat
+bash speedrun.sh
+```
+
+Ruby nanochat loads any checkpoint from python-nanochat.
+
+## Contributing
+
+Pull requests welcome. Priorities: evaluation benchmarks, base training, performance.
+
+## Credits
+
+Andrej Karpathy for nanochat. Andrew Kane for torch-rb and tokenizers-ruby.
+
+## Links
+
+- [python-nanochat](https://github.com/karpathy/nanochat)
+- [torch-rb](https://github.com/ankane/torch-rb)
+- [tokenizers-ruby](https://github.com/ankane/tokenizers-ruby)

data/bin/nanochat-setup

@@ -0,0 +1,186 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Setup script for Ruby nanochat - downloads pretrained checkpoint
+#
+# Usage:
+#   nanochat-setup [--tiny|--url URL]
+
+require 'fileutils'
+require 'net/http'
+require 'uri'
+require 'optparse'
+
+CACHE_DIR = File.expand_path('~/.cache/nanochat')
+CHECKPOINT_URLS = {
+  tiny: 'https://github.com/havenwood/nanochat/releases/download/v0.1.0/nanochat-tiny-d4.tar.gz'
+}.freeze
+
+def parse_options
+  options = {model: :tiny}
+
+  OptionParser.new do |opts|
+    opts.banner = 'Usage: nanochat-setup [options]'
+    opts.separator ''
+    opts.separator 'Download and setup a pretrained nanochat model'
+    opts.separator ''
+
+    opts.on('--tiny', 'Download tiny d4 model (default, ~30MB)') do
+      options[:model] = :tiny
+    end
+
+    opts.on('--url URL', 'Download from custom URL') do |url|
+      options[:url] = url
+    end
+
+    opts.on('-h', '--help', 'Show this help') do
+      puts opts
+      exit
+    end
+  end.parse!
+
+  options
+end
+
+def download_file(url, destination)
+  puts "Downloading from: #{url}"
+
+  uri = URI(url)
+  Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https') do |http|
+    request = Net::HTTP::Get.new(uri)
+
+    http.request(request) do |response|
+      case response
+      when Net::HTTPSuccess
+        total_size = response['content-length']&.to_i
+        downloaded = 0
+
+        File.open(destination, 'wb') do |file|
+          response.read_body do |chunk|
+            file.write(chunk)
+            downloaded += chunk.size
+
+            if total_size
+              percent = (downloaded.to_f / total_size * 100).round(1)
+              print "\r  Progress: #{percent}% (#{downloaded / 1_024_024}MB / #{total_size / 1_024_024}MB)"
+            else
+              print "\r  Downloaded: #{downloaded / 1_024_024}MB"
+            end
+          end
+        end
+        puts # Newline after progress
+      when Net::HTTPRedirection
+        # Follow redirect
+        redirect_url = response['location']
+        puts "  Following redirect to: #{redirect_url}"
+        return download_file(redirect_url, destination)
+      else
+        raise "Failed to download: #{response.code} #{response.message}"
+      end
+    end
+  end
+end
+
+def extract_tarball(archive_path, destination)
+  puts "Extracting to: #{destination}"
+
+  # Use tar command for extraction
+  system('tar', '-xzf', archive_path, '-C', destination, '--strip-components=1') ||
+    raise('Failed to extract tarball')
+
+  puts 'Extraction complete'
+end
+
+def verify_checkpoint(cache_dir)
+  model_path = File.join(cache_dir, 'model.pt')
+  tokenizer_path = File.join(cache_dir, 'tokenizer', 'tokenizer.json')
+
+  missing = []
+  missing << 'model.pt' unless File.exist?(model_path)
+  missing << 'tokenizer/tokenizer.json' unless File.exist?(tokenizer_path)
+
+  if missing.empty?
+    puts 'Checkpoint verified:'
+    puts "  Model: #{model_path}"
+    puts "  Tokenizer: #{tokenizer_path}"
+    true
+  else
+    puts "Missing files: #{missing.join(', ')}"
+    false
+  end
+end
+
+def main
+  options = parse_options
+
+  puts 'Nanochat Setup'
+  puts '=' * 50
+  puts
+
+  # Check if already setup
+  if File.exist?(File.join(CACHE_DIR, 'model.pt'))
+    puts 'Checkpoint already exists at ~/.cache/nanochat/'
+    print 'Overwrite? [y/N]: '
+    response = $stdin.gets&.chomp&.downcase
+
+    unless %w[y yes].include?(response)
+      puts 'Setup cancelled'
+      exit 0
+    end
+  end
+
+  # Prepare cache directory
+  FileUtils.mkdir_p(CACHE_DIR)
+
+  # Determine download URL
+  url = options[:url] || CHECKPOINT_URLS[options[:model]]
+
+  unless url
+    puts "No URL available for model: #{options[:model]}"
+    exit 1
+  end
+
+  # Download
+  archive_path = File.join(CACHE_DIR, 'download.tar.gz')
+
+  begin
+    download_file(url, archive_path)
+  rescue StandardError => e
+    puts
+    puts "Download failed: #{e.message}"
+    puts
+    puts 'Alternative: Train your own model with python-nanochat'
+    puts '  bash bin/speedrun.sh  # ~30 mins on CPU'
+    puts "  Checkpoint will be at: #{CACHE_DIR}"
+    exit 1
+  end
+
+  # Extract
+  begin
+    extract_tarball(archive_path, CACHE_DIR)
+  rescue StandardError => e
+    puts "Extraction failed: #{e.message}"
+    exit 1
+  ensure
+    FileUtils.rm_f(archive_path)
+  end
+
+  # Verify
+  puts
+  unless verify_checkpoint(CACHE_DIR)
+    puts
+    puts 'Setup incomplete - checkpoint verification failed'
+    exit 1
+  end
+
+  puts
+  puts '=' * 50
+  puts 'Setup complete!'
+  puts
+  puts 'Try it out:'
+  puts '  ruby examples/chat_cli.rb'
+  puts '  ruby examples/chat_web.rb  # Web UI on http://localhost:8000'
+  puts
+end
+
+main if __FILE__ == $PROGRAM_NAME

data/bin/package-checkpoint

@@ -0,0 +1,122 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Package a trained nanochat checkpoint for distribution
+#
+# This creates a tarball suitable for uploading to GitHub releases.
+# End users can then download it with `nanochat-setup`.
+#
+# Usage:
+#   bin/package-checkpoint [--output nanochat-tiny-d4.tar.gz]
+
+require 'fileutils'
+require 'optparse'
+
+CACHE_DIR = File.expand_path('~/.cache/nanochat')
+
+def parse_options
+  options = {output: 'nanochat-tiny-d4.tar.gz'}
+
+  OptionParser.new do |opts|
+    opts.banner = 'Usage: package-checkpoint [options]'
+    opts.separator ''
+    opts.separator 'Package a trained checkpoint for distribution'
+    opts.separator ''
+
+    opts.on('-o', '--output PATH', 'Output tarball path (default: nanochat-tiny-d4.tar.gz)') do |path|
+      options[:output] = path
+    end
+
+    opts.on('-h', '--help', 'Show this help') do
+      puts opts
+      exit
+    end
+  end.parse!
+
+  options
+end
+
+def verify_checkpoint
+  model_path = File.join(CACHE_DIR, 'model.pt')
+  tokenizer_path = File.join(CACHE_DIR, 'tokenizer', 'tokenizer.json')
+
+  missing = []
+  missing << 'model.pt' unless File.exist?(model_path)
+  missing << 'tokenizer/tokenizer.json' unless File.exist?(tokenizer_path)
+
+  if missing.empty?
+    puts "Checkpoint verified at: #{CACHE_DIR}"
+    puts "  Model: #{model_path}"
+    puts "  Tokenizer: #{tokenizer_path}"
+    true
+  else
+    puts "Missing files: #{missing.join(', ')}"
+    puts
+    puts 'Train a checkpoint first:'
+    puts '  bash bin/train-with-python-nanochat.sh'
+    false
+  end
+end
+
+def package_checkpoint(output_path)
+  output_path = File.expand_path(output_path)
+  cache_parent = File.dirname(CACHE_DIR)
+  cache_name = File.basename(CACHE_DIR)
+
+  puts
+  puts 'Creating tarball...'
+  puts "  Source: #{CACHE_DIR}"
+  puts "  Output: #{output_path}"
+  puts
+
+  # Create tarball
+  success = system(
+    'tar', '-czf', output_path,
+    '-C', cache_parent,
+    cache_name
+  )
+
+  unless success
+    puts 'Failed to create tarball'
+    exit 1
+  end
+
+  # Get size
+  size_mb = (File.size(output_path) / (1024.0 * 1024.0)).round(1)
+
+  puts "Tarball created: #{output_path}"
+  puts "  Size: #{size_mb} MB"
+  puts
+end
+
+def show_next_steps(output_path)
+  puts '=' * 70
+  puts 'Next Steps'
+  puts '=' * 70
+  puts
+  puts 'Test locally:'
+  puts "  tar -xzf #{output_path} -C #{File.dirname(CACHE_DIR)}"
+  puts '  ruby examples/chat_cli.rb'
+  puts
+  puts 'Upload to GitHub releases and users can download with nanochat-setup'
+  puts
+end
+
+def main
+  options = parse_options
+
+  puts 'Package Nanochat Checkpoint'
+  puts '=' * 70
+  puts
+
+  # Verify checkpoint exists
+  exit 1 unless verify_checkpoint
+
+  # Package it
+  package_checkpoint(options[:output])
+
+  # Show next steps
+  show_next_steps(options[:output])
+end
+
+main if __FILE__ == $PROGRAM_NAME

data/bin/speedrun.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# Train a tiny nanochat checkpoint for Ruby nanochat
+#
+# This is a simplified version adapted for CPU/single-GPU training.
+# For the full $100 8×H100 training, use python-nanochat directly.
+#
+# This script trains a d4 model (4 layers, ~30 mins on CPU)
+# The Python nanochat speedrun.sh trains a d20 model (~4 hours on 8×H100)
+
+set -e
+
+echo "======================================================================"
+echo "🚀 Ruby Nanochat - Quick Training Demo"
+echo "======================================================================"
+echo ""
+echo "This trains a tiny d4 model suitable for testing Ruby nanochat."
+echo ""
+echo "📊 Model specs:"
+echo "  • 4 layers (vs 20 in Python speedrun.sh)"
+echo "  • Runs on CPU or single GPU"
+echo "  • ~30 minutes training time"
+echo "  • Good for demos and development"
+echo ""
+echo "💡 For production models, use python-nanochat/speedrun.sh directly"
+echo "   (requires 8×H100 GPUs, ~4 hours, ~$100)"
+echo ""
+echo "======================================================================"
+echo ""
+
+# Run the actual training script
+bash bin/train-with-python-nanochat.sh

data/bin/train-tiny-model

@@ -0,0 +1,190 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Train a tiny nanochat model using Python nanochat's CPU training script
+#
+# This wraps Python nanochat's dev/runcpu.sh to make it easy for Ruby users
+# to train a minimal checkpoint without needing GPU clusters.
+#
+# Attribution: Uses training scripts from https://github.com/karpathy/nanochat
+#
+# Usage:
+#   bin/train-tiny-model [--quick|--standard]
+
+require 'English'
+require 'fileutils'
+require 'optparse'
+
+PYTHON_NANOCHAT_DIR = File.expand_path('python-nanochat')
+CACHE_DIR = File.expand_path('~/.cache/nanochat')
+
+def parse_options
+  options = {mode: :quick}
+
+  OptionParser.new do |opts|
+    opts.banner = 'Usage: train-tiny-model [options]'
+    opts.separator ''
+    opts.separator 'Train a tiny nanochat checkpoint using Python nanochat'
+    opts.separator ''
+
+    opts.on('--quick', 'Quick training (d4, 50 iterations, ~30 mins CPU, default)') do
+      options[:mode] = :quick
+    end
+
+    opts.on('--standard', 'Standard training (d4, 500 iterations, ~5 hours CPU)') do
+      options[:mode] = :standard
+    end
+
+    opts.on('-h', '--help', 'Show this help') do
+      puts opts
+      exit
+    end
+  end.parse!
+
+  options
+end
+
+def check_python_nanochat
+  unless Dir.exist?(PYTHON_NANOCHAT_DIR)
+    puts "Python nanochat not found at: #{PYTHON_NANOCHAT_DIR}"
+    puts
+    puts 'Clone it first:'
+    puts '  git clone https://github.com/karpathy/nanochat python-nanochat'
+    puts
+    exit 1
+  end
+
+  script_path = File.join(PYTHON_NANOCHAT_DIR, 'dev', 'runcpu.sh')
+  unless File.exist?(script_path)
+    puts "Training script not found: #{script_path}"
+    exit 1
+  end
+
+  puts "Python nanochat found at: #{PYTHON_NANOCHAT_DIR}"
+end
+
+def train_model(mode)
+  puts
+  puts 'Starting Training'
+  puts '=' * 70
+  puts
+  puts "Mode: #{mode} (d4 model, 4 layers)"
+  puts "Output: #{CACHE_DIR}"
+  puts "Time: ~#{mode == :quick ? '30 minutes' : '5 hours'} on CPU"
+  puts
+  puts 'Attribution: Using training scripts from'
+  puts 'https://github.com/karpathy/nanochat by Andrej Karpathy'
+  puts
+  puts '=' * 70
+  puts
+
+  # Change to python-nanochat directory
+  Dir.chdir(PYTHON_NANOCHAT_DIR) do
+    if mode == :quick
+      # Use the existing dev/runcpu.sh as-is (50 iterations)
+      puts 'Running: bash dev/runcpu.sh'
+      puts
+      system('bash', 'dev/runcpu.sh') || raise('Training failed')
+    else
+      # Modify runcpu.sh to run more iterations for better quality
+      puts 'Running modified training (500 iterations)'
+      puts
+
+      # Create a modified version inline
+      modified_script = File.read('dev/runcpu.sh')
+                            .gsub('--num_iterations=50', '--num_iterations=500')
+                            .gsub('--num_iterations=100', '--num_iterations=1000')
+
+      IO.popen('bash', 'w') do |bash|
+        bash.puts modified_script
+      end
+
+      raise 'Training failed' unless $CHILD_STATUS.success?
+    end
+  end
+
+  puts
+  puts 'Training complete!'
+end
+
+def verify_checkpoint
+  puts
+  puts 'Verifying checkpoint...'
+
+  model_path = File.join(CACHE_DIR, 'model.pt')
+  tokenizer_path = File.join(CACHE_DIR, 'tokenizer', 'tokenizer.json')
+
+  missing = []
+  missing << 'model.pt' unless File.exist?(model_path)
+  missing << 'tokenizer/tokenizer.json' unless File.exist?(tokenizer_path)
+
+  if missing.empty?
+    puts 'Checkpoint verified:'
+    puts "  Model: #{model_path}"
+    puts "  Tokenizer: #{tokenizer_path}"
+    true
+  else
+    puts "Missing files: #{missing.join(', ')}"
+    false
+  end
+end
+
+def show_next_steps
+  puts
+  puts '=' * 70
+  puts 'Try it out:'
+  puts '  ruby examples/chat_cli.rb'
+  puts '  ruby examples/chat_web.rb  # Web UI on http://localhost:8000'
+  puts
+end
+
+def main
+  options = parse_options
+
+  puts 'Train Tiny Nanochat Model'
+  puts '=' * 70
+  puts
+
+  # Check prerequisites
+  check_python_nanochat
+
+  # Warn about existing checkpoint
+  if File.exist?(File.join(CACHE_DIR, 'model.pt'))
+    puts
+    puts 'Warning: Checkpoint already exists'
+    puts "Location: #{CACHE_DIR}"
+    print 'Overwrite with new training? [y/N]: '
+    response = $stdin.gets&.chomp&.downcase
+
+    unless %w[y yes].include?(response)
+      puts 'Training cancelled'
+      exit 0
+    end
+
+    FileUtils.rm_rf(CACHE_DIR)
+  end
+
+  # Train
+  begin
+    train_model(options[:mode])
+  rescue StandardError => e
+    puts
+    puts "Training failed: #{e.message}"
+    puts
+    puts 'Make sure Python nanochat dependencies are installed'
+    puts 'Check python-nanochat/README.md for setup instructions'
+    exit 1
+  end
+
+  # Verify
+  unless verify_checkpoint
+    puts
+    puts 'Training completed but checkpoint verification failed'
+    exit 1
+  end
+
+  # Success!
+  show_next_steps
+end
+
+main if __FILE__ == $PROGRAM_NAME