bit-ttt-engine 0.6.0__tar.gz → 0.6.2__tar.gz

{bit_ttt_engine-0.6.0 → bit_ttt_engine-0.6.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: bit-ttt-engine
-Version: 0.6.0
+Version: 0.6.2
 Classifier: Development Status :: 4 - Beta
 Classifier: Programming Language :: Rust
 Classifier: Programming Language :: Python :: Implementation :: CPython
@@ -10,6 +10,7 @@ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 License-File: LICENSE
 Summary: 1.58-bit Quantization + Test-Time Training (TTT) Implementation in Pure Rust
 Keywords: llm,rust,ttt,quantization,ai
+Home-Page: https://github.com/imonoonoko/Bit-TTT-Engine
 Author: imonoonoko
 License: MIT
 Requires-Python: >=3.8
@@ -21,14 +22,13 @@ Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/bit-ttt-engine.svg)](https://pypi.org/project/bit-ttt-engine/)
 
-**1.58-bit Quantization + Test-Time Training (TTT)** Implementation in Pure Rust.
-
-This package provides Python bindings for the Bit-TTT Engine, allowing you to run ultra-light ternary LLMs with real-time adaptation.
+**4-bit Quantization + Test-Time Training (TTT)** Implementation in Pure Rust.
 
 ## ✨ Features
-1. **Ultra-Light**: Runs large LLMs on cheap hardware using **1.58-bit (ternary) weights**.
-2. **Adaptive (TTT)**: Learns *while* inferring, adapting to context in real-time.
+1. **Fast**: **40 tokens/second** on GPU (RTX 4060 Ti).
+2. **Adaptive (TTT)**: Learns *while* inferring - unique to Bit-TTT!
 3. **Pure Rust**: High performance with minimal dependencies.
+4. **Easy**: Load GGUF models directly.
 
 ## 🚀 Installation
 
@@ -36,13 +36,60 @@ This package provides Python bindings for the Bit-TTT Engine, allowing you to ru
 pip install bit-ttt-engine
 ```
 
-## 💻 Usage
+## 💻 Quick Start (GGUF Models)
+
+```python
+from cortex_rust import GgufModel
+
+# Load model
+model = GgufModel("model.gguf", tokenizer="tokenizer.json")
+
+# Generate text
+output = model.generate(
+    "Hello, how are you?",
+    max_tokens=50,
+    temperature=0.7
+)
+print(output)
+
+# Streaming output
+model.generate_with_callback(
+    "Tell me a story",
+    lambda t: print(t, end="", flush=True),
+    max_tokens=100
+)
+```
+
+## 🧠 TTT (Test-Time Training)
+
+**TTT makes the model adapt during inference** - something no other local LLM can do!
+
+```python
+from cortex_rust import GgufModel
+
+model = GgufModel("model.gguf", tokenizer="tokenizer.json")
+
+# Enable TTT
+model.enable_ttt(layers=4, learning_rate=0.1)
+
+# Without TTT: Pass 1 == Pass 2 (same output)
+# With TTT:    Pass 1 != Pass 2 (model is learning!)
+
+out1 = model.generate("My name is Alice.", max_tokens=20)
+out2 = model.generate("My name is Alice.", max_tokens=20)
+print(f"Different: {out1 != out2}")  # True!
+
+# TTT controls
+model.disable_ttt()
+model.reset_ttt_state()
+print(model.ttt_enabled)  # False
+```
+
+## 🏗️ Legacy API (BitLlama)
 
 ```python
 import cortex_rust
-import json
 
-# Initialize Configuration
 config = cortex_rust.BitLlamaConfig(
     vocab_size=32000,
     hidden_dim=512,
@@ -50,34 +97,14 @@ config = cortex_rust.BitLlamaConfig(
     inner_lr=0.001
 )
 
-# Initialize Model (Inference)
 model = cortex_rust.BitLlama(
     config=config,
     checkpoint_path="path/to/model.safetensors",
-    device="cpu", # or "cuda"
+    device="cpu",
     tokenizer_path="path/to/tokenizer.json"
 )
 
-# Generate Text
-output = model.generate(prompt="Hello, world!", max_tokens=50)
-print(output)
-```
-
-## 🏗️ Training (TTT)
-
-```python
-trainer = cortex_rust.PyTrainer(
-    config=config,
-    checkpoint_path="path/to/model.safetensors",
-    device="cuda"
-)
-
-# Single training step
-loss = trainer.train_step(input_ids=[...], targets=[...])
-print(f"Loss: {loss}")
-
-# Save checkpoint
-trainer.save_checkpoint("model_updated.safetensors")
+output = model.generate(prompt="Hello!", max_tokens=50)
 ```
 
 ## 📖 Documentation

bit_ttt_engine-0.6.2/README_PYPI.md

@@ -0,0 +1,99 @@
+# Bit-TTT Engine: High-Performance Brain Core
+
+[![Rust](https://img.shields.io/badge/rust-1.70+-orange.svg)](https://www.rust-lang.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![PyPI](https://img.shields.io/pypi/v/bit-ttt-engine.svg)](https://pypi.org/project/bit-ttt-engine/)
+
+**4-bit Quantization + Test-Time Training (TTT)** Implementation in Pure Rust.
+
+## ✨ Features
+1. **Fast**: **40 tokens/second** on GPU (RTX 4060 Ti).
+2. **Adaptive (TTT)**: Learns *while* inferring - unique to Bit-TTT!
+3. **Pure Rust**: High performance with minimal dependencies.
+4. **Easy**: Load GGUF models directly.
+
+## 🚀 Installation
+
+```bash
+pip install bit-ttt-engine
+```
+
+## 💻 Quick Start (GGUF Models)
+
+```python
+from cortex_rust import GgufModel
+
+# Load model
+model = GgufModel("model.gguf", tokenizer="tokenizer.json")
+
+# Generate text
+output = model.generate(
+    "Hello, how are you?",
+    max_tokens=50,
+    temperature=0.7
+)
+print(output)
+
+# Streaming output
+model.generate_with_callback(
+    "Tell me a story",
+    lambda t: print(t, end="", flush=True),
+    max_tokens=100
+)
+```
+
+## 🧠 TTT (Test-Time Training)
+
+**TTT makes the model adapt during inference** - something no other local LLM can do!
+
+```python
+from cortex_rust import GgufModel
+
+model = GgufModel("model.gguf", tokenizer="tokenizer.json")
+
+# Enable TTT
+model.enable_ttt(layers=4, learning_rate=0.1)
+
+# Without TTT: Pass 1 == Pass 2 (same output)
+# With TTT:    Pass 1 != Pass 2 (model is learning!)
+
+out1 = model.generate("My name is Alice.", max_tokens=20)
+out2 = model.generate("My name is Alice.", max_tokens=20)
+print(f"Different: {out1 != out2}")  # True!
+
+# TTT controls
+model.disable_ttt()
+model.reset_ttt_state()
+print(model.ttt_enabled)  # False
+```
+
+## 🏗️ Legacy API (BitLlama)
+
+```python
+import cortex_rust
+
+config = cortex_rust.BitLlamaConfig(
+    vocab_size=32000,
+    hidden_dim=512,
+    num_layers=12,
+    inner_lr=0.001
+)
+
+model = cortex_rust.BitLlama(
+    config=config,
+    checkpoint_path="path/to/model.safetensors",
+    device="cpu",
+    tokenizer_path="path/to/tokenizer.json"
+)
+
+output = model.generate(prompt="Hello!", max_tokens=50)
+```
+
+## 📖 Documentation
+For more details, please visit the [GitHub repository](https://github.com/imonoonoko/Bit-TTT-Engine).
+
+## 🙏 Acknowledgments
+This project incorporates ideas and techniques inspired by the DroPE method published by Sakana AI.
+
+## 💖 License
+MIT License

{bit_ttt_engine-0.6.0 → bit_ttt_engine-0.6.2}/crates/rust_engine/Cargo.toml

@@ -1,8 +1,15 @@
 [package]
 name = "cortex_rust"
-version = "0.6.0"
+version = "0.6.2"
 edition = "2021"
-readme = "README.md"
+description = "High-performance LLM inference with 4-bit quantization and Test-Time Training (TTT)"
+license = "MIT"
+repository = "https://github.com/imonoonoko/Bit-TTT-Engine"
+homepage = "https://github.com/imonoonoko/Bit-TTT-Engine"
+documentation = "https://docs.rs/cortex_rust"
+readme = "README_PYPI.md"
+keywords = ["llm", "rust", "ttt", "quantization", "inference"]
+categories = ["science", "algorithms"]
 
 [lib]
 crate-type = ["cdylib", "rlib"]

bit_ttt_engine-0.6.2/crates/rust_engine/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 imonoonoko
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

bit_ttt_engine-0.6.2/crates/rust_engine/README_PYPI.md

@@ -0,0 +1,99 @@
+# Bit-TTT Engine: High-Performance Brain Core
+
+[![Rust](https://img.shields.io/badge/rust-1.70+-orange.svg)](https://www.rust-lang.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![PyPI](https://img.shields.io/pypi/v/bit-ttt-engine.svg)](https://pypi.org/project/bit-ttt-engine/)
+
+**4-bit Quantization + Test-Time Training (TTT)** Implementation in Pure Rust.
+
+## ✨ Features
+1. **Fast**: **40 tokens/second** on GPU (RTX 4060 Ti).
+2. **Adaptive (TTT)**: Learns *while* inferring - unique to Bit-TTT!
+3. **Pure Rust**: High performance with minimal dependencies.
+4. **Easy**: Load GGUF models directly.
+
+## 🚀 Installation
+
+```bash
+pip install bit-ttt-engine
+```
+
+## 💻 Quick Start (GGUF Models)
+
+```python
+from cortex_rust import GgufModel
+
+# Load model
+model = GgufModel("model.gguf", tokenizer="tokenizer.json")
+
+# Generate text
+output = model.generate(
+    "Hello, how are you?",
+    max_tokens=50,
+    temperature=0.7
+)
+print(output)
+
+# Streaming output
+model.generate_with_callback(
+    "Tell me a story",
+    lambda t: print(t, end="", flush=True),
+    max_tokens=100
+)
+```
+
+## 🧠 TTT (Test-Time Training)
+
+**TTT makes the model adapt during inference** - something no other local LLM can do!
+
+```python
+from cortex_rust import GgufModel
+
+model = GgufModel("model.gguf", tokenizer="tokenizer.json")
+
+# Enable TTT
+model.enable_ttt(layers=4, learning_rate=0.1)
+
+# Without TTT: Pass 1 == Pass 2 (same output)
+# With TTT:    Pass 1 != Pass 2 (model is learning!)
+
+out1 = model.generate("My name is Alice.", max_tokens=20)
+out2 = model.generate("My name is Alice.", max_tokens=20)
+print(f"Different: {out1 != out2}")  # True!
+
+# TTT controls
+model.disable_ttt()
+model.reset_ttt_state()
+print(model.ttt_enabled)  # False
+```
+
+## 🏗️ Legacy API (BitLlama)
+
+```python
+import cortex_rust
+
+config = cortex_rust.BitLlamaConfig(
+    vocab_size=32000,
+    hidden_dim=512,
+    num_layers=12,
+    inner_lr=0.001
+)
+
+model = cortex_rust.BitLlama(
+    config=config,
+    checkpoint_path="path/to/model.safetensors",
+    device="cpu",
+    tokenizer_path="path/to/tokenizer.json"
+)
+
+output = model.generate(prompt="Hello!", max_tokens=50)
+```
+
+## 📖 Documentation
+For more details, please visit the [GitHub repository](https://github.com/imonoonoko/Bit-TTT-Engine).
+
+## 🙏 Acknowledgments
+This project incorporates ideas and techniques inspired by the DroPE method published by Sakana AI.
+
+## 💖 License
+MIT License

bit_ttt_engine-0.6.2/crates/rust_engine/examples/basic_generate.rs

@@ -0,0 +1,134 @@
+//! Basic Text Generation Example
+//!
+//! Simple example showing how to generate text with a GGUF model.
+//!
+//! Usage:
+//!   cargo run --release --example basic_generate -- \
+//!     --model path/to/model.gguf --prompt "Hello, world!"
+
+use anyhow::{Context, Result};
+use candle_core::{Device, Tensor};
+use cortex_rust::GgufModel;
+use std::path::PathBuf;
+
+fn parse_args() -> (PathBuf, String, usize) {
+    let args: Vec<String> = std::env::args().collect();
+    let mut model_path = PathBuf::from("model.gguf");
+    let mut prompt = "Hello, how are you?".to_string();
+    let mut max_tokens = 50usize;
+
+    let mut i = 1;
+    while i < args.len() {
+        match args[i].as_str() {
+            "--model" | "-m" => {
+                if i + 1 < args.len() {
+                    model_path = PathBuf::from(&args[i + 1]);
+                    i += 1;
+                }
+            }
+            "--prompt" | "-p" => {
+                if i + 1 < args.len() {
+                    prompt = args[i + 1].clone();
+                    i += 1;
+                }
+            }
+            "--tokens" | "-t" => {
+                if i + 1 < args.len() {
+                    max_tokens = args[i + 1].parse().unwrap_or(50);
+                    i += 1;
+                }
+            }
+            "--help" | "-h" => {
+                println!("Usage: basic_generate [OPTIONS]");
+                println!();
+                println!("Options:");
+                println!("  -m, --model <PATH>   Path to GGUF model");
+                println!("  -p, --prompt <TEXT>  Prompt text");
+                println!("  -t, --tokens <N>     Max tokens to generate [default: 50]");
+                std::process::exit(0);
+            }
+            _ => {}
+        }
+        i += 1;
+    }
+
+    (model_path, prompt, max_tokens)
+}
+
+/// Simple greedy sampling
+fn sample_greedy(logits: &Tensor) -> Result<u32> {
+    let (batch, seq_len, _) = logits.dims3()?;
+    let last_logits = logits.narrow(1, seq_len - 1, 1)?.squeeze(1)?;
+    let token_ids = last_logits.argmax(1)?;
+    let token_id = if batch == 1 {
+        token_ids.squeeze(0)?.to_scalar::<u32>()?
+    } else {
+        token_ids.get(0)?.to_scalar::<u32>()?
+    };
+    Ok(token_id)
+}
+
+/// Simple byte-level tokenization (for demo purposes)
+fn tokenize(text: &str) -> Vec<i64> {
+    text.bytes().map(|b| b as i64).collect()
+}
+
+fn main() -> Result<()> {
+    let (model_path, prompt, max_tokens) = parse_args();
+
+    println!("🚀 Bit-TTT-Engine Basic Generation Example");
+    println!("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━");
+    println!("Model:  {:?}", model_path);
+    println!("Prompt: \"{}\"", prompt);
+    println!("Tokens: {}", max_tokens);
+    println!();
+
+    // Load model
+    println!("📦 Loading model...");
+    let device = Device::Cpu;
+    let mut model = GgufModel::load(&model_path, &device)
+        .context("Failed to load model")?;
+    println!("✅ Model loaded!");
+    println!("   Layers: {}", model.config().num_layers);
+    println!("   Hidden: {}", model.config().hidden_dim);
+    println!();
+
+    // Tokenize
+    let mut tokens = tokenize(&prompt);
+    let input = Tensor::from_vec(tokens.clone(), (1, tokens.len()), &device)?;
+
+    // Prefill
+    println!("⚡ Generating...");
+    let logits = model.forward(&input, 0)?;
+    let first_token = sample_greedy(&logits)?;
+    tokens.push(first_token as i64);
+
+    // Generate
+    for _ in 0..max_tokens {
+        let pos = tokens.len() - 1;
+        let input = Tensor::from_vec(vec![tokens[pos]], (1, 1), &device)?;
+        let logits = model.forward(&input, pos)?;
+        let next_token = sample_greedy(&logits)?;
+        tokens.push(next_token as i64);
+    }
+
+    // Decode (simple byte->char)
+    let output: String = tokens.iter()
+        .skip(prompt.len())
+        .filter_map(|&t| {
+            if t >= 0 && t < 256 {
+                Some(t as u8 as char)
+            } else {
+                None
+            }
+        })
+        .collect();
+
+    println!();
+    println!("📝 Output:");
+    println!("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━");
+    println!("{}", output);
+    println!("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━");
+
+    Ok(())
+}

bit_ttt_engine-0.6.2/crates/rust_engine/examples/interactive_chat.rs

@@ -0,0 +1,161 @@
+//! Interactive Chat Example
+//!
+//! A simple REPL for chatting with a GGUF model.
+//!
+//! Usage:
+//!   cargo run --release --example interactive_chat -- --model path/to/model.gguf
+
+use anyhow::{Context, Result};
+use candle_core::{Device, Tensor};
+use cortex_rust::GgufModel;
+use std::io::{self, BufRead, Write};
+use std::path::PathBuf;
+
+fn parse_args() -> (PathBuf, usize) {
+    let args: Vec<String> = std::env::args().collect();
+    let mut model_path = PathBuf::from("model.gguf");
+    let mut max_tokens = 100usize;
+
+    let mut i = 1;
+    while i < args.len() {
+        match args[i].as_str() {
+            "--model" | "-m" => {
+                if i + 1 < args.len() {
+                    model_path = PathBuf::from(&args[i + 1]);
+                    i += 1;
+                }
+            }
+            "--tokens" | "-t" => {
+                if i + 1 < args.len() {
+                    max_tokens = args[i + 1].parse().unwrap_or(100);
+                    i += 1;
+                }
+            }
+            "--help" | "-h" => {
+                println!("Interactive Chat - Bit-TTT-Engine");
+                println!();
+                println!("Usage: interactive_chat [OPTIONS]");
+                println!();
+                println!("Options:");
+                println!("  -m, --model <PATH>  Path to GGUF model");
+                println!("  -t, --tokens <N>    Max tokens per response [default: 100]");
+                std::process::exit(0);
+            }
+            _ => {}
+        }
+        i += 1;
+    }
+
+    (model_path, max_tokens)
+}
+
+fn sample_greedy(logits: &Tensor) -> Result<u32> {
+    let (batch, seq_len, _) = logits.dims3()?;
+    let last_logits = logits.narrow(1, seq_len - 1, 1)?.squeeze(1)?;
+    let token_ids = last_logits.argmax(1)?;
+    let token_id = if batch == 1 {
+        token_ids.squeeze(0)?.to_scalar::<u32>()?
+    } else {
+        token_ids.get(0)?.to_scalar::<u32>()?
+    };
+    Ok(token_id)
+}
+
+fn tokenize(text: &str) -> Vec<i64> {
+    text.bytes().map(|b| b as i64).collect()
+}
+
+fn generate(model: &mut GgufModel, prompt: &str, max_tokens: usize, device: &Device) -> Result<String> {
+    let mut tokens = tokenize(prompt);
+    let input = Tensor::from_vec(tokens.clone(), (1, tokens.len()), device)?;
+
+    // Prefill
+    let logits = model.forward(&input, 0)?;
+    let first_token = sample_greedy(&logits)?;
+    tokens.push(first_token as i64);
+
+    // Generate
+    for _ in 0..max_tokens {
+        let pos = tokens.len() - 1;
+        let input = Tensor::from_vec(vec![tokens[pos]], (1, 1), device)?;
+        let logits = model.forward(&input, pos)?;
+        let next_token = sample_greedy(&logits)?;
+        
+        // Stop on newline or special tokens
+        if next_token == 10 || next_token == 0 {
+            break;
+        }
+        tokens.push(next_token as i64);
+    }
+
+    // Decode
+    let output: String = tokens.iter()
+        .skip(prompt.len())
+        .filter_map(|&t| {
+            if t >= 32 && t < 127 {
+                Some(t as u8 as char)
+            } else {
+                None
+            }
+        })
+        .collect();
+
+    Ok(output)
+}
+
+fn main() -> Result<()> {
+    let (model_path, max_tokens) = parse_args();
+
+    println!("╔════════════════════════════════════════════╗");
+    println!("║     🤖 Bit-TTT-Engine Interactive Chat     ║");
+    println!("╚════════════════════════════════════════════╝");
+    println!();
+    println!("Model: {:?}", model_path);
+    println!("Type 'quit' or 'exit' to end.");
+    println!();
+
+    // Load model
+    println!("📦 Loading model...");
+    let device = Device::Cpu;
+    let mut model = GgufModel::load(&model_path, &device)
+        .context("Failed to load model")?;
+    println!("✅ Ready!");
+    println!();
+
+    let stdin = io::stdin();
+    let mut stdout = io::stdout();
+
+    loop {
+        print!("You: ");
+        stdout.flush()?;
+
+        let mut input = String::new();
+        stdin.lock().read_line(&mut input)?;
+        let input = input.trim();
+
+        if input.is_empty() {
+            continue;
+        }
+
+        if input.eq_ignore_ascii_case("quit") || input.eq_ignore_ascii_case("exit") {
+            println!("👋 Goodbye!");
+            break;
+        }
+
+        // Reset cache for new conversation
+        model.reset_cache();
+
+        // Generate response
+        match generate(&mut model, input, max_tokens, &device) {
+            Ok(response) => {
+                println!("🤖: {}", response.trim());
+            }
+            Err(e) => {
+                println!("❌ Error: {}", e);
+            }
+        }
+        println!();
+    }
+
+    Ok(())
+}