programasweights 0.1.0__tar.gz → 0.1.0.dev1__tar.gz

programasweights-0.1.0.dev1/.gitignore

@@ -0,0 +1,76 @@
+# Dependencies
+node_modules/
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+
+# Build outputs
+dist/
+build/
+
+# Cache directories
+.vite/
+.cache/
+
+# Python
+__pycache__/
+*.pyc
+*.pyo
+.venv
+.env
+
+# System files
+.DS_Store
+Thumbs.db
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*.swm
+*.swn
+
+# Compiled models and temp files
+web-app/backend/compiled_models/
+web-app/backend/temp/
+web-app/backend/uploads/
+outputs/
+outputs_onnx/
+
+# SQLite caches
+cache.sqlite3
+verify_cache.sqlite3
+filter_cache.sqlite3
+
+# Training data (large generated files)
+data/batch_gpt52*/
+data/merged_gpt52*/
+data/regen_fuzzy_bench/
+data/dry_run_gpt52/
+data/pilot_test/
+data/vqa_images/
+data/ocr_data/
+*.jsonl
+
+# ONNX models
+*.onnx
+*.onnx_data
+
+# Compiled neural programs
+*.paw
+
+# Log files
+*.log
+
+# Handwriting/OCR data
+*.inkml
+
+# Private config
+init_private.sh
+
+# Playground scratch space
+playground/
+
+# Large inspection logs (generated on server)
+inspect_*.txt

programasweights-0.1.0.dev1/.hatch_build.toml

@@ -0,0 +1,28 @@
+[targets.sdist]
+# Explicitly include only the source package folder
+include = ["programasweights"]
+
+# Exclude large or non-source folders
+exclude = [
+  "outputs",
+  "outputs_1spec",
+  "outputs_1spec_bak",
+  "demo_weights.safetensors",
+  "dist",
+  "training",
+  "web-app",
+  "tests",
+  "data",
+  "__pycache__",
+  "*.egg-info",
+  "*.pt",
+  "*.safetensors",
+  "*.sqlite3",
+  "eval.py",
+  "train.py",
+  "upload_model.py",
+  "test_*.py",
+  ".git",
+  ".gitignore",
+  ".DS_Store"
+]

programasweights-0.1.0.dev1/.readthedocs.yaml

@@ -0,0 +1,13 @@
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"   # avoid 3.13 for now; many packages lag support
+
+sphinx:
+  configuration: docs/conf.py
+
+python:
+  install:
+    - requirements: docs/requirements.txt

programasweights-0.1.0.dev1/FLOW_SUMMARY.md

@@ -0,0 +1,202 @@
+# ProgramAsWeights Data Flow Summary
+
+## Overview
+
+This document explains how prefix tokens flow through training, compilation, and inference.
+
+## Three Stages
+
+### 1. Training (`train.py` → `training/loops/prefix_tuning_sft.py`)
+
+**Purpose**: Learn to map specs to KV caches via prefix tokens
+
+```python
+# Stage: TRAINING
+spec = "Parse (A) (B) (C) into JSON list"
+input = "(A) cat (B) dog"
+output = '["cat", "dog"]'
+
+# Flow:
+1. Add prefix tokens to spec:
+   spec_tokens = tokenize(spec)  # [tok1, tok2, ..., tokN]
+   spec_with_prefix = spec_tokens + [<PREFIX_0>, <PREFIX_1>, ..., <PREFIX_4>]
+   
+2. Run through compiler:
+   hidden = compiler(spec_with_prefix)  # [batch, seq_len, hidden_dim]
+   
+3. Extract prefix token hidden states:
+   prefix_hiddens = hidden[:, -5:, :]  # Last 5 positions
+   
+4. Map to KV cache:
+   kv_cache = mapper(prefix_hiddens)  # [batch, layers, 2, heads, prefix_steps, head_dim]
+   
+5. Use KV cache in interpreter:
+   logits = interpreter(input_tokens, past_key_values=kv_cache)
+   
+6. Compute loss against output:
+   loss = cross_entropy(logits, output_tokens)
+   
+7. Backprop through entire chain:
+   loss.backward()  # Updates: compiler embeddings, mapper, (optionally) models
+```
+
+**Saves**:
+- `checkpoint/compiler/` - Compiler model with prefix token embeddings
+- `checkpoint/interpreter/` - Interpreter model  
+- `checkpoint/compiler/mapper.pt` - Mapper weights + metadata
+
+---
+
+### 2. Compilation (`eval.py` or `paw.compile()`)
+
+**Purpose**: Convert a new spec into a standalone `.paw` file
+
+```python
+# Stage: COMPILATION  
+spec = "Extract numbers from text"  # New, unseen spec
+
+# Flow:
+1. Load trained model:
+   model = JointCompilerInterpreter(
+       compiler_model_name="checkpoint/compiler",  # Has prefix tokens!
+       interpreter_model_name="checkpoint/interpreter",
+       prefix_steps=5
+   )
+   # Note: Tokenizer loaded from checkpoint already has prefix tokens
+   
+2. Add prefix tokens to new spec:
+   spec_tokens = tokenize(spec)
+   spec_with_prefix = spec_tokens + model.prefix_token_ids
+   
+3. Run through trained compiler:
+   hidden = model.compiler(spec_with_prefix)
+   
+4. Extract prefix token hidden states:
+   prefix_hiddens = hidden[:, -5:, :]
+   
+5. Map to KV cache using trained mapper:
+   kv_cache = model.mapper(prefix_hiddens)
+   
+6. Save KV cache to .paw file:
+   save_paw_program(
+       filepath="program.paw",
+       kv_layers=kv_cache,
+       spec=spec,  # For reference only
+       base_model="interpreter_model_name",
+       prefix_steps=5
+   )
+```
+
+**Saves**:
+- `program.paw` - Contains only the KV cache (not the spec with prefix tokens!)
+
+---
+
+### 3. Inference (`paw.function()`)
+
+**Purpose**: Run a compiled program on new inputs
+
+```python
+# Stage: INFERENCE
+f = paw.function("program.paw")
+output = f("Extract from: 123 and 456")
+
+# Flow:
+1. Load .paw file:
+   kv_cache = load_paw_program("program.paw")
+   # kv_cache = precomputed KV prefix from compilation
+   # NO spec, NO prefix tokens - just the cache!
+   
+2. Tokenize input and add separator:
+   input_tokens = tokenize(input) + [<BOS>]  # Separator for generation
+   
+3. Run through interpreter with KV cache:
+   logits = interpreter(input_tokens, past_key_values=kv_cache)
+   
+4. Generate output:
+   output_tokens = generate(logits, max_new_tokens=128)
+   output = detokenize(output_tokens)
+```
+
+**No compilation needed** - the KV cache is already computed and saved!
+
+---
+
+## Key Points
+
+### Prefix Tokens:
+- ✅ **Training**: Added to every spec, embeddings learned
+- ✅ **Compilation**: Added to new spec, uses learned embeddings
+- ❌ **Inference**: NOT used - we use precomputed KV cache
+
+### What Gets Saved:
+- **Training checkpoint**: Compiler (with prefix tokens) + Interpreter + Mapper
+- **.paw file**: Only KV cache (no spec, no prefix tokens)
+
+### Why This Works:
+```
+Training:    spec + prefix_tokens → compiler → hidden → mapper → KV cache
+                                                                    ↓
+Compilation: new_spec + prefix_tokens → trained_compiler → hidden → trained_mapper → KV cache → save to .paw
+                                                                                                    ↓
+Inference:   input → interpreter(past_kv=loaded_from_.paw) → output
+```
+
+The prefix tokens are a **compilation-time mechanism** to create good KV caches. Once compiled, the KV cache is all you need!
+
+---
+
+## Checkpoint Loading
+
+When loading from checkpoint during compilation:
+
+```python
+# 1. Create model (loads tokenizer from checkpoint directory)
+model = JointCompilerInterpreter(
+    compiler_model_name="checkpoint/compiler",  # Directory path
+    ...
+)
+
+# 2. Tokenizer already has prefix tokens (saved during training)
+# 3. Model detects this and skips adding them again:
+if tokens_already_exist:
+    print("Prefix tokens already exist in tokenizer (loaded from checkpoint)")
+    num_added = 0
+else:
+    # Only happens during initial training
+    num_added = tokenizer.add_special_tokens(...)
+    compiler.resize_token_embeddings(len(tokenizer))
+```
+
+---
+
+## Summary Table
+
+| Stage        | Uses Prefix Tokens? | Uses Compiler? | Uses Interpreter? | Input                | Output              |
+|-------------|---------------------|----------------|-------------------|----------------------|---------------------|
+| Training    | ✅ Yes              | ✅ Yes         | ✅ Yes            | spec, input, output  | Updated models      |
+| Compilation | ✅ Yes              | ✅ Yes         | ❌ No             | spec                 | .paw file (KV cache)|
+| Inference   | ❌ No               | ❌ No          | ✅ Yes            | input                | output              |
+
+---
+
+## Example End-to-End
+
+```python
+# 1. TRAINING (once)
+python train.py --train-jsonl data/samples_train.jsonl
+# → Creates checkpoint/ with compiler+interpreter+mapper
+
+# 2. COMPILATION (per program)
+import programasweights as paw
+paw.compile("my_parser.paw", spec="Parse (A)(B)(C) format", checkpoint_dir="checkpoint/")
+# → Creates my_parser.paw with KV cache
+
+# 3. INFERENCE (many times)
+f = paw.function("my_parser.paw")
+result = f("(A) apple (B) banana (C) cherry")
+print(result)  # ["apple", "banana", "cherry"]
+# → Fast! No compilation, just interpreter + cached KV
+```
+
+The magic: **Train once, compile many programs, run them infinitely fast!**

{programasweights-0.1.0 → programasweights-0.1.0.dev1}/MANIFEST.in

@@ -22,3 +22,4 @@ exclude upload_model.py
 exclude *.pt
 exclude *.safetensors
 exclude *.sqlite3
+exclude outputs/*

programasweights-0.1.0.dev1/ONNX_MIGRATION_PLAN.md

@@ -0,0 +1,84 @@
+# ONNX Migration Plan
+
+Complete plan for migrating programasweights to ONNX Runtime.
+
+## Goals
+
+- **10-15x faster Time-to-First-Result** (2 min vs 15-20 min)
+- **5-10x smaller disk usage** (500MB vs 3-4GB)
+- **Maintain correctness** (outputs match PyTorch exactly)
+- **Backward compatible** (same API: `paw.function()`)
+
+## Architecture
+
+### Three ONNX Models:
+
+**1. Text Embeddings (`text_embeddings.onnx`)**
+- Input: `input_ids` [batch, seq_len] int64
+- Output: `embeddings` [batch, seq_len, hidden_size] float32
+- Source: `model.get_input_embeddings()`
+
+**2. Image Encoder (`image_encoder.onnx`)**  
+- Input: `pixel_values` [batch, 3, 224, 224] float32
+- Output: `image_embeddings` [batch, 196, hidden_size] float32
+- Source: CLIP vision encoder
+
+**3. Interpreter (`interpreter.onnx`)**
+- Inputs:
+  - `embeddings` [batch, seq_len, hidden_size] float32
+  - `past_key_values` [2*num_layers, batch, num_heads, past_len, head_dim] float32
+- Outputs:
+  - `logits` [batch, seq_len, vocab_size] float32
+  - `new_key_values` [2*num_layers, batch, num_heads, NEW_seq_len, head_dim] float32
+- Source: Interpreter model (forward pass only, KV handled externally)
+
+### KV Cache Strategy:
+
+**Return NEW cache only** (not full updated cache):
+```python
+# Python handles concatenation
+past_kv = initial_kv  # [2*L, B, H, 5, D] (prefix)
+new_kv = interpreter(emb, past_kv)  # [2*L, B, H, 1, D] (new token)
+updated_kv = concat(past_kv, new_kv, axis=3)  # [2*L, B, H, 6, D]
+```
+
+This matches transformers' behavior and is memory-efficient.
+
+## Implementation Steps
+
+### Phase 1: Export Models (Text-only)
+1. Export text embeddings model
+2. Export interpreter model (with KV cache I/O)
+3. Test correctness (compare PyTorch vs ONNX outputs)
+
+### Phase 2: ONNX Runtime
+1. Create ONNX-based interpreter class
+2. Implement KV cache management
+3. Implement generation loop
+4. Test end-to-end
+
+### Phase 3: Image Support
+1. Export image encoder
+2. Update runtime to handle images
+3. Test multimodal programs
+
+### Phase 4: Optimization
+1. Quantization (INT8)
+2. Graph optimization
+3. Benchmark improvements
+
+## Scripts to Create
+
+1. `export_to_onnx.py` - Export PyTorch models to ONNX
+2. `test_onnx_correctness.py` - Verify ONNX outputs match PyTorch
+3. `onnx_runtime/interpreter.py` - ONNX-based runtime
+4. `benchmark_onnx.py` - Benchmark ONNX version
+
+## Success Criteria
+
+- ✅ All tests pass (outputs match within 1e-5)
+- ✅ Time-to-First-Result < 2 minutes
+- ✅ Disk usage < 500MB
+- ✅ Inference latency < 100ms
+- ✅ API unchanged (backward compatible)
+

programasweights-0.1.0.dev1/PKG-INFO

@@ -0,0 +1,127 @@
+Metadata-Version: 2.4
+Name: programasweights
+Version: 0.1.0.dev1
+Summary: Compile natural language specifications into neural programs that run locally via llama.cpp.
+Author-email: ProgramAsWeights <support@programasweights.com>
+License: MIT
+Keywords: inference,llama-cpp,lora,neural-programs,nlp
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Requires-Dist: httpx>=0.27.0
+Requires-Dist: llama-cpp-python>=0.3.0
+Provides-Extra: test
+Requires-Dist: pytest; extra == 'test'
+Description-Content-Type: text/markdown
+
+# ProgramAsWeights
+
+**Compile natural language specifications into neural programs (.paw files) that run locally.**
+
+Programs are stored as weight blobs (KV cache prefix + optional LoRA adapters) interpreted by a small fixed model. No API calls needed at runtime — fully deterministic, local execution.
+
+## Installation
+
+```bash
+pip install programasweights
+```
+
+## Quick Start
+
+### Run a Program
+
+```python
+import programasweights as paw
+
+# Load and run a compiled program
+fn = paw.function("program_id_or_path.paw")
+result = fn("Contact alice@company.com or bob@example.org")
+print(result)  # ["alice@company.com", "bob@example.org"]
+```
+
+### Compile a Program
+
+```python
+import programasweights as paw
+
+# Compile from natural language specification
+paw.compile(
+    "output.paw",
+    spec="Extract all email addresses from text and return as JSON list",
+    checkpoint_dir="path/to/trained/compiler",
+)
+```
+
+## LoRA Support (PEFT Compatible)
+
+Already using PEFT for LoRA training? Convert to .paw in one line:
+
+```python
+import programasweights as paw
+
+# Standard PEFT workflow:
+# model = get_peft_model(base_model, LoraConfig(r=16, target_modules=["q_proj", "v_proj"]))
+# trainer.train()
+# model.save_pretrained("my_adapter/")
+
+# Convert to .paw:
+paw.from_peft(
+    "my_adapter/",       # Your PEFT checkpoint
+    "sentiment.paw",     # Output .paw file
+    spec="Classify sentiment as positive or negative",
+    tags=["sentiment", "classification"],
+    examples=[
+        {"input": "Great movie!", "output": "positive"},
+        {"input": "Terrible film.", "output": "negative"},
+    ],
+)
+
+# Use it:
+fn = paw.function("sentiment.paw")
+print(fn("This is amazing!"))  # → "positive"
+```
+
+Load LoRA from a .paw file:
+
+```python
+lora_weights, lora_config = paw.load_paw_lora("sentiment.paw")
+print(lora_config)  # {"rank": 16, "alpha": 32, ...}
+```
+
+Or use `save_lora_to_paw()` directly if you have raw tensors instead of a PEFT checkpoint.
+
+## .paw File Format v2
+
+A `.paw` file is a self-contained neural program that includes:
+
+| Component | Description | Required |
+|-----------|-------------|----------|
+| KV cache prefix | Continuous program (prefix weights) | Optional |
+| Pseudo-program | Discrete text instructions | Optional |
+| LoRA adapter | Fine-tuned adapter weights | Optional |
+| Generation config | Temperature, top_p, max_tokens | Optional |
+| Metadata | Interpreter model, spec, author, tags | Required |
+
+## Program Hub
+
+Browse and share programs at [hub.programasweights.com](https://hub.programasweights.com)
+
+## Links
+
+- **Website**: [programasweights.com](https://programasweights.com)
+- **Documentation**: [programasweights.readthedocs.io](https://programasweights.readthedocs.io)
+- **GitHub**: [github.com/programasweights/programasweights](https://github.com/programasweights/programasweights)
+- **Program Hub**: [hub.programasweights.com](https://hub.programasweights.com)
+
+## License
+
+MIT