programasweights 0.1.1__tar.gz → 0.2.1__tar.gz

programasweights-0.2.1/.cursor/rules/vllm-deployment.mdc

@@ -0,0 +1,106 @@
+---
+description: vLLM deployment rules and known issues for PAW servers
+alwaysApply: true
+---
+
+# vLLM Deployment Rules
+
+## Server: 31.22.104.226 (4x RTX PRO 6000 Blackwell, 98 GB each)
+
+### Code Location
+
+All server code lives at `/root/paw-website/` (git clone of `programasweights/website`).
+
+**Deployment (the ONLY way to update code):**
+```bash
+# Local: push changes
+git push origin main
+
+# Server: pull and restart
+cd /root/paw-website && git pull
+cd server && bash scripts/start_all.sh
+```
+
+**NEVER use rsync or scp for code.** Model weights use scp between servers.
+
+### GPU Layout
+
+| GPU | Service | Port | VRAM |
+|-----|---------|------|------|
+| 0 | Pseudo-gen (Qwen3-4B) | 8001 | 90% |
+| 1 | Qwen3 compiler pooling (4B) | 8002 | 85% |
+| 2 | GPT-2 compiler pooling (4B) | 8012 | 85% |
+| 3 | Qwen3 inference (0.6B) + GPT-2 inference (124M) | 8003 + 8013 | 50% + 40% |
+
+GPU 3 services must start sequentially: GPT-2 first (smaller, faster), then Qwen3 after GPT-2 claims its VRAM.
+
+### Services
+
+| Service | Port | Working Dir |
+|---------|------|-------------|
+| vLLM pseudo-gen | 8001 | - |
+| vLLM Qwen3 compiler | 8002 | /root/paw-website/server |
+| vLLM GPT-2 compiler | 8012 | /root/paw-website/server |
+| vLLM Qwen3 inference | 8003 | - |
+| vLLM GPT-2 inference | 8013 | - |
+| Qwen3 CompileProvider | 9000 | /root/paw-website/server |
+| GPT-2 CompileProvider | 9001 | /root/paw-website/server |
+| API gateway | 8000 | /root/paw-website/server |
+
+### Startup
+
+```bash
+cd /root/paw-website/server
+bash scripts/start_all.sh    # starts everything, waits for models to load
+```
+
+Or for full redeploy (git pull + patch + start):
+```bash
+cd /root/paw-website
+bash server/scripts/restart.sh
+```
+
+### Key Files
+
+- `.env`: `/root/paw-website/server/.env` (secrets, NOT in git)
+- Provider configs: `server/provider_configs/qwen3-0.6b.json`, `server/provider_configs/gpt2.json`
+- Compiler registry: `server/compiler_registry.json`
+- vLLM launchers: `server/scripts/launch_qwen3_compiler.py`, `server/scripts/launch_gpt2_compiler.py`
+- Patch script: `server/scripts/patch_vllm.sh` (run after every pip install vllm)
+
+## Critical Environment Variables
+
+### VLLM_ALLOW_RUNTIME_LORA_UPDATING=1
+MUST be set for inference servers with --enable-lora.
+
+### VLLM_TORCH_COMPILE_LEVEL=0
+Set on servers without full CUDA dev toolchain.
+
+## vLLM Compiler Pooling (PawCompilerForPooling)
+
+- Must use `--enforce-eager` flag
+- Must use `--runner pooling` flag
+- Must run `scripts/patch_vllm.sh` after every vLLM install
+- Endpoint: `POST /v1/embeddings`
+- Launcher scripts use `if __name__ == '__main__'` guard for multiprocessing
+
+## Post-Install Patch
+
+```bash
+cd /root/paw-website/server && bash scripts/patch_vllm.sh /root/paw-venv
+```
+
+## GPT-2 Specifics
+
+- Interpreter uses NO chat template (use_chat_template: false)
+- Compiler port: 8012 (NOT 8002 which is Qwen3)
+- Inference port: 8013 (NOT 8003 which is Qwen3)
+
+## Testing
+
+Full benchmark: 12 specs, 36 examples, expected 30/36 (83.3%) for Qwen3
+```bash
+cd /root/paw-website/server
+python benchmarks/benchmark_api.py --server https://programasweights.com
+python benchmarks/stress_test.py --server https://programasweights.com --scenario all
+```

programasweights-0.2.1/.git

@@ -0,0 +1 @@
+gitdir: ../.git/modules/programasweights

programasweights-0.2.1/.github/workflows/web-sdk.yml

@@ -0,0 +1,51 @@
+name: Web SDK CI
+
+on:
+  push:
+    paths: ['web-sdk/**']
+  pull_request:
+    paths: ['web-sdk/**']
+
+jobs:
+  lint-and-test:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: web-sdk
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: recursive
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+          cache: npm
+          cache-dependency-path: web-sdk/package-lock.json
+      - run: npm ci
+      - run: npm run typecheck
+      - run: npm test
+      - run: npm run build
+
+  publish:
+    if: startsWith(github.ref, 'refs/tags/web-sdk-v')
+    needs: [lint-and-test]
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: web-sdk
+    permissions:
+      contents: read
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: recursive
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+          registry-url: https://registry.npmjs.org
+      - run: npm ci
+      - run: npm run build
+      - run: npm publish --access public --provenance
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

programasweights-0.2.1/.gitignore

@@ -0,0 +1,76 @@
+# Dependencies
+node_modules/
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+
+# Build outputs
+dist/
+build/
+
+# Cache directories
+.vite/
+.cache/
+
+# Python
+__pycache__/
+*.pyc
+*.pyo
+.venv
+.env
+
+# System files
+.DS_Store
+Thumbs.db
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*.swm
+*.swn
+
+# Compiled models and temp files
+web-app/backend/compiled_models/
+web-app/backend/temp/
+web-app/backend/uploads/
+outputs/
+outputs_onnx/
+
+# SQLite caches
+cache.sqlite3
+verify_cache.sqlite3
+filter_cache.sqlite3
+
+# Training data (large generated files)
+data/batch_gpt52*/
+data/merged_gpt52*/
+data/regen_fuzzy_bench/
+data/dry_run_gpt52/
+data/pilot_test/
+data/vqa_images/
+data/ocr_data/
+*.jsonl
+
+# ONNX models
+*.onnx
+*.onnx_data
+
+# Compiled neural programs
+*.paw
+
+# Log files
+*.log
+
+# Handwriting/OCR data
+*.inkml
+
+# Private config
+init_private.sh
+
+# Playground scratch space
+playground/
+
+# Large inspection logs (generated on server)
+inspect_*.txt

programasweights-0.2.1/.gitmodules

@@ -0,0 +1,3 @@
+[submodule "web-sdk/wllama"]
+	path = web-sdk/wllama
+	url = git@github.com:programasweights/wllama.git

programasweights-0.2.1/.hatch_build.toml

@@ -0,0 +1,28 @@
+[targets.sdist]
+# Explicitly include only the source package folder
+include = ["programasweights"]
+
+# Exclude large or non-source folders
+exclude = [
+  "outputs",
+  "outputs_1spec",
+  "outputs_1spec_bak",
+  "demo_weights.safetensors",
+  "dist",
+  "training",
+  "web-app",
+  "tests",
+  "data",
+  "__pycache__",
+  "*.egg-info",
+  "*.pt",
+  "*.safetensors",
+  "*.sqlite3",
+  "eval.py",
+  "train.py",
+  "upload_model.py",
+  "test_*.py",
+  ".git",
+  ".gitignore",
+  ".DS_Store"
+]

programasweights-0.2.1/.readthedocs.yaml

@@ -0,0 +1,13 @@
+version: 2
+
+build:
+  os: ubuntu-24.04
+  tools:
+    python: "3.12"
+
+mkdocs:
+  configuration: mkdocs.yml
+
+python:
+  install:
+    - requirements: docs/requirements.txt

programasweights-0.2.1/AGENTS.md

@@ -0,0 +1,129 @@
+# ProgramAsWeights (PAW)
+
+PAW compiles natural language specifications into tiny neural functions that run locally. Use it when you need fuzzy text processing — classification, extraction, format repair, search, triage — that regex can't handle but a full LLM is overkill for.
+
+Website: https://programasweights.com
+Docs: https://programasweights.readthedocs.io
+
+## Install
+
+```bash
+pip install programasweights --extra-index-url https://pypi.programasweights.com/simple/
+```
+
+## Usage
+
+```python
+import programasweights as paw
+
+# Use a pre-compiled function (downloads once, runs locally forever)
+fn = paw.function("email-triage")
+fn("Urgent: server is down!")  # "immediate"
+fn("Newsletter: spring picnic")  # "wait"
+
+# Compile your own from a description
+program = paw.compile(
+    "Fix malformed JSON: repair missing quotes and trailing commas",
+    compiler="paw-4b-qwen3-0.6b"  # or "paw-4b-gpt2" for smaller/faster
+)
+fn = paw.function(program.id)
+fn("{name: 'Alice',}")  # '{"name": "Alice"}'
+```
+
+## Two Compilers
+
+| | Standard (Qwen3 0.6B) | Compact (GPT-2 124M) |
+|---|---|---|
+| Compiler name | `paw-4b-qwen3-0.6b` | `paw-4b-gpt2` |
+| Accuracy | Higher | Lower |
+| Base model size | 594 MB | 105 MB |
+| Program size | ~22 MB | ~5 MB |
+| Inference speed | ~90ms (server) | ~50ms (server) |
+| Runs in browser | No | Yes |
+
+Default is Standard (Qwen3 0.6B). Use Compact (GPT-2) when you need smaller files or browser deployment.
+
+## Browser / JavaScript SDK
+
+Programs compiled with GPT-2 also run entirely in the browser via WebAssembly — no server needed. Users' data never leaves their device.
+
+```bash
+npm install @programasweights/web
+```
+
+```javascript
+import paw from '@programasweights/web';
+
+const fn = await paw.function('programasweights/email-triage');
+const result = await fn('Urgent: server is down!');
+// result: "immediate"
+```
+
+Use this when you want to offload all computation to the user's browser: zero server costs, zero API keys, full data privacy. The 105 MB base model downloads once and is cached.
+
+## When to Use PAW
+
+- **Fuzzy search** — typo-tolerant matching, semantic search, near-duplicate detection
+- **Format repair** — fix broken JSON, normalize dates, repair malformed inputs
+- **Classification** — sentiment, urgency, categories defined in your own words
+- **Extraction** — emails, names, dates from messy unstructured text
+- **Log triage** — extract errors from verbose output, filter noise
+- **Intent routing** — map user descriptions to the closest URL, menu item, or setting
+- **Agent preprocessing** — parse tool calls, validate outputs, route tasks
+
+## Authentication (optional)
+
+Everything works without authentication. Sign in for higher rate limits and program naming.
+
+```bash
+export PAW_API_KEY=paw_sk_...
+```
+
+Generate API keys at https://programasweights.com/settings. The SDK reads `PAW_API_KEY` from the environment automatically.
+
+| | Anonymous | Authenticated |
+|---|---|---|
+| Compile rate limit | 20/hr | 60/hr |
+| Name programs (slugs) | No | Yes |
+
+## Program Naming
+
+Every compiled program gets an immutable hash ID (e.g. `a6b454023d41ac9ca845`). Authenticated users can also assign a human-readable slug:
+
+```python
+program = paw.compile("Classify sentiment", slug="my-classifier")
+# program.id   -> "a6b454023d41ac9ca845"
+# program.slug -> "da03/my-classifier"
+
+fn = paw.function("da03/my-classifier")  # by slug (requires auth to create)
+fn = paw.function("a6b454023d41ac9ca845") # by hash (always works)
+```
+
+## CLI
+
+```bash
+paw compile --spec "Classify sentiment" --json
+paw run --program <program_id> --input "I love this!" --json
+paw info <program_id>             # show program metadata
+paw rename <program_id> my-slug   # name a program (requires auth)
+paw login                         # save API key
+```
+
+`--json` gives structured output for programmatic use.
+
+## API
+
+```python
+paw.compile(spec)                 # compile (returns Program with .id)
+paw.function(program_id_or_slug)  # load for local inference
+paw.login()                       # save API key
+```
+
+## Browse Programs
+
+https://programasweights.com/hub
+
+## Add PAW to Your Project
+
+Copy this file into your project as AGENTS.md:
+https://programasweights.com/agents

programasweights-0.2.1/CHANGELOG.md

@@ -0,0 +1,13 @@
+# Changelog
+
+## 0.2.0 (2026-04-01)
+
+First public release.
+
+- Compile natural language specs into neural programs via `paw.compile()`
+- Load and run programs locally via `paw.function()` (llama.cpp backend)
+- HuggingFace-style slug naming: `paw.function("da03/my-classifier")`
+- Offline support: slug cache + program cache, no internet after first use
+- CLI: `paw compile`, `paw run`, `paw info`, `paw rename`, `paw login`
+- Pre-built wheels via pypi.programasweights.com for fast install
+- Privacy enforcement: private programs only accessible by owner

programasweights-0.2.1/FLOW_SUMMARY.md

@@ -0,0 +1,202 @@
+# ProgramAsWeights Data Flow Summary
+
+## Overview
+
+This document explains how prefix tokens flow through training, compilation, and inference.
+
+## Three Stages
+
+### 1. Training (`train.py` → `training/loops/prefix_tuning_sft.py`)
+
+**Purpose**: Learn to map specs to KV caches via prefix tokens
+
+```python
+# Stage: TRAINING
+spec = "Parse (A) (B) (C) into JSON list"
+input = "(A) cat (B) dog"
+output = '["cat", "dog"]'
+
+# Flow:
+1. Add prefix tokens to spec:
+   spec_tokens = tokenize(spec)  # [tok1, tok2, ..., tokN]
+   spec_with_prefix = spec_tokens + [<PREFIX_0>, <PREFIX_1>, ..., <PREFIX_4>]
+   
+2. Run through compiler:
+   hidden = compiler(spec_with_prefix)  # [batch, seq_len, hidden_dim]
+   
+3. Extract prefix token hidden states:
+   prefix_hiddens = hidden[:, -5:, :]  # Last 5 positions
+   
+4. Map to KV cache:
+   kv_cache = mapper(prefix_hiddens)  # [batch, layers, 2, heads, prefix_steps, head_dim]
+   
+5. Use KV cache in interpreter:
+   logits = interpreter(input_tokens, past_key_values=kv_cache)
+   
+6. Compute loss against output:
+   loss = cross_entropy(logits, output_tokens)
+   
+7. Backprop through entire chain:
+   loss.backward()  # Updates: compiler embeddings, mapper, (optionally) models
+```
+
+**Saves**:
+- `checkpoint/compiler/` - Compiler model with prefix token embeddings
+- `checkpoint/interpreter/` - Interpreter model  
+- `checkpoint/compiler/mapper.pt` - Mapper weights + metadata
+
+---
+
+### 2. Compilation (`eval.py` or `paw.compile()`)
+
+**Purpose**: Convert a new spec into a standalone `.paw` file
+
+```python
+# Stage: COMPILATION  
+spec = "Extract numbers from text"  # New, unseen spec
+
+# Flow:
+1. Load trained model:
+   model = JointCompilerInterpreter(
+       compiler_model_name="checkpoint/compiler",  # Has prefix tokens!
+       interpreter_model_name="checkpoint/interpreter",
+       prefix_steps=5
+   )
+   # Note: Tokenizer loaded from checkpoint already has prefix tokens
+   
+2. Add prefix tokens to new spec:
+   spec_tokens = tokenize(spec)
+   spec_with_prefix = spec_tokens + model.prefix_token_ids
+   
+3. Run through trained compiler:
+   hidden = model.compiler(spec_with_prefix)
+   
+4. Extract prefix token hidden states:
+   prefix_hiddens = hidden[:, -5:, :]
+   
+5. Map to KV cache using trained mapper:
+   kv_cache = model.mapper(prefix_hiddens)
+   
+6. Save KV cache to .paw file:
+   save_paw_program(
+       filepath="program.paw",
+       kv_layers=kv_cache,
+       spec=spec,  # For reference only
+       base_model="interpreter_model_name",
+       prefix_steps=5
+   )
+```
+
+**Saves**:
+- `program.paw` - Contains only the KV cache (not the spec with prefix tokens!)
+
+---
+
+### 3. Inference (`paw.function()`)
+
+**Purpose**: Run a compiled program on new inputs
+
+```python
+# Stage: INFERENCE
+f = paw.function("program.paw")
+output = f("Extract from: 123 and 456")
+
+# Flow:
+1. Load .paw file:
+   kv_cache = load_paw_program("program.paw")
+   # kv_cache = precomputed KV prefix from compilation
+   # NO spec, NO prefix tokens - just the cache!
+   
+2. Tokenize input and add separator:
+   input_tokens = tokenize(input) + [<BOS>]  # Separator for generation
+   
+3. Run through interpreter with KV cache:
+   logits = interpreter(input_tokens, past_key_values=kv_cache)
+   
+4. Generate output:
+   output_tokens = generate(logits, max_new_tokens=128)
+   output = detokenize(output_tokens)
+```
+
+**No compilation needed** - the KV cache is already computed and saved!
+
+---
+
+## Key Points
+
+### Prefix Tokens:
+- ✅ **Training**: Added to every spec, embeddings learned
+- ✅ **Compilation**: Added to new spec, uses learned embeddings
+- ❌ **Inference**: NOT used - we use precomputed KV cache
+
+### What Gets Saved:
+- **Training checkpoint**: Compiler (with prefix tokens) + Interpreter + Mapper
+- **.paw file**: Only KV cache (no spec, no prefix tokens)
+
+### Why This Works:
+```
+Training:    spec + prefix_tokens → compiler → hidden → mapper → KV cache
+                                                                    ↓
+Compilation: new_spec + prefix_tokens → trained_compiler → hidden → trained_mapper → KV cache → save to .paw
+                                                                                                    ↓
+Inference:   input → interpreter(past_kv=loaded_from_.paw) → output
+```
+
+The prefix tokens are a **compilation-time mechanism** to create good KV caches. Once compiled, the KV cache is all you need!
+
+---
+
+## Checkpoint Loading
+
+When loading from checkpoint during compilation:
+
+```python
+# 1. Create model (loads tokenizer from checkpoint directory)
+model = JointCompilerInterpreter(
+    compiler_model_name="checkpoint/compiler",  # Directory path
+    ...
+)
+
+# 2. Tokenizer already has prefix tokens (saved during training)
+# 3. Model detects this and skips adding them again:
+if tokens_already_exist:
+    print("Prefix tokens already exist in tokenizer (loaded from checkpoint)")
+    num_added = 0
+else:
+    # Only happens during initial training
+    num_added = tokenizer.add_special_tokens(...)
+    compiler.resize_token_embeddings(len(tokenizer))
+```
+
+---
+
+## Summary Table
+
+| Stage        | Uses Prefix Tokens? | Uses Compiler? | Uses Interpreter? | Input                | Output              |
+|-------------|---------------------|----------------|-------------------|----------------------|---------------------|
+| Training    | ✅ Yes              | ✅ Yes         | ✅ Yes            | spec, input, output  | Updated models      |
+| Compilation | ✅ Yes              | ✅ Yes         | ❌ No             | spec                 | .paw file (KV cache)|
+| Inference   | ❌ No               | ❌ No          | ✅ Yes            | input                | output              |
+
+---
+
+## Example End-to-End
+
+```python
+# 1. TRAINING (once)
+python train.py --train-jsonl data/samples_train.jsonl
+# → Creates checkpoint/ with compiler+interpreter+mapper
+
+# 2. COMPILATION (per program)
+import programasweights as paw
+paw.compile("my_parser.paw", spec="Parse (A)(B)(C) format", checkpoint_dir="checkpoint/")
+# → Creates my_parser.paw with KV cache
+
+# 3. INFERENCE (many times)
+f = paw.function("my_parser.paw")
+result = f("(A) apple (B) banana (C) cherry")
+print(result)  # ["apple", "banana", "cherry"]
+# → Fast! No compilation, just interpreter + cached KV
+```
+
+The magic: **Train once, compile many programs, run them infinitely fast!**

programasweights-0.2.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ProgramAsWeights
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.