programasweights 0.3.1__tar.gz → 0.4.0__tar.gz

{programasweights-0.3.1 → programasweights-0.4.0}/AGENTS.md

@@ -38,7 +38,7 @@ fn("I love this!")  # "positive"
 ## Two Compilers
 
 - **Standard** (`paw-4b-qwen3-0.6b`) — higher accuracy, 594 MB base + ~22 MB/program. Default.
-- **Compact** (`paw-4b-gpt2`) — smaller (105 MB base + ~5 MB/program), runs in browser via WebAssembly.
+- **Compact** (`paw-4b-gpt2`) — smaller (134 MB base + ~5 MB/program), runs in browser via WebAssembly.
 
 ## When to Use PAW
 

{programasweights-0.3.1 → programasweights-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: programasweights
-Version: 0.3.1
+Version: 0.4.0
 Summary: Compile natural language specifications into neural programs that run locally via llama.cpp.
 Project-URL: Homepage, https://programasweights.com
 Project-URL: Repository, https://github.com/programasweights/programasweights-python
@@ -71,7 +71,7 @@ fn("I love this!")  # "positive"
 | --------------- | --------------------- | -------------------- |
 | Compiler name   | `paw-4b-qwen3-0.6b`   | `paw-4b-gpt2`        |
 | Accuracy        | Higher                | Lower                |
-| Base model size | 594 MB                | 105 MB               |
+| Base model size | 594 MB                | 134 MB               |
 | Program size    | ~22 MB                | ~5 MB                |
 | Local inference | ~0.05-0.5s per call   | ~0.03-0.3s per call  |
 | Runs in browser | No                    | Yes (WebAssembly)    |

{programasweights-0.3.1 → programasweights-0.4.0}/PYPI_README.md

@@ -41,7 +41,7 @@ fn("I love this!")  # "positive"
 | --------------- | --------------------- | -------------------- |
 | Compiler name   | `paw-4b-qwen3-0.6b`   | `paw-4b-gpt2`        |
 | Accuracy        | Higher                | Lower                |
-| Base model size | 594 MB                | 105 MB               |
+| Base model size | 594 MB                | 134 MB               |
 | Program size    | ~22 MB                | ~5 MB                |
 | Local inference | ~0.05-0.5s per call   | ~0.03-0.3s per call  |
 | Runs in browser | No                    | Yes (WebAssembly)    |

{programasweights-0.3.1 → programasweights-0.4.0}/docs/adr/001-llama-cpp-over-pytorch.md

@@ -17,5 +17,5 @@ Replace the PyTorch runtime with llama-cpp-python (~80MB install). Use GGUF mode
 - Inference uses Metal (Mac), CPU (Linux/Windows) — no GPU required
 - Must pre-render chat templates server-side (no transformers tokenizer on client)
 - .paw format must change from v2 (safetensors) to v3 (GGUF adapter)
-- Base model is downloaded once (~594 MB for Q6_K) and shared across all functions
+- Base model is downloaded once (~594 MB Q6_K for Qwen3, ~134 MB Q8_0 for GPT-2) and shared across all functions
 - Per-function adapter download is ~23MB (Q4_0, confirmed lossless at 4096-scale eval)

{programasweights-0.3.1 → programasweights-0.4.0}/docs/advanced/adrs.md

@@ -14,16 +14,17 @@ Concise records of major technical choices. Full ADR files may live elsewhere in
 
 ---
 
-## ADR 002: Q4_0 adapter and Q6_K base quantization
+## ADR 002: Quantization levels for base models and adapters
 
-**Decision:** Use **Q4_0** for adapters and **Q6_K** for the base model in shipped bundles.
+**Decision:** Use **Q4_0** for adapters. For base models, use **Q6_K** for Qwen3 0.6B and **Q8_0** for GPT-2.
 
 **Context:** Empirical evaluation on **4096** held-out examples across quantization settings informed the trade-off.
 
 **Consequence:**
 
-- **Q6_K base** — quality is preserved while the footprint is roughly **60% smaller** than fp16.
-- **Q4_0 adapter** — quality loss is negligible; adapter size drops to about **23 MB** versus **78 MB** for a heavier format at comparable settings in prior experiments.
+- **Q6_K base (Qwen3 0.6B)** — quality is preserved while the footprint is roughly **60% smaller** than fp16. 4096-sample eval shows no accuracy loss vs fp16.
+- **Q8_0 base (GPT-2)** — Q6_K caused ~3.5% accuracy loss on GPT-2 (4096-sample eval). Q8_0 closes the gap at only ~29 MB additional cost (134 MB vs 105 MB).
+- **Q4_0 adapter** — quality loss is negligible for both models; adapter size drops to about **23 MB** (Qwen3) / **5 MB** (GPT-2) versus **78 MB** / **19 MB** at fp16.
 
 ---
 

{programasweights-0.3.1 → programasweights-0.4.0}/docs/case-studies/site-navigation.md

@@ -21,20 +21,20 @@ Pages: /playground, /hub, /browser, /docs, /settings, /about
 
 ### Attempt 2: Reframe as classification
 
-Instead of generating URLs, output a digit: 1 = Playground, 2 = Hub, 3 = Browser, etc. The frontend maps digits to actual routes.
+Instead of generating URLs, output a short semantic label such as `playground`, `docs`, or `feedback`. The frontend maps those labels to actual destinations.
 
 ```
-Classify the user's intent. Return ONLY a single digit.
-1 = Create or compile a new program
-2 = Browse or search existing programs
-3 = Run a program in the browser
+Classify the user's intent. Return ONLY a single label.
+playground = Create or compile a new program
+hub = Browse or search existing programs
+browser = Run a program in the browser
 ...
-0 = None of the above
+none = None of the above
 ```
 
 **Result:** Accuracy jumped from ~67% to ~88%. Classification is far more reliable than generation for routing.
 
-**Lesson:** Reframe generation tasks as classification whenever possible. Output a label or digit, map it to the real value in your code.
+**Lesson:** Reframe generation tasks as classification whenever possible. Output a short label, map it to the real value in your code.
 
 ### Attempt 3: Add Q&A support
 
@@ -71,8 +71,8 @@ validator = paw.function("my-answer-validator")
 
 def handle_query(user_query: str):
     destination = router(user_query)
-    if destination != "0":
-        return {"action": "navigate", "page": PAGES[int(destination)]}
+    if destination != "none":
+        return {"action": "navigate", "page": PAGE_MAP[destination]}
 
     category = q_type(user_query)
     if category == "yes_no":
@@ -92,36 +92,36 @@ Each program has a focused spec. For example, the page classifier:
 
 ```python
 router = paw.compile("""
-Classify the user's intent. Return ONLY a single digit.
-1 = Create or compile something new
-2 = Browse or search existing items
-3 = Run something in the browser
-4 = Read documentation
-5 = Manage account or API keys
-0 = None of the above (likely a question)
+Classify the user's intent. Return ONLY a single label.
+playground = Create or compile something new
+hub = Browse or search existing items
+browser = Run something in the browser
+docs = Read documentation
+settings = Manage account or API keys
+none = None of the above (likely a question)
 
 Input: how do I get started
-Output: 4
+Output: docs
 
 Input: browse community programs
-Output: 2
+Output: hub
 
 Input: is it free?
-Output: 0
+Output: none
 """)
 ```
 
 ## Adapting this for your site
 
 1. **List your pages** with short descriptions of what users do there
-2. **Compile a classifier** that maps intents to page numbers
+2. **Compile a classifier** that maps intents to short labels
 3. **Test with 20-30 real queries** your users would type — iterate on the spec wording
 4. If users also ask questions, add Q&A programs and a validator
 5. Each program compiles once and is cached forever — the pipeline runs locally with no API calls
 
 ## Takeaways
 
-- **Classification beats generation** for routing. Output a label, map it in code.
+- **Classification beats generation** for routing. Output a short label, map it in code.
 - **Multiple small programs beat one complex program.** When accuracy drops, split.
 - **A validator catches failures** the other programs miss — cheap insurance.
 - **Iterate with real queries.** Build a small test set, measure, adjust wording, repeat.

{programasweights-0.3.1 → programasweights-0.4.0}/docs/guide/browser-inference.md

@@ -2,7 +2,7 @@
 
 Run PAW programs directly in the browser via WebAssembly. No server, no API key, no setup.
 
-Programs compiled with the compact interpreter (GPT-2 124M) run entirely client-side. The base model (105 MB) downloads once and is cached; each program adds only ~5 MB.
+Programs compiled with the compact interpreter (GPT-2 124M) run entirely client-side. The base model (134 MB) downloads once and is cached; each program adds ~12 MB total (~5 MB adapter + ~7 MB prefix cache).
 
 ## Quick Start
 
@@ -12,7 +12,7 @@ Programs compiled with the compact interpreter (GPT-2 124M) run entirely client-
 <script type="module">
   import paw from 'https://cdn.jsdelivr.net/npm/@programasweights/web';
 
-  const fn = await paw.function('programasweights/email-triage');
+  const fn = await paw.function('email-triage');
   const result = await fn('Urgent: server is down!');
   console.log(result); // "immediate"
 </script>
@@ -27,7 +27,7 @@ npm install @programasweights/web
 ```javascript
 import paw from '@programasweights/web';
 
-const fn = await paw.function('programasweights/email-triage', {
+const fn = await paw.function('email-triage', {
   onProgress: ({ loaded, total, stage }) => {
     console.log(`${stage}: ${Math.round(loaded/total*100)}%`);
   },
@@ -42,7 +42,7 @@ await fn.free();
 
 ## How It Works
 
-1. **Base model** — Compact interpreter (GPT-2 124M, 105 MB) downloads from HuggingFace CDN and is cached in the browser after first load.
+1. **Base model** — Compact interpreter (GPT-2 124M, 134 MB) downloads from HuggingFace CDN and is cached in the browser after first load.
 2. **LoRA adapter** — Each program is a ~5 MB Q4_0 GGUF LoRA adapter that specializes the base model for a specific task.
 3. **Prefix cache** — A precomputed KV cache (~7 MB) eliminates the prompt prefill step, making the first inference call fast.
 4. **Inference** — Runs via WebAssembly (llama.cpp compiled to WASM with SIMD). ~200ms per call on Chrome.
@@ -84,10 +84,10 @@ paw.configure({
 
 | Component | Size | When |
 |-----------|------|------|
-| Base model (GPT-2 Q6_K) | 105 MB | First program load (cached) |
+| Base model (GPT-2 Q8_0) | 134 MB | First program load (cached) |
 | LoRA adapter | ~5 MB | Per program |
 | Prefix cache | ~7 MB | Per program |
-| **First load total** | **~117 MB** | |
+| **First load total** | **~146 MB** | |
 | **Switching programs** | **~12 MB** | |
 
 ## Browser Compatibility
@@ -135,5 +135,5 @@ Without these headers, the SDK falls back to single-threaded WASM (still functio
 ## Limitations
 
 - Only programs compiled with the **compact** interpreter (GPT-2 124M) are supported. Programs compiled with the standard interpreter (Qwen3 0.6B) are too large for browser inference (~594 MB base model).
-- The 105 MB base model download may be slow on mobile connections.
+- The 134 MB base model download may be slow on mobile connections.
 - Performance varies by browser (Chrome is fastest).

{programasweights-0.3.1 → programasweights-0.4.0}/docs/index.md

@@ -56,7 +56,7 @@ Each compiled program has two parts:
 1. **Discrete pseudo-program** -- text instructions generated by the neural compiler
 2. **Continuous neural adapter** -- LoRA weights (~23 MB) that tune the interpreter model
 
-At runtime, the SDK loads a Q6_K base model (~594 MB, downloaded once) and applies the LoRA adapter. Inference runs entirely locally via llama.cpp.
+At runtime, the SDK loads a quantized base model (Q6_K ~594 MB for Qwen3, Q8_0 ~134 MB for GPT-2, downloaded once) and applies the LoRA adapter. Inference runs entirely locally via llama.cpp.
 
 ## Browser Inference
 
@@ -65,7 +65,7 @@ Programs compiled with the compact interpreter (GPT-2 124M) also run directly in
 ```html
 <script type="module">
   import paw from '@programasweights/web';
-  const fn = await paw.function('programasweights/email-triage');
+  const fn = await paw.function('email-triage');
   const result = await fn('Urgent: server is down!');
 </script>
 ```

{programasweights-0.3.1 → programasweights-0.4.0}/programasweights/__init__.py

@@ -27,18 +27,19 @@ try:
     from importlib.metadata import version as _meta_version
     __version__ = _meta_version("programasweights")
 except Exception:
-    __version__ = "0.3.1"
+    __version__ = "0.4.0"
 
 from .config import get_api_url, get_api_key, set_api_key
 
 
 def compile(
     spec: str,
-    compiler: str = "paw-4b-qwen3-0.6b",
+    compiler: str | None = None,
     name: str | None = None,
     tags: list[str] | None = None,
     public: bool = True,
     slug: str | None = None,
+    ephemeral: bool = False,
 ):
     """Compile a natural language specification into a neural program.
 
@@ -48,7 +49,8 @@ def compile(
 
     Args:
         spec: Full specification text. Include examples in the text if desired.
-        compiler: Compiler model (alias or snapshot name).
+        compiler: Compiler model (alias or snapshot name). If omitted, the
+            server chooses the current default compiler.
         name: Human-readable program name (display title for the hub).
         tags: Tags for hub discovery.
         public: Whether to list on the public hub.
@@ -71,7 +73,7 @@ def compile(
     from ._output import status
     status("Compiling...")
     client = PAWClient(api_url=get_api_url(), api_key=get_api_key())
-    result = client.compile(spec, compiler=compiler, name=name, tags=tags, public=public, slug=slug)
+    result = client.compile(spec, compiler=compiler, name=name, tags=tags, public=public, slug=slug, ephemeral=ephemeral)
     label = f"{result.id}"
     if result.slug:
         ver_str = f" v{result.version}" if result.version and result.version > 1 else ""
@@ -125,6 +127,7 @@ def function(
     from .cache import is_program_cached, get_program_dir, get_cached_slug, save_slug_mapping
     from .runtime_llamacpp import PawFunction
 
+    offline = offline or os.environ.get("PAW_OFFLINE", "").strip() in ("1", "true", "yes")
     if n_gpu_layers is None:
         n_gpu_layers = int(os.environ.get("PAW_GPU_LAYERS", "-1"))
 
@@ -133,7 +136,7 @@ def function(
     resolved_id = program_id
     if not re.fullmatch(r"[a-f0-9]{16,64}", program_id):
         is_pinned = bool(re.search(r"@v\d+$", program_id))
-        use_offline = offline or os.environ.get("PAW_OFFLINE", "").strip() in ("1", "true", "yes")
+        use_offline = offline
 
         if is_pinned:
             cached = get_cached_slug(program_id)
@@ -176,7 +179,13 @@ def function(
 
     program_dir = get_program_dir(resolved_id)
     return PawFunction(
-        program_dir, n_ctx=n_ctx, n_gpu_layers=n_gpu_layers, verbose=verbose,
+        program_dir,
+        n_ctx=n_ctx,
+        n_gpu_layers=n_gpu_layers,
+        verbose=verbose,
+        api_url=get_api_url(),
+        api_key=get_api_key(),
+        offline=offline,
     )
 
 
@@ -227,7 +236,7 @@ def login(key: str | None = None):
 
 def compile_and_load(
     spec: str,
-    compiler: str = "paw-4b-qwen3-0.6b",
+    compiler: str | None = None,
     n_ctx: int = 2048,
     n_gpu_layers: int | None = None,
     verbose: bool = False,
@@ -240,7 +249,8 @@ def compile_and_load(
 
     Args:
         spec: Natural language specification.
-        compiler: Compiler model name.
+        compiler: Compiler model name. If omitted, the server chooses the
+            current default compiler.
         n_ctx: Context window size for llama.cpp.
         n_gpu_layers: GPU layers (-1 = all, 0 = CPU only).
         verbose: Print llama.cpp debug output.
@@ -293,10 +303,18 @@ def list_programs(sort: str = "recent", per_page: int = 20, page: int = 1) -> di
     return client.list_programs(sort=sort, per_page=per_page, page=page)
 
 
+def list_compilers() -> list[dict]:
+    """List available compilers from the server."""
+    from .client import PAWClient
+    client = PAWClient(api_url=get_api_url(), api_key=get_api_key())
+    return client.list_compilers()
+
+
 __all__ = [
     "compile",
     "compile_and_load",
     "function",
+    "list_compilers",
     "list_programs",
     "list_versions",
     "login",

programasweights-0.4.0/programasweights/cache.py

@@ -0,0 +1,323 @@
+"""
+Local cache management for base models and compiled programs.
+
+Cache structure:
+    ~/.cache/programasweights/
+        base_models/
+            qwen3-0.6b-q6_k.gguf      # ~594 MB, downloaded once
+            gpt2-q8_0.gguf            # ~134 MB, downloaded once
+        programs/
+            <program_id>/
+                adapter.gguf            # ~23 MB, Q4_0 LoRA
+                prompt_template.txt
+                meta.json
+        slug_cache.json                 # slug -> program_id mapping
+"""
+
+from __future__ import annotations
+
+import json
+import os
+from pathlib import Path
+
+import httpx
+
+from . import config
+
+BASE_MODEL_URLS = {
+    "qwen3-0.6b-q6_k": "https://huggingface.co/programasweights/Qwen3-0.6B-GGUF-Q6_K/resolve/main/qwen3-0.6b-q6_k.gguf",
+    "gpt2-q8_0": "https://huggingface.co/programasweights/GPT2-GGUF-Q8_0/resolve/main/gpt2-q8_0.gguf",
+}
+
+INTERPRETER_TO_GGUF = {
+    "Qwen/Qwen3-0.6B": "qwen3-0.6b-q6_k",
+    "gpt2": "gpt2-q8_0",
+}
+
+
+LEGACY_RUNTIME_MANIFESTS = {
+    "qwen3-0.6b-q6_k": {
+        "runtime_id": "qwen3-0.6b-q6_k",
+        "manifest_version": 1,
+        "display_name": "Qwen3 0.6B (Q6_K)",
+        "interpreter": "Qwen/Qwen3-0.6B",
+        "adapter_format": "gguf_lora",
+        "local_sdk": {
+            "supported": True,
+            "base_model": {
+                "provider": "huggingface",
+                "repo": "programasweights/Qwen3-0.6B-GGUF-Q6_K",
+                "file": "qwen3-0.6b-q6_k.gguf",
+                "url": BASE_MODEL_URLS["qwen3-0.6b-q6_k"],
+                "sha256": None,
+            },
+            "n_ctx": 2048,
+        },
+        "js_sdk": {
+            "supported": False,
+            "base_model": None,
+            "prefix_cache_supported": False,
+        },
+    },
+    "gpt2-q8_0": {
+        "runtime_id": "gpt2-q8_0",
+        "manifest_version": 1,
+        "display_name": "GPT-2 124M (Q8_0)",
+        "interpreter": "gpt2",
+        "adapter_format": "gguf_lora",
+        "local_sdk": {
+            "supported": True,
+            "base_model": {
+                "provider": "huggingface",
+                "repo": "programasweights/GPT2-GGUF-Q8_0",
+                "file": "gpt2-q8_0.gguf",
+                "url": BASE_MODEL_URLS["gpt2-q8_0"],
+                "sha256": None,
+            },
+            "n_ctx": 2048,
+        },
+        "js_sdk": {
+            "supported": True,
+            "base_model": {
+                "provider": "huggingface",
+                "repo": "programasweights/GPT2-GGUF-Q8_0",
+                "file": "gpt2-q8_0.gguf",
+                "url": BASE_MODEL_URLS["gpt2-q8_0"],
+                "sha256": None,
+            },
+            "prefix_cache_supported": True,
+        },
+    },
+}
+
+
+def _runtime_cache_dir() -> Path:
+    d = config.get_cache_dir() / "runtimes"
+    d.mkdir(parents=True, exist_ok=True)
+    return d
+
+
+def _runtime_manifest_path(runtime_id: str) -> Path:
+    return _runtime_cache_dir() / f"{runtime_id}.json"
+
+
+def get_cached_runtime_manifest(runtime_id: str) -> dict | None:
+    path = _runtime_manifest_path(runtime_id)
+    if not path.exists():
+        return None
+    try:
+        return json.loads(path.read_text())
+    except (json.JSONDecodeError, OSError):
+        return None
+
+
+def save_runtime_manifest(runtime_manifest: dict) -> None:
+    runtime_id = runtime_manifest.get("runtime_id")
+    if not runtime_id:
+        return
+    path = _runtime_manifest_path(runtime_id)
+    path.write_text(json.dumps(runtime_manifest, indent=2))
+
+
+def _legacy_runtime_manifest(interpreter: str | None) -> dict | None:
+    if not interpreter:
+        return None
+    runtime_id = INTERPRETER_TO_GGUF.get(interpreter)
+    if not runtime_id:
+        return None
+    manifest = LEGACY_RUNTIME_MANIFESTS.get(runtime_id)
+    return json.loads(json.dumps(manifest)) if manifest else None
+
+
+def _is_runtime_manifest_complete(runtime_manifest: dict | None) -> bool:
+    if not runtime_manifest:
+        return False
+    base_model = runtime_manifest.get("local_sdk", {}).get("base_model")
+    return bool(runtime_manifest.get("runtime_id") and base_model and base_model.get("file"))
+
+
+def fetch_runtime_manifest(
+    runtime_id: str,
+    api_url: str | None = None,
+    api_key: str | None = None,
+) -> dict:
+    base_url = (api_url or config.get_api_url()).rstrip("/")
+    headers = {}
+    if api_key:
+        headers["X-API-Key"] = api_key
+    resp = httpx.get(
+        f"{base_url}/api/v1/models/runtimes/{runtime_id}",
+        headers=headers,
+        timeout=10.0,
+    )
+    resp.raise_for_status()
+    runtime_manifest = resp.json()
+    save_runtime_manifest(runtime_manifest)
+    return runtime_manifest
+
+
+def resolve_runtime_manifest(
+    program_meta: dict,
+    api_url: str | None = None,
+    api_key: str | None = None,
+    offline: bool = False,
+) -> dict | None:
+    embedded = program_meta.get("runtime")
+    if isinstance(embedded, dict) and _is_runtime_manifest_complete(embedded):
+        save_runtime_manifest(embedded)
+        return embedded
+
+    runtime_id = program_meta.get("runtime_id")
+    if runtime_id:
+        cached = get_cached_runtime_manifest(runtime_id)
+        if _is_runtime_manifest_complete(cached):
+            return cached
+        if not offline:
+            try:
+                return fetch_runtime_manifest(runtime_id, api_url=api_url, api_key=api_key)
+            except Exception:
+                pass
+
+    return _legacy_runtime_manifest(program_meta.get("interpreter"))
+
+
+def _base_model_info_from_runtime(runtime_manifest: dict) -> dict | None:
+    return runtime_manifest.get("local_sdk", {}).get("base_model")
+
+
+def _build_hf_url(repo: str, file_name: str) -> str:
+    return f"https://huggingface.co/{repo}/resolve/main/{file_name}"
+
+
+def get_base_model_path(
+    interpreter: str = "Qwen/Qwen3-0.6B",
+    runtime_manifest: dict | None = None,
+) -> Path:
+    """Get the path to the base model GGUF, downloading if needed."""
+    if runtime_manifest:
+        local_sdk = runtime_manifest.get("local_sdk", {})
+        if not local_sdk.get("supported", False):
+            raise ValueError(
+                f"Runtime '{runtime_manifest.get('runtime_id', interpreter)}' is not supported by the local SDK."
+            )
+
+        base_model = _base_model_info_from_runtime(runtime_manifest)
+        if base_model:
+            file_name = base_model.get("file")
+            if not file_name:
+                raise ValueError(
+                    f"Runtime '{runtime_manifest.get('runtime_id', interpreter)}' is missing a base model file."
+                )
+
+            gguf_path = config.get_base_models_dir() / file_name
+            if gguf_path.exists():
+                return gguf_path
+
+            url = base_model.get("url")
+            if not url and base_model.get("provider") == "huggingface":
+                url = _build_hf_url(base_model["repo"], file_name)
+            if not url:
+                raise ValueError(
+                    f"Runtime '{runtime_manifest.get('runtime_id', interpreter)}' is missing a downloadable base model URL."
+                )
+
+            from ._output import status
+            label = runtime_manifest.get("display_name") or runtime_manifest.get("runtime_id") or interpreter
+            status(f"Downloading interpreter {label} (one-time download)...")
+            _download_file(url, gguf_path)
+            status(f"Saved to {gguf_path}")
+            return gguf_path
+
+    gguf_name = INTERPRETER_TO_GGUF.get(interpreter)
+    if not gguf_name:
+        raise ValueError(
+            f"Unknown interpreter: '{interpreter}'. "
+            f"Supported: {list(INTERPRETER_TO_GGUF.keys())}"
+        )
+    gguf_path = config.get_base_models_dir() / f"{gguf_name}.gguf"
+
+    if gguf_path.exists():
+        return gguf_path
+
+    url = BASE_MODEL_URLS.get(gguf_name)
+    if not url:
+        raise ValueError(
+            f"Unknown interpreter model: {interpreter}. "
+            f"Available: {list(INTERPRETER_TO_GGUF.keys())}"
+        )
+
+    from ._output import status
+    status(f"Downloading interpreter {gguf_name} (one-time download)...")
+    _download_file(url, gguf_path)
+    status(f"Saved to {gguf_path}")
+    return gguf_path
+
+
+def get_program_dir(program_id: str) -> Path:
+    """Get the local cache directory for a program."""
+    return config.get_programs_dir() / program_id
+
+
+def is_program_cached(program_id: str) -> bool:
+    """Check if a program's artifacts are already cached locally."""
+    d = get_program_dir(program_id)
+    return (d / "adapter.gguf").exists() and (d / "prompt_template.txt").exists()
+
+
+def _download_file(url: str, dest: Path):
+    """Download a file atomically with progress indication."""
+    dest.parent.mkdir(parents=True, exist_ok=True)
+    tmp = dest.with_suffix(dest.suffix + ".tmp")
+    try:
+        with httpx.stream("GET", url, follow_redirects=True, timeout=300.0) as resp:
+            resp.raise_for_status()
+            total = int(resp.headers.get("content-length", 0))
+            downloaded = 0
+            with open(tmp, "wb") as f:
+                for chunk in resp.iter_bytes(chunk_size=8192):
+                    f.write(chunk)
+                    downloaded += len(chunk)
+                    if total > 0:
+                        pct = downloaded / total * 100
+                        mb = downloaded / 1024 / 1024
+                        from ._output import status_inline, status_end
+                        status_inline(f"  {mb:.1f} MB ({pct:.0f}%)")
+            from ._output import status_end
+            status_end()
+        os.replace(str(tmp), str(dest))
+    except BaseException:
+        tmp.unlink(missing_ok=True)
+        raise
+
+
+def _slug_cache_path() -> Path:
+    return config.get_cache_dir() / "slug_cache.json"
+
+
+def get_cached_slug(slug: str) -> str | None:
+    """Look up a slug in the local cache. Returns program_id or None."""
+    path = _slug_cache_path()
+    if not path.exists():
+        return None
+    try:
+        data = json.loads(path.read_text())
+        program_id = data.get(slug)
+        if program_id and is_program_cached(program_id):
+            return program_id
+    except (json.JSONDecodeError, OSError):
+        pass
+    return None
+
+
+def save_slug_mapping(slug: str, program_id: str) -> None:
+    """Save a slug -> program_id mapping to the local cache."""
+    path = _slug_cache_path()
+    data: dict = {}
+    if path.exists():
+        try:
+            data = json.loads(path.read_text())
+        except (json.JSONDecodeError, OSError):
+            pass
+    data[slug] = program_id
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(json.dumps(data))

{programasweights-0.3.1 → programasweights-0.4.0}/programasweights/cli.py

@@ -169,7 +169,7 @@ def main():
 
     p = sub.add_parser("compile", help="Compile a spec on the server")
     p.add_argument("--spec", required=True, help="Natural language specification")
-    p.add_argument("--compiler", default="paw-4b-qwen3-0.6b", help="Compiler model")
+    p.add_argument("--compiler", default=None, help="Compiler model (omit to use the server default)")
     p.add_argument("--slug", default=None, help="URL-safe handle (e.g. 'message-classifier')")
     p.add_argument("--private", action="store_true", help="Make program private (not listed on hub)")
     p.add_argument("--json", action="store_true", help="JSON output")

{programasweights-0.3.1 → programasweights-0.4.0}/programasweights/client.py

@@ -6,6 +6,7 @@ Handles compilation, program download, and authentication.
 
 from __future__ import annotations
 
+import json
 import os
 import time
 import zipfile
@@ -25,6 +26,10 @@ class Program:
     status: str
     slug: Optional[str] = None
     compiler_snapshot: Optional[str] = None
+    compiler_kind: Optional[str] = None
+    pseudo_program_strategy: Optional[str] = None
+    runtime_id: Optional[str] = None
+    runtime_manifest_version: Optional[int] = None
     timings: Optional[dict] = None
     error: Optional[str] = None
     version: Optional[int] = None
@@ -47,11 +52,12 @@ class PAWClient:
     def compile(
         self,
         spec: str,
-        compiler: str = "paw-4b-qwen3-0.6b",
+        compiler: str | None = None,
         name: str | None = None,
         tags: list[str] | None = None,
         public: bool = True,
         slug: str | None = None,
+        ephemeral: bool = False,
     ) -> Program:
         """Compile a spec into a neural program on the server.
 
@@ -70,7 +76,11 @@ class PAWClient:
         Raises:
             httpx.HTTPStatusError: On API errors (422 for validation, 429 for rate limit).
         """
-        body: dict = {"spec": spec, "compiler": compiler, "public": public}
+        body: dict = {"spec": spec, "public": public}
+        if compiler:
+            body["compiler"] = compiler
+        if ephemeral:
+            body["ephemeral"] = True
         if name:
             body["name"] = name
         if tags:
@@ -92,6 +102,10 @@ class PAWClient:
             status=data.get("status", "unknown"),
             slug=data.get("slug"),
             compiler_snapshot=data.get("compiler_snapshot"),
+            compiler_kind=data.get("compiler_kind"),
+            pseudo_program_strategy=data.get("pseudo_program_strategy"),
+            runtime_id=data.get("runtime_id"),
+            runtime_manifest_version=data.get("runtime_manifest_version"),
             timings=data.get("timings"),
             error=data.get("error"),
             version=data.get("version"),
@@ -121,7 +135,7 @@ class PAWClient:
         from ._output import status
 
         status(f"Downloading program {program_id[:12]}...")
-        max_wait = 30
+        max_wait = 60
         elapsed = 0
         resp = None
         waiting_logged = False
@@ -174,8 +188,36 @@ class PAWClient:
                     raise ValueError(f"Unsafe path in .paw archive: {member}")
             zf.extractall(program_dir)
 
+        self._hydrate_runtime_manifest(program_dir)
+
         return program_dir
 
+    def _hydrate_runtime_manifest(self, program_dir: Path) -> None:
+        meta_path = program_dir / "meta.json"
+        if not meta_path.exists():
+            return
+
+        try:
+            meta = json.loads(meta_path.read_text())
+        except (json.JSONDecodeError, OSError):
+            return
+
+        runtime_id = meta.get("runtime_id")
+        runtime = meta.get("runtime")
+        if isinstance(runtime, dict) and runtime.get("runtime_id") and runtime.get("local_sdk", {}).get("base_model"):
+            return
+        if not runtime_id:
+            return
+
+        try:
+            runtime_manifest = self.get_runtime_manifest(runtime_id)
+        except Exception:
+            return
+
+        meta["runtime"] = runtime_manifest
+        meta.setdefault("runtime_manifest_version", runtime_manifest.get("manifest_version"))
+        meta_path.write_text(json.dumps(meta, indent=2))
+
     def get_program_meta(self, program_id: str) -> dict:
         """Get program metadata from the server."""
         resp = httpx.get(
@@ -186,6 +228,30 @@ class PAWClient:
         resp.raise_for_status()
         return resp.json()
 
+    def get_runtime_manifest(self, runtime_id: str) -> dict:
+        """Fetch a runtime manifest from the server and cache it locally."""
+        from . import cache
+
+        cached = cache.get_cached_runtime_manifest(runtime_id)
+        if cached:
+            return cached
+
+        return cache.fetch_runtime_manifest(
+            runtime_id,
+            api_url=self._api_url,
+            api_key=self._api_key,
+        )
+
+    def list_compilers(self) -> list[dict]:
+        """List available compilers from the server."""
+        resp = httpx.get(
+            f"{self._api_url}/api/v1/models/compilers",
+            headers=self._headers(),
+            timeout=10.0,
+        )
+        resp.raise_for_status()
+        return resp.json()["compilers"]
+
     def list_slug_versions(self, slug: str) -> dict:
         """List all versions of a slug. Slug format: 'username/slug-name' or bare 'slug-name'."""
         resp = httpx.get(

{programasweights-0.3.1 → programasweights-0.4.0}/programasweights/runtime_llamacpp.py

@@ -1,7 +1,8 @@
 """
 llama.cpp runtime for local inference with LoRA adapters.
 
-Loads a base GGUF model (Q6_K) and applies a Q4_0 LoRA adapter per-program.
+Loads a base GGUF model (Q6_K for Qwen3, Q8_0 for GPT-2) and applies a
+Q4_0 LoRA adapter per-program.
 Uses the pre-rendered prompt template from the .paw bundle.
 
 Prefix KV cache is saved to disk after the first call and reloaded on
@@ -33,6 +34,9 @@ class PawFunction:
         n_ctx: int = 2048,
         n_gpu_layers: int = 0,
         verbose: bool = False,
+        api_url: str | None = None,
+        api_key: str | None = None,
+        offline: bool = False,
     ):
         program_dir = Path(program_dir)
         self._program_dir = program_dir
@@ -53,7 +57,17 @@ class PawFunction:
 
         from ._output import status
         interpreter = self._meta.get("interpreter", "Qwen/Qwen3-0.6B")
-        base_model_path = cache.get_base_model_path(interpreter)
+        runtime_manifest = cache.resolve_runtime_manifest(
+            self._meta,
+            api_url=api_url,
+            api_key=api_key,
+            offline=offline,
+        )
+        if runtime_manifest:
+            self._meta["runtime"] = runtime_manifest
+            self._meta.setdefault("runtime_id", runtime_manifest.get("runtime_id"))
+            self._meta.setdefault("runtime_manifest_version", runtime_manifest.get("manifest_version"))
+        base_model_path = cache.get_base_model_path(interpreter, runtime_manifest=runtime_manifest)
 
         status("Loading interpreter...")
 
@@ -88,7 +102,6 @@ class PawFunction:
                 _os.close(_old_stderr)
 
         placeholder = "{INPUT_PLACEHOLDER}"
-        self._use_special = interpreter not in ("gpt2",)
 
         if placeholder in self._template:
             prefix_text = self._template.split(placeholder)[0]
@@ -99,8 +112,8 @@ class PawFunction:
 
         self._prefix_tokens = self._llm.tokenize(
             prefix_text.encode("utf-8"),
-            add_bos=not self._use_special,
-            special=self._use_special,
+            add_bos=False,
+            special=True,
         )
         self._suffix_text = suffix_text
         self._n_prefix = len(self._prefix_tokens)
@@ -183,7 +196,7 @@ class PawFunction:
         input_tokens = self._llm.tokenize(
             input_with_suffix.encode("utf-8"),
             add_bos=False,
-            special=self._use_special,
+            special=True,
         )
 
         tokens_used = self._n_prefix + len(input_tokens)

{programasweights-0.3.1 → programasweights-0.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "programasweights"
-version = "0.3.1"
+version = "0.4.0"
 description = "Compile natural language specifications into neural programs that run locally via llama.cpp."
 readme = "PYPI_README.md"
 requires-python = ">=3.9"

programasweights-0.4.0/tests/test_runtime_registry_sdk.py

@@ -0,0 +1,81 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import pytest
+
+from programasweights import cache
+
+
+RUNTIME = {
+    "runtime_id": "gpt2-q8_0",
+    "manifest_version": 1,
+    "display_name": "GPT-2 124M (Q8_0)",
+    "interpreter": "gpt2",
+    "adapter_format": "gguf_lora",
+    "local_sdk": {
+        "supported": True,
+        "base_model": {
+            "provider": "huggingface",
+            "repo": "programasweights/GPT2-GGUF-Q8_0",
+            "file": "gpt2-q8_0.gguf",
+            "url": "https://example.com/gpt2-q8_0.gguf",
+            "sha256": None,
+        },
+        "n_ctx": 2048,
+    },
+    "js_sdk": {
+        "supported": True,
+        "base_model": {
+            "provider": "huggingface",
+            "repo": "programasweights/GPT2-GGUF-Q8_0",
+            "file": "gpt2-q8_0.gguf",
+            "url": "https://example.com/gpt2-q8_0.gguf",
+            "sha256": None,
+        },
+        "prefix_cache_supported": True,
+    },
+}
+
+
+@pytest.fixture(autouse=True)
+def _cache_dir(tmp_path, monkeypatch):
+    monkeypatch.setenv("PAW_CACHE_DIR", str(tmp_path / "cache"))
+
+
+def test_resolve_runtime_manifest_prefers_embedded_manifest():
+    meta = {
+        "interpreter": "gpt2",
+        "runtime_id": "gpt2-q8_0",
+        "runtime": RUNTIME,
+    }
+    resolved = cache.resolve_runtime_manifest(meta, offline=True)
+    assert resolved["runtime_id"] == "gpt2-q8_0"
+    cached = cache.get_cached_runtime_manifest("gpt2-q8_0")
+    assert cached["runtime_id"] == "gpt2-q8_0"
+
+
+def test_resolve_runtime_manifest_uses_legacy_fallback_when_offline():
+    meta = {
+        "interpreter": "gpt2",
+        "runtime_id": "gpt2-q8_0",
+    }
+    resolved = cache.resolve_runtime_manifest(meta, offline=True)
+    assert resolved is not None
+    assert resolved["runtime_id"] == "gpt2-q8_0"
+    assert resolved["local_sdk"]["base_model"]["file"] == "gpt2-q8_0.gguf"
+
+
+def test_get_base_model_path_uses_runtime_manifest(tmp_path, monkeypatch):
+    target = tmp_path / "cache" / "base_models" / "gpt2-q8_0.gguf"
+
+    def fake_download(url: str, dest: Path):
+        assert url == "https://example.com/gpt2-q8_0.gguf"
+        dest.parent.mkdir(parents=True, exist_ok=True)
+        dest.write_bytes(b"gguf")
+
+    monkeypatch.setattr(cache, "_download_file", fake_download)
+    path = cache.get_base_model_path("gpt2", runtime_manifest=RUNTIME)
+    assert path == target
+    assert path.read_bytes() == b"gguf"

programasweights-0.3.1/programasweights/cache.py

@@ -1,131 +0,0 @@
-"""
-Local cache management for base models and compiled programs.
-
-Cache structure:
-    ~/.cache/programasweights/
-        base_models/
-            qwen3-0.6b-q6_k.gguf      # ~594 MB, downloaded once
-        programs/
-            <program_id>/
-                adapter.gguf            # ~23 MB, Q4_0 LoRA
-                prompt_template.txt
-                meta.json
-        slug_cache.json                 # slug -> program_id mapping
-"""
-
-from __future__ import annotations
-
-import json
-import os
-from pathlib import Path
-
-import httpx
-
-from . import config
-
-BASE_MODEL_URLS = {
-    "qwen3-0.6b-q6_k": "https://huggingface.co/programasweights/Qwen3-0.6B-GGUF-Q6_K/resolve/main/qwen3-0.6b-q6_k.gguf",
-    "gpt2-q6_k": "https://huggingface.co/programasweights/GPT2-GGUF-Q6_K/resolve/main/gpt2-q6_k.gguf",
-}
-
-INTERPRETER_TO_GGUF = {
-    "Qwen/Qwen3-0.6B": "qwen3-0.6b-q6_k",
-    "gpt2": "gpt2-q6_k",
-}
-
-
-def get_base_model_path(interpreter: str = "Qwen/Qwen3-0.6B") -> Path:
-    """Get the path to the base model GGUF, downloading if needed."""
-    gguf_name = INTERPRETER_TO_GGUF.get(interpreter)
-    if not gguf_name:
-        raise ValueError(
-            f"Unknown interpreter: '{interpreter}'. "
-            f"Supported: {list(INTERPRETER_TO_GGUF.keys())}"
-        )
-    gguf_path = config.get_base_models_dir() / f"{gguf_name}.gguf"
-
-    if gguf_path.exists():
-        return gguf_path
-
-    url = BASE_MODEL_URLS.get(gguf_name)
-    if not url:
-        raise ValueError(
-            f"Unknown interpreter model: {interpreter}. "
-            f"Available: {list(INTERPRETER_TO_GGUF.keys())}"
-        )
-
-    from ._output import status
-    status(f"Downloading interpreter {gguf_name} (one-time download)...")
-    _download_file(url, gguf_path)
-    status(f"Saved to {gguf_path}")
-    return gguf_path
-
-
-def get_program_dir(program_id: str) -> Path:
-    """Get the local cache directory for a program."""
-    return config.get_programs_dir() / program_id
-
-
-def is_program_cached(program_id: str) -> bool:
-    """Check if a program's artifacts are already cached locally."""
-    d = get_program_dir(program_id)
-    return (d / "adapter.gguf").exists() and (d / "prompt_template.txt").exists()
-
-
-def _download_file(url: str, dest: Path):
-    """Download a file atomically with progress indication."""
-    dest.parent.mkdir(parents=True, exist_ok=True)
-    tmp = dest.with_suffix(dest.suffix + ".tmp")
-    try:
-        with httpx.stream("GET", url, follow_redirects=True, timeout=300.0) as resp:
-            resp.raise_for_status()
-            total = int(resp.headers.get("content-length", 0))
-            downloaded = 0
-            with open(tmp, "wb") as f:
-                for chunk in resp.iter_bytes(chunk_size=8192):
-                    f.write(chunk)
-                    downloaded += len(chunk)
-                    if total > 0:
-                        pct = downloaded / total * 100
-                        mb = downloaded / 1024 / 1024
-                        from ._output import status_inline, status_end
-                        status_inline(f"  {mb:.1f} MB ({pct:.0f}%)")
-            from ._output import status_end
-            status_end()
-        os.replace(str(tmp), str(dest))
-    except BaseException:
-        tmp.unlink(missing_ok=True)
-        raise
-
-
-def _slug_cache_path() -> Path:
-    return config.get_cache_dir() / "slug_cache.json"
-
-
-def get_cached_slug(slug: str) -> str | None:
-    """Look up a slug in the local cache. Returns program_id or None."""
-    path = _slug_cache_path()
-    if not path.exists():
-        return None
-    try:
-        data = json.loads(path.read_text())
-        program_id = data.get(slug)
-        if program_id and is_program_cached(program_id):
-            return program_id
-    except (json.JSONDecodeError, OSError):
-        pass
-    return None
-
-
-def save_slug_mapping(slug: str, program_id: str) -> None:
-    """Save a slug -> program_id mapping to the local cache."""
-    path = _slug_cache_path()
-    data: dict = {}
-    if path.exists():
-        try:
-            data = json.loads(path.read_text())
-        except (json.JSONDecodeError, OSError):
-            pass
-    data[slug] = program_id
-    path.parent.mkdir(parents=True, exist_ok=True)
-    path.write_text(json.dumps(data))