programasweights 0.3.1__tar.gz → 0.3.2__tar.gz

{programasweights-0.3.1 → programasweights-0.3.2}/AGENTS.md

@@ -38,7 +38,7 @@ fn("I love this!")  # "positive"
 ## Two Compilers
 
 - **Standard** (`paw-4b-qwen3-0.6b`) — higher accuracy, 594 MB base + ~22 MB/program. Default.
-- **Compact** (`paw-4b-gpt2`) — smaller (105 MB base + ~5 MB/program), runs in browser via WebAssembly.
+- **Compact** (`paw-4b-gpt2`) — smaller (134 MB base + ~5 MB/program), runs in browser via WebAssembly.
 
 ## When to Use PAW
 

{programasweights-0.3.1 → programasweights-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: programasweights
-Version: 0.3.1
+Version: 0.3.2
 Summary: Compile natural language specifications into neural programs that run locally via llama.cpp.
 Project-URL: Homepage, https://programasweights.com
 Project-URL: Repository, https://github.com/programasweights/programasweights-python
@@ -71,7 +71,7 @@ fn("I love this!")  # "positive"
 | --------------- | --------------------- | -------------------- |
 | Compiler name   | `paw-4b-qwen3-0.6b`   | `paw-4b-gpt2`        |
 | Accuracy        | Higher                | Lower                |
-| Base model size | 594 MB                | 105 MB               |
+| Base model size | 594 MB                | 134 MB               |
 | Program size    | ~22 MB                | ~5 MB                |
 | Local inference | ~0.05-0.5s per call   | ~0.03-0.3s per call  |
 | Runs in browser | No                    | Yes (WebAssembly)    |

{programasweights-0.3.1 → programasweights-0.3.2}/PYPI_README.md

@@ -41,7 +41,7 @@ fn("I love this!")  # "positive"
 | --------------- | --------------------- | -------------------- |
 | Compiler name   | `paw-4b-qwen3-0.6b`   | `paw-4b-gpt2`        |
 | Accuracy        | Higher                | Lower                |
-| Base model size | 594 MB                | 105 MB               |
+| Base model size | 594 MB                | 134 MB               |
 | Program size    | ~22 MB                | ~5 MB                |
 | Local inference | ~0.05-0.5s per call   | ~0.03-0.3s per call  |
 | Runs in browser | No                    | Yes (WebAssembly)    |

{programasweights-0.3.1 → programasweights-0.3.2}/docs/adr/001-llama-cpp-over-pytorch.md

@@ -17,5 +17,5 @@ Replace the PyTorch runtime with llama-cpp-python (~80MB install). Use GGUF mode
 - Inference uses Metal (Mac), CPU (Linux/Windows) — no GPU required
 - Must pre-render chat templates server-side (no transformers tokenizer on client)
 - .paw format must change from v2 (safetensors) to v3 (GGUF adapter)
-- Base model is downloaded once (~594 MB for Q6_K) and shared across all functions
+- Base model is downloaded once (~594 MB Q6_K for Qwen3, ~134 MB Q8_0 for GPT-2) and shared across all functions
 - Per-function adapter download is ~23MB (Q4_0, confirmed lossless at 4096-scale eval)

{programasweights-0.3.1 → programasweights-0.3.2}/docs/advanced/adrs.md

@@ -14,16 +14,17 @@ Concise records of major technical choices. Full ADR files may live elsewhere in
 
 ---
 
-## ADR 002: Q4_0 adapter and Q6_K base quantization
+## ADR 002: Quantization levels for base models and adapters
 
-**Decision:** Use **Q4_0** for adapters and **Q6_K** for the base model in shipped bundles.
+**Decision:** Use **Q4_0** for adapters. For base models, use **Q6_K** for Qwen3 0.6B and **Q8_0** for GPT-2.
 
 **Context:** Empirical evaluation on **4096** held-out examples across quantization settings informed the trade-off.
 
 **Consequence:**
 
-- **Q6_K base** — quality is preserved while the footprint is roughly **60% smaller** than fp16.
-- **Q4_0 adapter** — quality loss is negligible; adapter size drops to about **23 MB** versus **78 MB** for a heavier format at comparable settings in prior experiments.
+- **Q6_K base (Qwen3 0.6B)** — quality is preserved while the footprint is roughly **60% smaller** than fp16. 4096-sample eval shows no accuracy loss vs fp16.
+- **Q8_0 base (GPT-2)** — Q6_K caused ~3.5% accuracy loss on GPT-2 (4096-sample eval). Q8_0 closes the gap at only ~29 MB additional cost (134 MB vs 105 MB).
+- **Q4_0 adapter** — quality loss is negligible for both models; adapter size drops to about **23 MB** (Qwen3) / **5 MB** (GPT-2) versus **78 MB** / **19 MB** at fp16.
 
 ---
 

{programasweights-0.3.1 → programasweights-0.3.2}/docs/guide/browser-inference.md

@@ -84,7 +84,7 @@ paw.configure({
 
 | Component | Size | When |
 |-----------|------|------|
-| Base model (GPT-2 Q6_K) | 105 MB | First program load (cached) |
+| Base model (GPT-2 Q8_0) | 134 MB | First program load (cached) |
 | LoRA adapter | ~5 MB | Per program |
 | Prefix cache | ~7 MB | Per program |
 | **First load total** | **~117 MB** | |

{programasweights-0.3.1 → programasweights-0.3.2}/docs/index.md

@@ -56,7 +56,7 @@ Each compiled program has two parts:
 1. **Discrete pseudo-program** -- text instructions generated by the neural compiler
 2. **Continuous neural adapter** -- LoRA weights (~23 MB) that tune the interpreter model
 
-At runtime, the SDK loads a Q6_K base model (~594 MB, downloaded once) and applies the LoRA adapter. Inference runs entirely locally via llama.cpp.
+At runtime, the SDK loads a quantized base model (Q6_K ~594 MB for Qwen3, Q8_0 ~134 MB for GPT-2, downloaded once) and applies the LoRA adapter. Inference runs entirely locally via llama.cpp.
 
 ## Browser Inference
 

{programasweights-0.3.1 → programasweights-0.3.2}/programasweights/__init__.py

@@ -39,6 +39,7 @@ def compile(
     tags: list[str] | None = None,
     public: bool = True,
     slug: str | None = None,
+    ephemeral: bool = False,
 ):
     """Compile a natural language specification into a neural program.
 
@@ -71,7 +72,7 @@ def compile(
     from ._output import status
     status("Compiling...")
     client = PAWClient(api_url=get_api_url(), api_key=get_api_key())
-    result = client.compile(spec, compiler=compiler, name=name, tags=tags, public=public, slug=slug)
+    result = client.compile(spec, compiler=compiler, name=name, tags=tags, public=public, slug=slug, ephemeral=ephemeral)
     label = f"{result.id}"
     if result.slug:
         ver_str = f" v{result.version}" if result.version and result.version > 1 else ""

{programasweights-0.3.1 → programasweights-0.3.2}/programasweights/cache.py

@@ -5,6 +5,7 @@ Cache structure:
     ~/.cache/programasweights/
         base_models/
             qwen3-0.6b-q6_k.gguf      # ~594 MB, downloaded once
+            gpt2-q8_0.gguf            # ~134 MB, downloaded once
         programs/
             <program_id>/
                 adapter.gguf            # ~23 MB, Q4_0 LoRA
@@ -25,12 +26,12 @@ from . import config
 
 BASE_MODEL_URLS = {
     "qwen3-0.6b-q6_k": "https://huggingface.co/programasweights/Qwen3-0.6B-GGUF-Q6_K/resolve/main/qwen3-0.6b-q6_k.gguf",
-    "gpt2-q6_k": "https://huggingface.co/programasweights/GPT2-GGUF-Q6_K/resolve/main/gpt2-q6_k.gguf",
+    "gpt2-q8_0": "https://huggingface.co/programasweights/GPT2-GGUF-Q8_0/resolve/main/gpt2-q8_0.gguf",
 }
 
 INTERPRETER_TO_GGUF = {
     "Qwen/Qwen3-0.6B": "qwen3-0.6b-q6_k",
-    "gpt2": "gpt2-q6_k",
+    "gpt2": "gpt2-q8_0",
 }
 
 

{programasweights-0.3.1 → programasweights-0.3.2}/programasweights/client.py

@@ -52,6 +52,7 @@ class PAWClient:
         tags: list[str] | None = None,
         public: bool = True,
         slug: str | None = None,
+        ephemeral: bool = False,
     ) -> Program:
         """Compile a spec into a neural program on the server.
 
@@ -71,6 +72,8 @@ class PAWClient:
             httpx.HTTPStatusError: On API errors (422 for validation, 429 for rate limit).
         """
         body: dict = {"spec": spec, "compiler": compiler, "public": public}
+        if ephemeral:
+            body["ephemeral"] = True
         if name:
             body["name"] = name
         if tags:
@@ -121,7 +124,7 @@ class PAWClient:
         from ._output import status
 
         status(f"Downloading program {program_id[:12]}...")
-        max_wait = 30
+        max_wait = 60
         elapsed = 0
         resp = None
         waiting_logged = False

{programasweights-0.3.1 → programasweights-0.3.2}/programasweights/runtime_llamacpp.py

@@ -88,7 +88,6 @@ class PawFunction:
                 _os.close(_old_stderr)
 
         placeholder = "{INPUT_PLACEHOLDER}"
-        self._use_special = interpreter not in ("gpt2",)
 
         if placeholder in self._template:
             prefix_text = self._template.split(placeholder)[0]
@@ -99,8 +98,8 @@ class PawFunction:
 
         self._prefix_tokens = self._llm.tokenize(
             prefix_text.encode("utf-8"),
-            add_bos=not self._use_special,
-            special=self._use_special,
+            add_bos=False,
+            special=True,
         )
         self._suffix_text = suffix_text
         self._n_prefix = len(self._prefix_tokens)
@@ -183,7 +182,7 @@ class PawFunction:
         input_tokens = self._llm.tokenize(
             input_with_suffix.encode("utf-8"),
             add_bos=False,
-            special=self._use_special,
+            special=True,
         )
 
         tokens_used = self._n_prefix + len(input_tokens)

{programasweights-0.3.1 → programasweights-0.3.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "programasweights"
-version = "0.3.1"
+version = "0.3.2"
 description = "Compile natural language specifications into neural programs that run locally via llama.cpp."
 readme = "PYPI_README.md"
 requires-python = ">=3.9"