programasweights 0.4.5__tar.gz → 0.4.7__tar.gz

programasweights-0.4.7/.cursor/rules/releases.mdc

@@ -0,0 +1,17 @@
+---
+description: Publish and verify complete Python SDK releases
+alwaysApply: true
+---
+
+# SDK releases
+
+For release or publication work, read `RELEASING.md` first.
+
+- A release is complete only when its annotated remote tag, PyPI wheel and
+  source distribution, and published GitHub Release all exist and agree.
+- Build from the clean, tested release commit; push its annotated version tag
+  before uploading the exact checked artifacts to PyPI.
+- Verify the `GitHub release` workflow succeeded and the latest GitHub Release
+  agrees with PyPI. A tag alone is not a GitHub Release entry.
+- Repair missing entries using the documented workflow dispatch. Never move a
+  published tag, rebuild an existing version, or bypass artifact verification.

programasweights-0.4.7/.github/workflows/release.yml

@@ -0,0 +1,72 @@
+name: GitHub release
+
+on:
+  push:
+    tags: ['v*']
+  workflow_dispatch:
+    inputs:
+      tag:
+        description: 'Existing stable version tag, e.g. v0.4.6'
+        required: true
+        type: string
+
+permissions:
+  contents: write
+
+concurrency:
+  group: github-release-${{ inputs.tag || github.ref_name }}
+  cancel-in-progress: false
+
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    env:
+      RELEASE_TAG: ${{ inputs.tag || github.ref_name }}
+      GH_TOKEN: ${{ github.token }}
+      GH_REPO: ${{ github.repository }}
+    steps:
+      - name: Set the verification output directory
+        run: echo "RELEASE_DIR=$RUNNER_TEMP/verified-release" >> "$GITHUB_ENV"
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      - name: Verify the remote tag and both published PyPI artifacts
+        # Tags are pushed before uploading. Wait for the wheel AND sdist;
+        # never announce an unpublished version or rebuild release assets.
+        run: python scripts/release_metadata.py --tag "$RELEASE_TAG" --output-dir "$RELEASE_DIR" --wait-seconds 600
+      - name: Publish the missing GitHub release
+        shell: bash
+        run: |
+          set -euo pipefail
+          if gh release view "$RELEASE_TAG" --json tagName >/dev/null 2>&1; then
+            echo "Release already exists; preserving its notes and assets."
+          else
+            latest=$(jq -r .latest "$RELEASE_DIR/verified.json")
+            gh release create "$RELEASE_TAG" "$RELEASE_DIR"/artifacts/* \
+              --verify-tag --title "$RELEASE_TAG" \
+              --notes-file "$RELEASE_DIR/notes.md" --latest="$latest"
+          fi
+          gh release view "$RELEASE_TAG" --json tagName,isDraft,isPrerelease,url \
+            | jq -e --arg tag "$RELEASE_TAG" '.tagName == $tag and .isDraft == false and .isPrerelease == false'
+      - name: Verify the uploaded release assets and latest marker
+        shell: bash
+        run: |
+          set -euo pipefail
+          gh release download "$RELEASE_TAG" --pattern 'programasweights-*' \
+            --dir "$RELEASE_DIR/github-assets"
+          for artifact in "$RELEASE_DIR"/artifacts/*; do
+            cmp "$artifact" "$RELEASE_DIR/github-assets/$(basename "$artifact")"
+          done
+          if [ "$(jq -r .latest "$RELEASE_DIR/verified.json")" = true ]; then
+            gh api "repos/$GH_REPO/releases/latest" \
+              | jq -e --arg tag "$RELEASE_TAG" '.tag_name == $tag'
+          fi
+      - name: Retain the verification record
+        uses: actions/upload-artifact@v4
+        with:
+          name: release-verification-${{ inputs.tag || github.ref_name }}
+          path: ${{ runner.temp }}/verified-release/verified.json

{programasweights-0.4.5 → programasweights-0.4.7}/.github/workflows/test.yml

@@ -6,6 +6,17 @@ on:
   pull_request:
 
 jobs:
+  release-metadata:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      - run: python -m pip install pytest
+      - name: Test release verification without network or publishing
+        run: python -m pytest tests/test_release_metadata.py
+
   test:
     runs-on: ubuntu-latest
     strategy:
@@ -41,6 +52,7 @@ jobs:
             tests/test_local_program.py \
             tests/test_base_interpreter.py \
             tests/test_cli_auth.py \
+            tests/test_remote_inference.py \
             tests/test_desktop_sdk.py \
             tests/test_runtime_registry_sdk.py \
             tests/test_sdk.py::TestInstallAndImport
@@ -61,7 +73,7 @@ jobs:
           python -m pip install httpx pytest
           python -m pip install -e . --no-deps
       - name: Test Windows local paths, cache locks, and compile errors
-        run: python -m pytest tests/test_api_errors.py tests/test_compile_timeouts.py tests/test_local_program.py --junitxml=test-results.xml
+        run: python -m pytest tests/test_api_errors.py tests/test_compile_timeouts.py tests/test_local_program.py tests/test_remote_inference.py --junitxml=test-results.xml
       - name: Annotate Windows test failures
         if: failure()
         shell: python

{programasweights-0.4.5 → programasweights-0.4.7}/AGENTS.md

@@ -43,8 +43,46 @@ fn = paw.compile_and_load("Classify sentiment as positive or negative")
 fn("I love this!")  # "positive"
 ```
 
+Load a local `.paw` file with `paw.function("./classifier.paw")` (SDK 0.4.5+).
+
 If you want the smaller browser-compatible runtime explicitly, pass `compiler="paw-4b-gpt2"`. Otherwise, omit `compiler` and let the server default decide.
 
+## Remote inference (optional)
+
+Use the hosted API for fast inference in around 150 ms, without a local model download.
+
+### Python SDK
+
+```python
+import programasweights as paw
+
+with paw.function("email-triage", remote=True) as remote_fn:
+    print(remote_fn("Urgent: the server is down!"))
+```
+
+For authenticated requests, set `PAW_API_KEY` or use `paw.login()`.
+
+### Direct HTTP
+
+Use `httpx` directly without installing the PAW SDK.
+
+```python
+import httpx
+
+with httpx.Client(timeout=60.0) as client:
+    response = client.post(
+        "https://programasweights.com/api/v1/infer",
+        json={
+            "program_id": "email-triage",
+            "input": "Urgent: server is down!"
+        },
+    )
+    response.raise_for_status()
+    print(response.json()["output"])
+```
+
+For authenticated requests, pass `headers={"X-API-Key": api_key}` to `client.post()`.
+
 ## Current Public Compilers
 
 - **Standard** (`paw-4b-qwen3-0.6b`) — higher accuracy, 594 MB base + ~22 MB/program. This is the current server default.
@@ -89,6 +127,8 @@ Output: delete
 - Spec + input + output share a ~2048 token context window. Inputs that exceed it will error.
 - `max_tokens` defaults to `None`: generation runs until EOS or the context limit.
 - Compile runs on the hosted PAW API. Inference should usually run locally through the SDK.
+- Synchronous compile requests use a 40-minute read timeout.
+- **Run local inference sequentially by default.** With PAW’s current llama.cpp backend, simultaneous inference calls often perform worse. Reuse loaded functions and process inputs one at a time; never call the same function instance concurrently.
 - **GPU acceleration** is enabled by default (`n_gpu_layers=-1`). Uses Metal on Mac, CUDA on Linux, and falls back to CPU automatically. If GPU causes issues, set `PAW_GPU_LAYERS=0` or pass `n_gpu_layers=0`.
 - **First call** is usually ~1-5s because it loads the base model. Subsequent calls are typically ~0.05-0.5s depending on input length and GPU availability.
 - **Base model files are shared** across programs on disk. Each Standard LoRA adapter is ~22 MB; each Compact LoRA adapter is ~5 MB.
@@ -102,6 +142,8 @@ Output: delete
 
 ## Common Errors
 
+Compile API HTTP errors raise `paw.APIError`. Check `error.code` and `error.message` for details. The SDK does not retry automatically.
+
 | Error | Cause | Fix |
 |-------|-------|-----|
 | `RuntimeError: assets not ready` on download | Program is still generating after compile | The SDK polls automatically for up to 60s. If it still fails, retry shortly or recompile. |

{programasweights-0.4.5 → programasweights-0.4.7}/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## 0.4.7 (2026-09-20)
+
+- Add `remote=True` to `paw.function` and `paw.compile_and_load`, plus
+  `paw run --remote`, for hosted inference without downloading model assets
+  or loading the local runtime. Local inference remains the default.
+- Use server generation defaults unless explicitly overridden. Reuse HTTP
+  connections, support `with` and `.close()`, and preserve structured
+  `paw.APIError` details.
+- Reject remote inference combined with offline mode or incompatible local
+  runtime options.
+
+## 0.4.6 (2026-09-13)
+
+- Add an optional `logits_processor` argument to a compiled or base program
+  call for caller-supplied token constraints in llama.cpp's sampler.
+  Defaults to `None`; sampling is unchanged when it is unset.
+- Propagate processor failures to the caller instead of letting native
+  callback errors silently continue with unconstrained output.
+
 ## 0.4.5 (2026-09-10)
 
 - Expose structured compile API failures as `paw.APIError`, compatible with

{programasweights-0.4.5 → programasweights-0.4.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: programasweights
-Version: 0.4.5
+Version: 0.4.7
 Summary: Compile natural language specifications into neural programs that run locally via llama.cpp.
 Project-URL: Homepage, https://programasweights.com
 Project-URL: Repository, https://github.com/programasweights/programasweights-python
@@ -66,6 +66,19 @@ fn("I love this!")  # "positive"
 
 If you specifically want the smaller browser-compatible runtime, pass `compiler="paw-4b-gpt2"`. Otherwise, omit `compiler` and let the server default decide.
 
+## Remote inference (optional)
+
+Use the hosted API for fast inference in around 150 ms, without a local model download.
+
+```python
+import programasweights as paw
+
+with paw.function("email-triage", remote=True) as remote_fn:
+    print(remote_fn("Urgent: the server is down!"))
+```
+
+For direct HTTP calls, see the [REST API reference](https://programasweights.readthedocs.io/en/latest/api-reference/rest-api/#post-infer).
+
 ## Current Public Compilers
 
 
@@ -84,6 +97,18 @@ If you need to inspect available compiler aliases programmatically, use `paw.lis
 
 GPU acceleration is enabled by default (Metal on Mac, CUDA on Linux, falls back to CPU). Set `PAW_GPU_LAYERS=0` to force CPU if GPU causes issues.
 
+## Constrained Decoding
+
+In SDK 0.4.6+, a call accepts an optional `logits_processor`: an advanced hook for caller-supplied llama.cpp-compatible token constraints, not built-in regex or JSON-schema validation. It runs at every generation step. The default, `None`, keeps sampling unchanged.
+
+```python
+import llama_cpp
+# my_processor is your compatible callable: (input_ids, scores) -> scores.
+fn("Office line: +1-555-666-7777", logits_processor=llama_cpp.LogitsProcessorList([my_processor]))
+```
+
+Processors see the full prompt and generated-token history, not just the output. Create or reset stateful processors for each call. Token limits and output whitespace trimming still apply, so validate the returned result.
+
 ## Desktop and Offline Workflows
 
 Prepare and inspect validated local assets without keeping a model loaded:
@@ -177,6 +202,7 @@ Generate API keys at [programasweights.com/settings](https://programasweights.co
 paw compile --spec "Extract error lines from logs" --json
 paw run --program <program_id> --input "[ERROR] timeout" --json
 paw run --program <program_id> --input "[ERROR] timeout" --offline --json
+paw run --program <program_id> --input "[ERROR] timeout" --remote --json
 paw login
 ```
 

{programasweights-0.4.5 → programasweights-0.4.7}/PYPI_README.md

@@ -35,6 +35,19 @@ fn("I love this!")  # "positive"
 
 If you specifically want the smaller browser-compatible runtime, pass `compiler="paw-4b-gpt2"`. Otherwise, omit `compiler` and let the server default decide.
 
+## Remote inference (optional)
+
+Use the hosted API for fast inference in around 150 ms, without a local model download.
+
+```python
+import programasweights as paw
+
+with paw.function("email-triage", remote=True) as remote_fn:
+    print(remote_fn("Urgent: the server is down!"))
+```
+
+For direct HTTP calls, see the [REST API reference](https://programasweights.readthedocs.io/en/latest/api-reference/rest-api/#post-infer).
+
 ## Current Public Compilers
 
 
@@ -53,6 +66,18 @@ If you need to inspect available compiler aliases programmatically, use `paw.lis
 
 GPU acceleration is enabled by default (Metal on Mac, CUDA on Linux, falls back to CPU). Set `PAW_GPU_LAYERS=0` to force CPU if GPU causes issues.
 
+## Constrained Decoding
+
+In SDK 0.4.6+, a call accepts an optional `logits_processor`: an advanced hook for caller-supplied llama.cpp-compatible token constraints, not built-in regex or JSON-schema validation. It runs at every generation step. The default, `None`, keeps sampling unchanged.
+
+```python
+import llama_cpp
+# my_processor is your compatible callable: (input_ids, scores) -> scores.
+fn("Office line: +1-555-666-7777", logits_processor=llama_cpp.LogitsProcessorList([my_processor]))
+```
+
+Processors see the full prompt and generated-token history, not just the output. Create or reset stateful processors for each call. Token limits and output whitespace trimming still apply, so validate the returned result.
+
 ## Desktop and Offline Workflows
 
 Prepare and inspect validated local assets without keeping a model loaded:
@@ -146,6 +171,7 @@ Generate API keys at [programasweights.com/settings](https://programasweights.co
 paw compile --spec "Extract error lines from logs" --json
 paw run --program <program_id> --input "[ERROR] timeout" --json
 paw run --program <program_id> --input "[ERROR] timeout" --offline --json
+paw run --program <program_id> --input "[ERROR] timeout" --remote --json
 paw login
 ```
 

{programasweights-0.4.5 → programasweights-0.4.7}/README.md

@@ -35,6 +35,19 @@ fn("I love this!")  # "positive"
 
 If you specifically want the smaller browser-compatible runtime, pass `compiler="paw-4b-gpt2"`. Otherwise, omit `compiler` and let the server default decide.
 
+## Remote inference (optional)
+
+Use the hosted API for fast inference in around 150 ms, without a local model download.
+
+```python
+import programasweights as paw
+
+with paw.function("email-triage", remote=True) as remote_fn:
+    print(remote_fn("Urgent: the server is down!"))
+```
+
+For direct HTTP calls, see the [REST API reference](docs/api-reference/rest-api.md#post-infer).
+
 ## Current Public Compilers
 
 
@@ -53,6 +66,18 @@ If you need to inspect available compiler aliases programmatically, use `paw.lis
 
 GPU acceleration is enabled by default (Metal on Mac, CUDA on Linux, falls back to CPU). Set `PAW_GPU_LAYERS=0` to force CPU if GPU causes issues.
 
+## Constrained Decoding
+
+In SDK 0.4.6+, a call accepts an optional `logits_processor`: an advanced hook for caller-supplied llama.cpp-compatible token constraints, not built-in regex or JSON-schema validation. It runs at every generation step. The default, `None`, keeps sampling unchanged.
+
+```python
+import llama_cpp
+# my_processor is your compatible callable: (input_ids, scores) -> scores.
+fn("Office line: +1-555-666-7777", logits_processor=llama_cpp.LogitsProcessorList([my_processor]))
+```
+
+Processors see the full prompt and generated-token history, not just the output. Create or reset stateful processors for each call. Token limits and output whitespace trimming still apply, so validate the returned result.
+
 ## Desktop and Offline Workflows
 
 Prepare and inspect validated local assets without keeping a model loaded:
@@ -147,6 +172,7 @@ Generate API keys at [programasweights.com/settings](https://programasweights.co
 paw compile --spec "Extract error lines from logs" --json
 paw run --program <program_id> --input "[ERROR] timeout" --json
 paw run --program <program_id> --input "[ERROR] timeout" --offline --json
+paw run --program <program_id> --input "[ERROR] timeout" --remote --json
 paw login
 ```
 

programasweights-0.4.7/RELEASING.md

@@ -0,0 +1,82 @@
+# Releasing the Python SDK
+
+A release has three separate records: an annotated Git tag identifies the source,
+PyPI serves the installable wheel and source distribution, and a GitHub Release
+shows the version and release notes on the repository. A tag or PyPI upload alone
+does not create a GitHub Release. Finish and verify all three before reporting a
+release complete.
+
+## Publish a new version
+
+1. Update `pyproject.toml`, the fallback `__version__` in
+   `programasweights/__init__.py`, and the matching section in `CHANGELOG.md`.
+   Commit and push these changes to `main` using the existing user Git identity.
+   Release from a clean checkout of that pushed commit; confirm `HEAD` equals
+   `origin/main` and the `tests` workflow is green for that exact commit.
+2. Set the intended version, then build a wheel and source distribution into a
+   new, empty directory. Install `build` and `twine` in the release environment if
+   needed. Use the two exact filenames below, not a shared `dist/*` directory.
+
+   ```bash
+   SDK_RELEASE_VERSION=0.4.7  # replace with the version being released
+   SDK_RELEASE_DIR=$(mktemp -d)
+   python -m build --outdir "$SDK_RELEASE_DIR"
+   python -m twine check \
+     "$SDK_RELEASE_DIR/programasweights-$SDK_RELEASE_VERSION-py3-none-any.whl" \
+     "$SDK_RELEASE_DIR/programasweights-$SDK_RELEASE_VERSION.tar.gz"
+   ```
+
+3. Create and push the annotated version tag **before uploading to PyPI**. Check
+   that the remote tag resolves to the intended commit and is an annotated tag.
+   Never replace an existing release tag with a new target.
+
+   ```bash
+   git tag -a "v$SDK_RELEASE_VERSION" -m "Release v$SDK_RELEASE_VERSION"
+   git push origin "v$SDK_RELEASE_VERSION"
+   git ls-remote --tags origin "refs/tags/v$SDK_RELEASE_VERSION*"
+   ```
+
+4. Upload the exact wheel and source distribution that passed `twine check`:
+
+   ```bash
+   python -m twine upload \
+     "$SDK_RELEASE_DIR/programasweights-$SDK_RELEASE_VERSION-py3-none-any.whl" \
+     "$SDK_RELEASE_DIR/programasweights-$SDK_RELEASE_VERSION.tar.gz"
+   ```
+
+5. The `release.yml` workflow starts on the `v*` tag push. It waits up to 600
+   seconds for PyPI, verifies the remote annotated tag, version metadata, and
+   published wheel/source distribution against the tagged source, then creates
+   the GitHub Release with notes from that tag's `CHANGELOG.md` and the published
+   artifacts. It preserves an existing release and marks a new release as latest
+   only when its version is the latest on PyPI. If upload takes longer than the
+   wait window, dispatch the workflow again after PyPI publication succeeds.
+6. Verify the workflow succeeded, the GitHub Releases page has the expected entry
+   and notes, and the latest-release entry agrees with PyPI's latest version.
+   Compare both uploaded files' SHA-256 hashes with the version's PyPI JSON
+   (`https://pypi.org/pypi/programasweights/<version>/json`, `urls[].digests.sha256`)
+   and with the GitHub Release assets. Do not treat a tag listing as confirmation
+   that the release entry exists.
+
+## Recover a missing GitHub Release
+
+When a version already exists on PyPI, keep its existing tag and published files.
+Do not rebuild or republish that version, and never move its tag. Verify the
+remote annotated tag, matching version metadata and changelog section, and the
+published artifacts first. `scripts/release_metadata.py` requires Python 3.11+
+and performs those checks, preparing the notes and assets in a new directory:
+
+```bash
+SDK_RELEASE_RECOVERY_DIR=$(mktemp -d)
+git fetch origin --tags
+python scripts/release_metadata.py --tag v0.4.6 \
+  --output-dir "$SDK_RELEASE_RECOVERY_DIR/verified" --wait-seconds 600
+gh workflow run release.yml --ref main -f tag=v0.4.6
+```
+
+Replace `v0.4.6` with the verified existing tag to repair. Run this from current
+`main`, which contains the recovery workflow and script; the release content is
+read from the requested tag. The workflow is idempotent: an existing GitHub
+Release is preserved. Recheck the release entry, latest-version status, and
+artifact hashes after it finishes. A source or artifact mismatch needs
+investigation; do not retag or overwrite published artifacts to make it pass.

{programasweights-0.4.5 → programasweights-0.4.7}/docs/api-reference/cli.md

@@ -33,10 +33,14 @@ paw compile --spec "Classify message urgency" [--compiler paw-4b-qwen3-0.6b] [--
 
 Run inference locally against a compiled program (the normal mode), or
 explicitly against a bare base interpreter (advanced mode).
+Add `--remote` for hosted inference without downloading local model assets.
 
 ```bash
 paw run --program <id_or_slug> --input "your text" [--offline] [--json]
 
+# Optional hosted inference
+paw run --program email-triage --input "Urgent: the server is down!" --remote --json
+
 # Advanced adapter-free mode
 paw run --base --interpreter gpt2 --input "raw prompt" [--offline] [--json]
 ```
@@ -47,10 +51,11 @@ paw run --base --interpreter gpt2 --input "raw prompt" [--offline] [--json]
 | `--base` | Select adapter-free base mode. Mutually exclusive with `--program` and requires `--interpreter`. |
 | `--interpreter` | Base interpreter: `Qwen/Qwen3-0.6B` or `gpt2`. Only valid with `--base`. |
 | `--input` | Input text for the program. |
-| `--max-tokens` | Maximum tokens to generate (default: 512). |
-| `--temperature` | Sampling temperature (default: 0.0). |
+| `--max-tokens` | Maximum tokens to generate. Local default: 512. Remote inference uses the server default when omitted. |
+| `--temperature` | Sampling temperature. Local default: 0.0. Remote inference uses the server default when omitted. |
 | `--verbose` | Print llama.cpp debug output. |
 | `--offline` | Require all selected assets to already be cached and make zero network calls. |
+| `--remote` | Use hosted inference for `--program`. Incompatible with `--base`, `--offline`, `--verbose`, and `PAW_OFFLINE=1`. |
 | `--json` | JSON output with `mode`, `program`, `interpreter`, `input`, and `output`. |
 
 Exactly one of `--program` and `--base` is required. An empty or

{programasweights-0.4.5 → programasweights-0.4.7}/docs/api-reference/python-sdk.md

@@ -24,11 +24,13 @@ fn = paw.function(
     verbose=False,
     offline=False,
     *,
+    remote=False,
     interpreter=None,
 )
 ```
 
-Loads a compiled program and returns a callable. Hub references download the
+Returns a callable for a compiled program. Local inference is the default.
+Hub references download the
 program and base model on first use; local `.paw` files supply the program
 bundle directly. Required runtime metadata and base models are cached for reuse.
 
@@ -39,12 +41,13 @@ bundle directly. Required runtime metadata and base models are cached for reuse.
 | `n_gpu_layers` | GPU layers to offload (`0` = CPU-only, `-1` = all). The default is `-1`, or `PAW_GPU_LAYERS` when set. |
 | `verbose` | Enable verbose logging (default `False`). |
 | `offline` | Use only local files/cache and make zero network calls; fail if required validated assets are missing. `PAW_OFFLINE=1` has the same effect. |
+| `remote` | Run hosted inference without downloading model assets (default `False`). Accepts a `Program` object, ID, or slug. Cannot be combined with offline mode, local file paths, `interpreter`, or non-default local runtime options. |
 | `interpreter` | Advanced adapter-free mode only. Must be passed by keyword and only when `program_id` is explicitly `None`. Supported values are `Qwen/Qwen3-0.6B` and `gpt2`. |
 
-The returned callable:
+For local inference, the returned callable accepts:
 
 ```python
-output: str = fn(input_text, max_tokens=None, temperature=0.0)
+output: str = fn(input_text, max_tokens=None, temperature=0.0, logits_processor=None)
 ```
 
 | Parameter | Description |
@@ -52,6 +55,9 @@ output: str = fn(input_text, max_tokens=None, temperature=0.0)
 | `input_text` | Input string for the program. |
 | `max_tokens` | Maximum tokens to generate. `None` (default) = use all remaining context window. |
 | `temperature` | Sampling temperature (default `0.0`). |
+| `logits_processor` | SDK 0.4.6+. Optional `llama_cpp.LogitsProcessorList` of caller-supplied processors, applied at every generation step. `None` (default) keeps sampling unchanged. |
+
+This advanced hook is not built-in regex or JSON-schema validation. Each processor takes `(input_ids, scores)` and returns modified scores. Its token history includes the full prompt (including any compiled prefix and suffix or base-model template) plus generated tokens. Create or reset stateful processors for each call; processor exceptions propagate to the caller. Token limits and the usual output whitespace trimming still apply, so validate the returned result.
 
 **Context limits:** Spec + input + output share a ~2048 token window. Inputs that exceed it will error. `max_tokens` defaults to `None`: generation runs until EOS or the context limit.
 
@@ -64,6 +70,25 @@ same canonical integrity metadata, so missing server-side checksum fields
 cannot weaken validation. Missing or failed adapters raise an error; the SDK
 never silently falls back to an unadapted base model.
 
+### Remote inference
+
+```python
+with paw.function("email-triage", remote=True) as remote_fn:
+    output = remote_fn("Urgent: the server is down!")
+```
+
+The callable returns a string and accepts optional `max_tokens` and `temperature`.
+Omitting either argument, or passing `None`, uses the server's default for that
+setting. `logits_processor` is supported only for local inference.
+
+Slugs are resolved once when the function is loaded. The `with` block closes
+the HTTP client; otherwise, call `remote_fn.close()` when finished.
+
+Remote calls use the existing [SDK configuration](#configuration) for the API
+URL and API key. HTTP 4xx/5xx responses raise `paw.APIError`, preserving the
+response status, headers, and structured error details. Transport errors
+propagate unchanged.
+
 ### Loading a local `.paw` file
 
 Version 0.4.5 adds local-file inputs to `paw.function`:
@@ -271,12 +296,21 @@ attempt to load a function when compilation raises.
 ## `paw.compile_and_load`
 
 ```python
-fn = paw.compile_and_load(spec, compiler="paw-4b-qwen3-0.6b", **kwargs)
+fn = paw.compile_and_load(spec, compiler=None, remote=False, **kwargs)
 ```
 
-Convenience method that compiles a spec and immediately loads the result for local inference. Equivalent to `paw.function(paw.compile(spec, ...).id)`. Returns a callable.
+Compiles a spec and returns a callable. Inference runs locally by default.
+Pass `remote=True` for hosted inference without downloading model assets.
+
+Accepts the parameters of `paw.compile`, plus `n_ctx`, `n_gpu_layers`, `verbose`,
+and `remote`. Local runtime options must retain their defaults with `remote=True`.
 
-Accepts all the same parameters as `paw.compile`.
+```python
+with paw.compile_and_load(
+    "Classify sentiment as positive or negative", remote=True,
+) as remote_fn:
+    output = remote_fn("I love this!")
+```
 
 ## `paw.list_programs`
 

{programasweights-0.4.5 → programasweights-0.4.7}/docs/api-reference/rest-api.md

@@ -42,7 +42,10 @@ Compile a specification.
 
 ### `POST /infer`
 
-Run inference for a compiled program (server-side execution).
+Use the hosted API for fast inference in around 150 ms, without a local model download.
+
+The Python SDK exposes hosted inference through `paw.function(..., remote=True)`.
+See the [remote inference reference](python-sdk.md#remote-inference).
 
 **Request body (JSON):**
 
@@ -55,6 +58,23 @@ Run inference for a compiled program (server-side execution).
 
 **Response (JSON):** includes `output`, `tokens_generated`, and `latency_ms`.
 
+**Example:**
+
+```python
+import httpx
+
+with httpx.Client(timeout=60.0) as client:
+    response = client.post(
+        "https://programasweights.com/api/v1/infer",
+        json={
+            "program_id": "email-triage",
+            "input": "Urgent: server is down!"
+        },
+    )
+    response.raise_for_status()
+    print(response.json()["output"])
+```
+
 ### `GET /programs`
 
 List or search programs.
@@ -136,7 +156,16 @@ List available compiler models and identifiers for use with compile requests.
 
 ### `GET /health`
 
-Liveness or readiness style health check for the API service.
+Returns HTTP 200 for API liveness. The JSON `status` is `healthy` only when
+all enabled public compilers pass their provider checks; otherwise it is
+`degraded`, with details in `warnings` and `gpu_services`, keyed by compiler
+name. Finetune checks include its base compiler, durable Redis, and at least
+one healthy worker. A healthy worker remains available while busy.
+
+Checks run concurrently with a three-second timeout and share a five-second
+cache. `queue_depth` counts waiting finetune jobs across distinct dispatchers,
+not running jobs; it is `null` when a queue cannot be verified. This is a
+readiness observation, not a guarantee that a new compilation will succeed.
 
 ## Errors
 

{programasweights-0.4.5 → programasweights-0.4.7}/docs/getting-started/first-program.md

@@ -20,6 +20,17 @@ print(result)
 
 The first call may download the program and runtime assets; later calls use the local cache.
 
+### Optional: remote inference
+
+For fast inference without downloading model assets, pass `remote=True`:
+
+```python
+with paw.function("email-triage", remote=True) as remote_fn:
+    print(remote_fn("Urgent: the server is down!"))
+```
+
+For direct HTTP calls, see the [REST API reference](../api-reference/rest-api.md#post-infer).
+
 ## Step 2: Compile your own program
 
 Describe the behavior you want in natural language, compile it, then load the result by `program_id`:

{programasweights-0.4.5 → programasweights-0.4.7}/docs/getting-started/installation.md

@@ -16,6 +16,9 @@ The `--extra-index-url` flag provides pre-built binaries for `llama-cpp-python`,
 GPU offload defaults to all available layers (`n_gpu_layers=-1`). Set
 `PAW_GPU_LAYERS=0` to force CPU-only execution.
 
+The same installation supports [remote inference](../api-reference/python-sdk.md#remote-inference)
+with `paw.function(..., remote=True)`, without downloading local model assets.
+
 ## Anaconda on Linux: OpenMP / libgomp errors
 
 If the build fails with `libgomp`-related errors, disable OpenMP: