jevkit-runtime 0.2.0__tar.gz → 0.3.1__tar.gz

jevkit_runtime-0.3.1/CHANGELOG.md

@@ -0,0 +1,42 @@
+# Changelog
+
+`jevkit-runtime` is 0.x: a minor version may break the API. The JevKit tools pin `<0.(n+1)` so a
+new minor release never changes an installed tool.
+
+## 0.3.1
+
+- Documentation and package metadata only; the code is the same as 0.3.0.
+- The DiffusionGemma and Laya guides report what to expect from the four tools' benchmarks.
+- Add this changelog and a security policy, and list the repository and changelog on PyPI.
+- README links point to GitHub, so they also work on PyPI.
+
+## 0.3.0
+
+- Add the local decision servers `diffusiongemma` (OpenJev) and `laya` (laya-mlx) to the catalog.
+  They need no key, are chosen only by name and never in place of a configured hosted provider, and
+  are priced at zero API fees.
+- A provider may carry its own price; `JEV_PRICE_PER_MTOK` overrides it and `Settings.list_price`
+  is the fallback.
+- Providers with joint reads key every answer on the whole ordered batch and are re-sent whole when
+  any slot is missing.
+- Setup guides for both servers in `docs/`, and the Laya server adapter in `scripts/`.
+
+## 0.2.0
+
+Breaking redesign: one request pipeline, one answer store, one provider catalog.
+
+- `Client.ask` owns the whole pipeline: request sharing, hedging, cache-only runs, charge callbacks
+  and provenance are per-call keyword arguments. It returns `Answers`, a dict with an `origins`
+  attribute.
+- The import is now `jevkit_runtime`, matching the distribution.
+- The answer store keeps answer and provenance in one row under a versioned schema; caches from
+  0.1 are reset and re-asked.
+- HTTP/2 is used whenever the `http2` extra is installed.
+- Removed: the legacy key recipes, the two-table cache, the `DecisionClient` seam, and the unused
+  local providers.
+
+## 0.1.0
+
+- First release: shared transport, settings, answer cache and metering for jgrep, jsort, jlink,
+  jselect and jcol, published as `jevkit-runtime` (the PyPI name `jevkit-core` belongs to another
+  project).

jevkit_runtime-0.2.0/README.md → jevkit_runtime-0.3.1/PKG-INFO

@@ -1,3 +1,20 @@
+Metadata-Version: 2.5
+Name: jevkit-runtime
+Version: 0.3.1
+Summary: Shared transport, configuration, caching, and accounting for JevKit tools
+Project-URL: Homepage, https://github.com/keltokhy/jevkit-core
+Project-URL: Repository, https://github.com/keltokhy/jevkit-core
+Project-URL: Changelog, https://github.com/keltokhy/jevkit-core/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/keltokhy/jevkit-core/issues
+Author: Khaled Eltokhy
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.10
+Requires-Dist: httpx>=0.27
+Provides-Extra: http2
+Requires-Dist: httpx[http2]>=0.27; extra == 'http2'
+Description-Content-Type: text/markdown
+
 # JevKit core
 
 Distribution **`jevkit-runtime`**, import **`jevkit_runtime`**. The PyPI name `jevkit-core` belongs
@@ -48,14 +65,32 @@ HTTP/2 is used whenever the `http2` extra is installed.
 - **Credentials** come from the provider's variable, then `$XDG_CONFIG_HOME/jev/<provider>.key`.
   Gateways take their URL from `JEV_GATEWAY_URL` or `<provider>.url`. `JEV_URL` overrides any endpoint.
 - **Metering** refuses malformed usage rather than under-counting; a response without a reported
-  cost is priced from its tokens at the run's list price.
+  cost is priced from its tokens at the provider's price, zero for local servers, or the list price.
+  `JEV_PRICE_PER_MTOK` overrides both.
 - **Errors** keep their wording across tools: a fatal status reads `PROVIDER said 401: detail`, a
   bad request reads `HTTP 400: detail`, and exhaustion reads `gave up after 15s (last failure)`.
   Both status errors carry `provider`, `status` and `detail` for tools that word or redact them.
 
+## Local servers
+
+Two catalog entries point at System One servers on your own machine: `diffusiongemma`, an
+[OpenJev](https://github.com/razorback16/openjev) server on port 8080, and `laya`, a
+[laya-mlx](https://github.com/mizorewww/laya-mlx) server on port 8081. Every JevKit tool names
+them in its catalog, so `--api laya` or `JEV_API=laya` works everywhere. They are never chosen
+in place of a configured hosted provider, need no key, and are metered at zero API fees unless
+`JEV_PRICE_PER_MTOK` says otherwise. `JEV_LAYA_URL` and `JEV_DIFFUSIONGEMMA_URL`, or the matching
+`.url` files, point at a server elsewhere.
+
+DiffusionGemma reads every question in a batch together, so the runtime keys each of its answers
+on the whole ordered batch and re-sends a batch whole when any slot is missing.
+
+No package ships the models. [docs/diffusiongemma.md](https://github.com/keltokhy/jevkit-core/blob/main/docs/diffusiongemma.md) and
+[docs/laya.md](https://github.com/keltokhy/jevkit-core/blob/main/docs/laya.md) explain how to run the servers, and `scripts/laya_server.py` is the
+adapter the Laya guide starts.
+
 ## Development
 
-Keep the six checkouts as siblings. Each consumer depends on `jevkit-runtime>=0.2.0,<0.3.0` and
+Keep the six checkouts as siblings. Each consumer depends on `jevkit-runtime>=0.3.0,<0.4.0` and
 overrides it for development with `jevkit-runtime = { path = "../jevkit-core", editable = true }`
 under `[tool.uv.sources]`.
 

jevkit_runtime-0.2.0/PKG-INFO → jevkit_runtime-0.3.1/README.md

@@ -1,18 +1,3 @@
-Metadata-Version: 2.5
-Name: jevkit-runtime
-Version: 0.2.0
-Summary: Shared transport, configuration, caching, and accounting for JevKit tools
-Project-URL: Homepage, https://github.com/keltokhy/jevkit-core
-Project-URL: Issues, https://github.com/keltokhy/jevkit-core/issues
-Author: Khaled Eltokhy
-License-Expression: MIT
-License-File: LICENSE
-Requires-Python: >=3.10
-Requires-Dist: httpx>=0.27
-Provides-Extra: http2
-Requires-Dist: httpx[http2]>=0.27; extra == 'http2'
-Description-Content-Type: text/markdown
-
 # JevKit core
 
 Distribution **`jevkit-runtime`**, import **`jevkit_runtime`**. The PyPI name `jevkit-core` belongs
@@ -63,14 +48,32 @@ HTTP/2 is used whenever the `http2` extra is installed.
 - **Credentials** come from the provider's variable, then `$XDG_CONFIG_HOME/jev/<provider>.key`.
   Gateways take their URL from `JEV_GATEWAY_URL` or `<provider>.url`. `JEV_URL` overrides any endpoint.
 - **Metering** refuses malformed usage rather than under-counting; a response without a reported
-  cost is priced from its tokens at the run's list price.
+  cost is priced from its tokens at the provider's price, zero for local servers, or the list price.
+  `JEV_PRICE_PER_MTOK` overrides both.
 - **Errors** keep their wording across tools: a fatal status reads `PROVIDER said 401: detail`, a
   bad request reads `HTTP 400: detail`, and exhaustion reads `gave up after 15s (last failure)`.
   Both status errors carry `provider`, `status` and `detail` for tools that word or redact them.
 
+## Local servers
+
+Two catalog entries point at System One servers on your own machine: `diffusiongemma`, an
+[OpenJev](https://github.com/razorback16/openjev) server on port 8080, and `laya`, a
+[laya-mlx](https://github.com/mizorewww/laya-mlx) server on port 8081. Every JevKit tool names
+them in its catalog, so `--api laya` or `JEV_API=laya` works everywhere. They are never chosen
+in place of a configured hosted provider, need no key, and are metered at zero API fees unless
+`JEV_PRICE_PER_MTOK` says otherwise. `JEV_LAYA_URL` and `JEV_DIFFUSIONGEMMA_URL`, or the matching
+`.url` files, point at a server elsewhere.
+
+DiffusionGemma reads every question in a batch together, so the runtime keys each of its answers
+on the whole ordered batch and re-sends a batch whole when any slot is missing.
+
+No package ships the models. [docs/diffusiongemma.md](https://github.com/keltokhy/jevkit-core/blob/main/docs/diffusiongemma.md) and
+[docs/laya.md](https://github.com/keltokhy/jevkit-core/blob/main/docs/laya.md) explain how to run the servers, and `scripts/laya_server.py` is the
+adapter the Laya guide starts.
+
 ## Development
 
-Keep the six checkouts as siblings. Each consumer depends on `jevkit-runtime>=0.2.0,<0.3.0` and
+Keep the six checkouts as siblings. Each consumer depends on `jevkit-runtime>=0.3.0,<0.4.0` and
 overrides it for development with `jevkit-runtime = { path = "../jevkit-core", editable = true }`
 under `[tool.uv.sources]`.
 

jevkit_runtime-0.3.1/SECURITY.md

@@ -0,0 +1,18 @@
+# Security
+
+## Reporting a vulnerability
+
+Please report security problems privately through GitHub:
+[Report a vulnerability](https://github.com/keltokhy/jevkit-core/security/advisories/new).
+Do not open a public issue. This is maintained by one person in spare time; expect a first reply
+within a week.
+
+Only the latest release is supported. Fixes ship as a new release, not as patches to older ones.
+
+## What to know when using it
+
+- Text you pass in is sent to the model provider you configure (TypeSafe, OpenRouter, a gateway,
+  or a server on your own machine). Do not send data you are not allowed to share with that provider.
+- API keys are read from environment variables or `~/.config/jev/<provider>.key` and are sent only
+  to the provider they belong to. They are never written to the answer cache or to logs.
+- Answers are cached on disk in `~/.cache/jev`. Delete it to remove stored answers.

jevkit_runtime-0.3.1/docs/diffusiongemma.md

@@ -0,0 +1,98 @@
+# DiffusionGemma (local, experimental)
+
+Any JevKit tool can send its decisions to a DiffusionGemma decision server on your own machine
+instead of a hosted provider. The server is chosen only by name, never automatically, and its
+calls are metered at zero API fees:
+
+```bash
+jgrep --api diffusiongemma -j 1 "a complaint about noise" complaints.txt
+jcol --api diffusiongemma run table.csv codebook.json
+```
+
+No JevKit package ships the model; it runs in its own environment and process. Model quality
+and useful probability thresholds need evaluation on your own inputs.
+
+## Apple silicon
+
+[OpenJev](https://github.com/razorback16/openjev) supplies an MLX implementation of the
+structured-read approach from [vLLM PR #57250](https://github.com/vllm-project/vllm/pull/57250).
+Its 4-bit checkpoint is roughly 16.6 GB to download and needs about 16 GB of model memory plus
+working memory. Install the server in its own directory and environment:
+
+```bash
+git clone https://github.com/razorback16/openjev.git
+cd openjev
+git checkout e04794ab36e4f7e6040c2547baecdb2737ce2e79
+uv sync --python 3.12 --extra mlx
+OPENJEV_BACKEND=mlx uv run --extra mlx python -m openjev
+```
+
+The server downloads `mlx-community/diffusiongemma-26B-A4B-it-4bit` on first start. Wait for
+`Application startup complete` before querying it. `OPENJEV_MLX_MODEL` can point at a downloaded
+snapshot to pin the weights; the revision used in the original experiment was
+`a7a81407613811e8ba63af92ac0d852b809e191f`. Then, in another terminal:
+
+```bash
+printf '%s\n' 'The music next door keeps me awake.' 'The elevator is broken.' |
+  jgrep --api diffusiongemma --model openjev-0.1 -j 1 --timeout 120 --stats -o 'a complaint about noise'
+```
+
+Start with one request at a time: the MLX backend executes model work serially, and a deep
+queue can exceed a tool's per-request deadline. The longer timeout covers the first inference.
+Raise concurrency from measured throughput once the model is warm.
+
+## NVIDIA / vLLM
+
+Run OpenJev's documented vLLM deployment, or the prototype `structured_server.py` from
+[PR #57250](https://github.com/vllm-project/vllm/pull/57250). That PR was unmerged at the time of
+writing, so a released vLLM is not enough on its own; follow the server's pinned build
+instructions. Its `/v1/systemone` adapter sits in front of vLLM. For the PR example server's
+default port:
+
+```bash
+JEV_DIFFUSIONGEMMA_URL=http://127.0.0.1:8011/v1/systemone \
+  jgrep --api diffusiongemma --model jev-latest 'a stack trace' build.log
+```
+
+## Configuration
+
+| Setting | Meaning |
+|---|---|
+| `--api diffusiongemma` or `JEV_API=diffusiongemma` | Select it; it is never picked by discovery |
+| `JEV_DIFFUSIONGEMMA_URL` or `~/.config/jev/diffusiongemma.url` | Full endpoint, default `http://127.0.0.1:8080/v1/systemone` |
+| `JEV_DIFFUSIONGEMMA_API_KEY` or `~/.config/jev/diffusiongemma.key` | Optional bearer token; no `Authorization` header when absent |
+| `--model` or `JEV_MODEL` | Default `openjev-latest`; `openjev-0.1` pins the server's decision model |
+| `JEV_PRICE_PER_MTOK` | Overrides the zero price for `--stats`, `--estimate`, and dollar budgets |
+
+`JEV_URL` overrides every endpoint. A cost the server reports always wins over the price.
+Zero means no API fee, not zero compute. Configure a price before using a dollar budget against
+a metered remote server; an offline estimate is a byte-based approximation and cannot predict
+the server's adaptive re-reads.
+
+## Joint reads and the cache
+
+A diffusion read answers every question in a batch in the light of the others, so the runtime
+keys each of this provider's answers on the provider, endpoint, model, state, and the entire
+ordered batch of questions, IDs included. A batch is reused only whole: if any slot is missing
+from the cache, the whole batch is sent again. Hosted providers and Laya keep per-question
+caching. Pin both server and weights; after changing weights or inference settings behind the
+same URL and model, use `--no-cache` or a separate `XDG_CACHE_HOME`.
+
+## What to expect
+
+On September 22, 2026, DiffusionGemma (`openjev-0.1`) ran each tool's benchmarks on an Apple M3
+Ultra with 96 GiB of unified memory, beside Jev 1.13's recorded runs. It is a reasonable
+substitute for Jev when the text must stay on your machine and each decision is about one record,
+or about long ones. In
+[jgrep](https://github.com/keltokhy/jgrep/blob/main/docs/benchmarks/local-models-2026-09-22.md)
+its spam F1 was 0.87 against Jev's 0.91, with the same news accuracy; in
+[jlink](https://github.com/keltokhy/jlink/blob/main/docs/benchmarks/local-models-2026-09-22.md)
+it came within 0.03 of Jev's F1 on four of five benchmarks; in
+[jcol](https://github.com/keltokhy/jcol/blob/main/benchmarks/README.md#local-models-2026-09-22)
+it agreed with the product label on 78 of 100 complaints against Jev's 75; and in
+[jsort](https://github.com/keltokhy/jsort/blob/main/docs/benchmarks/local-models-2026-09-22.md)
+its scores for whole FOMC statements correlated 0.89 with Jev's. It is not a substitute for
+sorting single short lines, where its comparisons had a reliability of 0.62 and 0.39 against
+Jev's 0.97 and 0.96, or for judging isolated diff lines. The server runs one call at a time on
+the GPU: about a third of a second per short record, and 14 to 18 minutes per 3,000 record pairs.
+Keep it experimental and evaluate its cutoff on your own data.

jevkit_runtime-0.3.1/docs/laya.md

@@ -0,0 +1,93 @@
+# Laya (local, experimental)
+
+Any JevKit tool can send its decisions to a Laya server on your own machine instead of a hosted
+provider. The server is chosen only by name, never automatically, and its calls are metered at
+zero API fees:
+
+```bash
+jgrep --api laya "a complaint about noise" complaints.txt
+jsort --api laya "more urgent" tickets.txt
+JEV_API=laya jlink link left.csv right.csv --on name
+```
+
+This guide runs the general English 421M-parameter [Laya](https://github.com/NandhaKishorM/laya)
+checkpoint through the independent [laya-mlx port](https://github.com/mizorewww/laya-mlx) on
+Apple silicon. No JevKit package ships the model; it runs in its own environment and process.
+The multilingual and newer typed-decision checkpoints have not been tried.
+
+## Setup (Apple silicon)
+
+Keep the model dependencies in a separate environment. From a directory outside this repository:
+
+```bash
+git clone https://github.com/mizorewww/laya-mlx.git
+cd laya-mlx
+git checkout fc1df62828a3fedf4d8229fdac1cbd85f1cdf337
+uv sync --python 3.12
+uv pip install --python .venv/bin/python fastapi==0.141.1 uvicorn==0.53.0
+.venv/bin/python -c 'from huggingface_hub import snapshot_download; print(snapshot_download("aac6fef/laya-mlx", revision="047678560251f28113ee8f5df4be82102c7bf336"))'
+```
+
+Use the printed snapshot path as `--checkpoint`, then start the adapter from this repository
+with that environment's Python:
+
+```bash
+/path/to/laya-mlx/.venv/bin/python scripts/laya_server.py \
+  --checkpoint /path/to/downloaded/snapshot \
+  --audit /path/to/laya-audit.jsonl
+```
+
+It binds to loopback port 8081, loads FP16 weights, warms the model, and serves `/v1/systemone`.
+`/health` reports the checkpoint, context budget, and truncation policy. Then, in another terminal:
+
+```bash
+printf '%s\n' 'The music next door keeps me awake.' 'The elevator is broken.' |
+  jgrep --api laya -j 4 --timeout 120 --stats -o 'a complaint about noise'
+```
+
+## Context limit
+
+The model's 512-token window includes the question's instructions and options, leaving less
+room for the record. The adapter checks the remaining budget separately for every question and
+answers HTTP 422 if any state would be cropped; the tool reports an error rather than a negative
+answer. `--allow-truncation` enables the native runtime's prefix retention instead, and every
+request then records the available and dropped state tokens per question in the audit log.
+Ordinary use should keep rejection on. Long question instructions can also be shortened by the
+runtime's own question budget; the adapter audits state truncation, not question truncation.
+
+## Configuration
+
+| Setting | Meaning |
+|---|---|
+| `--api laya` or `JEV_API=laya` | Select it; it is never picked by discovery |
+| `JEV_LAYA_URL` or `~/.config/jev/laya.url` | Full endpoint, default `http://127.0.0.1:8081/v1/systemone` |
+| `JEV_LAYA_API_KEY` or `~/.config/jev/laya.key` | Optional bearer token for a server that authenticates; the supplied adapter does not |
+| `--model` or `JEV_MODEL` | `laya-421m`; the supplied adapter rejects other IDs |
+| `JEV_PRICE_PER_MTOK` | Overrides the zero price for `--stats`, `--estimate`, and dollar budgets |
+
+`JEV_URL` overrides every endpoint. Zero API fees exclude hardware and electricity.
+
+Laya reads each question on its own, so its answers share the ordinary per-question cache with
+the hosted providers, keyed by provider, endpoint, model, state, and question. Pin the server
+and the weights; after changing the model behind the same URL and model ID, use `--no-cache`
+or a separate `XDG_CACHE_HOME`.
+
+## What to expect
+
+On September 22, 2026, Laya (`laya-421m`) ran each tool's benchmarks on an Apple M3 Ultra with
+96 GiB of unified memory, beside Jev 1.13's recorded runs. It is a fast classifier of short texts
+into broad topics, and not a substitute for Jev elsewhere. In
+[jgrep](https://github.com/keltokhy/jgrep/blob/main/docs/benchmarks/local-models-2026-09-22.md)
+it had the best news accuracy of the three models, 92% against Jev's 87%, and finished 2,000
+messages in 46 s; at the default cutoff it also flagged 235 messages that were not spam, against
+Jev's 41. Its 512-token window, question included, decides what it can read: the adapter refuses
+longer requests with HTTP 422, which the tools report as failed decisions, so in
+[jcol](https://github.com/keltokhy/jcol/blob/main/benchmarks/README.md#local-models-2026-09-22)
+it refused 27 of 100 complaint narratives and in
+[jsort](https://github.com/keltokhy/jsort/blob/main/docs/benchmarks/local-models-2026-09-22.md)
+every whole FOMC statement. On what it could read, its pairwise comparisons in jsort did not
+follow Jev's order, and in
+[jlink](https://github.com/keltokhy/jlink/blob/main/docs/benchmarks/local-models-2026-09-22.md)
+it scored most candidate pairs as matches, for an F1 of 0.16 to 0.22 on firms, products and
+software where Jev's was 0.67 to 0.94. Evaluate thresholds on your own labeled inputs before
+relying on the scores.

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "jevkit-runtime"
-version = "0.2.0"
+version = "0.3.1"
 description = "Shared transport, configuration, caching, and accounting for JevKit tools"
 readme = "README.md"
 license = "MIT"
@@ -10,6 +10,8 @@ dependencies = ["httpx>=0.27"]
 
 [project.urls]
 Homepage = "https://github.com/keltokhy/jevkit-core"
+Repository = "https://github.com/keltokhy/jevkit-core"
+Changelog = "https://github.com/keltokhy/jevkit-core/blob/main/CHANGELOG.md"
 Issues = "https://github.com/keltokhy/jevkit-core/issues"
 
 [project.optional-dependencies]

jevkit_runtime-0.3.1/scripts/laya_server.py

@@ -0,0 +1,116 @@
+"""Local System One adapter for a separately installed, pinned laya-mlx runtime.
+
+python scripts/laya_server.py --checkpoint /path/to/snapshot --audit /path/to/audit.jsonl
+State truncation fails explicitly unless --allow-truncation is supplied.
+Dependencies belong in the server environment: laya-mlx, fastapi, uvicorn.
+"""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import time
+from pathlib import Path
+
+
+def make_app(agent, *, audit_path, served_model="laya-421m", allow_truncation=False):
+    from fastapi import FastAPI, HTTPException
+    from laya_mlx.common import build_prefix, serialize_state
+
+    app = FastAPI()
+
+    @app.get("/health")
+    async def health():
+        return {
+            "model": served_model,
+            "max_tokens": agent.cfg["max_len"],
+            "allow_truncation": allow_truncation,
+            "checkpoint": str(agent.model_dir),
+        }
+
+    @app.post("/v1/systemone")
+    async def decide(body: dict):
+        if body.get("model") != served_model:
+            raise HTTPException(400, f"This server only serves {served_model!r}")
+        if "state" not in body or not isinstance(body.get("questions"), dict) or not body["questions"]:
+            raise HTTPException(400, "state and a nonempty questions object are required")
+        started = time.perf_counter()
+        state, questions = body["state"], body["questions"]
+        entry = {
+            "state_sha256": hashlib.sha256(
+                json.dumps(state, ensure_ascii=False, sort_keys=True).encode()
+            ).hexdigest(),
+            "questions_sha256": hashlib.sha256(
+                json.dumps(questions, ensure_ascii=False, sort_keys=True).encode()
+            ).hexdigest(),
+            "status": "started",
+            "questions": {},
+        }
+        try:
+            tokens = agent.tok(
+                serialize_state(state).replace(agent.tok.mask_token, " "), add_special_tokens=False
+            )["input_ids"]
+            for qid, definition in questions.items():
+                q = agent._to_internal(definition)
+                prefix, _ = build_prefix(agent.tok, q, agent.cfg.get("head_max_len", 192))
+                room = max(0, agent.cfg["max_len"] - len(prefix) - 1)
+                entry["questions"][qid] = {
+                    "state_tokens": len(tokens),
+                    "available_state_tokens": room,
+                    "dropped_state_tokens": max(0, len(tokens) - room),
+                }
+            if not allow_truncation and any(q["dropped_state_tokens"] for q in entry["questions"].values()):
+                entry["status"] = "context_rejected"
+                raise HTTPException(
+                    422,
+                    "State exceeds the Laya context budget; use shorter records "
+                    "or explicitly enable audited truncation",
+                )
+            # An async route intentionally runs MLX on one event-loop thread, serially.
+            result = agent.predict(state, questions)
+            result["model"] = served_model
+            result["usage"]["cost"] = 0.0
+            result["context_audit"] = entry["questions"]
+            entry["status"] = "ok"
+            entry["usage"] = result["usage"]
+            return result
+        except (ValueError, TypeError, KeyError) as error:
+            entry["status"] = "invalid_request"
+            raise HTTPException(400, str(error)) from error
+        finally:
+            entry["seconds"] = time.perf_counter() - started
+            with audit_path.open("a") as stream:
+                stream.write(json.dumps(entry, allow_nan=False) + "\n")
+
+    return app
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("--checkpoint", type=Path, required=True)
+    ap.add_argument("--audit", type=Path, required=True)
+    ap.add_argument("--port", type=int, default=8081)
+    ap.add_argument("--served-model", default="laya-421m")
+    ap.add_argument("--allow-truncation", action="store_true")
+    options = ap.parse_args()
+    import laya_mlx
+    import uvicorn
+
+    agent = laya_mlx.load(str(options.checkpoint), dtype="float16", device="gpu")
+    warmup = {"warmup": {"type": "noul", "instructions": "The text is a greeting."}}
+    agent.predict("Hello.", warmup)
+    options.audit.parent.mkdir(parents=True, exist_ok=True)
+    # Do not accidentally mix distinct server sessions in one audit file.
+    options.audit.touch(exist_ok=False)
+    app = make_app(
+        agent,
+        audit_path=options.audit,
+        served_model=options.served_model,
+        allow_truncation=options.allow_truncation,
+    )
+    uvicorn.run(app, host="127.0.0.1", port=options.port, log_level="warning", access_log=False)
+
+
+if __name__ == "__main__":
+    main()

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/src/jevkit_runtime/__init__.py

@@ -15,6 +15,7 @@ from .protocol import (
     QUESTION_TYPES,
     Usage,
     answer_key,
+    answer_keys,
     digest,
     error_detail,
     parse_answers,
@@ -27,7 +28,7 @@ from .settings import DEFAULT_PRICE_PER_MTOK, Settings
 from .store import AnswerStore, Entry
 from .transport import FATAL, RETRYABLE, post
 
-__version__ = "0.2.0"
+__version__ = "0.3.1"
 __all__ = [
     "AnswerStore",
     "Answers",
@@ -51,6 +52,7 @@ __all__ = [
     "Settings",
     "Usage",
     "answer_key",
+    "answer_keys",
     "catalog",
     "digest",
     "error_detail",

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/src/jevkit_runtime/client.py

@@ -14,6 +14,7 @@ from .errors import JevBudgetExceeded
 from .meter import Meter
 from .protocol import (
     answer_key,
+    answer_keys,
     answer_origin,
     parse_answers,
     parse_usage,
@@ -106,6 +107,9 @@ class Client:
     def key(self, state, question: dict) -> str:
         return answer_key(self.backend, state, question)
 
+    def keys(self, state, questions: dict[str, dict]) -> dict[str, str]:
+        return answer_keys(self.backend, state, questions)
+
     async def ask(
         self,
         state,
@@ -123,7 +127,7 @@ class Client:
         `on_cost` is charged only by the caller whose request actually went out. `hedge_after`
         sends a slow call a second time and keeps the first answer.
         """
-        keys = dict(keys) if keys is not None else {qid: self.key(state, q) for qid, q in questions.items()}
+        keys = dict(keys) if keys is not None else self.keys(state, questions)
         answers: dict[str, dict] = {}
         origins: dict[str, dict] = {}
         if self.store is not None:
@@ -133,6 +137,11 @@ class Client:
                     answers[qid] = entry.answer
                     origins[qid] = dict(entry.metadata or {}) | {"source": "cache"}
         misses = {qid: q for qid, q in questions.items() if qid not in answers}
+        if misses and self.backend.joint_reads and len(misses) != len(questions):
+            # Each slot was answered in the light of the others; asking for some alone would change that.
+            answers.clear()
+            origins.clear()
+            misses = dict(questions)
         if not misses:
             self.meter.cached += 1
         else:

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/src/jevkit_runtime/protocol.py

@@ -23,6 +23,14 @@ def answer_key(backend: Backend, state, question: dict) -> str:
     return digest([ANSWER_KEY_VERSION, backend.name, backend.url, backend.model, state, question])
 
 
+def answer_keys(backend: Backend, state, questions: dict[str, dict]) -> dict[str, str]:
+    """Each question's identity. Under joint reads it is the whole batch, which every answer depends on."""
+    if not backend.joint_reads:
+        return {qid: answer_key(backend, state, question) for qid, question in questions.items()}
+    batch = list(questions.items())
+    return {qid: answer_key(backend, state, {"slot": qid, "batch": batch}) for qid in questions}
+
+
 def request_body(model: str, state, questions: dict[str, dict]) -> dict:
     return {"model": model, "state": state, "questions": questions}
 

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/src/jevkit_runtime/providers.py

@@ -22,6 +22,8 @@ class Provider:
     url_env: str | None = None
     requires_key: bool = True
     auto_select: bool = True
+    price_per_mtok: float | None = None  # when the server reports no cost; None means the list price
+    joint_reads: bool = False  # every answer depends on the whole batch of questions, not on its own
 
     def key_file(self, settings: Settings) -> Path:
         return settings.config_dir / f"{self.name}.key"
@@ -55,6 +57,7 @@ class Backend:
     key: str = ""
     key_source: str = "none"
     price_per_mtok: float = DEFAULT_PRICE_PER_MTOK
+    joint_reads: bool = False
 
 
 PROVIDERS = {
@@ -68,6 +71,29 @@ PROVIDERS = {
         "OPENROUTER_API_KEY",
     ),
     "gateway": Provider("gateway", "", "jev-latest", "JEV_GATEWAY_API_KEY", url_env="JEV_GATEWAY_URL"),
+    # Local servers: chosen only by name, never in place of a configured hosted provider, and free
+    # of API fees. The models run in their own processes; no JevKit package ships them.
+    "diffusiongemma": Provider(
+        "diffusiongemma",
+        "http://127.0.0.1:8080/v1/systemone",
+        "openjev-latest",
+        "JEV_DIFFUSIONGEMMA_API_KEY",
+        url_env="JEV_DIFFUSIONGEMMA_URL",
+        requires_key=False,
+        auto_select=False,
+        price_per_mtok=0.0,
+        joint_reads=True,  # a diffusion read answers every slot in the light of the others
+    ),
+    "laya": Provider(
+        "laya",
+        "http://127.0.0.1:8081/v1/systemone",
+        "laya-421m",
+        "JEV_LAYA_API_KEY",
+        url_env="JEV_LAYA_URL",
+        requires_key=False,
+        auto_select=False,
+        price_per_mtok=0.0,
+    ),
 }
 
 
@@ -139,6 +165,19 @@ def _backend(provider: Provider, key: str, source: str, model: str | None, setti
         parsed = None
     if parsed is None or parsed.scheme not in ("http", "https") or not parsed.host:
         raise JevFatal(f"{provider.name} endpoint must be a complete HTTP or HTTPS URL, not {url!r}")
+    price = (
+        settings.price_per_mtok
+        if settings.price_per_mtok is not None
+        else provider.price_per_mtok
+        if provider.price_per_mtok is not None
+        else DEFAULT_PRICE_PER_MTOK
+    )
     return Backend(
-        provider.name, url, model or settings.model or provider.model, key, source, settings.price_per_mtok
+        provider.name,
+        url,
+        model or settings.model or provider.model,
+        key,
+        source,
+        price,
+        provider.joint_reads,
     )

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/src/jevkit_runtime/settings.py

@@ -22,7 +22,7 @@ class Settings:
     api: str | None = None
     url: str | None = None
     model: str | None = None
-    price_per_mtok: float = DEFAULT_PRICE_PER_MTOK
+    price_per_mtok: float | None = None  # JEV_PRICE_PER_MTOK when set; see list_price
     environ: Mapping[str, str] = field(default_factory=lambda: os.environ, repr=False, compare=False)
 
     @classmethod
@@ -30,12 +30,14 @@ class Settings:
         env = os.environ if environ is None else environ
         home = Path.home()
         raw_price = env.get("JEV_PRICE_PER_MTOK", "").strip()
-        try:
-            price = float(raw_price) if raw_price else DEFAULT_PRICE_PER_MTOK
-        except ValueError:
-            raise JevFatal(f"JEV_PRICE_PER_MTOK must be a number, not {raw_price!r}") from None
-        if not math.isfinite(price) or price < 0:
-            raise JevFatal("JEV_PRICE_PER_MTOK must be finite and nonnegative")
+        price = None
+        if raw_price:
+            try:
+                price = float(raw_price)
+            except ValueError:
+                raise JevFatal(f"JEV_PRICE_PER_MTOK must be a number, not {raw_price!r}") from None
+            if not math.isfinite(price) or price < 0:
+                raise JevFatal("JEV_PRICE_PER_MTOK must be finite and nonnegative")
         return cls(
             config_dir=Path(env.get("XDG_CONFIG_HOME") or home / ".config") / "jev",
             cache_dir=Path(env.get("XDG_CACHE_HOME") or home / ".cache") / "jev",
@@ -46,6 +48,11 @@ class Settings:
             environ=env,
         )
 
+    @property
+    def list_price(self) -> float:
+        """Dollars per million input tokens when neither the environment nor a provider says otherwise."""
+        return DEFAULT_PRICE_PER_MTOK if self.price_per_mtok is None else self.price_per_mtok
+
     def credential(self, name: str, variable: str) -> tuple[str, str]:
         """A provider key and where it came from: `env`, `file`, or `none`."""
         key = self.environ.get(variable, "").strip()

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/tests/test_client.py

@@ -198,6 +198,30 @@ def test_a_slow_call_is_hedged_and_the_first_answer_wins():
     run(exercise())
 
 
+def test_a_joint_read_is_reused_whole_or_repeated_whole(tmp_path):
+    fake = Fake()
+    store = AnswerStore(tmp_path / "answers.sqlite")
+    joint = Backend(
+        "diffusiongemma", "http://127.0.0.1:8080/v1/systemone", "openjev-latest", joint_reads=True
+    )
+
+    async def exercise():
+        async with Client(joint, store=store, transport=httpx.MockTransport(fake)) as client:
+            first = await client.ask("s", QUESTIONS)
+            assert await client.ask("s", QUESTIONS) == first and len(fake.bodies) == 1
+            await client.ask("s", dict(reversed(QUESTIONS.items())))
+            assert len(fake.bodies) == 2  # the same questions in another order are another read
+            await client.ask("s", {"q": QUESTIONS["q"]})
+            assert len(fake.bodies) == 3  # and so is one of them alone
+            store.db.execute("DELETE FROM answers WHERE key = ?", (client.keys("s", QUESTIONS)["r"],))
+            await client.ask("s", QUESTIONS)
+            assert len(fake.bodies) == 4 and list(fake.bodies[-1]["questions"]) == ["q", "r"]
+            assert client.meter.cached == 1
+
+    run(exercise())
+    store.close()
+
+
 def test_callers_may_supply_their_own_answer_identity(tmp_path):
     fake = Fake()
     store = AnswerStore(tmp_path / "answers.sqlite")

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/tests/test_protocol.py

@@ -5,6 +5,7 @@ from jevkit_runtime import (
     JevError,
     JevFatal,
     answer_key,
+    answer_keys,
     parse_answers,
     parse_usage,
     validate_answer,
@@ -23,6 +24,18 @@ def test_answer_key_depends_on_who_answers_and_what_was_asked():
     assert key != answer_key(BACKEND, "évidence ", question)
 
 
+def test_joint_reads_key_every_slot_on_the_whole_batch():
+    q = {"type": "noul", "instructions": "rule"}
+    plain = answer_keys(BACKEND, "s", {"a": q, "b": q})
+    assert plain == {"a": answer_key(BACKEND, "s", q), "b": answer_key(BACKEND, "s", q)}
+    joint = Backend("typesafe", "https://one.invalid", "m", joint_reads=True)
+    keys = answer_keys(joint, "s", {"a": q, "b": q})
+    assert len(set(keys.values())) == 2 and not set(keys.values()) & set(plain.values())
+    assert answer_keys(joint, "s", {"a": q, "b": q}) == keys
+    assert answer_keys(joint, "s", {"b": q, "a": q})["a"] != keys["a"]  # order is part of the read
+    assert answer_keys(joint, "s", {"a": q})["a"] != keys["a"]  # so is company
+
+
 @pytest.mark.parametrize(
     "question,answer",
     [

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/tests/test_settings_and_providers.py

@@ -24,6 +24,9 @@ def test_settings_read_every_convention_once(monkeypatch, tmp_path):
     settings = Settings.from_env()
     assert settings.config_dir == tmp_path / "jev"
     assert (settings.api, settings.model, settings.price_per_mtok) == ("gateway", "pinned", 0.5)
+    assert settings.list_price == 0.5
+    monkeypatch.delenv("JEV_PRICE_PER_MTOK")
+    assert Settings.from_env().price_per_mtok is None and Settings.from_env().list_price == 0.042
     monkeypatch.setenv("JEV_PRICE_PER_MTOK", "lots")
     with pytest.raises(JevFatal, match="JEV_PRICE_PER_MTOK"):
         Settings.from_env()
@@ -93,14 +96,44 @@ def test_gateway_needs_a_complete_endpoint(monkeypatch, tmp_path):
     assert resolve(providers, "gateway").url == "https://override.invalid/v1"
 
 
-def test_a_tool_can_add_its_own_provider(monkeypatch):
+def test_local_servers_are_free_keyless_and_chosen_only_by_name(monkeypatch):
+    providers = catalog("typesafe", "diffusiongemma", "laya")
+    monkeypatch.setenv("JEV_DIFFUSIONGEMMA_API_KEY", "optional")
+    with pytest.raises(JevFatal, match="Set TYPESAFE_API_KEY, or put"):
+        resolve(providers)
+    gemma = resolve(providers, "diffusiongemma")
+    assert (gemma.url, gemma.model, gemma.key, gemma.price_per_mtok, gemma.joint_reads) == (
+        "http://127.0.0.1:8080/v1/systemone",
+        "openjev-latest",
+        "optional",
+        0.0,
+        True,
+    )
+    laya = resolve(providers, "laya")
+    assert (laya.key, laya.price_per_mtok, laya.joint_reads) == ("", 0.0, False)
+    monkeypatch.setenv("JEV_LAYA_URL", "http://gpu-box:8081/v1/systemone")
+    assert resolve(providers, "laya").url == "http://gpu-box:8081/v1/systemone"
+
+
+def test_price_comes_from_the_environment_then_the_provider_then_the_list(monkeypatch):
+    monkeypatch.setenv("TYPESAFE_API_KEY", "key")
+    monkeypatch.setenv("M_KEY", "key")
+    metered = Provider("metered", "https://m.invalid/v1", "m", "M_KEY", price_per_mtok=0.25)
+    providers = catalog("typesafe", "laya", metered)
+    assert resolve(providers, "typesafe").price_per_mtok == 0.042
+    assert resolve(providers, "laya").price_per_mtok == 0.0
+    assert resolve(providers, "metered").price_per_mtok == 0.25
+    monkeypatch.setenv("JEV_PRICE_PER_MTOK", "0.5")
+    assert {resolve(providers, name).price_per_mtok for name in providers} == {0.5}
+
+
+def test_a_tool_can_add_its_own_provider():
     local = Provider(
-        "local", "http://127.0.0.1:8080/v1", "local-v1", "LOCAL_KEY", requires_key=False, auto_select=False
+        "local", "http://127.0.0.1:9000/v1", "local-v1", "LOCAL_KEY", requires_key=False, auto_select=False
     )
     providers = catalog("typesafe", local)
     assert list(providers) == ["typesafe", "local"]
     with pytest.raises(JevFatal, match="no API key"):
         resolve(providers)
-    monkeypatch.setenv("JEV_PRICE_PER_MTOK", "0")
     backend = resolve(providers, "local")
-    assert (backend.key, backend.price_per_mtok, backend.url) == ("", 0.0, "http://127.0.0.1:8080/v1")
+    assert (backend.key, backend.price_per_mtok, backend.url) == ("", 0.042, "http://127.0.0.1:9000/v1")

{jevkit_runtime-0.2.0 → jevkit_runtime-0.3.1}/uv.lock

@@ -139,7 +139,7 @@ wheels = [
 
 [[package]]
 name = "jevkit-runtime"
-version = "0.2.0"
+version = "0.3.1"
 source = { editable = "." }
 dependencies = [
     { name = "httpx" },