rag-wright 0.5.1__tar.gz → 0.6.0__tar.gz

{rag_wright-0.5.1 → rag_wright-0.6.0}/.claude/skills/qwen-vllm-modal/SKILL.md

@@ -48,15 +48,17 @@ different servers (and URLs), for example one per tenant.
 
 **The exact working deploy (this succeeded):**
 ```
-MODAL_IMAGE_BUILDER_VERSION=2025.06 \
+VLLM_API_KEY=<a long random string> MODAL_IMAGE_BUILDER_VERSION=2025.06 \
   APP_NAME=rw-qwen3-modal MODEL=Qwen/Qwen3.8-27B-FP8 KV_CACHE_DTYPE=fp8 \
   SERVED_NAME=Qwen/Qwen3.8-27B TOOL_PARSER=hermes REASONING_PARSER=qwen3 \
   GPU=A100-80GB:1 TP=1 MAX_LEN=16384 GPU_UTIL=0.95 MAX_NUM_SEQS=<target> \
   uv run --no-sync modal deploy scripts/modal_qwen3_vllm_server.py
 ```
 (From a product repo, deploy the copy in this skill's directory: `.claude/skills/qwen-vllm-modal/scripts/modal_qwen3_vllm_server.py` once the skills are linked.)
-Then wire the engine: `.env` `VLLM_BASE_URL=https://<workspace>--rw-qwen3-modal-serve.modal.run/v1`,
-`VLLM_API_KEY=rw-vllm-dev-key`; the profile `qwen3.8-27b-modal` routes there. **Stop billing when done:**
+The script refuses to deploy without `VLLM_API_KEY`: Modal's web endpoint is public and unauthenticated, so vLLM's
+`--api-key` is the only gate on a billing A100, and there is no default key. Then wire the engine: `.env`
+`VLLM_BASE_URL=https://<workspace>--rw-qwen3-modal-serve.modal.run/v1` and the same `VLLM_API_KEY`; the profile
+`qwen3.8-27b-modal` routes there (the seam also refuses an unset key). **Stop billing when done:**
 `uv run --no-sync modal app stop rw-qwen3-modal --yes` (A100 is expensive).
 
 **The engine does NOT reach Modal by default.** Every role's default is `_PRODUCT_LLM = "qwen3.8-27b-modal-or"`,

{rag_wright-0.5.1/src/rag_wright/.agents → rag_wright-0.6.0/.claude}/skills/qwen-vllm-modal/scripts/modal_qwen3_vllm_server.py

@@ -34,7 +34,9 @@ SERVED = os.environ.get("SERVED_NAME", "qwen3-eval")         # served-model-name
 TOOL_PARSER = os.environ.get("TOOL_PARSER", "")             # "" off | "hermes" -> tool calling (the product agent)
 REASONING_PARSER = os.environ.get("REASONING_PARSER", "")   # "" off | "qwen3" -> reasoning coexists with a tool call
 APP_NAME = os.environ.get("APP_NAME", "rw-qwen3-vllm-eval")  # distinct app (=distinct URL) per purpose
-API_KEY = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+API_KEY = os.environ.get("VLLM_API_KEY")  # vLLM's --api-key: the only gate on the server's public URL
+if not API_KEY:  # no default key (PS-21): a default would be public, and would gate a billing GPU
+    raise SystemExit("set VLLM_API_KEY to the server's API key (a long random string) before deploying")
 
 HF_CACHE = "/root/.cache/huggingface"
 VLLM_CACHE = "/root/.cache/vllm"

{rag_wright-0.5.1/src/rag_wright/.agents → rag_wright-0.6.0/.claude}/skills/using-the-rag-wright-engine/SKILL.md

@@ -141,7 +141,11 @@ Work the domain-adaptation guide, grounding each engine call (step 2) and writin
 `server = await adeploy_model_server(ModelServerSpec(name="<app>", max_num_seqs=<concurrency target>))` (the locked
 setup of the `qwen-vllm-modal` skill; needs `rag-wright[modal]`, a Modal login and `VLLM_API_KEY` in `.env`), then
 `EngineConfig(models={"general": server.model_id, ...})`. Other processes reconnect with `amodel_server(spec)`;
-call `await_model_server(server)` before traffic (cold start is minutes); `stop_model_server(name)` stops billing.
+call `await_model_server(server, timeout_s=<startup budget>)` before traffic (cold start is minutes; the default
+waits 25 min); `stop_model_server(name)` stops billing. `amodel_server_status(spec)` looks without waking it (deployed,
+running, healthy, weights cached). There is no default `VLLM_API_KEY`. Count concurrency end to end: `aingest` runs
+`min(document_concurrency, documents) x extract_concurrency` calls per run, parallel runs add up, and the sum must
+stay below `max_num_seqs` (`aingest` warns when its own peak exceeds it).
 Full walk-through: `docs/configuration.md`, "A self-hosted model server".
 
 **Using the decision model (Jev).** Register `jev_decision` (from `engine_capabilities()`) and set

rag_wright-0.6.0/.release-please-manifest.json

@@ -0,0 +1,3 @@
+{
+  ".": "0.6.0"
+}

{rag_wright-0.5.1 → rag_wright-0.6.0}/CHANGELOG.md

@@ -4,6 +4,17 @@ All notable changes to RAG_Wright are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project aims to follow
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.6.0](https://github.com/fzaidi-dai/RAG_Wright/compare/0.5.1...0.6.0) (2026-10-10)
+
+
+### ⚠ BREAKING CHANGES
+
+* **PS-21:** a self-hosted server call or deploy with VLLM_API_KEY (or the profile's api_key_env) unset now raises instead of using the default key "rw-vllm-dev-key".
+
+### Bug Fixes
+
+* **PS-21:** no default model-server key; concurrency ceiling, status and clearer errors (ADR-0131) ([a73a4fd](https://github.com/fzaidi-dai/RAG_Wright/commit/a73a4fd9595a9f8d70d95d393147ba10a1f1718b))
+
 ## [0.5.1](https://github.com/fzaidi-dai/RAG_Wright/compare/0.5.0...0.5.1) (2026-10-10)
 
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/CLAUDE.md

@@ -58,7 +58,7 @@ The failure mode to avoid, which has happened before: a command fails with "modu
 > **CURRENT STATE (2026-10-09): the public-surface workstream is COMPLETE and released as `rag-wright` 0.5.0** (ledger
 > `docs/specs/public-surface/plan.md`, ADR-0128: two public tiers, `rag_wright.api` for a product's seam and
 > `rag_wright.pack_sdk` for a pack's code, enforced by import contracts and a domain-vocabulary guard). Next:
-> RuleWright moves from `docs/specs/public-surface/rulewright-migration-0.3.0.md` then `rulewright-migration-0.4.0.md` and `-0.5.0.md`; PS-R4 (the unit-labelling eval) is done, unreleased. Before it,
+> RuleWright moves from `docs/specs/public-surface/rulewright-migration-0.3.0.md` then `rulewright-migration-0.4.0.md`, `-0.5.0.md` and `-0.6.0.md`; PS-R4 (the unit-labelling eval) is done, unreleased. Before it,
 > the **ingestion-hooks** workstream (ADR-0124) is
 > COMPLETE and released as **`rag-wright` 0.2.0** (PyPI + GitHub tag/release `0.2.0`; release process
 > `docs/releasing.md`): the generic `build_ingestion` with hooks and domain-neutral defaults, the neutral default

{rag_wright-0.5.1 → rag_wright-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: rag-wright
-Version: 0.5.1
+Version: 0.6.0
 Summary: A domain-retargetable, open-core engine for hybrid retrieval + knowledge-graph RAG: parsing, chunking, embedding, hybrid search, reranking, graph extraction, entity resolution, ontology-driven knowledge, and a capability runtime.
 Project-URL: Homepage, https://github.com/fzaidi-dai/RAG_Wright
 Project-URL: Repository, https://github.com/fzaidi-dai/RAG_Wright

rag_wright-0.6.0/docs/adr/0131-model-server-hardening-after-rulewright-review.md

@@ -0,0 +1,34 @@
+# ADR-0131: No default server key anywhere; a model's concurrency ceiling travels with its id
+
+**Status:** accepted · **Date:** 2026-10-11 · **Related:** ADR-0130 (deploy the model server from the engine),
+ADR-0110 (`max_num_seqs` is the concurrency target), PS-21 (RuleWright's model-server review)
+
+## Context
+
+RuleWright's first use of the model-server API found five gaps. The deploy script (and six dev scripts) fell back to
+a key committed in the repo when `VLLM_API_KEY` was unset, and Modal's web endpoint is public and unauthenticated, so
+the raw `modal deploy` path the skill documents put a billing A100 behind a published key; the seam sent the same
+default. A product could not tell whether its load fits the server, because the engine's own ingestion fan-out
+multiplies with the product's and nothing compared it to `max_num_seqs`. Cache state, a not-deployed error and the
+startup timeout were also unclear.
+
+## Decision
+
+1. No default key. Every Modal vLLM server script raises on an unset `VLLM_API_KEY` (and bakes it into the image
+   env, which the container reads); the seam's `vllm` backend raises instead of sending a default. A test fails if
+   the old default string reappears under `src/`, `scripts/` or `docs/*.md`. Breaking, approved.
+2. `ModelProfile.max_concurrency` (optional; approved schema change): the most requests the model's server batches
+   at once. A deployed server registers its `max_num_seqs` there. `aingest` warns (a `RuntimeWarning` and a progress
+   line) when `min(document_concurrency, documents) x extract_concurrency` exceeds the ceiling of any model the
+   workspace resolves. It cannot see a product's other parallel runs; the docs give the arithmetic.
+3. `amodel_server_status(spec)`: deployed, running, healthy, weights cached. It asks `/health` only of a running
+   container (a request to a scaled-down server starts a cold start) and reads the weights Volume (`rw-hf-cache`,
+   pinned to the script by a test).
+4. `ModelServerNotDeployed` (a `RuntimeError`) replaces Modal's raw `NotFoundError`; `astop_model_server` is the async
+   stop.
+
+## Consequences
+
+- A hand deploy or a client without `VLLM_API_KEY` now fails with a message naming the variable.
+- The ingestion warning covers the generic pipeline's extractor fan-out; other callers (a pack's own concurrency,
+  queries) are covered by the documented arithmetic, not a check.

{rag_wright-0.5.1 → rag_wright-0.6.0}/docs/adr/README.md

@@ -1,7 +1,7 @@
 # Architecture Decision Records — index
 
-128 ADR files (0001–0119, plus 0120 — the former duplicate `0015`, renumbered 2026-10-05 to repair the collision —
-and 0121–0130). ADRs are an
+129 ADR files (0001–0119, plus 0120 — the former duplicate `0015`, renumbered 2026-10-05 to repair the collision —
+and 0121–0131). ADRs are an
 immutable, cross-referenced decision log: nothing here is moved or renumbered — legacy records carry a `Status:`
 banner pointing to what replaced them. This index groups them so a newcomer isn't drowned.
 
@@ -186,4 +186,5 @@ framing that still appears in older ADRs and the root `SPEC.md`/`plan.md`.
 0127 Retire the OKF code ·
 0128 Two public tiers: `rag_wright.api` for products, `rag_wright.pack_sdk` for packs ·
 0129 The shipped Laya temperatures stay as fitted; scikit-learn floor at 1.9.1 ·
-0130 A product deploys the self-hosted model server from the engine and uses it like any model
+0130 A product deploys the self-hosted model server from the engine and uses it like any model ·
+0131 No default server key anywhere; a model's concurrency ceiling travels with its id

{rag_wright-0.5.1 → rag_wright-0.6.0}/docs/api/README.md

@@ -40,6 +40,14 @@ The deploy command's settings, as the skill's documented command passes them.
 
 A deployed server: `model_id` is the model id to use (in `EngineConfig.models` or a `model=` argument); `base_url` is its OpenAI-compatible endpoint; `root` is its URL (for `/health`).
 
+### `ModelServerNotDeployed(name: 'str')`
+
+No deployed model server has this name (never deployed, or stopped).
+
+### `ModelServerStatus(name: 'str', deployed: 'bool', running: 'bool', healthy: 'bool', weights_cached: 'bool') -> None`
+
+What `amodel_server_status` found. `deployed`: the Modal app exists. `running`: a container is up (the GPU is billing); False when scaled to zero after idling. `healthy`: a running container answers `/health` (False while it starts). `weights_cached`: the spec's model weights are in the weights Volume, so a deploy or cold start skips the ~27 GB download (an empty `modal app list` says nothing about this).
+
 ### `ModelRole(*values)`
 
 Which model does which job. The mapping to ids lives in config, not in capability code.
@@ -256,7 +264,7 @@ Deploy (or redeploy) the server described by `spec` on Modal, register its model
 
 ### `amodel_server(spec: 'ModelServerSpec') -> 'ModelServer'`
 
-Reconnect to a server deployed earlier (by this process or another) without redeploying: look up its URL and register its model profile. `spec` must match the deployed server (its name and served model).
+Reconnect to a server deployed earlier (by this process or another) without redeploying: look up its URL and register its model profile. `spec` must match the deployed server (its name and served model). Raises `ModelServerNotDeployed` when no server has that name.
 
 ### `await_model_server(server: 'ModelServer', *, timeout_s: 'float' = 1500, poll_s: 'float' = 15, progress: 'Callable[[str], object]' = print) -> 'ModelServer'`
 
@@ -266,6 +274,14 @@ Wait until the server answers `/health` (a cold start is about 8 minutes, ADR-01
 
 Stop the Modal app `name` (stops billing); its URL stops answering.
 
+### `amodel_server_status(spec: 'ModelServerSpec') -> 'ModelServerStatus'`
+
+Look, without deploying or waking anything: is the server deployed, is a container running (billing), does it answer `/health`, and are its model weights cached. `/health` is asked only of a running container: a request to a scaled-down server would start a cold start.
+
+### `astop_model_server(name: 'str') -> 'None'`
+
+`stop_model_server` for async code.
+
 ### `ainvoke_subgraph(name: 'str', inputs: 'dict', *, resources: 'WorkspaceHandle') -> 'Any'`
 
 Invoke a subgraph-kind capability by name over the workspace, inside a trace span. The implementation is resolved lazily from the manifest `impl_ref` (no central adapter dict). Retry/dead-letter comes from the LangGraph scaffold the subgraph is built on; usage is captured by the caller's `measure_usage()` (EP-API-5).

{rag_wright-0.5.1 → rag_wright-0.6.0}/docs/configuration.md

@@ -130,7 +130,9 @@ uv run modal token new          # log in to Modal once per machine
 ```
 
 and put the server's API key in your `.env` (any secret string; the server is deployed with it and every call
-sends it): `VLLM_API_KEY=<a long random string>`. There is no default key.
+sends it): `VLLM_API_KEY=<a long random string>`. There is no default key: Modal's web endpoint is public, so this key
+is the only thing between the internet and a billing A100. Deploying (through the engine or the skill's raw
+`modal deploy` command) and calling the server both refuse to run without it.
 
 **Deploy, then use it like any model.**
 
@@ -159,9 +161,37 @@ without redeploying, which only looks up the server and registers its model id:
 from rag_wright.api import amodel_server, await_model_server
 
 server = await amodel_server(ModelServerSpec(name="acme-qwen", max_num_seqs=32))   # same spec as the deploy
-await await_model_server(server)    # returns at once if it is up; waits through a cold start if it scaled down
+await await_model_server(server, timeout_s=600)   # at once if it is up; through a cold start if it scaled down
 ```
 
+`amodel_server` raises `ModelServerNotDeployed` (a `RuntimeError`) when no server has that name. Choose
+`await_model_server`'s `timeout_s` for your startup: the default (1500 s) covers the worst cold start, but it also
+holds a service's startup that long when the server is down. Pick what your orchestrator tolerates, and on
+`TimeoutError` either fail the readiness check (the orchestrator retries) or start without the server and fall back
+to a provider-served model id for those roles.
+
+**Looking without waking it.** `await amodel_server_status(spec)` reports `deployed`, `running` (a container is up,
+so the GPU is billing), `healthy` and `weights_cached`. It asks `/health` only of a running container, because any
+request to a scaled-down server starts a cold start. `weights_cached` is what decides a deploy's cost: the weights
+live in the Modal Volume `rw-hf-cache` (the vLLM compile cache in `rw-vllm-serve-compile-cache`), and stay there when
+the app is stopped, so an empty `modal app list` does not mean a cold cache. With the weights cached, a deploy is a
+short image build and a cold start; without them, it also downloads about 27 GB. `astop_model_server` is
+`stop_model_server` for async code.
+
+**Concurrency: the engine's fan-out multiplies with yours.** `max_num_seqs` is the most requests the server batches
+at once; requests above it queue, and under load queued reasoning calls time out and retry. Count everything that
+can call the server at the same moment:
+
+- `build_ingestion`'s `aingest` runs up to `min(document_concurrency, documents) x extract_concurrency` extractor
+  calls at once (`IngestionTuning`, defaults 2 x 8 = 16), per `aingest` call. Two `aingest` runs in parallel are 32.
+- The reference contracts pack runs `CLAUSE_CONCURRENCY` (default 8) extractions per document (Jev calls by default;
+  Qwen calls only when Qwen is its extraction model), capped process-wide by `RAG_EXTRACT_WORKERS` (default 32).
+- Plus your own concurrent queries and answers.
+
+The server's profile carries its ceiling (`max_concurrency`, set from `max_num_seqs`), and `aingest` warns before it
+starts (a `RuntimeWarning` and an `[ingest] warning:` progress line) when its own peak can exceed the ceiling of a
+model the workspace resolves. It cannot see your other parallel runs: keep their sum below `max_num_seqs`.
+
 **What to know before you rely on it.**
 
 - **Cold start.** A new or scaled-down server takes minutes to become ready (226 s in the last live run; up to about
@@ -169,16 +199,15 @@ await await_model_server(server)    # returns at once if it is up; waits through
   deployed; the next request waits through a cold start, so call `await_model_server` before sending traffic.
 - **Cost.** An A100 bills while a container runs. `stop_model_server("acme-qwen")` removes the app entirely; deploy
   again to bring it back.
-- **Concurrency.** `max_num_seqs` is how many requests the server batches at once (ADR-0110 sets it to your real
-  concurrency target); keep your own client concurrency below it.
 - **Timeouts.** Reasoning-on structured calls under full load can take about a minute; raise
   `RAG_STRUCTURED_TIMEOUT_S` (default 60) for bulk work, or calls time out and retry.
 - **Process-wide.** A registered server model id is known to the whole process, like every model profile; after a
   restart, call `amodel_server` again.
 
-**A runnable example.** `examples/model_server.py` does all of this through `rag_wright.api`: `deploy` brings a
-server up, asks one question on it through a workspace, checks the call was metered on the server's model id, then
-stops it and confirms it is gone; `reconnect` is the service-process path (look up, wait, ask, leave it running).
+**A runnable example.** `examples/model_server.py` does all of this through `rag_wright.api`: `deploy` reports the
+status, brings a server up, asks one question on it through a workspace, checks the call was metered on the
+server's model id, then stops it and confirms it is gone; `reconnect` is the service-process path (look up, wait
+with a startup timeout, ask, leave it running).
 
 The `qwen-vllm-modal` skill has the operational detail (the image fixes, serving gotchas, the manual `modal deploy`
 command) and ADR-0130 the design.

{rag_wright-0.5.1 → rag_wright-0.6.0}/docs/specs/public-surface/plan.md

@@ -94,3 +94,4 @@ in two at the PS-6 gate so the store rewrite is its own visible task. One task,
 | PS-18 | RuleWright's request of 2026-10-10: raise the floor to `scikit-learn>=1.9.1` (8 shipped heads were pickled with 1.9.1) with a guard test that loads every shipped head with `InconsistentVersionWarning` as an error; the Laya temperatures (9 of 15 above Laya 0.3.22's [0.5, 5]) left as they are with the cause and the measured effect written down (ADR): fitted by Laya's training script within [0.1, 10], clamped at serving; labels unaffected (the fleet uses the ranking, not the probability; 5,400 paired predictions: 0 abstain changes, 45 rounding-tie differences); a reply to RuleWright | guard test; full suite | done: `scikit-learn>=1.9.1` (lock 1.9.0 -> 1.9.1); guard loads the 8 heads in the shipped model folders with the version warning as an error; ADR-0129; reference-pack note; the `laya` skill's post-training temperature checklist + scikit-learn lesson; reply to RuleWright; upstream Laya report drafted at `docs/upstream/laya-temperature-range-mismatch.md` (not filed) |
 | PS-19 | A product deploys the self-hosted Qwen server from the engine and uses it like any model: `ModelServerSpec` (defaults = the `qwen-vllm-modal` skill / ADR-0110 locked Config B: Qwen3.8-27B-FP8 + fp8 KV, 1x A100-80GB, TP 1, 16K, 0.95, served name `Qwen/Qwen3.8-27B`, hermes + qwen3 parsers, image builder 2025.06), `adeploy_model_server` (deploys the shipped script, waits for `/health` with X/N progress), reconnect by name, status, stop; each server registered as a model profile with its own endpoint (no `VLLM_BASE_URL` needed), so its `model_id` goes in `EngineConfig.models` or any `model=`; the API key from a named environment variable, never a default; `rag-wright[modal]` extra; Modal grounded in the framework graph | hermetic tests (Modal and HTTP mocked); live: deploy, one call through a workspace, stop, confirm stopped | done: `ModelServerSpec` (Config B defaults, `max_num_seqs` required), `adeploy_model_server` / `amodel_server` / `await_model_server` / `stop_model_server`; profiles carry an explicit `base_url`, `register_model_profile`; `rag-wright[modal]`; ADR-0130; configuration.md walk-through linked from the engine-usage skill, domain-adaptation guide, workspaces, templates; RuleWright 0.5 guide; live: ready in 226 s, one call metered on the server's model id, stopped (239 s of A100). Follow-up: `examples/model_server.py` (`deploy` and `reconnect` modes, public API only) for RuleWright and other products, and the URL lookup moved to Modal's async interface (no `AsyncUsageWarning`); live: all three steps passed (ready after a 254 s cold start, reconnect ready in 1 s, each call metered on the server's model id, stopped, no longer resolves) |
 | PS-20 | Cold start of the self-hosted Qwen server (~8 min, CPU-bound engine init + CUDA-graph capture; ADR-0109): GPU snapshot / sleep mode, measured against the current baseline. Later, by your decision | live cold-start measurement | todo (later) |
+| PS-21 | RuleWright's model-server review (2026-10-11): (1) no default API key anywhere: every Modal vLLM server script raises on an unset `VLLM_API_KEY`, the seam's `vllm` backend raises instead of sending a default; (2) the concurrency ceiling made computable: `ModelProfile.max_concurrency` (approved schema change), set from `max_num_seqs` on registration, and `aingest` warns when `document_concurrency x extract_concurrency` can exceed a resolved model's ceiling, with the arithmetic documented; (3) `amodel_server_status` (deployed, healthy, weights cached) and the Volumes documented; (4) `ModelServerNotDeployed` instead of Modal's raw error; (5) startup timeout guidance; `astop_model_server`. Breaking (approved): an unset key now fails | hermetic tests; live: deploy, status, one call, stop, status, reconnect error | done: all five points and `astop_model_server`; 7 server scripts refuse an unset key (two Granite scripts also never baked the key into the container, fixed); status asks `/health` only of a running container; ADR-0131; RuleWright 0.6 guide; live: status before (weights cached), ready after 177 s, one call metered on the server id, stopped, status after (not deployed), reconnect gives the `ModelServerNotDeployed` message |

rag_wright-0.6.0/docs/specs/public-surface/rulewright-migration-0.6.0.md

@@ -0,0 +1,43 @@
+# RuleWright: moving to rag-wright 0.6 (the model-server review)
+
+The engine's answer to RuleWright's model-server review (2026-10-11), and what changes for RuleWright. Do the 0.3,
+0.4 and 0.5 guides first. Nothing here is edited from the engine repo. The design is ADR-0131.
+
+**Short version.** One break: nothing defaults `VLLM_API_KEY` any more. Everything else is new API and docs that
+answer the five points raised.
+
+## 1. The one break: no default server key
+
+- The deploy script (the skill's raw `modal deploy` path too) refuses to run without `VLLM_API_KEY`, and so does the
+  model seam's `vllm` backend: a call to a self-hosted server with the variable unset raises
+  `ValueError("set VLLM_API_KEY ...")` instead of sending `rw-vllm-dev-key`. A profile's own `api_key_env` is
+  required the same way.
+- RuleWright already deploys with a real key, so the only thing to check is that every process that calls the
+  server (services, workers, scripts, tests that reach a live server) has `VLLM_API_KEY` set. Hermetic tests that set
+  `RAG_SERVING=vllm` need a dummy key in the environment.
+- RuleWright's product-side guards stay valid; the engine's own documented path is now guarded too (point 1 of the
+  review).
+
+## 2. The answers to the review
+
+| Review point | What the engine did |
+|---|---|
+| 1. Published default key | Removed everywhere (above), with a test that fails if it returns. |
+| 2. Concurrency ceiling not computable | `docs/configuration.md` "Concurrency: the engine's fan-out multiplies with yours" gives the arithmetic. A server's profile now carries its ceiling (`ModelProfile.max_concurrency`, from `max_num_seqs`), and `aingest` warns before it starts when its own peak, `min(document_concurrency, documents) x extract_concurrency`, can exceed it. Correction to the review: `RAG_EXTRACT_WORKERS` is a process-wide cap of 32 threads on the reference pack's docling-graph extraction, not a per-document fan-out; the per-document knob there is `CLAUSE_CONCURRENCY` (8). The warning cannot see RuleWright's own parallel runs, so their sum is still yours to keep below `max_num_seqs`. |
+| 3. Cache state invisible | `await amodel_server_status(spec)` returns `deployed`, `running` (a container is up, billing), `healthy` and `weights_cached`. It never wakes a scaled-down server (it asks `/health` only of a running container). The Volumes (`rw-hf-cache` for weights, `rw-vllm-serve-compile-cache`) are documented. |
+| 4. Modal's error leaks | `amodel_server` raises `ModelServerNotDeployed` (a `RuntimeError`) naming the server and saying to deploy it. |
+| 5. 25-minute startup block | Docs on choosing `await_model_server(..., timeout_s=...)` for startup and what to do on `TimeoutError` (fail readiness, or start on a provider-served model id). The example uses 600 s. |
+| `stop_model_server` is sync | `astop_model_server` added; the sync one stays. |
+
+## 3. What to change in RuleWright (optional except section 1)
+
+- Replace the `asyncio.to_thread(stop_model_server, ...)` workaround with `await astop_model_server(name)`.
+- Catch `ModelServerNotDeployed` at worker startup instead of Modal's `NotFoundError`.
+- Give the worker's `await_model_server` a startup-sized `timeout_s`.
+- Use `amodel_server_status` in the pre-flight instead of reading `modal app list`.
+
+## 4. Verify
+
+The engine's `examples/model_server.py` (read it from the engine checkout) now prints the status before deploying
+and confirms the server is gone with `amodel_server_status` after stopping; `reconnect` fails with the
+`ModelServerNotDeployed` message when nothing is deployed.

{rag_wright-0.5.1 → rag_wright-0.6.0}/examples/model_server.py

@@ -9,7 +9,7 @@ Prerequisites (see docs/installation.md):
   - ArcadeDB running locally, with ARCADEDB_* set in .env.
 
 Run from the repo root:
-    uv run python examples/model_server.py deploy      # deploy, ask one question on it, stop it, confirm it is gone
+    uv run python examples/model_server.py deploy      # status, deploy, ask one question, stop, confirm it is gone
     uv run python examples/model_server.py reconnect   # reconnect to a server already deployed, ask one question
 
 `deploy` is what an operator script or startup job does, and it always stops the server at the end (an A100 bills
@@ -30,11 +30,13 @@ from rag_wright.api import (
     EvidenceItem,
     ModelRole,
     ModelServer,
+    ModelServerNotDeployed,
     ModelServerSpec,
     StoreConfig,
     adeploy_model_server,
     agenerate_answer,
     amodel_server,
+    amodel_server_status,
     await_model_server,
     measure_usage,
     open_workspace,
@@ -62,6 +64,9 @@ async def ask_one_question(server: ModelServer) -> None:
 
 
 async def deploy() -> None:
+    before = await amodel_server_status(SPEC)  # looks only: deploys nothing, wakes nothing
+    print(f"before: deployed={before.deployed} running={before.running} weights_cached={before.weights_cached} "
+          f"({'a short image build and a cold start' if before.weights_cached else 'plus a ~27 GB weight download'})")
     start = time.monotonic()
     try:
         server = await adeploy_model_server(SPEC, poll_s=20)  # deploys, waits for /health, registers the model id
@@ -69,17 +74,22 @@ async def deploy() -> None:
         await ask_one_question(server)
     finally:
         stop_model_server(SPEC.name)  # always, even if something above failed
-    try:
-        await amodel_server(SPEC)
-    except Exception as exc:  # a stopped app no longer resolves
-        print(f"stopped: {SPEC.name} no longer resolves ({type(exc).__name__}); {time.monotonic() - start:.0f}s total")
-    else:
-        raise SystemExit(f"{SPEC.name} still resolves after stop")
+    after = await amodel_server_status(SPEC)
+    if after.deployed:
+        raise SystemExit(f"{SPEC.name} is still deployed after stop")
+    print(f"stopped: {SPEC.name} is no longer deployed (weights still cached: {after.weights_cached}); "
+          f"{time.monotonic() - start:.0f}s total")
 
 
 async def reconnect() -> None:
-    server = await amodel_server(SPEC)  # looks the server up and registers its model id; does not deploy
-    await await_model_server(server, poll_s=20)  # at once if it is up; through a cold start if it scaled down
+    try:
+        server = await amodel_server(SPEC)  # looks the server up and registers its model id; does not deploy
+    except ModelServerNotDeployed as exc:
+        raise SystemExit(str(exc))
+    try:  # a startup-sized timeout: on expiry, fail readiness (or fall back to a provider-served model id)
+        await await_model_server(server, timeout_s=600, poll_s=20)
+    except TimeoutError as exc:
+        raise SystemExit(f"{exc}; failing startup")
     await ask_one_question(server)
 
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "rag-wright"
-version = "0.5.1"
+version = "0.6.0"
 description = "A domain-retargetable, open-core engine for hybrid retrieval + knowledge-graph RAG: parsing, chunking, embedding, hybrid search, reranking, graph extraction, entity resolution, ontology-driven knowledge, and a capability runtime."
 readme = "README.md"
 requires-python = ">=3.12,<3.13"

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/compliance_engine_smoke.py

@@ -11,7 +11,7 @@ Capped to a few §255.5 disclosure requirements to sidestep the CC-6 cross-produ
   # on Modal (A100 Granite judge + extraction, Modal compliance KG) -- MODAL-COMPLIANCE:
   ARCADEDB_HOST=farhan-zaidi--rw-arcadedb-serve.modal.run ARCADEDB_PORT=443 ARCADEDB_PROTOCOL=https \
     ARCADEDB_USER=root ARCADEDB_PASSWORD=rag_wright_dev_2026 \
-    RAG_SERVING=vllm VLLM_BASE_URL=<a100>/v1 VLLM_API_KEY=rw-vllm-dev-key \
+    RAG_SERVING=vllm VLLM_BASE_URL=<a100>/v1 VLLM_API_KEY=<the server key> \
     uv run --no-sync python -m scripts.compliance_engine_smoke
 """
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/eval_compliance_gold.py

@@ -15,7 +15,7 @@ runs local or fully on Modal (KG on the rw-arcadedb Volume + Granite/BGE on the
   # fully on Modal (A100 Granite + A100 BGE + Modal compliance KG):
   ARCADEDB_HOST=farhan-zaidi--rw-arcadedb-serve.modal.run ARCADEDB_PORT=443 ARCADEDB_PROTOCOL=https \
     ARCADEDB_USER=root ARCADEDB_PASSWORD=rag_wright_dev_2026 \
-    RAG_SERVING=vllm VLLM_BASE_URL=<a100>/v1 VLLM_API_KEY=rw-vllm-dev-key STACK_URL=<a100> \
+    RAG_SERVING=vllm VLLM_BASE_URL=<a100>/v1 VLLM_API_KEY=<the server key> STACK_URL=<a100> \
     uv run --no-sync python -m scripts.eval_compliance_gold
 """
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/granite_chunker_assess.py

@@ -67,7 +67,7 @@ def main() -> None:
     )
 
     vbase = os.environ["VLLM_URL"].rstrip("/") + "/v1"
-    vkey = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+    vkey = os.environ["VLLM_API_KEY"]
     n_docs = int(os.environ.get("DOCS", "4"))
     runs = int(os.environ.get("RUNS", "3"))
     model_id = "ibm-granite/granite-4.1-8b"

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/measure_generation_robustness.py

@@ -15,7 +15,7 @@ Metrics per strategy, over answerable queries where evidence is non-empty (so an
 Run on the A100 (streams X/N progress):
   ARCADEDB_HOST=<rw-arcadedb>.modal.run ARCADEDB_PORT=443 ARCADEDB_PROTOCOL=https ARCADEDB_USER=root \
     ARCADEDB_PASSWORD=rag_wright_dev_2026 ARCADEDB_DATABASE=ragwright_cuad_full \
-    RAG_SERVING=vllm VLLM_BASE_URL=<a100>/v1 VLLM_API_KEY=rw-vllm-dev-key STACK_URL=<a100> \
+    RAG_SERVING=vllm VLLM_BASE_URL=<a100>/v1 VLLM_API_KEY=<the server key> STACK_URL=<a100> \
     uv run --no-sync python -m scripts.measure_generation_robustness [REPEATS] [N_SAMPLES]
 """
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/modal_gemma4_vllm.py

@@ -7,7 +7,7 @@ model config (no --quantization flag for the 4-bit dense model).
 
   MODEL=google/gemma-4-31B-it-qat-w4a16-ct  uv run --no-sync modal deploy scripts/modal_gemma4_vllm.py
   # then, self-hosted silver run:
-  #   RAG_SERVING=vllm VLLM_BASE_URL=https://<app>.modal.run/v1 VLLM_API_KEY=rw-vllm-dev-key \
+  #   RAG_SERVING=vllm VLLM_BASE_URL=https://<app>.modal.run/v1 VLLM_API_KEY=<the server key> \
   #     SILVER_MODELS="gemma4-31b-qat:google/gemma-4-31B-it-qat-w4a16-ct" \
   #     uv run --no-sync python -m scripts.measure_silver 3
   uv run --no-sync modal app stop rw-gemma4
@@ -24,7 +24,9 @@ import modal
 MODEL = os.environ.get("MODEL", "google/gemma-4-31B-it-qat-w4a16-ct")
 QUANT = os.environ.get("QUANT", "")  # empty for the W4A16 compressed-tensors dense model (auto-detected);
 # for the 26B-A4B MoE (no 4-bit checkpoint) set QUANT=int8_per_channel_weight_only (Google's vLLM rec).
-API_KEY = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+API_KEY = os.environ.get("VLLM_API_KEY")  # vLLM's --api-key: the only gate on the server's public URL
+if not API_KEY:  # no default key (PS-21): a default would be public, and would gate a billing GPU
+    raise SystemExit("set VLLM_API_KEY to the server's API key (a long random string) before deploying")
 GPU = os.environ.get("GPU", "A100-40GB")
 GPU_UTIL = os.environ.get("VLLM_GPU_UTIL", "0.80")  # LEAVE HEADROOM: json_schema guided decoding (xgrammar)
 # needs GPU workspace beyond weights+KV; at 0.90 a guided request OOM-killed the engine. 0.80 keeps room.

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/modal_gemma4_vllm_snapshot.py

@@ -33,7 +33,9 @@ import modal
 
 MODEL = os.environ.get("MODEL", "google/gemma-4-31B-it-qat-w4a16-ct")
 QUANT = os.environ.get("QUANT", "")  # e.g. int8_per_channel_weight_only for the 26B-A4B MoE
-API_KEY = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+API_KEY = os.environ.get("VLLM_API_KEY")  # vLLM's --api-key: the only gate on the server's public URL
+if not API_KEY:  # no default key (PS-21): a default would be public, and would gate a billing GPU
+    raise SystemExit("set VLLM_API_KEY to the server's API key (a long random string) before deploying")
 GPU = os.environ.get("GPU", "A100-40GB")
 GPU_UTIL = os.environ.get("VLLM_GPU_UTIL", "0.80")
 MAX_LEN = os.environ.get("VLLM_MAX_LEN", "16384")

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/modal_granite_vllm_server.py

@@ -16,7 +16,9 @@ import modal
 
 MODEL_HF = os.environ.get("GRANITE_HF", "ibm-granite/granite-4.1-8b")
 GPU = os.environ.get("GPU", "A100-40GB")
-API_KEY = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")  # vLLM's --api-key; clients send it as the bearer
+API_KEY = os.environ.get("VLLM_API_KEY")  # vLLM's --api-key: the only gate on the server's public URL
+if not API_KEY:  # no default key (PS-21): a default would be public, and would gate a billing GPU
+    raise SystemExit("set VLLM_API_KEY to the server's API key (a long random string) before deploying")
 HF_CACHE = "/root/.cache/huggingface"
 
 app = modal.App("rw-granite-vllm")
@@ -24,7 +26,7 @@ hf_vol = modal.Volume.from_name("rw-hf-cache", create_if_missing=True)
 vllm_image = (
     modal.Image.from_registry("nvidia/cuda:12.8.1-devel-ubuntu22.04", add_python="3.12")
     .pip_install("vllm", "huggingface_hub[hf_transfer]")
-    .env({"HF_HUB_ENABLE_HF_TRANSFER": "1", "HF_HOME": HF_CACHE})
+    .env({"HF_HUB_ENABLE_HF_TRANSFER": "1", "HF_HOME": HF_CACHE, "VLLM_API_KEY": API_KEY})  # the container reads its own env
 )
 
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/modal_qwen3_27b_bench.py

@@ -35,7 +35,9 @@ KV_DTYPE = os.environ.get("KV_CACHE_DTYPE", "auto")      # "auto" (unquantized)
 N = int(os.environ.get("N", "48"))                       # requests per concurrency level (small -> cheap, meaningful)
 MAX_TOK = int(os.environ.get("MAX_TOK", "256"))          # output tokens/request (a clause-property JSON size)
 CSWEEP = [int(x) for x in os.environ.get("CSWEEP", "1,4,8,16,32,64").split(",")]
-API_KEY = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+API_KEY = os.environ.get("VLLM_API_KEY")  # vLLM's --api-key: the only gate on the server's public URL
+if not API_KEY:  # no default key (PS-21): a default would be public, and would gate a billing GPU
+    raise SystemExit("set VLLM_API_KEY to the server's API key (a long random string) before deploying")
 
 app = modal.App("rw-qwen3-27b-bench")
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/modal_qwen3_27b_snapshot.py

@@ -23,7 +23,9 @@ import modal
 MODEL = os.environ.get("MODEL", "Qwen/Qwen3.8-27B")
 GPU = os.environ.get("GPU", "A100-80GB:2")
 TP = os.environ.get("TP", "2")
-API_KEY = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+API_KEY = os.environ.get("VLLM_API_KEY")  # vLLM's --api-key: the only gate on the server's public URL
+if not API_KEY:  # no default key (PS-21): a default would be public, and would gate a billing GPU
+    raise SystemExit("set VLLM_API_KEY to the server's API key (a long random string) before deploying")
 GPU_UTIL = os.environ.get("VLLM_GPU_UTIL", "0.90")
 MAX_LEN = os.environ.get("VLLM_MAX_LEN", "16384")
 MAX_NUM_SEQS = os.environ.get("MAX_NUM_SEQS", "256")

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/modal_stack_a100.py

@@ -23,7 +23,9 @@ import subprocess
 import modal
 
 MODEL_HF = os.environ.get("GRANITE_HF", "ibm-granite/granite-4.1-8b")
-API_KEY = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+API_KEY = os.environ.get("VLLM_API_KEY")  # vLLM's --api-key: the only gate on the server's public URL
+if not API_KEY:  # no default key (PS-21): a default would be public, and would gate a billing GPU
+    raise SystemExit("set VLLM_API_KEY to the server's API key (a long random string) before deploying")
 GPU_UTIL = os.environ.get("VLLM_GPU_UTIL", "0.75")  # leave ~10GB of the 40GB for BGE (~2.3GB) + LegalBERT (~0.5GB)
 HF_CACHE = "/root/.cache/huggingface"
 LEGALBERT_PATH = "/models/legalbert_function"
@@ -34,7 +36,7 @@ models_vol = modal.Volume.from_name("rw-models", create_if_missing=True)
 image = (
     modal.Image.from_registry("nvidia/cuda:12.8.1-devel-ubuntu22.04", add_python="3.12")
     .pip_install("vllm", "FlagEmbedding", "transformers", "fastapi", "httpx", "huggingface_hub[hf_transfer]")
-    .env({"HF_HUB_ENABLE_HF_TRANSFER": "1", "HF_HOME": HF_CACHE})
+    .env({"HF_HUB_ENABLE_HF_TRANSFER": "1", "HF_HOME": HF_CACHE, "VLLM_API_KEY": API_KEY})  # the container reads its own env
 )
 
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/phase_a_leg_validate.py

@@ -7,7 +7,7 @@ the STACK_URL adapters).
 
   ARCADEDB_HOST=<rw-arcadedb>.modal.run ARCADEDB_PORT=443 ARCADEDB_PROTOCOL=https \
     ARCADEDB_USER=root ARCADEDB_PASSWORD=rag_wright_dev_2026 ARCADEDB_DATABASE=ragwright_cuad_full \
-    RAG_SERVING=vllm VLLM_BASE_URL=<a100>/v1 VLLM_API_KEY=rw-vllm-dev-key STACK_URL=<a100> \
+    RAG_SERVING=vllm VLLM_BASE_URL=<a100>/v1 VLLM_API_KEY=<the server key> STACK_URL=<a100> \
     uv run --no-sync python -m scripts.phase_a_leg_validate A
 
 Arg selects the leg(s): A (intra_document_qa), Crel (relational_qa, graph-structural), B

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/semantic_judge_live_validate.py

@@ -50,7 +50,7 @@ def main() -> None:
     from rag_wright.util.concurrent import map_concurrent
 
     vbase = os.environ["VLLM_URL"].rstrip("/") + "/v1"
-    vkey = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+    vkey = os.environ["VLLM_API_KEY"]
     n = int(os.environ.get("N", "12"))
     model_id = "ibm-granite/granite-4.1-8b"
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/semantic_judge_probe.py

@@ -48,7 +48,7 @@ def main() -> None:
     from rag_wright.packs.contracts.spans.semantic_judge import build_semantic_judge_fn
 
     vbase = os.environ["VLLM_URL"].rstrip("/") + "/v1"
-    vkey = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+    vkey = os.environ["VLLM_API_KEY"]
     method = os.environ.get("METHOD", "json_schema")
     model_id = "ibm-granite/granite-4.1-8b"
     print(f"[probe] METHOD={method} ; warming {model_id}", flush=True)

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/stack_correctness_validate.py

@@ -23,7 +23,7 @@ from dotenv import load_dotenv
 def _post(url: str, payload: dict, timeout: int = 60) -> dict:
     req = urllib.request.Request(url, data=json.dumps(payload).encode(), method="POST",
                                  headers={"Content-Type": "application/json",
-                                          "Authorization": "Bearer rw-vllm-dev-key"})
+                                          "Authorization": f"Bearer {os.environ['VLLM_API_KEY']}"})
     with urllib.request.urlopen(req, timeout=timeout) as r:
         return json.loads(r.read())
 

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/vllm_extraction_ab.py

@@ -38,7 +38,7 @@ def main() -> None:
     from rag_wright.util.concurrent import map_concurrent
 
     vllm_base = os.environ["VLLM_URL"].rstrip("/") + "/v1"
-    key = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+    key = os.environ["VLLM_API_KEY"]
     n = int(os.environ.get("N", "30"))
     temp = float(os.environ.get("TEMP", "0"))  # pin to 0 (greedy) to isolate model diff from sampling noise
     mode = os.environ.get("MODE", "vllm")  # "vllm" = vLLM-vs-OpenRouter; "control" = OpenRouter-vs-OpenRouter

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/vllm_kg_query_validate.py

@@ -36,7 +36,7 @@ def main() -> None:
     from rag_wright.util.concurrent import map_concurrent
 
     vbase = os.environ["VLLM_URL"].rstrip("/") + "/v1"
-    vkey = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+    vkey = os.environ["VLLM_API_KEY"]
     n = int(os.environ.get("N", "15"))
 
     # a DIVERSE sample: distinct functions so the queries span the property space

{rag_wright-0.5.1 → rag_wright-0.6.0}/scripts/vllm_raw_diagnostic.py

@@ -60,7 +60,7 @@ def _show(name: str, res: dict) -> None:
 def main() -> None:
     load_dotenv()
     vbase = os.environ["VLLM_URL"].rstrip("/") + "/v1"
-    vkey = os.environ.get("VLLM_API_KEY", "rw-vllm-dev-key")
+    vkey = os.environ["VLLM_API_KEY"]
     obase = os.getenv("OPENROUTER_BASE_URL", "https://openrouter.ai/api/v1")
     okey = os.environ["OPENROUTER_API_KEY"]
     m = "ibm-granite/granite-4.1-8b"