rag-wright 0.8.0__tar.gz → 0.9.0__tar.gz

{rag_wright-0.8.0 → rag_wright-0.9.0}/.claude/skills/authoring-a-capability/SKILL.md

@@ -37,9 +37,13 @@ MCP) follows automatically for invokable kinds; 4b (a bespoke MCP server) is opt
 1. **Implementation** — the real code, in that kind's home (see per-kind below).
 2. **The invoke factory + `impl_ref`** (invokable kinds: subgraph/model) — write a co-located
    `async def ainvoke(resources, inputs)` (subgraph) / `def <name>(resources, inputs)` (model) in the capability's
-   own module. A subgraph factory builds over the opaque `WorkspaceHandle` (`resources._store`,
-   `resources._embedder`, `resources.model_id(role)`), never env; a model factory is store-independent and ignores
-   `resources`. The manifest's `impl_ref="module:attr"` points to it. The invoker imports
+   own module. A subgraph factory builds over the opaque `WorkspaceHandle`, never env: its store access with
+   `pack_store(resources, YourStore)` (your pack's store extension over the workspace's store) and its models with
+   `resources.model_id(role)`; its query embedder with `workspace_embedder(resources)` (`rag_wright.pack_sdk`, built
+   from the workspace's embedding profile) and its options with `resources.config`; a model factory is
+   store-independent and ignores `resources`. A builder that takes the store as a keyword goes through `pack_store`
+   too: `pack_store(resources, lambda store: production_x(store=store, ...))`. The handle's underscored attributes
+   are engine-internal; the reference pack's factories show the public form. The manifest's `impl_ref="module:attr"` points to it. The invoker imports
    it LAZILY and calls it — there is **NO central adapter dict** (EP-CORE-2). A plain function/agent_skill/mcp_tool
    declares no `impl_ref`.
 3. **ARD manifest + register it** — a `CapabilityManifest(slug, kind, display_name, description,
@@ -57,8 +61,9 @@ MCP) follows automatically for invokable kinds; 4b (a bespoke MCP server) is opt
    A downstream product can register without touching the canonical set (`register_capability` does not check
    slugs), BUT `manifests.author()` rejects a non-canonical slug, so a product that publishes ARD JSON must call
    `register_canonical_slugs` for its slugs first.
-   Publish to `~/.air/registry` with `uv run python scripts/publish_manifests.py`; that script publishes only the
-   engine's reference pack (it calls `load_reference_pack()`), so a product publishes its own with `publish_all`.
+   Publish the ARD JSON (`urn:air`) of what is registered with `publish(slug, root=...)` or `publish_all(root=...)`
+   from `rag_wright.api` (the root defaults to `ARD_REGISTRY_ROOT`, else `~/.air/registry`); a slug must be canonical
+   (`register_canonical_slugs`, above). The engine's `scripts/publish_manifests.py` does this for its reference pack.
    Callable kinds get `ResponseBounds` (defaulted); `agent_skill` must NOT declare bounds (loaded, not called).
 4. **Invocable + MCP for free** — once registered with an `impl_ref`, the capability is callable as
    `ainvoke_subgraph(slug, inputs, resources=ws)` / `invoke_model(slug, inputs, resources=ws)` (or
@@ -79,7 +84,7 @@ MCP) follows automatically for invokable kinds; 4b (a bespoke MCP server) is opt
 - **Invoke:** the co-located `async def ainvoke(resources, inputs)` factory (impl_ref target) builds + awaits the graph.
 - Retry/dead-letter come from the graph scaffold, not the invoker. `CapabilityManifest` has no contract field (only
   the internal `CapabilityRegistry.register(contract=...)` takes one); where the typed I/O must be declared, set
-  `capability_interface` on the manifest.
+  `capability_interface` on the manifest (`CapabilityInterface`, from `rag_wright.pack_sdk`).
 
 ### function — a plain, typed callable
 - **Home:** `capabilities/<slug>.py` (generic) or `packs/<pack>/capabilities/<slug>.py` (domain). A deterministic or model-backed callable with a Pydantic in/out contract.
@@ -103,7 +108,9 @@ MCP) follows automatically for invokable kinds; 4b (a bespoke MCP server) is opt
 - **Home:** `skills/<slug>/SKILL.md` (generic) or `packs/<pack>/skills/<slug>/SKILL.md` (domain, e.g.
   `packs/compliance/skills/compliance_judgment/SKILL.md`) + the agent runtime (e.g. `skills/rlm/`). It is LOADED (progressive
   disclosure), not called: no `ResponseBounds`. Declare `requires=(...)` for a closure over other skills and
-  `skill_runtime` for its intrinsic runtime.
+  `skill_runtime` (`SkillRuntime`, from `rag_wright.pack_sdk`) for its intrinsic runtime.
+- A product's agent (a Deep Agents harness) is not a capability kind yet: an `agent` kind is a later engine workstream
+  (PS-33, engine ADR-0139). Build agents in the product, on `agent_chat_model` for their model.
 
 ### mcp_tool — a capability exposed over MCP
 - A distinct ARD identity (`<slug>_mcp`) for the same underlying capability exposed as a cross-agent MCP tool.
@@ -129,7 +136,7 @@ add it to `_RESERVED_WITHOUT_MANIFEST` with a one-line reason. The engine's `tes
 pack, so these tests see its slugs.
 
 **The import boundary (engine repo).** `pyproject.toml` `[tool.importlinter]` has three forbidden contracts: every
-generic engine package (`source_modules`: `rag_wright.api`, `capabilities`, `contracts`, `corpus`, `ingestion`,
+generic engine package (`source_modules`: `rag_wright.api`, `capabilities`, `contracts`, `corpus`, `evals`, `ingestion`,
 `models`, `ontology`, `pack_sdk`, `skills`, `spans`, `store`, `subgraphs`, `util`) must never import
 `rag_wright.packs`; pack code (`rag_wright.packs`) imports only `rag_wright.api` and `rag_wright.pack_sdk` (plus
 itself), never another engine package directly; and `rag_wright.packs.contracts` must never import

{rag_wright-0.8.0 → rag_wright-0.9.0}/.claude/skills/building-an-ingestion-capability/SKILL.md

@@ -149,12 +149,32 @@ network-bound work inside it; the engine runs up to `IngestionTuning.extract_con
   to its parent, and an `AttachedTo` edge from the child to the table-row span it belongs to, with a confidence and
   the identifier evidence (`IngestionTuning.identifier` sets what counts as an identifier). The report lists them in
   `DocumentReport.children`, `links` and `unmapped_links`.
+- **Record table or form, decided per table.** In `auto` mode a table is a database (one unit per row) when its
+  header has `RecordTableRule.min_header_cols`+ named columns, `distinct_ratio`+ of them distinct, and either a serial
+  first column in `serial_ratio`+ of the rows or `min_cols`+ columns; anything else stays whole (a form). A test log
+  or results sheet judged wrongly is fixed with `IngestionTuning(record_table=RecordTableRule(...))`, or per source
+  with `IngestSource(table_mode="record"|"block")`. Check the decision on your samples with
+  `evaluate_ingestion(..., table_labels=[{"pattern": <regex on the header row>, "label": "record"|"block"}])`
+  (first match wins): a block table split per row, or a wide record table not split, is a failure, and tables no
+  pattern matched are listed in `unlabelled_tables`.
+- **How an embedded file finds its row.** An identifier is a token of 4+ characters containing a digit that appears
+  on at most `IdentifierRule.max_rows` table rows and in at most `max_files` embedded files (defaults 3 and 3; set
+  them with `IngestionTuning(identifier=IdentifierRule(...))`). A file whose identifiers match no row, or too many,
+  stays a child document with no `AttachedTo` edge and is listed in `unmapped_links`: for a workbook with dozens of
+  embedded lab reports, check `links` against what you expect before trusting the graph.
+- **Nesting.** A file embedded in an embedded file is ingested; anything deeper is listed in
+  `DocumentReport.embedded_skipped`, not walked.
+- **Stable ids.** `IngestSource(doc_id=...)` gives a document your own id (an upload's record id, say), so a re-ingest
+  of the same file lands on the same `Document`.
 
 ## 6. Verify (definition of done)
 
-1. `evaluate_ingestion` passes on a representative sample of YOUR documents (not a hand-picked easy one). If your
-   units carry labels, its `labels` report on your gold meets the bar you set, with the confusions understood.
-2. A live ingest of that sample: no dead letters, `extraction_failures` explained, records cite spans
+1. `evaluate_ingestion` passes on a representative sample of YOUR documents (not a hand-picked easy one), with
+   `table_labels` covering every table type you have. If your units carry labels, its `labels` report on your gold
+   meets the bar you set, with the confusions understood. Inside a web backend use `aevaluate_ingestion` (with
+   `progress`) as a background job.
+2. A live ingest of that sample, in a disposable database (`with temporary_workspace(config, prefix="trial") as ws:`),
+   no dead letters, `extraction_failures` explained, records cite spans
    (`kg_read(ws, "<your type>", fields=["span_id"])`), and `span_positions(ws, doc_id)` gives the citation positions.
 3. A hermetic test of your extractor on fixed units (no model calls in the default suite; the engine's test network
    guard fails any unmarked test that reaches the network).

{rag_wright-0.8.0 → rag_wright-0.9.0}/.claude/skills/creating-evals/SKILL.md

@@ -63,8 +63,10 @@ worked examples; your evals live in your repo.
   GoldMapping(id=..., input=..., expected=..., group=...), name=..., version=...)` reads them as written -- the header
   found under a title, blank rows skipped, values kept exactly, each item's place recorded -- and refuses anything
   that would make the gold wrong (a missing column, a row without its id, a duplicate id) with where it is. Map
-  `group` to the source document so splits stay leakage-safe. A gold set is `local_only` by default: client data
-  never leaves the machine.
+  `group` to the source document so splits stay leakage-safe. A workbook with several sheets: name one with
+  `sheet=`. Say how the gold was made with `provenance=` (`hand`, `silver` or `generated`; or a per-row column through
+  `GoldMapping(provenance=...)`), so silver is never conflated with gold. A gold set is `local_only` by default:
+  client data never leaves the machine.
 - **Gold stays local when the corpus is restrictively licensed** (CUAD/ACORD-derived gold lives under the
   gitignored `data/eval/`); commit the builder and the scorer, never the data.
 
@@ -145,10 +147,10 @@ or a plain function, which runs off the event loop) on every item, `repeats` tim
 answer), scores each repeat, and passes a scorer only if it passes on every repeat (its gate is the worst repeat;
 `per_repeat_gate` lists each). `run_eval` is the synchronous form; inside a web backend, await `arun_eval` as a
 background job. What is still yours:
-- **Isolate the capability under test**: the task calls ONE capability, injected (a seam / `extract_override` / an
-  injected `retrieve`), so the eval measures it, not the whole pipeline. Invoke production code THROUGH the
+- **Isolate the capability under test**: the task calls ONE capability, injected (a seam, such as the reference pack's
+  `extract_override` or an injected `retrieve`), so the eval measures it, not the whole pipeline. Invoke production code THROUGH the
   capability layer (`ainvoke_subgraph`/`ainvoke_model`), never a hand-built copy. Score the **shipped request** (the
-  exact prompt/question builder production uses, e.g. `judge_request`), not a copy of it re-typed in the eval.
+  exact prompt/question builder production uses, e.g. the reference pack's `judge_request`), not a copy of it re-typed in the eval.
 - **Repeatability for non-deterministic models.** Use `repeats` and read `report.flips`; flips cluster near the
   threshold and usually mean the QUESTION is ambiguous: fix the question, do not vote it away (Jev flipped 5/181
   answers at scores 0.47-0.56; temperature/seed did not help, majority voting barely helped, rewriting the ambiguous
@@ -160,7 +162,7 @@ background job. What is still yours:
 - **Keep the report.** `report.model_dump_json()` holds every item's outputs; save it next to the gold version.
   `rescore_eval(report, gold, scorers)` re-scores it with new thresholds, a new scorer or a corrected gold, free. An
   eval kept only as aggregate numbers can never be checked again.
-- **Pre-flight paid bulk** (>~50 paid calls): print the exact count + cost and wait (`warn-before-bulk` rule).
+- **Pre-flight paid bulk** (>~50 paid calls): print the exact count + cost and wait for approval before launching.
 - **Hermetic tests never touch the network**: `tests/conftest.py` fails a test that resolves a non-local host unless
   it carries a live marker (`model`, `store`, ...). Live evals run as scripts (outside pytest) or carry a live
   marker.

{rag_wright-0.8.0 → rag_wright-0.9.0}/.claude/skills/laya/SKILL.md

@@ -25,9 +25,10 @@ you can fine-tune. In RAG_Wright a decision model is wired as a `kind="model"` c
 
 **Swapping Jev for Laya is config, but not free.** `DECISION_PROFILES` has only Jev entries (`jev-1.13`, the
 default, and `jev-latest`). A Laya swap needs (1) a server that speaks the Decisions API (POST
-`{model, state, questions}`, return `{answers: {<id>: {type, noul|choice|score}}}`), and (2) a `DecisionModelProfile`
-entry in `DECISION_PROFILES` with its `endpoint`, `served_model_id` and `api_key_env` (an unknown id falls back to
-the OpenRouter endpoint). Then select it with `RAG_DECISION_MODEL=<profile id>`;
+`{model, state, questions}`, return `{answers: {<id>: {type, noul|choice|score}}}`), and (2) its profile, registered
+at startup from your product: `register_decision_profile(DecisionModelProfile(model_id="laya-onprem",
+served_model_id=..., endpoint=..., api_key_env=...))`, both on `rag_wright.api` (an unregistered id falls back to the
+OpenRouter endpoint). Then select it with `RAG_DECISION_MODEL=<profile id>` (or the `jev_decision` input `model`);
 `RAG_JEV_TIMEOUT_S` overrides the request timeout.
 
 **Batch the questions.** One call carries one `state` and many keyed questions, so ask everything about a unit in
@@ -68,7 +69,7 @@ server you swap in.
 ## Data — the JSONL schema + criteria (the differentiator)
 One JSONL line per case (`research/scripts/finetune_single_device.py` reads this exact shape):
 ```
-{"state": "<clause text>",
+{"state": "<unit text>",
  "questions": {"<dim>": {"type":"choice","instructions":"<the question>","criteria":{"<value>":"<description>", ...}}},
  "gold": {"<dim>": {"probabilities": {"<value>": <p>, ...}, "label": "<value>"}}}
 ```
@@ -78,7 +79,7 @@ One JSONL line per case (`research/scripts/finetune_single_device.py` reads this
 - **`gold` is a DISTRIBUTION (soft targets), not a hard label** — RLCD trains on the teacher's probability per option.
   One-hot (`{gold:1.0, other:0.0}`) works and is the pragmatic start; true soft targets (teacher per-option probs)
   are the design-intended enhancement.
-- Reuse your existing **contract-disjoint, symmetric splits** so floors are directly comparable to SetFit. Silver
+- Reuse your existing **document-disjoint, symmetric splits** so floors are directly comparable to SetFit. Silver
   goes in TRAIN only; val/test stay gold.
 
 ## Fine-tune — single T4, RLCD, calibration built in
@@ -116,11 +117,12 @@ One JSONL line per case (`research/scripts/finetune_single_device.py` reads this
   the model from the engine (never a hardcoded/stale id; bulk teacher labeling on Modal Qwen, not OpenRouter), load
   `.env` by EXPLICIT path from an out-of-repo script, SMOKE one item before the fan-out, and stream X/N to a log you
   actively monitor. Those exact mistakes cost runs on 2026-10-04.
-- **Modal harness = reuse the [[setfit]] hardened pattern:** `.uv_pip_install("laya")` + `.add_local_file` the
+- **Modal harness = follow the [[setfit]] hardened pattern** (the pattern described there; that harness's code is
+  not shipped): `.uv_pip_install("laya")` + `.add_local_file` the
   finetune script; launcher BLOCKS on `.get()` per spawn (no spawn-and-return), stamps + verifies a `data_sha`,
   writes a manifest, streams X/N; snapshot keepers server-side to a `/checkpoints/<name>` path (a small copy fn) —
   the laya volume has no built-in snapshot, add one. Never lose a fine-tune.
-- **ACCOUNT CONTAINER CAP = 10 (fzaidi2014).** Spawning more than 10 fine-tunes at once does NOT run them all —
+- **ACCOUNT CONTAINER CAP (10 on the engine authors' account; check your workspace's).** Spawning more than 10 fine-tunes at once does NOT run them all —
   Modal queues the rest and runs ~10 at a time (correct, but ~N/10 waves of wall-clock, and a "why only 10 running?"
   surprise). Size a `groupbake`/dimbatch fan-out to ≤10 in flight (chunk into waves + gather between), or state the
   wave count honestly. This is a DIFFERENT knob from the vLLM `max_containers=1` + `@modal.concurrent` batching in the

{rag_wright-0.8.0/src/rag_wright/.agents → rag_wright-0.9.0/.claude}/skills/qwen-vllm-modal/SKILL.md

@@ -46,7 +46,13 @@ config = EngineConfig(store=..., models={"general": server.model_id})   # or any
 stop_model_server("acme-qwen")  # only to retire it: stopping discards the snapshots (see "Cold start & cost")
 ```
 
-`amodel_server(spec)` reconnects to a server deployed earlier (a restart, another process) without redeploying.
+`server.model_id` is `qwen3.8-27b-modal@<name>`: a copy of the `qwen3.8-27b-modal` profile pointed at this server
+(reasoning on for structured calls and agent turns, off for free text). Use that id, never `VLLM_BASE_URL`. A
+reasoning-off structured pick (bulk closed-vocab labeling) has no `@<name>` variant: point `VLLM_BASE_URL` at
+`server.base_url` and use the `qwen3.8-27b-modal-nothink` profile.
+`amodel_server(spec)` reconnects to a server deployed earlier (a restart, another process) without redeploying;
+`await_model_server(server, timeout_s=...)` waits for readiness; `amodel_server_status(spec)` looks without waking it
+(`deployed`, `running` (a container bills), `healthy`, `weights_cached`); `astop_model_server(name)` is the async stop.
 The API key comes from the variable `ModelServerSpec.api_key_env` names (default `VLLM_API_KEY`); there is no default
 key, so set a real one. Needs `rag-wright[modal]` and a Modal login. One app per server name: different names give
 different servers (and URLs), for example one per tenant.
@@ -66,8 +72,9 @@ The script refuses to deploy without `VLLM_API_KEY`: Modal's web endpoint is pub
 `qwen3.8-27b-modal` routes there (the seam also refuses an unset key). When to stop it: "Cold start & cost" below.
 
 **The engine does NOT reach Modal by default.** Every role's default is `_PRODUCT_LLM = "qwen3.8-27b-modal-or"`,
-the same model routed through OpenRouter (a Modal cold start can still take minutes when Modal builds a snapshot). To send calls to the Modal server, set
-`RAG_MODEL_ALL=qwen3.8-27b-modal` (all roles), a per-role `RAG_MODEL_<ROLE>`, or `EngineConfig.models`. For bulk
+the same model routed through OpenRouter (a Modal cold start can still take minutes when Modal builds a snapshot). To send calls to the Modal server, put
+`server.model_id` in `EngineConfig.models` (the API route), or on the raw route set `RAG_MODEL_ALL=qwen3.8-27b-modal`
+(all roles) or a per-role `RAG_MODEL_<ROLE>`. For bulk
 closed-vocab labeling, the reasoning-off profile `qwen3.8-27b-modal-nothink` exists (opt-in; see the reasoning
 caveat below). `VISION_OCR` now defaults to the product LLM too, but image inputs through the Modal vLLM server are
 UNVERIFIED (the serve script passes no multimodal flags): confirm an image call works before pointing
@@ -130,7 +137,7 @@ the OFFICIAL prebuilt vLLM image** rather than pip-building vLLM:
 - **When to stop it -- one rule (EV-6; it used to contradict the line above).** The A100 is billed only while a
   container runs; the server scales to zero after 10 idle minutes (`scaledown_window=600`, no warm minimum) and then
   costs nothing and KEEPS its snapshots, so the next request is back in ~13 s. `modal app stop` saves at most those
-  10 minutes and makes the next deploy pay a 9-15 min snapshot-building boot (2026-10-12: a stop after a failed smoke
+  10 minutes and makes the next deploy pay a snapshot-building boot (usually 4-9 min; 15 min seen once) (2026-10-12: a stop after a failed smoke
   run, then a redeploy, cost a 911 s boot).
   1. **Deploy once per server name**; reuse the name (a new name builds new snapshots).
   2. **Build the consumer first** (the bulk / label / eval script, its smoke test, its progress logging) while no
@@ -138,8 +145,8 @@ the OFFICIAL prebuilt vLLM image** rather than pip-building vLLM:
      a smoke-warmed A100 sat idle ~20 min while unrelated work ran).
   3. **Between runs, and when you are done, let it scale to zero.** Never stop it between runs; a script's error
      path must not stop it either when a fix-and-rerun is likely.
-  4. **Check it reached zero**: `modal app list` shows the app `deployed` with 0 tasks about 10 minutes after the
-     last call. If tasks remain past the window, find out why (that is the 2026-10-04 burn) and stop it.
+  4. **Check it reached zero**: `amodel_server_status(spec).running` is False (or `modal app list` shows the app
+     `deployed` with 0 tasks) about 10 minutes after the last call. If tasks remain past the window, find out why (that is the 2026-10-04 burn) and stop it.
   5. **Stop (`modal app stop <app> --yes`, or `stop_model_server`) only to retire a server**: you will not use it
      for a long time and do not want its endpoint up, or you are replacing its deployment. Expect the next deploy to
      rebuild the snapshots.

{rag_wright-0.8.0/src/rag_wright/.agents → rag_wright-0.9.0/.claude}/skills/setfit/SKILL.md

@@ -219,11 +219,14 @@ after the 21-classifier runs — each one cost real time and each is avoidable w
 order before you fan out any bulk job.
 
 1. **Resolve the model from the engine, never hardcode a model id.** A hardcoded id rots: `ibm-granite/granite-4.1-8b`
-   now 404s on OpenRouter ("No endpoints found"). Use the engine resolver (`models.profiles.model_for(ModelRole.<ROLE>)`,
+   now 404s on OpenRouter ("No endpoints found"). Use the engine resolver (`rag_wright.pack_sdk.model_for(ModelRole.<ROLE>)`,
    or `ws.model_id(role)` on a workspace; the model-profile seam) so the id and backend come from config, and **confirm it with a one-call smoke before any
    bulk run**. Bulk teacher labeling runs on **self-hosted Qwen on Modal** (`qwen-vllm-modal` skill + ADR-0110), NOT
-   OpenRouter (flat-GPU = the warn-before-bulk rule); reasoning ON for label fidelity; client concurrency ≤16; raise
-   the structured timeout for reasoning-ON bulk; `modal app stop` when done.
+   OpenRouter (a flat GPU rate, not a per-call bill: before more than ~50 paid calls, print the exact count and cost
+   and wait for approval); reasoning ON for label fidelity; client concurrency below the server's `max_num_seqs`
+   (counting every concurrent run); raise the structured timeout for reasoning-ON bulk; between runs let the server
+   scale to zero, and stop it only to retire it (`qwen-vllm-modal`, "Cold start & cost": a stop discards its
+   snapshots).
 2. **Load `.env` by EXPLICIT path in any script run from outside the repo root** (a scratchpad / one-off script).
    `load_dotenv()` with no path searches the CALLING SCRIPT's directory, so a script in `/tmp/...` silently misses
    the project `.env` → `KeyError` or wrong creds. Pass the repo `.env` path (`load_dotenv("/abs/repo/.env")`) or
@@ -257,7 +260,7 @@ Concrete tool gotchas from a real run; following them saves real time.
   per-run progress log on the volume. A run whose app shows `stopped / 0 tasks` seconds after launch did NOT train —
   confirm via the platform task count and the registry, not by assuming.
 - **Mind the ACCOUNT CONTAINER CAP when sizing the fan-out.** A workspace has a max concurrent-container count (10 on
-  the fzaidi2014 account). Spawning more than that does NOT run them all at once — the excess QUEUES and runs in waves
+  the engine authors' account; check your own workspace's limit). Spawning more than that does NOT run them all at once — the excess QUEUES and runs in waves
   (correct + no double-spend, but ~N/cap× the wall-clock, and only `cap` show as running). Size the wave to ≤ the cap
   (chunk spawns + gather between waves) or report the wave count honestly. Distinct from any in-container concurrency.
 - **When multiple entrypoints exist, name the one to run** (`script.py::entrypoint`) and use the FULL script path;

{rag_wright-0.8.0/src/rag_wright/.agents → rag_wright-0.9.0/.claude}/skills/using-the-rag-wright-engine/SKILL.md

@@ -17,14 +17,18 @@ open-core and domain-neutral; your product brings the domain. Hard rules:
 - **You own your capabilities.** The engine's catalog starts empty; your seam registers everything you invoke, at
   startup (section 1). The contract/compliance capabilities are the engine's reference pack, not the engine.
 - **Product → Engine, one way.** Never fork, edit, or reach into engine internals. If the engine needs a change,
-  flag it upstream — don't work around it here.
+  write an engine request (section 6) — don't work around it here: a product-side patch hides the defect from
+  every other product built on the engine.
+- **Your product owns its store and its data; the engine mediates access.** Your product runs its own ArcadeDB and
+  has no ArcadeDB client of its own (section 1).
 - **The public surface is `rag_wright.api`; a pack also uses `rag_wright.pack_sdk`.** From `rag_wright.api` import
-  `EngineConfig`/`StoreConfig` (`StoreConfig.from_env()` reads `ARCADEDB_*`)/`open_workspace`, the invokers
+  `EngineConfig`/`StoreConfig` (`StoreConfig.from_env()` reads `ARCADEDB_*`)/`open_workspace`
+  (`temporary_workspace`/`drop_workspace` for disposable and removed databases), the invokers
   (`ainvoke_subgraph`/`invoke_model`/`ainvoke_model`), discovery (`capability_index`/`discover`), the KG accessors
   (`kg_read` with `key_range`, `kg_write`, `kg_edges` with `NOT_NULL`, `kg_count`/`kg_delete`/`kg_update`,
   `entities_by_name`, `span_positions`), `pack_store(ws, cls)` (build your pack's store extension over the
   workspace's store; the store itself stays private), metering and tracing (`measure_usage`, `record_usage`,
-  `traced_run`/`traced_step`), answers (`agenerate_answer`, `ajudge_spans` with `EvidenceItem`/`Condition`, the
+  `traced_run`/`traced_step`), `agent_chat_model` (an agent's chat model from an engine model id), answers (`agenerate_answer`, `ajudge_spans` with `EvidenceItem`/`Condition`, the
   models from the workspace's roles, your domain wording via `guidance=`), `ModelRole`, pack loading
   (`register_capability`/`load_pack`/`engine_capabilities`/`load_reference_pack`), the document helpers
   (`parse_document`/`aparse_document`, `parse_document_bytes`/`aparse_document_bytes` for uploads,
@@ -47,13 +51,19 @@ open-core and domain-neutral; your product brings the domain. Hard rules:
 ## 1. Install
 
 ```sh
-uv add 'rag-wright>=0.3.0'              # released-product mode (PyPI); see below for local co-development
+uv add 'rag-wright>=0.8.0'              # released-product mode (PyPI); see below for local co-development
 ```
 
 Stand up the runtime prerequisites you provide: ArcadeDB (the store), a model provider (OpenRouter or self-hosted
 vLLM), and — only for NER — `uv add 'rag-wright[ner]'` + `uv run python -m spacy download en_core_web_sm`.
 Read the engine's **installation** + **configuration** docs for the exact `.env` and `EngineConfig` fields.
 
+**ArcadeDB is your product's, all of it.** Run your own instance (installation, "Run your product's ArcadeDB"): its
+own container name, its own host ports (a pair no other instance on the machine uses; the engine's test instance
+uses 2480/2424), its data directory in your repo (gitignored) or on storage you manage. Make your settings default
+to your own port, so an unset variable cannot reach another instance and appear to work. Your product code has no
+ArcadeDB client: every read and write goes through `rag_wright.api`.
+
 **Local co-development** (engine and product developed together on one machine): keep the floor and add an editable
 source so engine changes are picked up immediately: `[tool.uv.sources] rag-wright = { path = "<engine checkout>",
 editable = true }`. Check against the published release with `uv sync --no-sources` (never with `--locked`; never
@@ -78,6 +88,18 @@ per implementation chosen per tenant in your code.
 (`use_workspace_models(ws)` for a call that does not). To change a tenant's config, open its corpus again with the
 new config: a different config replaces the cached handle, no restart needed. `docs/workspaces.md` lists what applies per workspace, per call and per process.
 
+**Databases.** `open_workspace(config, corpus=...)` creates a missing database by default (and logs
+`created database '<name>'`); where the database must already exist (a read path, a query service, an explicit
+override), pass `create=False`, which raises `LookupError` naming the databases that do exist instead of opening an
+empty graph. A probe, measurement or test that needs its own database uses
+`with temporary_workspace(config, prefix="probe") as ws:`; the database is dropped when the block exits, also when it
+raises. Remove a database with `drop_workspace(config, corpus=...)` (it refuses a blank name and cannot be undone);
+never delete its files or call the ArcadeDB server yourself.
+
+**Hermetic tests stub the door the code opens.** Your code reaches the store through `open_workspace`; a fast test
+stubs that. A stub on an entry point the code no longer calls lets the test reach a real store and create a database
+on every run, without failing.
+
 Repository paths in this skill (`docs/`, `eval/`, `scripts/`, `tests/`, `src/`) are in the engine repository: read them there or on GitHub, at the tag matching your installed engine.
 
 ## 2. Ground before you write (graphify)
@@ -135,7 +157,25 @@ Work the domain-adaptation guide, grounding each engine call (step 2) and writin
    `build_ingestion(document_hook=...)`, which runs once per document after its records are written; a
    first-class resolver config is still engine gap G1. → `kg-construction.md`, `entity-resolution.md`.
 6. **Build the product seam** — your thin layer over the engine API + registered caps (tenancy, orchestration,
-   the product tool surface). Wrap `open_workspace`/the invokers/`kg_read`/`measure_usage`.
+   the product tool surface). Wrap `open_workspace`/the invokers/`kg_read`/`measure_usage`. Classify each operation
+   before building it: deterministic (the backend calls a pre-wired workflow), agent-fronted (a fixed workflow
+   reached through an agent), or composed by a model at run time; build each workflow once, as a tool that both the
+   UI and an agent call, and never put a model in the path of an unambiguous button click.
+
+**PDFs.** Call the parse and ingest functions concurrently as you like: the engine parses one PDF at a time inside
+the process (a crash in the PDF library otherwise, engine ADR-0138) and keeps everything after the parse concurrent.
+Count an upload's pages with `pdf_page_count(data)`, which reads under the same lock; product code never imports the
+PDF library itself.
+
+**Agents.** Build a product agent (Deep Agents or LangGraph) on the model the engine gives it, by engine model id:
+`create_deep_agent(model=agent_chat_model("<model id>", max_retries=0), tools=[...])` (`None` takes the `GENERAL`
+role's model). The profile decides endpoint, key and the agent turn's reasoning flag, so your code names no model,
+vendor or vendor flag; every call, a subagent's included, is recorded into `measure_usage()` with the provider's
+real cost and traced when tracing is on. The generic harness substrate (the loop, error levels and deadlines,
+retries, interrupts and human-in-the-loop, replay on resume, memory practice, metering) is engine scope (engine
+ADR-0139, PS-32): it arrives as a third public tier with the `building-an-agent-harness` skill. Until your installed
+engine has it, keep agents thin and never write a substrate of your own; your product keeps its agents, tools,
+prompts, skills, guardrails and decision models.
 
 **Using a self-hosted model (Qwen on Modal).** Deploy it from the engine and use its model id like any model:
 `server = await adeploy_model_server(ModelServerSpec(name="<app>", max_num_seqs=<concurrency target>))` (the locked
@@ -181,8 +221,15 @@ engine PR:
 
 - Depend on a floor (`rag-wright>=X.Y.Z`, never `==`); `uv.lock` holds the exact version. Dependabot (uv ecosystem)
   opens a PR per new release; your CI — with a LIVE engine-seam test — gates it.
-- Found an engine bug? Fix it **in the engine repo** (PR titled `fix(scope): …` so it lands in the next release),
-  not by patching around it in the product.
+- Found an engine bug, or need something the engine does not have? Write an **engine request** in your repo,
+  `docs/engine-requests/YYYY-MM-DD-<slug>.md`: what you found, the evidence (a reproduction, a measurement, the
+  exact error), what you want, and what you do meanwhile. The engine fixes it in its own repo (a `fix(scope): …`
+  commit, so the next release carries it); never patch around it in the product. Read the engine's
+  `docs/domain-adaptation/_engine-gaps.md` first: the gap may be known.
+- An engine symbol on the engine's main branch but not yet released is reached through the editable path
+  (local co-development), never through an internal import; move your floor only for a released symbol.
+- Keep a committed snapshot of the installed `rag_wright.api.__all__` and a test that compares them: a change to the
+  surface you build on is then named the day it arrives, and any other engine change needs nothing from you.
 - Need that fix before the next release? Temporarily pin
   `[tool.uv.sources] rag-wright = { git = "https://github.com/fzaidi-dai/RAG_Wright", rev = "<merged-sha>" }` and
   `uv lock`; drop the override when the release containing it arrives.
@@ -214,3 +261,7 @@ link). Link these, and any product-building skills another repo writes for produ
   domain instead of authoring your own `.ttl`.
 - Hand-constructing a capability's logic instead of invoking it by name through the invoker.
 - Skipping the eval: write it before the implementation.
+- Patching around an engine defect in product code instead of writing the engine request.
+- Giving product code an ArcadeDB client, pointing it at another product's or the engine's instance, or deleting a
+  database's files by hand (`drop_workspace` removes one; `temporary_workspace` keeps scratch from piling up).
+- Naming a model, vendor or vendor flag in product code; writing your own agent harness substrate.

{rag_wright-0.8.0 → rag_wright-0.9.0}/.env.example

@@ -6,8 +6,9 @@ OPENROUTER_API_KEY=
 # OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
 
 # --- ArcadeDB (single store: hybrid retrieval index + knowledge graph) ---
-# Run locally with Docker (see docs/ArcadeDB_Local.md). The password below is the container's
-# root password; set the same value here and in the `docker run` JAVA_OPTS.
+# Working on the engine: its own test instance, run locally with Docker (see docs/ArcadeDB_Local.md).
+# A product sets these in its own .env, pointing at its own instance (docs/installation.md).
+# The password below is the container's root password; set the same value here and in the `docker run` JAVA_OPTS.
 ARCADEDB_HOST=localhost
 ARCADEDB_PORT=2480
 ARCADEDB_USER=root

rag_wright-0.9.0/.release-please-manifest.json

@@ -0,0 +1,3 @@
+{
+  ".": "0.9.0"
+}

{rag_wright-0.8.0 → rag_wright-0.9.0}/CHANGELOG.md

@@ -4,6 +4,15 @@ All notable changes to RAG_Wright are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project aims to follow
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.9.0](https://github.com/fzaidi-dai/RAG_Wright/compare/0.8.0...0.9.0) (2026-10-13)
+
+
+### Features
+
+* **PS-36:** agent_chat_model, an agent's chat model from a model profile, metered and traced ([f1af92a](https://github.com/fzaidi-dai/RAG_Wright/commit/f1af92a848c2de7dd02cf1d99ce61cd3df8d20d9))
+* **PS-37:** drop_workspace and temporary_workspace, so a product removes the databases it creates ([44c0c78](https://github.com/fzaidi-dai/RAG_Wright/commit/44c0c78b0b8dd02724920391a564f3cda006c41c))
+* **PS-38:** close the public-surface gaps a new product hits (G12, G14, G22, G23) and require ARCADEDB_PORT ([0d0c1f1](https://github.com/fzaidi-dai/RAG_Wright/commit/0d0c1f16b95bc763dec348a59965de2e8b4a22c9))
+
 ## [0.8.0](https://github.com/fzaidi-dai/RAG_Wright/compare/0.7.0...0.8.0) (2026-10-12)
 
 

{rag_wright-0.8.0 → rag_wright-0.9.0}/CLAUDE.md

@@ -55,7 +55,7 @@ The failure mode to avoid, which has happened before: a command fails with "modu
 
 ## Start of every session
 
-> **CURRENT STATE (2026-10-12): the evals workstream is COMPLETE; `rag-wright` 0.8.0 is ready to release** (ledger
+> **CURRENT STATE (2026-10-12): the evals workstream is COMPLETE and RELEASED as `rag-wright` 0.8.0** (PyPI + GitHub release `0.8.0`; ledger
 > `docs/specs/evals/plan.md`, EV-0..EV-6; EV-7 optional): gold sets from experts' files, a scorer per capability kind,
 > the runner `arun_eval` with an optional Langfuse backend, `EvalReport`, `aevaluate_ingestion`; the reference is
 > `docs/evals.md`. With it in 0.8.0: PS-24 (async calls on every event loop, ADR-0135), PS-25 (the release PR bumps

{rag_wright-0.8.0 → rag_wright-0.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: rag-wright
-Version: 0.8.0
+Version: 0.9.0
 Summary: A domain-retargetable, open-core engine for hybrid retrieval + knowledge-graph RAG: parsing, chunking, embedding, hybrid search, reranking, graph extraction, entity resolution, ontology-driven knowledge, and a capability runtime.
 Project-URL: Homepage, https://github.com/fzaidi-dai/RAG_Wright
 Project-URL: Repository, https://github.com/fzaidi-dai/RAG_Wright
@@ -132,8 +132,8 @@ from git or a local path to get them (see [`docs/installation.md`](docs/installa
 0.1.0 migrates with the
 [breaking-changes record](docs/specs/ingestion-hooks/ing8-breaking-changes.md).
 
-Runtime prerequisites: **ArcadeDB** (the store — run it locally with Docker, see
-[`docs/installation.md`](docs/installation.md)); **a model provider** (an OpenRouter key by default, or a self-hosted
+Runtime prerequisites: **ArcadeDB** (the store: your product runs its own instance and owns its data, including
+the knowledge graph; see [`docs/installation.md`](docs/installation.md)); **a model provider** (an OpenRouter key by default, or a self-hosted
 endpoint); and, only for NER (named entity recognition), the optional extra `uv add 'rag-wright[ner]'` +
 `uv run python -m spacy download en_core_web_sm`.
 

{rag_wright-0.8.0 → rag_wright-0.9.0}/README.md

@@ -76,8 +76,8 @@ from git or a local path to get them (see [`docs/installation.md`](docs/installa
 0.1.0 migrates with the
 [breaking-changes record](docs/specs/ingestion-hooks/ing8-breaking-changes.md).
 
-Runtime prerequisites: **ArcadeDB** (the store — run it locally with Docker, see
-[`docs/installation.md`](docs/installation.md)); **a model provider** (an OpenRouter key by default, or a self-hosted
+Runtime prerequisites: **ArcadeDB** (the store: your product runs its own instance and owns its data, including
+the knowledge graph; see [`docs/installation.md`](docs/installation.md)); **a model provider** (an OpenRouter key by default, or a self-hosted
 endpoint); and, only for NER (named entity recognition), the optional extra `uv add 'rag-wright[ner]'` +
 `uv run python -m spacy download en_core_web_sm`.
 

{rag_wright-0.8.0 → rag_wright-0.9.0}/docs/ArcadeDB_Local.md

@@ -1,8 +1,15 @@
-# Running ArcadeDB locally (T13/T14)
+# ArcadeDB for engine development
 
-The single store (FR-S.1) runs as a local Docker container during development, pinned to **ArcadeDB 26.7.1** (the
-latest stable tag; `:latest` resolves to a moving `-SNAPSHOT` dev build, which we deliberately do not use). This is the version T13's schema and the `vector.fuse`
-de-risk are validated on. Sparse vector index and server-side hybrid retrieval exist since 26.5.1.
+**This page is for working on the engine itself.** A product runs its own ArcadeDB instance and owns its data,
+including the knowledge graph: see [Run your product's ArcadeDB](installation.md#run-your-products-arcadedb-the-single-store).
+The instance below is the engine developer's own: it holds the databases the engine's live store tests
+(`pytest -m store`) and its evals build, on public or locally acquired evaluation corpora. No product points at it,
+and no product's data goes in it.
+
+It runs as a local Docker container, pinned to **ArcadeDB 26.7.1** (the latest stable tag; `:latest` resolves to a
+moving `-SNAPSHOT` dev build, which we deliberately do not use). This is the version the engine's schema and the
+`vector.fuse` hybrid retrieval are validated on. Sparse vector index and server-side hybrid retrieval exist since
+26.5.1.
 
 ## Start
 
@@ -16,9 +23,10 @@ docker run -d --name arcadedb-ragwright \
 ```
 
 - Port `2480` is the HTTP API the `arcadedb_python` client uses; `2424` is the binary protocol.
-- Data persists under the gitignored `data/arcadedb/` (SPEC data directory; never committed).
+- Data persists under this checkout's gitignored `data/arcadedb/`: engine test and eval databases only, never
+  committed.
 - `ARCADEDB_OPTS_MEMORY` sets the JVM heap, the store's capacity limit (the image default is `-Xms2G -Xmx2G`, which
-  ran out of memory part-way through the full CUAD knowledge graph). It is read at container start, so changing it
+  ran out of memory part-way through the full CUAD evaluation knowledge graph). It is read at container start, so changing it
   means recreating the container.
 - Set the same password in `.env` as `ARCADEDB_PASSWORD` (with `ARCADEDB_HOST=localhost`,
   `ARCADEDB_PORT=2480`, `ARCADEDB_USER=root`, `ARCADEDB_DATABASE=rag_wright`). `.env` is gitignored.
@@ -58,5 +66,5 @@ A database created before the `Span` field rename (`contract_id`, `function`, `f
 
 ```bash
 docker stop arcadedb-ragwright && docker rm arcadedb-ragwright   # stop (data persists in the volume)
-rm -rf data/arcadedb                                            # wipe all local databases
+rm -rf data/arcadedb                                            # wipe the engine's local test databases
 ```

rag_wright-0.9.0/docs/adr/0139-the-agent-harness-substrate-is-engine-scope.md

@@ -0,0 +1,95 @@
+# ADR-0139: The agent harness substrate is engine scope
+
+**Status:** accepted · **Date:** 2026-10-12 · **Supersedes in part:** ADR-0052 (the placement of the agent harness
+in the product) · **Amends later:** ADR-0041 (an `agent` capability kind, PS-33) · **Related:** ADR-0128 (the public
+tiers), ADR-0134 (skill placement), ADR-0015 (the RLM Deep Agent), RuleWright's request of 2026-10-12 and its
+ADR-0021 amendment
+
+## Context
+
+ADR-0052 put the agent harness in the product: each product would hand-build its orchestration. RuleWright has
+since built one on Deep Agents and LangGraph, and measured it: 32 modules, 7,514 lines. Only three import anything
+from the product, and 21 name no domain concept. What it found expensive is domain-free: the loop, error levels and
+deadlines, retry composition, human-in-the-loop and the interrupt protocol, replay on resume, chat interaction,
+per-layer context management, memory practice (including cross-tenant namespaces), metering, bounded grading,
+repeat suppression and the decision registry. Each of those cost it a real defect to get right. What makes the
+product what it is lies elsewhere: its tools and the capabilities it gives each agent, the guardrails that read
+content, its prompts and skills, its decision models.
+
+Two more products on the engine are planned, and each needs a harness. Left in the product, the substrate would be
+written three times and drift three ways. Since ADR-0052 the engine has also been de-domained into a substrate for
+building domain products (ADR-0124, ADR-0128); driving its capabilities with agents is part of that job.
+
+A second reason surfaced on review. `create_deep_agent(...)` returns a LangGraph `CompiledStateGraph`, the same type
+as the engine's workflow subgraphs, and the capability invoker resolves any `impl_ref` factory without knowing what
+is behind it (`capabilities/invoke.py`). So a harness, customised for one job, can itself be registered as a
+capability: an agentic capability beside the deterministic functions, the models and the workflow subgraphs. The
+RLM capabilities already are this, built by hand (ADR-0015). Only the engine owns the capability layer and ARD, so
+only the engine can make that a supported pattern.
+
+## Decision
+
+1. **The agent harness substrate moves into the engine**, as a third public tier, `rag_wright.harness`, beside
+   `rag_wright.api` (a product's seam) and `rag_wright.pack_sdk` (a pack's code).
+2. **The substrate is what exists, separated; it is not redesigned for products not yet built.** It is RuleWright's
+   working harness with its product parts removed. No protocol is added to serve a future product's needs; a
+   product customises the substrate through what the agent framework already provides (middleware, tools, prompts,
+   skills, subagents and plain functions), and a need it cannot meet that way is an engine request like any other.
+3. **RuleWright does the separation**, because the code and the analysis are in its repo. It moves the substrate
+   into a package that imports nothing from the product and names no domain concept, its product parts consume
+   that package, and it hands the package over. The engine then adopts it (PS-32): the harness imports engine
+   code only through `rag_wright.api`, no other engine module imports the harness (import contracts), the
+   domain-vocabulary guard covers it, and its deployment-specific dependencies (the Postgres checkpointer) sit in an
+   optional extra, `rag-wright[harness]`, so a plain engine install stays as it is.
+4. **The product keeps** its tools and operations, its agents and their roster, prompts and skills, the guardrails
+   that read content, its decision models, its tenancy model and its UI.
+5. **The blueprint ships with the code.** RuleWright's `building-an-agent-harness` skill moves into
+   `src/rag_wright/.agents/skills/` (ADR-0134), version-matched to the substrate it describes.
+6. **Agentic capabilities are a later, separate workstream (PS-33).** It adds an `agent` capability kind (amending
+   ADR-0041: invoked like a subgraph, but declared as non-deterministic, tool-using, possibly pausing for a human,
+   and spending against a budget), typed contracts at the boundary, interrupts across the capability boundary,
+   deadlines and budgets passed down nested agents, and a worked example on public data. Not started until agreed.
+
+## Consequences
+
+- One substrate, fixed once, for every product's chat agent and later for agentic capabilities.
+- The engine's support surface and public API grow; a harness change ships as an engine release. The tier may
+  change between minor releases until a second product uses it, and every change goes in that release's migration
+  guide.
+- `deepagents`, `langgraph` and `langchain-quickjs`, until now used only by the parked RLM agent, move onto a
+  supported path. Their floors stay floors (ADR-0112). A product still declares its own floors for the agent
+  framework it calls directly (RuleWright's ADR-0021, decision 1).
+- Publishing the substrate in the open core cannot be undone. It is accepted because the substrate is not where a
+  product's value lies; the product keeps its tools, guardrails, prompts, skills and decision models closed.
+
+## Alternatives considered
+
+- **Each product writes its own** (ADR-0052 as it stood): the same expensive code three times, the same defects
+  rediscovered.
+- **Wait for a second product's harness, then decide:** guarantees a second copy to merge later.
+- **A separate open package, its own repo:** clean scope, but a second CI and release line and an engine-by-harness
+  compatibility matrix, every seam change becoming two releases; and agentic capabilities would still need the
+  capability layer, which is the engine's.
+- **A private shared package:** right only if the harness were where a product's value lies; RuleWright's
+  measurement says it is not.
+
+## Correction, 2026-10-13
+
+RuleWright re-measured its harness before starting the separation (its engine request
+`2026-10-12-correction-to-the-harness-measurement.md`). The Context's figures were wrong: **13** modules import
+something from the product (6 of them a domain package), not three; **18** name no domain concept once comments and
+docstrings are stripped, not 21; **12 modules (2,193 lines) lift unchanged and 20 modules (5,321 lines) need work**,
+so the separation is staged work, not three imports turned into protocols. A third axis, invisible to both of those
+measurements, also keeps code product-shaped: deployment choice, vendor names and tenancy shape (a model id and an
+`is_openrouter()` helper in code, a direct `psycopg` import, a `(customer, workspace)` memory namespace).
+
+The decision is unchanged; its reasons did not rest on these figures. Two details of it are sharpened:
+
+- **The substrate's seams are the agent framework's own types**, not new protocols: it accepts a LangGraph
+  checkpointer, a memory namespace prefix from the caller, and a chat model (or a model profile name), and names no
+  model, vendor, database driver or tenancy shape. The product builds the Postgres checkpointer and passes it in, so
+  **the engine needs no `rag-wright[harness]` Postgres extra** (Decision 3's extra is withdrawn).
+- **PS-32's acceptance gains two checks:** RuleWright's third guard (deployment, vendor and tenancy shape) runs over
+  `rag_wright.harness` in the engine's suite, and the engine repeats all three measurements itself at handover.
+  Building an agent's chat model from a model profile, with tracing and usage metering, goes on `rag_wright.api`
+  first (PS-36), so the product's model code is deleted rather than moved.