rag-wright 0.12.0__tar.gz → 0.13.0__tar.gz

{rag_wright-0.12.0 → rag_wright-0.13.0}/.claude/skills/building-an-ingestion-capability/SKILL.md

@@ -154,6 +154,10 @@ category.
 
 ## 5. Spreadsheets, tables and embedded files
 
+- **Blank PDF pages** are read as empty, never sent to the vision model (engine 0.13.0). On a parse cache built
+  by an earlier engine with `VISION_OCR` configured, run `find_blank_pages_with_text(cache_dir, sources)` once:
+  delete each finding's `cache_files` and re-ingest that document (its blank pages may hold invented text).
+
 - **Hidden sheets** are ingested by default; `IngestSource(include_hidden_sheets=False)` skips them (listed in
   `DocumentReport.skipped_hidden_sheets`).
 - **Blank rows inside a table**: the parse cuts a sheet's table at every blank row; the engine glues a piece back

{rag_wright-0.12.0 → rag_wright-0.13.0}/.claude/skills/specifying-a-product-on-the-engine/SKILL.md

@@ -58,7 +58,7 @@ Every product item lands in one or more of these. The route column is where the
 | **Domain knowledge**: vocabulary, schema, constraints, synonyms, decision criteria | "controlled vocabularies", "the ontology", "mandatory fields", "valid ranges" | `EngineConfig(pack=...)` (the store creates the declared types on `open_workspace`); your capabilities read value sets and SHACL shapes with `rdflib` / `pyshacl` | `authoring-a-capability` | ontology authoring |
 | **Ingestion**: documents, spreadsheets, images, embedded files into the graph | "parse the forms", "load the lab reports", "idempotent re-runs" | `build_ingestion` (your extractor + optional hooks) → `aingest`; `IngestSource`, `IngestionTuning`, `table_rows` / `TableRow`, `evaluate_ingestion`, `check_extraction`; `KgNode` / `KgEdge` | `building-an-ingestion-capability` | KG construction |
 | **Entity and registry resolution** | "resolve against the yarn register", "queue unresolved mentions" | your `document_hook`; the entity-resolution building blocks on `rag_wright.pack_sdk` | `building-an-ingestion-capability` | entity resolution |
-| **A typed query or retrieval** | "find similar cases", "filter the table", "the dossier" | a query capability of your pack composing the primitives (`kg_read`, `kg_edges`, `NOT_NULL`, hybrid search from `rag_wright.pack_sdk`), registered with `CapabilityManifest` + `register_capability` / `load_pack`, invoked with `ainvoke_subgraph` | `authoring-a-capability` | authoring capabilities |
+| **A typed query or retrieval** | "find similar cases", "filter the table", "the dossier" | a query capability of your pack composing the primitives (`kg_read`, `kg_edges`, `NOT_NULL`, `Range`, hybrid span search with `span_search` from `rag_wright.api`), registered with `CapabilityManifest` + `register_capability` / `load_pack`, invoked with `ainvoke_subgraph` | `authoring-a-capability` | authoring capabilities |
 | **A cited answer, or a refusal** | "every answer cites its source", "say when it is not in the records" | `agenerate_answer` over `EvidenceItem`s (an answer states its support; "none" abstains), `ajudge_spans` | `authoring-a-capability` | concepts (answers) |
 | **A typed decision or classification** | "classify the structure", "which of these is it" | `classifier-opportunity-analysis` first; then rules, a trained classifier, or the decision model (`jev_decision` from `engine_capabilities()`, criteria in the pack) | `classifier-opportunity-analysis` → `setfit` / `laya` | classification and decision models |
 | **A deterministic workflow behind a button** (mode **D**) | "upload, confirm, save", "export CSV" | a product **operation** (one function per user operation) calling your capabilities; the route calls the operation | `building-a-product-backend` | domain-adaptation step 8 |

{rag_wright-0.12.0/src/rag_wright/.agents → rag_wright-0.13.0/.claude}/skills/using-the-rag-wright-engine/SKILL.md

@@ -27,8 +27,11 @@ open-core and domain-neutral; your product brings the domain. Hard rules:
   `EngineConfig`/`StoreConfig` (`StoreConfig.from_env()` reads `ARCADEDB_*`)/`open_workspace`
   (`temporary_workspace`/`drop_workspace` for disposable and removed databases), the invokers
   (`ainvoke_subgraph`/`invoke_model`/`ainvoke_model`), discovery (`capability_index`/`discover`), the KG accessors
-  (`kg_read` with `key_range`, `kg_write`, `kg_edges` with `NOT_NULL`, `kg_count`/`kg_delete`/`kg_update`,
-  `entities_by_name`, `span_positions`), `pack_store(ws, cls)` (build your pack's store extension over the
+  (`kg_read`, `kg_write`, `kg_edges`, `kg_count`/`kg_delete`/`kg_update`, all filtering by `where` values that
+  are equality, a list for membership, `NOT_NULL` or a `Range(gt=, ge=, lt=, le=)` on any number of fields,
+  evaluated in the store;
+  `entities_by_name`, `span_positions`), hybrid span search (`span_search` / `aspan_search`: cited spans from
+  the dense and sparse index, fused in the store), `pack_store(ws, cls)` (build your pack's store extension over the
   workspace's store; the store itself stays private), metering and tracing (`measure_usage`, `record_usage`,
   `traced_run`/`traced_step`, and for code emitting its own generations `tracing_on`/`trace_level`/
   `start_generation`/`finish_generation`/`flush_tracing`), the query embedder (`ws.embedder` where you hold a

rag_wright-0.13.0/.release-please-manifest.json

@@ -0,0 +1,3 @@
+{
+  ".": "0.13.0"
+}

{rag_wright-0.12.0 → rag_wright-0.13.0}/CHANGELOG.md

@@ -4,6 +4,21 @@ All notable changes to RAG_Wright are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project aims to follow
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.13.0](https://github.com/fzaidi-dai/RAG_Wright/compare/0.12.0...0.13.0) (2026-10-15)
+
+
+### Features
+
+* **PS-35:** migrate a pre-0.2.0 database's Span fields through rag_wright.api ([856f956](https://github.com/fzaidi-dai/RAG_Wright/commit/856f956f674c731d170878d44ec3fef980a10ba9))
+* **PS-44:** range filters on any number of KG fields, evaluated in the store (TexWright ER-1) ([cc62c49](https://github.com/fzaidi-dai/RAG_Wright/commit/cc62c49000161567869d46127bb158db16106c28))
+* **PS-45:** a public hybrid span search on rag_wright.api (engine gap G24) ([4aa802f](https://github.com/fzaidi-dai/RAG_Wright/commit/4aa802fdc6cf12a3c732aa7a41e4de6240dd05da))
+* **PS-47:** find cached parses holding text a VLM invented for a blank page ([ff92d81](https://github.com/fzaidi-dai/RAG_Wright/commit/ff92d81956590002418ef50f41dee54e0b0b6ef8))
+
+
+### Bug Fixes
+
+* **PS-46:** a page with no ink is blank, never escalated to the VLM (TexWright 2026-10-15) ([fc633a8](https://github.com/fzaidi-dai/RAG_Wright/commit/fc633a84aa52a89de5b79d90b50d80f7d622d792))
+
 ## [0.12.0](https://github.com/fzaidi-dai/RAG_Wright/compare/0.11.0...0.12.0) (2026-10-15)
 
 

{rag_wright-0.12.0 → rag_wright-0.13.0}/CLAUDE.md

@@ -57,7 +57,7 @@ The failure mode to avoid, which has happened before: a command fails with "modu
 
 > **CURRENT STATE (2026-10-15): `rag-wright` 0.11.0 is RELEASED** (PR #13: PS-40 the product-building skills ship,
 > PS-41 the skills audit, PS-42 by-slug registration; RuleWright's guide `docs/specs/public-surface/rulewright-migration-0.11.0.md`).
-> **PS-43 done, unreleased (next: 0.12.0)**: `worker_spec(definition=)`, `docs/harness.md` (tables generated by
+> **PS-43 RELEASED as 0.12.0 (2026-10-15, PR #14)**: `worker_spec(definition=)`, `docs/harness.md` (tables generated by
 > `scripts/build_harness_doc.py`), the harness source cleared of RuleWright's vocabulary with a guard; guide
 > `rulewright-migration-0.12.0.md`. Before it,
 > **(2026-10-14): PS-32 done: the agent harness is the engine's third public tier `rag_wright.harness`**

{rag_wright-0.12.0 → rag_wright-0.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: rag-wright
-Version: 0.12.0
+Version: 0.13.0
 Summary: A domain-retargetable, open-core engine for hybrid retrieval + knowledge-graph RAG: parsing, chunking, embedding, hybrid search, reranking, graph extraction, entity resolution, ontology-driven knowledge, and a capability runtime.
 Project-URL: Homepage, https://github.com/fzaidi-dai/RAG_Wright
 Project-URL: Repository, https://github.com/fzaidi-dai/RAG_Wright

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/ArcadeDB_Local.md

@@ -60,7 +60,7 @@ typed edges, and `ComplianceStore` creates `Requirement`, so they appear the fir
 
 A database created before the `Span` field rename (`contract_id`, `function`, `functions` became `document_id`,
 `primary_tag`, `tags`) is refused by `ensure_schema` with a `RuntimeError`; migrate it once with
-`uv run python -u scripts/migrate_span_fields.py <database>`.
+`migrate_span_fields(config, corpus=...)` from `rag_wright.api` ([installation](installation.md)).
 
 ## Stop / reset
 

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/adr/0062-tiered-ocr-scan-quality-vlm-escalation.md

@@ -84,3 +84,14 @@ now, a self-hosted vLLM-Gemma-4 later for data sovereignty (only `base_url` chan
   as the escalation provider for data sovereignty; (3) routing the vision call **through the async model seam**
   for true thread cancellation (today `to_thread` unblocks the caller at the deadline but the docling thread
   runs on); (4) classical preprocessing stays available as a cheap first pass but is not the heavy-scan fix.
+
+## Amendment (2026-10-15, PS-46): a blank page is not degraded
+
+A page with no ink read as both "blurry" (a white page has Laplacian variance 0) and "faint" (no pixel below 128),
+so it was escalated, and a VLM asked to transcribe an empty page invents one (TexWright: meeting minutes in another
+language, kept in the parse). The gate now measures `ink_frac` (pixels below 200, faint grey included) and returns
+`ScanQuality.BLANK` for a page with no OCR text and `ink_frac` at most 1e-4: never escalated, never flagged
+needs-rescan, listed in `TieredOCRReport.blank_pages`. Measured margin: blank 0; the heavy faint scan 0.077-0.082
+(its `dark_frac` is 0.0025, which is why `dark_frac` alone cannot separate it); the sparsest real fixture page
+0.019. The post-escalation check stays a flag rather than dropping the VLM text: it is an English word-rate test,
+and a correct transcription of a page with few common words (a number-heavy table) fails it.

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/api/README.md

@@ -68,6 +68,10 @@ Members: `STRUCTURED_REASONING` (`'structured_reasoning'`), `STRUCTURED_REASONIN
 
 One ranked capability match from `discover` — enough for an agent to pick and invoke it by `slug`.
 
+### `SpanHit(*, span_id: str, text: str, document_id: str, doc_start: Optional[int] = None, doc_end: Optional[int] = None, pages: list[int] = [], bbox: Optional[tuple[float, float, float, float]] = None, primary_tag: str = '', rank: int, found_by: tuple[str, ...]) -> None`
+
+One cited span from `span_search`, best first. `found_by` says which legs found it: `"hybrid"` (the fused dense + sparse search) and/or `"dense"` (the dense floor). `bbox` is `(l, t, r, b)` or None.
+
 ### `KgNode(type: 'str', key_field: 'str', props: 'dict[str, object]') -> None`
 
 A typed KG node to upsert (DD-1b, ADR-0117): `type` is the vertex type, `key_field` the identity field to upsert on, `props` the fields (including `key_field`) as DOMAIN-NATIVE values. The store encodes each prop by its pack-declared storage type -- the caller never serializes to the store's wire format.
@@ -76,6 +80,18 @@ A typed KG node to upsert (DD-1b, ADR-0117): `type` is the vertex type, `key_fie
 
 A typed KG edge to create between two nodes identified by (type, key_field, key). `props` are native values (edge properties are type-driven: edges declare no storage schema).
 
+### `Range(gt: 'Optional[object]' = None, ge: 'Optional[object]' = None, lt: 'Optional[object]' = None, le: 'Optional[object]' = None) -> None`
+
+A `where` value meaning a comparison, evaluated in the store (ER-1): `gt` (>), `ge` (>=), `lt` (<), `le` (<=), at most one lower and one upper bound, at least one bound. Bounds are scalars (numbers compare as numbers, strings lexically). Any number of fields may carry one, beside equality, membership and `NOT_NULL`, on every KG call that takes `where`: `where={"weight": Range(ge=180, le=220), "tensile": Range(gt=400)}`.
+
+#### `Range.terms(self) -> 'list[tuple[str, object]]'`
+
+`[(operator, bound), ...]`, lower bound first.
+
+### `BlankPageText(*, source: str, pages: dict[int, str], cache_files: list[str]) -> None`
+
+One cached parse with text on blank pages. `source` is the PDF's path (or the name given with its bytes), `pages` maps each blank page to the text the cached parse holds there, and `cache_files` are the cached parse(s) and their OCR sidecar: delete them, then re-ingest the document so it is parsed again.
+
 ### `EvidenceItem(*, chunk_id: str, text: str, confidence: Optional[str] = None, properties: Optional[list[dict]] = None) -> None`
 
 One piece of grounding evidence: a chunk's text, its `chunk_id` (the citation), and — for a graph-derived fact — its confidence tag (surfaced to the generator, FR-S.4).
@@ -296,7 +312,7 @@ The semantic boundary-discovery seam: decide chunk boundaries as a partition of
 
 ### `open_workspace(config: 'EngineConfig', *, corpus: 'str', reset: 'bool' = False, create: 'bool' = True) -> 'WorkspaceHandle'`
 
-Resolve (and cache) the workspace for `corpus` (the backend database name) from `config`. Ensures the schema: the neutral engine types, plus `config.pack`'s declared types when set. Raises `RuntimeError` on a database whose `Span` type still has the pre-ING-8d field names (migrate it with `scripts/migrate_span_fields.py`). Returns an opaque `WorkspaceHandle`. `reset=True` drops + recreates the database (test/clean-slate) and bypasses the cache. A missing database is created (logged): that is how a new corpus is provisioned. `create=False` raises `LookupError` instead (PS-27) -- use it wherever the corpus must already exist, so a wrong name fails loudly rather than opening an empty graph.
+Resolve (and cache) the workspace for `corpus` (the backend database name) from `config`. Ensures the schema: the neutral engine types, plus `config.pack`'s declared types when set. Raises `RuntimeError` on a database whose `Span` type still has the pre-ING-8d field names (migrate it with `migrate_span_fields`). Returns an opaque `WorkspaceHandle`. `reset=True` drops + recreates the database (test/clean-slate) and bypasses the cache. A missing database is created (logged): that is how a new corpus is provisioned. `create=False` raises `LookupError` instead (PS-27) -- use it wherever the corpus must already exist, so a wrong name fails loudly rather than opening an empty graph.
 
 One handle per store and corpus is cached for the process. An equal `config` returns it; a different one (new models, options, credentials or pack) returns a new handle that replaces it from then on, so a configuration change takes effect without a restart (PS-16). The new handle reuses the store when the store settings and the pack are unchanged, and the query embedder when the embedding profile is unchanged; a call still holding the old handle finishes on the old config.
 
@@ -304,6 +320,10 @@ One handle per store and corpus is cached for the process. An equal `config` ret
 
 Drop the database `corpus` (PS-37): everything in it, the retrieval index and the knowledge graph, is deleted and cannot be recovered. Returns True if it existed, False if there was nothing to drop. A cached handle for it is closed and forgotten, so a later `open_workspace` starts fresh. A missing or blank `corpus` raises `ValueError`: the name is always explicit, never a default database.
 
+### `migrate_span_fields(config: 'EngineConfig', *, corpus: 'str', batch: 'int' = 5000, progress: 'Optional[Callable[[int, int], None]]' = None) -> 'int'`
+
+Migrate the database `corpus` from the pre-0.2.0 `Span` field names (`contract_id`, `function`, `functions`) to the generic ones (`document_id`, `primary_tag`, `tags`), the database `open_workspace` refuses until this has run (PS-35). Values are copied in batches of `batch` and the old properties dropped; `progress(done, total)` is called after each batch. Idempotent: returns the number of records moved, 0 when already migrated. The database must exist (`LookupError` otherwise; it is never created), and a missing or blank `corpus` raises `ValueError`. Afterwards the schema is checked, so a database this returns for opens with `open_workspace`.
+
 ### `temporary_workspace(config: 'EngineConfig', *, prefix: 'str' = 'tmp') -> 'Iterator[WorkspaceHandle]'`
 
 A workspace on a fresh database, `<prefix>_<12 hex>`, dropped when the block exits, whether it succeeds or raises (PS-37). For a probe or a measurement whose database must not outlive it: `with temporary_workspace(config, prefix="probe") as ws: ...`.
@@ -386,7 +406,7 @@ Rank the live ARD catalog by semantic match to `query`; return the top `k` (opti
 
 ### `kg_read(ws: 'WorkspaceHandle', node_type: 'str', *, where: 'Optional[dict]' = None, fields: 'Optional[list]' = None, distinct: 'Optional[str]' = None, order_by: 'Optional[str]' = None, limit: 'Optional[int]' = None, key_range: 'Optional[tuple]' = None) -> 'list[dict]'`
 
-Read typed nodes of `node_type` from the workspace (see `Store.kg_read`). Equality/`IN` filters, a `key_range=(field, lo, hi)` range, projection, distinct, order, limit; an empty list `where` value is scope-to-nothing -> `[]`.
+Read typed nodes of `node_type` from the workspace (see `Store.kg_read`). Each `where` value is a scalar (equality), a list (membership; an empty list is scope-to-nothing -> `[]`), `NOT_NULL` (presence) or a `Range` (comparisons, on any number of fields: `{"weight": Range(ge=180, le=220), "tensile": Range(gt=400)}`); all filters are AND-ed and evaluated in the store. `key_range=(field, lo, hi)` is the half-open shorthand (`lo <= field < hi`). Then projection, distinct, order and limit.
 
 ### `kg_write(ws: 'WorkspaceHandle', nodes: 'list', edges: 'Any' = ()) -> 'None'`
 
@@ -398,7 +418,7 @@ Generic edge TRAVERSAL over the workspace (see `Store.kg_edges`): node-start out
 
 ### `kg_count(ws: 'WorkspaceHandle', type_name: 'str', *, where: 'Optional[dict]' = None, key_range: 'Optional[tuple]' = None) -> 'int'`
 
-The number of vertices or edges of `type_name` matching `where` / `key_range` (see `Store.kg_count`).
+The number of vertices or edges of `type_name` matching `where` / `key_range` (see `Store.kg_count`); `where` as in `kg_read`, `Range` values included.
 
 ### `kg_delete(ws: 'WorkspaceHandle', type_name: 'str', *, where: 'Optional[dict]' = None, key_range: 'Optional[tuple]' = None) -> 'int'`
 
@@ -416,6 +436,14 @@ Resolve an entity NAME to every entity node it matches: `[{entity_id, name, enti
 
 Every span of `document` with its position provenance, ordered by document position. Each row: `span_id`, `parent_chunk_id`, `span_index`, `text`, `primary_tag` (the span tagger's primary tag, "" if untagged), `document_id`, `doc_start` / `doc_end` (document-absolute offsets), `pages`, and `bbox` DECODED to a `(l, t, r, b)` tuple or None. The engine MECHANISM behind a product's citation/highlight types -- the product wraps these rows into its own presentation type (e.g. `SpanLocation`).
 
+### `span_search(ws: 'WorkspaceHandle', query: 'str', *, k: 'int' = 10, documents: 'Optional[list]' = None, tag: 'Optional[str]' = None, dense_floor: 'int' = 3) -> 'list'`
+
+Hybrid search over the workspace's span index (every ingested span, embedded dense and sparse): the two legs fused by Reciprocal Rank Fusion in the store, with the top `dense_floor` pure-dense spans guaranteed a place, so a strong semantic match a common-token query would push out of the fused pool is kept (0 turns it off). Returns up to `k` `SpanHit`s, best first, each citation-ready: `span_id`, `text`, `document_id`, `doc_start` / `doc_end`, `pages`, `bbox`, `primary_tag`, `rank`, and `found_by` (`"hybrid"` and/or `"dense"`). `documents` scopes it to those source documents in the store (`None` = all, `[]` = nothing); `tag` keeps one primary tag. No reranking: compose `BGEReranker` over the hits if you want one. Raises `RuntimeError` when the workspace has no embedder.
+
+### `aspan_search(ws: 'WorkspaceHandle', query: 'str', *, k: 'int' = 10, documents: 'Optional[list]' = None, tag: 'Optional[str]' = None, dense_floor: 'int' = 3) -> 'list'`
+
+`span_search` off the event loop (the query embedding and the store calls block), for an async caller.
+
 ### `document_of(entity_id: 'str') -> 'str'`
 
 The source-document id embedded in a chunk/span/unit id (`<source_doc_id>:<idx>:<hash>` -> the first, delimiter-safe segment). Empty in -> empty out.
@@ -448,6 +476,10 @@ The number of pages in a PDF (a path or the PDF's bytes), read under the engine'
 
 The async, deadline-bounded twin of `parse_document_bytes` (ADR-0057): runs the docling parse off the event loop.
 
+### `find_blank_pages_with_text(cache_dir: 'Any', sources: 'Iterable[Source]' = (), *, progress: 'Optional[Callable[[int, int], None]]' = None) -> 'list[BlankPageText]'`
+
+Audit the parse cache under `cache_dir` (the one given to `parse_document`, or to `aingest`, which keeps its parses in `parsed/`; searched recursively) for text on blank pages. `sources` are the documents' PDFs, as paths or `(name, bytes)`; files embedded in a parsed package are stored in the cache and audited without being passed. A source that is not a PDF, or has no cached parse, is skipped. `progress(done, total)` is called after each PDF. Raises `FileNotFoundError` when `cache_dir` does not exist.
+
 ### `agenerate_answer(query: 'str', evidence: 'list[EvidenceItem]', *, ws: 'Any', guidance: 'Optional[str]' = None) -> 'GeneratedAnswer'`
 
 A grounded, cited answer to `query` over `evidence`, or an abstention. Empty evidence abstains without a model call; a citation not in the evidence is dropped, and an answer left with no valid citation becomes an abstention (no claim without a citation). `answer_kind` says whether the evidence fully supported the answer (`answered`), only partly (`partial`) or not at all (`abstained`). `guidance` adds your domain's wording to the domain-neutral method.

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/configuration.md

@@ -109,6 +109,11 @@ and how to manage workspaces.
   deployment serves a single model; OCR resolves its endpoint and flags through the same profile as every other
   call. Override one role with `RAG_MODEL_<ROLE>` (e.g. `RAG_MODEL_VISION_OCR`, `RAG_MODEL_GENERAL`), or every
   role at once with `RAG_MODEL_ALL`; the role-specific variable wins.
+- **Blank PDF pages** are never sent to `VISION_OCR` (0.13.0, PS-46): a page with no ink has nothing to read, and a
+  vision model asked to transcribe one invents a page. A parse cached by an earlier engine with `VISION_OCR`
+  configured may still hold such text: `find_blank_pages_with_text(cache_dir, sources)` lists every page that is
+  blank in the source PDF but has text in its cached parse (files embedded in a parsed package are found in the
+  cache without being passed). Delete each finding's `cache_files`, then re-ingest the document.
 - **Structured output** is client-side tag-parse (ADR-0045); any provider flags live in the profile.
 - **A self-hosted Qwen server**: see [A self-hosted model server](#a-self-hosted-model-server-qwen-on-modal) below.
 - **Decision models** (e.g. Jev) are reached through a `DecisionModelProfile` (default `jev-1.13`;

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/domain-adaptation/README.md

@@ -42,7 +42,7 @@ ADR-0117/0118 engine API + capability runtime). Read [Concepts](../concepts.md)
    (segmenter, span tagger, unit grouper, boundary decider, writer, a per-document hook)
    → [KG construction](kg-construction.md) and the `building-an-ingestion-capability` skill. Your query graphs, domain functions, and skills **compose the engine's
    generic primitives from the public tiers** (graph reads and query, embedding, typed ranking, reranking,
-   answers, parsing, chunking; hybrid search and fusion are not public yet, gap G24) and register only your OWN domain graphs/models/skills via `register_capability(manifest)` with an
+   answers, parsing, chunking, and hybrid span search with `span_search`) and register only your OWN domain graphs/models/skills via `register_capability(manifest)` with an
    `impl_ref`, invocable (and MCP-exposable) with zero engine edits. Package them as a pack module that exposes
    `register()` and load it with `load_pack("<your module>")`.
    → [Authoring capabilities](authoring-capabilities.md).

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/domain-adaptation/_engine-gaps.md

@@ -190,7 +190,7 @@ they are the real "does a new customer benefit?" items. The genuine engine ones
 - **Table rows:** `table_rows(source_document)` returns every parsed table's rows as exact cells (`TableRow`), and a
   one-row unit carries its row as `Unit.table_row` (ING-7).
 
-### G24: No public hybrid search or fusion primitive
+### G24: No public hybrid search or fusion primitive (closed)
 - **Where:** the engine's own retrieval legs run hybrid (dense + sparse) span search and rank fusion internally, and
   the authoring guide used to say a capability composes them "by direct import"; neither is on `rag_wright.api` or
   `rag_wright.pack_sdk`.
@@ -199,6 +199,10 @@ they are the real "does a new customer benefit?" items. The genuine engine ones
   what it embeds itself, beside a structured leg (`kg_read`, `typed_constraint_match_rank`).
 - **Proposed:** a public hybrid span search over a workspace (query text, filters, k) returning cited spans, with the
   fusion and reranking it already applies; on `rag_wright.pack_sdk` for pack capabilities.
+- **Closed by PS-45:** `span_search(ws, query, *, k, documents, tag, dense_floor)` and `aspan_search` on
+  `rag_wright.api` (which pack code may also import): the store-side RRF fusion of the dense and sparse span
+  legs plus the dense floor (ADR-0104), returning citation-ready `SpanHit`s. Reranking stays the caller's
+  (`BGEReranker`), as it was in the engine's own leg.
 - Surfaced: the TexWright specification (2026-10-14).
 
 ## Prerequisites a domain provides (not engine gaps)

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/domain-adaptation/authoring-capabilities.md

@@ -114,9 +114,10 @@ slug. Write good `representative_queries` (2–5, phrased like real tasks) — t
 Your capabilities **compose the engine's generic primitives from the public tiers** (`rag_wright.api` and
 `rag_wright.pack_sdk`: the knowledge-graph reads `kg_read` / `kg_edges`, graph query, embedding (`ws.embedder`,
 `cosine`), typed ranking (`typed_constraint_match_rank`), reranking (`BGEReranker`), answers (`agenerate_answer`,
-`ajudge_spans`), parsing and chunking); you register only your OWN domain graphs/models/skills. Don't wrap or
-re-register an engine primitive. Hybrid (dense + sparse) span search and rank fusion are not public yet (engine gap
-G24): check a primitive is on a tier before designing around it.
+`ajudge_spans`), hybrid span search (`span_search` / `aspan_search`: the dense and sparse legs fused in the store,
+with a dense floor, returning cited spans), parsing and chunking); you register only your OWN domain
+graphs/models/skills. Don't wrap or re-register an engine primitive, and check a primitive is on a tier before
+designing around it.
 
 ## Definition of done
 

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/installation.md

@@ -131,13 +131,20 @@ To serve open models yourself instead, set `RAG_SERVING=vllm` with `VLLM_BASE_UR
 
 `open_workspace` (through `ensure_schema`) refuses a database whose `Span` type still has the old field names
 (`contract_id`, `function`, `functions`; renamed to `document_id`, `primary_tag`, `tags`) and raises a `RuntimeError`
-naming the fix. Migrate each such database once, in place (it connects with the `ARCADEDB_*` settings in `.env`; it
-is idempotent and streams `X/N` progress):
+naming the fix. Migrate each such database once, in place, with `migrate_span_fields` from `rag_wright.api`. It is
+idempotent (it returns the number of spans moved, 0 when already migrated), never creates a database, and reports
+progress after each batch:
 
-```sh
-uv run python -u scripts/migrate_span_fields.py <database>
+```python
+from rag_wright.api import EngineConfig, StoreConfig, migrate_span_fields
+
+config = EngineConfig(store=StoreConfig.from_env())
+moved = migrate_span_fields(config, corpus="my_corpus",
+                            progress=lambda done, total: print(f"[migrate] {done}/{total}"))
 ```
 
+From an engine checkout, `uv run python -u scripts/migrate_span_fields.py <database>` does the same.
+
 Every other breaking change since 0.1.0, with the exact change a consumer makes, is in
 [`specs/ingestion-hooks/ing8-breaking-changes.md`](specs/ingestion-hooks/ing8-breaking-changes.md).
 

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/specs/public-surface/plan.md

@@ -107,7 +107,7 @@ in two at the PS-6 gate so the store rewrite is its own visible task. One task,
 | PS-32 | (waits on RuleWright) Adopt the agent harness substrate as `rag_wright.harness`, a third public tier (ADR-0139). RuleWright separates the substrate from its product parts in its own repo and hands over a package that imports nothing from the product; the engine takes it in as it stands, with import contracts (the harness reaches engine code only through `rag_wright.api`; no engine module imports the harness), the domain-vocabulary guard, no Postgres extra (the product passes its checkpointer in; ADR-0139 correction), RuleWright's third guard (deployment, vendor, tenancy shape; its handover version, which also matches field declarations) in the engine suite, RuleWright's greenfield conformance test (a second product on the seams; strict expected failures that may only shrink), the run context as the product's own type against a `scope` protocol (ADR-0139 addendum), and the three measurements repeated here, `rag_wright.harness.__all__` declared (RuleWright pins it), `tracing_on()` and a flush on `rag_wright.api` (the harness may import only `api`), and the handover shapes agreed 2026-10-13 (RuleWright's `2026-10-13-what-to-build-for-rag_wright-harness.md`): the query encoder is `api.query_embedder()` built once per process; run resources are an opaque product value passed to the product's worker factories (`RunResources` and `run_resources.py` stay in the product, which declares its tool names); the harness defines its own deadline exceptions and `error_middleware` takes `retry_on` and `message_for` from the product; tracing through `api.traced_run`/`traced_step`; the agent model through `api.agent_chat_model` with the model id passed in, the `building-an-agent-harness` skill moved into the shipped skills, docs, and a RuleWright migration guide. No new protocols for future products | the harness's own tests pass in the engine; import contracts and the vocabulary guard; a plain install imports without the extra; live: a chat run with an interrupt and a resume | done 2026-10-14 ( `rag_wright.harness` 26 modules with `__all__` (42 names); `AgentDefinition`/`AgentPrompts`/`ToolSurface` made the tier's; `agent_chat_model(trace=False)` and `tracing_on`/`trace_level`/`start_generation`/`finish_generation`/`flush_tracing` on `api`; two import contracts; the vocabulary guard clean; conformance 7/7, genericity guard, 39 harness tests; live interrupt-and-resume on `qwen3.8-27b-modal-or` (2 calls, $0.00038); `langchain`/`langchain-core` declared; `GenerationRecorder` fixed: only a retry of the same turn abandons an attempt, a failed call closes as an error) |
 | PS-33 | (later; its own workstream when agreed) Agentic capabilities: an `agent` capability kind (amends ADR-0041) for a harness compiled to a LangGraph graph and registered under a capability name; typed contracts at the boundary (inputs to messages and context, `response_format` back to the contract, the graceful-degrade rule); interrupts across the capability boundary (a suspended result with a resume handle, declared in the manifest); deadlines, budgets and dispatch limits passed down nested agents; a worked example on public data with an eval | to be planned | todo (later) |
 | PS-34 | RuleWright's 25 databases leave the engine's ArcadeDB (they sit in the engine checkout's `data/arcadedb`, served by the engine's container, which RuleWright's `scripts/arcadedb.sh` also manages by name). The product owns its store and its data, the knowledge graph included; the docs were corrected 2026-10-12 (`installation.md`, `ArcadeDB_Local.md`). RuleWright moves them to its own container, port and data directory per `rulewright-arcadedb-separation.md`; the engine then deletes its copies (user's go-ahead), decides `some-db`, and restores its container's own settings | RuleWright: every moved database opens with `create=False`, its live store tests pass; engine: `data/arcadedb` holds engine databases only | done 2026-10-13 (the engine dropped `rulewright_dev`, `rulewright_compliance` and `some-db` with `drop_workspace`; six engine databases remain) |
-| PS-35 | (small) The pre-0.2.0 `Span` field migration is reachable only through `scripts/migrate_span_fields.py`, which is not in the wheel, and `ArcadeDBStore.migrate_span_fields` is internal; `installation.md` tells a product to run the script. Put the migration on `rag_wright.api` (or state that it needs an engine checkout) | a hermetic test of the public call; `installation.md` names it | todo |
+| PS-35 | (small) The pre-0.2.0 `Span` field migration is reachable only through `scripts/migrate_span_fields.py`, which is not in the wheel, and `ArcadeDBStore.migrate_span_fields` is internal; `installation.md` tells a product to run the script. Put the migration on `rag_wright.api` (or state that it needs an engine checkout) | a hermetic test of the public call; `installation.md` names it | done 2026-10-15: `migrate_span_fields(config, *, corpus, batch, progress)` on `rag_wright.api` (opens the store without the schema check, never creates the database, checks the schema after); the refusal messages and docs name it; the script wraps it |
 | PS-36 | An agent's chat model from a model profile on `rag_wright.api` (ADR-0139 correction): the seam's builder is internal today (`models/seam.py:build_model`), so RuleWright's harness built its own and named a model and a vendor in code. Public, profile-driven, with tracing and `usage_scope` metering; precedes RuleWright's `models.py` stage of the separation | hermetic tests (profile resolution, metering, tracing); live: one agent call through a profile | done 2026-10-13 (`agent_chat_model`; profile slot `agent_extra_body`; live: a deep agent on `qwen3.8-27b-modal-or`, 2 calls metered with real cost) |
 | PS-37 | The public surface can drop a database it created (RuleWright request 2026-10-13: a create-only lifecycle leaks, and its only workarounds reach past the engine to the store). `drop_workspace(config, *, corpus) -> bool` (explicit name only, never a default database; closes and forgets the cached handle) and `temporary_workspace(config, *, prefix=)` (a fresh `<prefix>_<12 hex>` corpus dropped on exit, also on error); `ArcadeDBStore.drop_database` underneath | hermetic tests; live: create, write, drop on exit, `create=False` refuses after | done (2026-10-13): `tests/api/test_drop_workspace.py` (9) + `test_drop_workspace_live.py` (live, nothing left behind); `docs/workspaces.md` "Removing a workspace"; API reference regenerated; full suite 2169 passed. Housekeeping: `some-db` (empty, from before PS-27) dropped; the engine's container recreated on its own settings (6 GB heap, no cap) after RuleWright's script had set 3 GiB/2 GB on it; two stopped containers removed |
 | PS-38 | Close the public-surface gaps a new product (TexWright) hits, from the 2026-10-13 product-starter review (`_engine-gaps.md`): (a) ARD publication on `rag_wright.api` (`publish`/`publish_all` with a target root, G14); (b) `CapabilityInterface` and `SkillRuntime` on `rag_wright.pack_sdk`, so a product can fill those manifest fields (G23); (c) a public way to register a `DecisionModelProfile` (and a model profile), so a Laya or on-premises decision model is config, not an engine edit (G12); (d) a public accessor for the workspace's configured embedder, and the reference pack moved onto it and onto `pack_store` (no `resources._store`/`_embedder`) (G22); (e) `StoreConfig.from_env()` no longer defaults `ARCADEDB_PORT` to 2480, so an unset variable cannot reach another instance (it raises naming the variable); the skills and docs that name these gaps updated | hermetic tests per item; the import contracts and doc-reference guard green; RuleWright's suite checked for (e) before it lands (it sets `ARCADEDB_PORT`); a live publish of a product manifest to a temporary root | done 2026-10-13 (publish/publish_all and the profile registrations on `api`; CapabilityInterface, SkillRuntime, workspace_embedder on `pack_sdk`; `WorkspaceHandle.config`; reference pack off private attributes; `ARCADEDB_PORT` required; 6 live reference tests green) |
@@ -116,3 +116,7 @@ in two at the PS-6 gate so the store rewrite is its own visible task. One task,
 | PS-41 | Audit the engine-authored product-developer skills and the domain-adaptation guide for omissions a greenfield product hits (TexWright as the test case), and fix them: the specifying skill (capability catalog + definition-of-done requirement, models, the operations registry and parity, the agent's guardrails, settled data semantics, confirming a primitive is public, a live reconciliation); the consumer skill (three tiers, the harness first, specify first, client data, the surface snapshot over all tiers); capability authoring (product homes and slug prefixes, the eval in the definition of done, the harness for agents); ingestion (images and scans, a model-backed extractor as a closure, interpretations as records, per-sheet roles, multi-row records, re-ingest under G8, registering the pipeline, field-level accuracy); evals (gold from provided data, field-level ingestion accuracy, judged gold, agent behavior classes); the backend skill (the operations registry, parity, read-only agent tools); the guide (step 0 specify, the bridge pack for an existing ontology, G24: no public hybrid search, the false 'compose hybrid search by import' claim removed, a `uv pip` fixed) | the doc-reference, shipped-skills and ingestion-example guards green | done 2026-10-14 |
 | PS-42 | Register only the engine capabilities a product uses, by slug (TexWright T-0.2 finding): the product-starter templates, the consumer skill and the domain-adaptation docs showed a blanket `for m in engine_capabilities(): register_capability(m)`, which puts unused capabilities into `discover()`'s ranking pool; now by slug, failing loudly when a used slug is no longer offered | the doc-reference and shipped-skills guards green | done 2026-10-14 |
 | PS-43 | The harness tier's design reference and its quiet defaults (RuleWright's 2026-10-15 note): `docs/harness.md` taken in from RuleWright's draft, its tables generated from the live tier by `scripts/build_harness_doc.py`; `worker_spec(definition=)` fills the seven policy parameters (and the signposts) a worker otherwise left inert from the product's `AgentDefinition`, explicit arguments winning; the harness source's docstrings and comments cleared of the originating product's vocabulary, task, requirement and ADR ids, with a guard; two `AgentPrompts` fields described; the agent-harness skill's 'Four declared facts' corrected to fifteen | the product-text guard, the `definition=` conformance tests and the harness-design drift test green | done 2026-10-15 |
+| PS-44 | TexWright ER-1: range filters on any number of fields, evaluated in the store. `Range(gt=, ge=, lt=, le=)` on `rag_wright.api` is a `where` value beside equality, membership and `NOT_NULL`, on every KG call that takes `where` (`kg_read`, `kg_count`, `kg_delete`, `kg_update`, `kg_edges`); `kg_read` now shares the filter grammar (so `NOT_NULL` works there too); `key_range` stays as the half-open shorthand | hermetic SQL-contract tests and a live test of two numeric ranges together | done 2026-10-15 |
+| PS-45 | Engine gap G24: a public hybrid span search. `span_search(ws, query, *, k, documents, tag, dense_floor)` and `aspan_search` on `rag_wright.api`, over the span index ingestion fills: the store-side RRF fusion of the dense and sparse legs plus the dense floor (ADR-0104), returning citation-ready `SpanHit`s (text, document, offsets, pages, box, `found_by`); no reranking (the caller composes `BGEReranker`). G24 closed; the guide, the authoring page and the specifying and consumer skills name it | hermetic tests of the composition and the public call; a live test on real ArcadeDB and BGE-M3 (found by meaning, scoped, cited) | done 2026-10-15 |
+| PS-46 | TexWright's 2026-10-15 report: a blank page escalated to the VLM came back as an invented document the parse kept. The scan-quality gate gains `ScanQuality.BLANK` (no OCR text and no ink: `ink_frac`, pixels < 200, at most 1e-4; the heavy faint scan measures ~0.08) and `TieredOCRParser` never escalates or flags a blank page, reporting it in `TieredOCRReport.blank_pages`. The post-check stays a flag, not a filter (a verbatim transcription of a page with few common English words fails it). ADR-0062 amended | hermetic gate and tiered tests (a real 2-page PDF: the VLM gets page 1 only); a live docling parse reports the blank page; the heavy fixture still escalates all 10 pages | done 2026-10-15 |
+| PS-47 | The audit for PS-46: a parse cached before the fix keeps a blank page's invented text (the cache is keyed on content). `find_blank_pages_with_text(cache_dir, sources=(), *, progress)` and `BlankPageText` on `rag_wright.api` list every page blank in the source PDF (no ink, no text layer) with text in its cached parse, and the cache files to delete before re-ingesting. Searches the cache recursively (`aingest` keeps parses in `parsed/`); embedded files are audited from the cache's own copies; top-level sources are passed as paths or `(name, bytes)` | hermetic tests; a live `build_ingestion` of a workbook embedding a blank-tail PDF: clean, then a planted page found with nothing passed, then clean after delete and re-ingest | done 2026-10-15 |

{rag_wright-0.12.0 → rag_wright-0.13.0}/docs/specs/public-surface/rulewright-migration-0.12.0.md

@@ -1,7 +1,7 @@
 # RuleWright: moving to rag-wright 0.12
 
-What 0.12.0 changes for RuleWright (PS-43, from your note of 2026-10-15; unreleased until its release PR is
-merged). Do the 0.11 guide first. Nothing here is edited from the engine repo, and nothing breaks an existing
+What 0.12.0 changes for RuleWright (PS-43, from your note of 2026-10-15; released 2026-10-15: PyPI and GitHub
+release `0.12.0`). Do the 0.11 guide first. Nothing here is edited from the engine repo, and nothing breaks an existing
 call: every item is new or optional.
 
 ## 1. Build workers with `worker_spec(..., definition=...)` (PS-43)

{rag_wright-0.12.0 → rag_wright-0.13.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "rag-wright"
-version = "0.12.0"
+version = "0.13.0"
 description = "A domain-retargetable, open-core engine for hybrid retrieval + knowledge-graph RAG: parsing, chunking, embedding, hybrid search, reranking, graph extraction, entity resolution, ontology-driven knowledge, and a capability runtime."
 readme = "README.md"
 requires-python = ">=3.12,<3.13"

{rag_wright-0.12.0 → rag_wright-0.13.0}/scripts/migrate_span_fields.py

@@ -3,6 +3,7 @@
 `contract_id` -> `document_id`, `function` -> `primary_tag`, `functions` -> `tags` (values copied, old fields removed,
 old properties dropped). The legacy `parent_okf_path` locator is left in place. Idempotent: re-running on a migrated
 database moves nothing. After ING-8d, `ArcadeDBStore.ensure_schema` refuses an unmigrated database.
+A thin command over the public call `rag_wright.api.migrate_span_fields` (PS-35), which a product uses.
 
 Usage: uv run python -u scripts/migrate_span_fields.py <database> [--batch 5000]
 """
@@ -13,7 +14,7 @@ import time
 
 from dotenv import load_dotenv
 
-from rag_wright.store.arcadedb import ArcadeDBStore
+from rag_wright.api import EngineConfig, StoreConfig, migrate_span_fields
 
 
 def main() -> None:
@@ -22,16 +23,14 @@ def main() -> None:
     ap.add_argument("--batch", type=int, default=5000)
     args = ap.parse_args()
     load_dotenv(".env")
-    store = ArcadeDBStore.from_env(database=args.database)
+    config = EngineConfig(store=StoreConfig.from_env())
     t0 = time.time()
     print(f"[migrate] {args.database}: Span field rename (batch {args.batch})", flush=True)
-    moved = store.migrate_span_fields(
-        batch=args.batch,
+    moved = migrate_span_fields(
+        config, corpus=args.database, batch=args.batch,
         progress=lambda done, total: print(f"[migrate] {done}/{total} spans, {time.time() - t0:.0f}s", flush=True))
-    store.ensure_schema()  # the post-condition: the migrated database is accepted
     print(f"[migrate] done: {moved} spans moved in {time.time() - t0:.0f}s"
           + (" (already migrated)" if moved == 0 else ""), flush=True)
-    store.close()
 
 
 if __name__ == "__main__":

{rag_wright-0.12.0 → rag_wright-0.13.0}/src/rag_wright/.agents/skills/building-an-ingestion-capability/SKILL.md

@@ -154,6 +154,10 @@ category.
 
 ## 5. Spreadsheets, tables and embedded files
 
+- **Blank PDF pages** are read as empty, never sent to the vision model (engine 0.13.0). On a parse cache built
+  by an earlier engine with `VISION_OCR` configured, run `find_blank_pages_with_text(cache_dir, sources)` once:
+  delete each finding's `cache_files` and re-ingest that document (its blank pages may hold invented text).
+
 - **Hidden sheets** are ingested by default; `IngestSource(include_hidden_sheets=False)` skips them (listed in
   `DocumentReport.skipped_hidden_sheets`).
 - **Blank rows inside a table**: the parse cuts a sheet's table at every blank row; the engine glues a piece back

{rag_wright-0.12.0 → rag_wright-0.13.0}/src/rag_wright/.agents/skills/specifying-a-product-on-the-engine/SKILL.md

@@ -58,7 +58,7 @@ Every product item lands in one or more of these. The route column is where the
 | **Domain knowledge**: vocabulary, schema, constraints, synonyms, decision criteria | "controlled vocabularies", "the ontology", "mandatory fields", "valid ranges" | `EngineConfig(pack=...)` (the store creates the declared types on `open_workspace`); your capabilities read value sets and SHACL shapes with `rdflib` / `pyshacl` | `authoring-a-capability` | ontology authoring |
 | **Ingestion**: documents, spreadsheets, images, embedded files into the graph | "parse the forms", "load the lab reports", "idempotent re-runs" | `build_ingestion` (your extractor + optional hooks) → `aingest`; `IngestSource`, `IngestionTuning`, `table_rows` / `TableRow`, `evaluate_ingestion`, `check_extraction`; `KgNode` / `KgEdge` | `building-an-ingestion-capability` | KG construction |
 | **Entity and registry resolution** | "resolve against the yarn register", "queue unresolved mentions" | your `document_hook`; the entity-resolution building blocks on `rag_wright.pack_sdk` | `building-an-ingestion-capability` | entity resolution |
-| **A typed query or retrieval** | "find similar cases", "filter the table", "the dossier" | a query capability of your pack composing the primitives (`kg_read`, `kg_edges`, `NOT_NULL`, hybrid search from `rag_wright.pack_sdk`), registered with `CapabilityManifest` + `register_capability` / `load_pack`, invoked with `ainvoke_subgraph` | `authoring-a-capability` | authoring capabilities |
+| **A typed query or retrieval** | "find similar cases", "filter the table", "the dossier" | a query capability of your pack composing the primitives (`kg_read`, `kg_edges`, `NOT_NULL`, `Range`, hybrid span search with `span_search` from `rag_wright.api`), registered with `CapabilityManifest` + `register_capability` / `load_pack`, invoked with `ainvoke_subgraph` | `authoring-a-capability` | authoring capabilities |
 | **A cited answer, or a refusal** | "every answer cites its source", "say when it is not in the records" | `agenerate_answer` over `EvidenceItem`s (an answer states its support; "none" abstains), `ajudge_spans` | `authoring-a-capability` | concepts (answers) |
 | **A typed decision or classification** | "classify the structure", "which of these is it" | `classifier-opportunity-analysis` first; then rules, a trained classifier, or the decision model (`jev_decision` from `engine_capabilities()`, criteria in the pack) | `classifier-opportunity-analysis` → `setfit` / `laya` | classification and decision models |
 | **A deterministic workflow behind a button** (mode **D**) | "upload, confirm, save", "export CSV" | a product **operation** (one function per user operation) calling your capabilities; the route calls the operation | `building-a-product-backend` | domain-adaptation step 8 |

{rag_wright-0.12.0/.claude → rag_wright-0.13.0/src/rag_wright/.agents}/skills/using-the-rag-wright-engine/SKILL.md

@@ -27,8 +27,11 @@ open-core and domain-neutral; your product brings the domain. Hard rules:
   `EngineConfig`/`StoreConfig` (`StoreConfig.from_env()` reads `ARCADEDB_*`)/`open_workspace`
   (`temporary_workspace`/`drop_workspace` for disposable and removed databases), the invokers
   (`ainvoke_subgraph`/`invoke_model`/`ainvoke_model`), discovery (`capability_index`/`discover`), the KG accessors
-  (`kg_read` with `key_range`, `kg_write`, `kg_edges` with `NOT_NULL`, `kg_count`/`kg_delete`/`kg_update`,
-  `entities_by_name`, `span_positions`), `pack_store(ws, cls)` (build your pack's store extension over the
+  (`kg_read`, `kg_write`, `kg_edges`, `kg_count`/`kg_delete`/`kg_update`, all filtering by `where` values that
+  are equality, a list for membership, `NOT_NULL` or a `Range(gt=, ge=, lt=, le=)` on any number of fields,
+  evaluated in the store;
+  `entities_by_name`, `span_positions`), hybrid span search (`span_search` / `aspan_search`: cited spans from
+  the dense and sparse index, fused in the store), `pack_store(ws, cls)` (build your pack's store extension over the
   workspace's store; the store itself stays private), metering and tracing (`measure_usage`, `record_usage`,
   `traced_run`/`traced_step`, and for code emitting its own generations `tracing_on`/`trace_level`/
   `start_generation`/`finish_generation`/`flush_tracing`), the query embedder (`ws.embedder` where you hold a

{rag_wright-0.12.0 → rag_wright-0.13.0}/src/rag_wright/api/__init__.py

@@ -26,6 +26,7 @@ from rag_wright.api.answers import (AnswerKind, Condition, EvidenceItem, Generat
                                     RelevanceVerdict, agenerate_answer, ajudge_spans)
 from rag_wright.api.documents import (aparse_document, aparse_document_bytes, parse_document,
                                       parse_document_bytes, source_document, pdf_page_count)
+from rag_wright.capabilities.parse_audit import BlankPageText, find_blank_pages_with_text
 from rag_wright.api.ids import decode_bbox, document_of
 from rag_wright.api.discover import Discovered, discover
 from rag_wright.contracts.ingestion import (
@@ -70,10 +71,11 @@ from rag_wright.evals import (GoldImportError, GoldMapping, aload_gold_set, arun
                               score_candidate_choice, score_classification, score_extraction, score_generation,
                               score_ingestion, score_judgment, score_relational, score_retrieval)
 from rag_wright.ingestion.tables import table_rows
-from rag_wright.store.seam import NOT_NULL, KgEdge, KgNode
+from rag_wright.store.seam import NOT_NULL, KgEdge, KgNode, Range
 from rag_wright.api.invoke import ainvoke_model, ainvoke_subgraph, capability_index, invoke_model
-from rag_wright.api.kg import (entities_by_name, kg_count, kg_delete, kg_edges, kg_read, kg_update, kg_write,
-                               span_positions)
+from rag_wright.api.kg import (aspan_search, entities_by_name, kg_count, kg_delete, kg_edges, kg_read, kg_update,
+                               kg_write, span_positions, span_search)
+from rag_wright.capabilities.span_search import SpanHit
 from rag_wright.api.tracing import (finish_generation, flush_tracing, start_generation, trace_level, traced_run,
                                     traced_step, tracing_on)
 from rag_wright.api.usage import ModelUsage, UsageTotals, measure_usage, record_usage
@@ -82,6 +84,7 @@ from rag_wright.models.profiles import (DecisionModelProfile, ModelProfile, Mode
 from rag_wright.api.workspace import (
     WorkspaceHandle,
     drop_workspace,
+    migrate_span_fields,
     open_workspace,
     pack_store,
     temporary_workspace,
@@ -101,7 +104,7 @@ from rag_wright.models.model_servers import (
 )
 
 __all__ = [
-    "EngineConfig", "StoreConfig", "EngineOptions", "IngestOptions", "WorkspaceHandle", "open_workspace", "drop_workspace", "temporary_workspace", "pack_store", "use_workspace_models",
+    "EngineConfig", "StoreConfig", "EngineOptions", "IngestOptions", "WorkspaceHandle", "open_workspace", "drop_workspace", "migrate_span_fields", "temporary_workspace", "pack_store", "use_workspace_models",
     # PS-19: deploy the self-hosted model server and use it like any model
     "ModelServerSpec", "ModelServer", "adeploy_model_server", "amodel_server", "await_model_server", "stop_model_server",
     "ModelServerNotDeployed", "ModelServerStatus", "amodel_server_status", "astop_model_server",
@@ -111,9 +114,10 @@ __all__ = [
     "query_embedder",  # PS-39: the process-wide query embedder, for a seam that holds no workspace
     "ModelRole",  # PS-1 (G17): WorkspaceHandle.model_id(role) and EngineConfig.models need it
     "ainvoke_subgraph", "invoke_model", "ainvoke_model", "capability_index", "discover", "Discovered",
-    "kg_read", "kg_write", "kg_edges", "kg_count", "kg_delete", "kg_update", "entities_by_name", "span_positions", "KgNode", "KgEdge", "NOT_NULL",
+    "kg_read", "kg_write", "kg_edges", "kg_count", "kg_delete", "kg_update", "entities_by_name", "span_positions", "span_search", "aspan_search", "SpanHit", "KgNode", "KgEdge", "NOT_NULL", "Range",
     "document_of", "decode_bbox",
     "source_document", "parse_document", "aparse_document", "parse_document_bytes", "pdf_page_count", "aparse_document_bytes",
+    "find_blank_pages_with_text", "BlankPageText",  # PS-47: cached parses holding text a VLM invented for a blank page
     "agenerate_answer", "ajudge_spans", "EvidenceItem", "GeneratedAnswer", "AnswerKind",
     "RelevanceVerdict", "Relevance", "Condition",
     "measure_usage", "record_usage", "UsageTotals", "ModelUsage", "traced_run", "traced_step",

{rag_wright-0.12.0 → rag_wright-0.13.0}/src/rag_wright/api/kg.py

@@ -19,16 +19,19 @@ _SPAN_POSITION_FIELDS = [
 def kg_read(ws: WorkspaceHandle, node_type: str, *, where: Optional[dict] = None, fields: Optional[list] = None,
             distinct: Optional[str] = None, order_by: Optional[str] = None, limit: Optional[int] = None,
             key_range: Optional[tuple] = None) -> list[dict]:
-    """Read typed nodes of `node_type` from the workspace (see `Store.kg_read`). Equality/`IN` filters, a
-    `key_range=(field, lo, hi)` range, projection, distinct, order, limit; an empty list `where` value is
-    scope-to-nothing -> `[]`."""
+    """Read typed nodes of `node_type` from the workspace (see `Store.kg_read`). Each `where` value is a scalar
+    (equality), a list (membership; an empty list is scope-to-nothing -> `[]`), `NOT_NULL` (presence) or a `Range`
+    (comparisons, on any number of fields: `{"weight": Range(ge=180, le=220), "tensile": Range(gt=400)}`); all
+    filters are AND-ed and evaluated in the store. `key_range=(field, lo, hi)` is the half-open shorthand
+    (`lo <= field < hi`). Then projection, distinct, order and limit."""
     return ws._store.kg_read(node_type, where=where, fields=fields, distinct=distinct, order_by=order_by, limit=limit,
                              key_range=key_range)
 
 
 def kg_count(ws: WorkspaceHandle, type_name: str, *, where: Optional[dict] = None,
              key_range: Optional[tuple] = None) -> int:
-    """The number of vertices or edges of `type_name` matching `where` / `key_range` (see `Store.kg_count`)."""
+    """The number of vertices or edges of `type_name` matching `where` / `key_range` (see `Store.kg_count`); `where`
+    as in `kg_read`, `Range` values included."""
     return ws._store.kg_count(type_name, where=where, key_range=key_range)
 
 
@@ -85,3 +88,30 @@ def span_positions(ws: WorkspaceHandle, document: str) -> list[dict]:
     rows = ws._store.kg_read(SPAN_TYPE, fields=_SPAN_POSITION_FIELDS, where={"document_id": document},
                              order_by="doc_start")
     return [{**r, "bbox": decode_bbox(r.get("bbox"))} for r in rows]
+
+
+def span_search(ws: WorkspaceHandle, query: str, *, k: int = 10, documents: Optional[list] = None,
+                tag: Optional[str] = None, dense_floor: int = 3) -> list:
+    """Hybrid search over the workspace's span index (every ingested span, embedded dense and sparse): the two legs
+    fused by Reciprocal Rank Fusion in the store, with the top `dense_floor` pure-dense spans guaranteed a place, so
+    a strong semantic match a common-token query would push out of the fused pool is kept (0 turns it off). Returns
+    up to `k` `SpanHit`s, best first, each citation-ready: `span_id`, `text`, `document_id`, `doc_start` /
+    `doc_end`, `pages`, `bbox`, `primary_tag`, `rank`, and `found_by` (`"hybrid"` and/or `"dense"`). `documents`
+    scopes it to those source documents in the store (`None` = all, `[]` = nothing); `tag` keeps one primary tag.
+    No reranking: compose `BGEReranker` over the hits if you want one. Raises `RuntimeError` when the workspace
+    has no embedder."""
+    from rag_wright.capabilities.span_search import span_search as _search
+
+    if ws.embedder is None:
+        raise RuntimeError("span_search needs the workspace's embedder; this workspace was opened without one")
+    return _search(query, store=ws._store, embedder=ws.embedder, k=k, documents=documents, tag=tag,
+                   dense_floor=dense_floor)
+
+
+async def aspan_search(ws: WorkspaceHandle, query: str, *, k: int = 10, documents: Optional[list] = None,
+                       tag: Optional[str] = None, dense_floor: int = 3) -> list:
+    """`span_search` off the event loop (the query embedding and the store calls block), for an async caller."""
+    import asyncio
+
+    return await asyncio.to_thread(span_search, ws, query, k=k, documents=documents, tag=tag,
+                                   dense_floor=dense_floor)

{rag_wright-0.12.0 → rag_wright-0.13.0}/src/rag_wright/api/workspace.py

@@ -83,7 +83,7 @@ def use_workspace_models(ws: Any) -> Iterator[None]:
 def open_workspace(config: EngineConfig, *, corpus: str, reset: bool = False, create: bool = True) -> WorkspaceHandle:
     """Resolve (and cache) the workspace for `corpus` (the backend database name) from `config`. Ensures the schema:
     the neutral engine types, plus `config.pack`'s declared types when set. Raises `RuntimeError` on a database whose
-    `Span` type still has the pre-ING-8d field names (migrate it with `scripts/migrate_span_fields.py`). Returns an
+    `Span` type still has the pre-ING-8d field names (migrate it with `migrate_span_fields`). Returns an
     opaque `WorkspaceHandle`. `reset=True` drops + recreates the database (test/clean-slate) and bypasses the cache.
     A missing database is created (logged): that is how a new corpus is provisioned. `create=False` raises
     `LookupError` instead (PS-27) -- use it wherever the corpus must already exist, so a wrong name fails loudly
@@ -140,6 +140,27 @@ def drop_workspace(config: EngineConfig, *, corpus: str) -> bool:
     return existed
 
 
+def migrate_span_fields(config: EngineConfig, *, corpus: str, batch: int = 5000,
+                        progress: Optional[Callable[[int, int], None]] = None) -> int:
+    """Migrate the database `corpus` from the pre-0.2.0 `Span` field names (`contract_id`, `function`, `functions`)
+    to the generic ones (`document_id`, `primary_tag`, `tags`), the database `open_workspace` refuses until this
+    has run (PS-35). Values are copied in batches of `batch` and the old properties dropped; `progress(done, total)`
+    is called after each batch. Idempotent: returns the number of records moved, 0 when already migrated. The
+    database must exist (`LookupError` otherwise; it is never created), and a missing or blank `corpus` raises
+    `ValueError`. Afterwards the schema is checked, so a database this returns for opens with `open_workspace`."""
+    if not corpus or not corpus.strip():
+        raise ValueError("migrate_span_fields needs the corpus to migrate; it never falls back to a default database")
+    store = _build_store(config, corpus, reset=False, create=False)
+    try:
+        moved = store.migrate_span_fields(batch=batch, progress=progress)
+        store.ensure_schema()
+    finally:
+        store.close()
+    if moved:
+        logger.info("migrated %d spans in database %r to the generic field names", moved, corpus)
+    return moved
+
+
 @contextmanager
 def temporary_workspace(config: EngineConfig, *, prefix: str = "tmp") -> Iterator[WorkspaceHandle]:
     """A workspace on a fresh database, `<prefix>_<12 hex>`, dropped when the block exits, whether it succeeds or