@konneal/engine 0.1.0

package/docs/annealment/F5-workflow.md

@@ -0,0 +1,21 @@
+# F5 — Certification workflow statefulness
+
+**Powering objects:** `evaluation/processes.yaml` (layer-composed
+process tree; `validate_provision` binds the campaign to requirement
+URNs like /req/metrological/mpe), `evaluation/gateways.yaml`,
+`execution/` (test-report forms, checklist), entities/performance-test-
+evaluations, evaluation/sample-selection-rules.
+
+**The win:** the assistant knows WHERE an evaluation stands and WHAT
+GATES WHAT — not as prose but as traversable state:
+- *"what must be true before the creep test?"* → the sequence rule
+  (MDLO baseline first) + the gateway that checks it;
+- *"which requirements does this test campaign validate?"* → the
+  validate_provision URNs, exhaustively;
+- agentic follow-through: checklist items flip state as evidence
+  lands; the assistant can say "3 of 62 tests outstanding, blocked on
+  the humidity chamber" — a document lane can quote the procedure but
+  cannot know the POSITION.
+
+**Why documents can't follow:** procedures in prose are instructions;
+here they are a state machine with recorded progress.

package/docs/annealment/F6-impact.md

@@ -0,0 +1,21 @@
+# F6 — Impact analysis (committee tooling)
+
+**Powering objects:** the binding graph — requirements ↔ conformance
+tests ↔ formulas (`formulas-used.yaml` binds MDLO to conversion_factor_f,
+E_L, E_R, C_M), tables (mpe_tiers feeds lookupMPE), aspects
+(requirement-binding-targets), execution forms (E_R's report field).
+
+**The win:** DRAFTING becomes queryable. *"If the committee raises the
+class C limit_factor, what changes?"* → traverse the graph: the MPE
+verdicts for class C, the tests whose verdicts derive from lookupMPE,
+the R 60-3 report fields that record them, the instances already
+evaluated. The answer is an IMPACT SET — machine-derived, exhaustive
+(F3's completeness), with every hop's clause anchor.
+
+**Why it matters:** this is the first capability whose customer is the
+STANDARDS DEVELOPER, not the reader — the model pays for its own
+maintenance by making revision risk computable.
+
+**Why documents can't follow:** the binding "this test's verdict
+consumes this table's factor" exists nowhere in any single document —
+it is model knowledge spanning three parts.

package/docs/annealment/F7-editions.md

@@ -0,0 +1,18 @@
+# F7 — Semantic edition diffs + temporal jurisdiction
+
+**Powering objects:** editions/2017/*.prl — a FULL prior-edition model
+package (not a separate PDF) — plus edition lifecycle (status,
+supersedes, validity.from).
+
+**The win, beyond L6:** diffs become SEMANTIC, not textual:
+*"what changed in creep between 2017 and 2021?"* → diff the two MODELS:
+the constraint set (new OCL checks), attribute definitions (added
+fields, moved scopes), test sequences (reordered steps), tier tables
+(changed breakpoints) — each delta with both editions' clause anchors.
+Text diffs of two PDFs produce noise; model diffs produce CHANGE RECORDS.
+
+**Temporal jurisdiction:** validity windows answer retro-questions —
+*"an evaluation performed 2019-11: which edition governed it, and is
+its verdict still valid today?"* → 2017 governs; superseded-by chain
+says whether re-evaluation is required. Legal-metrology gold: the
+corpus knows TIME, not just content.

package/docs/annealment/F8-selfverify.md

@@ -0,0 +1,19 @@
+# F8 — Self-verification as a query (meta-grounding)
+
+**Powering objects:** the model-linker rule set (quantity-coherence,
+requirement-binding-targets) and the allowlist ledger whose history
+reads "burned to empty — every formerly allowlisted site was fixed at
+the data layer": the model is MACHINE-CHECKED consistent.
+
+**The win:** for model-grounded claims, execution replaces the judge.
+When the D lane answers "the class C limit factor is 0.35", it can
+VERIFY that claim against the model (the value is read from the typed
+table, not generated) — and the user can ask *"is this corpus
+internally consistent?"* and receive the linker verdict. The
+faithfulness judge (an LLM scoring prose) remains for synthesis; for
+model-grounded facts, the check is arithmetic.
+
+**Why it matters:** it closes the trust loop — the same mechanism that
+answers also proves. The strongest possible grounding story for the
+paper: not "the model was cited" but "the model was EXECUTED, and the
+execution is checkable".

package/docs/annealment/F9-projection-qa.md

@@ -0,0 +1,17 @@
+# F9 — Document-as-projection verification
+
+**Powering objects:** documents/{1,2,3,a}/document.presentation.xml —
+the model CARRIES its own rendered document projections per part.
+
+**The win:** publishing QA becomes a query. *"Does the published
+R 60-1 Table 4 match the model's mpe_tiers?"* → parse the projection's
+table, compare cell-by-cell against the model's typed table → a
+diff-and-verdict. The direction inverts: instead of the document being
+the source the model was derived from, the MODEL becomes the source and
+the document a VIEW to be verified. Forward: model → document
+GENERATION (the view rendered FROM truth, drift structurally
+impossible).
+
+**Why it matters:** every standards body's dirty secret is render drift
+(errata). Projection QA makes errata a computable diff. And it is the
+on-ramp to the fully closed loop: author → model → verified document.

package/docs/annealment/L0-locate.md

@@ -0,0 +1,19 @@
+# L0 — LOCATE (primitives P1–P2)
+
+**Definition:** find WHERE the corpus addresses a topic; return the clause
+anchor. No value, no synthesis — presence and position only.
+
+**Why it's the floor:** it isolates retrieval from understanding. A lane
+failing L0 is mis-built (chunking/embedding bug), not "less annealed" —
+the floor validates the harness before the ladder means anything.
+
+**Example** — *"Where does R 60 address creep?"*
+Witness: citation anchor ∈ {R 60-1 §3.x (definition), R 60-2 §2.11.x
+(test)} — either is a pass; the anchor must EXIST and be normative.
+Lane expectations: A/B/C/D/E all pass (sanity). Primmel additionally
+binds the term to its `behaviors: [creep]` entry with stimulus/response
+and the `load-cell` entity — same anchor, richer provenance.
+
+**Harness notes:** 6 questions spanning definition-anchors, test-anchors
+and annex-anchors; paraphrases use colloquial synonyms ("time drift under
+load") so lexical lanes can't win by string match alone.

package/docs/annealment/L1-extract.md

@@ -0,0 +1,18 @@
+# L1 — EXTRACT (P1–P2)
+
+**Definition:** return a normative value VERBATIM from prose — no
+disambiguation beyond finding the sentence.
+
+**Example** — *"Quote the note that governs which p_LC value the creep
+MPE must use."*
+Witness (real, from notes.yaml): "p_LC = 0.7" AND the qualifier
+"regardless of any value declared by the manufacturer"
+(note_creep_plc_always_0_7 — see also F10: this note is an OVERRIDE in
+the model, prose everywhere else).
+
+**Example** — *"What validity date does the current R 60 edition carry?"*
+Witness: "2021-01-01" (standard.yaml edition.validity.from — P2
+provenance makes this trivial in D, findable in C via the registry).
+
+Lane expectations: all pass; separation is noise-level. L1 exists to
+calibrate the harness's regex discipline before structure matters.

package/docs/annealment/L2-nomenclature.md

@@ -0,0 +1,22 @@
+# L2 — NOMENCLATURE (P4)
+
+**Definition:** resolve the user's vocabulary to the corpus's DEFINED
+terms, return the authoritative definition and its vocabulary anchor.
+
+**Sub-probes:**
+- **a) corpus-local** — *"my load cell keeps drifting over months — what
+  does R 60 call that?"* → durability; witness: the definition text
+  ("ability of a measuring instrument to maintain its performance
+  characteristics over a period of use") + anchor R 60-1 §3.1 with VIML
+  5.15 register (terminology.yaml: vocab_ref viml-2022 5.15).
+- **b) cross-register** — *"which VIM/VIML concept does 'durability'
+  come from?"* → viml-2022 clause 5.15. Only lanes carrying vocab_ref
+  answer the REGISTER part (D natively; C via the glossary lane's
+  concept sources).
+- **c) multilingual** — term spellings per language (Primmel
+  eng-Latn entries; multilingual spellings land with more languages).
+
+Lane expectations: A/B luck-dependent (embedding proximity); C strong at
+(a); D strong a–c. The colloquial gap ("drift"≠"durability") is the
+measurement: without P4 the lanes must bridge vocabulary by embeddings
+alone.

package/docs/annealment/L3-geometry.md

@@ -0,0 +1,23 @@
+# L3 — TABULAR GEOMETRY (P3 + P8)
+
+**Definition:** a value whose retrieval requires row×column
+disambiguation; delivery requires the TYPED BLOCK (the producer's
+object, never re-typed prose).
+
+**Sub-probes:**
+- **a) cell lookup** — *"minimum n_LC for class B"* → R 60-1 Table 3:
+  row B, lower-limit column. Witness: 5000 (family values: A 50000, B
+  5000, C 500, D 100 — verified in the corpus) + block artifact
+  `block:"table"`.
+- **b) unit-aware cell** — *"what quantity does `load_min` carry in the
+  MPE tier table, and in what unit?"* → tables.yaml column
+  `{name: load_min, type: number, unit: v}` → "load in verification
+  intervals (v)". Only D answers the UNIT question structurally (P8);
+  C's payload has unit strings (metanorma-document#55 GAP-1).
+- **c) derived cell** — *"the MPE limit factor for class C at a load
+  between the tier bounds"* → needs the tier BREAKPOINTS + factor: the
+  lookup operation (formula lookupMPE) — transitional to L8.
+
+Lane expectations: A ≤40% (flattened geometry), B partial (pipes as
+noise), C ≥90% at (a), D ≥90% a–c. **First hard separator; the
+side-by-side demo on the site.**

package/docs/annealment/L4-composition.md

@@ -0,0 +1,21 @@
+# L4 — INTRA-DOCUMENT COMPOSITION (P2 + P5)
+
+**Definition:** an answer that JOINS ≥2 clauses (often + a table) and
+returns both anchors plus the derived relation.
+
+**Sub-probes:**
+- **a) intra-document** — *"initial-verification MPE vs in-service
+  limits for a class C load cell, and how they scale across the
+  range"* → R 60-1 MPE table (tier breakpoints, limit_factor) + the
+  applicability clauses. Witness: both anchors + the scaling statement
+  expressed in v-units.
+- **b) cross-part** — *"which R 60-2 test validates the R 60-1
+  repeatability requirement, and where is its result recorded in
+  R 60-3?"* → requirement `/req/metrological/repeatability` (R 60-1) ↔
+  test `/conf/metrological-tests/...` (R 60-2) ↔ test-report form
+  (R 60-3 §2.1.3 E_R). Witness: the three part-anchors. In D the chain
+  is EDGES (formulas_used binds the test to
+  `repeatabilityError`); in C it must be co-retrieved.
+
+Lane expectations: C/D strong at (a); (b) separates C (partial — needs
+all three parts in the window) from D (edges). A/B partial.

package/docs/annealment/L5-cross-standard.md

@@ -0,0 +1,20 @@
+# L5 — CROSS-STANDARD LINKING (P5)
+
+**Definition:** answers that traverse a reference to ANOTHER standard,
+returning the linked identity and clause.
+
+**Sub-probes:**
+- **a) cites-edge** — *"which ISO/IEC test method does R 60-2 invoke for
+  humidity?"* → witness: the ISO/IEC document id + clause from the
+  cites edges / references registry. ISOLATION: public lanes assert the
+  LINK only — ISO/IEC text never enters a public index; content probes
+  are member/internal-only, structurally enforced.
+- **b) composition** — *"which CASCO vocabulary governs R 60's
+  certification activities, and which aspects of the load cell do its
+  requirements bind?"* → `uses: iso-iec-17000` + `iso-iec-17065` + the
+  aspects registry (markings, accompanying_document… via
+  requirement-binding-targets). Witness: both package ids + ≥1 aspect.
+
+Lane expectations: D strong (composition is a first-class relation);
+C partial at (a) (cites edges, no composition); A/B near-zero —
+cross-standard vocabulary rarely co-embeds.

package/docs/annealment/L6-diachrony.md

@@ -0,0 +1,21 @@
+# L6 — DIACHRONY / EDITIONS (P6)
+
+**Definition:** answers that depend on WHICH edition, or on WHAT
+CHANGED between editions.
+
+**Sub-probes:**
+- **a) current-edition selection** — *"which R 60 edition applies to a
+  type evaluation started this year?"* → 2021 (lifecycle: status
+  current, supersedes 2017). Witness: edition + status.
+- **b) delta extraction** — *"what changed in the creep requirements
+  between 2017 and 2021?"* → D computes the diff between EDITION
+  PACKAGES (editions/2017/*.prl vs current) at the model level — the
+  witness is the delta fact (constraint added/changed, limits moved).
+  C can align units by hash but has no semantic diff (metanorma-document
+  #53 item 5). A/B near-zero.
+- **c) temporal jurisdiction** — *"which edition governed an evaluation
+  performed in 2019, and under what validity rule?"* → 2017 (validity
+  windows: 2017 until 2021-01-01). Only D (validity.from on editions).
+
+Lane expectations: D strong a–c; C partial (a, steering already ships);
+A/B fail b/c. Deepened by frontier F7.

package/docs/annealment/L7-perception.md

@@ -0,0 +1,20 @@
+# L7 — MULTIMODAL PERCEPTION (P7)
+
+**Definition:** answers whose evidence is in PIXELS, not text.
+
+**Sub-probes:**
+- **a) caption/asset** — *"what does the R 60-2 test setup figure
+  show?"* → figure unit + stored caption/vision description.
+- **b) pixel-only content** — *"what are the three labeled cases in the
+  design-shapes example?"* → labels A/B/C that exist ONLY in the
+  drawing; witness: the label values (verified: the deployed system
+  reads them from pixels — attachFigureImages).
+- **c) user-image grounding** — a nameplate photo → "which accuracy
+  class marking does this carry / which OIML R applies?" (image
+  questions, shipped). Witness: classification.
+
+Lane expectations: C strong a–c (figure lane live + multimodal
+generation); D partial (references figures in sequences —
+`fig-2/fig-3` — asset binding pending); A/B zero (no units, no assets).
+The most DEMO-VISIBLE separator: same question, lane C shows the image
+and reads it, lane A shows prose soup.

package/docs/annealment/L8-computation.md

@@ -0,0 +1,22 @@
+# L8 — COMPUTATION (P9 + P8)
+
+**Definition:** answers the documents DEFINE but never PRINT — values
+that must be computed. Witnesses are COMPUTED OFFLINE from the model
+(deterministic; no judge, no regex luck).
+
+**Sub-probes:**
+- **a) pure calculation** — *"the conversion factor f for these
+  indications"* → calculations.yaml conversionFactor (inputs:
+  avgIndicationAt75pct, indicationAtDmin; R 60-3 2.1.2.4). Witness: the
+  computed number.
+- **b) constraint verdict** — *"is D_max = 0.8·E_max acceptable?"* → OCL
+  `dead_load_max_geometry` FAILS: d_max must lie in [0.9·E_max, E_max];
+  witness: verdict + the recorded violation_meaning ("the type
+  evaluation of this load cell is void") — F1's ladder form.
+- **c) unit coherence** — *"are 3000 kgf and 30 kN the same force?"* →
+  the unit REGISTER converts (quantity_kind force; conversion to SI);
+  witness: the equality verdict. P8 pure.
+
+Lane expectations: **D only, by construction** — A/B/C/E cannot compute
+(they can at best quote a formula; E proves it's not prose quality).
+The ladder's most objective rung: ground truth is arithmetic.

package/docs/annealment/L9-instance-process.md

@@ -0,0 +1,23 @@
+# L9 — INSTANCE & PROCESS (P10)
+
+**Definition:** answers about PARTICULAR things and ORDERED activities —
+knowledge that exists only where the model records instances and
+sequences.
+
+**Sub-probes:**
+- **a) process order & consequence** — *"what is contaminated if creep
+  runs before MDLO on the same sample?"* → test_sequences.yaml
+  `mdlo-creep-dr`: MDLO is the BASELINE (order 1), creep FOLLOWS (order
+  2), DR reads against creep's D_max; witness: the ordering verdict +
+  "contaminates the baseline". The rationale is ENCODED as sequence
+  semantics — prose lanes only pass if the corpus happens to print it
+  (containment-checked; the printed form exists in R 60-2 — so the
+  probe asks the COUNTERFACTUAL form: "may I run them in my own
+  order?" → D answers from the model rule).
+- **b) instance-grounded fact** — a sample instrument profile →
+  "what is ITS E_R budget?" — the model evaluates at the instance.
+- **c) execution form** — *"where is E_R recorded in the test report?"*
+  → execution/test-report.yaml + subforms (R 60-3 2.1.3).
+
+Lane expectations: **D only.** The deepest binding: the corpus doesn't
+describe the general — it RECORDED the particular and the ORDER.

package/docs/annealment/README.md

@@ -0,0 +1,10 @@
+# The annealment documentation set
+
+One file per ladder rung (L0–L9) and per frontier win (F1–F12). The
+instrument spec with the primitive taxonomy lives in
+../knowledge-annealment.md; each file here ELABORATES its level with
+worked examples, witnesses and lane expectations.
+
+Ladder: [L0](L0-locate.md) · [L1](L1-extract.md) · [L2](L2-nomenclature.md) · [L3](L3-geometry.md) · [L4](L4-composition.md) · [L5](L5-cross-standard.md) · [L6](L6-diachrony.md) · [L7](L7-perception.md) · [L8](L8-computation.md) · [L9](L9-instance-process.md)
+
+Frontier: [F1](F1-verdict.md) · [F2](F2-counterfactual.md) · [F3](F3-absence.md) · [F4](F4-instance.md) · [F5](F5-workflow.md) · [F6](F6-impact.md) · [F7](F7-editions.md) · [F8](F8-selfverify.md) · [F9](F9-projection-qa.md) · [F10](F10-notes.md) · [F11](F11-composition.md) · [F12](F12-passport.md)

package/docs/guidelines-metanorma-ai-programme.md

@@ -0,0 +1,279 @@
+# Metanorma AI — Building a Grounded RAG Service from Metanorma Sources
+
+*Programme guidelines: how any standards body or Metanorma user builds a
+citation-grounded question-answering service over their own corpus,
+from scratch, using the Metanorma document model as the source of
+truth. Companion to the OIML Bulletin article (2026-08-30). Everything
+below is extracted from a production deployment — the OIML SMART AI
+service — and is reproducible.*
+
+---
+
+## Who this is for
+
+You have documents authored in Metanorma (or convertible to it) and you
+want people to *ask questions* and get *verifiable, cited answers* —
+not a search box over PDFs. The path below assumes no prior RAG
+experience; every stage names the decision, the cost, and the
+measurement that proves it works.
+
+## Principles (the short list)
+
+1. **Grounding is the product.** Fluency is free; trust is engineered.
+   Every claim cites the exact publication, edition and clause. When
+   the corpus doesn't contain the answer, the system says so — one
+   canonical refusal sentence, never cached.
+2. **Facts from the source model, never from renderings.** Metanorma
+   documents are typed object models; consuming the model (via MKO)
+   beats scraping HTML or PDF on every axis that matters.
+3. **Input = full fidelity; output = references.** The model *reads*
+   tables, equations, figures; it *never re-types* them. It writes a
+   symbolic reference; your renderer draws the producer's payload.
+4. **Measure everything, ship nothing unmeasured.** A golden question
+   set with witness citations, retrieval metrics over repeated runs,
+   and a faithfulness judge that sees the passages the answer was
+   actually built from. CI runs the suites; a change without a number
+   is a guess.
+5. **Cost-first on the hot path, quality-first on one-time work.**
+   Every question pays the serving model; enrichment, captioning and
+   evaluation are one-time costs whose quality persists.
+
+---
+
+## Stage 0 — Decide your corpus and its identity backbone
+
+**What:** the documents you will index, and the canonical identifier
+that makes two spellings of the same publication collapse into one.
+
+- Establish a registry: publication families → editions → status
+  (current / superseded / withdrawn). **Derive status from
+  supersession links, never from a status field** — fields lie; edges
+  are structure.
+- Surface the gaps: families with no derivable current edition are your
+  upstream bibliographic worklist, not something to hide.
+- Record language and edition metadata on every document *before*
+  chunking; it becomes the filter your retrieval uses.
+
+**Cost:** $0. **Measurement:** registry completeness (families with a
+derivable current edition / total families).
+
+## Stage 1 — Ingest the Metanorma model (MKO), not renderings
+
+**What:** export each document as a Metanorma Knowledge Objects bundle
+(MN 116): typed units (clauses, tables, terms, equations, figures,
+requirements), a section graph (part_of, cites, defines edges), a
+native glossary, and native bibliographic objects.
+
+```
+metanorma compile document.adoc -x mko    # (or the Ruby exporter today)
+```
+
+**Key decisions:**
+- Chunks follow **clause boundaries** — the unit a human cites — never
+  token windows. Tables, equations and figures are **atomic units**
+  (never split; cite their parent clause).
+- Each chunk carries: docidentifier, edition, language, clause anchor,
+  status, and its **unit id** (stable content hash) for incremental
+  re-ingest.
+- Write a **data-quality report** on every build: counts by type,
+  missing anchors, oversized payloads. Serving keeps score; upstream
+  fixes follow.
+
+**Cost:** $0 (local compute). **Measurement:** unit counts per document
+against a hand-check; zero empty chunks for preface-only documents
+(a real producer bug we hit).
+
+## Stage 2 — Contextual enrichment (the highest-ROI one-time spend)
+
+**What:** a one-time pass where a strong model writes a 2–3 sentence
+preamble situating each chunk in its document ("This clause defines the
+accuracy-class limits for load cells in R 60-1:2017 §4.1.2"), then you
+embed **preamble + chunk text** and index that.
+
+- Cache preambles by chunk id (KV or object store); unchanged chunks
+  are free on re-runs. Invalidate by content hash.
+- Budget: ~260 in / ~210 out tokens per chunk. At our models that is
+  ≈$0.0017 per chunk; a 40,000-chunk corpus is ≈$70 one-time.
+- Expect a measurable retrieval lift and, more visibly, the recovery of
+  paraphrased questions that share no vocabulary with the corpus.
+
+**Cost:** ≈$0.002 per chunk, one-time. **Measurement:** retrieval
+recall@5 before/after on a paraphrase probe set.
+
+## Stage 3 — Hybrid retrieval (dense + full-corpus lexical)
+
+**What:** two retrieval lanes run in parallel on every question:
+
+1. **Dense** — the question is embedded (same embedding model as the
+   index; this is mandatory) and queried against the vector index.
+2. **Lexical** — a full-corpus BM25/FTS index over the *enriched* chunk
+   text, catching exact identifiers, part numbers and defined terms
+   that dense similarity misses.
+
+Fuse with reciprocal rank fusion (RRF, k=60). Then a cross-encoder
+reranks the fused candidates; optionally a stronger listwise pass for
+complex questions.
+
+**The lesson that cost us a baseline:** lexical retrieval must scan the
+*whole corpus* as a first stage. Re-scoring only what dense already
+found cannot recover what dense missed. On our corpus this single
+change lifted recall@5 from 86% to 95%.
+
+**Cost:** embedding ≈$0.00001 per query; lexical ≈free (D1/FTS5).
+**Measurement:** recall@5, AP@5, MRR@5 on the golden set, **mean over
+≥3 runs** (single runs are noise).
+
+## Stage 4 — Query understanding (LLM-decided, zero rules)
+
+**What:** a small model reads the question first: language, named
+publication, edition, defined terms, complexity, whether it's
+conversational. There are no keyword rules and no regex on user input —
+the same model judges every question, in every language.
+
+- Its output drives **metadata filters** (doc number, edition) on the
+  dense query — the exact-identifier query class embeddings handle
+  poorly.
+- A **terminology graph** (defined term → defining documents) resolves
+  colloquial phrasing to the corpus's own vocabulary.
+- Run the understanding call **in parallel** with the query embedding;
+  they have no dependency when no filter is emitted.
+
+**Cost:** ≈$0.0002 per question. **Measurement:** filter precision
+(named publication → correct doc_number).
+
+## Stage 5 — The answer contract
+
+**What:** the model generates from the retrieved passages only, under a
+data-file prompt contract:
+
+- **Inline citations** on every claim: `[OIML R 60-1:2021 §4.4.2]`
+- **Verbatim quote anchors** for normative values — the quoted phrase
+  must exist, character-for-character, in a cited passage
+- **Symbolic unit references** (`[[u:table-1]]`) when presenting a
+  whole table/equation/figure — the model *points*, your renderer
+  draws the producer's payload
+- **One canonical refusal sentence** for off-corpus questions
+
+**Verification (all mechanical, all post-generation):**
+
+| Check | Method | On failure |
+|---|---|---|
+| Quote anchors | deterministic text match against used passages | one corrective regeneration; never cache the unverified answer |
+| Unit references | every `[[u:…]]` must exist in the used passages | dropped, never rendered |
+| Table retyping | markdown table in output while a typed unit was available | corrective regeneration with a targeted note |
+| Faithfulness | independent judge scores claims vs the passages used | answer withheld or regenerated |
+
+The crucial correctness rule: **the faithfulness judge sees the
+passages the answer was actually built from** (return them in the
+response), not a fresh retrieval — otherwise you are measuring a
+different question's evidence.
+
+**Cost:** judge ≈$0.0003 per answer. **Measurement:** faithfulness
+score distribution; corrective-regen rate; zero cached violations.
+
+## Stage 6 — Serving, caching, and cost control
+
+- **Exact answer cache** (keyed by query hash + index version) and a
+  **semantic cache** (leading-dimension signature, cosine ≥0.97) serve
+  near-duplicate questions instantly. Refusals and conversational turns
+  are never cached. `fresh=true` bypasses *every* cache (we shipped a
+  bug where it missed the semantic one — test this).
+- **Cache versioning:** bump the version on every retrieval or prompt
+  change; old answers expire by key.
+- **Model policy:** all tiers can share one good model if it's cheap
+  enough (ours: ≈$0.001/answer). Hot-path understanding stays on a
+  smaller model; only the *answer* needs the quality.
+
+## Stage 7 — Evaluation as a CI gate
+
+Build these suites **before** you need them:
+
+1. **Golden set** — 20–30 questions with expected citations (regex over
+   docidentifier) and witness values for table questions. Run on every
+   change; report R/AP/MRR means over ≥3 runs.
+2. **Paraphrase probes** — the same question asked 8 ways must hit the
+   same sources.
+3. **End-to-end behavioural cases** — refusal correctness, language
+   fidelity, edition steering, block rendering, auth tiers.
+4. **Faithfulness battery** — judged scores over the golden set,
+   re-run after any generation-model change.
+
+Our suites caught: a corpus lane invisible to filtered retrieval (an
+identity-parse bug), a mangled CSS rule that blanked dark mode, a cache
+path that served stale answers to regeneration requests, and a
+diversity cap that structurally evicted typed tables. None were visible
+in code review; all were visible in a suite.
+
+## Stage 8 — The typed-block pipeline (tables, equations, figures)
+
+This is the stage that separates a chatbot from a *standards*
+assistant:
+
+1. **Typed payloads in a serving store** (D1/KV table: unit_id →
+   payload), written at ingest, validated against the MN 116 schemas.
+2. **Passage headers declare their units** so the model can reference
+   them: `[3] OIML R 60-1:2021 §5.1.2 unit u:table-1 (table) — …`
+3. **Server-side reference resolution**: validate against used
+   passages, fetch the payload, return a `blocks` array alongside the
+   prose.
+4. **Client rendering**: real `<table>` from columns/rows, KaTeX from
+   LaTeX, `<img>` from the asset store, term cards from designations +
+   definition. Unknown block types render nothing (forward
+   compatibility).
+5. **Figures**: upload image assets to immutable URLs; a vision-capable
+   model describes each from pixels (one-time, ≈cents per figure);
+   store the description so text-only tiers can explain figures too.
+
+## Stage 9 — Operations
+
+- **Incremental re-ingest:** diff bundles by (document, unit_id) +
+  content hash; a wording change re-processes only the affected chunks.
+  Our corpus-proven case: a synonym change diffed exactly 34 of 3,053
+  chunks; a no-change re-ingest reports zero.
+- **Spend ledger:** log per-request model spend to a queryable table;
+  review weekly against the model policy.
+- **Upstream loop:** every corpus defect the service surfaces (missing
+  successor links, malformed table serializations, preface-only
+  documents) goes back as an upstream PR or issue — the corpus gets
+  better because serving keeps score.
+
+---
+
+## Reference stack (as deployed)
+
+| Component | Choice | Notes |
+|---|---|---|
+| Platform | Cloudflare Workers only | minimal accounts, no proprietary model providers |
+| Answer model | GLM-5.3 Flash (open-weight, multimodal) | ≈$0.001/answer, all tiers |
+| Understanding | Qwen3-30B-A3B | cost-first hot path |
+| Embeddings | Qwen3-Embedding-0.6B | same model both sides (mandatory) |
+| Reranker | bge-reranker-base | ≈$0.00003/query |
+| Vector index | Cloudflare Vectorize | dense lane |
+| Lexical index | D1 + FTS5 (porter) | full-corpus BM25 |
+| Registry + graph | D1 (documents, graph_nodes/edges) | derived status |
+| Caches | KV (answer + semantic) | versioned by index |
+| Typed payloads | D1 (unit_payloads) | MN 116 schemas |
+| Assets | R2 (unit-keyed, immutable) | /assets/u:<id>.<ext> |
+| Ingest | Python (pydantic models) | MKO → chunks → enrich → embed → upsert |
+| Eval | Node (golden, paraphrase, e2e, RAGAS-style) | CI-gated |
+
+## What not to do (each one cost us a measurement)
+
+- Don't scrape rendered HTML when the document model exists.
+- Don't let the model re-type tables; it will, and the values will be
+  wrong in ways that pass review.
+- Don't trust a status field over the record's own edges.
+- Don't judge faithfulness against a fresh retrieval.
+- Don't report single-run metrics; the noise band is ±5pp.
+- Don't run your heavy eval/enrichment traffic against the same account
+  that serves users — you'll throttle yourself and mis-measure
+  everything.
+- Don't skip the refusal path in your test suite; it's the contract
+  users trust most.
+
+---
+
+*Production reference: ai.oimlsmart.org — OIML SMART AI. Format spec:
+Metanorma MN 116 (Metanorma Knowledge Objects). This guide is extracted
+from the system's own documentation set and can be adapted per corpus;
+the measurement discipline transfers unchanged.*