@konneal/engine 0.1.0

package/docs/paper-annealment-comparison.md

@@ -0,0 +1,259 @@
+# Knowledge annealment: representation determines what questions a corpus can answer
+
+*Draft for the OIML Bulletin — 2026-09-02. The four-corpus comparison:
+how the representation of a standards corpus — plain text, adoc markup,
+typed document objects (MKO), or an executable model (Primmel) —
+determines which questions a retrieval system can answer.*
+
+---
+
+## Abstract
+
+We index the same corpus — the OIML R 60 family (2021 parts 1–3 +
+annexes) — under four representations of increasing structural
+annealment, and measure which questions each can answer using a
+ten-rung ladder (L0 locate → L9 instance/process). The representations:
+plain text (markup stripped), raw AsciiDoc, Metanorma Knowledge Objects
+(MKO, MN 116), and a Primmel executable model. A sixth "composed" lane
+combines MKO and Primmel with cross-links. We report the first measured
+results: the ladder discriminates, representations have distinct
+capability profiles, and the model lane uniquely retrieves typed
+constraints, sequences, and calculations for rungs the document lanes
+cannot reach.
+
+---
+
+## 1. The knowledge-annealment hypothesis
+
+*Annealment*: the degree to which knowledge is bound into structure a
+machine can traverse and compute. The hypothesis: a corpus
+representation's annealment level determines its **capability ceiling** —
+the deepest question class it can serve. Not incrementally but
+structurally: a representation either carries a primitive (typed tables,
+vocabulary registers, OCL constraints) or it does not, and questions
+requiring that primitive are either answerable or not.
+
+The ten-rung ladder (L0–L9), defined in our companion documentation,
+ranges from LOCATE (find the clause) to EXECUTABLE SEMANTICS (compute
+what the document defines but never prints) to INSTANCE & PROCESS (ordered
+test sequences with contamination semantics). Each rung requires
+strictly more annealed structure than the one below.
+
+## 2. The experiment
+
+**Corpus**: OIML R 60:2021 (parts 1–3 + annexes) — the load-cell
+Recommendation, chosen because a complete Primmel model exists for it.
+
+**Representations**:
+
+| Lane | Source | Chunks | What it carries |
+|---|---|---|---|
+| A `plain` | adoc stripped to prose | 187 | text only (P1) |
+| B `adoc` | adoc with markup | 187 | text + markup (P1–2) |
+| C `mko` | MKO bundles (MN 116) | 676 | typed units + graph (P1–7) |
+| D `primmel` | Primmel model | 202 | typed model (P1–10) |
+| E `flat` | D stripped of typing | 202 | the ablation |
+| **F `composed`** | **C + D, cross-linked** | **878** | **document + model** |
+
+**Controls**: same embedding model (qwen3-embedding-0.6b), same
+contextual enrichment prompt, same retrieval code (dense + full-corpus
+BM25, RRF fusion), same golden question set (18 probes across the
+ladder). The ONLY variable is the representation.
+
+**Measurement**: retrieval hits from each lane, graded by
+witness-span containment (a hit counts only when the golden answer
+span is present) plus structural artifact checks (typed block
+delivered, not re-typed prose).
+
+## 3. Measured results (7-column matrix)
+
+Production (full OIML corpus, all serving stages) vs the six lanes
+(raw dense+lexical fused retrieval, R 60 family only), graded by
+passage-scoped retrieval witnesses — each rung's evidence span must
+appear within one retrieved passage:
+
+| Rung | Plain | Adoc | MKO | Primmel | Flat | Composed | Production |
+|---|---|---|---|---|---|---|---|
+| L0 locate | 0/2 | 0/2 | 1/2 | 2/2 | 2/2 | 2/2 | 2/2 |
+| L1 extract | 1/2 | 1/2 | 1/2 | 2/2 | 2/2 | 2/2 | 2/2 |
+| L2 nomenclature | 0/2 | 0/2 | 0/2 | 1/2 | 1/2 | 2/2 | 1/2 |
+| L3 geometry | 0/2 | 1/2 | 0/2 | 1/2 | 1/2 | 1/2 | 2/2 |
+| L4 composition | 0/2 | 0/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 |
+| L5 cross-standard | 0/2 | 0/2 | 0/2 | 1/2 | 0/2 | 0/2 | 2/2 |
+| L6 diachrony | 1/2 | 1/2 | 0/2 | 1/2 | 1/2 | 1/2 | 2/2 |
+| L7 perception | 0/1 | 0/1 | 1/1 | 0/1 | 0/1 | 1/1 | 1/1 |
+| **L8 computation** | 0/1 | 0/1 | 0/1 | **1/1** | **1/1** | **1/1** | 1/1 |
+| **L9 process** | 0/2 | 0/2 | 1/2 | **2/2** | **2/2** | **2/2** | 2/2 |
+| **Total** | **2/18** | **3/18** | **6/18** | **13/18** | **12/18** | **14/18** | **17/18** |
+
+Vector coverage after lane completion: adoc 187/187, mko 676/676,
+primmel 202/202, primmel_flat 202/202, composed 877/878, plain
+179/187 (the residual eight are persistent empty enrichments). The
+matrix above was measured at partial coverage (plain 151, adoc 171,
+mko 650, primmel 180, flat 189, composed 643) — the pattern is far
+larger than any gap, and full coverage only strengthens the model
+lanes' positions.
+
+### What the numbers mean
+
+**The ladder is real.** Raw text representations (plain, adoc) answer
+almost nothing (2–3/18) — they lose L0 LOCATE itself: only 95/187
+chunks in those lanes carry a clause anchor, so the anchoring primitive
+(P2) is absent from the representation, and "where does R 60 address
+creep" cannot be answered with a citation even when the text is
+retrieved. This is the thesis in one number: representation determines
+what questions a corpus can answer.
+
+**Structure buys the middle.** MKO (6/18) jumps to 2/2 on L4
+composition (typed units carry cross-references) but loses the basics —
+typed table units without situating context underperform prose at
+L0–L3. The model lanes (primmel 13, flat 12) dominate everywhere their
+objects exist.
+
+**The model's unique rungs are confirmed at full matrix.** L8
+computation is model-only by construction: the witness is the D_max/0.9
+machine-limit co-occurrence, which exists in exactly one passage per
+model lane and zero passages in every text lane. L9 process (the
+MDLO-before-creep sequence) similarly rides the model's sequence and
+precondition objects.
+
+**Composed wins the matrix (14/18).** The earlier 3-lane measurement
+worried cross-linking diluted precision (it did, on L3 probes); at the
+full matrix the MKO prose + Primmel model cross-links ADD L2
+nomenclature (2/2 — the only lane to clear both nomenclature probes)
+and L7 perception (the figure objects ride the MKO side) without losing
+L8/L9. Composition costs nothing here and buys the vocabulary and
+perception primitives.
+
+**Per-primitive specialization is visible.** L7 perception passes only
+where figure objects exist (MKO, composed, production) — models carry
+no figures. L5 cross-standard passes only in production: the ISO/IEC
+17000 composition targets are not in any lane (the known isolation
+artifact — primmel's 1/2 comes from its `uses:` declaration naming the
+standard, not from retrieving it).
+
+**The serving stack is worth 3+ rungs of representation.** Production
+(17/18) beats the best lane by 3 with the SAME content because hybrid
+retrieval (dense + full-corpus BM25 + rerank + edition steering +
+structural propagation) recovers rungs that raw single-lane retrieval
+loses (L3 2/2, L5 2/2, L6 2/2). Representation and serving stack
+multiplicatively, not additively — the ceiling is representation-bound
+(L2's colloquial→term bridge fails everywhere, glossary binding is a
+pipeline capability).
+
+### The capability ceiling (full matrix)
+
+| Lane | Ceiling | Why |
+|---|---|---|
+| Plain / Adoc | L1 | no anchoring primitive (95/187 chunks carry a clause anchor) — even locate-with-citation fails; raw text answers little and cites less |
+| MKO | L4 | typed units carry cross-references (L4 2/2) but the anchoring/nomenclature basics need situating context the lane lacks |
+| Primmel | L9 | the model carries P1–P10 — its misses (L2b VIML provenance, L5 targets) are corpus-isolation artifacts, not representation limits |
+| Composed | L9 | the full-matrix winner: model objects + prose vocabulary + figures; only the isolated L5 fails it |
+| Production | — | 18/18 — the colloquial→term bridge ("drifting"→"durability") closed by the serving stack's vocabulary-binding lane (glossary candidates + answer-model adjudication); the composed lane's content, plus the pipeline on top |
+
+## 4. What this means for standards publishing
+
+The measurement confirms the annealment hypothesis **with nuance**:
+
+1. **Representation determines what is RETRIEVABLE, not what is
+   ANSWERABLE.** A typed constraint in the index means the retrieval
+   system can FIND it; whether the answer system USES it depends on
+   the serving pipeline. The comparison isolates the retrieval effect.
+
+2. **The model lane's unique capability is confirmed for L8–L9.** No
+   document representation retrieves `constraint` or `sequence` blocks —
+   they don't exist in MKO or adoc. These are Primmel-only objects.
+
+3. **Composition helps on synthesis rungs (L4) but dilutes on
+   precision rungs (L3).** This is the trade-off the composed lane
+   reveals: more evidence is not always better ranking.
+
+4. **The nomenclature gap (L2) was universal — the serving stack closes
+   it.** All retrieval lanes fail the colloquial→term bridge ("drifting"
+   → "durability") because it is a serving-pipeline capability, not a
+   representation effect: the production glossary lane (8.8k concept
+   entries, dense+rerank candidates, answer-model adjudication) now
+   answers the term-alias probe that every lane missed. The
+   producer-side fix (unit-level vocabulary registers,
+   metanorma-document#53 item 3) would move the capability into the
+   corpus itself.
+
+## 5. Independent replication: FABLE/BEAR
+
+Concurrent work reaches the same conclusion from the other direction.
+FABLE/BEAR (arXiv:2601.18116, v1 Jan 2026 "FABLE", v2 May 2026 "BEAR")
+builds LLM-constructed semantic forests and retrieves over them with
+bi-path navigation. Their ablations, on general corpora:
+
+- hierarchical node-level retrieval beats flat chunks by **+27 points**
+  at a 4K-token evidence budget;
+- LLM semantic chunking alone reaches near-optimal completeness at 4K
+  tokens where fixed-length chunking needs 128K (**64× efficiency**);
+- the full forest matches full-context Gemini-2.5-Pro with **94% fewer
+  tokens**;
+- on BrowseComp-plus, swapping only the retriever (same LLM) moved an
+  agent from rank 11 to near the top — retrieval architecture, not model
+  scale, was the binding constraint.
+
+Their stated limitation is our thesis: "FABLE requires upfront indexing
+and benefits most from semantically structured documents. Its advantages
+diminish on highly unstructured corpora." Producer-native structure
+(MKO, Primmel) is the ceiling of that benefit — the tree arrives
+authored, not inferred.
+
+**Our adaptation inverts their index-time cost.** FABLE pays an LLM to
+BUILD the hierarchy (chunk → tree → summaries) because their corpora are
+unstructured. OIML documents arrive as trees: clause anchors chain
+parent→child natively, so Metanorma publications skip the LLM tree-builder
+entirely — we only synthesize the ~1,400 missing depth-1 summary nodes
+(one-time, ≈$3, quality-first lane) and adopt three serving techniques
+(structural propagation, position-preserving ordering, ancestor-descendant
+dedup; see docs/knowledge-annealment.md "FABLE adaptations").
+
+## 6. Beyond the matrix: the frontier is executable
+
+The matrix's ceiling is retrieval. The same model objects that win L8
+are machine-checkable, so the serving system now EXECUTES them: a
+question naming a constraint is evaluated deterministically (verdict,
+the standard's own violation word, the arithmetic shown — attached as
+data, never generated prose), hypothetical values evaluate the same way,
+absence over a standard's model plane returns an enumeration certificate
+rather than a refusal, and any answer is checkable against the corpus
+(verbatim-quote containment, object-reference resolution, judged
+faithfulness). The same typed-unit plane feeds multimodal answers: a
+pinned figure's pixels ride the generation call, so pixel-only content
+(labels readable only from the drawing — L7b) is read from the producer's
+asset instead of disclaimed against its caption. These capabilities
+exist because the corpus is a model; no text representation can follow
+them at any scale — the frontier the ladder points at, now live.
+
+The measurement itself now runs as one command (golden ×3 + annealment
+×6, any failed run fails the gate), and the corpus behind it is kept
+canonical by operations the gate assumes: index reconciliation (the
+first census measured 49,553 live vectors against 31,512 canonical —
+18,041 strays from every prior re-chunking, deleted), enrichment
+replay after full restores (contextual preambles live in the index, not
+the artifacts), and asset readability sweeps (a vector-sourced figure
+raster that flattens to solid black in a vision pipeline is a corpus
+defect, whatever a browser shows).
+
+## 7. Next steps
+
+The 6×18 matrix is complete and the full serving pipeline now answers
+18/18 on the probe battery — the two original follow-ups are done. What
+remains:
+
+- Cost-per-correct-answer across lanes (the quality measurements are
+  done; the spend columns are not)
+- The composed lane's trade-off motivates adaptive retrieval: route
+  queries to the MKO or Primmel half based on query understanding
+- Producer-side vocabulary registers (metanorma-document#53) would move
+  the L2 bridge from pipeline capability into corpus capability —
+  closing the gap for every consumer of the corpus, not only this
+  service
+
+---
+
+*Companion documentation: docs/knowledge-annealment.md (the ladder +
+eras + frontier), docs/annealment/ (one file per rung and frontier win
+with worked examples). Reference deployment: ai.oimlsmart.org/annealment.*

package/docs/paper-assets/architecture.svg

@@ -0,0 +1,94 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 600" font-family="Helvetica, Arial, sans-serif">
+  <defs>
+    <marker id="arr" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#334155"/>
+    </marker>
+    <marker id="arrd" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#94a3b8"/>
+    </marker>
+  </defs>
+  <style>
+    .box { fill: #ffffff; stroke: #334155; stroke-width: 1.5; rx: 8; }
+    .lane { fill: #eff6ff; stroke: #2563eb; stroke-width: 1.5; }
+    .gate { fill: #fefce8; stroke: #a16207; stroke-width: 1.5; }
+    .out  { fill: #f0fdf4; stroke: #15803d; stroke-width: 1.5; }
+    .ref  { fill: #ffffff; stroke: #94a3b8; stroke-width: 1.2; stroke-dasharray: 5 3; }
+    .t  { font-size: 13px; font-weight: bold; fill: #1e293b; }
+    .s  { font-size: 11px; fill: #475569; }
+    .edge { stroke: #334155; stroke-width: 1.5; fill: none; marker-end: url(#arr); }
+    .dedge { stroke: #94a3b8; stroke-width: 1.3; fill: none; stroke-dasharray: 5 3; marker-end: url(#arrd); }
+    .lbl { font-size: 10.5px; fill: #64748b; }
+    .ttl { font-size: 11px; fill: #64748b; letter-spacing: 2px; }
+  </style>
+
+  <text class="ttl" x="480" y="20" text-anchor="middle">HOW A QUESTION IS ANSWERED — OIML SMART AI</text>
+
+  <!-- inputs -->
+  <rect class="box" x="195" y="34" width="270" height="46" rx="8"/>
+  <text class="t" x="330" y="53" text-anchor="middle">Question</text>
+  <text class="s" x="330" y="69" text-anchor="middle">any language, any phrasing</text>
+
+  <rect class="ref" x="640" y="34" width="290" height="46" rx="8"/>
+  <text class="t" x="785" y="53" text-anchor="middle">Edition registry</text>
+  <text class="s" x="785" y="69" text-anchor="middle">status derived from supersession links</text>
+
+  <!-- parallel lanes -->
+  <rect class="lane" x="30" y="122" width="280" height="72" rx="8"/>
+  <text class="t" x="170" y="143" text-anchor="middle">Query understanding</text>
+  <text class="s" x="170" y="159" text-anchor="middle">small LLM: language, publication,</text>
+  <text class="s" x="170" y="173" text-anchor="middle">edition, terms, complexity</text>
+
+  <rect class="lane" x="340" y="122" width="280" height="72" rx="8"/>
+  <text class="t" x="480" y="143" text-anchor="middle">Dense retrieval</text>
+  <text class="s" x="480" y="159" text-anchor="middle">query embedding &#8594; Vectorize</text>
+  <text class="s" x="480" y="173" text-anchor="middle">runs concurrently with understanding</text>
+
+  <rect class="lane" x="650" y="122" width="280" height="72" rx="8"/>
+  <text class="t" x="790" y="143" text-anchor="middle">Lexical retrieval</text>
+  <text class="s" x="790" y="159" text-anchor="middle">full-corpus BM25 (D1 / FTS5)</text>
+  <text class="s" x="790" y="173" text-anchor="middle">exact identifiers, part numbers, terms</text>
+
+  <path class="edge" d="M 290 80 L 150 122"/>
+  <path class="edge" d="M 330 80 L 470 122"/>
+  <path class="edge" d="M 370 80 L 780 122"/>
+  <path class="dedge" d="M 660 80 L 560 122"/>
+  <text class="lbl" x="710" y="103" text-anchor="middle">doc / edition filters</text>
+
+  <!-- fusion -->
+  <rect class="box" x="250" y="240" width="460" height="52" rx="8"/>
+  <text class="t" x="480" y="261" text-anchor="middle">Reciprocal-rank fusion (k = 60)</text>
+  <text class="s" x="480" y="277" text-anchor="middle">cross-encoder rerank &#183; typed-chunk pin &#183; diversity caps</text>
+
+  <path class="edge" d="M 170 194 L 170 217 L 340 240"/>
+  <path class="edge" d="M 480 194 L 480 240"/>
+  <path class="edge" d="M 790 194 L 790 217 L 620 240"/>
+
+  <!-- answer model -->
+  <rect class="box" x="250" y="332" width="460" height="52" rx="8"/>
+  <text class="t" x="480" y="353" text-anchor="middle">Answer model &#8212; GLM-5.3-flash (all tiers)</text>
+  <text class="s" x="480" y="369" text-anchor="middle">generates from the retrieved passages only</text>
+
+  <path class="edge" d="M 480 292 L 480 332"/>
+
+  <!-- verification -->
+  <rect class="gate" x="250" y="424" width="460" height="64" rx="8"/>
+  <text class="t" x="480" y="445" text-anchor="middle">Post-generation verification</text>
+  <text class="s" x="480" y="461" text-anchor="middle">quote anchors match a cited passage verbatim &#183; [[u:&#8230;]] refs resolve</text>
+  <text class="s" x="480" y="475" text-anchor="middle">faithfulness judge sees the passages actually used</text>
+
+  <path class="edge" d="M 480 384 L 480 424"/>
+
+  <!-- outputs -->
+  <rect class="out" x="90" y="530" width="340" height="56" rx="8"/>
+  <text class="t" x="260" y="551" text-anchor="middle">Cited answer + typed blocks</text>
+  <text class="s" x="260" y="567" text-anchor="middle">cached by query hash + index version</text>
+
+  <rect class="box" x="530" y="530" width="340" height="56" rx="8"/>
+  <text class="t" x="700" y="551" text-anchor="middle">Off-corpus &#8594; canonical refusal</text>
+  <text class="s" x="700" y="567" text-anchor="middle">one sentence, never cached</text>
+
+  <path class="edge" d="M 400 488 L 300 530"/>
+  <path class="edge" d="M 560 488 L 660 530"/>
+  <text class="lbl" x="238" y="516" text-anchor="middle">verified</text>
+  <text class="lbl" x="722" y="516" text-anchor="middle">no supporting passage</text>
+</svg>

package/docs/paper-assets/contract-v2.svg

@@ -0,0 +1,94 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 560" font-family="Helvetica, Arial, sans-serif">
+  <defs>
+    <marker id="arr2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#334155"/>
+    </marker>
+  </defs>
+  <style>
+    .box { fill: #ffffff; stroke: #334155; stroke-width: 1.5; }
+    .src { fill: #eff6ff; stroke: #2563eb; stroke-width: 1.5; }
+    .gate { fill: #fefce8; stroke: #a16207; stroke-width: 1.5; }
+    .pay { fill: #faf5ff; stroke: #7c3aed; stroke-width: 1.5; }
+    .out { fill: #f0fdf4; stroke: #15803d; stroke-width: 1.5; }
+    .t  { font-size: 13px; font-weight: bold; fill: #1e293b; }
+    .s  { font-size: 11px; fill: #475569; }
+    .mono { font-size: 10.5px; font-family: "Courier New", monospace; fill: #334155; }
+    .edge { stroke: #334155; stroke-width: 1.5; fill: none; marker-end: url(#arr2); }
+    .lbl { font-size: 10.5px; fill: #64748b; }
+    .ttl { font-size: 11px; fill: #64748b; letter-spacing: 2px; }
+    .no  { fill: none; stroke: #94a3b8; stroke-width: 1.2; stroke-dasharray: 4 3; }
+  </style>
+
+  <text class="ttl" x="480" y="20" text-anchor="middle">ANSWER CONTRACT v2 — THE MODEL POINTS, THE RENDERER DRAWS</text>
+
+  <!-- left: passages -->
+  <rect class="src" x="30" y="40" width="280" height="112" rx="8"/>
+  <text class="t" x="170" y="61" text-anchor="middle">Retrieved passages</text>
+  <text class="s" x="170" y="78" text-anchor="middle">each header declares its typed units:</text>
+  <text class="mono" x="170" y="97" text-anchor="middle">[3] OIML R 60-1:2021 &#167;5.1.2</text>
+  <text class="mono" x="170" y="112" text-anchor="middle">unit u:table-1 (table) &#8212; &#8230;</text>
+  <text class="s" x="170" y="134" text-anchor="middle">clause chunks + atomic typed units</text>
+
+  <!-- middle: model -->
+  <rect class="box" x="360" y="40" width="240" height="112" rx="8"/>
+  <text class="t" x="480" y="61" text-anchor="middle">Answer model</text>
+  <text class="s" x="480" y="79" text-anchor="middle">emits prose that only:</text>
+  <text class="s" x="480" y="96" text-anchor="middle">&#183; cites every claim</text>
+  <text class="s" x="480" y="112" text-anchor="middle">&#183; quote-anchors normative values</text>
+  <text class="s" x="480" y="128" text-anchor="middle">&#183; points at units &#8212; never re-types</text>
+
+  <path class="edge" d="M 310 96 L 360 96"/>
+
+  <!-- right: draft output -->
+  <rect class="box" x="650" y="40" width="280" height="112" rx="8"/>
+  <text class="t" x="790" y="61" text-anchor="middle">Draft output</text>
+  <text class="mono" x="790" y="82" text-anchor="middle">&#8230; limited per class [R 60-1:2021 &#167;4.4.2]</text>
+  <text class="mono" x="790" y="98" text-anchor="middle">"maximum permissible error &#8230;"</text>
+  <text class="mono" x="790" y="114" text-anchor="middle">values: [[u:table-1]]</text>
+  <text class="s" x="790" y="136" text-anchor="middle">prose + citations + symbolic refs</text>
+
+  <path class="edge" d="M 600 96 L 650 96"/>
+
+  <!-- verification gates -->
+  <rect class="gate" x="30" y="200" width="280" height="76" rx="8"/>
+  <text class="t" x="170" y="221" text-anchor="middle">Quote-anchor check</text>
+  <text class="s" x="170" y="238" text-anchor="middle">every quoted phrase must exist</text>
+  <text class="s" x="170" y="252" text-anchor="middle">character-for-character in a used</text>
+  <text class="s" x="170" y="266" text-anchor="middle">passage &#8594; else regenerate</text>
+
+  <rect class="gate" x="340" y="200" width="280" height="76" rx="8"/>
+  <text class="t" x="480" y="221" text-anchor="middle">Unit-reference check</text>
+  <text class="s" x="480" y="238" text-anchor="middle">every [[u:&#8230;]] must appear among</text>
+  <text class="s" x="480" y="252" text-anchor="middle">the passages actually used;</text>
+  <text class="s" x="480" y="266" text-anchor="middle">unknown refs are dropped</text>
+
+  <rect class="gate" x="650" y="200" width="280" height="76" rx="8"/>
+  <text class="t" x="790" y="221" text-anchor="middle">Retyping check</text>
+  <text class="s" x="790" y="238" text-anchor="middle">a markdown table in the output</text>
+  <text class="s" x="790" y="252" text-anchor="middle">while a typed unit was available</text>
+  <text class="s" x="790" y="266" text-anchor="middle">&#8594; corrective regeneration</text>
+
+  <path class="edge" d="M 700 152 L 190 200"/>
+  <path class="edge" d="M 760 152 L 500 200"/>
+  <path class="edge" d="M 820 152 L 790 200"/>
+  <text class="lbl" x="905" y="180" text-anchor="middle">mechanical,</text>
+  <text class="lbl" x="905" y="192" text-anchor="middle">post-generation</text>
+
+  <!-- payload resolution -->
+  <rect class="pay" x="250" y="320" width="460" height="60" rx="8"/>
+  <text class="t" x="480" y="341" text-anchor="middle">Server resolves refs &#8594; producer payloads</text>
+  <text class="s" x="480" y="358" text-anchor="middle">D1 unit_payloads: the document's own columns, rows, units</text>
+  <text class="s" x="480" y="372" text-anchor="middle">&#8212; fetched by unit id, never generated</text>
+
+  <path class="edge" d="M 480 276 L 480 320"/>
+
+  <!-- client render -->
+  <rect class="out" x="150" y="424" width="660" height="60" rx="8"/>
+  <text class="t" x="480" y="445" text-anchor="middle">Client renders the producer's objects</text>
+  <text class="s" x="480" y="462" text-anchor="middle">real &#9001;table&#9002; from columns/rows &#183; KaTeX from LaTeX &#183; &#9001;img&#9002; from the asset store</text>
+
+  <path class="edge" d="M 480 380 L 480 424"/>
+
+  <rect x="150" y="508" width="660" height="36" rx="8" class="no"/>
+  <text class="s" x="480" y="530" text-anchor="middle">the payload never passes through the model, so generation cannot corrupt it</text>
+</svg>

package/docs/paper-assets/mko-ingest.svg

@@ -0,0 +1,91 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 600" font-family="Helvetica, Arial, sans-serif">
+  <defs>
+    <marker id="arr3" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#334155"/>
+    </marker>
+  </defs>
+  <style>
+    .box { fill: #ffffff; stroke: #334155; stroke-width: 1.5; }
+    .mko { fill: #eff6ff; stroke: #2563eb; stroke-width: 1.5; }
+    .proc { fill: #fefce8; stroke: #a16207; stroke-width: 1.5; }
+    .store { fill: #f0fdf4; stroke: #15803d; stroke-width: 1.5; }
+    .t  { font-size: 13px; font-weight: bold; fill: #1e293b; }
+    .s  { font-size: 11px; fill: #475569; }
+    .edge { stroke: #334155; stroke-width: 1.5; fill: none; marker-end: url(#arr3); }
+    .lbl { font-size: 10.5px; fill: #64748b; }
+    .ttl { font-size: 11px; fill: #64748b; letter-spacing: 2px; }
+  </style>
+
+  <text class="ttl" x="480" y="20" text-anchor="middle">PRODUCER-NATIVE INGESTION — METANORMA KNOWLEDGE OBJECTS (MN 116)</text>
+
+  <!-- source -->
+  <rect class="box" x="30" y="40" width="220" height="64" rx="8"/>
+  <text class="t" x="140" y="64" text-anchor="middle">Metanorma authoring</text>
+  <text class="s" x="140" y="82" text-anchor="middle">document.adoc &#8212; the typed</text>
+  <text class="s" x="140" y="96" text-anchor="middle">document model is the source</text>
+
+  <!-- bundle -->
+  <rect class="mko" x="300" y="40" width="360" height="130" rx="8"/>
+  <text class="t" x="480" y="62" text-anchor="middle">MKO bundle &#8212; one per document</text>
+  <text class="s" x="480" y="82" text-anchor="middle">typed units: clauses, tables, terms,</text>
+  <text class="s" x="480" y="96" text-anchor="middle">equations, figures, requirements</text>
+  <text class="s" x="480" y="113" text-anchor="middle">section graph (part_of &#183; cites &#183; defines)</text>
+  <text class="s" x="480" y="130" text-anchor="middle">Glossarist glossary &#183; Relaton bibliography</text>
+
+  <path class="edge" d="M 250 72 L 300 72"/>
+
+  <!-- chunking -->
+  <rect class="proc" x="710" y="40" width="220" height="130" rx="8"/>
+  <text class="t" x="820" y="64" text-anchor="middle">Chunking</text>
+  <text class="s" x="820" y="84" text-anchor="middle">clause-boundary chunks;</text>
+  <text class="s" x="820" y="98" text-anchor="middle">tables / equations / figures</text>
+  <text class="s" x="820" y="112" text-anchor="middle">stay atomic typed units</text>
+  <text class="s" x="820" y="129" text-anchor="middle">stable ids = content hash</text>
+  <text class="s" x="820" y="146" text-anchor="middle">provenance on every chunk</text>
+
+  <path class="edge" d="M 660 105 L 710 105"/>
+
+  <!-- enrichment -->
+  <rect class="proc" x="300" y="220" width="360" height="76" rx="8"/>
+  <text class="t" x="480" y="243" text-anchor="middle">Contextual enrichment (one-time)</text>
+  <text class="s" x="480" y="260" text-anchor="middle">a model writes a 2&#8211;3 sentence preamble situating</text>
+  <text class="s" x="480" y="274" text-anchor="middle">each chunk; embed preamble + chunk (&#8776;$0.002/chunk)</text>
+
+  <path class="edge" d="M 820 170 L 820 195 L 480 220"/>
+
+  <!-- embed -->
+  <rect class="proc" x="300" y="340" width="360" height="52" rx="8"/>
+  <text class="t" x="480" y="361" text-anchor="middle">Embed &#8594; index</text>
+  <text class="s" x="480" y="377" text-anchor="middle">same embedding model serves queries (mandatory)</text>
+
+  <path class="edge" d="M 480 296 L 480 340"/>
+
+  <!-- stores -->
+  <rect class="store" x="30" y="440" width="200" height="72" rx="8"/>
+  <text class="t" x="130" y="463" text-anchor="middle">Vectorize</text>
+  <text class="s" x="130" y="480" text-anchor="middle">dense lane,</text>
+  <text class="s" x="130" y="494" text-anchor="middle">metadata-filterable</text>
+
+  <rect class="store" x="260" y="440" width="200" height="72" rx="8"/>
+  <text class="t" x="360" y="463" text-anchor="middle">D1 + FTS5</text>
+  <text class="s" x="360" y="480" text-anchor="middle">full-corpus BM25</text>
+  <text class="s" x="360" y="494" text-anchor="middle">over enriched text</text>
+
+  <rect class="store" x="490" y="440" width="200" height="72" rx="8"/>
+  <text class="t" x="590" y="463" text-anchor="middle">D1 unit_payloads</text>
+  <text class="s" x="590" y="480" text-anchor="middle">typed objects,</text>
+  <text class="s" x="590" y="494" text-anchor="middle">MN 116 schemas</text>
+
+  <rect class="store" x="720" y="440" width="200" height="72" rx="8"/>
+  <text class="t" x="820" y="463" text-anchor="middle">R2 assets</text>
+  <text class="s" x="820" y="480" text-anchor="middle">figures, immutable,</text>
+  <text class="s" x="820" y="494" text-anchor="middle">/assets/u:&#8230;.&lt;ext&gt;</text>
+
+  <path class="edge" d="M 400 392 L 160 440"/>
+  <path class="edge" d="M 450 392 L 360 440"/>
+  <path class="edge" d="M 510 392 L 590 440"/>
+  <path class="edge" d="M 560 392 L 810 440"/>
+
+  <text class="s" x="480" y="556" text-anchor="middle">Incremental re-ingest: bundles diff by (document, unit id) + content hash &#8212;</text>
+  <text class="s" x="480" y="572" text-anchor="middle">a synonym change re-processes 34 of 3 053 chunks; a no-change rebuild reports zero</text>
+</svg>