@konneal/engine 0.1.0

package/docs/ARCHITECTURE.md

@@ -0,0 +1,302 @@
+# OIML RAG Architecture — 2026-08-26
+
+The complete serving pipeline, layer by layer. Every technique is
+grounded in 2025/2026 research and running in production at
+**ai.oimlsmart.org**. The layers below map 1:1 onto the stage registry
+(`workers/worker_public/src/stages/` — per-stage contracts, guards and
+failure modes in `docs/spec-pipeline.md`; the HTTP surface in
+`docs/spec-api.md`).
+
+```
+┌─────────────────────────────────────────────────────────────────────┐
+│                        USER (browser / API)                         │
+│  chat UI — Vue islands (Astro 7/Vite 8/Tailwind 4): sessions,      │
+│  fork, edit, export, markdown copy, globe spinner; suggestions +   │
+│  datasets panel render the API response verbatim (no hardcoded     │
+│  client content)                                                    │
+│  ↓ SSE (stream:true, stop, prev, history[20])                       │
+├─────────────────────────────────────────────────────────────────────┤
+│                        WORKER (rag-public)                          │
+│                                                                     │
+│  ┌─ 0. CONVERSATIONAL ROUTE (LLM-decided) ────────────────────────┐ │
+│  │  understanding.intent: conversational | knowledge               │ │
+│  │  · greetings, identity, capability, small talk (ANY language)  │ │
+│  │    → answered directly from the DATASETS catalog facts         │ │
+│  │    (prompts/conversational.md), no retrieval, never refused    │ │
+│  │  · off-topic SUBJECT questions are knowledge (honest refusal   │ │
+│  │    + redirect) — "conversational" never means off-topic        │ │
+│  │  · asymmetric default: null/doubt → knowledge path             │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 1. QUERY UNDERSTANDING (LLM) ─────────────────────────────────┐ │
+│  │  qwen3-30b-a3b → strict JSON (prompts/understanding.md):       │ │
+│  │  · intent (see layer 0)  · docidentifier/docnumber             │ │
+│  │  · process_intent  (certify/apply → B-series, not R-series)    │ │
+│  │  · term  ("what is a load cell" → "load cell")                 │ │
+│  │  · standalone_query  (follow-ups folded with context)          │ │
+│  │  · complexity  (simple vs complex → adaptive depth)            │ │
+│  │  · query_variants[2-3]  (alternative phrasings for fusion)    │ │
+│  │  · sub_queries[2-4]  (decomposition for complex questions)     │ │
+│  │  · hypothetical_answer  (→ HyDE embedding)                     │ │
+│  │  UNISON with deterministic regexes (union, not either-or)     │ │
+│  │  1200 max_tokens (reasoning shares the budget), fresh call     │ │
+│  │  per retry, per-attempt timeouts 7s/4s; failure → vanilla      │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│  │  Concurrently: the folded query is embedded while understanding │
+│  │  runs (warm embedding) — reused whenever the final retrieval    │
+│  │  query is unchanged; understand→embed collapses to max()        │ │
+│                              ↓                                      │
+│  ┌─ 2. MULTI-QUERY RAG-FUSION ────────────────────────────────────┐ │
+│  │  Primary query + each query_variant → own embedding            │ │
+│  │  Each → Vectorize query (topK=50, metadata-filtered)          │ │
+│  │  All rankings → Reciprocal Rank Fusion (k=60)                  │ │
+│  │  Ref: RAG-Fusion paper; arXiv 2604.01733 (hybrid+RRF > alone) │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 3. MULTI-HOP DECOMPOSITION (complex only) ────────────────────┐ │
+│  │  Each sub_query → own embedding + Vectorize (topK=15)          │ │
+│  │  Results merged as complementary perspectives (union, 0.8×)   │ │
+│  │  Ref: Agent-Orchestrated Adaptive RAG (arXiv 2606.05658)      │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 4. OVERVIEW PENALTY + FAMILY BOOST ───────────────────────────┐ │
+│  │  · Overview chunks ×0.85 (boilerplate drowns clause content)  │ │
+│  │  · Family chunks → top (multi-part structure: "R 60 = 3       │ │
+│  │    parts + 1 annex") when doc-scoped query                    │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 5. CROSS-ENCODER RERANKING ───────────────────────────────────┐ │
+│  │  bge-reranker-base over all candidates                          │ │
+│  │  Catches semantic relevance dense embeddings miss              │ │
+│  │  Retry ×2 on transient failure; vector order as fallback       │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 6. HYBRID SEARCH (BM25 + Dense + RRF) ────────────────────────┐ │
+│  │  Keyword (lexical) scoring alongside rerank ranking            │ │
+│  │  RRF fusion (k=60) — catches exact terms ("n_LC", "R 60-3")   │ │
+│  │  Ref: arXiv 2604.01733; Denser.ai (RRF k=60 default)          │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 7. TERM BOOST + LANGUAGE NUDGE + EDITION RECENCY ─────────────┐ │
+│  │  · Exact term match (clause_title/body starts with term) →    │ │
+│  │    decisive boost (definition questions)                       │ │
+│  │  · Query language → prefer matching chunks (scaled to spread) │ │
+│  │  · Edition recency → tie-break (newer > older, scaled)        │ │
+│  │  All boosts scaled to observed score spread                   │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 8. DIVERSITY (per-publication caps) ──────────────────────────┐ │
+│  │  · Overview: max 2 globally (6 for doc-scoped queries)        │ │
+│  │  · Clause: max 2 per docidentifier|language (3 for filtered)  │ │
+│  │  · Family chunks bypass (always included when relevant)       │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 9. CRAG GRADER ───────────────────────────────────────────────┐ │
+│  │  deepseek-v4-flash grades passages good/weak/bad               │ │
+│  │  · good → generate                                             │ │
+│  │  · weak → ONE corrective re-retrieval (doc ident made          │ │
+│  │    explicit); accepted only on strictly better grade           │ │
+│  │  · bad → strict generation (refuses honestly)                  │ │
+│  │  Ref: CRAG paper (arXiv 2401.15884)                            │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 10. GENERATION ───────────────────────────────────────────────┐ │
+│  │  Anon: qwen3-30b-a3b-fp8 | Member: qwen3.8-27b                │ │
+│  │  System prompt is DATA (prompts/system.md, {{PLACEHOLDERS}}   │ │
+│  │  filled from code; per-corpus notes travel with the DATASETS  │ │
+│  │  catalog entry)                                                 │ │
+│  │  CONTEXT BUDGET (16k tokens est., CJK/Arabic-aware):           │ │
+│  │  · history 30% slice, newest-first, turns clipped 600 tok     │ │
+│  │  · overflow turns SUMMARIZED into a continuity block          │ │
+│  │    (prompts/summarize.md) — never silently dropped            │ │
+│  │  · passages fill the rest, best-ranked first, 900 tok/chunk   │ │
+│  │  · citations built from passages actually included            │ │
+│  │  Streaming SSE (3072 max_tokens, reasoning_effort: low)        │ │
+│  │  Strict grounding + mandatory process-answer rule; refusals   │ │
+│  │  canonicalized to the exact contract sentence (model           │ │
+│  │  paraphrases are normalized server-side)                       │ │
+│  │  Citations sorted: in-force → unknown → superseded/withdrawn  │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 11. SELF-RAG REFLECTION ──────────────────────────────────────┐ │
+│  │  Model critiques: is every claim grounded in the passages?     │ │
+│  │  If not → re-retrieve targeting missing info → regenerate     │ │
+│  │  (max one retry; better-grounded answer wins)                  │ │
+│  │  Ref: selfrag.github.io; arXiv 2606.05658 (bounded reflection) │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                              ↓                                      │
+│  ┌─ 12. ANSWER CACHE (KV) ────────────────────────────────────────┐ │
+│  │  Keyed on INDEX_VERSION + query + lang                         │ │
+│  │  · Cache hit → SSE stream (must speak SSE)                    │ │
+│  │  · Regenerate (fresh=true) → skip cache read                   │ │
+│  │  · Contextual follow-ups → skip cache entirely                 │ │
+│  │  · Refusals are NEVER cached (they describe the moment, not    │ │
+│  │    the question — a cached refusal poisons retries)            │ │
+│  │  INDEX_VERSION bumped on any retrieval/logic change            │ │
+│  └────────────────────────────────────────────────────────────────┘ │
+│                                                                     │
+├─────────────────────────────────────────────────────────────────────┤
+│                        INDEX PIPELINE (ingest)                       │
+│                                                                     │
+│  ┌─ Source Corpora ────────────────────────────────────────────────┐│
+│  │  Clean: mn-samples-oiml (29 curated Metanorma collections)     ││
+│  │  Dirty: publications-private (880 OCR-derived docs)            ││
+│  │  ISO: smartcab-refs (16 ISO/IEC, internal-only, pending)      ││
+│  │  Precedence: clean wins over dirty on overlap                 ││
+│  └────────────────────────────────────────────────────────────────┘│
+│                              ↓                                      │
+│  ┌─ Extraction ────────────────────────────────────────────────────┐│
+│  │  HTML-first: BeautifulSoup over compiled Metanorma HTML        ││
+│  │  · Rendered tables (caption + rows)                            ││
+│  │  · Clause-number anchors from heading text                     ││
+│  │  · Literal-tag stripping (OCR `<td>` artifacts)               ││
+│  │  · GUID anchor dropping (machine-generated ids)               ││
+│  │  · collection.yml manifest as identity authority               ││
+│  │  English-only (user directive; language detection from        ││
+│  │  slug suffixes + adoc attrs)                                  ││
+│  └────────────────────────────────────────────────────────────────┘│
+│                              ↓                                      │
+│  ┌─ Family Chunks ─────────────────────────────────────────────────┐│
+│  │  47 synthetic chunks encoding multi-part structure             ││
+│  │  ("OIML R 60 comprises 3 parts and 1 annex")                  ││
+│  │  Deterministic IDs (content-hashed for re-embedding on        ││
+│  │  text change)                                                 ││
+│  └────────────────────────────────────────────────────────────────┘│
+│                              ↓                                      │
+│  ┌─ Status Enrichment ─────────────────────────────────────────────┐│
+│  │  relaton-data-oiml (5,707 YAML records) joined at ingest       ││
+│  │  · status: in-force / superseded / withdrawn / joint          ││
+│  │  · superseded_by: successor identifier                        ││
+│  └────────────────────────────────────────────────────────────────┘│
+│                              ↓                                      │
+│  ┌─ Contextual Enrichment (RUNNING over 31k chunks) ───────────────┐│
+│  │  Anthropic contextual retrieval technique                      ││
+│  │  POST /admin/enrich (Bearer ADMIN_TOKEN): deepseek-v4-pro      ││
+│  │  writes a 1-sentence situating context per chunk              ││
+│  │  (prompts/enrichment.md; 1600 max_tokens — reasoning          ││
+│  │  models starve below that), KV-cached 30d per chunk id;       ││
+│  │  context+text re-embedded and upserted in place (ctx flag)    ││
+│  │  Driver: ingest/enrich.py — paced (--rpm), rate-aware         ││
+│  │  backoff on Workers AI 3021s, resumable state file;           ││
+│  │  ~$0.0015/chunk → ≈$50 one-time (quality-first lane)          ││
+│  └────────────────────────────────────────────────────────────────┘│
+│                              ↓                                      │
+│  ┌─ Embedding + Indexing ──────────────────────────────────────────┐│
+│  │  @cf/qwen/qwen3-embedding-0.6b (1024-dim, 100+ languages)     ││
+│  │  → Vectorize idx_oiml_public_v2 (cosine, 31k English vectors) ││
+│  │  Metadata indexes: doctype, doc_number, edition, language     ││
+│  │  Resumable embed (25/batch, per-item fallback)               ││
+│  │  Resumable upsert (batch cursor, idempotent)                  ││
+│  │  Orphan deletion (stale IDs purged on re-index)              ││
+│  └────────────────────────────────────────────────────────────────┘│
+│                                                                     │
+├─────────────────────────────────────────────────────────────────────┤
+│                        EVALUATION                                   │
+│                                                                     │
+│  ┌─ Golden Set (19 cases) ────────────────────────────────────────┐│
+│  │  Doc-level, definitions, table values, edition filters,        ││
+│  │  multilingual, refusals, leakage probes, process intent,      ││
+│  │  family structure, test report forms                           ││
+│  │  npm run test:golden (90% threshold, exit 1 on regression)   ││
+│  └────────────────────────────────────────────────────────────────┘│
+│  ┌─ Faithfulness (RAGAS-style) ────────────────────────────────────┐│
+│  │  LLM-as-judge: every claim grounded in passages? (0-1)       ││
+│  │  Reported per case + averaged in eval-report.json             ││
+│  │  Ref: docs.ragas.io faithfulness metric                       ││
+│  └────────────────────────────────────────────────────────────────┘│
+│  ┌─ Live E2E (13 cases) ───────────────────────────────────────────┐│
+│  │  Health, ask quality, citations, refusals + redirect, meta     ││
+│  │  turns (EN/FR/DE), 200-word questions, French, auth, quotas   ││
+│  │  tests/e2e.mjs                                                  ││
+│  └────────────────────────────────────────────────────────────────┘│
+│  ┌─ Retrieval eval (hit@5) ─────────────────────────────────────────┐│
+│  │  Golden cases + 8 vocabulary-mismatch PARAPHRASE probes        ││
+│  │  (the contextual-enrichment failure mode) via /v1/search;     ││
+│  │  snapshots to artifacts/eval/ for before/after lift          ││
+│  │  tests/retrieval.mjs                                            ││
+│  └────────────────────────────────────────────────────────────────┘│
+│  ┌─ UI (Playwright, 30 checks, runs in CI) ─────────────────────────┐│
+│  │  Real Chromium over the built site with stubbed APIs (SSE      ││
+│  │  included): ask/stream/citations/superseded badges/sessions   ││
+│  │  CRUD/filter/persistence/XSS safety. tests/ui.mjs; CI also    ││
+│  │  builds the site for real (site-shell checked out)            ││
+│  └────────────────────────────────────────────────────────────────┘│
+│  ┌─ Browser E2E ────────────────────────────────────────────────────┐│
+│  │  Production layout + console-error probe. tests/browser.mjs   ││
+│  └────────────────────────────────────────────────────────────────┘│
+│                                                                     │
+├─────────────────────────────────────────────────────────────────────┤
+│                        AUTH & TIERS                                 │
+│                                                                     │
+│  · OIDC RP on id.oimlsmart.org (authorization code + PKCE,        │
+│    ES256 via JWKS, RFC 8414 discovery)                             │
+│  · RAG-minted HMAC session cookie (7-day, sliding renewal)        │
+│  · Anon: 20/day per IP (KV counter)                                │
+│  · Member: 300/day, qwen3.8-27b model                              │
+│  · API key: per-key limits, D1-backed                              │
+│  · Operator IP exemption (env + KV override)                       │
+│  · Role-gated datasets panel (ISO corpus: mc_member, etc.)        │
+│  · Conversations D1 API (member-only, sub-keyed)                  │
+│                                                                     │
+├─────────────────────────────────────────────────────────────────────┤
+│                        TECHNIQUE PROVENANCE                         │
+│                                                                     │
+│  ┌────────────────────────────┬───────────────────────────────────┐│
+│  │ Technique                   │ Source                            ││
+│  ├────────────────────────────┼───────────────────────────────────┤│
+│  │ Query understanding (LLM)   │ Agentic RAG survey (2501.09136) ││
+│  │ Multi-query RAG-Fusion      │ RAG-Fusion paper (Semantic      ││
+│  │                             │ Scholar b4d1da74)                ││
+│  │ Hybrid search (BM25+RRF)    │ arXiv 2604.01733; Denser.ai     ││
+│  │ Cross-encoder reranking     │ Standard (bge-reranker-base)     ││
+│  │ CRAG corrective grading     │ arXiv 2401.15884                 ││
+│  │ Self-RAG reflection         │ selfrag.github.io                ││
+│  │ Adaptive decomposition      │ arXiv 2606.05658                 ││
+│  │ Contextual enrichment       │ anthropic.com/engineering/      ││
+│  │                             │ contextual-retrieval              ││
+│  │ Late chunking (alternative) │ arXiv 2409.04701                 ││
+│  │ RAGAS faithfulness          │ docs.ragas.io                    ││
+│  │ Family chunks               │ novel (metanorma/metanorma#592) ││
+│  │ Publication status marking  │ novel (relaton join at ingest)  ││
+│  └────────────────────────────┴───────────────────────────────────┘│
+│                                                                     │
+│                        NOT YET IMPLEMENTED                          │
+│                                                                     │
+│  · GraphRAG (knowledge graph over terms/documents) — vocab repo    │
+│    has 6,031 concepts; relaton has 5,707 relations                │
+│  · FLARE (forward-looking active retrieval) — predict next        │
+│    sentence to anticipate retrieval needs                         │
+│  · Speculative RAG (parallel draft generation)                    │
+│  · Turnstile/WAF (needs dashboard sitekey)                        │
+│                                                                     │
+│  Shipped since the 08-24 revision: HyDE (hypothetical-answer       │
+│  embedding via understanding), ISO internal tier + federated       │
+│  retrieval (worker_internal /retrieve + RRF), citation deep        │
+│  links (R2 renderings), shareable permalinks, contextual           │
+│  enrichment (running), conversational routing, context budget      │
+│  + compaction, prompts-as-data, CI with real site build +          │
+│  Playwright UI suite, retrieval eval harness                       │
+└─────────────────────────────────────────────────────────────────────┘
+```
+
+
+## 2026-09 additions (delta over the 2026-08-26 map)
+
+- **Multimodal generation**: figure units among the used passages attach
+  their R2 pixels to the glm-5.3-flash call (`attachFigureImages`) — the
+  model reads labels that exist only in the image. Users can attach a
+  photo (`image` data URL on /v1/ask, validated, 6MB) — text still drives
+  retrieval; image asks bypass both caches.
+- **Edition steering**: family-relative demotion of superseded editions
+  (cross-publication recency kept); edition pins corroborated by the
+  corpus (<3 chunks → drop to doc-only). Fixed superseded-citation drift.
+- **Citation labels**: model-facing passage headers and UI chips strip
+  OIML language markers, dedupe editions, never show UUID anchors.
+- **Lexical lane carries unit identity** (chunks.unit_id/block) — typed
+  chunks arriving via BM25 keep their [[u:…]] contract.
+- **Research loop**: keep-recent-10 context folding (older evidence as
+  digests) + per-round focus folding.
+- **Witness-span eval**: a hit requires ≥75% containment of the golden
+  answer span (ETSI protocol); graph-lane probes.
+- **Pipeline**: unit-level langid tags bilingual annexes; typed tables in
+  the lexical lane; `fts --incremental`; per-doc ingest resilience.

package/docs/AUDIT-2026-08-24.md

@@ -0,0 +1,71 @@
+# Deep audit — 2026-08-24
+
+Evidence-based review of code, architecture, UX, and results. Every finding
+is grounded in this session's incidents, the live telemetry (642 asks, 7
+failures, 0 feedback votes, 0 member conversations yet), and 2025/2026 RAG
+literature. Fixed items are marked ✅ (this session); the rest are ranked.
+
+## 1. Results quality
+
+| Finding | Status / Priority |
+|---|---|
+| Regex heuristics were the decision-maker between users and the model — the top defect source (E2-class hijack, glued `r60`, `de`-as-German, `forms`-drift). Replaced by an LLM query-understanding layer, **unioned** with the regexes (regex = tested floor, LLM = creative phrasings + context rewriting). Pure-LLM gating was tried and REJECTED with evidence: flaky JSON/latency lost the working expansions | ✅ shipped (understand.ts) |
+| No corrective loop: weak retrieval went straight to generation | ✅ CRAG grader shipped (deepseek-v4-flash grades good/weak/bad; weak → one corrective re-retrieval, accepted only on a strictly better grade) |
+| Exact-count/TOC questions ("how many forms") — honest partial answers; needs synopsis/TOC chunks or table extraction (see Metanorma-for-AI P1 tables) | Medium |
+| Enrichment pass (contextual chunk prefixes, deepseek-v4-pro one-time) planned, never run — recall is the main retrieval ceiling | **High** |
+| Terminology (vocab, 6,031 concepts) and relaton bibliography ingested only for status — exact definition lookup from Glossarist datasets unused | **High** (cheap, big quality win) |
+| Doc-as-context mode (deepseek-v4-flash 1.31M ctx) for summarize-the-document questions — planned, not built | Medium |
+| Graph-augmented retrieval (CLAUDE.md architecture) — not started | Low |
+| Eval: 19 golden cases, manual runs, no trend history; no RAGAS-style faithfulness/relevancy metrics; 👍/👎 feedback never reviewed | **High** (nightly golden in CI + faithfulness scoring + feedback triage) |
+| Multilingual: English-only corpus (user directive); French questions answered from EN sources — works; reversal requires interlinear alignment (mn-for-AI P2) | Recorded |
+
+## 2. Architecture
+
+| Finding | Priority |
+|---|---|
+| worker_internal + ISO tier: datasets panel sells it, `idx_iso_internal` exists but is EMPTY — ingest smartcab-refs + federated internal worker | **Top gap** |
+| KV quota counter is get→put (not atomic) — concurrent requests can exceed limits; Workers Rate Limiting binding is the proper primitive | Medium |
+| No model fallback ladder (member qwen3.8-27b failure → 502; CLAUDE.md specifies a ladder ending at z.ai GLM) | Medium |
+| D1 unbounded growth: queries/spend/feedback/conversations never pruned — retention cron + delete >90d | Medium |
+| Session: 7d HMAC cookie, no OP revalidation mid-session; shorten members to 24h or re-check on /auth/me | Medium |
+| Conversations API: member write path unrate-limited (ask quota doesn't cover message appends) | Medium |
+| Deploy friction: INDEX_VERSION bumped 30+ times manually; stale-isolate poisoning twice → automate bump + 90s settle + smoke-test in a deploy script | **High** (scripts/deploy.sh) |
+| Ingest is laptop-manual; weekly reindex cron in the architecture diagram isn't wired; scripts/ pile of one-offs → fold into ingest subcommands | Medium |
+| Cloud sync is fire-and-forget (message loss on flaky networks); localStorage keeps last 100 convs only (silent guest loss) — outbox retry queue | Medium |
+| Observability: telemetry exists but nobody looks — /v1/admin/stats + spend alerts (50/80/100) + error-rate dashboard | **High** |
+| Turnstile absent (needs dashboard sitekey); hard cap 5000/day bounds worst case ≈ $2/day | Medium |
+
+## 3. Code quality
+
+- `ui.ts` renderConversation duplicates beginAssistantMessage's done() — extract a shared renderAssistantBlock (Medium).
+- Worker `any`-typing throughout (AI responses, env) — type the Workers AI result shapes (Low).
+- `selfquery.ts` retained as the union's deterministic floor — now has tests via golden; keep frozen, no new heuristics without a golden case.
+- linkifyCitations interpolates corpus identifiers into a title attribute — escape quotes (Low, hardening).
+- No worker unit tests (vitest-pool-workers); ui.mjs is monolithic but effective.
+
+## 4. UX (remaining)
+
+- **Citation deep links** — the biggest trust gap: citations show text but never link to the document. Link to oiml.org publication pages now; R2 clause-anchored renderings later (**High**).
+- Share conversation permalinks (public read route + rate limit) — marketing value for a public service (Medium).
+- Mobile: visualViewport handling for the composer above the keyboard; drawer focus-trap; aria-live is per-chat not per-message (Medium).
+- /api page is stale — document /v1/ask+search, conversations, /auth/*, /api/datasets, keys (Medium).
+- Feedback prompt is subtle — nudge after an answer settles; feedback is the eval flywheel and has 0 uses (Medium).
+- UI strings are English-only; empty-state suggestions static (Low).
+
+## 5. What today's incidents taught (process)
+
+1. **Bugs clustered at heuristic boundaries** (regex layers) — the union architecture exists so no single layer gates silently.
+2. **Deploy propagation (~90s) + answer cache = poisoned cache** — twice. Automate the settle-and-smoke step.
+3. **Live verification beats type-checks** — every real bug today was caught by Playwright/D1 probes, none by tsc.
+
+## Sources (2025/2026 SOTA survey)
+
+- [Engineering the RAG Stack: A Comprehensive Review (arXiv)](https://arxiv.org/html/2601.05264v1)
+- [Agentic RAG with LangGraph — retrieval grading, rewriting, corrective loops](https://www.langchain.com/blog/agentic-rag-with-langgraph)
+- [CRAG: Corrective Retrieval Augmented Generation (arXiv 2401.15884)](https://arxiv.org/html/2401.15884v3)
+- [Corrective RAG workflow — Meilisearch](https://www.meilisearch.com/blog/corrective-rag)
+- [GraphRAG vs CRAG vs Self-RAG — TowardsAI](https://pubs.towardsai.net/advanced-rag-comparing-graphrag-corrective-rag-and-self-rag-00491de494e4)
+- [RAG best practices from 100+ teams — kapa.ai](https://www.kapa.ai/blog/rag-best-practices)
+- [RAGAS metrics: faithfulness, answer relevancy](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/)
+- [RAGAS paper (arXiv 2309.15217)](https://arxiv.org/html/2309.15217v1)
+- [From RAG to Context — 2025 review (RAGFlow)](https://ragflow.io/blog/rag-review-2025-from-rag-to-context)

package/docs/CONTRIBUTOR-AUDIT-2026-08-25.md

@@ -0,0 +1,147 @@
+# Contributor Audit — 2026-08-25
+
+## Verdict: REJECT the unstaged site/ changes
+
+The contributor's changes convert the Astro site from **static output**
+to **SSR on Cloudflare Workers** via `@astrojs/cloudflare`. This breaks
+our architecture with zero functional benefit.
+
+### What the contributor did
+
+| File | Change | Intent |
+|------|--------|--------|
+| `site/astro.config.mjs` | Added `adapter: cloudflare()` | Convert static → SSR |
+| `site/package.json` | Added `@astrojs/cloudflare` dep | Enable SSR adapter |
+| `site/tsconfig.json` | Added `worker-configuration.d.ts` include | Types for SSR runtime |
+| `site/wrangler.jsonc` | NEW — second worker config | Deploy site as separate worker |
+| `site/public/.assetsignore` | NEW — exclude `_worker.js` | Framework-fighting artifact |
+| `site/public/spinning-globe-3d.js` | DELETED | We replaced with mini (intentional) |
+
+### Why this is wrong
+
+**Our architecture** (documented in CLAUDE.md, docs/ARCHITECTURE.md):
+```
+Browser → rag-public worker (ai.oimlsmart.org)
+    ├── /           → static HTML from ASSETS binding
+    ├── /api/*      → handler code
+    ├── /v1/*       → handler code
+    └── /auth/*     → handler code
+```
+ONE worker. Static assets served through the worker's `[assets]` binding.
+The chat app is a client-side SPA talking to the API via fetch.
+
+**What the contributor's changes would produce:**
+```
+Browser → which worker? (custom domain conflict)
+    ├── oimlsmart-rag-site (SSR worker)
+    │     └── / → server-rendered on every request
+    └── rag-public (API worker)
+          └── /api/* → handler code
+```
+TWO workers. SSR compute on every page load. Custom domain routing
+conflict. Cold starts for a page that was previously instant static HTML.
+
+### Specific violations
+
+1. **Breaks single-worker deployment** — `site/wrangler.jsonc` creates
+   a second deployment target (`oimlsmart-rag-site`) that conflicts with
+   `workers/worker_public/wrangler.toml` for the custom domain.
+
+2. **SSR is unnecessary** — the site has zero server-side rendering
+   needs. It's a chat UI that loads once and talks to the API via
+   fetch/SSE. Adding SSR adds compute cost and cold starts with no benefit.
+
+3. **Framework-fighting artifacts** — the `.assetsignore` file exists
+   solely to work around the adapter generating `_worker.js` in the
+   public directory. This is the framework telling you it doesn't fit.
+
+4. **Incomplete** — `@astrojs/cloudflare` is in package.json but
+   package-lock.json wasn't updated (0 new packages found). The
+   contributor didn't even install/test their own changes.
+
+5. **Violates CLAUDE.md ecosystem constraints** — "Deployment is purely
+   Cloudflare" is about the PLATFORM, not about adding more workers.
+   The minimal-infrastructure principle means one worker, not two.
+
+### What to do
+
+Revert all unstaged changes in `site/`:
+```bash
+git checkout -- site/astro.config.mjs site/package.json site/tsconfig.json
+rm site/wrangler.jsonc site/public/.assetsignore
+```
+
+If SSR is ever genuinely needed (e.g., for A/B testing, edge-side
+personalization, or server components), it should be an explicit
+architectural decision with a written proposal — not a drive-by adapter.
+
+---
+
+## Our code — quality audit
+
+### DRY violations (medium priority)
+
+- `json()` and `err()` helpers duplicated in **3 files**: index.ts,
+  conversations.ts, share.ts. Extract to `src/lib/http.ts`.
+
+- `Hit` construction (`matches.map((m) => ({ id: m.id, score: m.score,
+  metadata: m.metadata, text: ... }))`) duplicated **5 times** in
+  pipeline.ts (primary, fallback, RAG-Fusion variants, HyDE, multi-hop).
+  Extract to `toHit(m: any): Hit`.
+
+### OCP violations (medium priority)
+
+- **Router**: 16-branch if-chain in a 636-line index.ts. Adding a route
+  requires editing the router AND creating a handler — the router should
+  be a declarative route table (Map of path patterns to handlers). New
+  route = new entry, no router modification.
+
+- **Pipeline**: the `retrieve()` function at 377 lines contains all
+  retrieval techniques inline (embedding, vector search, HyDE, multi-query
+  fusion, multi-hop decomposition, overview penalty, family boost,
+  reranking, keyword ranking, RRF fusion, term boost, language nudge,
+  edition boost, diversity). Each new technique modifies the function.
+  Should be a composable pipeline: `const hits = pipe(query,
+  understanding, [embed, search, hyde, fuseVariants, decompose,
+  rerank, hybrid, boost, diversify])`.
+
+### `any` typing (low priority)
+
+Worker code uses `any` for env bindings, AI responses, and metadata.
+The Workers AI result shapes are known at each call site. Reduce by
+defining interfaces for the response shapes we actually use.
+
+### MECE concerns
+
+- `pipeline.ts` does retrieval + ranking + citation building + message
+  building — at least 4 responsibilities. Should be:
+  - `retrieval.ts` (embed, search, fuse)
+  - `ranking.ts` (rerank, boost, diversify)
+  - `citations.ts` (citation building)
+  - `messages.ts` (LLM message assembly)
+
+- `ui.ts` at 506 lines duplicates assistant-message rendering between
+  `beginAssistantMessage.done()` and `renderConversation` — extract a
+  shared `renderAssistantContent(wrap, msg, handlers)`.
+
+### Missing tests
+
+- No worker unit tests (vitest-pool-workers or miniflare). Route
+  handlers, retrieval pipeline stages, and auth logic are only tested
+  via live e2e. The retrieval pipeline especially would benefit from
+  deterministic unit tests (mock the AI bindings, verify ranking logic).
+
+### Recommended cleanup (prioritized)
+
+1. Reject contributor changes (above)
+2. Extract `lib/http.ts` (json/err helpers) — 30 minutes
+3. Extract `toHit()` helper — 15 minutes
+4. Route table replacing if-chain — 1 hour
+5. Pipeline middleware chain — 2 hours (careful: must preserve all
+   test behavior)
+6. Split pipeline.ts into retrieval/ranking/citations/messages — 1 hour
+7. Worker unit tests with miniflare — 2 hours
+8. Type Workers AI response shapes — 1 hour
+
+Total: ~8 hours of cleanup. Can be done incrementally; each step is
+independently valuable.