norn-cli 3.1.0 → 3.3.0

package/CLAUDE.md

@@ -31,6 +31,7 @@ Current skills and what they cover:
 | `add-sequence-step-type` | Adding a new sequence step type |
 | `add-imported-sidecar-language` | Adding a new imported sidecar file type |
 | `norn-cli-integration` | `src/cli.ts` and CLI parity |
+| `norn-editor-decorations` | Drawing onto the editor — end-of-line chips, hiding markup, folding, caret traps |
 | `norn-debugger-integration` | DAP / debugger work |
 | `norn-response-panel` | The response webview panel |
 | `norn-test-explorer` | VS Code Test Explorer integration |

package/NOW.md

@@ -25,28 +25,79 @@
 
 ## Active now
 
-_Nothing accepted right now._ The MCP-servers item finished 2026-08-12 — see **Recently done**.
-Slice 5f is the standing follow-on in the backlog below; it now has a spec and is ready for a
-fresh session the moment Peter accepts it here.
+### 📚 Knowledge Base — the third pillar
+**Accepted 2026-08-15.** Norn covers **models** (providers, contracts, evals, record/replay) and
+**capabilities** (MCP servers and tools). What an agent still cannot get is **context**. The user
+writes prose in a declared folder and Norn derives the structure; because our runs have ground
+truth, *"did editing this document make agent runs better?"* is answerable here and nowhere else.
+Eight stages, ordered by dependency — **1 → 2 → 3** is the spine and where a wrong call is most
+expensive. **Stages 1, 2, 3 and 4 are built as of 2026-08-15.** Stage 1 established declared membership,
+topic identity, source-bounded domains, tombstones and deterministic DAG lineage. Stage 2 replaced
+its temporary JSON cache with one project-global built-in SQLite authority: atomic content-addressed
+snapshots, occurrence-scoped authored metadata, versioned action caches, conservative typed entity
+crossing, FTS5 diagnostics, retention/pins, and a queued extension worker. The calibrated fixture
+kept all nine entity kinds; exact values cross automatically while service spelling aliases remain
+proposals only. The database/WAL byte test proves `.nornenv` values and ciphertext do not persist.
+The CLI remains deliberately narrow and generic at index-backed `knowledge ls --explain`. Stage 3
+now applies sidecar-local grants to one lazily pinned snapshot per top-level run, routes exact typed
+entities with one conservative lexical fallback, assembles literal cache-stable prompt blocks under
+four explicit budgets, offers the bounded `load_knowledge` tool, and writes durable masked SQLite
+run/invocation/domain/topic/load telemetry independently of rolling replay files. Pure replay stays
+offline; a live tail fails closed without its recorded snapshot. The extension awaits its worker
+queue and the compiled local CLI shares identical runner semantics without new subcommands.
+**Stage 4 turned the ordinary Markdown editor into the authoring surface** — extension-only, no CLI
+command. A declared document now shows its own interpretation (topics, effective type and its
+origin, `use_when`, token estimates, exact entities that cross into the project), honest recorded
+use from Stage 3 telemetry, folded metadata, dimmed exclusions, and actions that write
+`type`/`use_when` and exclusions without moving a topic hash. The cost boundary is the design: a
+keystroke parses the open buffer once per version and reads cached memory, while every SQLite read
+runs on the index worker. Counts are raw co-occurrence — distinct completed runs for the headline,
+invocations and earlier lineage versions in their own rows, no causal language anywhere. Nine
+Stage 1, eleven Stage 2, eleven Stage 3 and eighteen Stage 4 focused tests pass under Node 24; 345
+pass across every vscode-free suite. **One gate stays open: the visual smoke test
+(light/dark/high-contrast) needs Peter's eyes — fixture at
+[`demos/knowledge-editor`](demos/knowledge-editor).**
+**Stage 5, the coverage panel, is the next dependency boundary and its cold-session handoff is
+ready.** Reviewing it against the built code moved three things: the mock's `402 runs 94%` is two
+different measurements plus one that does not exist anywhere (`K-20`), a panel row must be an
+endpoint rather than an entity or it contradicts itself on the first screen (`K-48`), and the
+"stale ref" list is where extractor false positives accumulate (`K-46`). Three gates need Peter
+before code: those, plus CLI scope and `K-47`.
+→ **Stage 5 implementation handoff (start here next session):
+[`Docs/to-be-built/knowledge-base-stage-5.md`](Docs/to-be-built/knowledge-base-stage-5.md).**
+→ **Stage 4 build record:
+[`Docs/to-be-built/knowledge-base-stage-4.md`](Docs/to-be-built/knowledge-base-stage-4.md).**
+→ **Stage 3 build record:
+[`Docs/to-be-built/knowledge-base-stage-3.md`](Docs/to-be-built/knowledge-base-stage-3.md).**
+→ Stage 2 build record: [`Docs/to-be-built/knowledge-base-stage-2.md`](Docs/to-be-built/knowledge-base-stage-2.md).
+→ Stage 1 build record: [`Docs/to-be-built/knowledge-base-stage-1.md`](Docs/to-be-built/knowledge-base-stage-1.md).
+→ Decision record: [`knowledge-base-refinement.md`](Docs/to-be-built/knowledge-base-refinement.md)
+(`K-01`…`K-45`, every Stage 1/2 gate shut and the Stage 3 review carried forward). Reasoning lives
+here; **do not re-litigate** — if a decision looks wrong, say so and stop.
+→ Design and the *why*: [`knowledge-base.md`](Docs/to-be-built/knowledge-base.md).
+
+**The `K-28` spike answered its question — markdown cuts it, no custom renderer.** It and the
+`playground/knowledgeBase/` development corpus were deleted at the Stage 1 gate. Its durable
+findings remain in skill `norn-editor-decorations`; production knowledge code shares no spike
+foundation.
+
+Two decisions the spike overturned, both worth knowing before reading anything else: per-topic
+metadata sits **inline under each heading** (`K-13`), which removed the binding-key problem rather
+than solving it; and **depth counts** for domains (`K-30`), so a folder inside a domain is a
+sub-domain and splitting an oversized domain actually works.
+→ **Refinement worksheet (start here): [`Docs/to-be-built/knowledge-base-refinement.md`](Docs/to-be-built/knowledge-base-refinement.md)**
+→ Parent spec: [`Docs/to-be-built/knowledge-base.md`](Docs/to-be-built/knowledge-base.md)
+→ Runtime floor and SQLite authority are recorded in `K-11` / `K-12` / `K-31`; Stage 2 applied
+them and removed the temporary Stage 1 JSON store.
 
 ---
 
-## Backlog / accepted-but-not-started
-
-### 🧪 Agent Workbench — Slice 5f: eval fixtures, docs, positioning
-**Postponed 2026-08-11; the MCP work it was waiting on finished 2026-08-12, and it now has a
-spec.** Slice 5 is otherwise complete and reported across every surface (5a, 5c, 5e built; 5b and
-5d closed). 5f is smaller than its name suggests: the **website already documents `judge` well**,
-and a deterministic judge server already exists as a test fixture — so what is left is promoting
-that fixture to a demo anyone can run with **no API key** (one clearly-good case, one clearly-bad
-one), a README section that catches up with the website, and the explanation of what evals are
-for. **No `src/` change.** The spec leads with the constraint that matters: Norn's positioning is
-locked, so "positioning" here means explaining evals *from* the spine, never a new pitch.
-→ **5f spec: [`Docs/to-be-built/agent-workbench-slice-5f.md`](Docs/to-be-built/agent-workbench-slice-5f.md)**
-→ Slice 5 spec: [`Docs/to-be-built/agent-workbench-slice-5.md`](Docs/to-be-built/agent-workbench-slice-5.md)
-→ Built: [`5a`](Docs/to-be-built/agent-workbench-slice-5a.md) ·
-[`5c`](Docs/to-be-built/agent-workbench-slice-5c.md) ·
-[`5e`](Docs/to-be-built/agent-workbench-slice-5e.md)
+## Backlog / awaiting acceptance
+
+_Nothing awaiting acceptance._ (Slice 5f was scrapped 2026-08-15 — it had decayed into a reminder
+to update docs. Docs get reviewed as one pass when the website is updated for the knowledge base.
+Its spec remains at [`agent-workbench-slice-5f.md`](Docs/to-be-built/agent-workbench-slice-5f.md)
+if it is ever wanted.)
 
 ## Parked
 
@@ -58,6 +109,27 @@ narrative and how to resume:
 
 ## Recently done
 
+- **3.2.0 — the knowledge base ships, quietly** (2026-08-19) — **done.** Stages 1–4 of the
+  Knowledge Base are in the published build: declared `knowledge` blocks in `.nornagent`,
+  the project-global SQLite authority, grant-scoped routing with the bounded `load_knowledge`
+  tool, and the Markdown authoring surface. **Deliberately undocumented in public** — Peter's
+  call at release time. Stages 5–8 are unbuilt and the coverage panel is still the next
+  dependency boundary, so the website says nothing about it and neither does the README. The
+  feature is reachable by anyone who writes the block; that is accepted, not overlooked.
+  **Agent graph details now open on click, not hover.** The 650ms open delay and 120ms hide
+  delay are gone: the reader either waited on a canvas that looked inert or was chased across
+  it by a popover they never asked for. A second click closes, Escape dismisses, and the
+  navigation that used to live on the tile — reveal source, step replay, reveal declaration —
+  is now one labelled button in the detail footer, so a tile has exactly one gesture. The
+  website's two "hover an agent or a boundary" sentences were updated to match, and the
+  response panel's agent invocation tree got the docs line it never had.
+  Verified at 426 Extension Host tests and 198/200 regression; the two failures are the live
+  SQL Server Docker fixture, which was not running and was accepted as a known failure.
+
+- **Agent results view** (2026-08-14; performance follow-up 2026-08-15) — **done.** The normal
+  response panel has a safe themed invocation tree; rows/details now start collapsed and large
+  MCP values remain host-side until reveal/copy. Streamed contract/agent rows also have distinct
+  toggle identities. → [Spec](Docs/to-be-built/agent-results-view.md)
 - **3.1.0 — MCP authoring parity, and the changelog is gone** (2026-08-13) — **done.**
   Three fixes that all came out of one painful hour authoring a real server by hand. **`header`
   now reads like HTTP**: `header Name: value` is accepted alongside the original no-colon form,
@@ -147,61 +219,5 @@ narrative and how to resume:
   tests; 141 pass across every runnable suite, plus type-check, lint, bundles, regression, skill
   validation, and a real CLI run against a deterministic judge server.
   → [`Docs/to-be-built/agent-workbench-slice-5c.md`](Docs/to-be-built/agent-workbench-slice-5c.md)
-- **Agent Workbench Slice 5a — eval authoring** (2026-08-10) — **done.**
-  `judge <subject> with <Agent> expects "…"` in `.norn` sequences — **one line, like `assert`**,
-  with `expects file <path>` for a list. Built as a `judge … end judge` block first and reworked
-  on Peter's correction: the sequence is the orchestrator, so it stays a flat list of steps
-  rather than nesting a mini-language. Expectations are ordinary Norn strings, file lists resolve
-  at parse time with per-expectation origins, plus eleven `judge-*` diagnostics, grammar,
-  completions, hover, and CLI/JSON step output. Driven by Peter's answer that the first eval is a
-  checklist — all expectations must be met, exceeding them is not a failure. Reuse held:
-  expectation strings go through `agentPromptBlock.ts`'s existing escaping rather than a second
-  stack, one shared module serves runner/diagnostics/completions/hover, and statements validate
-  in the existing agent preflight so a malformed eval cannot spend a model call. A judge step
-  reports as explicitly *not judged* until 5c rather than silently passing. 15 new focused tests;
-  123 pass across every runnable suite, plus type-check, lint, bundles, regression, and skill
-  validation.
-  → [`Docs/to-be-built/agent-workbench-slice-5a.md`](Docs/to-be-built/agent-workbench-slice-5a.md)
-- **Agent Workbench Slice 4** (2026-08-09) — **done and reviewed.**
-  Default-on, secret-safe recordings under `.norn-cache/runs`; fixed 20-run retention;
-  byte-identical offline replay with zero provider/MCP side effects; strict CLI replay with JSON,
-  environment restoration, graph-drift failure, and live `--from`; agent frames/scopes,
-  breakpoints, stepping, and contract-failure navigation in the existing DAP; paused-hop canvas
-  highlighting and validated node-to-invocation navigation. Review: replay is a clean
-  `AgentProviderAdapter` with no runtime special-casing; the debugger is the existing DAP session
-  extended, not a second one; secrets are stable placeholders re-substituted from `.nornenv` at
-  replay time behind a preflight that fails naming the missing variables; and the debugger became
-  the third surface consuming the shared `agentHopPresentation` helper. 55 focused tests pass
-  across all agent suites, plus type-check, lint, and bundle build.
-  → [`Docs/to-be-built/agent-workbench-slice-4.md`](Docs/to-be-built/agent-workbench-slice-4.md)
-- **Agent Workbench Slice 3** (2026-08-09) — **done and reviewed.**
-  Read-only VS Code agent canvas with parser-first static graphs, redacted live/completed trace
-  overlays, deterministic cycle-safe layout, first-class contract states, hover/copy payloads,
-  source navigation, command and CodeLens entry points, and shared response-panel presentation.
-  Recovery resolves successful correction invocations rather than trusting a flag. Review: the
-  shared `agentHopPresentation` helper is consumed by both the canvas and the response panel so
-  they cannot drift; the trace is redacted before anything reads it; CSP is `default-src 'none'`
-  with nonce'd scripts and no external hosts; no graph library was added; the panel never writes.
-  43 focused tests pass across all agent suites, plus type-check, lint, and bundle build.
-  → [`Docs/to-be-built/agent-workbench-slice-3.md`](Docs/to-be-built/agent-workbench-slice-3.md)
-- **Agent Workbench Slice 2** (2026-08-08) — **done and reviewed.**
-  Model-directed agent-as-tool delegation, two-way boundary correction, subtree conversations,
-  configurable spend guards, recursive CLI/JSON/response-panel traces, language tooling, docs,
-  and a deterministic violate → correct → pass demo. Review: sub-agents go through the same tool
-  path as MCP tools, config precedence is agent → provider → global → built-in, parallel tool use
-  is disabled on both adapters, and the runner was extracted to `src/agents/agentRunner.ts` rather
-  than growing `sequenceRunner`. 32 focused tests pass; the language suite needs the VS Code host.
-  `max_tokens` default stays 16000 — a real OpenAI model rejected 32000 and reported a 16384
-  cap, so the generous default was walked back; raise it per-provider where a model allows it.
-  → [`Docs/to-be-built/agent-workbench-slice-2.md`](Docs/to-be-built/agent-workbench-slice-2.md)
-- **Agent Workbench Slice 1** (2026-08-08) — **done and verified running by Peter.**
-  `.nornagent` definitions, linear agent `run` handoffs, OpenAI/local/Anthropic/Google adapters,
-  MCP tools, automatic boundary contracts, live CLI + JSON traces, language tooling, focused
-  tests, and a runnable demo. Reviewed: reuse held (contracts go through `matchesSchema` and the
-  existing Contract View path), provider interface held (Google landed later as a pure new file).
-  → [`Docs/to-be-built/agent-workbench-slice-1.md`](Docs/to-be-built/agent-workbench-slice-1.md)
-- **Website k8s de-promotion** (2026-08-07) — promo surfaces stripped and deployed; docs
-  and functionality untouched. Was Phase 0 of the marketing plan.
-
 ---
-_Last updated: 2026-08-12_
+_Last updated: 2026-08-15_

package/demos/knowledge-editor/README.md

@@ -0,0 +1,74 @@
+# The knowledge editor
+
+A knowledge base small enough to read in a minute, written to show every state the Stage 4
+editor can be in. Open the files in [`handbook/`](./handbook) and the ordinary Markdown editor
+gains chips, hovers, folded metadata, dimmed exclusions and underlined entities — no custom
+editor, no preview pane, no new file type.
+
+Membership comes from [`agents.nornagent`](./agents.nornagent) and nothing else:
+
+```nornagent
+knowledge Handbook
+    source ./handbook
+    exclude **/*.draft.md
+end knowledge
+```
+
+The folder is called `handbook`, not `knowledge`. That is the point — a path convention never
+activates the editor, and a frontmatter marker never activates it either. The declaration does.
+
+## What each file demonstrates
+
+| File | What to look at |
+|---|---|
+| `handbook/payments/charges.md` | H1 and H2 treatment, a multi-line `norn:meta` block folded behind its marker, a single-line one, an excluded section, and underlined entities that cross into `api.nornapi` and `.nornenv` |
+| `handbook/payments/refunds/policy.md` | A sub-domain. `payments/refunds` is its own domain, shown separately from `payments` — loading the parent never loads the child |
+| `handbook/ops/runbook.md` | No H1, so document feedback moves to a CodeLens at the top instead of chips on a title line |
+| `handbook/ops/retired.md` | A whole-file exclusion: the document dims, stays editable, and raises no error |
+| `handbook/ops/flow.svg` | A declared member whose extension produces no topics — inside the base, no editor surface |
+| `handbook/payments/pricing-tiers.draft.md` | Excluded by the declaration's glob, so it is plain Markdown again |
+
+## See the raw text
+
+**Norn: Knowledge: Toggle Markup** puts the `##` markers and the `norn:meta` blocks back on screen
+with the chips still in place. It is also how you check the surface still works without the
+unsupported style injection that hides them.
+
+## Try the authoring actions
+
+On any topic heading, the lightbulb offers:
+
+- **Change topic type…** — writes `type:` into the adjacent metadata block, creating the block if
+  there isn't one. The topic hash does not move, because metadata is stripped before hashing.
+- **Add/Edit use_when…** — the sentence the router reads when no exact entity matches.
+- **Exclude topic** / **Include topic** — the marker is placed below any metadata block, so the
+  metadata stays bound to its heading.
+
+Inside a `norn:meta` block, completions offer `type` and `use_when`, and `task`/`reference`/
+`concept` after `type:`.
+
+## Recorded use
+
+Every count in the editor is raw co-occurrence read back from Stage 3 run telemetry: these runs
+loaded this text. Norn does not claim the text changed their outcome — that question belongs to
+Stage 8. Until you run something, the honest state is what you'll see:
+
+> No knowledge runs recorded yet.
+
+To fill it in, point the `Support` model at a provider you have keys for and run the sequence:
+
+```bash
+node ./dist/cli.js demos/knowledge-editor/support.norn --env prelive
+```
+
+Then reopen `handbook/payments/charges.md`: the title line gains a pill —
+`Loaded by 1 run (1 succeeded · 0 failed)` — and each topic hover shows its own invocations and,
+if you have since edited a topic, its earlier versions in a separate row.
+
+## Inspect the same base from the CLI
+
+```bash
+node ./dist/cli.js knowledge ls --explain demos/knowledge-editor/agents.nornagent
+```
+
+Stage 4 adds no CLI command: the editor is a VS Code surface over the same index.

package/demos/knowledge-editor/agents.nornagent

@@ -0,0 +1,16 @@
+model Support
+    provider openai
+    name gpt-4.1-mini
+end model
+
+knowledge Handbook
+    source ./handbook
+    exclude **/*.draft.md
+end knowledge
+
+agent SupportEngineer
+    model Support
+    knowledge Handbook
+    describe "Answer payment support questions from the handbook."
+    system "You are a payments support engineer. Answer only from the handbook you were given."
+end agent

package/demos/knowledge-editor/api.nornapi

@@ -0,0 +1,10 @@
+headers Json
+Content-Type: application/json
+Idempotency-Key: {{$env.IDEMPOTENCY_KEY}}
+end headers
+
+endpoints
+CreateCharge: POST /v1/charges
+GetRefund: GET /v1/refunds/{id}
+CancelCharge: DELETE /v1/charges/{id}
+end endpoints

package/demos/knowledge-editor/handbook/ops/flow.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="120" height="40"><text x="4" y="24" font-size="14">charge flow</text></svg>

package/demos/knowledge-editor/handbook/ops/retired.md

@@ -0,0 +1,9 @@
+<!-- norn:exclude -->
+
+# Retired: the old settlement runbook
+
+Excluded as a whole file, so the entire document dims. It is still ordinary Markdown you can
+read and edit — it simply contributes nothing to any run.
+
+## Nightly settlement
+We stopped running nightly settlement when the gateway moved to continuous capture.

package/demos/knowledge-editor/handbook/ops/runbook.md

@@ -0,0 +1,14 @@
+<!-- A file with no H1: the document feedback moves to a CodeLens at the top and the source
+     stays untouched. -->
+
+## Gateway timeouts
+<!-- norn:meta
+type: task
+use_when: the payment gateway is timing out
+-->
+1. Check the gateway dashboard before touching anything.
+2. Retry once. A second timeout is an incident, not a retry.
+3. HTTP status 504 from `payment-svc` means the gateway, not us.
+
+## Who to wake
+The on-call rota lives in the team handbook. Wake the payments on-call, not the platform one.

package/demos/knowledge-editor/handbook/payments/charges.md

@@ -0,0 +1,25 @@
+# Charges
+
+The charge lifecycle, written for a support engineer who has the customer on the line.
+
+## Create a charge
+<!-- norn:meta
+type: task
+use_when: creating a charge, or retrying one that failed
+-->
+1. Send `POST /v1/charges` with the amount in minor units.
+2. Always send the header `Idempotency-Key`. A retry without one can charge twice.
+3. A success is HTTP status 201 and a charge id in the body.
+
+The service `payment-svc` owns this endpoint in the `prelive` environment, and the key comes
+from {{$env.IDEMPOTENCY_KEY}}.
+
+## What CARD_DECLINED means
+<!-- norn:meta type: reference -->
+The error code "CARD_DECLINED" is the issuer's answer, not ours. Nothing we retry will change
+it inside the same hour, so the customer needs a different card or their bank.
+
+## Legacy charge lookup
+<!-- norn:exclude -->
+The endpoint `GET /v1/charges/legacy` was retired in 2024. Kept for archaeology; excluded from
+the knowledge base so no agent quotes it back to a customer.

package/demos/knowledge-editor/handbook/payments/pricing-tiers.draft.md

@@ -0,0 +1,7 @@
+# Pricing tiers (draft)
+
+Excluded by the declaration's `exclude **/*.draft.md` glob, so this file is not a member at
+all: no chips, no CodeLens, no hovers. It is a plain Markdown file again.
+
+## Tier 2
+Numbers not agreed yet.

package/demos/knowledge-editor/handbook/payments/refunds/policy.md

@@ -0,0 +1,12 @@
+# Refunds
+
+## When we refund without asking
+<!-- norn:meta
+type: reference
+use_when: deciding whether a refund needs approval
+-->
+Under £50, refund it. The endpoint is `GET /v1/refunds/{id}` to check state first, and the
+service is `payment-svc`.
+
+## Partial refunds
+Split the amount across the original charge. A partial refund still returns HTTP status 200.

package/demos/knowledge-editor/support.norn

@@ -0,0 +1,10 @@
+[CreateCharge]
+POST {{BASE_URL}}/v1/charges
+content-type: application/json
+Idempotency-Key: {{$env.IDEMPOTENCY_KEY}}
+
+{"amount": 1200, "currency": "gbp"}
+
+sequence AnswerASupportQuestion
+    run SupportEngineer "A charge came back CARD_DECLINED. What should the customer do?"
+end sequence