@onlooker-community/ecosystem 0.24.0 → 0.25.1

package/.claude-plugin/marketplace.json

@@ -5,26 +5,26 @@
     "email": "community@onlooker.dev"
   },
   "metadata": {
-    "description": "TODO Fill this out"
+    "description": "Composable observability, memory, and quality-gate plugins for Claude Code — all built on the Onlooker ecosystem event substrate."
   },
   "plugins": [
     {
       "name": "ecosystem",
       "source": "./",
-      "description": "Fill this out",
+      "description": "Observability substrate for Claude Code. Provides the shared $ONLOOKER_DIR storage root (default $HOME/.onlooker), canonical schema-validated event emission, session and tool tracking hooks, and prompt rules. Required by all other Onlooker plugins.",
       "author": {
         "name": "Onlooker Community"
       },
       "homepage": "https://onlooker.dev",
       "repository": "https://github.com/onlooker-community/ecosystem",
       "license": "MIT",
-      "keywords": [],
-      "tags": []
+      "keywords": ["observability", "substrate", "events", "hooks", "telemetry"],
+      "tags": ["observability", "substrate"]
     },
     {
       "name": "archivist",
       "source": "./plugins/archivist",
-      "description": "Structured session memory across context truncation. Extracts decisions, dead ends, and open questions on PreCompact and reinjects the most important items at SessionStart. Requires the ecosystem plugin.",
+      "description": "Structured session memory across context truncation: extracts decisions, dead ends, and open questions on PreCompact and reinjects the most important items at SessionStart. Requires the ecosystem plugin.",
       "author": {
         "name": "Onlooker Community"
       },
@@ -34,10 +34,23 @@
       "keywords": ["memory", "compaction", "context", "session"],
       "tags": ["memory", "context-engineering"]
     },
+    {
+      "name": "cartographer",
+      "source": "./plugins/cartographer",
+      "description": "Proactive periodic auditor of the persistent instruction layer (CLAUDE.md, AGENTS.md, .claude/rules/). Discovers all instruction files in the repo, extracts semantic maps, and surfaces contradictions, shadowing, gaps, and drift before they cause expensive agent misbehavior. Requires the ecosystem plugin.",
+      "author": {
+        "name": "Onlooker Community"
+      },
+      "homepage": "https://onlooker.dev",
+      "repository": "https://github.com/onlooker-community/ecosystem",
+      "license": "MIT",
+      "keywords": ["instructions", "audit", "claude-md", "agents-md", "drift", "consistency"],
+      "tags": ["instructions", "context-engineering"]
+    },
     {
       "name": "tribunal",
       "source": "./plugins/tribunal",
-      "description": "Multi-agent execution with LLM-as-a-Judge quality gates. An Actor performs work; a jury of typed Judges scores it against a project-overridable rubric; a Meta-Judge reviews the jury for bias; the gate decides accept, retry, or exhaust. Requires the ecosystem plugin.",
+      "description": "Multi-agent execution with LLM-as-a-Judge quality gates. An Actor performs work; a jury of typed Judges scores it against a project-overridable rubric; a Meta-Judge reviews the jury for bias; the gate decides accept, retry, or exhaust. Grounded in LLM-as-a-Judge (Zheng et al. 2023) and LLM-as-a-Meta-Judge (Wu et al. 2024). Requires the ecosystem plugin.",
       "author": {
         "name": "Onlooker Community"
       },
@@ -76,7 +89,7 @@
     {
       "name": "compass",
       "source": "./plugins/compass",
-      "description": "Pre-write intent clarity gate. Intercepts write-class tool calls and samples N=5 parallel evaluators to score intent clarity before allowing writes to proceed. Blocks when confidence is low or evaluators disagree, surfacing a structured clarification prompt. Requires the ecosystem plugin.",
+      "description": "Pre-write intent clarity gate. Intercepts write-class tool calls and requires a confidence threshold before allowing them to proceed. Evaluates the pending write against the prior assistant turn as context to avoid false positives on question-answer turns. Requires the ecosystem plugin.",
       "author": {
         "name": "Onlooker Community"
       },
@@ -89,7 +102,7 @@
     {
       "name": "scribe",
       "source": "./plugins/scribe",
-      "description": "Intent documentation from agent activity. Captures why changes were made — problem context, decisions, tradeoffs, and constraints — and distills them into readable Markdown artifacts at session end. Git logs record what changed; scribe records why. Requires the ecosystem plugin.",
+      "description": "Intent documentation from agent activity. Captures why changes were made — problem context, decisions, tradeoffs — and distills them into readable artifacts at session end. Requires the ecosystem plugin.",
       "author": {
         "name": "Onlooker Community"
       },
@@ -102,7 +115,7 @@
     {
       "name": "counsel",
       "source": "./plugins/counsel",
-      "description": "Weekly synthesis and recommendations from your full observability stack. Reads all plugin event logs, identifies patterns, surfaces improvement opportunities, and injects a structured brief at session start when the last brief is stale. Turns disparate logs into a coaching signal. Requires the ecosystem plugin.",
+      "description": "Weekly synthesis and recommendations from the full observability stack. Reads all plugin event logs, produces a structured improvement brief, and injects it at session start when the last brief is stale. Requires the ecosystem plugin.",
       "author": {
         "name": "Onlooker Community"
       },
@@ -115,7 +128,7 @@
     {
       "name": "warden",
       "source": "./plugins/warden",
-      "description": "Untrusted-content gate. Scans content flowing in through WebFetch and Read for prompt-injection patterns, and when a threat is detected closes a session-scoped gate that blocks Write, Edit, and Bash until the user explicitly clears it. Grounded in Meta's Agents Rule of Two — warden removes the agent's external-actions property while untrusted content is in play. Requires the ecosystem plugin.",
+      "description": "Untrusted-content gate. Scans content flowing in through WebFetch and Read for prompt-injection patterns, and when a threat is detected closes a session-scoped gate that blocks Write, Edit, and Bash until the user explicitly clears it. Grounded in Meta's Agents Rule of Two: an agent should hold no more than two of {private data, external actions, untrusted content} at once — warden removes the external-actions property while untrusted content is in play. Requires the ecosystem plugin.",
       "author": {
         "name": "Onlooker Community"
       },
@@ -128,7 +141,7 @@
     {
       "name": "librarian",
       "source": "./plugins/librarian",
-      "description": "Consolidation layer between archivist's per-session artifacts and the user's durable typed memory store. Reads archivist artifacts at SessionEnd, applies a durability filter, classifies survivors via Haiku into the four memory types (user, feedback, project, reference), and queues proposals for explicit confirmation via /librarian review. Auto-promotion is opt-in. Requires the ecosystem plugin.",
+      "description": "Consolidation layer between archivist's per-session artifacts and the user's durable typed memory store. Detects which session decisions, dead-ends, and open questions deserve to live across sessions, classifies them into the user/feedback/project/reference types, and queues them as proposals for explicit confirmation. Auto-promotion is opt-in. Requires the ecosystem plugin.",
       "author": {
         "name": "Onlooker Community"
       },
@@ -141,7 +154,7 @@
     {
       "name": "curator",
       "source": "./plugins/curator",
-      "description": "Maintenance layer for the typed auto-memory store. At every SessionStart, runs four cheap heuristic checks against the memories at ~/.claude/projects/<encoded>/memory/ — date_decayed (ISO-8601 dates past the grace period), path_broken (path-shaped references that don't resolve under the repo root), broken_index (MEMORY.md pointing at missing files), and orphaned_memory (files in the dir not referenced from MEMORY.md). Surfaces findings via /curator review; never edits the memory store directly. Parallel to cartographer (which audits hand-maintained instruction files), curator audits the auto-memory substrate. Requires the ecosystem plugin.",
+      "description": "Maintenance layer for the user's typed auto-memory store. At every SessionStart, runs four cheap heuristic checks (date_decayed, path_broken, broken_index, orphaned_memory) against the memories at ~/.claude/projects/<encoded>/memory/ inside a wall-clock budget. Surfaces findings as a one-line pointer to /curator review; never edits the memory store directly. Requires the ecosystem plugin.",
       "author": {
         "name": "Onlooker Community"
       },
@@ -154,7 +167,7 @@
     {
       "name": "historian",
       "source": "./plugins/historian",
-      "description": "Episodic memory layer for past Claude Code sessions. At SessionEnd, reads the session transcript, drops tool calls and tool results, chunks the remaining user + assistant turns at turn boundaries with overlap, redacts secret-shaped substrings (AWS keys, GitHub PATs, Anthropic API keys, KEY=value env assignments), and appends one JSONL line per surviving chunk to ~/.onlooker/historian/<project-key>/sessions/<session-id>.jsonl. Future-tense retrieval (vector embeddings + UserPromptSubmit similarity surfacer) lands in a follow-up; this version ships the indexing pipeline only. Requires the ecosystem plugin.",
+      "description": "Episodic memory layer. At SessionEnd, chunks and sanitizes the session transcript and stores chunks under $ONLOOKER_DIR/historian/<project-key>/sessions/ (default $HOME/.onlooker). On UserPromptSubmit, embeds the prompt and performs similarity retrieval over stored chunks to surface relevant past context. Requires the ecosystem plugin.",
       "author": {
         "name": "Onlooker Community"
       },
@@ -163,6 +176,19 @@
       "license": "MIT",
       "keywords": ["memory", "episodic", "transcript", "indexing", "session", "retrieval"],
       "tags": ["memory", "context-engineering"]
+    },
+    {
+      "name": "assayer",
+      "source": "./plugins/assayer",
+      "description": "Claim verification. At session end, parses the agent's final message for testable claims (\"I ran the tests, they pass\", \"the build is green\") and checks each against the actual command results in the session transcript, classifying it corroborated, contradicted, or unverifiable. Catches lying-without-malice. Advisory by default. Requires the ecosystem plugin.",
+      "author": {
+        "name": "Onlooker Community"
+      },
+      "homepage": "https://onlooker.dev",
+      "repository": "https://github.com/onlooker-community/ecosystem",
+      "license": "MIT",
+      "keywords": ["verification", "claims", "exit-codes", "honesty", "testing", "transcript"],
+      "tags": ["verification", "testing"]
     }
   ]
 }

package/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "ecosystem",
-  "version": "0.24.0",
-  "description": "Observability substrate for Claude Code. Provides the shared ~/.onlooker/ storage root, canonical schema-validated event emission, session and tool tracking hooks, and prompt rules. Required by all other Onlooker plugins.",
+  "version": "0.25.1",
+  "description": "Observability substrate for Claude Code. Provides the shared $ONLOOKER_DIR storage root (default $HOME/.onlooker), canonical schema-validated event emission, session and tool tracking hooks, and prompt rules. Required by all other Onlooker plugins.",
   "author": {
     "name": "Onlooker Community",
     "url": "https://onlooker.dev"

package/.release-please-manifest.json

@@ -1,15 +1,16 @@
 {
-  ".": "0.24.0",
+  ".": "0.25.1",
   "plugins/archivist": "0.1.0",
   "plugins/tribunal": "1.0.1",
   "plugins/echo": "0.2.0",
-  "plugins/cartographer": "0.2.0",
-  "plugins/governor": "0.2.0",
+  "plugins/cartographer": "0.2.1",
+  "plugins/governor": "0.2.1",
   "plugins/compass": "0.2.0",
   "plugins/scribe": "0.2.1",
   "plugins/counsel": "0.2.0",
   "plugins/warden": "0.2.0",
   "plugins/librarian": "0.2.0",
   "plugins/curator": "0.1.0",
-  "plugins/historian": "0.2.0"
+  "plugins/historian": "0.2.0",
+  "plugins/assayer": "1.0.0"
 }

package/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## [0.25.1](https://github.com/onlooker-community/ecosystem/compare/ecosystem-v0.25.0...ecosystem-v0.25.1) (2026-06-10)
+
+
+### Bug Fixes
+
+* vendor portable-lock.sh into cartographer and governor ([#73](https://github.com/onlooker-community/ecosystem/issues/73)) ([ab2c354](https://github.com/onlooker-community/ecosystem/commit/ab2c354b131c26cc642ebb51e84a043dc43cbaa1))
+
+## [0.25.0](https://github.com/onlooker-community/ecosystem/compare/ecosystem-v0.24.0...ecosystem-v0.25.0) (2026-06-04)
+
+
+### Features
+
+* **assayer:** introduce claim-verification plugin ([#70](https://github.com/onlooker-community/ecosystem/issues/70)) ([1d0500b](https://github.com/onlooker-community/ecosystem/commit/1d0500b64f8cd670d1cfa1ac070182d72696bdfd))
+
 ## [0.24.0](https://github.com/onlooker-community/ecosystem/compare/ecosystem-v0.23.1...ecosystem-v0.24.0) (2026-06-04)
 
 

package/CLAUDE.md

@@ -37,6 +37,7 @@ scripts/lib/onlooker-event.mjs  ← canonical event builder; all plugins route t
 | governor | SessionStart, PreToolUse (Task), PostToolUse (Task), Stop | Budget gates on subagent spawns; tracks spend per session |
 | tribunal | Stop + skill invocation | Post-task quality gate; also invokable via `/tribunal` |
 | warden | PostToolUse (WebFetch, Read), PreToolUse (Write, Edit, MultiEdit, Bash), SessionStart + skill invocation | Scans ingested content for injection; closes a content gate that blocks write-class tools until cleared via `/warden` |
+| assayer | Stop | Verifies the agent's final-message claims against actual command results in the transcript; advisory |
 
 Plugins communicate by emitting events to the JSONL log — they do not call each other directly. All plugins depend on the ecosystem substrate; no plugin depends on another plugin directly.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@onlooker-community/ecosystem",
-  "version": "0.24.0",
+  "version": "0.25.1",
   "description": "Agents, skills, hooks, commands, rules, and MCP configurations that power [Onlooker](https://onlooker.dev)",
   "author": {
     "name": "Onlooker Community",
@@ -19,14 +19,14 @@
     "onlooker-install": "install.sh"
   },
   "dependencies": {
-    "@onlooker-community/schema": "^2.5.0"
+    "@onlooker-community/schema": "^2.6.0"
   },
   "scripts": {
     "postinstall": "echo '\\n  onlooker-ecosystem installed!\\n  Run: npx onlooker-install typescript\\n  Docs: https://github.com/onlooker-community/ecosystem\\n'",
     "test": "npm run test:bats && npm run test:schema",
     "test:bats": "bats test/bats",
     "test:schema": "node --test test/node/*.test.mjs",
-    "test:shellcheck": "shellcheck -S error -x install.sh scripts/common.sh scripts/hooks/*.sh scripts/lib/*.sh plugins/archivist/scripts/hooks/*.sh plugins/archivist/scripts/lib/*.sh plugins/tribunal/scripts/hooks/*.sh plugins/tribunal/scripts/lib/*.sh plugins/echo/scripts/hooks/*.sh plugins/echo/scripts/lib/*.sh plugins/governor/scripts/hooks/*.sh plugins/governor/scripts/lib/*.sh plugins/compass/scripts/hooks/*.sh plugins/compass/scripts/lib/*.sh plugins/scribe/scripts/hooks/*.sh plugins/scribe/scripts/lib/*.sh plugins/counsel/scripts/hooks/*.sh plugins/counsel/scripts/lib/*.sh plugins/warden/scripts/hooks/*.sh plugins/warden/scripts/lib/*.sh plugins/librarian/scripts/hooks/*.sh plugins/librarian/scripts/lib/*.sh plugins/curator/scripts/hooks/*.sh plugins/curator/scripts/lib/*.sh plugins/historian/scripts/hooks/*.sh plugins/historian/scripts/lib/*.sh",
+    "test:shellcheck": "shellcheck -S error -x install.sh scripts/common.sh scripts/hooks/*.sh scripts/lib/*.sh plugins/archivist/scripts/hooks/*.sh plugins/archivist/scripts/lib/*.sh plugins/tribunal/scripts/hooks/*.sh plugins/tribunal/scripts/lib/*.sh plugins/echo/scripts/hooks/*.sh plugins/echo/scripts/lib/*.sh plugins/governor/scripts/hooks/*.sh plugins/governor/scripts/lib/*.sh plugins/compass/scripts/hooks/*.sh plugins/compass/scripts/lib/*.sh plugins/scribe/scripts/hooks/*.sh plugins/scribe/scripts/lib/*.sh plugins/counsel/scripts/hooks/*.sh plugins/counsel/scripts/lib/*.sh plugins/warden/scripts/hooks/*.sh plugins/warden/scripts/lib/*.sh plugins/librarian/scripts/hooks/*.sh plugins/librarian/scripts/lib/*.sh plugins/curator/scripts/hooks/*.sh plugins/curator/scripts/lib/*.sh plugins/historian/scripts/hooks/*.sh plugins/historian/scripts/lib/*.sh plugins/assayer/scripts/hooks/*.sh plugins/assayer/scripts/lib/*.sh plugins/cartographer/scripts/hooks/*.sh plugins/cartographer/scripts/lib/*.sh",
     "lint:references": "node scripts/lint/check-references.mjs",
     "lint:manifests": "node scripts/lint/check-manifests.mjs",
     "coverage:node": "node scripts/coverage/run-coverage.mjs",

package/plugins/assayer/.claude-plugin/plugin.json

@@ -0,0 +1,14 @@
+{
+  "name": "assayer",
+  "version": "1.0.0",
+  "description": "Claim verification. At session end, parses the agent's final message for testable claims (\"I ran the tests, they pass\", \"the build is green\") and checks each against the actual command results in the session transcript, classifying it corroborated, contradicted, or unverifiable. Catches lying-without-malice. Advisory by default. Builds on the Onlooker ecosystem plugin.",
+  "author": {
+    "name": "Onlooker Community",
+    "url": "https://onlooker.dev"
+  },
+  "homepage": "https://onlooker.dev",
+  "repository": "https://github.com/onlooker-community/ecosystem",
+  "license": "MIT",
+  "skills": [],
+  "agents": []
+}

package/plugins/assayer/CHANGELOG.md

@@ -0,0 +1,10 @@
+# Changelog
+
+## 1.0.0 (2026-06-04)
+
+
+### Features
+
+* **assayer:** introduce claim-verification plugin ([#70](https://github.com/onlooker-community/ecosystem/issues/70)) ([1d0500b](https://github.com/onlooker-community/ecosystem/commit/1d0500b64f8cd670d1cfa1ac070182d72696bdfd))
+
+## Changelog

package/plugins/assayer/README.md

@@ -0,0 +1,114 @@
+# Assayer
+
+Claim verification — does the agent's story match the session's receipts?
+
+When an agent finishes, it tells you what it did: "I ran the tests, they pass," "the build is green," "lint is clean." Assayer treats those as **testable claims** and checks each against what actually happened in the session — the Bash commands that ran and whether they errored. A claim that a passing run contradicts is surfaced, not silently trusted. This catches lying-without-malice: the agent isn't deceiving you, it just misremembered, or assumed, or never re-ran after a change.
+
+Assayer is a sibling plugin to [`ecosystem`](../../) and assumes the Onlooker observability substrate (`~/.onlooker/`) is present.
+
+## How it works
+
+| Hook | What Assayer does |
+|------|-------------------|
+| `Stop` | Reads the just-finished session's transcript (`transcript_path`). Extracts the agent's testable success claims from its final message with a single `claude -p` pass, cross-checks each against the actual Bash command results in the same transcript, and emits a verdict per claim plus an audit summary. Advisory only — always exits 0, never blocks Stop. |
+
+The pipeline:
+
+```
+Stop → Transcript Reader → Claim Extractor (claude -p) → Deterministic Verifier → Events
+```
+
+- **Transcript reader** pulls two things from the JSONL transcript: the **final assistant message** (where claims live) and every **Bash command** paired with its result status. Claude Code records a command as a `tool_use` block and its outcome as a `tool_result` carrying an `is_error` flag — there is no per-call numeric exit code, so `is_error` is the success/failure signal.
+- **Claim extractor** (LLM) reads only the final message and identifies success claims, tagging each with a `type` (`tests_pass`, `build_succeeds`, `lint_clean`, `types_check`, `command_succeeds`, `generic`) and a `command_keyword` — the substring it expects in the verifying command. The LLM does **not** judge truth; it only identifies claims and what would settle them.
+- **Verifier** (deterministic bash) is the factual half: for each claim it finds the most recent command matching the claim's keywords and reads its `is_error`. Same inputs always produce the same verdict.
+
+## Verdicts
+
+| Verdict | Meaning |
+|---------|---------|
+| **corroborated** | A matching command ran and succeeded. |
+| **contradicted** | A matching command ran and **failed** — the claim is not backed by the evidence. |
+| **unverified** | No matching command (`no_matching_command`), or the claim implies no checkable command (`ambiguous`). |
+
+The most **recent** matching command wins: an agent may fail, fix, and re-run, and the last run reflects the state the final message describes.
+
+## Activation
+
+Assayer is **off by default** — it calls `claude -p` on every Stop, so it is opt-in. Enable per-project in `.claude/settings.json`:
+
+```json
+{
+  "assayer": {
+    "enabled": true
+  }
+}
+```
+
+Or globally in `~/.claude/settings.json`.
+
+## Configuration
+
+All keys are optional. Unset keys fall back to the plugin's `config.json` defaults.
+
+```json
+{
+  "assayer": {
+    "enabled": false,
+    "evaluation": {
+      "model": "claude-haiku-4-5-20251001",
+      "timeout_seconds": 60
+    },
+    "max_claims": 12,
+    "min_confidence": 0.5,
+    "final_message_chars": 6000
+  }
+}
+```
+
+| Key | Default | Description |
+|-----|---------|-------------|
+| `enabled` | `false` | Must be `true` for any audit to run. |
+| `evaluation.model` | `claude-haiku-4-5-20251001` | Model used for claim extraction. Haiku is fast and cheap; the task is structured and shallow. |
+| `evaluation.timeout_seconds` | `60` | Per-pass wall-clock timeout passed to the `timeout` command. |
+| `max_claims` | `12` | Maximum number of claims to extract from a final message. |
+| `min_confidence` | `0.5` | Claims the extractor scores below this are dropped before verification. |
+| `final_message_chars` | `6000` | How many characters of the final assistant message to feed into extraction. |
+
+## Storage layout
+
+```text
+~/.onlooker/assayer/<project-key>/
+└── audit-<session-id>.json     # advisory summary written at end of each audit
+```
+
+Each audit file records the claim count, the corroborated / contradicted / unverified tallies, the overall verdict, and the per-claim list for review in the next session.
+
+Project key: first 12 hex chars of SHA256 of `git remote get-url origin`, falling back to a hash of the repo root realpath — stable across directory moves, clones, and worktrees of the same repo.
+
+## Events emitted
+
+Assayer emits the canonical `assayer.*` event surface from [`@onlooker-community/schema`](https://github.com/onlooker-community/schema). All events land in `~/.onlooker/logs/onlooker-events.jsonl` and are validated against the schema before write.
+
+| Event | When |
+|-------|------|
+| `assayer.audit.started` | Before verification begins. Includes `claim_count` and `command_count`. |
+| `assayer.claim.contradicted` | A claim is contradicted by a failing command. Includes the `claim`, the `evidence_command`, and a `result_excerpt`. |
+| `assayer.claim.unverified` | A claim has no supporting evidence (`reason`: `no_matching_command` or `ambiguous`). |
+| `assayer.audit.complete` | After all claims are checked. Includes the tallies, the `verdict` (`clean`, `contradictions_found`, `nothing_to_verify`), and `duration_ms`. |
+
+Corroborated claims are counted in the summary rather than emitted individually — the happy path is the quiet path.
+
+## Requirements
+
+- The `ecosystem` plugin installed (for the `~/.onlooker/` substrate and canonical event emission).
+- A release of `@onlooker-community/schema` that includes the `assayer.*` event types (the emitter validates every envelope against the installed schema; older versions reject `assayer.*`).
+- `claude` CLI on `PATH` (the hook shells out to `claude -p` for the extraction pass).
+- `jq` for JSON manipulation.
+- `node` for canonical-event emission.
+- `python3` for millisecond timestamps (standard on macOS and most Linux distributions).
+
+## Architecture decisions
+
+Key decisions made during initial design are recorded in [`docs/adr/`](docs/adr/):
+
+- [ADR-001](docs/adr/001-verify-claims-against-transcript-evidence.md) — Verify claims at Stop against transcript evidence (and why `is_error`, not exit codes, and why advisory)

package/plugins/assayer/config.json

@@ -0,0 +1,14 @@
+{
+  "plugin_name": "assayer",
+  "storage_path": "~/.onlooker",
+  "assayer": {
+    "enabled": false,
+    "evaluation": {
+      "model": "claude-haiku-4-5-20251001",
+      "timeout_seconds": 60
+    },
+    "max_claims": 12,
+    "min_confidence": 0.5,
+    "final_message_chars": 6000
+  }
+}

package/plugins/assayer/docs/adr/001-verify-claims-against-transcript-evidence.md

@@ -0,0 +1,57 @@
+# ADR-001: Verify Claims at Stop Against Transcript Evidence
+
+- Status: Accepted
+- Date: 2026-06-04
+- Deciders: Meagan
+- Tags: assayer, verification, stop-hook, transcript, honesty
+
+## Context and Problem Statement
+
+Every other plugin in the ecosystem assumes the agent's account of its own work is true. Tribunal judges the output, echo scores the prompt, governor counts the spend — but none of them check the most basic thing: when the agent says "I ran the tests and they pass," did the tests actually pass?
+
+This failure mode is not malice. An agent claims success because it intended to verify, or it verified an earlier revision, or it ran the command, saw red, fixed something, and never re-ran. The final message reflects a belief, and the belief can be stale. The session transcript already holds the ground truth — the commands that ran and whether they errored — but nothing reconciles the two.
+
+The question: **how do we check the agent's claims against what actually happened, cheaply and without false alarms?**
+
+## Decision Drivers
+
+- The evidence (commands + results) must already exist and be trustworthy — not reconstructed or re-run.
+- Verification must be deterministic: the same session must always produce the same verdict.
+- False positives are expensive. Flagging a true claim as a lie destroys trust in the plugin faster than missing a false one.
+- It must not interrupt the user's flow for an advisory signal.
+
+## Decision
+
+**Assayer runs at `Stop`, reads the session transcript, and reconciles the agent's final-message claims against the Bash command results recorded in that same transcript.**
+
+Three sub-decisions follow:
+
+### 1. Stop, reading the committed transcript
+
+`Stop` fires once the turn is over and the transcript is fully written to disk — the same `transcript_path` tribunal and compass read. There is no timing-skew risk: every command the agent ran, and every result, is already on disk before assayer looks. Running earlier (e.g. `PostToolUse`) would mean verifying claims that have not been made yet.
+
+### 2. `is_error`, not exit codes
+
+Claude Code's transcript represents a command as a `tool_use` block and its outcome as a `tool_result` carrying an `is_error` boolean. It does **not** expose a per-call numeric exit code. So `is_error` is the success/failure signal: a claim of success contradicted by a matching command whose `is_error` is true. The schema's `assayer.claim.contradicted` payload reflects this honestly — `evidence_command` is required, `exit_code` is optional (populated only when a code is recoverable from output), and a `result_excerpt` captures the failing output for the reader.
+
+### 3. Split the work: LLM identifies, bash verifies
+
+Claim extraction is a language problem — what counts as a testable success claim, and what command would settle it — so an LLM (`claude -p`, Haiku) does it. The factual cross-check is not a language problem; it is a lookup. So a deterministic bash/jq verifier matches each claim to the most recent command containing its keywords and reads `is_error`. The LLM never judges truth; the verifier never interprets language. This keeps the verdict reproducible and unit-testable, and confines the non-determinism to the one step that genuinely needs it.
+
+### 4. Advisory, not blocking
+
+Assayer always exits 0. A contradicted claim is emitted as an event and written to an advisory file, not used to block `Stop`. The turn is already over; the high-value action is a durable, queryable signal ("the agent claimed X; the evidence says otherwise"), not interrupting a finished session. A blocking/enforce mode that re-prompts the agent on contradiction is a plausible future opt-in, but the safe default is advisory.
+
+## Consequences
+
+**Positive**
+
+- Closes the "did it actually work?" gap with zero new infrastructure — the evidence is already in the transcript.
+- Deterministic and testable: the verifier is pure bash/jq with no LLM in the factual path.
+- Off by default and advisory, so it can never block or surprise a session it was not invited to.
+
+**Negative / accepted trade-offs**
+
+- Keyword matching is heuristic. A claim whose verifying command uses unexpected wording falls to `unverified` rather than being checked — a miss, not a false alarm, which is the safer direction to err.
+- `is_error` is coarser than an exit code. A command that exits 0 but prints failures (a test runner that swallows its own status) reads as success. Documented; acceptable for v0.1.
+- One `claude -p` call per Stop. This is why the plugin is off by default.

package/plugins/assayer/docs/design.md

@@ -0,0 +1,72 @@
+# Assayer — Design
+
+Assayer is the verification layer of the Onlooker ecosystem. Where tribunal judges *quality* and echo tracks *prompt drift*, assayer answers a narrower, more literal question: **did the things the agent said it did actually happen?**
+
+## The problem: lying-without-malice
+
+An agent's final message is a self-report. It says "tests pass," "build is green," "lint is clean." These read as facts but are really *beliefs*, and beliefs drift from reality in ordinary ways:
+
+- The agent ran the check against an earlier revision, then changed code, and never re-ran.
+- It intended to run the check and narrated as if it had.
+- It misread a noisy command's output.
+
+None of this is deception. But a user who trusts the self-report ships on a false premise. The session transcript already contains the ground truth — every command and its result — so the gap is purely one of reconciliation.
+
+## Pipeline
+
+```
+Stop
+  → Transcript Reader      (final message + commands-with-status)
+  → Claim Extractor        (claude -p, Haiku — language understanding)
+  → Deterministic Verifier (bash/jq — factual lookup)
+  → Events + advisory file
+```
+
+### Transcript reader (`assayer-transcript.sh`)
+
+Two extractions from the JSONL transcript at `transcript_path`:
+
+- `assayer_final_assistant_message` — the text blocks of the last assistant turn that contains any, truncated to `final_message_chars`. This is where claims live.
+- `assayer_collect_commands` — every `Bash` `tool_use` joined to its `tool_result` by `tool_use_id`, yielding `{ command, is_error, excerpt }`. Claude Code does not record a numeric exit code per call; `is_error` is the success/failure signal. See ADR-001.
+
+### Claim extractor (`assayer-extract.sh`)
+
+A single `claude -p` pass reads only the final message and returns a JSON array of claims, each `{ text, type, command_keyword, confidence }`. `type` is one of `tests_pass | build_succeeds | lint_clean | types_check | command_succeeds | generic`. The model identifies claims and what command would settle them; it does not judge whether they are true. `assayer_parse_claims` strips fences, validates the array, coerces unknown types to `generic`, and drops entries without text.
+
+### Deterministic verifier (`assayer-verify.sh`)
+
+`assayer_classify_claim` derives keywords from the claim's `type` (e.g. `tests_pass → ["test"]`) plus the LLM-supplied `command_keyword`, finds the **most recent** command containing any keyword, and classifies on its `is_error`:
+
+- matched + not errored → **corroborated**
+- matched + errored → **contradicted**
+- no match → **unverified** (`no_matching_command`, or `ambiguous` when the claim implies no checkable command)
+
+"Most recent wins" handles the fail-fix-rerun pattern: the last run reflects the final state. The function is pure — no LLM, no filesystem — so it is fully unit-tested.
+
+`assayer_audit_verdict` rolls the per-claim counts into `clean`, `contradictions_found`, or `nothing_to_verify`.
+
+## Events
+
+| Event | Payload highlights |
+|-------|--------------------|
+| `assayer.audit.started` | `audit_id`, `claim_count`, `command_count`, `trigger` |
+| `assayer.claim.contradicted` | `claim`, `claim_type`, `evidence_command`, `result_excerpt`, `confidence` |
+| `assayer.claim.unverified` | `claim`, `claim_type`, `reason` |
+| `assayer.audit.complete` | `corroborated`, `contradicted`, `unverified`, `verdict`, `duration_ms` |
+
+Corroborated claims are counted in the summary, not emitted individually — the happy path stays quiet.
+
+## Non-goals (v0.1)
+
+- **Blocking.** Assayer is advisory; it never blocks `Stop`. An enforce mode is a future opt-in.
+- **Re-running commands.** Assayer reconciles against what already ran; it does not execute anything.
+- **Parsing exit codes from output.** It relies on `is_error`. A command that exits 0 while printing failures reads as success.
+- **Non-Bash evidence.** Only `Bash` results are treated as evidence today.
+
+## Relationship to other plugins
+
+Assayer occupies the verification/execution layer, empty until now. It is complementary to:
+
+- **tribunal** — judges whether the work is *good*; assayer checks whether the *claims about it* are true.
+- **scribe** — writes the narrative of *why*; assayer checks the factual assertions in that narrative's neighbor, the final message.
+- **counsel** — can consume `assayer.*` events to surface recurring honesty gaps over time.

package/plugins/assayer/hooks/hooks.json

@@ -0,0 +1,15 @@
+{
+  "hooks": {
+    "Stop": [
+      {
+        "matcher": "*",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "\"$CLAUDE_PLUGIN_ROOT\"/scripts/hooks/assayer-stop.sh"
+          }
+        ]
+      }
+    ]
+  }
+}