@event4u/agent-config 9.8.0 → 9.9.0

package/docs/proof.md

@@ -1,252 +0,0 @@
-<!-- GENERATED by src/scripts/build_proof.ts — do NOT hand-edit.
-     Drift-checked in CI (`task build-proof-check`). Regenerate with
-     `./scripts-run src/scripts/build_proof` after editing docs/CLAIMS.md. -->
-
-# Proof — verify our claims yourself
-
-We sell falsifiability, so the selling is machine-checked. Every
-public claim binds to resolvable evidence; a skeptic can reproduce
-every check below on a fresh checkout. This page is itself generated
-from those sources and fails CI if it drifts.
-
-## What this prevents
-
-Each row is a failure mode, not a feature — and each cites the ledger
-claim that backs it. A row whose claim stopped resolving would fail this
-page's own generator, so the table cannot outlive its evidence.
-
-| Failure this prevents | How | Backed by |
-|---|---|---|
-| A rule says MUST, and nothing anywhere enforces it — so the rule reads as a guarantee while being honour-system. | Each rule declares `enforced_by:`, and the check **resolves** it: a validator reachable from no taskfile, workflow, or hook manifest counts as unwired, not as covered. Undeclared counts as uncovered. | [`enforcement-coverage-resolved`](CLAIMS.md) ⟳ |
-| A shipped artefact carries a hidden-Unicode or instruction-smuggling payload, and reaches consumers because only the source tree was scanned. | Source **and** the condensed projection are scanned in CI; a finding blocks the release before publish, not merely the merge. | [`shipped-artifacts-hidden-instruction-scanned`](CLAIMS.md) ⟳ |
-| A count in public prose drifts from the source, and the marketing number quietly stops being true. | Counts are generated from source and drift-checked; the build fails on a count-shaped prose mention that disagrees, or on two different numbers for the same artefact kind. | [`skill-count`](CLAIMS.md) ⟳ |
-| A host-coverage claim understates or overstates what actually works — for months, on the one surface everybody reads. | The number is pinned by a test over real detection, and the test is re-run as the claim evidence. | [`host-agent-count`](CLAIMS.md) ⟳ |
-| A non-coding domain skill implies proven correctness it never had, because it was forged on a different domain and never checked against one. | Domain skills are labelled unvalidated until they pass a sourced domain-truth fixture, and the validated count is ratcheted so it cannot quietly fall. | [`domain-soundness-scoped`](CLAIMS.md) ⟳ |
-| Behavioural-eval coverage regresses as skills are added, and the suite looks healthy because only the absolute number is reported. | Coverage is measured per tier and ratcheted in CI, so it can only rise; the gap is published rather than implied away. | [`eval-coverage-ratcheted`](CLAIMS.md) ⟳ |
-
-⟳ — the claim carries `exec:` evidence: CI re-runs the command and
-compares its exit code, so a stale row turns the build red rather than
-ageing quietly into marketing.
-
-## See it run (< 60s, real output)
-
-![The trust surface running green — every "verify it yourself" command below, recorded from a real run](media/proof-demo.gif)
-
-The recording is of the exact commands in § 5, nothing staged. Its
-commands are re-executed in CI (`.github/workflows/proof-demo.yml`), so a
-demo that showed something broken would turn CI red — the recording
-cannot drift from current behavior.
-
-## 1. Every public claim binds to evidence
-
-Rendered from [`docs/CLAIMS.md`](CLAIMS.md). A `<!-- claim:ID -->` marker in
-README/docs must resolve to a `backed` entry here with a resolving
-evidence pointer, or `task check-claims` fails the build.
-
-| Claim | Kind | Evidence | Resolves |
-|---|---|---|---|
-| The release process is documented as an inheritable runbook + succession doc, and the project's bus-factor (trailing-90-day distinct human reviewers) is tracked and reported truthfully — currently 1, not implied to be more. | qual | `docs/succession.md#trailing 90 days` | ✅ |
-| 190 commands. | quant | `exec:check_artefact_count_messaging -> 0` | ✅ |
-| A MEASURED-BUT-NOT-SHIPPED experiment — the thin rule projection reduced eager rule load 78,513 → 13,881 GPT-tokens (whole always-loaded projection 98,529 → 33,897, ~65.6%), but FAILED the quality gate (thin win-rate 36.2% vs required 48%) and does not ship; it un-defers only behind `discipline_profile: essential`. Shipped behavior does NOT include this reduction. Method: `agent-config benchmark` over the pinned token baseline; the baseline is the honest "what the user pays if everything loads eagerly", NOT a synthetic full-corpus strawman (council Q4); quality gate per the Phase-0 paired judge run. | quant | `internal/bench/reports/token-baseline.json#eager_rule_load` | ✅ |
-| The first cross-vendor parity pass (5 orchestration-corpus tasks × 2 vendors × 3 repeats, identical prompts via the council transport, $0.16) measured real per-host finding-count differences — claude-sonnet-4-5 surfaced ~2× the findings of gpt-4o on the multi-file analysis task (median 11 vs 5) while both vendors were identical on the planted hollow-implementation task (2 vs 2) and perfectly silent on the clean-code negative control (0 vs 0, no spurious findings). The per-task `finding_floor` values are calibrated from the cross-host lower envelope and the gate is armed. | quant | `internal/bench/reports/parity-count.json#min over hosts of median` | ✅ |
-| On a weak host (claude-haiku-4-5) the package produces a significant, placebo-controlled discipline lift on scope/downstream traps; on a strong host the same measurement is a published null — the package transplants discipline a weak model lacks, not model intelligence. | quant | `docs/benchmark.md#weak-host-specific` | ✅ |
-| The non-coding domain skills (finance/founder/ops/content) are forged on TS/PHP and labeled unvalidated until they pass a sourced domain-truth fixture; no public prose implies proven domain correctness, and the validated count is CI-ratcheted. | qual | `exec:domain_soundness_status --check -> 0` | ✅ |
-| The validated non-coding domain-skill count is pinned and CI-ratcheted at a maintainer-set floor (9 of 20 default-surface skills carry a sourced `evals/domain-truth.json` fixture at pin time, 2026-07-11 — 5 deterministic, keys from cited formulas; 4 rubric, criteria matching a named external practice); the floor only rises via a maintainer `--write-floor` after a new sourced fixture lands. | quant | `exec:domain_soundness_status -> 0` | ✅ |
-| On the READ-ONLY FAN-OUT slice family, tier-downshifted subagent dispatch (lite/haiku vs session-tier-proxy sonnet) nets a ≥30% USD-weighted token-cost reduction at held quality — measured 2026-07-08 (n=10 paired live dispatches, 20 telemetry lines): 10/10 exact-match on BOTH arms, 29.4% fewer raw tokens, 76.5% USD-weighted cost reduction at the 3x haiku↔sonnet price ratio. FAMILY-SCOPED — the mechanical-edit family is unmeasured and its downshift (incl. the deferred tier downgrades of existing units) stays gated. Negative control held: an open-ended synthesis/unknown slice never resolves below the session tier (inferSliceTier → medium/inherit, never lite). | quant | `internal/bench/routing-downshift/results-2026-07-08.md#FAMILY-SCOPED PROVE` | ✅ |
-| 14 of 107 governed rules (13.1%) carry a backstop that fails a CI build. The number is RESOLVED, not declared — a `validator:` counts only when the script exists AND a GITHUB WORKFLOW reaches it (transitively, so a sub-check under a wired umbrella counts), and a hook registered `fail_closed: false` resolves to `observer`, never `validator`. The figure was 14 before this correction too, and it was wrong: the resolver treated `taskfiles/` and `.github/workflows/` as one corpus, so "named in a taskfile" counted as blocking — while NO workflow invokes `task ci`, `ci-strict`, or `ci-fast`. Nine of the thirteen validators only ran when a human typed the command. Split into `validator` (CI runs it) and `validator-local` (only a taskfile does), the honest figure was 5 of 107; wiring the nine into `rule-backstops.yml` returned it to 14, this time meaning what the headline says. `local_only` is now 0 and is ratcheted, so a gate cannot drop back out silently. Wiring them also surfaced that FIVE were already failing invisibly — 37 findings, baselined in `rule-backstop-debt.json` and ratcheted against growth. 86 rules declare nothing and count as uncovered, not excluded. | quant | `exec:check_enforcement_coverage --check -> 0` | ✅ |
-| The lift-carrying essential cut (kernel + downstream-changes) keeps a significant weak-host discipline lift at a fraction of the full load's tokens, and the lift is FAMILY- and HOST-SCOPED — measured on three hosts: claude-haiku-4-5 (weak) shows the family-scoped lift (trapE 0.533→1.000, 7/7 discordant, corpus cost 1.71x); claude-sonnet-4-6 (strong) is a ceiling null; gpt-5-mini (non-Claude weak, codex prompt-prepend surface) FAILED replication with headroom (corpus Δ=+0.024 p=0.70, capability trend n.s. — no harm claimed, injection-surface confound documented). Therefore discipline_profile: auto enables the lift only where measured (vendor-granular unknown_defaults). Non-claims — the balanced router profile was removed after a NULL measurement (p=0.81, n=24); no full-tier recommendation exists; no cross-vendor lift is claimed. | quant | `docs/benchmark.md#REPLICATION FAILED` | ✅ |
-| Behavioural-eval coverage is measured per tier and CI-ratcheted so it can only rise; the current coverage and its gap are published, never implied as "264 evaluated skills". | qual | `exec:skill_eval_coverage --check -> 0` | ✅ |
-| A credential-free prescription layer reads content the host's own web tools cannot fetch at all — Reddit thread text (Atom feeds) AND comment ranking plus reply nesting (server-rendered HTML), and a single named tweet (the platform's own oEmbed endpoint). Measured 2026-07-25 from a residential network against a pre-registered 6-task-per-channel set with a native control and thresholds frozen before the run: reddit tier 1 6/6, reddit tier 2 6/6, twitter-oembed 6/6, native 0/6 on both Reddit tiers. Zero credentials, zero resident processes, zero auto-installs. Scope bounds that travel WITH the claim: (a) the twitter gap is narrower than 6/6 suggests — native also passed 2 of those 6 via third-party mirrors, so the channel earns its place only on tweets nothing mirrors; (b) reddit tier 2 is on an announced closing path and ships with a kill-switch keyed on an OBSERVED login wall; (c) youtube-transcripts is PARKED, not shipped — its backend is human-installed by contract and was never exercised; (d) residential network is load-bearing, and CI is explicitly not a bench environment. | quant | `docs/benchmark.md#ship-gated-reach` | ✅ |
-| 23 host agents are detected and inventoried; 20 receive a written config surface (18 projection + 1 plugin + 1 bundle target) and 3 are export-only (aider, zed, jetbrains). The count is enforced, not asserted — `knownToolIds()` is pinned at 23 by a test whose assertion literal IS the number, and `src/config/surface-matrix.yml` is held in set-equality with the installer's own user-scope path map by `lint_surface_matrix`, so a host added to one and not the other fails the build. This entry stood `unbacked` while naming its own unblocking condition ("once `surface-matrix.yml` exists, bind the count to that file and flip"); the condition was met and nothing fired, so the shipped figure stayed "7+" — understating real coverage by 3x. | quant | `exec:vitest run tests/install/toolDetection.test.ts -> 0` | ✅ |
-| On the fixture corpus (n = 20 before/after pairs, 16 length-controlled within ±25%), the humanizer pass removes every mechanically detected AI-writing tell (mean hard hits 0.9 → 0, cluster score 53.97 → 0 per 500 words, dash density 9.22 → 0), and a blind judge (claude-sonnet-4-5, deterministic per-pair A/B seed) preferred the humanized text in 16/16 length-controlled pairs. Scope note — the "before" fixtures were deliberately tell-seeded, so this measures seeded-tell removal on a self-constructed corpus, NOT real-draft improvement; real-world lift is unmeasured until step 4b has processed real ghostwriter drafts (see the road-to-humanizer-hardening live-usage blocker). | quant | `internal/bench/reports/humanizer-v1.md#prefers the humanized text` | ✅ |
-| NONE of the backed ledger claims are machine-re-verifiable today — every evidence pointer is checked for existence (file present, substring present, URL carries a date), never for truth, so a claim pointing at a stale artefact stays backed indefinitely. A measured minority COULD carry a re-executing `exec:` form, clearing the >= 10 pp threshold that was pre-registered before the count was taken, which is why that form is scheduled rather than assumed. The rest cannot: paid or stochastic benchmark runs no CI job can re-derive, and prose contracts. Exact counts live in the evidence file and are NOT restated here on purpose — this entry hard-coded its denominator twice and drifted within a day both times (25 when the ledger held 26, then 26 when it held 27) while CI stayed green, because the pointer resolved. `check_claims` now compares the stored denominator against the live ledger and fails on divergence. A number a human retypes on every ledger edit will drift; the fix was to stop retyping it. | quant | `internal/reports/exec-evidence-feasibility.json#"exec_feasible"` | ✅ |
-| A hand-rolled, dependency-free BM25 + trigram lexical index resolves the "recalls but does not rank" gap: on the retrieval-precision corpus (9 keyword-overlapping-confuser tasks) it drives the mean top tie-set from 3.333 (the `_score` bucket scorer) to 1.0 — every needed decision uniquely top-ranked — with precision@1 and precision@5 unchanged at 1.0. Method: deterministic, model-free re-ranking of the SAME retrieved entry set; both scorers measured over the identical store via `measure_lexical_ranking.ts`. Cross-artefact note (2026-07-25): this baseline (3.333) is the value in THIS claim's own artefact and is cited correctly, but the retrieval-precision artefact records 4.111 for the same scorer on the same corpus — two bench scripts disagree. The lift direction (ties collapse to 1.0) holds under either baseline; the discrepancy itself is unresolved and recorded rather than smoothed. | quant | `exec:measure_lexical_ranking -> 0` | ✅ |
-| The whole layer is compiled into host agents with zero runtime daemon. | qual | `docs/contracts/no-runtime-boundary.md#file-first, no-runtime suite` | ✅ |
-| On a 12-fixture option-decision corpus (3 arms × 2 providers, blind rubric judge claude-opus-4-8, pre-registered hypotheses), famous-figure identity framing added nothing beyond the underlying method text (method 5.04 vs figure 4.88, Δ=0.17, sign-test p=0.607), and provider diversity moved judged quality ~15× more than persona identity (provider Δ=2.58 vs identity Δ=0.17); the whole persona layer lifted only +0.08 over bare prompts. Honest null — persona panel-mode stays CUT, evidence-closed. | quant | `internal/bench/reports/persona-placebo.json#honest-null` | ✅ |
-| The only agent layer that publishes the runs where it changed nothing. Deliberately falsifiable — if a reader finds a comparable agent layer publishing its own honest-null benchmark runs, this line updates; that is the point. | comparative | `docs/benchmark.md#honest` | ✅ |
-| On the live end-to-end retrieval benchmark (9 tasks × 3 arms × 3 seeds on claude-haiku, fixture store with keyword-overlapping confusers), the memory retrieval substrate scored precision@5 = 100% (9/9) with 100% poisoned-entry rejection, and the retrieval-on arm passed 27/27 model-scored tasks vs 2/27 with retrieval off and 4/27 with a placebo injection. Known limit stays published: mean tie-set 4.111 means top-k ties break by store order, not relevance (the ADR-116/FTS5 signal). SOLE RECORD for this artefact as of 2026-07-25: a second entry (`second-brain-retrieval-precision`) described the same measurement from the precision angle and had drifted to 5/27, 5/27 and tie-set 3.3 — figures absent from the shared artefact. Two entries over one artefact is what allowed them to disagree while both resolved, so the pair was folded into this one. | quant | `internal/bench/reports/second-brain-retrieval.json#retrieval-on` | ✅ |
-| 107 governed rules. | quant | `exec:check_artefact_count_messaging -> 0` | ✅ |
-| On a deterministic multi-session recall corpus, the memory substrate produces a measured, placebo-controlled recall lift — memory-on 27/27 vs no-memory 10/27 and vs equal-byte placebo 9/27 (claude-haiku-4-5, n=9 tasks x 3 seeds, sign test p=0.031 for BOTH pairings). Scoped honestly: this is the context-value upper bound (perfect retrieval on a one-fact-per-task corpus), not retrieval precision under a large store. | quant | `internal/bench/reports/second-brain-delta.json` | ✅ |
-| Every artifact the package ships — source AND the condensed projection that reaches consumers — is machine-scanned in CI for hidden-Unicode, mixed-script-confusable, and instruction-smuggling payloads (the rules-file-backdoor class); a finding blocks the release before `npm publish`, not just the merge. | qual | `exec:lint_agent_security -> 0` | ✅ |
-| 281 skills. | quant | `exec:check_artefact_count_messaging -> 0` | ✅ |
-| Removes only its own keys from a shared host config (matched by JSON-pointer + SHA-256), never a neighbour tool's entries. | qual | `docs/contracts/install-layout.md#JSON-pointer` | ✅ |
-| On the pre-registered 12-fixture defect-finding corpus (three arms, deterministic file-level recall, codex reviewer gpt-5.5), the cross-model team-review arm produced NO recall lift over single-model self-review — all three arms recalled every planted defect (Δ = 0, H1 not met). Honest null, ceiling-limited (recall 1.00 everywhere: the seeded defects are too obvious to discriminate the arms on recall); the only non-null signal is a single self-review false positive on the controversial-but-correct control vs 0 for team/council. No cross-model quality/lift claim binds; team mode stays workflow-value-only. Re-open: a judge-survivable-subtlety corpus or a new model generation. | quant | `internal/bench/reports/defect-finding.json#honest_null` | ✅ |
-| On the A3 orchestration eval (deterministic verify, measured token deltas), the production-validator subagent returned the correct verdict on both fixtures — NOT READY with the exact `file:line` citation on a planted hollow implementation, READY with zero spurious findings on the clean control — while consuming ~45k fewer tokens than the inline-host baseline on each task. Scope: two planted fixtures on a Claude Code host, not a broad hit-rate. | quant | `internal/bench/orchestration/pv-a3-results.md#token_delta_provenance: measured` | ✅ |
-
-**27 backed claim(s)** — all evidence pointers resolve in CI.
-
-### How many of those re-derive themselves
-
-**10 of 27** backed claims carry `exec:` evidence —
-CI re-runs the command and compares its exit code to the claim, so a
-stale one turns the build red. The other
-**17** rest on a pointer: CI checks that the artefact
-exists and contains what it should, which cannot distinguish a live
-claim from one whose producer nobody has run in months.
-
-That residue is not an oversight, and it is listed rather than rounded
-away. A claim can only re-derive itself when a deterministic command
-carries the verdict in its exit code. These cannot, for the reasons
-given:
-
-| Claim | Why it cannot re-execute |
-|---|---|
-| The release process is documented as an inheritable runbook + succession doc, and the project's bus-factor (tr | prose or contract artefact — no exit code carries the verdict |
-| A MEASURED-BUT-NOT-SHIPPED experiment — the thin rule projection reduced eager rule load 78,513 → 13,881 GPT-t | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| The first cross-vendor parity pass (5 orchestration-corpus tasks × 2 vendors × 3 repeats, identical prompts vi | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| On a weak host (claude-haiku-4-5) the package produces a significant, placebo-controlled discipline lift on sc | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| On the READ-ONLY FAN-OUT slice family, tier-downshifted subagent dispatch (lite/haiku vs session-tier-proxy so | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| The lift-carrying essential cut (kernel + downstream-changes) keeps a significant weak-host discipline lift at | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| A credential-free prescription layer reads content the host's own web tools cannot fetch at all — Reddit threa | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| On the fixture corpus (n = 20 before/after pairs, 16 length-controlled within ±25%), the humanizer pass remove | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| NONE of the backed ledger claims are machine-re-verifiable today — every evidence pointer is checked for exist | prose or contract artefact — no exit code carries the verdict |
-| The whole layer is compiled into host agents with zero runtime daemon. | prose or contract artefact — no exit code carries the verdict |
-| On a 12-fixture option-decision corpus (3 arms × 2 providers, blind rubric judge claude-opus-4-8, pre-register | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| The only agent layer that publishes the runs where it changed nothing. Deliberately falsifiable — if a reader  | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| On the live end-to-end retrieval benchmark (9 tasks × 3 arms × 3 seeds on claude-haiku, fixture store with key | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| On a deterministic multi-session recall corpus, the memory substrate produces a measured, placebo-controlled r | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| Removes only its own keys from a shared host config (matched by JSON-pointer + SHA-256), never a neighbour too | prose or contract artefact — no exit code carries the verdict |
-| On the pre-registered 12-fixture defect-finding corpus (three arms, deterministic file-level recall, codex rev | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-| On the A3 orchestration eval (deterministic verify, measured token deltas), the production-validator subagent  | benchmark output — regenerating it needs paid or stochastic model calls no CI job can re-derive |
-
-Artefact counts in public prose (skills, commands, governed rules,
-guidelines, personas) are **generated from source and CI-drift-checked**:
-`update_counts.ts` writes the numbers, `check_artefact_count_messaging.ts`
-fails the build on any count-shaped prose mention that drifts from the
-source count — or on two different numbers for the same artefact kind.
-
-We also publish our **debt**: 4 claim(s) are logged as
-`unbacked` inventory in the ledger — not yet bound, and therefore not
-allowed to carry a marker in public prose. Hiding them would be the
-opposite of the point.
-
-## 2. We publish honest nulls
-
-Benchmark results — **including the runs where the package changed nothing** —
-live in [`docs/benchmark.md`](benchmark.md). We do not delete a measured null to
-make a number look better; the null is the evidence of honesty.
-
-The freshest example: the **persona-placebo benchmark** <!-- claim:persona-identity-placebo-null -->
-(2026-07-12, 3 arms × 2 providers, blind rubric judge, pre-registered
-hypotheses) measured whether famous-figure persona framing improves decision
-answers over the bare method text it wraps. It does not (Δ=0.17, p=0.607) —
-while provider diversity moved judged quality ~15× more than persona identity.
-The measured null closed a planned feature (persona panel-mode) instead of
-shipping theater; the full per-cell data is committed at
-`internal/bench/reports/persona-placebo.json`.
-
-**Behavioural-eval coverage — the honest baseline.** Skill *quality* is only
-as good as its measurement. Today **42 of 281** skills carry a behavioural
-`evals.json`; the highest-traffic / highest-cost tiers (default-surface +
-`rich` + routers) are **fully covered (35 of 35)**, the long tail
-(7 of 246) is not. We publish that gap rather than imply
-"281 evaluated skills": coverage is measured per tier
-(`./scripts-run src/scripts/skill_eval_coverage`), **CI-ratcheted so it can
-only rise**, and the priority tiers carry a hard **tier floor**: every
-rich / default-surface / router skill MUST have a behavioural eval or an
-explicit exemption-with-reason in `internal/evals/tier-floor-exemptions.json`
-— no silent exemptions (the bright line deliberately replaces a
-weighted-coverage score). Authoring long-tail evals is gated on per-case
-human ratification (a generated assertion that checks the wrong property is
-worse than none), so the number grows deliberately, not overnight.
-
-**Non-coding domain soundness — scoped, not proven.** The `finance` /
-`founder` / `ops` / `content` profiles sell concrete domain value (DCF,
-runway, RICE, incident command, messaging), but the skills are forged on
-TS/PHP — "promising, not proven" off those stacks. A disclaimer floor
-bounds *liability*, not *correctness*: a skill can be format-correct,
-disclaimered, and still embed a wrong domain assumption. Today **9 of 20**
-default-surface domain skills carry a sourced `domain-truth` fixture
-(`./scripts-run src/scripts/domain_soundness_status`); the rest are labeled
-`unvalidated` and the validated count is CI-ratcheted. The fixtures landed
-so far are the **deterministic** targets (runway, unit-economics, DCF,
-forecasting, scenario band/sensitivity), whose answer keys are computed
-from cited standard formulas — never the skill's own output; the
-**rubric** targets (incident command, messaging, fundraising, editorial)
-need domain-competent grounding and remain unvalidated, so validation
-lands deliberately — the gap is published, never implied away.
-
-**Second-brain substrate — measured recall lift, honestly bounded.** On a
-deterministic multi-session recall corpus, the memory substrate beats a
-no-memory baseline AND an equal-byte placebo: memory-on **27/27** vs
-no-memory **10/27** vs placebo **9/27** (claude-haiku-4-5, 9 tasks × 3
-seeds, sign test **p = 0.031** for both pairings) — a real,
-placebo-controlled lift (`internal/bench/reports/second-brain-delta.json`).
-Scoped, not oversold: this is the context-value upper bound (perfect
-retrieval on a one-fact-per-task corpus), NOT retrieval precision under a
-large store; the lift concentrates exactly where memory is the only source
-and ties where the prompt self-contains the fact. Boundary vs a human PKM,
-and why the Obsidian export stays declined, are in
-[`docs/second-brain-scope.md`](second-brain-scope.md).
-
-A follow-up removed the perfect-retrieval assumption: against a store of
-keyword-overlapping confusers the REAL retrieval recalls the needed
-decision into the top-5 (9/9) and the model disambiguates it — retrieval-on
-**27/27** vs no-memory **5/27** and vs placebo **5/27** (p=0.008 both). The
-honest limit: the keyword scorer recalls but does not rank (mean tie-set
-3.3), which is the SQLite-FTS5 activation signal (ADR-116) at scale.
-
-## 3. Known limits (published, witness-tested)
-
-Each limitation below is stated by the skill itself and carries a witness
-test that reproduces it. If the limitation is ever fixed, its witness goes
-red — so a "Known limit" can never quietly become false.
-
-| skill | known limit | witness |
-|---|---|---|
-| `check-refs` | Only validates references to known-root paths (docs/, skills/, rules/, commands/, contexts/, personas/, …). A relative-path link such as `./sibling.md` or `../foo.md` is not matched, so a broken relative link is never reported. | [`tests/scripts/witness/check_refs_relative_gap.test.ts`](../tests/scripts/witness/check_refs_relative_gap.test.ts) |
-
-## 4. What is checkable — us vs. the category
-
-This is not a takedown — the point is the last column. For each claim,
-our evidence is a pointer you can resolve on a fresh checkout; the wider
-category is described only by what is publicly observable, never a named
-competitor and never a counter-claim to anyone's headline number. A claim
-is "checkable" only when its `our evidence` pointer resolves — CI enforces
-that (`task check-comparison`), so this column can never lie.
-
-| Claim | Our evidence | The category | Checkable? |
-|---|---|---|---|
-| No resident runtime — no background daemon, no state database or service, no auto-write memory; deterministic, rebuildable file indexes only. | [`docs/contracts/no-runtime-boundary.md#file-first`](../docs/contracts/no-runtime-boundary.md) | Swarm-runtime tools in this category ship a background process and/or a resident state store by design; that is an architectural fact of the runtime approach, not a defect. | ✅ |
-| Capability/benchmark results are published — including the runs where the package changed nothing. | [`docs/benchmark.md`](../docs/benchmark.md) | A category headline figure (e.g. an '84.8%' score) appears across marketing surfaces with no reproducible methodology published — so it cannot be verified either way. | ✅ |
-| Every public claim binds to machine-checked evidence. | [`docs/CLAIMS.md`](../docs/CLAIMS.md) | Marketing claims in this category are not bound to a machine-checked ledger a reader can reproduce. | ✅ |
-| Skills publish their own known limits, each with a witness test. | [`tests/scripts/witness/check_refs_relative_gap.test.ts`](../tests/scripts/witness/check_refs_relative_gap.test.ts) | Published known-limitation surfaces backed by reproducing tests are not a standard artifact in this category. | ✅ |
-| A 30-second single-file wedge exists — one read-only subagent, one command, verdict with file:line — and its promise is a scoped, published eval, not a feature list. | [`internal/bench/orchestration/pv-a3-results.md#token_delta_provenance: measured`](../internal/bench/orchestration/pv-a3-results.md) | Category entry points typically install the full platform (profiles, packs, runtime) before the first felt win; single-artifact entries with a published eval behind the promise are not a standard artifact. | ✅ |
-| The persona-identity question was tested and published as an honest null (identity swap ~zero effect, provider choice ~15x larger) instead of being shipped as theater. | [`internal/bench/reports/persona-placebo.json#honest-null`](../internal/bench/reports/persona-placebo.json) | Named-persona prompts are shipped as a feature across the category without a published A/B separating identity from method. | ✅ |
-| Every upstream-tool install prescription the package ships is version-pinned, intake-recorded, and enforced offline in CI — an unpinned version, a branch/archive install source, or a pipe-remote-to-shell instruction fails the build. | [`src/scripts/validate_reach_prescriptions.ts`](../src/scripts/validate_reach_prescriptions.ts) | Capability layers in this category commonly install themselves and their upstream dependencies from a moving target (a branch archive or an unpinned package) via a piped remote installer; whether any given one pins is observable per repo, but a CI gate that fails the build on an unpinned prescription is not a standard artifact. | ✅ |
-| Platform reads that the host's own web tools cannot perform at all (a refused domain, an HTTP 402) are shipped as credential-free prescriptions whose reliability was measured per channel against a native control, with the thresholds frozen before the run and the narrowing caveat published alongside the pass. | [`docs/benchmark.md#ship-gated-reach`](../docs/benchmark.md) | Reaching gated platforms is commonly solved in this category by asking for an API key or a session cookie, and the capability is asserted from a feature list rather than a per-channel reliability measurement with a native-arm control; publishing the case where the native tools already suffice, next to the pass, is not a standard artifact. | ✅ |
-
-### What each one prevents
-
-The same rows, read as failure modes rather than as comparisons. Each names
-something that goes wrong without the control and carries the identical
-resolvable pointer — a projection of the table above, not a second list to
-keep in sync.
-
-| Without it | The control | Evidence |
-|---|---|---|
-| A background process keeps running after you stop using it, holds state you did not write, and has to be kept alive and upgraded. | No resident runtime — no background daemon, no state database or service, no auto-write memory; deterministic, rebuildable file indexes only. | [`docs/contracts/no-runtime-boundary.md#file-first`](../docs/contracts/no-runtime-boundary.md) |
-| You adopt on a headline number nobody can re-run, and never learn which of its measurements changed nothing. | Capability/benchmark results are published — including the runs where the package changed nothing. | [`docs/benchmark.md`](../docs/benchmark.md) |
-| A shipped number drifts from the artefact it came from and stays published, because nothing re-checks the claim against its source. | Every public claim binds to machine-checked evidence. | [`docs/CLAIMS.md`](../docs/CLAIMS.md) |
-| A limitation is discovered by you, in your repo, at the moment it costs you something — instead of being stated up front and held by a test. | Skills publish their own known limits, each with a witness test. | [`tests/scripts/witness/check_refs_relative_gap.test.ts`](../tests/scripts/witness/check_refs_relative_gap.test.ts) |
-| Evaluating the tool means installing the whole platform first, so the first thing you learn is the setup cost rather than whether it works. | A 30-second single-file wedge exists — one read-only subagent, one command, verdict with file:line — and its promise is a scoped, published eval, not a feature list. | [`internal/bench/orchestration/pv-a3-results.md#token_delta_provenance: measured`](../internal/bench/orchestration/pv-a3-results.md) |
-| You pay tokens for prompt theater that was never measured against the method text it wraps. | The persona-identity question was tested and published as an honest null (identity swap ~zero effect, provider choice ~15x larger) instead of being shipped as theater. | [`internal/bench/reports/persona-placebo.json#honest-null`](../internal/bench/reports/persona-placebo.json) |
-| An install instruction silently follows a moving target — an unpinned package or a piped remote script — so what lands in your repo is whatever the source held that day. | Every upstream-tool install prescription the package ships is version-pinned, intake-recorded, and enforced offline in CI — an unpinned version, a branch/archive install source, or a pipe-remote-to-shell instruction fails the build. | [`src/scripts/validate_reach_prescriptions.ts`](../src/scripts/validate_reach_prescriptions.ts) |
-| You hand over a credential — or your account — for a read the tools could already do, and you cannot tell which channels actually work because the capability was claimed rather than measured. | Platform reads that the host's own web tools cannot perform at all (a refused domain, an HTTP 402) are shipped as credential-free prescriptions whose reliability was measured per channel against a native control, with the thresholds frozen before the run and the narrowing caveat published alongside the pass. | [`docs/benchmark.md#ship-gated-reach`](../docs/benchmark.md) |
-
-## 5. Verify it yourself
-
-On a fresh checkout, reproduce the claims above:
-
-```bash
-task check-claims   # every markered public claim binds to resolvable evidence
-task check-refs     # no broken internal references
-task check-skill-gaps    # every logged known-limit cites a real witness test
-task check-comparison    # every comparison-table "our evidence" pointer resolves
-./scripts-run src/scripts/skill_eval_coverage         # behavioural-eval coverage, per tier
-./scripts-run src/scripts/skill_eval_coverage --check # the ratchet: coverage may not drop
-task build-proof-check   # this page is in sync with its sources
-```
-
-If a claim ever loses its binding, or this page drifts from the ledger,
-CI goes red. Reproducibility is the proof.

package/docs/quality.md

@@ -1,98 +0,0 @@
-# Quality & CI
-
-## CI Pipeline
-
-Run all checks before pushing:
-
-```bash
-task ci
-```
-
-This runs, in order:
-
-1. **Sync check** — `dist/agent-src/` matches `src/` (non-`.md` files)
-2. **Condensation hashes** — Condensed `.md` hashes match source
-3. **Reference check** — No broken cross-references between files
-4. **Portability check** — No project-specific paths in shared files
-5. **Schema validation** — Frontmatter of every skill/rule/command/persona matches its JSON-Schema contract
-6. **Skill linter** — All skills, rules, commands pass structural validation
-7. **README linter** — README passes quality checks
-8. **Tests** — All bash + Python tests pass
-9. **Consistency** — No uncommitted changes from generated outputs
-
----
-
-## Skill Linter
-
-The linter (`scripts/skill_linter.py`) validates:
-
-- **Required structure** — YAML frontmatter, description, triggers
-- **Frontmatter schema** — Each artefact type has a JSON-Schema in `scripts/schemas/`; violations surface as `schema_<rule>` errors (see [frontmatter contract](../agents/reference/docs/frontmatter-contract.md))
-- **Anti-patterns** — Procedural rules in behavior rules, overlong skills, scope creep
-- **Condensation quality** — Key sections preserved after condensation
-
-Schema validation also runs standalone via `task validate-schema` — fast
-fail before the full linter.
-
-### Quality levels
-
-| Level | Meaning |
-|---|---|
-| **PASS** | Meets all requirements |
-| **WARN** | Advisory — may need attention but not blocking |
-| **FAIL** | Must be fixed before merge |
-
-### CI integration
-
-- GitHub Actions runs the linter on every PR
-- Results are posted as PR comments with quality counts
-- **0 FAIL required** — no merge with failures
-
----
-
-## Condensation System
-
-Content flows from verbose (`src/`) to condensed (`dist/agent-src/`),
-which is then projected into `.augment/` for Augment Code.
-
-### Rules
-
-- Source of truth is **always** `src/`
-- Never edit `dist/agent-src/` or `.augment/` directly
-- The `/condense` command produces token-efficient output
-- Condensation hashes track which files have been condensed
-
-### Verification
-
-```bash
-task sync-check          # Non-.md files in sync?
-task sync-check-hashes   # .md hashes current?
-task check-condensation   # Condensation quality OK?
-task lint-skills-pairs   # Source vs condensed comparison
-```
-
-### Fixing
-
-```bash
-task sync                # Copy non-.md files
-task sync-mark-all-done  # Mark all hashes as current (after manual condense)
-task consistency-fix     # Regenerate ALL derived outputs
-```
-
----
-
-## Runtime artifacts
-
-Skills dispatched via `scripts/runtime_dispatcher.py run --skill NAME --output FILE`
-write a typed `ExecutionResult` JSON (exit code, stdout, stderr, duration,
-artifacts) to the given path. In CI, the two pilot skills `lint-skills`
-and `check-refs` write to `agents/runtime/reports/runs/` and
-`scripts/ci_summary.py` renders them into the GitHub Step Summary so
-failures are visible on the PR page.
-
-No data is auto-injected into agent context and no persistent metrics,
-feedback, or audit logs are collected.
-
----
-
-← [Back to README](../README.md)

package/docs/readme-split-plan.md

@@ -1,102 +0,0 @@
-# README three-audience split — plan
-
-Annotated outline for `P2.2a` in
-[`road-to-proof-not-features.md`](../agents/roadmaps/road-to-proof-not-features.md).
-Decides the **information architecture**, not the prose. No content
-rewrite happens in this step; `P2.2b` applies the mapping.
-
-## Target headings (top of README, in order)
-
-1. **Use it in your project** — anchor `#use-it`
-2. **Prove it** — anchor `#prove-it`
-3. **Contribute** — anchor `#contribute`
-
-Each branch opens with one paragraph + one primary CTA. AI Council is
-not mentioned in any branch (verified by `P3.4`).
-
-### Anchor-stability promise
-
-`P2.2b` must keep these existing anchors intact so external inbound
-links survive:
-
-| Anchor today | Lives under (new) | Why |
-|---|---|---|
-| `#quickstart` | `#use-it` | npm/composer search results, social links |
-| `#supported-tools` | `#use-it` | most-cited section on the web |
-| `#what-your-agent-is-asked-to-do` | `#prove-it` | linked from blog posts |
-| `#documentation` | `#use-it` | docs portal entry |
-| `#development` | `#contribute` | contributor guides |
-
-Other section anchors may be renamed; `lint-readme` checks the table
-above and the three new audience anchors only.
-
-## Block-by-block mapping
-
-Every existing top-of-README block, in source order, mapped to
-exactly one branch. "Drop" = block is retired; "Move" = relocated as-
-is; "Reframe" = block stays but its lead-in / CTA changes (still no
-copy rewrite in this step — the reframe direction is decided here,
-applied in `P2.2b`).
-
-| # | Block (current heading) | Lines | Branch | Action | Notes |
-|---|---|---|---|---|---|
-| 1 | Title + tagline + stats badge | 1–13 | — | Keep above branches | Survives unchanged; counts updated by `update_readme_counts`. |
-| 2 | `## Start here` (three-paths table) | 15–25 | — | **Drop** | Replaced by the three branch sections themselves; rows map cleanly: `/onboard` → Use, `task ci` → Contribute, `task generate-tools` → Use. |
-| 3 | `## Quickstart` lead-in | 27–39 | Use it | Move | Becomes the opening paragraph under `#use-it`. |
-| 4 | `### For teams (recommended)` | 40–79 | Use it | Move | Primary CTA for `#use-it`. |
-| 5 | `### Pick specific AIs` | 81–101 | Use it | Move | Stays under Quickstart subtree. |
-| 6 | `#### Global install` | 103–124 | Use it | Move | Subsection of Pick specific AIs. |
-| 7 | `### For individual use (optional)` | 126–144 | Use it | Move | Alternate install path. |
-| 8 | `### Self-hosted MCP on Cloudflare` | 146–226 | Use it | Move | Operator install path; deep but consumer-facing. |
-| 9 | `#### Lock your Worker behind Bearer` | 196–213 | Use it | Move | Subsection of MCP block; stays nested. |
-| 10 | `### Optional: persistent agent memory` | 228–247 | Use it | Move | Companion package install. |
-| 11 | `## 2-minute demo: /implement-ticket` | 251–285 | Prove it | Move | Flagship evidence surface. Primary CTA for `#prove-it`. |
-| 12 | `### Sibling entrypoint: /work` | 287–316 | Prove it | Move | Same engine, second envelope. |
-| 13 | `### Product UI track` | 318–347 | Prove it | Move | Third evidence surface. |
-| 14 | `## What your agent is asked to do` | 351–365 | Prove it | Move | Intent table — proof of behaviour, not features. |
-| 15 | `## What this package is — and what it isn't` | 369–398 | Prove it | Move | Scope-honesty surface; loadbearing for the "proof" framing. |
-| 16 | `## You don't need everything` (cost profiles) | 402–423 | Prove it | Reframe | Currently sits as "feature" prose; the new framing is "proof that the package shrinks to fit". |
-| 17 | `## Who this is for` (stack coverage) | 427–439 | Prove it | Move | Honest depth claim — also evidence-side. |
-| 18 | `## Featured Skills` | 443–462 | Use it | Move | Catalog teaser → consumer surface. |
-| 19 | `## Featured Commands` | 466–481 | Use it | Move | Catalog teaser → consumer surface. |
-| 20 | `## Supported Tools / Project-installed` | 487–527 | Use it | Move | Per-tool install matrix. |
-| 21 | `## Supported Tools / Plugin-installed` | 529–541 | Use it | Move | Subsection. |
-| 22 | `## Supported Tools / Cloud / Hosted-agent` | 543–558 | Use it | Move | Subsection. |
-| 23 | `## Core Principles` | 562–570 | Prove it | Move | Behavioural floor — proof-side. |
-| 24 | `## Documentation` (index table) | 574–589 | Use it | Move | Doc portal entry. |
-| 25 | `### Maintainer telemetry (opt-in)` | 591–608 | Contribute | Move | Engagement measurement — maintainer / contributor surface. |
-| 26 | `### Context-aware command suggestion` | 610–629 | Use it | Move | Consumer-facing feature toggle. |
-| 27 | `## Development` | 633–642 | Contribute | Move | Primary CTA for `#contribute`. |
-| 28 | `## Requirements` | 644–649 | Use it | Move | Install gate — Use-side, not Contribute. |
-| 29 | `## License` | 651–653 | — | Keep at bottom | Footer; outside the three branches. |
-
-## Branch outlines (post-migration shape)
-
-### `## Use it in your project`
-
-Opening paragraph: one-line "Two minutes from npx to a better-behaved
-agent." Primary CTA: `npx @event4u/agent-config init`. Children:
-Quickstart subtree (#3–#7), MCP operator path (#8–#9), optional memory
-(#10), Featured Skills + Commands (#18–#19), Supported Tools (#20–#22),
-Documentation (#24), Command suggestion (#26), Requirements (#28).
-
-### `## Prove it`
-
-Opening paragraph: one-line "What the agent actually does, with
-evidence." Primary CTA: `/implement-ticket` demo (#11). Children:
-`/work` (#12), Product UI track (#13), Intent table (#14), Scope
-statement (#15), Cost profiles reframed (#16), Stack coverage (#17),
-Core Principles (#23).
-
-### `## Contribute`
-
-Opening paragraph: one-line "Editing rules, skills, commands — the
-contributor loop." Primary CTA: `task ci` (#27). Children: Maintainer
-telemetry (#25). External links: `CONTRIBUTING.md`, `AGENTS.md`,
-`docs/development.md`.
-
-## Verification (P2.2c preview)
-
-Grep-based test asserts `## Use it in your project`, `## Prove it`,
-`## Contribute` appear in that order. `lint-readme` keeps anchor
-stability for the rows in the Anchor-stability promise table.

package/docs/recruits/_template.md

@@ -1,81 +0,0 @@
-# Recruit — `<short-handle>`
-
-> **step-13 Phase 1 intake template.** One file per recruit. Filename:
-> `<YYYY-MM-DD>-<source>-<handle>.md` (e.g. `2026-05-20-indiehackers-jsmith.md`).
-> The maintainer fills this in **before** the recruit's first session; the
-> session-log section gets appended live during the walkthrough.
-
-## Identity & source
-
-- **Handle / display name:** `<as-they-want-to-be-credited>`
-- **Source channel:** (indie-hackers | r/ContentWritingJobs | product-hunt | direct-dm | other)
-- **First contact:** YYYY-MM-DD via `<thread-url-or-DM-context>`
-- **Role / self-description:** `<one-line, in their words>`
-- **User-type fit:** (consultant | creator | founder | finance | ops | gtm | other)
-- **Tool host they already use:** (claude-desktop | claude-code | cursor | windsurf | copilot | chatgpt-web | none)
-
-## Consent
-
-- **Attribution:** (full-name-OK | first-name-OK | role-only | fully-anonymous)
-- **Screenshot publication:** (yes | yes-with-redaction | no)
-- **Verbatim quote publication:** (yes | yes-with-review | no)
-- **Case-study publication if outcome lands:** (yes | maybe-revisit | no)
-- **Consent record:** `<link-to-dm-or-email-thread-or-signed-form>`
-
-> Consent gate per step-13 Phase 1 row 3 — a recruit without an explicit
-> consent record cannot anchor `agents/evidence/eval-findings/`. Anonymised
-> finding is acceptable; missing consent record is not.
-
-## Pre-session readiness
-
-- [ ] Recruit has Claude Desktop installed *(or equivalent MCP host)*
-- [ ] Recruit has a real task ready to bring to the session
-- [ ] Maintainer has [`docs/mcp-server.md`](../mcp-server.md) open in another tab
-- [ ] Maintainer has stopwatch / screen-recording ready for the install-time
-  measurement (Phase 1 row 2 — `< 10 minutes` gate)
-
-## Session log
-
-Append-only during the walkthrough. Timestamp every observation;
-"silent" minutes are also a signal.
-
-```
-HH:MM  start — recruit shares screen, opens Claude Desktop
-HH:MM  install step 1 (`task mcp:setup`): outcome / friction / quote
-HH:MM  install step 2 (Claude Desktop config paste): outcome / friction / quote
-HH:MM  first invocation attempt: prompt / which skill fired / verdict
-...
-HH:MM  end — total install time: __ min __ s
-```
-
-## Friction inventory
-
-At least **two** real friction points (per step-12 Phase 7 L130 spirit).
-Empty list = the session wasn't real.
-
-1. <where they stalled, in their words> → <what we fixed / parked>
-2. <...>
-
-## Outcome verdict
-
-- **MCP setup time:** `__ minutes __ seconds` (gate: `< 10 min`)
-- **First useful invocation reached without terminal?:** (yes | no | partial)
-- **Recruit would recommend to a peer?:** (yes | maybe | no | declined-to-answer)
-- **Eligible to anchor `agents/evidence/eval-findings/`?:** (yes | no — reason)
-
-## Cross-links
-
-- **Eval-finding file (if logged):** `agents/evidence/eval-findings/YYYY-MM-DD-<slug>.md`
-- **Case-study file (if subject opted in):** `docs/case-studies/YYYY-MM-DD-<type>-<slug>.md`
-- **Roadmap rows closed by this session:**
-  - step-13 P1 row 1 (recruit) — yes / no
-  - step-13 P1 row 2 (`< 10 min` gate) — yes / no
-  - step-13 P1 row 3 (consent record) — yes / no
-
-## Maintainer follow-up
-
-- [ ] Eval-finding committed
-- [ ] Recruit thanked + sent the published finding link
-- [ ] Skill-description deltas filed *(any friction that maps to a
-  skill description widening or a missing trigger phrase)*
-- [ ] Case-study slot offered *(if outcome was useful + consent allows)*