instar 1.3.976 → 1.3.978

package/upgrades/side-effects/assessment-confidence-unverified.md

@@ -0,0 +1,125 @@
+# Side-Effects Review — assessmentConfidence: my own honesty field was overclaiming
+
+## Summary of the change
+
+`CoverageSummary.assessmentTrustworthy` (shipped hours earlier in PR #1641) reported **`true` over
+the exact defect it was added to expose.** Verified on the LIVE endpoint after v1.3.975 deployed:
+
+```
+total: 22 | enforcedRatio: 0.0455 | assessmentTrustworthy: TRUE | canaryOk: true | canaryFailures: 0
+registry.parsed: 22 | articleHeadings: 22 | bytes: 46606 | families: [5 of the 6]
+```
+
+The source registry carries **81** articles. The stale agent-home copy is not malformed — it is a
+self-consistent document that is a quarter of the real one, so it passes every INTERNAL check by
+construction (all headings parse, none dropped, anchors present, count above the floor).
+
+`assessmentTrustworthy` was computed as `total > 0 && registry.canaryOk` — i.e. "my internal checks
+passed". It READ as "this assessment is trustworthy". Conflating those is the failure the whole
+instrument was being fixed for.
+
+Replaced by a tri-state `assessmentConfidence` (`'verified' | 'unverified' | 'untrustworthy'`) plus
+an always-populated `confidenceReason`. The boolean is retained, deprecated, TRUE only on
+`'verified'`.
+
+**Root insight:** nothing INSIDE a 22-rule document says it should have 81. Trustworthiness is not
+obtainable by looking harder at the file — it needs an EXTERNAL expectation. That mechanism is the
+blocked `standards-registry-snapshot-refresh` spec, so `'verified'` is unreachable today, and the
+honest verdict for every passing read is `'unverified'`.
+
+Proven against the real live registry (all four states):
+
+| input | before | after |
+|---|---|---|
+| the live 22-of-81 copy | `trustworthy: true` | `unverified` + reason |
+| the real 81-article registry | `true` | `unverified` — still correct; nothing to check against |
+| a MATCHING expectation | unreachable | `verified` |
+| a MISMATCHED expectation | unreachable | `untrustworthy` |
+
+## Decision-point inventory
+
+- `computeCoverage` / `GET /conformance/coverage{,/health}` — OBSERVE-ONLY. Gates nothing, blocks
+  nothing, no code path branches on the verdict. This changes what is REPORTED, never what anything
+  DOES.
+- `deriveAssessmentConfidence` — new, exported, pure. Extracted deliberately: inline, TypeScript's
+  flow analysis proved `'verified'` unreachable and REJECTED the comparison deriving the deprecated
+  boolean. That rejection is TRUE and is the point; a function keeps the union open for when an
+  expectation ships without pretending it is reachable now. (TS caught this, which is worth noting —
+  the compiler objected to a claim I could not yet honour.)
+- No gate, hook, sentinel, reaper, scheduler, migration or config surface touched.
+
+## 1. Over-block
+
+Nothing blocks. The adjacent risk is over-alarming: `assessmentTrustworthy` is now `false`
+EVERYWHERE, including for a complete and correct registry. That is intentional and argued in the
+ELI16: the instrument genuinely cannot verify completeness today, so anything stronger is the same
+overclaim in nicer clothes.
+
+The tri-state exists precisely so this is not alarming: `'unverified'` says *I do not know*, which is
+neither reassuring nor a red flag. A consumer reading only the deprecated boolean sees `false` and
+may over-read it — which is why the boolean is deprecated in its doc comment and the reason string is
+always populated.
+
+## 2. Under-block
+
+No detection is weakened: `'untrustworthy'` still fires on every case the boolean fired on (nothing
+parsed, canary objected), asserted in the updated tests. The change only ADDS a state between "fine"
+and "broken" where none existed.
+
+The residual gap, stated plainly: **a coherent stale registry is still reported with real-looking
+figures** (`total: 22, enforcedRatio: 0.0455`). It is now labelled `unverified` with a reason naming
+the possibility of "a coherent older copy", but the instrument cannot say *which* is the case. Only
+the blocked spec closes that. This change makes the uncertainty visible; it does not resolve it, and
+it must not be described as resolving it.
+
+## 3. Blast radius
+
+One field's semantics in `src/core/StandardsEnforcementAuditor.ts`; the route spreads the summary so
+the new fields flow automatically. Consumers checked with a full-text scan (`grep` silently returned
+nothing for a pattern present twice in this very file — python was used instead, and that tooling
+anomaly is recorded):
+
+- `assessmentTrustworthy` appears in exactly three test files (all updated) and nowhere in `src/`
+  outside its declaration and derivation. No runtime branch reads it.
+- The response is additive apart from the boolean's meaning narrowing; no field renamed or removed.
+
+## 4. Rollback plan
+
+Single-commit revert; no state, config, migration or persisted artifact. The audit recomputes per
+request.
+
+## 5. Test coverage (all three tiers)
+
+- **Unit** (`standards-enforcement-auditor.test.ts`, 16 pass): a coherent-but-stale registry (a real
+  whole-family subset of the live constitution) has zero dropped headings and `parsed ===
+  articleHeadings` — i.e. passes every internal check — and STILL must not read `verified`; plus
+  `deriveAssessmentConfidence`'s four verdicts in both directions, including that a mismatched
+  expectation is `untrustworthy` rather than merely `unverified`.
+- **Integration** (`conformance-dev-gate-route.test.ts`): the HTTP body carries
+  `assessmentConfidence: 'untrustworthy'` for an empty registry.
+- **E2E** (`standards-coverage-lifecycle.test.ts`): the verdict and its reason survive the production
+  initialization path.
+
+## 6. A fourth test found encoding the defect
+
+`standards-enforcement-auditor.test.ts` asserted `assessmentTrustworthy === true` for the full
+registry — pinning the overclaim as a specification. That is the FOURTH test found this evening
+asserting the behaviour it should have been challenging (the others: the standards parser's
+five-family allowlist, a fixture whose meaning changed with the wall clock, and the
+learning-velocity route's filed-items-count-as-learning fixtures).
+
+Four in one day is not a coincidence about those four tests. It is a property of how tests get
+written here: they pin what the code currently does. A green suite therefore attests to
+self-consistency, not correctness — which is the same shape as everything else this project has
+found, applied to our own verification. Recorded as a class; not fixed here, because the fix is a
+standard about how assertions are chosen, not a code change.
+
+## 7. How this was found, which is the transferable part
+
+Not by a test. By deploying and reading the live surface. The suite was fully green — it tested the
+cases I had thought of, which were exactly the cases my fix handled. The stale-but-coherent case was
+outside my imagination, and no amount of internal testing would have surfaced it.
+
+Two hours from shipping to discovery. The generalisable rule: **after shipping an honesty fix, read
+the live surface for the case you were fixing.** If the number you set out to correct still looks
+fine, you have not corrected it.

package/upgrades/side-effects/ci-green-latest-run.md

@@ -0,0 +1,119 @@
+# Side-Effects Review — CI greenness must use the LATEST run per check
+
+## Summary of the change
+
+`StageTransitionValidator.ciIsGreen` iterated EVERY entry in GitHub's `statusCheckRollup` and refused
+on any non-success conclusion. GitHub returns every run of every check, **including superseded ones**,
+so a check that failed and was then re-run to green appears twice and the failed entry never
+disappears.
+
+Verified on the live PR #1641 (merged legitimately at 02:07:44Z):
+
+```
+rollup entries: 31 | names appearing more than once: eli16 (2), UX impact declaration (2),
+                                                     UX assertions (messaging E2E) (2), UX merge safety (2)
+eli16  COMPLETED  FAILURE  completedAt 2026-07-26T01:44:01Z
+eli16  COMPLETED  SUCCESS  completedAt 2026-07-26T01:45:32Z
+```
+
+Branch protection merged the PR because it evaluates each check's CURRENT state. The tracker refused
+it with `CI_NOT_GREEN` because it evaluated the stale run too — so **no correct merge whose CI had
+ever been red could ever be recorded.**
+
+Fix: `latestRunPerCheck` collapses the rollup to the newest run per check name (by `completedAt`,
+falling back to `startedAt`) before `ciIsGreen` evaluates it. `GhPrView.statusCheckRollup`'s type
+gains the `name`/`context`/`startedAt`/`completedAt` fields the dedupe needs; the route's existing
+`gh pr view --json …statusCheckRollup` already returns them, so no query change.
+
+## Decision-point inventory
+
+- `ciIsGreen` feeds the `building → merged` GATE. This change makes the gate ACCEPT a case it
+  previously refused, so it is the direction that needs the most scrutiny — see §1/§2.
+- No other decision point, hook, sentinel, reaper, scheduler, migration or config surface is touched.
+- `latestRunPerCheck` is a pure function over the rollup array.
+
+## 1. Over-block
+
+Strictly reduced, and that is the intent: a merged PR with a superseded failure is now recordable.
+Nothing that used to pass now fails.
+
+The one preserved refusal worth naming: a **latest** run that is still in flight (`status` not
+`COMPLETED`) is still not green, asserted in its own test, so the dedupe cannot be used to skip a
+pending check by pairing it with an older success.
+
+## 2. Under-block — the part that matters
+
+A dedupe is a potential laundering mechanism, so three properties are enforced and each is tested:
+
+1. **Latest is authoritative in BOTH directions.** SUCCESS followed by a later FAILURE is NOT green.
+   Without this, "keep the latest" would let anyone bury a red by arranging run order.
+2. **Unorderable runs FAIL CLOSED.** Equal or missing timestamps ⇒ the FAILING run wins. An undatable
+   success must never mask a red. (Three variants tested: no timestamps either way round, and equal
+   timestamps.)
+3. **Unnamed entries are keyed individually.** Bare status contexts with neither `name` nor `context`
+   are never collapsed into each other, so a failing one cannot be replaced by a passing one.
+
+Residual, stated: the dedupe trusts GitHub's timestamps. A platform that reported them wrongly could
+mis-order runs — but the fail-closed rule bounds the damage to the unorderable case, and the
+alternative (ignoring the rollup entirely, or trusting merge status alone) is strictly weaker.
+
+## 3. Blast radius
+
+One function plus one type widening in `src/core/StageTransitionValidator.ts`. `ciIsGreen` is private
+to that module (`grep` for callers: only the `to === 'merged'` branch). The route passes the rollup
+through unchanged. No response shape changes; no new code.
+
+## 4. Rollback plan
+
+Single-commit revert; no state, config, migration or artifact. Reverting restores the previous
+(over-refusing) behaviour immediately.
+
+## 5. Test coverage
+
+`tests/unit/StageTransitionValidator.test.ts` (46 pass), five new cases: the live #1641 shape
+(superseded FAILURE + later SUCCESS ⇒ green); the reverse (SUCCESS + later FAILURE ⇒ not green);
+unorderable-runs-fail-closed across three variants; unnamed entries not collapsed; and an in-flight
+latest run still not green.
+
+No integration/e2e tier added: this is a pure predicate inside an existing gate whose route path is
+already covered, and the decisive evidence is the end-to-end check below.
+
+## 6. End-to-end evidence, gathered BEFORE the fix and after
+
+The validator's real logic was run against the two real merged PRs with the route's exact helper
+wiring:
+
+```
+before:  item 1 (PR 1641) -> CI_NOT_GREEN
+         item 2 (PR 1644) -> MERGE_BASE_UNVERIFIABLE   (stale local ref; resolved by git fetch)
+after:   item 1 (PR 1641) -> WOULD SUCCEED
+         item 2 (PR 1644) -> WOULD SUCCEED
+```
+
+Note the second line: the `MERGE_BASE_UNVERIFIABLE` code shipped hours earlier in #1643 did its job
+here on a real case — an unknown local SHA reported as *unverifiable* rather than as a false "not an
+ancestor".
+
+## 7. Third instance in one code path, in one evening
+
+The completion-recording path has now yielded three independent defects tonight, each the same
+underlying error:
+
+| | defect | what it reported |
+|---|---|---|
+| #1640 | the `gh` binary was unreachable | a flat refusal |
+| #1643 | a SourceTreeGuard refusal was caught and returned as `false` | "the merge is not on main" |
+| this | a superseded check run read as the check's current state | "CI is not green" |
+
+All three presented identically from outside — an opaque refusal — and each masked the next. That is
+worth recording as a property of the path rather than three coincidences: **a code path whose failures
+all render as one indistinguishable refusal will hide its own defects in series.** The general fix is
+the one #1643 started: distinguish "could not determine" from "determined, and the answer is no".
+
+## 8. How it was found
+
+By refusing to assert. I was about to report that recording would work once the server updated, and
+instead ran the validator's own logic against the real PRs. It returned two different refusals. That
+cost two minutes and avoided discovering both blockers serially, each appearing to be a fresh
+mystery. Habit worth keeping: **before claiming something will work once a dependency lands, execute
+its logic now and read what it returns.**

package/upgrades/side-effects/learning-velocity-counts-finishing.md

@@ -0,0 +1,102 @@
+# Side-Effects Review — learning velocity counts finishing, not filing (ACT-1244)
+
+## Summary of the change
+
+`GET /metrics/learning-velocity` counted EVERY evolution action at its `createdAt`, so the metric
+answering "are we learning?" was almost purely a measure of FILING RATE. Verified 2026-07-25: 739 of
+771 events (96%) were filed items; on the real queue 494 of those carry the sweep's resolution
+"Abandoned without active tracking since creation date". The metric read 88/100 "accelerating" on
+the morning the operator halted work because the opposite was visibly true — **the faster work was
+abandoned, the higher the score climbed.**
+
+An action now contributes a learning event ONLY when `status === 'completed'`, stamped at
+`completedAt` (falling back to `updatedAt`). Everything else is excluded and ACCOUNTED FOR by
+reason. The response gains `counting` (the rule, in one line) and `evolutionActions`
+(`considered` / `counted` / `excluded` by reason, with `auto-abandoned` named specifically).
+
+Measured before/after on this agent's live queue, 30-day window, rule change only:
+**784 events → 18.** Considered 1,288; excluded 743 `not-completed:pending`, 494 `auto-abandoned`,
+29 `not-completed:cancelled`, 2 `not-completed:in_progress`.
+
+## Decision-point inventory
+
+- `GET /metrics/learning-velocity` — a READ-ONLY observability route. It gates nothing, blocks
+  nothing, and no code path branches on its output. The change alters what it REPORTS, never what
+  anything DOES.
+- `computeLearningVelocity` (`src/core/LearningVelocityScorer.ts`) — **untouched.** It remains a
+  pure function over an event list; only the route's event GATHERING changed. Its 6 unit tests pass
+  unmodified, which is the evidence that the scorer's contract is unchanged.
+- No gate, hook, reaper, sentinel, scheduler, migration or config surface is touched.
+
+## 1. Over-block
+
+Not applicable — nothing here blocks. The adjacent risk is over-EXCLUSION: an action that genuinely
+represents learning but is not marked `completed` no longer counts. That is the intended semantics
+(the whole finding is that filing ≠ learning), and it is not silent: every exclusion is itemised by
+reason in the response, so a reader can see exactly what was set aside and disagree with the rule if
+they wish.
+
+Two exclusions were chosen deliberately and could be argued:
+
+- **`completed` with no `completedAt`/`updatedAt` → excluded, not back-dated to `createdAt`.**
+  Back-dating would re-import the entire filing bias through the one remaining gap. On the live
+  queue this bucket is EMPTY (all 20 completions carry `completedAt`), so the strict choice costs
+  nothing today and closes the door for later.
+- **`cancelled` for a considered reason → still excluded.** A deliberate cancellation may well
+  embody a lesson, but it is not a completion, and admitting it would reopen the "closing a row
+  counts as learning" path that produced the inversion.
+
+## 2. Under-block
+
+The score can now READ LOWER than reality if completions are recorded without status/timestamp
+hygiene. That is a reporting risk, not a safety one, and it is bounded by the accounting block: a
+low score with `counted: 1, excluded: {not-completed:pending: 743}` is legible as "almost nothing
+has finished", which is the true statement. The previous behaviour had the opposite failure — a high
+score that was structurally unfalsifiable — and an honest low number is strictly better than a
+flattering unfalsifiable one.
+
+`byType.learning` (registered learnings) and `byType.correction` (the correction ledger) are
+unchanged: both are genuine learning artifacts rather than filings, so neither was touched.
+
+## 3. Blast radius
+
+One route handler in `src/server/routes.ts`. Consumers checked:
+
+- `grep -rn "learning-velocity"` across `src/`, `scripts/`, `tests/` — the route is read by the
+  CLAUDE.md template docs and the dashboard's plain-language surface; no code branches on
+  `adaptabilityScore` or `byType.evolution`.
+- The response is ADDITIVE (`counting`, `evolutionActions`); no existing field was renamed or
+  removed, so a consumer reading `totalEvents`/`byType`/`adaptabilityScore` still parses.
+- `tests/unit/LearningVelocityScorer.test.ts` — 6 tests, unmodified, pass. The scorer is untouched.
+
+## 4. Rollback plan
+
+Single-commit revert. No state, no config key, no migration, no persisted artifact; the metric is
+recomputed from disk on every request. Reverting restores the old counting immediately.
+
+## 5. Test coverage (all three tiers)
+
+- **Unit** — `LearningVelocityScorer.test.ts` unchanged and passing (proves the pure scorer's
+  contract did not move).
+- **Integration** — `tests/integration/learning-velocity-routes.test.ts`, 7 tests. Three existing
+  cases UPDATED (see §6) and three added: the real queue in miniature (5 filed-or-abandoned + 1
+  completion → exactly 1 event, `auto-abandoned` counted as 2 and named); a `completed` action with
+  no timestamp excluded rather than back-dated; and a completion outside the window excluded by
+  DATE while still appearing in `counted`, so "counts completions" cannot become "counts every
+  completion ever".
+- **E2E** — `tests/e2e/learning-velocity-lifecycle.test.ts`: the `counting` rule and the
+  `evolutionActions` accounting must survive the PRODUCTION initialization path, because the live
+  endpoint is where a reader actually reads the number.
+
+## 6. Three existing tests were encoding the defect
+
+`learning-velocity-routes.test.ts` used action fixtures carrying only `createdAt` and asserted they
+counted as learning events. Those assertions were a specification of the inversion: they would have
+failed any correct implementation. Rewritten to use completions, with the filed-but-unfinished case
+retained and asserted as an EXCLUSION rather than deleted.
+
+This is the **third** time in one day a passing test was found protecting the defect beside it (the
+others: the standards parser's five-family allowlist, and a fixture whose meaning changed with the
+wall clock). Recorded rather than tidied away, because three instances in a day is a pattern about
+how our tests get written — they pin current behaviour, and current behaviour is what the audit
+found wanting. The class is worth a standard; it is not fixed here.