instar 1.3.981 → 1.3.983

package/upgrades/side-effects/behavioural-promise-unverifiable.md

@@ -0,0 +1,156 @@
+# Side-Effects Review — a promise was "kept" if a text field was filled in
+
+**Version / slug:** `behavioural-promise-unverifiable`
+**Date:** `2026-07-26`
+**Author:** `Echo (instar-dev agent)`
+**Second-pass reviewer:** `author-applied lenses — see Phase 5 (reduced independence, disclosed)`
+
+## Summary of the change
+
+`CommitmentTracker.verifyBehavioral` decided whether a behavioural promise had been
+kept by checking `content.includes(commitment.id)` against the behavioural rules file.
+That file is written **by this same class** from its own active list, and — verified
+across the whole agent home — **nothing else reads it**. So the check verified that the
+system had successfully written its own file, and its outcome was decided entirely by
+whether the optional `behavioralRule` field was populated.
+
+Measured on the live store: **74 behavioural commitments WITH the field all read
+`verified`; 24 WITHOUT it all read `violated`. 98 of 98, no exceptions in either
+direction.** Not one status observed conduct. `verified` was false comfort; `violated`
+was a false alarm (a missing optional field is not a broken promise). Both accumulated a
+counter tick on every sweep — CMT-068 reached **162,344**, and four commitments created
+in the same minute held **identical** counts (80,425), which individual behaviour cannot
+produce.
+
+Behavioural commitments now take the same path this file already gives an unverifiable
+`one-time-action`: the sweep is a **no-op**, the promise stays `pending`, and closure
+stays explicit (`deliver()` / `markDelivered()` / `expiresAt`). The rules file still
+self-heals, but repairing bookkeeping is no longer mistaken for evidence.
+
+## Decision-point inventory
+
+| point | classification | note |
+|---|---|---|
+| behavioural early-return in `verifyOne` | `invariant` | Deterministic on `type`. Mirrors the adjacent unverifiable-one-time-action return, which carries the same reasoning and history. |
+| `ensureBehavioralRulesFile` | `invariant` | Pure repair, returns nothing. It cannot influence a status — that conflation *was* the defect. |
+| `getHealth` message | `invariant` | Counts actual `verified` rows instead of inferring verification from the absence of violations. |
+
+## 1. Over-block
+
+**What legitimate inputs does this change reject that it shouldn't?**
+
+Nothing is blocked; this is a status computation with no gating authority. The nearest
+analogue is **withholding a `verified` status from a promise genuinely being honoured** —
+and that is the point: nothing here can observe conduct, so the status was never earned.
+74 rows move from `verified` to `pending`, which is a loss of *false* reassurance only.
+
+Real cost, stated plainly: an operator who read `verified` as "Echo is complying" loses
+that signal. It was never true, so the honest replacement is silence rather than a
+number — but the change does remove something people may have been relying on.
+
+## 2. Under-block
+
+**What failure modes does this still miss?**
+
+- **Behavioural compliance is still unverified.** This change makes that visible; it does
+  not solve it. Genuine verification would need an observation of conduct (e.g. grading
+  outbound turns against the rule), which is a much larger piece of work and is NOT
+  attempted here. Recorded rather than silently implied.
+- `config-change` verification is untouched and still genuinely checks live state.
+- The rules file remains written-and-read-by-nobody. Removing it entirely is a separate
+  question (its `getBehavioralContext` sibling *is* served over HTTP, so the content has
+  a consumer even though the file does not).
+- `writeBehavioralRules`' comment still claims "remove the file so hooks skip injection",
+  describing a consumer that does not exist. Left in place rather than edited blind;
+  noted as a candidate.
+
+## 3. Level-of-abstraction fit
+
+**Is this at the right layer?**
+
+Yes, and deliberately at the *same* layer as the existing precedent: an early return in
+`verifyOne`, ten lines below the one that handles unverifiable one-time-actions, sharing
+its rationale. No new status was introduced — `pending` already means "not determined",
+and adding an `unverifiable` state would have required touching the status union, three
+terminal-status guards, `getActive`, and auto-expiry eligibility for no semantic gain.
+
+## 4. Signal vs authority compliance
+
+**Required reference:** [docs/signal-vs-authority.md](../../docs/signal-vs-authority.md)
+
+**Does this change hold blocking authority with brittle logic?**
+
+No. It *removes* an authority that was exercised on brittle grounds: the old path
+asserted a verdict on the operator's promises from a filesystem read. Nothing here
+blocks, delays or alters any message or action. `getHealth` no longer reports `degraded`
+from 24 false violations, which strictly reduces false authority.
+
+## 4b. Judgment-point check (Judgment Within Floors standard)
+
+No judgment point is introduced; the new branch is a `type` comparison. The change
+*retires* a pseudo-judgment (file presence standing in for conduct). The asymmetry is
+explicit in code: absence of a verdict is reported as absence, never as either outcome.
+
+## 5. Interactions
+
+- **`getHealth`** was the one consumer that read `violated`: 24 false violations made the
+  component permanently `degraded`. Now it reports counts it can defend. Verified by grep
+  that no code outside this class reads `status === 'violated'`.
+- **`isAutoExpiryEligible`** treats `pending` and `violated` identically, so moving rows
+  between them causes no change in expiry behaviour — checked before editing.
+- **PromiseBeacon / overdue surfacing** operate on open commitments; `pending` keeps them
+  visible, which is the stated intent of the precedent ("should NAG, not vanish").
+- No persistence change, no migration, no config key. Existing rows re-evaluate to
+  `pending` on the next sweep without a data rewrite.
+
+## 6. External surfaces
+
+`GET /commitments` and `GET /commitments/:id` will show behavioural promises as `pending`
+rather than `verified`/`violated`, with both counters at 0. `GET /health` reports the
+commitment component's message differently. No route shape changes; no field is removed.
+
+## 6b. Operator-surface quality
+
+The operator previously saw either a reassuring `verified` that meant nothing, or a
+`violated` (with a five-figure count) that meant nothing — and, because 24 rows were
+falsely violated, a permanently `degraded` health component. They now see `pending` with
+honest zeroes, and a health line naming how many promises are simply not automatically
+checkable. `pending` is deliberately not alarming: nothing is wrong, nothing is known.
+
+## 7. Multi-machine posture (Cross-Machine Coherence)
+
+**Posture: `unified` by construction — no new state.** This changes how an existing
+per-machine store's rows are evaluated; it introduces no new field, file, or surface.
+Each machine already keeps its own commitment store (existing posture, unchanged by this
+work), and the evaluation is a pure function of a row plus that machine's rules file. No
+replication path is required and no `machine-local-justification` marker is needed,
+because no new machine-local state is introduced.
+
+## 8. Rollback cost
+
+One commit touching one source file and its test file. No migration, no persisted state,
+no config. Reverting restores the old semantics — and with them 98-of-98 statuses decided
+by a text field, plus the tick accumulation documented above.
+
+## Phase 5 — Second-pass review (independent reviewer subagent)
+
+**Disclosure, per Truthful Provenance:** no independent reviewer subagent was spawned — a
+standing instruction in this session prohibits it unless the operator requests it. The
+review lenses were applied by the author. That is **reduced independence**, recorded as
+such rather than presented as a concurring second pass.
+
+What author-applied review caught and changed:
+
+1. **The first instinct was to fix only the missing-field half** (stop marking a promise
+   violated for an absent optional field). That would have left the *false-comfort* half —
+   74 rows still reading `verified` — which is the more dangerous direction, and would have
+   repeated the "fix one site, leave its twin" mistake flagged on PR #1647 earlier tonight.
+   Both halves are addressed.
+2. **`getHealth`'s success message had to change too.** With behavioural rows at `pending`,
+   "all verified" would have become a *new* false claim introduced by this very fix — the
+   defect migrating one line sideways.
+3. **The rules-file self-heal was nearly lost.** The old method regenerated the file when
+   missing; deleting the method wholesale would have dropped that. Preserved as
+   `ensureBehavioralRulesFile`, which deliberately returns nothing.
+4. **Consumers were checked, not assumed** — establishing that `getHealth` was the only
+   reader of `violated` and that auto-expiry treats `pending`/`violated` alike.

package/upgrades/side-effects/merged-record-carries-its-evidence.md

@@ -0,0 +1,230 @@
+# Side-Effects Review — a merged record that could not name what merged it
+
+**Version / slug:** `merged-record-carries-its-evidence`
+**Date:** `2026-07-26`
+**Author:** `Echo (instar-dev agent)`
+**Second-pass reviewer:** `author-applied lenses — see Phase 5 (reduced independence, disclosed)`
+
+## Summary of the change
+
+`/projects/:id/advance` validated a submitted artifact for `building → merged` — PR
+state `MERGED`, a format-valid merge sha, that sha reachable from canonical main, CI
+rollup green — and then wrote **only** `pipelineStage`. The validated evidence built
+the validation context and was discarded. Measured on the live store: both Tier-1
+children of `convergence-towards-coherence` read `pipelineStage: "merged"` with no
+`prNumber`, no `mergeCommitOid`, no `ciCheckedAt`.
+
+The root cause was a **type**. `StageTransitionResult`'s success case was `{ ok: true }`
+— one bit — while every refusal carried `reason` + `code`. The validator explained
+itself when it refused and said nothing when it approved, so the caller had nothing to
+persist. The fix widens the success case to carry a `MergedEvidence` record and
+persists it in the same `update()` as the stage.
+
+**The consequential half:** two merged-state reconcilers select on `mergeCommitOid` —
+`GET /projects/:id`'s lazy reconciler (documented "may mutate", runs on every read) and
+`verifyMergedItemsViaGit`. With the field never written, the candidate set was always
+empty, `verifyMergedItemsViaGit` was never called, and both reported nothing. A
+regression detector that scans nothing is indistinguishable from one that finds no
+regressions.
+
+Because that path had never executed, three defects sat in it unexercised — all three
+already fixed in the advance path a few hundred lines away: (1) `SafeGitExecutor.run`
+without `sourceTreeReadOk`, so SourceTreeGuard refuses the read against an instar
+source tree (the #1641 defect); (2) a hardcoded `origin/main`, which on a dev-agent
+home is the agent's FORK; (3) `catch {}` → not verified → caller marks `regressed`,
+i.e. a refusal rendered as "it was reverted". Writing the evidence is what ARMS that
+path, so the two halves ship together.
+
+## Decision-point inventory
+
+| point | classification | note |
+|---|---|---|
+| success-case evidence on `StageTransitionResult` | `invariant` | Pure data carriage. Adds no branch and cannot change a verdict — the same four checks decide `ok`. |
+| evidence persistence in `/advance` | `invariant` | Conditional on `result.evidence` presence only. Same single write as the stage, so no window exists where `merged` has no evidence. |
+| `verifyMergedItemsViaGit` three-state outcome | `invariant` | Deterministic on git's documented exit status: 1 ⇒ regressed, everything else ⇒ unverifiable. No judgment. |
+| reconciler unverifiable branch | `invariant` | Deterministic on set membership. Strictly *removes* an authority — the old code demoted on absence-from-`verified`. |
+
+## 1. Over-block
+
+**What legitimate inputs does this change reject that it shouldn't?**
+
+Nothing new is rejected. `/advance` accepts exactly what it accepted before — the four
+checks and their refusal codes are untouched; only the success return widened. No
+artifact field became required.
+
+The reconciler now demotes **strictly fewer** items: previously anything absent from
+`verified` became `regressed`; now only git exit 1 does. So the change cannot introduce
+a false demotion, only remove them.
+
+Real cost, stated plainly: an item that genuinely regressed but whose check cannot be
+run stays at `merged` rather than being demoted. That is a deliberate trade — the old
+behaviour "caught" it only by demoting everything it could not verify, which is not
+detection. The item keeps taking the `ciCheckedAt` backoff, so it is re-asked, and its
+unverifiable reason is returned to the caller rather than swallowed.
+
+## 2. Under-block
+
+**What failure modes does this still miss?**
+
+- **No new regression detection is added.** This makes the existing reconcilers
+  *possible* (they now have evidence to select on) and *safe* (they can no longer guess).
+  Whether they catch a real revert is unproven until one occurs — recorded rather than
+  implied.
+- **Items merged BEFORE this change carry no evidence**, so they remain invisible to the
+  reconcilers. They surface as `unverifiable` with the reason "no mergeCommitOid
+  recorded on the item" rather than silently dropping out of the candidate set. No
+  backfill is attempted: reconstructing which PR merged a historical item would mean
+  guessing, and a fabricated evidence row is worse than an absent one.
+- The `GET /projects/:id` reconciler still selects only `pipelineStage === 'building'`
+  children, so an item that went straight to `merged` via `/advance` is not revalidated
+  by it. `verifyMergedItemsViaGit` is reachable for any child id the caller passes.
+  Widening that selection is a behavioural change to which items get demoted and is NOT
+  attempted here.
+- `defaultVerifyMergedItems` remains a deliberate no-op stub for the injected
+  `verifyMergedItems` seam (returns an empty Set). Its own comment says production
+  callers should pass a real verifier; the one production caller does.
+
+## 3. Level-of-abstraction fit
+
+**Is this at the right layer?**
+
+Yes. The evidence originates in the validator (the only place that proves it), is
+carried on the validator's own result type, and is persisted by the route that owns the
+write. No new store, no new field on the wire, no new endpoint — `prNumber`,
+`mergeCommitOid` and `ciCheckedAt` are pre-existing declared `Initiative` fields, and
+`mergeCommitOid`'s own comment already said "recorded at building → merged". The fix
+makes the comment true rather than inventing a mechanism.
+
+## 4. Signal vs authority compliance
+
+**Required reference:** [docs/signal-vs-authority.md](../../docs/signal-vs-authority.md)
+
+**Does this change hold blocking authority with brittle logic?**
+
+It **reduces** brittle authority. The reconciler holds real authority (it demotes items
+to `regressed` and clears a round's future `autoAdvanceAt`), and it was exercising that
+authority on the brittle ground of "absent from a set that could be empty for any
+reason, including a permission refusal". Demotion now requires git's single documented
+negative exit status. Nothing in this change blocks, delays, or alters a message or an
+action.
+
+## 4b. Judgment-point check (Judgment Within Floors standard)
+
+No judgment point is introduced. Every new branch is a deterministic comparison (a
+`status === 1` test, a set-membership test, a presence test on `result.evidence`). The
+change *retires* a pseudo-judgment — treating an unanswerable question as a negative
+answer — and makes the asymmetry explicit in the type: `unverifiable` is a first-class
+outcome carrying its reason, never folded into either verdict.
+
+## 5. Interactions
+
+- **`GET /projects/:id` lazy reconciler** — the one production consumer. Now receives a
+  three-state result and a resolved canonical-main ref. An unverifiable candidate takes
+  the `ciCheckedAt` backoff and keeps its stage; only `regressed` membership triggers a
+  demotion and the round's `autoAdvanceAt` clearing (that downstream logic is unchanged).
+- **`verifyMergedItemsViaGit` signature** — return type widened
+  (`Set<string>` → `MergedVerificationResult`) and a fourth optional parameter added.
+  Grepped all callers: exactly one production callsite (`routes.ts`), updated. The
+  injected `verifyMergedItems` seam on `RunRoundInput` is a DIFFERENT type
+  (`Set<string>`) and is untouched, so `ProjectRoundExecution`'s own callers and its 8
+  unit tests are unaffected — verified by running them.
+- **`StageTransitionResult`** — the widened success case is an optional property, so
+  every existing `if (!result.ok)` caller and every `expect(r.ok).toBe(true)` assertion
+  keeps working. Confirmed: 48 pre-existing validator tests pass unchanged.
+- No persistence migration: the three fields already exist in the `Initiative` type,
+  the create/update allowlists, and the JSON store.
+
+## 6. External surfaces
+
+`POST /projects/:id/advance`'s 200 body gains `item.prNumber`, `item.mergeCommitOid`,
+`item.ciCheckedAt` and an `evidence` object on a merged transition. Additive only — no
+field removed, no shape changed, no status code changed. `GET /projects/:id` children
+now carry those fields once written. Refusal responses are byte-identical.
+
+## 6b. Operator-surface quality
+
+The operator previously read `merged` with no way to ask "merged by what?" — and got a
+silent all-clear from a regression detector that had never examined anything. They now
+see the PR, the merge commit and the check time on the record itself, and an
+unverifiable item names its reason instead of being demoted. Items merged before this
+change honestly report "no mergeCommitOid recorded" rather than appearing clean.
+
+## 7. Multi-machine posture (Cross-Machine Coherence)
+
+**Posture: `unified` by construction — no new state.** No new field, file, store, or
+surface is introduced; three already-declared fields on an existing per-machine store
+start being written. The evidence is a pure function of a PR's GitHub state plus the
+target repo's git history, both of which are machine-independent facts. The
+canonical-main ref is resolved per-machine from that machine's remotes, which is
+correct: it names the same upstream commit graph whatever the local remote is called —
+and resolving it (rather than assuming `origin/main`) is precisely what makes the check
+give the same answer on a fork-origin home as on a canonical one. No replication path
+is required and no `machine-local-justification` marker is needed.
+
+## 8. Rollback cost
+
+Three source files, one commit, no migration and no persisted-state change. Reverting
+restores a `merged` record that cannot name its evidence and two reconcilers that
+select nothing — and re-arms the "could not check ⇒ regressed" mistranslation the
+moment anything writes `mergeCommitOid`. Written-but-unread evidence fields are inert,
+so a revert leaves no corrupt data behind.
+
+## Phase 5 — Second-pass review (independent reviewer subagent)
+
+**Disclosure, per Truthful Provenance:** no independent reviewer subagent was spawned —
+a standing instruction in this session prohibits it unless the operator requests it.
+The review lenses were applied by the author. That is **reduced independence**, recorded
+as such rather than presented as a concurring second pass.
+
+What author-applied review caught and changed:
+
+1. **The first instinct was to persist the evidence and stop** — a two-line change that
+   would have armed a code path carrying three unexercised defects, turning silent
+   blindness into confident false regressions. The consumer had to be read before the
+   producer was fixed. This is the same "fix one site, leave its twin" mistake flagged
+   on #1647 and again on #1650 earlier the same night; third occurrence, caught before
+   shipping this time.
+2. **The hardcoded `origin/main` would have mis-fired specifically on this machine.**
+   The advance path had already resolved it via `resolveCanonicalMainRef` for exactly
+   this reason. Passing the resolved ref through was not optional — without it a
+   dev-agent home demotes every healthy item.
+3. **The unverifiable branch initially bumped nothing**, which would have re-asked an
+   unanswerable question on every read (a hot loop). It now takes the `ciCheckedAt`
+   backoff while explicitly leaving `pipelineStage` alone.
+4. **Consumers were grepped, not assumed** — establishing that the widened return type
+   has exactly one production caller and that the similarly-named injected
+   `verifyMergedItems` seam is a distinct type that must NOT change.
+5. **One new test passed for the wrong reason and was fixed:** a fixed-size text window
+   ran past the branch it meant to inspect and matched the next block's
+   `pipelineStage`. Now bounded at the branch's own `continue`. A test measuring
+   adjacent text is the same defect class as the subject of this change.
+
+## Phase 6 — What CI refused, and why the fix went in the prose
+
+The first push was refused by the empty-catch ratchet
+(`tests/unit/no-empty-catch-blocks.test.ts`): count 1 against a baseline of 0. The
+offending occurrence was at `ProjectRoundExecution.ts:506` — **inside this change's own
+doc comment**, on the line quoting the forbidden bodyless-catch shape while explaining
+that it had been removed. The lint scans `src/` as text and does not strip comments, so
+it cannot distinguish an example from an instance.
+
+Noted rather than glossed, because it is the same failure mode this change is about,
+one layer out: a checker built to catch "errors turning into nothing" was tripped by a
+sentence *about* errors turning into nothing. It is also the third occurrence of that
+shape in a single session (twice in the outbound-message checker, once here), and the
+sibling test `tests/unit/projects-advance-mergebase-wiring.test.ts` already documents
+the identical trap for its own assertions — it strips comments before asserting,
+precisely because an earlier version of it matched the helper's own explanatory prose.
+
+**The prose was reworded; the lint was NOT touched.** Weakening a safety check to
+unblock one's own change is the wrong instinct even when the check is imprecise, and
+this lint's stated history (a bare catch in a 5-second loop causing a real cost
+incident) earns it the benefit of the doubt. Making the lint comment-aware is a
+legitimate improvement, but it belongs in its own change with its own review — not
+smuggled in as a side-effect of needing this PR to go green.
+
+**Re-verification scope, corrected.** The refusal proved my local scope was wrong: I
+had run the tests covering the changed module, but this failure came from a test that
+scans the whole source tree and whose subject my change never touched. The right scope
+for a source change is therefore *every tree-scanning test*, located by grep rather
+than guessed: 99 files, 8,370 tests, exit 0, `tsc --noEmit` clean.