instar 1.3.989 → 1.3.991

package/upgrades/1.3.991.md

@@ -0,0 +1,57 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: patch -->
+
+## What Changed
+
+`IntentDriftDetector.alignmentScore()` reported `score: 0, grade: 'F'` when its analysis window held
+no decisions. The `summary` field said "alignment cannot be assessed", but `score` and `grade` are
+what consumers read — so "no data" and "assessed, catastrophically bad" were identical on every field
+in use.
+
+The grade union is now `'A'|'B'|'C'|'D'|'F'|'N/A'`, and the result carries `assessable: boolean`.
+`GET /intent/alignment` returns both. `instar intent drift` prints the reason instead of a fabricated
+grade when nothing was assessed.
+
+Root cause was vocabulary: with no grade meaning *no verdict*, absence had to borrow the worst real
+grade — the same shape as a status set with no way to say "undetermined".
+
+## What to Tell Your User
+
+If you look at the alignment score, a flat zero with an F used to mean one of two very different
+things: that your decisions genuinely scored badly, or that there was nothing to score. You could not
+tell which from the number.
+
+Now an unassessed period says so plainly, and the score carries a flag saying whether anything was
+actually measured.
+
+The honest scope: this changes what you are told, not how aligned anything is. It also does not
+change the default command view, where the unassessable case was already handled well — that path
+stops early with a helpful message and never reached the bad grade. The fabricated F is visible to
+anything reading the score programmatically, and on the command line only if you widen the window
+past thirty days by hand.
+
+## Summary of New Capabilities
+
+No new endpoint or command. An existing score gained an honest "not assessed" state and a flag
+distinguishing a placeholder from a measurement.
+
+## Evidence
+
+- Restoring the fabricated `'F'`: 3 failed | 25 passed. Making `assessable` constant: 5 failed.
+- Disabling the renderer's honest branch left all 28 module and route tests green — the wiring was
+  unguarded until a test was added that drives the real command and reads its output. With it, that
+  same change fails immediately.
+- Restored: 42 passed across the five affected files; `tsc --noEmit` exit 0.
+- Two existing tests asserted `grade === 'F'` on the empty case — they encoded the defect rather than
+  merely lagging it. Both updated, with the reason recorded in place.
+
+## Known limits
+
+The early-return window and the scoring window differ (the command's `--window`, default 14, versus a
+fixed 30), which is what makes the command-line case reachable at all; reconciling them changes what
+every user sees and is deliberately not bundled. `score` stays `0` on the unassessable case rather
+than becoming `null`, to avoid a breaking type change — `assessable` is the additive signal. Nothing
+here improves alignment or judges whether a cited principle was genuinely consulted.
+<!-- tracked: CMT-1044 -->

package/upgrades/side-effects/alignment-score-not-assessed.md

@@ -0,0 +1,178 @@
+# Side-Effects Review — an alignment score that can say "not assessed"
+
+**Version / slug:** `alignment-score-not-assessed`
+**Date:** `2026-07-26`
+**Author:** `Echo (instar-dev agent)`
+**Second-pass reviewer:** `see Phase 5`
+
+## Summary of the change
+
+`IntentDriftDetector.alignmentScore()` returned `score: 0, grade: 'F'` when the analysis window
+contained no decisions. The `summary` field was honest ("No decisions logged — alignment cannot be
+assessed") but no consumer read it; `score` and `grade` are what get rendered and compared. So
+"nothing to assess" and "assessed, catastrophically bad" were identical on every field in use — on
+the instrument whose purpose is honest alignment measurement.
+
+Root cause is vocabulary, the same shape as the channel registry one increment earlier: the grade
+union was `'A'|'B'|'C'|'D'|'F'` with no member meaning *no verdict*, so absence had to borrow the
+worst real grade.
+
+Adds `'N/A'` to the grade union, adds `assessable: boolean`, and makes `instar intent drift` print
+the reason instead of a fabricated grade.
+
+## Refusal evidence (constraint 2)
+
+```
+REFUSAL 1 — restore the fabricated 'F' on the unassessable case
+  × handles empty journal — not assessable, grade N/A      → expected 'F' to be 'N/A'
+  × GET /intent/alignment reports NOT ASSESSED             → expected 'F' to be 'N/A'
+  × an empty journal grades 'N/A', never 'F'               → expected 'F' to be 'N/A'
+  Tests  3 failed | 25 passed (28)
+
+REFUSAL 2 — always report assessable:true
+  × an empty journal is flagged unassessable               → expected true to be false
+  × a real assessment is DISTINGUISHABLE from an empty one → expected true not to be true
+  × assessable tracks sampleSize exactly                   → expected true to be false
+  (+2)                                                     Tests  5 failed
+
+REFUSAL 3 — disable the CLI's honest branch (`if (false && !alignment.assessable)`)
+  BEFORE the CLI test existed:  Tests  28 passed (28)   <-- the blindness, again
+  AFTER:  × a STALE journal prints "not assessed", never a red F
+          Tests  1 failed | 4 passed (5)
+```
+
+Restored: **42 passed** across the five affected files, `tsc --noEmit` exit 0.
+
+**REFUSAL 3 is the finding, and it is the THIRD occurrence of this class tonight** (#1658 route
+registry, #1659 route validator, now a CLI renderer). Each time the logic was thoroughly guarded and
+the wiring to the surface a human or API client actually reads was not. This one is the sharpest:
+the module returning `'N/A'` is worth precisely nothing if the renderer ignores it, and 28 green
+tests said everything was fine while the renderer was disabled.
+
+## Two of my own claims were falsified during this work
+
+Recorded because the corrections are the useful part, and because an artifact that hides them is the
+failure mode this tier exists to remove.
+
+1. **"`instar intent drift` has been showing a red F for the journal's whole life."** FALSE. The
+   command returns early with a genuinely helpful message when the window holds no decisions; it
+   never reaches the scoring block. The empty case was already handled honestly there.
+2. **"Then it is reachable whenever the journal is merely stale."** ALSO FALSE. The early return
+   checks `windowDays` (default 14) and `alignmentScore()` is fixed at 30 — and 14 ⊂ 30, so anything
+   clearing the early return is inside the alignment window by construction.
+
+**The actual reachable CLI case is narrow:** the operator must widen the window past 30
+(`--window 60`), so a 40-day-old decision clears the early return and falls outside the fixed 30-day
+alignment window. That is what the regression test constructs. I asserted twice before checking; the
+test is what settled it.
+
+## Decision-point inventory
+
+| point | classification | note |
+|---|---|---|
+| `sampleSize === 0` → `grade: 'N/A'`, `assessable: false` | `invariant` | Deterministic count check. No model. |
+| `assessable` mirrors `sampleSize > 0` | `invariant` | Asserted by test; the two can never disagree. |
+| CLI branches on `assessable` | `invariant` | Renders `summary` instead of a grade. |
+
+No judgment points, no LLM, nothing gated or blocked.
+
+## 1. Over-block
+
+Nothing is blocked — this is a read surface. The available harm is **misinforming a reader**, and
+this change strictly reduces it in the direction that mattered (absence no longer reads as failure).
+
+The mirror over-block is real and guarded: a genuinely-assessed period must never report `'N/A'` or
+`assessable: false`, or a real alignment problem would be hidden as "no data" — strictly worse than
+the original bug. Asserted by two tests (`a genuinely assessed period still reports a real letter
+grade`, and the CLI's `a populated journal still prints a real graded score`).
+
+**Caller sweep, run BEFORE writing this section** (the correction from #1659, where I wrote a
+confident risk claim from the wrong measurement and CI falsified it): `alignmentScore()` has exactly
+two production callers — `routes.ts:24305` (passes through verbatim) and `commands/intent.ts:431`
+(now branches). Every `.grade` hit elsewhere in `src/` belongs to `DecisionQualityRecorder`'s
+unrelated `right|wrong|unknown` grade, checked rather than assumed. Two tests asserted the old shape;
+both updated.
+
+## 2. Under-block
+
+**The mismatched windows are NOT fixed.** The early return uses `windowDays` while `alignmentScore()`
+is hardcoded to 30. That divergence is what makes the CLI case reachable at all, and reconciling them
+changes what the command reports for every user. Deliberately not folded in. <!-- tracked: CMT-1044 -->
+
+**`score: 0` is retained on the unassessable case.** Changing it to `null` would be a breaking type
+change for a field two consumers read; `assessable` is the additive signal instead. A consumer that
+reads `.score` and ignores both `assessable` and `sampleSize` still sees a 0 — it can no longer see
+an F, which is the part that read as a verdict.
+
+**It does not improve alignment,** and it does not judge whether a cited principle was genuinely
+consulted. Same honest limit as the increment before it.
+
+## 3. Level-of-abstraction fit
+
+The honest state lives in the returned value, not in the renderer, so every consumer inherits it —
+the route needed no change at all. The renderer's job is narrowed to *presenting* a state it no
+longer has to infer. Had I fixed only the CLI, the API consumer (the one that is actually reachable
+by default) would still have been lied to.
+
+## 4. Signal vs authority compliance
+
+Pure signal. `docs/signal-vs-authority.md` is satisfied trivially: it produces a read-only score that
+gates nothing, blocks nothing, and is consumed by one route and one command.
+
+## 4b. Judgment-point check (Judgment Within Floors standard)
+
+None introduced. Two deterministic branches on a count.
+
+## 5. Interactions
+
+- **`GET /intent/alignment`** — response gains `assessable`; `grade` may now be `'N/A'`. Additive plus
+  one widened union member.
+- **`IntentDriftDetector.analyze()`** — untouched; drift scoring is a separate path.
+- **`tests/unit/IntentDriftDetector.test.ts`** and **`tests/integration/drift-routes.test.ts`** — each
+  had a test asserting `grade === 'F'` on the empty case. **Both were encoding the defect**, not
+  merely stale. Updated with comments recording that, since a test that locks in a wrong answer is
+  the same instrument-honesty class as the defect itself.
+- **`CapabilityIndex`** — unchanged; `intent` is already `INTERNAL_PREFIXES`.
+
+## 6. External surfaces
+
+One API response shape change (additive field + widened union), one CLI rendering change. No config
+key, no persisted state, no migration, no message to any user.
+
+## 6b. Operator-surface quality
+
+The unassessable CLI output prints the reason (`No decisions logged — alignment cannot be assessed`)
+in dim rather than a red grade, so it reads as an absence of data rather than an alarm. The component
+breakdown is suppressed in that branch: four zeroed rows invite exactly the misreading being fixed.
+
+## 7. Multi-machine posture (Cross-Machine Coherence)
+
+**Machine-local BY DESIGN.** The journal is a per-machine JSONL under `stateDir` and the score is
+computed from it, so `assessable` answers "on this machine". An agent running on two machines has two
+journals and two scores. That predates this change and is unaddressed here. <!-- tracked: CMT-1044 -->
+
+## 8. Rollback cost
+
+Low. One union member, one boolean, one CLI branch, three test updates. No persisted state, no
+migration; existing journal rows are read unchanged. Reverting restores the fabricated F.
+
+## Phase 5 — Second-pass review
+
+Not a gate, sentinel, guard or watchdog; holds no block/allow authority; touches no session lifecycle
+or trust level. The high-risk trigger list is not engaged. Author lenses, disclosed:
+
+**Adversarial — "how would I make this useless?"** Three ways, all closed and asserted: report `'F'`
+again (refusal 1), make `assessable` constant (refusal 2), or let the renderer ignore it entirely
+(refusal 3 — the one that was genuinely open until I wrote the CLI test).
+
+**"Would it have caught the incident?"** The incident here is my own: I read `topPrinciples: []` and
+`score: 0 (F)` on a journal I already knew was empty, and had to reason my way to "that F is
+meaningless" instead of being told. With this, the surface says it.
+
+**"Symptom or cause?"** Cause, for the reporting defect — absence can no longer render as a verdict
+because the type now has somewhere honest to put it. Symptom-level for the window mismatch, which is
+named and left.
+
+**Weakest point:** the CLI case is genuinely narrow (`--window > 30`), and I over-claimed its reach
+twice before testing settled it. The API consumer is the one that matters by default. An artifact
+claiming broad user impact here would be overstating it, so this one does not.

package/upgrades/side-effects/decision-journal-principle-required.md

@@ -0,0 +1,227 @@
+# Side-Effects Review — a decision log that refuses a decision which does not say why
+
+**Version / slug:** `decision-journal-principle-required`
+**Date:** `2026-07-26`
+**Author:** `Echo (instar-dev agent)`
+**Second-pass reviewer:** `see Phase 5`
+
+## Summary of the change
+
+Operator directive (topic 29723, 20:4xZ): decide against the goal hierarchy instead of escalating,
+log the reasoning for later review, and **enforce it via infrastructure rather than by remembering**.
+
+Investigating what to build produced the finding that the machinery already exists — hierarchy
+(`GET /intent/org`, `/intent/tradeoff-resolve`), recorder (`DecisionJournal`, `POST /intent/journal`),
+and drift detector (`IntentDriftDetector`) — and had recorded **zero decisions, ever**. What is
+missing is not machinery; it is anything that forces its use or reveals its disuse.
+
+Then, using it, I produced the defect this PR fixes:
+
+- I POSTed `reasoning` and `checkedAgainst`. **Neither is a field.** The route spread `...rest`
+  straight through with no validation, so both were persisted where no reader consumes them. The
+  write returned 201. I then told my operator the reasoning was recorded — believing it.
+- `principle`, the typed field documented as *"Which AGENT.md principle or intent guided the choice"*,
+  was empty on all five entries.
+- `stats()` therefore reported `topPrinciples: []` — **byte-identical to an empty journal.** The
+  instrument built to detect unreasoned decisions could not distinguish its own worst case from a
+  clean slate.
+
+Adds `validateDecisionSubmission()` (pure) + `principledCount`/`unprincipledCount` on
+`DecisionJournalStats`, wires the validator into `POST /intent/journal`, and registers the behaviour
+change for new and existing agents.
+
+## Refusal evidence (constraint 2)
+
+```
+REFUSAL 1 — unwire the validator from the ROUTE (`if (false && !verdict.ok)`)
+  UNIT:        Tests  11 passed (11)      <-- the blindness, reproduced deliberately
+  INTEGRATION: × the ROUTE refuses a decision that names no principle
+               × the ROUTE refuses fields no reader consumes
+               × a refused submission writes NOTHING
+               × the refusal message tells the caller where the content belongs
+               Tests  4 failed | 2 passed (6)
+
+REFUSAL 2 — swallow unknown fields instead of refusing
+  × a field no reader consumes is REFUSED, not swallowed
+  × missing-required outranks unknown-fields, and still reports both
+  × (+3 integration)                       Tests  5 failed | 12 passed (17)
+
+REFUSAL 3 — drop `principle` from the required set
+  × a decision naming no guiding principle is REFUSED
+  × a blank or whitespace principle does not satisfy the requirement
+  × (+3)                                   Tests  5 failed | 12 passed (17)
+
+REFUSAL 4 — revert stats to the ambiguous shape
+  × entries with no principle are COUNTED, not silently absent   → expected +0 to be 2
+  × an empty journal is DISTINGUISHABLE from an unprincipled one → expected +0 not to be +0
+  × (+2)                                   Tests  4 failed | 13 passed (17)
+```
+
+Restored: **188 passed (188)** across the five affected files, `tsc --noEmit` exit 0.
+
+**REFUSAL 1 is the finding, and it is the second occurrence tonight of the same class.** One feature
+earlier (#1658) I emptied a route's registry and all 19 unit tests passed — module guarded, wiring
+not. So here I went looking for it deliberately: unwiring the validator leaves **11/11 unit tests
+green** while the route accepts everything it is supposed to refuse. The integration file exists for
+exactly this assertion and nothing else.
+
+## Decision-point inventory
+
+| point | classification | note |
+|---|---|---|
+| missing required field → refuse | `invariant` | Deterministic key/type check. No model call. |
+| unknown field → refuse | `invariant` | Allowlist comparison; allowlist asserted against the documented field set by test. |
+| missing-required outranks unknown-fields | `invariant` | Both are still reported, so one round trip surfaces both problems. |
+| machine dispatch path exempt | `invariant` | Scoped by callsite, not by inspecting content. |
+
+No judgment points. No LLM. Nothing is inferred about the *quality* of a cited principle — only that
+one was cited. Judging whether a decision genuinely followed the principle it names is the drift
+detector's job and is deliberately not attempted here.
+
+## 1. Over-block
+
+**This is the section that shaped the design.** A blanket requirement would have been wrong.
+
+`journal.log()` has two callers: the HTTP route (agent-authored) and `DispatchDecisionJournal
+.logDispatchDecision` via `AutoDispatcher`, which writes auto-applied dispatch decisions whose own
+documented shape is `{ dispatchDecision: 'accept', reasoning: 'auto-applied' }`. Those have no
+principle to cite. **Enforcing inside `log()` would have broken automatic dispatch to buy nothing**,
+so the refusal lives at the route and `log()` is untouched — asserted by a test that the module path
+still accepts an entry with no principle.
+
+Residual over-block risk on the agent path: a caller with a legitimate decision and genuinely no
+guiding principle now gets a 400. I accept this deliberately — under the operator directive, a
+decision made during operations without checking the stated goals is precisely what we are
+eliminating. The failure is loud, names the missing field, and is fixed by adding one string.
+
+Unknown-field rejection is a strict-schema change and could in principle break an existing caller.
+The refusal names the offending keys and the correct destination, so a broken caller is told exactly
+what to change rather than failing opaquely.
+
+**CORRECTION — my first version of this section was too confident, and CI proved it.** I wrote that
+"the journal had `count: 0` on this agent, so no caller has ever successfully written to it" and
+treated that as evidence the back-compat risk was near-nil. That measurement was about the *live
+agent's data file*; it says nothing about *callers in the codebase*. There was one:
+`tests/integration/intent-routes.test.ts` POSTed three decisions with no `principle`, and my change
+refused all three, so the journal file was never created and a later read failed `ENOENT`.
+
+I did not catch it locally because I ran a targeted file set that did not include that test. CI did.
+Two failures, both mine, neither a flake. Resolution: the tests now supply a `principle` — which is
+the *correct* fix, since they were writing exactly the unreasoned decisions this gate exists to
+refuse, and their failure is the refusal working on a real caller rather than a synthetic one.
+
+The honest generalisation: "no rows in the data file" is not evidence of "no callers in the code".
+Those are different questions and I conflated them. A repo-wide sweep afterwards found three files
+POSTing to the route and six asserting on `stats()` shape; two needed changes, and both are fixed.
+
+## 2. Under-block
+
+**It does not force the check, only the citation.** Nothing here fires at the moment a decision is
+made and puts the hierarchy in front of the agent. An agent can still decide without consulting
+`GET /intent/org` and then name a principle after the fact. This PR makes it impossible to *record*
+a decision that claims no guiding intent; it does not make it impossible to *make* one. The
+consult-side trigger is genuinely separate work and is not bundled. <!-- tracked: CMT-1044 -->
+
+**It does not judge the principle.** Any non-empty string satisfies the requirement. A caller citing
+"because I felt like it" passes. Judging alignment is `IntentDriftDetector`'s job.
+
+**The five existing entries are not repaired.** They keep their unread `reasoning`/`checkedAgainst`
+keys. The new counters will report them as unprincipled, which is accurate. Rewriting my own history
+to look better than it was is the opposite of this tier's purpose.
+
+**`GET /intent/journal` (read) is unchanged** and will still return the legacy rows with their
+unread fields. Nothing marks those fields as unread on read.
+
+## 3. Level-of-abstraction fit
+
+The validator is pure — no `fs`, no clock, no config, no server import — so the refusal is unit
+testable without a server, and the route owns transport only. This mirrors the structure used one
+feature earlier and, more importantly, is what made REFUSAL 1 possible to demonstrate: a pure
+validator can be perfect while nothing calls it, which is precisely the failure mode being guarded.
+
+The counters live on `stats()` rather than in a new surface, because the ambiguity being fixed is a
+property of that existing return value. A new endpoint would have left the misleading one in place.
+
+## 4. Signal vs authority compliance
+
+This **is** an authority — it blocks a write — so `docs/signal-vs-authority.md` applies directly
+rather than trivially. It qualifies because the logic is deterministic and total: an allowlist
+comparison and a set of required-key checks, no heuristics, no model, no inference about content.
+That is the category the principle permits to hold blocking authority. Every uncertain input
+(`null`, `undefined`, a string, a number) resolves to a refusal with a named reason rather than a
+throw or a pass — asserted by test.
+
+The blocked path writes nothing: a refused submission leaves `count: 0`, asserted by integration
+test. A refusal that still recorded the row would be worse than no refusal, because the journal
+would carry entries the caller was told were rejected.
+
+## 4b. Judgment-point check (Judgment Within Floors standard)
+
+None introduced. Every branch is a deterministic key or type comparison.
+
+## 5. Interactions
+
+- **`DispatchDecisionJournal` / `AutoDispatcher`** — deliberately NOT gated (see §1). Asserted.
+- **`IntentDriftDetector`** — reads the journal; gains higher-quality input (entries now carry
+  `principle`) and is otherwise untouched. Its 16 tests pass unchanged.
+- **`instar intent reflect` / `commands/intent.ts`** — calls `journal.log()` directly, module path,
+  unaffected by the route gate.
+- **`tests/unit/DecisionJournal.test.ts`** — one test pinned the exact empty-`stats()` object shape
+  and now asserts the two new counters. This is a shape assertion updated to a new shape, not a
+  behavioural assertion weakened; the comment records why the zero values matter.
+- **`CapabilityIndex`** — no change needed. `intent` is already classified under
+  `INTERNAL_PREFIXES` ("surfaced inside `evolution` subsystems"); this adds no new route prefix.
+
+## 6. External surfaces
+
+`POST /intent/journal` changes response behaviour: submissions that previously returned 201 may now
+return 400 with `{ error, reason, unknownFields, missingFields }`. `GET /intent/journal/stats` gains
+two fields (additive). No config key, no new route, no persisted-state migration, no message to any
+user. No credentials or content beyond what the caller submitted appear in any response.
+
+## 6b. Operator-surface quality
+
+The refusal message names the offending fields AND the correct destination (`context` for reasoning,
+`principle` for guiding intent) plus the full writable-field list. A refusal that only says "invalid"
+relocates the failure rather than fixing it; asserted by a test that the message mentions `context`.
+
+## 7. Multi-machine posture (Cross-Machine Coherence)
+
+**Machine-local BY DESIGN.** The journal is a per-machine JSONL under `stateDir`; the validator is
+pure and stateless, so the refusal is identical on every machine with no coordination required.
+There is no replication, no lease interaction, no generated URL, and no cross-machine read. Honest
+limitation: an agent running on two machines keeps two separate decision journals, so
+`principledCount` answers "on this machine". That predates this change and is not addressed here.
+<!-- tracked: CMT-1044 -->
+
+## 8. Rollback cost
+
+Low. One pure function, two counters, one route guard, plus doc/migration lines. No persisted-state
+change and no data migration — existing rows are read unchanged. Reverting restores the permissive
+route; already-written entries remain valid under both versions.
+
+## Phase 5 — Second-pass review
+
+This change **does** hold block/allow authority on a write path, so the high-risk trigger list is
+engaged. It does not touch session lifecycle, messaging, dispatch, trust levels, or recovery. Author
+lenses, disclosed:
+
+**Adversarial — "how would I make this useless?"** Three ways, all closed and asserted: let the
+route skip the validator (REFUSAL 1 — the real one); accept unknown keys silently (REFUSAL 2); drop
+the principle requirement (REFUSAL 3). A fourth — refuse but write the row anyway — is closed by the
+`a refused submission writes NOTHING` test.
+
+**"Would it have caught the incident?"** Yes, on the first attempt, which is the strongest thing I
+can say for it. My literal opening submission is now a test fixture and returns 400 naming
+`checkedAgainst` and `reasoning`. I would have lost seconds instead of finding out by reading my own
+file afterwards and having already told my operator otherwise.
+
+**"Symptom or cause?"** Partly cause, and I want this stated plainly rather than implied: it makes
+an unreasoned decision unrecordable, which is a real structural gate. It does not make an unreasoned
+decision unmakeable. The operator asked for decisions checked against the hierarchy; this delivers
+the enforcement half of "log the reasoning" and none of the consult half.
+
+**Weakest point:** the requirement is satisfiable by any non-empty string. Nothing distinguishes a
+genuinely-consulted principle from a plausible-sounding one typed to clear the gate — and the agent
+clearing it is the same one whose unreliability motivated the gate. That limit is inherent to a
+deterministic check and is the reason the consult-side trigger is not optional future work.