instar 1.3.1304 → 1.3.1306

package/upgrades/1.3.1306.md

@@ -0,0 +1,125 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: patch -->
+
+## What Changed
+
+The operated feedback drain refuses every tick with 403 "current registered
+readiness agent required" until an operator registers the readiness
+authority, which is the record that says which model may decide which
+feedback reports become work items. Registering was possible only through
+the PIN-gated `POST /feedback-factory/readiness-authorities` with about a
+dozen technical fields, and there was no screen for it. On the Mac Studio
+this left 1,001 reports waiting.
+
+- New read-only `GET /feedback-factory/readiness-authorities/proposal`
+  (Bearer). The server computes every field the runtime later compares: the
+  agent id (`projectName`, which the drain job sends), the drain's owner
+  machine and epoch, the provider and model that the intelligence router
+  would actually pick for the readiness call, and the prompt, schema and
+  decision-point ids. Only the envelope can be edited: batch size (default
+  50), daily spend cap (default $5), and token limit (default 1200). The
+  response also carries the status (none / active / proposal-only /
+  revoked), any plain-language blockers, and which action an Approve tap
+  would perform.
+- The existing PIN route now accepts `useProposal: true` for create and
+  replace. When it is set, the server fills the binding fields from its own
+  proposal and takes only the envelope from the request. It stays PIN-only
+  and same-origin, and the agent still cannot register itself.
+- New `IntelligenceRouter.previewPrimary()`, a read-only preview of the
+  primary (framework, model) that `evaluate()` would choose. It follows the
+  same enforced-nature or category logic.
+- New dashboard card on the **Feedback Drain** tab: "Who decides which
+  feedback becomes work". It shows a one-line status, a plain sentence
+  ("…up to 50 reports per batch, at most $5 per day. Anything outside that
+  comes to you."), an "Adjust limits" drawer, a PIN box, and Approve (or
+  Restore / Save new limits) and Revoke buttons.
+- CLAUDE.md template and migration: new "Feedback Readiness Authority
+  (operator approval)" section, also mirrored to Codex/Gemini agents.
+
+The worktree monitor posted the same two notices ("Worktree activity detected
+after session …" and "Stale worktrees detected") after almost every job
+session and every five-minute scan. On 29 September that was 549 of 724
+outbound messages, all about the same unmerged branches. It now remembers the
+set of branches it last announced for each notice and posts only when that set
+changes: a branch appears, or one resolves. A set that stays open is repeated
+once a day. The remembered set is saved in
+`state/worktree-monitor/announced.json`, so a restart does not post it again.
+A failed send is not recorded, so the next scan retries it.
+
+The topic-intent capture loop no longer sends `provenance: automation`
+messages to the extractor LLM. That covers scheduler job sends and monitor,
+sentinel and system notices. Those messages used about 64% of the extractor's
+input tokens, and most of what they stored was repeated "unmerged branches"
+facts. The check uses the structural provenance stamp set when a message is
+sent, never the message text. User and agent conversation is extracted as
+before, and so are unstamped legacy rows. The rolling topic summary still
+includes automated messages, so the next conversational extraction still sees
+them as context.
+
+## What to Tell Your User
+
+The feedback drain has been waiting for your go-ahead before it sorts
+reports into work items. You can now give it from your phone: open the
+dashboard, go to the Feedback Drain tab, and find the card "Who decides
+which feedback becomes work". It says in one sentence what you're agreeing
+to (by default, up to 50 reports per batch and at most $5 a day). Enter
+your PIN and tap Approve. You can revoke it from the same card at any time.
+I can't approve it myself.
+
+You'll see far fewer worktree reminders. A notice now means something changed,
+and a branch that is still open gets one reminder a day.
+
+## Summary of New Capabilities
+
+- Dashboard → Feedback Drain → "Who decides which feedback becomes work":
+  approve, change limits, restore or revoke the readiness authority with
+  the PIN.
+- `GET /feedback-factory/readiness-authorities/proposal`: the read-only
+  answer to "what exactly would be approved, and why can't it be yet?"
+
+- Worktree notices are sent only when the flagged set changes, with one reminder a day; `/hooks/worktrees/last-report` shows when an unchanged finding was suppressed.
+- Automated messages are no longer run through topic-intent extraction.
+
+## Evidence
+
+- `tests/integration/feedback-readiness-authority-routes.test.ts`, run
+  against a real AgentServer over HTTP. The test reads the proposal, checks
+  that a tick 403s, and checks that Bearer without a PIN and a wrong PIN are
+  both refused. It then approves with the PIN while also sending
+  attacker-supplied `agentId`/`provider`, which are ignored. The next tick
+  returns 202 and the run succeeds. The arbiter is called once and its
+  model check passes, so the authority stays `active`. It then revokes,
+  confirms the tick 403s again, and confirms an out-of-range batch is
+  refused.
+- `tests/unit/intelligence-router-preview-primary.test.ts`: the preview
+  equals what the real `evaluate()` reports through `onModel` in four modes:
+  unconfigured, category override, enforced nature plan, and dryRun nature
+  plan. It returns null when the routed framework is unavailable.
+- `tests/unit/feedback-readiness-authority-proposal.test.ts` covers every
+  status→action branch, the blockers and the envelope bounds. It also
+  replays real recorded shapes from the Mac Studio's `feature-metrics.db`
+  (221 codex-cli `gpt-6-astra` `capable` calls). An authority built from the
+  proposal passes the real arbiter on that shape and is refused on the
+  `gpt-5.6-sol` retirement-fallback shape.
+- `tests/unit/feedback-readiness-authority-ui.test.ts`: covers the card's
+  buttons per state, the request bodies (no binding fields are sent), the
+  PIN and envelope submitted on Approve, and that values are rendered with
+  textContent only. It also checks that the page mounts the card, imports
+  the module, and renders the card on tab open but not on the 15-second
+  poll.
+- `tests/e2e/feedback-factory-drain-lifecycle.test.ts`: on the production
+  init path the route answers 200, tracks the post-failover owner epoch,
+  offers no Approve when no routed model exists, and refuses the new path
+  without a PIN.
+
+- A replay of the real 29 September message log through the new
+  `captureTurn`: 549 worktree notices and 64 other automated rows skipped; all
+  111 agent and 19 user rows extracted. That is 130 extractor calls instead of
+  743.
+- A replay of the same day's worktree notice texts under the new rule: 11 of
+  549 would be sent, each because the flagged set changed.
+- Unit, integration (`/hooks/worktrees/last-report`, `/telegram/reply` →
+  capture metrics) and e2e tests cover both sides. The new tests fail when
+  the new logic is disabled.

package/upgrades/side-effects/feedback-readiness-authority-card.md

@@ -0,0 +1,105 @@
+# Side-Effects Review — Readiness authority approval from the dashboard
+
+**Version / slug:** `feedback-readiness-authority-card`
+**Date:** `2026-09-30`
+**Author:** `Echo`
+**Second-pass reviewer:** `independent reviewer subagent (see below)`
+
+## Summary of the change
+
+The operated feedback drain on the Mac Studio is live-healthy with 1,001 unprocessed reports, but every tick 403s ("current registered readiness agent required"). The readiness authority can only be registered through the PIN-gated `POST /feedback-factory/readiness-authorities`, which has no human surface and needs a dozen technical fields. The operator (Justin, topic 95267, 2026-09-30 10:26 PDT) approved registering it with maxBatch 50 and maxDailySpendUsd 5. This change adds the operator surface:
+
+- `src/feedback-factory/drain/readinessAuthorityProposal.ts` (new, pure): builds the record the runtime will accept. It also returns status, blockers, the Approve action and a plain summary.
+- `src/core/IntelligenceRouter.ts`: new read-only `previewPrimary()` that mirrors `evaluateRouted`'s primary selection.
+- `src/server/routes.ts`: new `GET /feedback-factory/readiness-authorities/proposal` (Bearer, read-only); `useProposal: true` on the existing PIN route makes the server fill the binding fields.
+- `src/server/AgentServer.ts`: exposes `authorityBinding()`, the owner machine and epoch that `canAgentMutateReadiness` compares.
+- `dashboard/feedback-readiness-authority.js` (new) + `dashboard/index.html`: the card on the Feedback Drain tab.
+- CLAUDE.md template, migration, and shadow marker for Agent Awareness.
+
+## Decision-point inventory
+
+- `POST /feedback-factory/readiness-authorities` authorization (PIN + X-Instar-Request + same-origin): **pass-through**. It is unchanged and still runs before any new code.
+- `FeedbackDrainStore.mutateAuthority` validation: **pass-through**. The proposal mirrors its envelope bounds for display, and the store remains the authority.
+- `FeedbackDrainService.canAgentMutateReadiness` and `FeedbackReadinessArbiter.decideBatch` provider/model/prompt checks: **pass-through**. Nothing loosens them; the proposal computes values that satisfy them.
+
+---
+
+## 1. Over-block
+
+No new block/allow surface. The proposal's `blockers` only decide whether the card offers Approve, and the PIN route re-checks the same conditions and refuses with 409 when `useProposal` cannot build a record. One case: an operator who deliberately wants to register while no routed model exists (for example, before enabling a framework) cannot use the card. The raw-field PIN route still works for that expert path.
+
+## 2. Under-block
+
+- The proposal predicts the primary route. If the call later lands elsewhere (the Codex retirement self-heal reports `gpt-5.6-sol`; a failure swap to another framework), the arbiter's existing check refuses and demotes the authority to `proposal-only`. The card then shows "Paused by a safety brake" and offers Approve (replace). This is the designed fail-safe direction and is replayed in `feedback-readiness-authority-proposal.test.ts`. The operator may need to approve again after a model retirement.
+- Found during this build and not fixed here: the live `Config.ts` loader drops `sessions.natureRouting`, so the Studio runs nature routing observe-only even though its config says `dryRun:false`. `previewPrimary` reads the same runtime closure, so the proposal matches the real call (codex-cli / `gpt-6-astra`). If that loader bug is fixed, the next tick after a routing change demotes the authority, and the card offers a re-approval bound to the new route. Fixing the loader would make MessageSentinel's FAST chain fail closed (pi-cli is not enabled), so it is a separate decision. It is reported to the operator in the PR summary.
+
+## 3. Level-of-abstraction fit
+
+This is the right layer. The server already owns the facts: the owner binding in AgentServer, routing in IntelligenceRouter, and the prompt/schema ids in the arbiter module. Computing them server-side and letting the operator choose only the envelope keeps the human decision where it belongs. `previewPrimary` sits on the router beside `for()` (the existing diagnostic resolver) instead of re-implementing routing in the route handler.
+
+## 4. Signal vs authority compliance
+
+- [x] No — this change has no block/allow surface.
+
+The proposal is advisory display. Authority stays with the PIN (human), the store's validation, and the arbiter's runtime checks, none of which changed.
+
+## 4b. Judgment-point check
+
+No new static heuristic at a competing-signals decision point. The status→action mapping follows the store's own state machine: a revoked authority must be restored before it can be replaced, and create is refused if a record already exists.
+
+## 5. Interactions
+
+- **Shadowing:** `useProposal` is read only after `feedbackMutationIntentValid` passes. Revoke and restore ignore it and reuse the stored record as before.
+- **Double-fire:** the card renders on tab open and after an action, not on the 15-second status poll, so a PIN being typed is never wiped. The status poll is unchanged.
+- **Races:** two approvals at once are serialized by the store's `.immediate()` transaction. The second create gets 409 "authority already exists", which the card shows verbatim.
+- **Feedback loops:** none. The proposal reads state and writes nothing.
+
+## 6. External surfaces
+
+- **Operator surface (Mobile-Complete):** this change adds the missing phone surface. It works from the tunnel URL with the dashboard PIN, and the POST is same-origin.
+- Other agents and users: the new route is Bearer-only and read-only. Without the drain (fleet default) it returns 503, like its siblings.
+- Persistent state: nothing new. Approvals write the same `authority_records` and `authority_audit` rows as before, with an `operatorDecisionRef` of the form `dashboard:<timestamp>:<random>`.
+
+## 6b. Operator-surface quality
+
+1. **Leads with the primary action?** Yes. The card shows its title, a one-line status, the plain sentence of what is being approved, the PIN box, then Approve. The numeric limits sit in a collapsed "Adjust limits" drawer because the approved defaults (50 / $5) are prefilled.
+2. **Zero raw internals as primary content?** Yes. The machine id, epoch and model appear only as a muted "Bound to:" line inside the collapsed drawer. Statuses read "Not set up yet", "Active (version 1): up to 50 reports per batch…", "Paused by a safety brake", "Revoked". No enum values are shown.
+3. **Destructive actions de-emphasized?** Yes. Revoke is a plain button after the green Approve / Save new limits, never above it, and appears only when there is something to revoke.
+4. **Plain language + phone width?** Plain language is checked in the UI tests. The card reuses the Spend tab's mobile-first classes: 44px tap targets, 16px inputs (no iOS zoom), full-width stacked fields. I did not take a separate phone-width screenshot of this card.
+
+## 7. Multi-machine posture
+
+**Machine-local by design, proxied on the owner.** The authority binds to the drain's canonical owner machine and epoch, and `authorityBinding()` reports this server's view: the configured `operatedHostMachineId` and the coordinator lease epoch. On a non-owner machine, `ownerMachineId` is still the configured owner, so a record approved there binds to the owner. The store is per machine, though, so the approval must be made on the owner's dashboard; the tick proxy already routes ticks to the owner. The card emits no user notices and generates no URLs. Its only durable state is the existing authority tables, which are already in the drain backup/restore path.
+
+## 8. Rollback cost
+
+This is a pure code change. Reverting removes the card, the proposal route, `useProposal` and `previewPrimary`. Records approved through the card are ordinary authority generations and keep working, and they can be revoked through the raw PIN route. No migration is needed. The CLAUDE.md section would remain in existing agents' files, pointing at a card that no longer exists, until a follow-up migration removes it by the same marker.
+
+## Conclusion
+
+The review changed the design in three ways. Approval uses `useProposal` (server-derived fields) rather than posting client-side copies of technical fields. The card renders outside the 15-second poll. The model binding comes from a router preview that is proven equal to the real `evaluate()` selection, rather than from the static routing map. The static map would have proposed `gpt-5.5`, a retired id that the live call never uses, and the first tick would have demoted the authority. The change is clear to ship. The natureRouting loader drop is flagged to the operator as a separate finding.
+
+---
+
+## Second-pass review (if required)
+
+**Reviewer:** independent reviewer subagent
+**Independent read of the artifact: concur**
+
+The reviewer found no path for a Bearer-only caller to register, widen or restore. `useProposal` is read only after the PIN, origin and header checks pass, and the store's envelope bounds still apply. They confirmed that `previewPrimary` matches `evaluateRouted` on the enforced, fail-closed, no-route and category paths. They also confirmed that the binding equals what `canAgentMutateReadiness` and the tick job compare. They raised three minor points:
+- Restore ignores the edited limits. Fixed: the card now says Restore brings back the last approved limits, and a UI test covers it.
+- An emptied limit field turned into 0. Fixed: empty fields are omitted, so the server default applies (JS and GET), with a UI test.
+- A `claude-code` subscription-path agent reports `interactive-pool` as its model, so the arbiter would pause the authority (a false pause, fail-safe). This is noted here, and it does not affect the Studio route.
+
+---
+
+## Evidence pointers
+
+- `tests/integration/feedback-readiness-authority-routes.test.ts`: before approval the tick 403s; after the PIN approval it returns 202 with a succeeded run and the authority stays active.
+- `tests/unit/intelligence-router-preview-primary.test.ts`: the preview equals the real `onModel` in four routing modes.
+- `tests/unit/feedback-readiness-authority-proposal.test.ts`: replays the recorded live shape codex-cli/`gpt-6-astra` (Mac Studio `.instar/server-data/feature-metrics.db`, 221 rows) and the `gpt-5.6-sol` retirement-fallback shape.
+- `tests/e2e/feedback-factory-drain-lifecycle.test.ts`: the route is alive on the production init path.
+
+## Class-Closure Declaration (display-only mirror)
+
+No agent-authored-artifact defect — not applicable. No self-triggered controller is added.

package/upgrades/side-effects/worktree-monitor-dedupe.md

@@ -0,0 +1,133 @@
+# Side-Effects Review — Worktree notices on change; automated sends skip intent extraction
+
+**Version / slug:** `worktree-monitor-dedupe`
+**Date:** `2026-09-30`
+**Author:** `Echo`
+**Second-pass reviewer:** `not required`
+
+## Summary of the change
+
+Two small changes, both driven by offline measurements on 29 Sep traffic
+(docs/research/jev/field-notes/2026-09-30-idea1-intent-skip.md and
+2026-09-30-idea4-notification-tiering.md, in the agent home). (1)
+`src/monitoring/WorktreeMonitor.ts`: each notice kind (`session` = post-session
+unmerged/orphan notice, `stale` = periodic stale-worktree notice) keeps a
+fingerprint of the item set it last announced, persisted in
+`state/worktree-monitor/announced.json`. A notice is sent only when the set
+differs, or when `reminderIntervalMs` (default 24h) has passed. An empty set
+clears the fingerprint, and a failed send is not recorded. (2)
+`src/core/TopicIntentCapture.ts`: `captureTurn` returns `skipped-automation`
+for `provenance === 'automation'` before any LLM call, counted under
+`prefilter_skipped`. `src/commands/server.ts` forwards `entry.provenance` from
+`onMessageLogged` into the loop. `src/scaffold/templates.ts` and
+`src/core/PostUpdateMigrator.ts` add a short "Worktree Notices" CLAUDE.md
+section (template, add-if-absent migration, and framework-shadow marker).
+
+## Decision-point inventory
+
+- `WorktreeMonitor.onSessionComplete` / `periodicScan`: whether to send a notice — **modify**. It was "any finding → send". It is now "finding set changed, or 24h since last announcement → send".
+- `captureTurn` pre-filter: whether a turn reaches the extractor LLM — **modify**. It adds a structural skip for `automation` provenance.
+- `migrateClaudeMd`: whether to add the section — **add**. It is content-sniffed on the heading.
+
+---
+
+## 1. Over-block
+
+- Worktree: a finding whose *details* change while the branch set does not (for example, more commits on the same unmerged branch) is not re-announced until the daily reminder. This is intentional. The branch was already flagged, and commit-count churn was part of the spam.
+- Intent: an automated message that carries a real decision (the idea-1 note found a few, such as "framework switch deferred until the build is idle") is no longer extracted directly. Mitigation: TopicSummarizer builds the rolling summary from every message regardless of provenance (`getMessagesSinceSummary` has no provenance filter), and that summary is passed to the next conversational extraction as context. Justin's own messages are stamped `user` and are never skipped.
+
+---
+
+## 2. Under-block
+
+- A set that flaps (a branch that keeps being created and deleted) re-announces on each change. That is correct, since each flap is a real change, but it is not rate-limited beyond that.
+- Automated sends that reach the log without a provenance stamp (legacy rows, any path that does not stamp) are still extracted. This fails open on purpose.
+- Other LLM consumers of the same messages (Usher, ArcCheck) are unchanged. They are outside this change.
+
+---
+
+## 3. Level-of-abstraction fit
+
+The dedupe sits at the source, the monitor that decides to speak, rather than in NotificationBatcher, because only the monitor knows the item set. The intent skip uses the existing structural provenance authority, stamped at the send seam (`/telegram/reply` derives it from messageKind/isSystemTemplate/proxy, and `sendToTopic` defaults to `automation`), rather than re-deriving automation from text.
+
+---
+
+## 4. Signal vs authority compliance
+
+**Required reference:** [docs/signal-vs-authority.md](../../docs/signal-vs-authority.md)
+
+- [x] No — this change has no block/allow surface.
+
+Neither change blocks a user or agent action. The monitor chooses whether to emit its own informational notice. The capture loop chooses whether to spend an LLM call on a best-effort background extraction, and it was already fail-open and never on the delivery path.
+
+---
+
+## 4b. Judgment-point check (Judgment Within Floors standard)
+
+No new static heuristic at a competing-signals decision point. "Has the flagged set changed since the last announcement" is an enumerable set comparison. "Is this message automation-provenance" reads an existing structural stamp and does not weigh competing live signals.
+
+---
+
+## 5. Interactions
+
+- **Shadowing:** the provenance skip runs before `isSubstantiveTurn`. Both skips count as `prefilter_skipped`, and the new status `skipped-automation` distinguishes them for callers. User turns still bump the turn counter first, because the provenance check comes after `bumpTurn`, and automation is never `fromUser`.
+- **Double-fire:** the post-session and periodic notices keep separate fingerprints, as they did before. The dedupe cannot suppress one because of the other.
+- **Races:** `onSessionComplete` and `periodicScan` can overlap. The worst case is one duplicate notice in the overlap window: both see "due" before either records. The state write is synchronous and last-writer-wins, and both write the same fingerprint.
+- **Feedback loops:** this removes one. Worktree notices fed the extractor, which filed "unmerged branches" facts into the intent store on every notice.
+
+---
+
+## 6. External surfaces
+
+- Telegram: far fewer worktree notices in the attention/system topic (replay of 29 Sep notices: 549 → 11).
+- Persistent state: a new small file `state/worktree-monitor/announced.json`, holding at most two entries.
+- CLAUDE.md: a new "Worktree Notices" section for new and existing agents, plus the Codex/Gemini shadow marker.
+- Operator surface: no operator-facing actions.
+
+---
+
+## 6b. Operator-surface quality (Operator-Surface Quality standard)
+
+No operator surface — not applicable.
+
+---
+
+## 7. Multi-machine posture (Cross-Machine Coherence)
+
+**Machine-local by design.** The worktree monitor scans the local git checkout. Worktrees are per-machine filesystem state, so the announced-set memory belongs with the scan that produced it. Two machines with different worktrees correctly announce their own findings. This change does not alter the existing notice routing, including one-voice gating. It only reduces how often the notice fires. The intent skip is a pure function of each logged row, so its behaviour is identical on every machine.
+
+---
+
+## 8. Rollback cost
+
+Pure code change. Revert and ship a patch. `announced.json` is ignored by older code and harmless to leave. After rollback, the old behaviour (a notice every session) returns immediately. The CLAUDE.md section would stay on migrated agents but describes behaviour that would no longer hold. A rollback should drop it with a follow-up migration.
+
+---
+
+## Conclusion
+
+Both changes are deterministic and reduce volume. They were measured on real 29 Sep data before building. The review added two design points: a failed send is not recorded (otherwise one Telegram outage would suppress the finding for a day), and an empty set clears the fingerprint (so a finding that resolves and later returns is announced). Clear to ship.
+
+---
+
+## Second-pass review (if required)
+
+Not required (Tier 1).
+
+---
+
+## Evidence pointers
+
+- Unit: `tests/unit/worktree-monitor.test.ts` (announce on change only: 8 cases), `tests/unit/TopicIntentCapture.test.ts` (automation skipped; agent/user/unstamped captured), `tests/unit/PostUpdateMigrator-worktreeNotices.test.ts`.
+- Integration: `tests/integration/worktree-monitor-announce-routes.test.ts`, `tests/integration/topic-intent-capture-automation-skip.test.ts`.
+- E2E: `tests/e2e/worktree-monitor-announce-lifecycle.test.ts` (10 sessions → 1 notice; restart; changed set), `tests/e2e/topic-intent-capture-lifecycle.test.ts`.
+- Wrong-side proof: with the new logic disabled, all new integration/e2e assertions fail (8 failures).
+- Real-shape replay (read-only, live telegram-messages.jsonl, 29 Sep): 743 rows → 130 extractor calls; automation skipped (e.g. ids 115564, 115575, 115608), agent captured (115622, 115667), user captured (115774, 115822).
+
+---
+
+## Class-Closure Declaration (display-only mirror)
+
+- **`defectClass`**: `unbounded-self-action`
+- **`closure`**: `n/a`
+- **Reason**: this change adds a settling brake to an existing self-triggered notify. It does not add a new action. Under a sustained, unchanged finding, the notice rate falls from one per session or scan (unbounded; 549 a day observed) to at most one per `reminderIntervalMs` (24h) per notice kind. A change in the set re-arms it once. `tests/e2e/worktree-monitor-announce-lifecycle.test.ts` proves 10 sessions → 1 notice.