instar 1.3.71 → 1.3.73

package/upgrades/1.3.73.md

@@ -0,0 +1,73 @@
+# Instar Upgrade Guide — NEXT
+
+<!-- bump: patch -->
+
+## What Changed
+
+**The Usher's precision loop is now wired — `acted` can finally move off zero.**
+
+The Usher (rung 4 of continuous-working-awareness) fires "re-surface" signals
+when a faded context becomes relevant again, and exposes a precision read
+(`acted / fired`) that is the hard precondition for ever letting it interrupt
+mid-task (rung 5). But `UsherSignalStore.markActed` — the precision *numerator* —
+had **no caller anywhere**. So `acted` stayed 0 on every topic and precision was
+structurally pinned at 0: the gate could never be satisfied by data. The Usher's
+accuracy half had shipped asleep — the exact "shipped but asleep" trap the rest
+of this project exists to kill, hiding inside our own accuracy meter.
+
+This closes the loop with two correlation paths, both reducing to "does some
+probe text COVER a fired signal's proposition?":
+
+1. **Auto-use (path a).** When the agent's genuine reply goes out on a topic and
+   it actually *uses* a re-surfaced context the Usher flagged, that signal is
+   marked acted (`via: 'use'`). Wired on the outbound `/telegram/reply` path.
+2. **Miss-map (path b).** When the user has to *correct* the agent (a
+   HumanAsDetector signal) on a context a recent nudge had already flagged, that
+   nudge was a genuine catch the agent ignored — still a true positive. It's
+   marked acted (`via: 'miss'`). Wired right after the inbound human-detector
+   observe seam.
+
+Matching is deliberately precision-over-recall (≥2 shared salient terms AND ≥50%
+coverage, within a recency window): a falsely-*high* precision is the dangerous
+direction since it gates interruption, so under-crediting a fast reply is
+acceptable, inflating the gate is not. Everything is best-effort and never throws
+into the message/delivery path.
+
+`GET /usher/metrics` now also reports `acted_by_use` and `acted_by_miss` so the
+precision numerator is visible split by which path confirmed usefulness.
+
+## What to Tell Your User
+
+The Usher now measures whether its mid-task reminders were actually useful — on
+its own, two ways: when my next reply uses a reminder, and when you later have to
+correct me on something a reminder had already flagged. That useful rate is the
+number that has to look good before the Usher ever earns the right to interrupt me
+(the final rung), so instead of sitting stuck at zero it can now gather real
+numbers as we work. Nothing for you to set up — it just runs, and I can show you
+the per-conversation numbers whenever you want them.
+
+## Summary of New Capabilities
+
+| Capability | How to Use |
+|-----------|-----------|
+| Usher precision numerator wired | Automatic — `markActed` is now called from the outbound-reply path (agent used the nudge) and the inbound human-detector seam (user corrected on it). |
+| `acted_by_use` / `acted_by_miss` split | `GET /usher/metrics?topicId=N` returns the precision numerator broken down by which correlation path confirmed usefulness. |
+| `UsherActedCorrelator` | Pure, exported coverage matcher (`salientTerms`, `contextCoveredBy`, `findCoveredSignalIds`, `markActedByCoverage`) + two thin wiring wrappers (`creditUsherOnOutbound`, `creditUsherOnMiss`). Unit-tested both-sides-of-the-boundary. |
+
+## Evidence
+
+- Unit: `tests/unit/UsherActedCorrelator.test.ts` — 20 cases covering salient-term
+  extraction, coverage thresholds (covers vs single-coincidental-word vs
+  low-coverage vs empty), recency-window filtering, the via-split + backward-compat
+  on `markActed`, and the two wrappers' guards.
+- Integration: `tests/integration/usher-routes.test.ts` — `GET /usher/metrics`
+  exposes the `acted_by_use`/`acted_by_miss` split + precision; legacy topics
+  report the split as 0; source-guards prove `routes.ts` calls
+  `creditUsherOnOutbound` and `server.ts` calls `creditUsherOnMiss` after the
+  human-detector observe (anti-shipped-but-asleep).
+- E2E: `tests/e2e/usher-precision-lifecycle.test.ts` — boots the real route tree
+  on a live HTTP server, proves the pre-fix state (precision pinned at 0 with a
+  fired-but-unacted nudge), then drives BOTH paths: a reply that uses the context
+  flips precision 0 → 1.0 (`acted_by_use`), an off-topic reply does not, and an
+  inbound correction credits a prior ignored nudge (`acted_by_miss`) — all visible
+  over real HTTP at `GET /usher/metrics`.

package/upgrades/side-effects/attention-topic-flood-guard.md

@@ -0,0 +1,74 @@
+# Side-Effects Review — Attention Topic-Flood Guard
+
+**Spec:** `docs/specs/attention-topic-flood-guard.md` (converged, 4-reviewer panel)
+**Change:** per-source + global forum-topic circuit breaker at
+`TelegramAdapter.createAttentionItem` (the structural backstop). The redrive
+offender fix shipped separately in PR #495 (merged); the redrive edits originally
+bundled here were dropped to defer to it.
+**Files:** `src/messaging/AttentionTopicGuard.ts` (new),
+`src/messaging/TelegramAdapter.ts`, `src/core/PostUpdateMigrator.ts`, tests,
+`tests/unit/feature-delivery-completeness.test.ts`, `upgrades/NEXT.md`.
+
+## Phase 4 — the seven questions
+
+1. **Over-block.** The guard never blocks an agent action or drops an item — it
+   only changes *delivery form* (own topic vs. one coalesced topic + a log line)
+   for **non-critical** items. The one legitimate behavior it changes: a
+   non-critical source raising >3 items / 10 min has the surplus coalesced. That
+   is the *intended* behavior (the operator asked for exactly this) and is tunable
+   per adapter. HIGH/URGENT are never affected. No agent capability is rejected.
+
+2. **Under-block.** Before the global ceiling was added, a source varying its
+   `sourceContext` per item would have dodged the per-source budget — the review
+   caught this; the global cap + key eviction now bound it regardless of source
+   cardinality. Remaining under-block: a flood composed entirely of HIGH/URGENT
+   items is not coalesced (by design — critical items must always be visible). A
+   feature mis-marking routine noise as HIGH would still flood; that is a
+   mis-classification bug in that feature, not in the guard, and is the correct
+   place to fix it.
+
+3. **Level-of-abstraction fit.** Correct layer: the breaker sits at the single
+   chokepoint (`createAttentionItem`) where every attention item becomes a topic,
+   so it covers all current and future callers. It is deliberately *below* the
+   tone-gate authority on the `/attention` route (a transport-mechanics
+   rate-counter, not a content-judgment filter). The per-feature redrive fix
+   (PR #495) is at the feature layer; the guard is the substrate backstop — two
+   layers, by intent.
+
+4. **Signal vs authority.** Compliant. The guard holds no blocking authority over
+   agent behavior or information flow; it is a delivery *shaper* (the rate-counter
+   / transport-dedup carve-out in `docs/signal-vs-authority.md`), the same class as
+   `SentinelNotifier`. It never withholds a critical notice and never drops an
+   item.
+
+5. **Interactions.** Audited against `SentinelNotifier` (separate path → the
+   single reused system topic; no collision — it never flows through the guard) and
+   the `agent-attention-topic` (not registered in the attention maps, untouched).
+   Restart interaction (coalesced items vs. `loadAttentionItems` reverse-map
+   corruption) and concurrency (double-create race) were both found by review and
+   fixed. `updateAttentionStatus` no longer closes a shared topic when one
+   coalesced sibling resolves (coalesced items aren't in the per-item maps).
+
+6. **External surfaces.** User-visible change: a flooding source now produces one
+   "🔁 …: notices coalesced (flood guard)" topic instead of a wall; the rest goes
+   to `state/attention-suppressed.jsonl`. Fleet-wide via dist update, default-ON,
+   no config. No new network calls; no new external dependency. The CLAUDE.md
+   awareness section is backfilled to existing agents (idempotent migrator).
+
+7. **Rollback cost.** Cheap. Single config flag:
+   `messaging[].config.attentionTopicGuard.enabled = false` restores per-item-topic
+   behavior. No data migration, no agent-state repair (the guard holds only
+   in-memory counters). Worst case is a `src/` revert + patch release.
+
+## Phase 5 — second-pass review (REQUIRED: touches a "guard" + outbound messaging)
+
+An independent four-reviewer convergence panel (adversarial/security, integration,
+architecture, scalability) audited the change against the diff. It raised 7
+material concerns (global-cap dodge, `/ack`+shared-topic-close, case-sensitive
+critical bypass, double-create race, hex plaintext to unverified address, NaN
+config disable, unbounded audit log) plus framing notes. **6 of the 7 were fixed
+in code this iteration; the 7th (hex plaintext) was a finding on the redrive edits
+that are now deferred to the merged PR #495.** Full record:
+`docs/specs/attention-topic-flood-guard.convergence.md`. **Concur with the review —
+the design after this iteration addresses every raised concern; no blocking
+authority on brittle logic remains.**

package/upgrades/side-effects/usher-precision-loop.md

@@ -0,0 +1,131 @@
+# Side-Effects Review — Usher precision numerator wiring (rung 4)
+
+**Version / slug:** `usher-precision-loop`
+**Date:** `2026-05-28`
+**Author:** `echo`
+**Second-pass reviewer:** `not required` (signal-only observability change, no blocking surface)
+**Spec:** `docs/specs/cwa-usher.md` (approved:true by justin; review-note explicitly names "the usher_acted precision definition that gates rung 5" as the in-scope follow-up this change implements)
+
+## Summary of the change
+
+The Usher fires re-surface signals (`UsherSignalStore.recordSignal`) and exposes a
+precision read (`acted / fired`) that the spec designates as the hard precondition
+for rung 5 (mid-task injection). But `UsherSignalStore.markActed` — the precision
+*numerator* — had no caller anywhere, so `acted` was pinned at 0 on every topic
+and precision could never move. This wires the numerator via two correlation paths
+in a new pure helper `src/core/UsherActedCorrelator.ts`: (a) **auto-use** — when the
+agent's outbound reply on a topic uses a re-surfaced context, mark that signal
+`acted(via:'use')`, called from the `POST /telegram/reply` handler in
+`src/server/routes.ts` after a genuine (non-proxy, non-system) reply sends; (b)
+**miss-map** — when the user has to correct the agent (a `HumanAsDetector` signal)
+on a context a recent nudge flagged, mark it `acted(via:'miss')`, called right after
+the inbound `observeInboundMessage` seam in `src/commands/server.ts`. `markActed` now
+also stamps `actedVia`/`actedAt`, and `GET /usher/metrics` reports
+`acted_by_use`/`acted_by_miss`. Files: `UsherSignalStore.ts`, `UsherActedCorrelator.ts`
+(new), `usherRoutes.ts`, `routes.ts` (+`RouteContext.usherSignalStore`),
+`AgentServer.ts` (route ctx wiring), `server.ts` (path-b seam).
+
+## Decision-point inventory
+
+The only "decision" this change makes is whether to increment an observability
+counter (`acted`). It gates nothing — no message is blocked, delayed, or altered.
+
+- `UsherActedCorrelator.contextCoveredBy` — **add** — pure text-coverage match deciding whether a probe references a fired signal's proposition. Output feeds `markActed` only; no authority over delivery.
+- `routes.ts POST /telegram/reply` (path a) — **pass-through** — the existing send decision is untouched; crediting runs *after* the reply has already gone out.
+- `server.ts` inbound human-detector seam (path b) — **pass-through** — crediting runs after `observeInboundMessage`, which itself is signal-only.
+
+---
+
+## 1. Over-block
+
+**What legitimate inputs does this change reject that it shouldn't?**
+
+No block/allow surface — over-block not applicable. `markActed` only increments a
+metric; it cannot reject, delay, or modify any inbound or outbound message. Both
+call sites run strictly *after* the message has been processed/sent, and both are
+wrapped best-effort (never throw into the message path).
+
+## 2. Under-block
+
+**What failure modes does this still miss?**
+
+By design (precision-over-recall — a falsely-HIGH precision is the dangerous
+direction since precision gates rung 5), the correlator deliberately UNDER-credits:
+
+- **Path (a) timing race:** the Usher's signal is recorded by an async, LLM-backed
+  fire-and-forget loop (~8–10s). A very fast agent reply can land before the signal
+  is persisted, so that use goes uncredited. Acceptable: the full reply (after the
+  agent's work) usually lands well after the signal persists, and under-crediting
+  never inflates the gate.
+- **Short contexts:** a context with fewer than 2 salient terms can never reach the
+  `MIN_SHARED=2` threshold, so single-word propositions are never auto-credited.
+- **Recency window:** uses/corrections outside `USE_WINDOW_MS` (6h) / `MISS_WINDOW_MS`
+  (24h) are not credited.
+
+These are recall gaps, not correctness bugs — the precision number will be a *floor*
+on true usefulness, which is the conservative posture the rung-5 gate wants.
+
+## 3. Level-of-abstraction fit
+
+**Is this at the right layer?**
+
+Yes. `markActed` already existed on `UsherSignalStore` (the spec's numerator); this
+change only supplies its missing callers. The matching logic lives in a pure,
+unit-tested helper (`UsherActedCorrelator`) rather than inline at the call sites,
+and the two call sites are the two existing seams where the relevant text is already
+in hand (outbound reply text; inbound correction text + its `HumanAsDetector`
+verdict). No higher layer owns "was this nudge used" — this is the layer that has
+both the fired signals and the probe text. No smarter gate exists that should own
+it instead.
+
+## 4. Signal vs authority compliance
+
+**Does this hold blocking authority with brittle logic, or produce a signal?**
+
+Compliant (`docs/signal-vs-authority.md`). The coverage matcher is intentionally
+brittle (keyword overlap, no LLM) — and that is acceptable *precisely because it has
+zero authority*. It produces a signal (the precision numerator) and nothing else. It
+never blocks, never gates delivery, and feeds only the operator-facing
+`GET /usher/metrics`. A brittle detector with no blocking power is the correct
+pattern; the anti-pattern (brittle check WITH blocking authority) is not present.
+
+## 5. Interactions
+
+**Does it shadow another check, get shadowed, double-fire, or race?**
+
+- **Idempotency:** `markActed` guards `if (sig.acted) return false`, so paths (a) and
+  (b) marking the same signal, or repeated calls, increment `acted` at most once per
+  signal. The split counters (`acted_by_use`/`acted_by_miss`) only increment on the
+  successful (first) mark.
+- **Ordering on the inbound seam:** path (b) runs in the human-detector callback,
+  which is chained *before* the Usher loop callback on the same `onMessageLogged`.
+  It correlates the correction against signals fired on PRIOR turns (already
+  persisted) — it does not race the current turn's not-yet-fired signal.
+- **No shadowing:** the outbound tone-gate / dedup / ArcCheck path is unchanged;
+  crediting runs after the gate passed and the send succeeded, so it cannot affect
+  any block/allow outcome.
+
+## 6. External surfaces
+
+**Does it change anything visible to other agents, users, or systems?**
+
+- `GET /usher/metrics` gains two additive fields (`acted_by_use`, `acted_by_miss`)
+  and `acted`/`precision` now actually move. This endpoint is operator-only
+  (`INTERNAL_PREFIXES`), not surfaced to end users or other agents.
+- The per-topic `usher/<topicId>.json` store files gain optional `actedVia`/`actedAt`
+  on signals — additive; `load()` already tolerates unknown/missing fields and old
+  readers ignore them.
+- No change to any message content, no new outbound traffic, nothing visible to
+  peer agents. Behavior depends on conversation/topic state but is fully best-effort.
+
+## 7. Rollback cost
+
+**If this turns out wrong in production, what's the back-out?**
+
+Cheap and clean: revert the commit (a normal patch release). No data migration — the
+new optional fields are tolerated by existing `load()` and ignored by older binaries;
+a topic file written by the new code is readable by the old code and vice-versa.
+Worst case after revert: `precision` returns to its prior pinned-at-0 state (the bug
+this fixes). No agent-state repair, no hot-data fix. Because the change is
+observability-only, a wrong precision number cannot itself cause an outage — at most
+it would mislead the rung-5 decision, which is human-gated and not part of this change.