instar 1.3.423 → 1.3.424

package/upgrades/1.3.424.md

@@ -0,0 +1,28 @@
+# Upgrade Guide — vNEXT
+
+<!-- assembled-by: assemble-next-md -->
+<!-- bump: minor -->
+
+## What Changed
+
+The per-feature LLM metrics ledger now records, for every call the system makes on its own behalf (sentinels, gates, reflectors, background checks), **which provider + model actually ran it** and **whether the check acted (fired) or found nothing (noop)** — closing two long-standing blind spots: the `model` column was never populated, and the `fired` verdict was deferred to a "Phase 2" that never shipped, so codex/gemini calls were unattributable and every completed call read as a no-op. Recording is enforced at the single funnel (`CircuitBreakingIntelligenceProvider`) via two additive, optional seams on `IntelligenceOptions` — `onModel` (every provider surfaces its resolved model/framework, independent of token usage) and `classifyVerdict` (the caller classifies act-vs-no-act). A new **LLM Activity** dashboard tab renders it in plain language over a 24h/7d/30d window. Bounded retention (default 30 days, `monitoring.featureMetrics.retentionDays`) ages the audit trail out so it is never hoarded forever.
+
+## What to Tell Your User
+
+You can now audit what your agent's "autopilot" is actually doing: open the **LLM Activity** dashboard tab to see, per check, which AI provider ran it, how often it acted on something vs. found nothing, how often it was skipped to save rate limits, and what it cost — over time. This is the new *Observable Intelligence* standard: no autonomous AI action the system takes is allowed to be invisible. Nothing for you to do; it takes effect on the next restart.
+
+## Summary of New Capabilities
+
+- `/metrics/features` rows now include `frameworks[]`, `models[]`, real `fired`/`fireRate`, and `shed` — answering "which provider ran this sentinel?" and "is it doing real work or just being skipped?".
+- New **LLM Activity** dashboard tab (read-only) over a selectable window.
+- `IntelligenceOptions.onModel` + `classifyVerdict` seams; every provider (Claude/Codex/Gemini/Pi/InteractivePool) surfaces its model/framework; `MessageSentinel` + `CommitmentSentinel` classify their verdicts.
+- Bounded retention via `FeatureMetricsLedger.pruneOlderThan` + `monitoring.featureMetrics.retentionDays` (default 30).
+- New constitutional standard *Observable Intelligence — No Autonomous LLM Action Is Unauditable* (proposed, pending ratification) + spec `docs/specs/observable-intelligence.md`.
+
+## Evidence
+
+`tests/unit/FeatureMetricsLedger.test.ts` (12, +4: framework rollup, prune, idempotent column add), `tests/unit/CircuitBreaking-feature-metrics-tap.test.ts` (19, +7: onModel/classifyVerdict/error-path attribution + real-ledger rollup), `tests/integration/metrics-features-routes.test.ts` (+1: provider/model/fired through the route), `tests/e2e/metrics-features-lifecycle.test.ts` green; no-silent-fallbacks ratchet + feature-delivery-completeness parity green; `tsc` clean.
+
+## Scope (honest)
+
+Provider/model attribution + the fired/noop verdict are recorded for all providers. Per-call **token cost** is captured where the provider surfaces it (Claude, Pi); codex/gemini exec output carries no usage block, so their per-call tokens stay null (model/framework + outcome are still recorded; account-level codex cost is visible via `/codex/usage`) — per-call token parsing for those via their `--json` modes is a bounded follow-up. Read-only observability throughout: it never gates, blocks, or mutates the path it observes.

package/upgrades/eli16/observable-intelligence-llm-activity.md

@@ -0,0 +1,26 @@
+# Observable Intelligence — in plain language
+
+## The problem
+
+Your agent has a bunch of little "autopilot" helpers running in the background — the things we call sentinels and gates. They watch your messages, decide whether something needs action, and quietly handle housekeeping. Each one can ask an AI model a quick question to make its call.
+
+Someone asked a simple, fair question: *"Which AI provider are these helpers actually using, and are they even doing any real work?"* And the honest answer was: **we couldn't fully tell.** The log that's supposed to track this had two holes in it:
+
+1. It never wrote down **which AI** ran each check. So if you'd switched your sentinels over to a different provider (say, Codex instead of Claude), the log couldn't prove it.
+2. It never wrote down whether a check **actually did something** versus just looked and found nothing. So every check looked identical in the data — you couldn't tell a hard-working guard from a useless one.
+
+A helper that acts on your behalf but can't show what it chose to do is a helper you can't hold accountable. That's the gap this closes.
+
+## What changed
+
+Now, every time one of these helpers asks an AI a question, the system writes down: **which provider and model answered, whether the helper acted or found nothing, whether it got skipped to save on rate limits, how much it cost, and how long it took.** It records this at one shared "chokepoint" that every helper already passes through — so it covers all of them automatically, today and in the future, with nothing to remember.
+
+There's a new **"LLM Activity" tab** in your dashboard that shows all of this in plain English: one row per helper, over a window you pick (last day, week, or month). You can finally see, at a glance, which helpers earn their keep and which are mostly idle or skipped.
+
+## The balance
+
+We don't keep this log forever — that would just be hoarding. It's kept about a month (you can change that), then old entries age out. Long enough to spot trends, not so long it piles up.
+
+## Is it risky?
+
+No. This only **watches** — it never blocks, changes, or slows down anything the helpers do. If the new tracking ever hiccupped, the helper would carry on exactly as before. It's all extra, additive recording, fully covered by tests.

package/upgrades/side-effects/observable-intelligence-llm-activity.md

@@ -0,0 +1,25 @@
+# Side-effects review — Observable Intelligence / LLM Activity
+
+**Tier:** 1 (large but purely additive, read-only observability; no existing behavior changes; reversible; full 3-tier tests). Risk floor flags "new capability" — declared below floor with this rationale, recorded in the decision audit.
+
+## Surface touched
+
+- `IntelligenceOptions` gains two **optional** callbacks (`onModel`, `classifyVerdict`). Additive: every existing caller is byte-identical (evaluate() still returns `Promise<string>`).
+- `CircuitBreakingIntelligenceProvider` (the single funnel) captures model/framework + classifies the verdict and records them. Pure side-channel — wrapped in try/catch; a throw in either callback cannot change what `evaluate()` returns or break the observed path.
+- Five providers (Claude/Codex/Gemini/Pi/InteractivePool) each call `onModel` once per call. Each call is try/caught (`@silent-fallback-ok`) so it can never break the LLM path.
+- Two sentinels (MessageSentinel, CommitmentSentinel) pass a `classifyVerdict`; the classification reuses their existing parse and defaults to noop on throw.
+- `FeatureMetricsLedger`: new `framework` column via idempotent pragma-guarded `ALTER TABLE ADD COLUMN` (existing DBs migrate at open, no data loss); new `pruneOlderThan`; rollup gains `frameworks[]`/`models[]`.
+- `AgentServer`: a retention prune timer (boot + every 6h, `unref`'d), cleared at shutdown.
+
+## Risks considered
+
+- **Schema migration**: covered by a unit test that opens an old-schema DB and confirms the column is added without losing the legacy row. The ALTER is pragma-guarded and idempotent.
+- **Performance**: recording is one extra SQLite insert per call (already happening) plus two cheap callback invocations. No new network calls.
+- **Retention deletion**: `pruneOlderThan` only deletes rows older than the cutoff; fail-open (a failed prune leaves rows for the next tick). Default 30d, configurable, `0` disables.
+- **Observability never gates**: same guarantee as TokenLedger. A failed metric write, failed prune, or thrown callback degrades silently; the LLM path is unaffected.
+- **No new HTTP route** (reuses `/metrics/features`), so no new auth/route surface. Dashboard tab is read-only.
+- **Migration parity**: existing agents get the enhanced CLAUDE.md section via the existing `/metrics/features` migrateClaudeMd block (enhanced in place, idempotent — verified by `PostUpdateMigrator-metricsFeatures.test.ts`); new agents via the template.
+
+## Constitutional fit
+
+Implements the proposed **Observable Intelligence** standard (pending operator ratification); balanced by Responsible-Resource bounded retention. Sharpens the existing **Observability** standard.