instar 1.3.486 → 1.3.487

package/upgrades/side-effects/token-audit-completeness.md

@@ -0,0 +1,84 @@
+# Side-Effects Review — Token-Audit Completeness
+
+**Version / slug:** `token-audit-completeness`
+**Date:** `2026-06-11`
+**Author:** `echo (instar-dev agent)`
+**Second-pass reviewer:** `independent reviewer subagent (Phase 5 — change touches the funnel + a sentinel-adjacent tripwire)`
+
+## Summary of the change
+
+Implements docs/specs/token-audit-completeness.md (converged r6, operator-approved): codex judgment calls switch to `codex exec --json` with a streaming spawn helper (`spawnCodexExecJson` in `codexSpawn.ts`) so per-call token usage (in/out/cached) reaches the FeatureMetricsLedger; the funnel's error path now records already-burned tokens; `/metrics/features` gains a feature×model partition (`byModel`), per-framework `usageCoverage`, and unlabeled-spend shares; every previously-unattributed funnel callsite is tagged (`attribution.component`) and a new lint (`scripts/lint-llm-attribution.js`) + empty-baseline ratchet test keeps the baseline at zero; a `codex-usage-parse-drift` degradation tripwire (once per process) catches future parse rot; `appendAuditEntry` gains 16 MB rotation (the per-call SafeFs deletions make destructive-ops.jsonl hot-path). Files: `codexSpawn.ts`, `codexUsageParser.ts` (new), `CodexCliIntelligenceProvider.ts`, `ClaudeCliIntelligenceProvider.ts`, `CircuitBreakingIntelligenceProvider.ts`, `FeatureMetricsLedger.ts`, `SafeGitExecutor.ts`, `routes.ts`, `intelligenceProviderFactory.ts`, `server.ts`/`route.ts`/`reflect.ts` (closure threading), 8 tagged callsites, `componentCategories.ts`, `templates.ts`, `PostUpdateMigrator.ts`, `STANDARDS-REGISTRY.md`, lint + 8 test files.
+
+## Decision-point inventory
+
+- `CircuitBreakingIntelligenceProvider.evaluate` (the funnel) — **pass-through** — the breaker/shed/error decision logic is UNCHANGED; the error path *adds token fields to a row it already writes*, and an `unlabeled` row additionally fires a once-per-process signal event. No block/allow behavior changes.
+- `CodexCliIntelligenceProvider.evaluate` mode selection — **add (non-gating)** — chooses `--json` vs plain invocation per call via the kill-switch resolver. Both modes produce the same evaluate() contract; this selects a transport, it does not gate any caller.
+- `scripts/lint-llm-attribution.js` — **add (build-time only)** — blocks COMMITS (via `npm run lint`), never runtime actions. Deterministic structural validator (the explicitly-allowed class in signal-vs-authority's "when this does NOT apply").
+- DegradationReporter emissions (`codex-usage-parse-drift`, `unlabeled-llm-call`) — **add (signal-only)** — fixed constants, once per process, never block/delay/rewrite anything.
+- No gate, sentinel, watchdog, or recovery path changes its verdict logic anywhere in this change.
+
+---
+
+## 1. Over-block
+
+**What legitimate inputs does this change reject that it shouldn't?**
+
+- **Lint:** a funnel callsite whose attribution is injected via a helper-wrapper in a DIFFERENT file fails the lexical rule even though it is attributed at runtime. Sanctioned fix is inlining (documented in the lint header). Today's tree has zero such cases (full-repo lint is clean). A receiver named e.g. `myProvider` that is NOT an LLM provider would be flagged only if it also has an `.evaluate(` method and no attribution — no such case exists in src/ (verified by the clean run).
+- **Result-file 16 MB cap:** a legitimate >16 MB single codex answer would be rejected loudly. No instar judgment call expects answers near that size (the old path capped at 1 MB total stdout — the new cap is 16× looser than the previous behavior).
+- **Missing-file-after-exit-0 reject:** if a future codex omits the `--output-last-message` file for an empty answer, json mode rejects where plain mode resolved `''`. Documented asymmetry, decided in-spec: masking would hide argument rot (the worse failure). The kill-switch restores plain behavior per machine instantly.
+- No other rejection surface was added. **No further issue identified.**
+
+## 2. Under-block
+
+**What failure modes does this still miss?**
+
+- **Lint lexical limits:** conditional attribution (`attribution: x ?? {component:'Y'}`) and wrapper indirection pass lexically without proving runtime attribution. Backstopped at runtime by the `unlabeled-llm-call` event + `unlabeledCallShare` metric (any escape is visible, just not commit-blocked).
+- **Usage truthfulness:** the parser trusts codex's self-reported `token_count` totals (clamped + reconciled). A codex that mis-reports usage in a self-consistent way passes reconciliation. Account-level `/codex/usage` remains the independent cross-check.
+- **Per-machine ledger:** pool-wide audit still requires reading each machine (pre-existing, noted in-spec).
+- **Wiring-test exclusions:** 18 pre-existing component labels remain unregistered in COMPONENT_CATEGORY (registering them changes live framework routing — deliberately out of scope). The pinned exclusion list prevents NEW ones; the existing ones still route to the default framework.
+
+## 3. Level-of-abstraction fit
+
+**Is this at the right layer?** Yes. Codex child mechanics stay in `transport/codexSpawn.ts` (the single home — no second spawn path in the provider; the helper is consumed, mirroring `spawnCodexAndWait`). Usage parsing is a pure module. Token recording rides the EXISTING funnel tap (no new parallel recording path). The per-model aggregation lives in the ledger (the only component that owns the SQL), and the route only composes. The lint follows the established `lint-no-direct-llm-http.js` shape. Audit rotation lives in `appendAuditEntry` itself, benefiting every SafeFs/SafeGit caller — not bolted onto the codex cleanup path.
+
+## 4. Signal vs authority compliance
+
+Compliant (re-checked against `docs/signal-vs-authority.md`):
+- The drift tripwire and unlabeled backstop are **signals** (DegradationReporter events + durable metrics) with zero blocking power.
+- The only blocking surface (the lint) is a deterministic build-time structural validator — the explicitly-exempt class ("hard-invariant validation" / enumerable domain), like the existing dark-gate and destructive lints. It gates commits, not judgments.
+- The funnel's breaker logic (the existing authority for shed decisions) is untouched.
+- Result extraction reads ONE file written by codex itself; stdout events are observability signal only and can never become the result (events-as-authority would have been the violation; the spec pinned this).
+
+## 5. Interactions
+
+- **onUsage contract widening** ("fires even on calls that subsequently reject"): the only funnel consumer composes caller callbacks; callers receive at most one extra invocation on failed codex calls. Claude's parse path is unchanged (still success-only by construction). pi unchanged. No double-fire: the accumulator finalizes exactly once (settlement contract unit-pinned).
+- **Audit-log rotation vs concurrent writers:** re-stat-immediately-before-rename shrinks the double-rotate window; worst case under a still-possible race is losing one predecessor segment, never the live log; the rotation-marker makes any rotation visible.
+- **Sweep vs live calls:** in-flight Set + 6h age threshold + own-uid lstat verification; cross-process race requires a >6h call (none exists).
+- **Duplicate sweep triggers:** rate floor set BEFORE the pass starts — concurrent evaluates can't double-trigger.
+- **lint-no-direct-destructive:** the new lint's read-only `git diff` execSync needed the standard bootstrap-escape allowlist entry (added with the standard comment).
+- **Contract-evidence gate (pre-push):** fired on SlackAdapter.ts even though this diff touches no Slack API surface (type widening + attribution tag on an internal LLM call) and no Slack token exists in the vault to run the live tier. Fixed the hook rather than bypassing it (instar-dev: "if the hook is genuinely broken, fix the hook"): `scripts/pre-push-gate.js` gains the same in-diff marker escape `check-e2e-pairing.cjs` already uses (`CONTRACT-EVIDENCE: EXEMPT — <reason>`), accepted only when EVERY changed adapter file carries the marker, logged loudly. Over-block resolved; under-block unchanged (real API edits still require live evidence — a marker on an API-changing diff would be visible to the PR reviewer next to the change it covers).
+- **lint-degradation-emit-sites** auto-discovers the two new `.report()` sites (verified: legacy count includes them; warning-only).
+- No shadowing/double-fire with BurnDetector: it reads attribution keys, which only get MORE populated.
+
+## 6. External surfaces
+
+- `/metrics/features` response is **additive** (new fields; existing fields unchanged) — existing consumers (dashboard LLM Activity tab, agents) keep working. The `?feature=` filter now also narrows `totals.byModel` (new field, so no consumer regression).
+- Codex child invocation changes (argv + stdin): kill-switch `intelligence.codexExecJson:false` / `INSTAR_CODEX_EXEC_JSON=0` restores today's invocation byte-for-byte; older CLIs (<0.20) fail loudly naming both levers. Live canary validated against the real installed codex CLI (real call, usage recorded).
+- CLAUDE.md template + migrateClaudeMd addendum (idempotent `unlabeledCallShare` sniff) + shadow-mirror markers: existing agents learn the new surface on update (Migration Parity). No config migration needed (absence = on, by design).
+- Timing dependence: the post-SIGTERM flush requires the child's TERM trap to run within the 2s SIGTERM→SIGKILL grace — when it doesn't, the error row simply carries the last pre-kill cumulative (degraded, never wrong-direction).
+
+## 7. Rollback cost
+
+- **Codex json mode:** per-machine config flip (30s TTL, no restart) or env var — instant, no release needed. When off, codex calls are token-blind again and `usageCoverage` reports it honestly (never silent).
+- **Lint/ratchet:** revert the PR (no data surface).
+- **Ledger column:** `tokens_cached` is an idempotent ALTER; rollback releases simply ignore the column (additive, no migration down needed).
+- **Audit rotation:** revert restores unbounded growth (the prior behavior); rotated `.1` segments remain readable on disk.
+- No agent state repair or data migration in any rollback path.
+
+---
+
+## Second-pass review
+
+**Concur with the review.** (Independent reviewer subagent, 2026-06-11.)
+
+> I independently verified the artifact against the code. (1) Signal-vs-authority holds: the only new blocking surfaces are the build-time lint (deterministic, full-repo run exits 0, allowlist seeded empty and pinned by the ratchet test) and the disclosed in-call rejections (missing result file after exit 0, 16 MB result cap) — both honestly documented in §1 with the per-machine kill-switch as the rollback; the drift tripwire and unlabeled backstop are once-per-process DegradationReporter emissions with zero blocking power, and the result authority is the `--output-last-message` file only (codexSpawn.ts events feed only the usage accumulator). (2) The funnel diff in CircuitBreakingIntelligenceProvider.ts is purely additive — error rows gain tokensIn/Out/Cached and the `unlabeled` signal fires in recordMetric; breaker/shed/error verdict logic is byte-untouched, matching the artifact's "pass-through" classification. (3) The spawn helper's settlement is single-settle by construction (`settled` flag guards close/exit-grace/error races), stdin EPIPE/ERR_STREAM_DESTROYED is absorbed, buffers are bounded (2 MB carry cap, 600-char stderr tail), the out-dir cleanup's try/finally cannot leak the in-flight Set entry, and the sweep's rate-floor/in-flight/lstat-uid brakes match the artifact's interaction claims — all new unit tests pass including timeout-mid-stream flush ordering, held-fd grace, and stderr-drain cases. The only theoretical residual found (a child whose `kill()` itself throws on both rungs could leave evaluate() unsettled) is the same hazard class the prior execFile path had and does not rise to a concern.