instar 1.2.26 → 1.2.27

package/upgrades/1.2.26.md

@@ -1,51 +1,58 @@
 # Upgrade Guide — vNEXT
 
-<!-- bump: patch -->
-<!-- patch = bug fixes, refactors, test additions, doc updates -->
+<!-- bump: minor -->
+<!-- minor = new capability (ORG-INTENT drift detection — Phase 4, completes the four-phase project). Aggregates fix(heal) from PR #316 too. -->
 
 ## What Changed
 
-**fix(heal): better-sqlite3 self-heal now survives Homebrew Node updates that strand process.execPath.**
+**feat(org-intent-runtime): periodic drift detection job + analyzer (Phase 4 of 4 — completes the project).**
 
-When Homebrew (or any package manager) updates Node while an instar server is running, the brew install swaps the `/opt/homebrew/bin/node` symlink and removes the previous Cellar directory. The server process keeps an open file descriptor to the deleted binary, so it continues executing — but any new `spawnSync(process.execPath, ...)` call returns ENOENT. The native-module self-heal path runs through exactly that kind of spawn, so the heal silently fails. The agent then degrades to legacy-memory fallbacks and stays there across restarts (the next startup hits the same stale execPath).
+The per-message Coherence Gate from Phase 1 catches individual constraint violations. The session-start injection from Phase 2 ensures the agent reasons with the contract from message one. The tradeoff helper from Phase 3 lets non-reviewer code paths resolve values collisions deterministically. But none of these catch the slow accumulation pattern — every individual message passes the gate, yet week-over-week the agent has gradually drifted toward optimizing for the wrong objective. That is the Klarna failure mode at action scale, and it is what this phase catches.
 
-This was observed live on luna 2026-05-21. Server log showed the heal firing at 02:46:00, then 150ms later: `"rebuild failed (spawnSync /opt/homebrew/Cellar/node/25.6.1/bin/node ENOENT). SQLite subsystems may degrade."` Knowledge graph, conversation summaries, and the pending-relay queue all went offline while a healthy Node (a different Cellar version) was sitting on disk a few millimetres away.
+Three new surfaces, all signal-only:
 
-The fix:
+- **`src/core/OrgIntentDriftAnalyzer.ts`** — pure function. Given recent Coherence Gate review history + parsed `ORG-INTENT.md`, produces a `DriftAnalysis` with a trend label (`stable` | `rising` | `concerning` | `insufficient-data` | `no-org-intent`), per-reviewer block-rate stats, half-window comparison, and cross-references against ORG-INTENT constraints / goals / values.
+- **`GET /intent/org/drift?lookbackDays=N`** — HTTP route exposing the analyzer. Returns 503 when the response review pipeline is not enabled (graceful degradation).
+- **Weekly job template** (`src/scaffold/templates/jobs/instar/org-intent-drift-audit.md`) — `enabled: false` by default. Operators with an authored `ORG-INTENT.md` can flip it on; the job runs every Monday at 10:00 local time, calls the route, and sends a Telegram heads-up only when `shouldSurface: true` (trend is rising or concerning). Most weeks stay silent — that means the agent is staying aligned.
 
-1. New `resolveStableNodeBinary()` helper at `src/utils/resolveNodeBinary.ts` (with a CommonJS twin at `scripts/resolve-node-binary.cjs`). It walks a fallback chain — `process.execPath` → `fs.realpathSync(execPath)` → optional caller-supplied bundled Node → `/opt/homebrew/bin/node` (the stable symlink Homebrew always maintains) → `/usr/local/bin/node` → `/usr/bin/node` → `which node` — and returns the first existing-and-executable path, or null when every candidate fails.
+Per `feedback_signal_vs_authority`: every surface in this phase is SIGNAL. The Coherence Gate from Phase 1 remains AUTHORITY. Nothing here writes to `ORG-INTENT.md`, modifies the gate's behavior, or blocks anything.
 
-2. `ensureSqliteBindings()` in `src/commands/server.ts` now resolves the spawn target through `resolveStableNodeBinary()` before invoking either the bundled `fix-better-sqlite3.cjs` or the npm-rebuild fallback. If resolution returns null, a structured DegradationReporter event is emitted with an explicit recovery hint instead of the previous "rebuild failed (ENOENT)" log line.
+This is the final phase of the four-phase ORG-INTENT runtime project that started with topic 11378 on 2026-05-21. With this release, `ORG-INTENT.md` is fully load-bearing infrastructure: enforced at message review (Phase 1), surfaced at session start (Phase 2), consultable for tradeoffs (Phase 3), and monitored for drift (Phase 4).
 
-3. `scripts/fix-better-sqlite3.cjs` resolves once at module load and uses the resolved path for every internal spawn (testBinary probe, verifyChildAbiMatches defence, source-build via npm). `findNpmCli` also resolves the npm sibling relative to the resolved Node so a stale execPath doesn't poison npm discovery.
+Spec: `docs/specs/ORG-INTENT-DRIFT-DETECTION-SPEC.md`. ELI16 companion: `docs/specs/ORG-INTENT-DRIFT-DETECTION-SPEC.eli16.md`. Side-effects review: `upgrades/side-effects/org-intent-drift-detection.md`.
 
-4. When the rebuild itself still fails after resolution, `ensureSqliteBindings()` emits a second DegradationReporter event (`ensureSqliteBindings.rebuildFailed`) with a user-actionable recovery hint. Previously the failure was a single yellow log line; now it surfaces through the same degradation channel operators already monitor.
+---
 
-Fail-closed semantics preserved: if no Node binary is reachable anywhere, the heal does NOT proceed (the alternative — guessing wrong — risks producing an ABI-mismatched binary that fails silently later). The DegradationReporter event names the missing Node so the operator can fix it.
+**Also included: fix(heal) — better-sqlite3 self-heal now survives Homebrew Node updates that strand `process.execPath`.**
 
-## Evidence
+When Homebrew (or any package manager) updates Node while an instar server is running, the brew install swaps the `/opt/homebrew/bin/node` symlink and removes the previous Cellar directory. The server process keeps an open file descriptor to the deleted binary, so it continues executing — but any new `spawnSync(process.execPath, ...)` call returns ENOENT. The native-module self-heal path runs through exactly that kind of spawn, so the heal silently fails. The agent then degrades to legacy-memory fallbacks and stays there across restarts (the next startup hits the same stale execPath).
 
-Failure mode reproduced and verified to stop:
+This was observed live on luna 2026-05-21. The fix adds a `resolveStableNodeBinary()` helper that walks a fallback chain — `process.execPath` → `fs.realpathSync(execPath)` → optional caller-supplied bundled Node → `/opt/homebrew/bin/node` (the stable symlink Homebrew always maintains) → `/usr/local/bin/node` → `/usr/bin/node` → `which node` — and returns the first existing-and-executable path. `ensureSqliteBindings()` in `src/commands/server.ts` and `scripts/fix-better-sqlite3.cjs` both route through the resolver before spawning. Fail-closed semantics preserved: if no Node binary is reachable, the heal does not proceed and a structured DegradationReporter event surfaces.
 
-Reproduction (`tests/integration/heal-execpath-staleness.test.ts`) covers six scenarios drawn directly from the luna incident:
+## What to Tell Your User
 
-1. `spawnSync` against a stale Cellar path (`/opt/homebrew/Cellar/node/0.0.0-removed-by-brew/bin/node`) returns ENOENT — the same shape as the luna server log line.
-2. The resolver returns a working fallback when execPath is stale. A child process spawned against the resolved path executes successfully.
-3. `fix-better-sqlite3.cjs` resolves a stable Node binary at module load (resolver and script both present, exported helpers callable).
-4. The resolver fails closed when no Node is reachable anywhere — does not silently pick a phantom path.
-5. The resolver rejects non-executable files even when they exist at the requested path (defence against `chmod -x` or write-only files).
+If you have an organizational intent file authored AND you opt into the new weekly job, you'll get a Telegram heads-up on Monday mornings when your agent's outbound-message review block rate is rising or concerning. Most weeks stay silent — that means the agent is staying aligned with your organizational intent. The heads-up tells you the trend, what flagged most, and one or two suggestions for where to look. It surfaces patterns; it doesn't fix anything autonomously. The fix is your call.
 
-Plus seven unit tests in `tests/unit/resolveNodeBinary.test.ts` covering the full fallback chain branch-by-branch.
+If you haven't authored an organizational intent file, nothing changes. This is a zero-cost upgrade for those agents.
 
-The live fix on luna (manually-invoked `fix-better-sqlite3.cjs` via the resolved Node binary) brought her degradation count from 8 to 1, with SemanticMemory, TopicMemory, FeatureRegistry, and the pending-relay queue all returning to healthy. With this PR landed, the same recovery will run automatically on every server restart that detects the mismatch.
+This is the last of four phases in the ORG-INTENT runtime project. With this release, `ORG-INTENT.md` is fully load-bearing infrastructure: enforced at message review, surfaced at session start, consultable for tradeoffs, and monitored for drift.
 
-## What to Tell Your User
-
-If your agent ever logged "rebuild failed (spawnSync ... ENOENT)" after a Homebrew Node update — most often visible as your knowledge graph or conversation summaries going offline while the agent still appears to be running — this fix lets the agent recover on the next restart without manual help. The heal now tries multiple Node paths instead of just the one the server happened to launch with, so a brew-swept-out-from-under-us Cellar path no longer blocks rebuild.
+A second change in this release (the heal fix): if your agent ever logged "rebuild failed (spawnSync ... ENOENT)" after a Homebrew Node update — most often visible as your knowledge graph or conversation summaries going offline while the agent still appears to be running — this fix lets the agent recover on the next restart without manual help.
 
 ## Summary of New Capabilities
 
-| Capability | How to Use |
-|-----------|-----------|
-| Heal survives stale `process.execPath` | automatic on next server restart |
-| DegradationReporter event on heal failure | automatic — surfaces through existing `/degradations` and any consumers watching that channel |
+- **GET /intent/org/drift** — new HTTP route returning a drift digest derived from recent Coherence Gate review history.
+- **`analyzeOrgIntentDrift()` exported function** — pure logic helper for any callsite that wants the same analysis.
+- **Weekly drift audit job** — agentmd template, off by default, sends a Telegram heads-up only when drift surfaces.
+- **Stable-Node-resolver self-heal** — automatic on next server restart; surfaces a structured DegradationReporter event if no Node binary is reachable.
+- **Migration parity** — existing agents' CLAUDE.md gains a Phase 4 curl line automatically.
+
+## Evidence
+
+- Tier 1 unit tests: `tests/unit/OrgIntentDriftAnalyzer.test.ts` (11 new tests, all passing). `tests/unit/PostUpdateMigrator-org-intent-runtime.test.ts` extended with 1 new test for Phase 4 migration (9 total, all passing).
+- Tier 2 integration tests: `tests/integration/org-intent-routes.test.ts` extended with 1 new test for the drift route (17 total, all passing).
+- Tier 3 E2E lifecycle tests: `tests/e2e/org-intent-drift-lifecycle.test.ts` (4 tests mirroring production wiring through `AgentServer` and `createRoutes` with a real `CoherenceGate` seeded with synthetic history, all passing — including the "feature is alive" 200-not-503 check).
+- Heal fix evidence: `tests/integration/heal-execpath-staleness.test.ts` reproduces the luna failure mode (6 scenarios) + 7 unit tests in `tests/unit/resolveNodeBinary.test.ts` covering the resolver fallback chain branch-by-branch. The live fix on luna brought her degradation count from 8 to 1.
+- Type-check: `npx tsc --noEmit` clean.
+- Lint: clean.
+- The full test suite must remain green before merge per Zero-Failure Standard.

package/upgrades/1.2.27.md

@@ -0,0 +1,40 @@
+# Upgrade Guide — vNEXT
+
+<!-- bump: minor -->
+<!-- minor = new capability (ORG-INTENT drift detection — Phase 4, code already on main but unreleased due to a rebase artifact). -->
+
+## What Changed
+
+**chore(release): cut Phase 4 of the ORG-INTENT runtime project.**
+
+Phase 4 (periodic drift detection) merged on main in PR #319 but did not publish to npm at that time. The Phase 4 NEXT.md content was absorbed into `upgrades/1.2.26.md` (the heal-fix release) during the rebase that resolved the conflict between my branch and the just-cut release. The Phase 4 CODE — `OrgIntentDriftAnalyzer`, the `GET /intent/org/drift` route, the weekly audit job template, and the CLAUDE.md migration — has been sitting on main since PR #319 merged but is not yet on npm.
+
+This PR is a fresh NEXT.md to trigger the publish workflow. No code changes — Phase 4 is already in the tree.
+
+After this publishes, the complete four-phase ORG-INTENT runtime project will be available on npm:
+
+- Phase 1 (v1.2.23): gate-time enforcement of constraints.
+- Phase 2 (v1.2.24): session-start injection of the three-rule contract.
+- Phase 3 (v1.2.25): deterministic tradeoff helper.
+- Phase 4 (this release): periodic drift detection + weekly audit job.
+
+Spec: `docs/specs/ORG-INTENT-DRIFT-DETECTION-SPEC.md`.
+ELI16 companion: `docs/specs/ORG-INTENT-DRIFT-DETECTION-SPEC.eli16.md`.
+Side-effects review: `upgrades/side-effects/org-intent-drift-detection.md`.
+
+## What to Tell Your User
+
+The drift detection capability that was supposed to ship in the prior release was a release-pipeline misfire — the code landed but the publish skipped. This release re-cuts that work so the new endpoint and the weekly audit job become available through the normal instar update path. No behavior change from the prior release if you don't act on it; if you have an organizational intent file authored and want the weekly drift check, ask your agent to enable the new drift-audit job for you.
+
+## Summary of New Capabilities
+
+(All shipped in PR #319 — this release just publishes them.)
+
+- **GET /intent/org/drift** — HTTP route returning a drift digest derived from recent Coherence Gate review history.
+- **`analyzeOrgIntentDrift()` exported function** — pure logic helper.
+- **Weekly drift audit job template** — `.instar/jobs/instar/org-intent-drift-audit.md`, off by default.
+- **Migration parity** — CLAUDE.md gains Phase 4 curl line automatically on next update.
+
+## Evidence
+
+All test evidence is in PR #319's release notes (now in `upgrades/1.2.26.md`). This PR is a no-op release cut.

package/upgrades/side-effects/org-intent-drift-detection.md

@@ -0,0 +1,101 @@
+# Side-effects review — org-intent drift detection (Phase 4)
+
+Spec: `docs/specs/ORG-INTENT-DRIFT-DETECTION-SPEC.md`
+ELI16: `docs/specs/ORG-INTENT-DRIFT-DETECTION-SPEC.eli16.md`
+Phase: 4 of 4 (final). Phases 1, 2, and 3 shipped in prior PRs.
+
+## Surface map
+
+| Change | File | Type |
+|---|---|---|
+| Pure `analyzeOrgIntentDrift()` + `DriftAnalysis` type | `src/core/OrgIntentDriftAnalyzer.ts` (new file) | Additive module |
+| New `GET /intent/org/drift` HTTP route | `src/server/routes.ts` | Additive route |
+| Weekly drift-audit agentmd job template, `enabled: false` | `src/scaffold/templates/jobs/instar/org-intent-drift-audit.md` (new file) | Job template — distributed via `installBuiltinJobs()` |
+| CLAUDE.md ORG-INTENT subsection adds Phase 4 curl line | `src/scaffold/templates.ts` + `src/core/PostUpdateMigrator.ts` | Doc + migration |
+| Tier 1 unit tests (analyzer + migration) | `tests/unit/OrgIntentDriftAnalyzer.test.ts` (new), `tests/unit/PostUpdateMigrator-org-intent-runtime.test.ts` (extended) | Test addition |
+| Tier 2 integration test | `tests/integration/org-intent-routes.test.ts` (extended) | Test addition |
+| Tier 3 E2E test | `tests/e2e/org-intent-drift-lifecycle.test.ts` (new) | Test addition |
+
+## Over-block analysis
+
+**Could the new route or analyzer block anything?**
+
+No. Pure read path. The analyzer takes a snapshot of recent review history (already retained by the gate per its existing retention policy) and produces a derived digest. The route returns the digest. Nothing writes to `ORG-INTENT.md` or to the gate's history. Per `feedback_signal_vs_authority`: SIGNAL only.
+
+**Could the weekly job spam an operator?**
+
+The job ships `enabled: false`. Operators must explicitly opt in. Once enabled, the digest is only sent when `shouldSurface: true` (trend = `rising` or `concerning`). The default thresholds (15% concerning, 5% rising-delta) are calibrated so most weeks stay silent.
+
+**Could the analyzer flag false positives?**
+
+Yes — by design. The trend labels are heuristic. A week with a single bad day might surface as `rising` even if the underlying pattern is noise. This is acceptable because:
+
+- The job's output is a Telegram message, not a code change. The operator reads it on their phone and decides.
+- The suggestions are advisory, not directive. "Check whether the constraints need updating" — not "I updated the constraints."
+- False negatives (missing real drift) are worse than false positives (occasional noise on a stable agent). The defaults err toward surfacing.
+
+Operators who want different thresholds can pass `lookbackDays` to the route or update the job template's body to override.
+
+## Under-block analysis
+
+**What does the analyzer NOT catch?**
+
+- Drift that doesn't show up in gate review history. If the agent does something off-mission that the gate's reviewers don't flag (e.g. answer the wrong question instead of violating a constraint), the analyzer has no signal to work with. This is the natural limit of any system that operates on gate output.
+- Channel-specific patterns. The analyzer aggregates across channels. A surge of blocks on Telegram but stable performance on direct/CLI would show up as overall rising. Future iteration may add per-channel breakdowns.
+- Recovery patterns. If the agent drifted in week 1 and corrected in week 2, the half-window comparison would show falling rates, but the analyzer doesn't surface "drift detected but recovered" as a positive signal. Out of scope for v1.
+
+## Level-of-abstraction fit
+
+The analyzer is a pure function over a typed input. The route is a thin wrapper. The job is an agentmd template that just calls the route and renders. No new abstractions introduced. The decomposition mirrors `IntentDriftDetector` (the existing decision-journal-based detector) but operates on a different input space (gate review history vs decision journal), so they're complementary rather than overlapping.
+
+## Signal-vs-authority compliance
+
+Every surface in this phase is **SIGNAL**:
+
+- The analyzer produces a digest, never blocks.
+- The route returns the digest, never blocks.
+- The job sends a Telegram message, never blocks.
+
+The Coherence Gate from Phase 1 remains **AUTHORITY**. It continues to enforce constraints at message-review time. This phase consumes its output as one input among several; it never modifies the gate's behavior.
+
+This is the right architectural pattern per `feedback_signal_vs_authority`. The drift surface can be wrong without compromising any actual outcome — the worst case is "operator gets a false-alarm Telegram message," not "agent loses a constraint enforcement."
+
+## Interactions with existing systems
+
+| System | Interaction | Risk |
+|---|---|---|
+| Coherence Gate (Phase 1) | Reads `getReviewHistory()` | None — read-only |
+| Session-start injection (Phase 2) | None — different surface | None |
+| Tradeoff helper (Phase 3) | None — orthogonal | None |
+| Decision journal `IntentDriftDetector` | Sibling system on a different input space | None — they observe different things |
+| `installBuiltinJobs()` | New template added to the source dir; will be copied on update | None — non-destructive (installs missing, doesn't overwrite user edits) |
+| Existing weekly job slots | New job is independent and off by default | None |
+
+## Rollback cost
+
+Low. Pure additive feature. Three options:
+
+1. **Code revert**: `git revert <PR-merge-sha>` removes new module + route + job template + test files. No data migration to roll back.
+2. **Disable the job**: The job ships `enabled: false`. Operators who never enable it see zero behavior change.
+3. **Ignore the route**: 404 the route via a config flag if desired (future iteration); currently the route is opt-in via being called.
+
+## Test coverage summary
+
+| Tier | File | Tests | Status |
+|---|---|---|---|
+| 1 (unit) | `tests/unit/OrgIntentDriftAnalyzer.test.ts` (new) | 11 | ✓ passing |
+| 1 (unit) | `tests/unit/PostUpdateMigrator-org-intent-runtime.test.ts` (extended) | 9 (1 new for Phase 4) | ✓ passing |
+| 2 (integration) | `tests/integration/org-intent-routes.test.ts` (extended) | 17 (1 new for Phase 4) | ✓ passing |
+| 3 (E2E lifecycle) | `tests/e2e/org-intent-drift-lifecycle.test.ts` (new) | 4 | ✓ passing |
+
+## This is the final phase
+
+This PR closes the four-phase ORG-INTENT runtime project that started with topic 11378 on 2026-05-21. All four phases shipped — gate / session-start / tradeoff / drift — with three-tier test coverage at every step.
+
+## Open follow-ups (NOT this PR — future iterations)
+
+- Per-channel drift breakdowns.
+- LLM-supervised drift narrative (Tier 1 supervisor over the digest output).
+- Recovery-pattern detection (positive signal when drift corrects).
+- Drift digest persistence (week-over-week comparison artifacts).
+- Action-level drift (catching off-mission actions that don't trip the gate).