instar 1.2.22 → 1.2.24

package/upgrades/1.2.24.md

@@ -0,0 +1,44 @@
+# Upgrade Guide — vNEXT
+
+<!-- bump: minor -->
+<!-- minor = new capability (ORG-INTENT session-start injection — Phase 2). -->
+
+## What Changed
+
+**feat(org-intent-runtime): inject `ORG-INTENT.md` at session-start so the agent reasons with organizational intent from message one (Phase 2 of 4).**
+
+Phase 1 (shipped in v1.2.23) wired ORG-INTENT.md into the Coherence Gate so constraint violations are blocked at outbound-message review time. Phase 2 closes the other half of the loop: the agent now sees the structured three-rule contract in its working context at the start of every session, so it reasons with the constraints, goals, values, and tradeoff hierarchy as it drafts — instead of just being blocked after the fact.
+
+Two delivery surfaces, both light:
+
+1. **New HTTP route** — `GET /intent/org/session-context` returns the parsed three-rule contract formatted as a session-start text block. Returns `{ present: false }` when ORG-INTENT.md is absent, template-only, or unparseable.
+
+2. **Session-start hook** — the canonical session-start hook (installed by `PostUpdateMigrator.getSessionStartHook()`) now fetches that route and injects the labeled block alongside identity, topic context, and other grounding sections. Fail-open: route unreachable / 503 / timeout → silent skip; the Coherence Gate (Phase 1) still enforces the contract at message-review time.
+
+The new formatter is also exported (`formatOrgIntentForSessionStart()` from `src/core/OrgIntentManager.ts`) for any future callsite that wants the same labeled rendering.
+
+Phase 3 (tradeoff helper) and Phase 4 (drift detection job) remain queued.
+
+Spec: `docs/specs/ORG-INTENT-SESSION-START-INJECTION-SPEC.md`. ELI16 companion: `docs/specs/ORG-INTENT-SESSION-START-INJECTION-SPEC.eli16.md`. Side-effects review: `upgrades/side-effects/org-intent-session-start-injection.md`.
+
+## What to Tell Your User
+
+If your agent already has an organizational intent file (ORG-INTENT.md in the .instar directory), this release makes the agent see it at the start of every session — not just get blocked by it afterward. Practically: the agent now drafts responses with the organization's constraints, goals, values, and tradeoff hierarchy already in mind. Most constraint violations will never be drafted in the first place. The gate from the previous release remains the safety net for edge cases.
+
+If you have not authored an ORG-INTENT.md, nothing changes. This is a zero-cost upgrade for those agents.
+
+## Summary of New Capabilities
+
+- **GET /intent/org/session-context** — new HTTP route returning a session-start-ready text block parsed from ORG-INTENT.md.
+- **Session-start injection** — the canonical session-start hook now surfaces the three-rule contract alongside identity, topic context, soul, and working memory.
+- **Exported `formatOrgIntentForSessionStart()`** — pure formatter usable by any new callsite that wants the same labeled rendering.
+- **Migration parity** — existing agents' CLAUDE.md ORG-INTENT subsection is upgraded automatically to mention both surfaces (Phase 1 gate + Phase 2 session-start injection).
+
+## Evidence
+
+- Tier 1 unit tests: `tests/unit/OrgIntentManager-session-start-format.test.ts` (6 new tests, all passing). `tests/unit/PostUpdateMigrator-org-intent-runtime.test.ts` extended with 2 new tests for Phase 2 migration (7 total tests, all passing).
+- Tier 2 integration tests: `tests/integration/org-intent-routes.test.ts` extended with 4 new tests for the session-context route (11 total tests, all passing).
+- Tier 3 E2E lifecycle tests: `tests/e2e/org-intent-session-context-lifecycle.test.ts` (3 tests mirroring production wiring through `AgentServer` and `createRoutes`, all passing — including the "feature is alive" 200-not-503 check).
+- Type-check: `npx tsc --noEmit` clean.
+- Lint: clean.
+- The full test suite must remain green before merge per Zero-Failure Standard.

package/upgrades/side-effects/org-intent-runtime-gate.md

@@ -0,0 +1,117 @@
+# Side-effects review — org-intent runtime gate (Phase 1)
+
+Spec: `docs/specs/ORG-INTENT-RUNTIME-GATE-SPEC.md`
+ELI16: `docs/specs/ORG-INTENT-RUNTIME-GATE-SPEC.eli16.md`
+Phase: 1 of 4 (this PR). Phases 2-4 (session-start injection, tradeoff helper, drift detection job) are queued.
+
+## Surface map
+
+| Change | File | Type |
+|---|---|---|
+| `OrgIntentReviewContext` interface + `ReviewContext.orgIntent` field | `src/core/CoherenceReviewer.ts` | Type extension (additive, optional field) |
+| `OrgIntentManager.parse()` wired into `loadValueDocs()` | `src/core/CoherenceGate.ts` | Behavior change (structured parsing replaces flat extraction for `ORG-INTENT.md` only) |
+| Reviewer prompt rewrite with three-rule contract | `src/core/reviewers/value-alignment.ts` | Behavior change (prompt content) |
+| Criticality auto-promotion for value-alignment when ORG-INTENT has constraints | `src/core/CoherenceGate.ts` | Behavior change (failure-mode handling) |
+| CLAUDE.md template: ORG-INTENT runtime subsection | `src/scaffold/templates.ts` | Doc change (fresh agents) |
+| CLAUDE.md migration: ORG-INTENT runtime subsection patched into existing files | `src/core/PostUpdateMigrator.ts` | Migration parity (existing agents) |
+| Tier 1 unit tests | `tests/unit/CoherenceGate.test.ts`, `tests/unit/PostUpdateMigrator-org-intent-runtime.test.ts` | Test addition |
+| Tier 2 integration test | `tests/integration/coherence-gate-org-intent.test.ts` | Test addition |
+| Tier 3 E2E lifecycle test | `tests/e2e/org-intent-runtime-lifecycle.test.ts` | Test addition |
+
+No new agent-installed config keys. No new hooks. No agent-installed code changes — the runtime wiring is entirely in instar source. Migration is to `CLAUDE.md` only.
+
+## Over-block analysis
+
+**Could the new wiring block messages that previously passed?**
+
+Yes — that is exactly the intended effect for agents with an authored `ORG-INTENT.md`. The behavior change is: constraints written in `ORG-INTENT.md` now actually block outbound messages that violate them. Prior to this change, the same agent with the same file would have let those messages through.
+
+For agents without an `ORG-INTENT.md`, behavior is unchanged. The structured-intent path is gated on `OrgIntentManager.parse()` returning non-null; absent, template-only, or unparseable files return null and fall back to the legacy flat-blob path (which itself is what shipped before).
+
+**Could the criticality auto-promotion over-block?**
+
+The promotion only affects timeout behavior. Specifically: when `value-alignment` reviewer TIMES OUT on an external-facing message AND `ORG-INTENT.md` has at least one constraint, the gate fails closed instead of falling open. Pre-change behavior: a timeout on `value-alignment` led to `ALL_ABSTAIN` or `WARN_ONLY`, which fell open for internal channels and queued/fail-closed for external depending on config. Post-change behavior: external timeout under this condition triggers `HIGH_CRIT_TIMEOUT` which fails closed.
+
+Net effect on operator: slightly higher rate of "Review system unavailable" responses on external channels under reviewer-timeout stress, instead of unreviewed messages slipping through. Override available: explicit `config.reviewerCriticality['value-alignment'] = 'standard'` restores prior behavior.
+
+**Could the migration over-write user-customized CLAUDE.md content?**
+
+No. Both insertion paths use content-sniff guards (`!content.includes('ORG-INTENT.md (Organizational Intent at Runtime)') && !content.includes('Organizational Intent at Runtime')`). The migration is purely additive — it inserts before an anchor (`**Topic-Project Bindings**`) when present, falls back to appending when absent, and skips entirely when the subsection is already present. Re-runs are idempotent (verified by unit test).
+
+## Under-block analysis
+
+**What does the new wiring NOT catch?**
+
+- **Semantic constraint violations the reviewer LLM does not detect.** The reviewer relies on the LLM (`sonnet` by default) to recognize when a draft message violates a written constraint. A subtle violation — say, hinting at internal pricing without quoting a specific number — may slip through if the LLM does not connect the constraint to the draft.
+- **Constraint scoping**: the current model treats all constraints as universal. There is no way to say "this constraint applies only to external contacts." Phase 3 may add channel/recipient scoping.
+- **Tradeoff hierarchy use outside the reviewer**: only the value-alignment reviewer consults it. Code paths that make decisions (research agents, planning) cannot ask the hierarchy for a tiebreaker yet. Phase 3 adds a tradeoff helper API.
+- **Drift accumulation**: a stream of messages each individually passing review can collectively drift from organizational intent. Phase 4 adds periodic drift detection.
+
+**What if ORG-INTENT.md is mutated after the gate boots?**
+
+The 60-minute `ValueDocCache` TTL means edits do not take effect until cache expiry (or process restart). This is documented in the Phase 4 E2E test (`cache-stale` case) so a future refactor cannot silently break the invariant. A future improvement (Phase 2 or 3) may add fs.watch-based invalidation.
+
+## Level-of-abstraction fit
+
+The runtime wiring lives in `src/core/CoherenceGate.ts` and `src/core/reviewers/value-alignment.ts` — the right layer. The parser already lived at `src/core/OrgIntentManager.ts`; this change connects the two at the layer that actually evaluates outbound messages.
+
+The migration lives in `src/core/PostUpdateMigrator.ts` — the right layer for changes that need to propagate to existing agents.
+
+No new abstractions introduced. The change adds one new type (`OrgIntentReviewContext`), one new private method (`resolveCriticality`), and one new helper function (`formatOrgIntent`) — all minimal and proportional to the work.
+
+## Signal-vs-authority compliance
+
+This is the right place in the stack for ORG-INTENT enforcement. The Coherence Gate is the authoritative pre-delivery review for outbound messages. The reviewer prompt explicitly distinguishes:
+
+- **Constraints** → AUTHORITY: severity MUST be `block`. The reviewer is told to block constraint violations.
+- **Goals** → mixed: contradictions warn or block depending on severity. The reviewer has discretion.
+- **Values** → SIGNAL: drift warns. Not authoritative.
+- **Tradeoff hierarchy** → SIGNAL feeding AUTHORITY: it tells the reviewer how to resolve ties; the reviewer is still authoritative on the verdict.
+
+Lower-precision signals (e.g. deterministic keyword scans in the PEL) could be added later for blatant constraint violations, but those would be signals; the reviewer remains the authority. Per `feedback_signal_vs_authority` from agent memory.
+
+## Interactions with existing systems
+
+| System | Interaction | Risk |
+|---|---|---|
+| `value-alignment` reviewer (existing) | Prompt rewritten; receives `orgIntent` field in context | Low — prompt change tested at all three tiers |
+| Other reviewers (existing) | No change | None |
+| `loadValueDocs` cache | Extended to include `orgIntent` field | Low — additive |
+| `ValueDocCache` TTL | Unchanged (60 minutes) | Documented; locked in by E2E test |
+| Legacy `orgValues` flat blob | Still loaded and passed to reviewers | None — backwards compatible |
+| Custom reviewers reading `context.orgValues` | Still work as before | None |
+| `instar intent` CLI surface (existing) | Unchanged | None |
+| `/intent/org`, `/intent/validate` HTTP routes (existing) | Unchanged | None |
+| PostUpdateMigrator other migrations | New migration is independent | None |
+| Husky pre-push tone-gate / version-gate | Should pass — new content has Evidence section, version bump is minor | Verified by full-suite run before push |
+
+No other instar systems consume `ORG-INTENT.md`. The structured parser was a one-purpose primitive used only by the offline analyzer CLI and now by the gate.
+
+## Rollback cost
+
+Low. The legacy flat-blob path (`extractValueSection` → `orgValues`) is retained and still passed to reviewers. Three options:
+
+1. **Code revert**: `git revert <PR-merge-sha>` restores prior behavior. Tests will fail until the new test files are also removed, but production behavior reverts cleanly.
+2. **Soft revert via reviewer config**: explicitly set `config.responseReview.reviewers['value-alignment'].mode = 'warn'` to demote constraint violations from `block` to `warn`. This restores most of the prior behavior without reverting code.
+3. **Soft revert via criticality config**: explicitly set `config.responseReview.reviewerCriticality['value-alignment'] = 'standard'` to undo the timeout-fail-closed change without touching the rest.
+
+No data migration to roll back. No file format changes. No agent-installed code changes.
+
+## Test coverage summary
+
+| Tier | File | Tests | Status |
+|---|---|---|---|
+| 1 (unit) | `tests/unit/CoherenceGate.test.ts` (new describe block) | 5 | ✓ passing |
+| 1 (unit) | `tests/unit/PostUpdateMigrator-org-intent-runtime.test.ts` | 5 | ✓ passing |
+| 2 (integration) | `tests/integration/coherence-gate-org-intent.test.ts` | 4 | ✓ passing |
+| 3 (E2E lifecycle) | `tests/e2e/org-intent-runtime-lifecycle.test.ts` | 4 | ✓ passing |
+
+All three tiers exercise the same parser + gate + reviewer + HTTP route + production wiring, with each tier proving a different layer of the invariant.
+
+## Open follow-ups (deferred to later phases, NOT this PR)
+
+- Phase 2: Inject parsed intent at session-start (alongside identity/topic context).
+- Phase 3: Standalone `POST /intent/tradeoff-resolve` helper consulted by non-reviewer code.
+- Phase 4: Periodic drift detection job sampling recent outbound actions vs intent.
+- Cache invalidation on `ORG-INTENT.md` mutation (fs.watch or signal-based).
+- Per-channel constraint scoping (some constraints external-only, others universal).

package/upgrades/side-effects/org-intent-session-start-injection.md

@@ -0,0 +1,87 @@
+# Side-effects review — org-intent session-start injection (Phase 2)
+
+Spec: `docs/specs/ORG-INTENT-SESSION-START-INJECTION-SPEC.md`
+ELI16: `docs/specs/ORG-INTENT-SESSION-START-INJECTION-SPEC.eli16.md`
+Phase: 2 of 4. Phase 1 (gate wiring) shipped as v1.2.23 (PR #315). Phase 3 (tradeoff helper) and Phase 4 (drift detection job) are queued.
+
+## Surface map
+
+| Change | File | Type |
+|---|---|---|
+| Exported `formatOrgIntentForSessionStart()` formatter | `src/core/OrgIntentManager.ts` | Additive function export |
+| New `GET /intent/org/session-context` HTTP route | `src/server/routes.ts` | Additive route |
+| Session-start hook fetches + injects parsed ORG-INTENT block | `src/core/PostUpdateMigrator.ts` (`getSessionStartHook`) | Behavior change to agent-installed hook script |
+| CLAUDE.md ORG-INTENT subsection updated (mentions Phase 2) | `src/scaffold/templates.ts` + `PostUpdateMigrator.migrateClaudeMd()` | Doc + migration |
+| Tier 1 unit tests (formatter + migration) | `tests/unit/OrgIntentManager-session-start-format.test.ts`, extended `PostUpdateMigrator-org-intent-runtime.test.ts` | Test addition |
+| Tier 2 integration tests | Extended `tests/integration/org-intent-routes.test.ts` | Test addition |
+| Tier 3 E2E test | `tests/e2e/org-intent-session-context-lifecycle.test.ts` | Test addition |
+
+## Over-block analysis
+
+**Could the injection push the agent's context over a token budget?**
+
+The block size scales with the `ORG-INTENT.md` file size — the formatter doesn't add content; it just labels and structures what's already there. A typical authored `ORG-INTENT.md` is dozens of bullets across four sections — comfortably under 500 tokens. The cap is the source file itself; an organization with a 10-page ORG-INTENT.md will see 10 pages injected. That's a feature, not a bug — but operators should be aware that very large intent files consume context.
+
+**Could the hook fail in a way that disrupts session start?**
+
+No. The hook uses `curl -sf --max-time 4` and a wrapping `if [ -n "$ORG_INTENT_RESPONSE" ]; then ... fi`. Any failure path (route unreachable, 503, network timeout, malformed JSON) results in silent skip. The session continues normally. The Coherence Gate from Phase 1 still enforces the contract at outbound message time, so missing the session-start injection never compromises constraint enforcement.
+
+**Could the new route be abused / DoS'd?**
+
+The route reads `ORG-INTENT.md` from the agent's state dir on every request — bounded local disk read, no LLM call, no DB query. The auth middleware enforces Bearer-token gating. Rate at which an attacker could call it is also bounded by the same auth gate. No new abuse surface beyond `GET /intent/org` (which already existed).
+
+## Under-block analysis
+
+**What does the injection NOT catch?**
+
+- A subtle constraint violation the agent doesn't recognize when drafting. The injection helps the agent *understand* the constraints; the gate is what actually *enforces* them. If the agent's reasoning misses a constraint applicability, the gate catches it.
+- Constraint applicability scoping. A constraint like "never quote internal pricing externally" depends on knowing if the recipient is external. The injection surfaces the constraint; the agent must reason about applicability. Phase 3+ may add scoping metadata to constraints.
+- Mid-session ORG-INTENT.md changes. The injection happens at session start. Edits during a session do not propagate until next session.
+
+## Level-of-abstraction fit
+
+The formatter lives in `src/core/OrgIntentManager.ts` alongside the parser — the right place for a pure rendering function. The HTTP route lives in `src/server/routes.ts` next to `GET /intent/org`. The hook update lives in `PostUpdateMigrator.getSessionStartHook()`, the canonical source for session-start hook content. No new abstractions introduced.
+
+## Signal-vs-authority compliance
+
+The session-start injection is **SIGNAL** — it informs the agent's reasoning but has no authority to refuse or modify outbound messages. The Coherence Gate (Phase 1) remains the **AUTHORITY**: constraint violations are blocked at outbound-message review time regardless of whether the agent saw the injection at session start.
+
+This separation is intentional. A signal-only injection is robust to malformed `ORG-INTENT.md`, route failures, and hook timeouts — the worst case is "the agent doesn't see the contract this session," not "the agent ignores the contract." The authority layer (the gate) catches what the signal misses.
+
+## Interactions with existing systems
+
+| System | Interaction | Risk |
+|---|---|---|
+| Coherence Gate (Phase 1) | None — different layer, different lifecycle | None |
+| `GET /intent/org` | Sibling route — same data, different formatting | None |
+| Session-start hook (existing) | Adds one new fetch + inject block alongside identity, topic, integrated-being, soul, working-memory, etc. | Low — fail-open, bounded by curl --max-time 4 |
+| Working memory injection | Independent — both contribute to session-start context | Token cost is additive but bounded |
+| PostUpdateMigrator other migrations | New migration path operates on an isolated content region | Low — content-sniff guards |
+| Existing `getSessionStartHook()` content | Added inline block; rest unchanged | Low — verified by template parity test pattern |
+
+## Rollback cost
+
+Low. Three options:
+
+1. **Code revert**: `git revert <PR-merge-sha>` removes route + hook block + format function. Tests would need removal too.
+2. **Soft revert via hook bypass**: edit installed `session-start.sh` to remove the curl/python3 block. The route remains but is just an unused HTTP endpoint.
+3. **Soft revert via empty file**: replace `ORG-INTENT.md` with template-only content; the route returns `{ present: false }` and no injection happens. The Phase 1 gate also gracefully degrades on template-only.
+
+No data migration to roll back. No file format changes. The Phase 1 wiring is unaffected by this change.
+
+## Test coverage summary
+
+| Tier | File | Tests | Status |
+|---|---|---|---|
+| 1 (unit) | `tests/unit/OrgIntentManager-session-start-format.test.ts` | 6 | ✓ passing |
+| 1 (unit) | `tests/unit/PostUpdateMigrator-org-intent-runtime.test.ts` (extended) | 7 (2 new for Phase 2) | ✓ passing |
+| 2 (integration) | `tests/integration/org-intent-routes.test.ts` (extended) | 11 (4 new for Phase 2) | ✓ passing |
+| 3 (E2E lifecycle) | `tests/e2e/org-intent-session-context-lifecycle.test.ts` | 3 | ✓ passing |
+
+## Open follow-ups (deferred to later phases, NOT this PR)
+
+- Phase 3: `POST /intent/tradeoff-resolve` standalone helper.
+- Phase 4: periodic drift detection job sampling recent outbound actions vs intent.
+- Mid-session ORG-INTENT.md change propagation (fs.watch or signal-based reinject).
+- Per-channel constraint scoping (some constraints external-only, others universal).
+- Cache layer in front of the route if call rate becomes notable (currently bounded by session-start frequency, which is ~1 per session).