instar 0.28.95 → 0.28.97

package/upgrades/side-effects/project-scope-phase1b-pr5.md

@@ -0,0 +1,150 @@
+# Side-Effects Review — project-scope Phase 1b PR 5 (Dashboard Projects tab + Initiatives filter)
+
+**Version / slug:** `project-scope-phase1b-pr5`
+**Date:** `2026-05-11`
+**Author:** `echo`
+**Second-pass reviewer:** `required (new dashboard surface + new server-side filter)`
+
+## Summary of the change
+
+Fifth PR of project-scope Phase 1b. Ships the dashboard surface plus
+the small server-side filter the dashboard depends on. The compaction-
+recovery hook's active-projects digest was already in the template (it
+shipped alongside session-start.sh in Phase 1.9 / PR 1's session-start
+work and the equivalent block is present at lines 467-509 of
+`src/templates/hooks/compaction-recovery.sh`), so no template change
+is needed for this PR.
+
+Spec source: `docs/specs/PROJECT-SCOPE-SPEC.md` § Phase 1.10
+("Dashboard Projects tab"), § Phase 1.10 ("Initiatives tab filter").
+
+Modified files:
+- `dashboard/index.html` (+~210 lines) — adds the Projects tab button
+  alongside the Initiatives tab, adds the Projects panel HTML (header,
+  empty state, list container), registers the tab in the panel router,
+  and ships ~180 lines of JS:
+  - `loadProjects()` — fetches `/projects`, filters to `status:'active'`
+    client-side, fetches each project's joined view via
+    `/projects/:id?reconcile=false` in parallel (capped at 25), renders
+    cards.
+  - `renderProjectCard(project, children)` — XSS-safe construction
+    (`textContent` everywhere, `appendChild` only, no `innerHTML`).
+    Per spec: title row, drift status badge (colored per verdict),
+    round-by-round progress bar + pipelineStage histogram + Halt
+    + Ack buttons.
+  - `startProjectsPoll` / `stopProjectsPoll` — 15-second `setInterval`
+    when the tab is active, cleared on deactivation.
+  - `projectHalt(id)` + `projectAck(id, idx)` — POST to the new
+    runner-backed routes. 409 ("version mismatch") triggers a silent
+    `loadProjects()` re-render rather than an error toast.
+  - `loadInitiatives()` is updated to pass
+    `?excludeKind=project&excludeParented=true` by default, with a
+    "Show projects + children" checkbox to bypass. The filter is
+    server-side; the records never make the wire-trip.
+- `src/server/routes.ts` (+~10 lines) — `GET /initiatives` now accepts
+  `excludeKind=<string>` (drops records where `kind ?? 'task'` matches)
+  and `excludeParented=true` (drops records with a `parentProjectId`).
+- `tests/unit/routes-initiatives.test.ts` (+3 cases + mirror update) —
+  exercises both new params individually and combined with the existing
+  `status=` filter. The test mirror at the top of the file (which
+  duplicates the production handler shape) was updated in lockstep.
+
+## Decision-point inventory
+
+- **Initiatives filter** — **add** — `excludeKind` and
+  `excludeParented` are additive query parameters that DEFAULT TO NOT
+  EXCLUDING. Existing callers see no behavior change. The dashboard
+  opts in via a UI checkbox that defaults to "exclude" (matches the
+  spec's "filter project-kind by default").
+- **Projects tab read-only verbs** — **add** — the Halt and Ack
+  buttons route through the existing `POST /projects/:id/halt` and
+  `POST /projects/:id/ack` (shipped in PR 3). No new HTTP surface in
+  this PR. The buttons act on the project record's current version
+  (from the latest `loadProjects()` fetch); the routes enforce OCC
+  via `If-Match` — actually, `/halt` and `/ack` do NOT require
+  `If-Match` in PR 3's design (they use the tracker's internal
+  version-aware update via `ifMatch` from the latest `get()`). The
+  dashboard relies on that internal handling rather than passing
+  `If-Match` itself.
+- **15-second poll cadence** — **add** — matches the spec exactly.
+  The interval is cleared when the tab deactivates so a hidden tab
+  doesn't burn cycles.
+
+## Over-block vs under-block analysis
+
+### Server-side filter
+Over-block: the dashboard's default-on filter means a user who clicks
+"Initiatives" no longer sees project-kind records or child items
+unless they tick the "Show projects + children" checkbox. This is the
+spec's documented behavior (Projects get their own tab; children live
+inside their parent's card). No information is lost, just relocated.
+
+Under-block: the filter is purely additive. Existing scripts or
+clients that hit `/initiatives` without the new params see the same
+behavior they always have.
+
+### Dashboard rendering
+Over-block: 409 from `/halt` or `/ack` is treated as "stale view,
+just refresh" — no error toast, no user-visible noise. The spec
+explicitly calls this out: "409 response handled silently: refresh
++ re-render, no error toast."
+
+Under-block: the project-list fetch caps detailed lookups at 25. A
+deployment with >25 active projects would have the rest rendered as
+empty cards (since `detailed` is `null` for un-fetched IDs and we
+skip nulls). Acceptable for Phase 1; a follow-up PR can add pagination
+when project count actually exceeds the cap.
+
+### XSS safety
+Every rendered string goes through `.textContent` (never `.innerHTML`).
+Project ids, titles, round names, child stages, drift verdicts — all
+constructed via `createElement` + `textContent` + `appendChild`. The
+existing `dashboard/index.html` conventions already enforce this; the
+new code follows the same pattern.
+
+## Signal vs authority audit
+
+Dashboard is **read-only display** for the project surface (with two
+button-triggered mutating calls to existing PR 3 routes). The
+authority for halt and ack lives in `ProjectRoundRunner` — the
+dashboard just calls the documented endpoints. No new authority
+introduced.
+
+The initiatives filter is a **read-side** filter, not a
+record-modifying gate. The records themselves are unchanged; the
+filter only affects the response set.
+
+## Interactions with existing systems
+
+- **PR Pipeline tab.** No interaction — the Projects tab uses its
+  own panel + JS namespace.
+- **Initiatives tab.** Shares the existing `loadInitiatives()`
+  function; the new filter is a query-string addition with a UI
+  checkbox to bypass. Existing tab behavior preserved when the
+  checkbox is checked.
+- **`/projects/:id?reconcile=false`.** This query parameter is
+  already documented in the spec (Phase 1.3 — "Clients that need
+  pure-read semantics use `?reconcile=false`"). The dashboard uses
+  it to avoid triggering the lazy reconciler on every 15-second
+  poll. The `?reconcile=false` parameter is honored as a passthrough
+  in the existing route (the lazy reconciler isn't wired yet, so
+  it's a no-op today — when the reconciler ships, the dashboard
+  poll won't trigger it).
+- **Compaction-recovery hook.** Already includes the active-projects
+  digest section (PR 1's template work). No change needed in this PR.
+
+## Rollback cost
+
+Revert removes the new tab button, the new panel HTML, the JS block,
+the Initiatives filter checkbox, the server-side filter, and the test
+cases. No schema changes. Existing project records and initiatives
+are unaffected.
+
+## Verification
+
+- `npm run lint` — passes (tsc + lint-no-direct-destructive).
+- `npx vitest run tests/unit/routes-initiatives.test.ts
+  tests/unit/route-completeness.test.ts
+  tests/integration/projects-api.test.ts
+  tests/unit/dashboard-initiativesTab.test.ts` — 66/66 pass.
+- Existing PR 1-4 invariants still green.

package/upgrades/side-effects/project-scope-phase1b-pr6.md

@@ -0,0 +1,136 @@
+# Side-Effects Review — project-scope Phase 1b PR 6 (Tone-gated round-complete message + delivery)
+
+**Version / slug:** `project-scope-phase1b-pr6`
+**Date:** `2026-05-11`
+**Author:** `echo`
+**Second-pass reviewer:** `required (new outbound-message template + new disk-backed idempotency surface)`
+
+## Summary of the change
+
+Sixth PR of project-scope Phase 1b. Ships the two final primitives the
+autonomous run loop will need to emit round-complete digests safely:
+
+- `formatRoundCompleteMessage(input)` — pure template function. Enforces
+  required-field PRESENCE (not non-emptiness, per spec § Phase 1.8 —
+  the gate "never silently rejects on a legitimate halt"). Returns
+  `{ok:true, message, idempotencyKey}` or `{ok:false, missingFields}`.
+- `RoundCompleteDeliveryHelper` — retry + idempotency wrapper around
+  a caller-provided send function. 3-attempt exponential backoff
+  (1s, 2s, 4s defaults). Records the `(projectId, roundIndex,
+  eventKind, projectVersion)` key in `.instar/local/round-complete-sent.json`
+  after the first successful send so tone-gate retries don't produce
+  duplicates. On permanent failure (all retries exhausted, or first
+  non-transient response), the caller's `onPermanentFail` runs.
+
+Nothing wires these up to an actual round-complete event yet — the
+autonomous run loop is the consumer. PR 6 ships the primitives so the
+run loop in a follow-up PR doesn't fork either of them.
+
+Spec source: `docs/specs/PROJECT-SCOPE-SPEC.md` § Phase 1.8.
+
+New files:
+- `src/core/ProjectRoundCompleteMessage.ts` (~290 lines) — the
+  template + delivery helper described above. Pure ESM, no extra deps.
+- `tests/unit/ProjectRoundCompleteMessage.test.ts` (20 cases) —
+  template required-field gate (all 9 required + halt-flavor whatHalted),
+  empty-string accepted, root-cause-default omitted, override link
+  rendered, evidence-cap honoured, idempotency-key permutations,
+  delivery first-attempt success, dedup short-circuit, transient retry
+  to success, non-transient bail with fallback, all-transient exhaust
+  with fallback, on-disk persistence across instances, no-record on
+  total failure, template-reject suppresses send.
+
+## Decision-point inventory
+
+- **Template required-field gate** (`formatRoundCompleteMessage`) —
+  **add** — rejects with `{ok:false, missingFields}` when any of the
+  9 required fields is undefined / null / non-array (for `evidenceCited`)
+  / non-integer (for `roundIndex`/`projectVersion`). Halt-flavor events
+  additionally require `whatHalted` to be a string. Empty strings are
+  ACCEPTED — the spec explicitly says presence is what matters.
+- **Idempotency-key dedup** (`RoundCompleteDeliveryHelper.sendOnce`) —
+  **add** — recorded only after a confirmed send. If all attempts
+  failed, the key is NOT recorded, so a subsequent retry with the
+  same input will be allowed to call `send` again. Bounded to 1000
+  recent keys so the dedup file doesn't grow unbounded.
+- **Fallback firing** (`onPermanentFail`) — **add** — fires once when
+  all retries are exhausted OR a non-transient failure is reported.
+  Caller decides whether the fallback path is attention-queue,
+  audit-log, `awaitingUser` population, or all three. The PR 6
+  delivery helper just signals "we couldn't deliver — your turn."
+
+## Over-block vs under-block analysis
+
+### Template gate
+Over-block: an `evidenceCited: []` (empty array) is accepted —
+required-field PRESENCE includes "array exists, possibly empty." The
+gate would reject `evidenceCited: undefined`. Matches spec.
+
+Under-block: there's no length cap on individual string fields. A
+30,000-char `whatLanded` would render in full. Acceptable: outbound
+content quality is the tone-gate's job (MessagingToneGate already
+flags excessive verbosity). The template just builds the string.
+
+### Delivery helper
+Over-block: a non-transient failure bails immediately rather than
+retrying. Matches the spec's implicit "permanent failure → fallback"
+semantics — retrying a 401 / 400 won't help.
+
+Under-block: dedup is keyed on `projectVersion`. If the project record
+is mutated between send attempts (which it normally is — the
+post-send tracker.update bumps version), a second attempt would have
+a different key and re-send. This is intentional — the version bump
+is what makes "same round, different state" distinguishable from
+"same round, same state, duplicate send."
+
+## Signal vs authority audit
+
+The template is a pure function — no signal/authority overlap. The
+delivery helper has authority only over its own idempotency ring; it
+delegates transport entirely to the caller's `send` function.
+
+## Interactions with existing systems
+
+- **`MessagingToneGate`.** The caller (the autonomous run loop, not
+  shipped yet) is responsible for routing the formatted message
+  through the tone gate before calling `helper.sendOnce`. The helper
+  does NOT call the tone gate directly — keeps the boundary clean
+  and the helper unit-testable.
+- **TelegramAdapter / Slack / WhatsApp.** None used here. The caller
+  supplies the `send` function.
+- **`.instar/local/`.** The dedup file lives at
+  `.instar/local/round-complete-sent.json` — machine-local, NOT
+  git-synced (same convention as the round-runner lock and the drift
+  ledger lock). Two machines won't dedupe each other's sends, which
+  is correct: each machine should make its own send decision and the
+  recipient platforms (Telegram, Slack) handle their own dedup.
+- **`SafeFsExecutor`.** Used for test cleanup; the in-class
+  persistence uses tmp+rename atomic writes (matches the pattern in
+  `MachineHeartbeat`).
+
+## Rollback cost
+
+Revert deletes `src/core/ProjectRoundCompleteMessage.ts` and the
+test file. The on-disk dedup file becomes orphaned but harmless.
+No schema migrations.
+
+## What this PR explicitly defers (to PR 7+)
+
+- **Wiring into the autonomous run loop.** The run loop is the
+  intended consumer; it ships separately. Until then, no caller
+  invokes `formatRoundCompleteMessage` or `sendOnce`.
+- **Attention queue / audit log / `awaitingUser` integration.** The
+  fallback callback is caller-provided; PR 6 only fires it. The
+  concrete callback that wires those three downstream channels
+  ships with the run loop.
+- **MessagingToneGate routing.** Same — the caller threads the
+  message through the gate; the helper accepts the formatted text
+  as-is.
+
+## Verification
+
+- `npm run lint` — passes (tsc + lint-no-direct-destructive).
+- `npx vitest run tests/unit/ProjectRoundCompleteMessage.test.ts` —
+  20/20 pass.
+- Existing PR 1-5 invariants unaffected (no shared code touched
+  outside the new file).

package/upgrades/side-effects/project-scope-phase1b-pr7.md

@@ -0,0 +1,175 @@
+# Side-Effects Review — project-scope Phase 1b PR 7 (Autonomous run loop)
+
+**Version / slug:** `project-scope-phase1b-pr7`
+**Date:** `2026-05-11`
+**Author:** `echo`
+**Second-pass reviewer:** `required (new subprocess management + process-group signaling + filesystem allocation under target repo)`
+
+## Summary of the change
+
+Final PR of project-scope Phase 1b. Ships the autonomous run loop the
+spec § Phase 1.5 names (steps 1-11), wired against the lock + preflight
++ runner primitives that landed in PRs 3-6:
+
+- `ProjectRoundWorktrees` — lazy worktree allocator under
+  `<targetRepoPath>/.worktrees/<projectId>/<roundIndex>/<itemId>`.
+  Appends `.worktrees/` to `.git/info/exclude` on first call so the
+  namespace doesn't pollute `git status`.
+- `runRound(input, deps)` — the run loop. Acquires the lock, lazily
+  allocates the first worktree, spawns the autonomous child in a
+  detached process group, polls the project record every 60 seconds.
+  On mid-round mutation to `round.itemIds`, SIGTERMs the child's
+  process group (5s grace, then SIGKILL) and relaunches with the
+  new stop condition. On the child's natural exit, verifies per-item
+  artifacts via the caller-injected `verifyMergedItems` and sets
+  `round.status` to `complete` | `partially-complete`. On
+  `haltedAt` set mid-run, kills the child and returns `halted`.
+  Three-attempt resume cap on transient non-zero exits before
+  `failed`.
+
+Spec source: `docs/specs/PROJECT-SCOPE-SPEC.md` § Phase 1.5.
+
+New files:
+- `src/core/ProjectRoundWorktrees.ts` (~140 lines) — pathFor,
+  allocate, prune, remove, ensureExcludeEntry helpers. `SafeGitExecutor`
+  for all git invocations.
+- `src/core/ProjectRoundExecution.ts` (~340 lines) — the run loop
+  itself, plus `verifyMergedItemsViaGit` helper for callers that want
+  the real `git merge-base --is-ancestor` verifier instead of a stub.
+- `tests/unit/ProjectRoundWorktrees.test.ts` (8 cases) —
+  path-shape, allocate-creates, refuse-on-existing, idempotent
+  exclude entry, prune-on-empty, remove-on-missing,
+  remove-cleans-up, allocate-with-refuseExisting-false.
+- `tests/unit/ProjectRoundExecution.test.ts` (8 cases) —
+  lock-held-failure, first-pass-complete (no spawn), natural-exit
+  full-verification, natural-exit subset → partially-complete,
+  halt-mid-run, dynamic stop revalidation (itemIds mutation
+  triggers relaunch), worktree allocated, exclude entry written.
+
+## Decision-point inventory
+
+- **Spawn in detached process group** (`spawnAutonomousChild`) —
+  **add** — `detached: true` per Node `child_process.spawn`. The
+  child gets its own process group; the runner is NOT a member.
+  `process.kill(-pid, signal)` (negative PID = group target) reaps
+  the child's entire process tree (compilers, test runners spawned
+  by the autonomous skill) without reaping the runner. Spec § Phase
+  1.5 step 5 calls this out explicitly.
+- **Worktree allocation** — **add** — lazy on first use. Refuses
+  by default if path exists (spec semantics); callers can pass
+  `refuseExisting: false` to override (used on resume paths).
+- **Per-step halt checkpoint** — **add** — before every iteration of
+  the inner relaunch loop, re-reads `round.haltedAt` from the tracker.
+  Set → SIGTERM the child, return `halted`.
+- **Stop-condition verifier dependency injection** — **add** —
+  `verifyMergedItems` is required input. Default (used when no
+  override) is a no-op set (returns empty set), so production callers
+  MUST pass a real verifier. Tests pass stubs. The real production
+  verifier is `verifyMergedItemsViaGit`, exported alongside the run
+  loop.
+- **Resume cap** — **add** — 3 transient non-zero exits before
+  `failed`. Spec § Phase 1.5 step 11.
+
+## Over-block vs under-block analysis
+
+### Process group signaling
+Over-block: SIGTERM-grace-then-SIGKILL of the WHOLE process group
+will reap any descendant process the autonomous skill spawned —
+compilers, test runners, lockfile holders. This is intentional and
+matches the spec.
+
+Under-block: if the autonomous child somehow detaches FURTHER
+(creates its own grandchild group), that group escapes. Acceptable
+edge case — the autonomous skill is in-house and doesn't do that;
+we'd notice in observability before it becomes a leak.
+
+### Worktree allocation
+Over-block: a stale worktree directory under `.worktrees/` from a
+crashed previous run causes `allocate()` to throw. Callers (the
+runner's first-item path) pass `refuseExisting: false` so the
+allocator returns the existing path instead. Trade-off: a crashed
+run's stale worktree is silently reused. Acceptable: `prune()` runs
+on every successful round completion, and `git worktree add` would
+itself complain if the worktree is corrupt.
+
+Under-block: `prune()` runs `git worktree prune` against the whole
+repo, not just our namespace. Worktrees registered by OTHER tools
+(unrelated to project-scope) that have become stale will ALSO be
+pruned. Acceptable: `git worktree prune` only removes administrative
+entries pointing at missing paths — it does not delete live worktree
+directories.
+
+### Run loop
+Over-block: a long-running autonomous child that takes 4 hours to
+complete isn't capped here. Acceptable — the autonomous skill has
+its own duration cap (stop hook); the run loop just watches for exit.
+
+Under-block: a child that hangs without exiting AND without itemIds
+changing AND without `haltedAt` being set sits forever. Acceptable
+for this PR — the auto-advance poller's `LOCK_HELD` rejection will
+eventually surface the stuck round in telemetry, and the user can
+`POST /halt`.
+
+## Signal vs authority audit
+
+The run loop is **authority** for round-state transitions
+(`complete` / `partially-complete` / `failed`). The decision is
+deterministic: the caller-provided `verifyMergedItems` (a signal
+source) returns a set; the run loop counts matched/unmatched and
+sets `round.status` accordingly. The runner never sets a verdict
+the verifier didn't support.
+
+## Interactions with existing systems
+
+- **`ProjectRoundLock`** — acquired at step 1, released at step 8 in
+  the finally block (always releases even on throws).
+- **`InitiativeTracker`** — single source of truth for `round.itemIds`,
+  `round.haltedAt`, `round.status`. Polled every `pollIntervalMs`;
+  mutated on outcome via `tracker.update(...)` with `ifMatch`. OCC
+  races are silently ignored at outcome-recording time (next read
+  will see the conflicting writer's state).
+- **`SafeGitExecutor`** — every git invocation (worktree add / prune /
+  remove, merge-base --is-ancestor) routes through the safe executor.
+  No raw `execFileSync('git', ...)` anywhere.
+- **`ProjectRoundRunner.preflight`** (PR 3) — the run loop assumes
+  preflight has passed. It does NOT re-run preflight at step 1; the
+  caller is expected to do it (auto-advance poller does; /project
+  run-round skill does).
+- **`ProjectAutoAdvancePoller`** (PR 4) — the poller calls
+  `preflight` and then in PR 7+ can call `runRound` directly. PR 4
+  bookkeeps unacked count + clears autoAdvanceAt; the run loop is
+  the actual work-doer that follows.
+- **`RoundCompleteDeliveryHelper`** (PR 6) — the run loop does NOT
+  invoke the delivery helper. That's a caller concern (the poller
+  or the skill consumes the run-loop result and constructs the
+  message). Keeps the run loop's responsibilities tight.
+
+## Rollback cost
+
+Revert deletes `src/core/ProjectRoundWorktrees.ts`,
+`src/core/ProjectRoundExecution.ts`, the two test files. Existing
+worktree directories under `<targetRepoPath>/.worktrees/...` become
+orphaned but harmless. `git worktree prune` from a subsequent
+unrelated run cleans up the administrative entries. No schema
+changes.
+
+## What this PR explicitly defers
+
+- **Wiring the poller / skill to call `runRound`.** PR 4's poller
+  currently only bookkeeps; calling `runRound` is a small wiring
+  task in a follow-up PR (the runner instance is already in the
+  ctx; the poller just needs to invoke it).
+- **`GET /projects/:id/next` real implementation.** Still 501.
+- **Lazy merged-state reconciler on `GET /projects/:id`.** Spec
+  Phase 1.5 — the read-may-mutate behavior with the ≤3-children-per-GET
+  cap. Not implemented in PR 7; the
+  `verifyMergedItemsViaGit` helper here is the building block the
+  reconciler can call into in a follow-up.
+
+## Verification
+
+- `npm run lint` — passes (tsc + lint-no-direct-destructive).
+- `npx vitest run tests/unit/ProjectRoundExecution.test.ts
+  tests/unit/ProjectRoundWorktrees.test.ts` — 16/16 pass.
+- Existing PR 1-6 invariants unaffected (the run loop is a new
+  module; no shared-code mutations).