@yemi33/minions 0.1.2448 → 0.1.2449

package/docs/worktree-lifecycle.md

@@ -6,10 +6,10 @@ the **quarantine path** (dirty/divergent → quarantine dir + retry), and the
 **Windows file-lock retry** (EPERM/EBUSY footgun). CLAUDE.md → Worktree
 Lifecycle keeps the cross-cutting invariants; the detail lives here.
 
-> Source of truth: `engine/worktree-pool.js`, `engine/shared.js#removeWorktree`
+> Source of truth: `engine/worktrees/pool.js`, `engine/core/shared.js#removeWorktree`
 > + `_retryFsOp`, `engine.js` (`_quarantineDirtyWorktree`,
 > `_killGitDescendantsForWorktree`, `pruneOrphanWorktrees*`, `gcDispatchWorktreeIfOrphan`),
-> `engine/cleanup.js`. Last verified: 2026-07-23.
+> `engine/orchestration/cleanup.js`. Last verified: 2026-07-23.
 
 ## Operator-checkout boundary
 
@@ -76,14 +76,14 @@ dispatch was still active.
   (`excludeDispatchId: id` so the dispatch can clean up its own worktree);
   dispatch-end orphan GC (`gcDispatchWorktreeIfOrphan`); `_quarantineDirtyWorktree`
   (returns `{ skipped: true, quarantinedPath: null }` on skip — callers
-  MUST honor `skipped` and not set `quarantined: true`); `engine/cleanup.js`
+  MUST honor `skipped` and not set `quarantined: true`); `engine/orchestration/cleanup.js`
   orphan-dir sweep.
 - **On skip:** drops a deduped operator note at
   `notes/inbox/engine-worktree-skip-live-<basename>-<date>.md`.
 
 ## Worktree pool (opt-in)
 
-`ENGINE_DEFAULTS.worktreePoolSize > 0` enables `engine/worktree-pool.js` to
+`ENGINE_DEFAULTS.worktreePoolSize > 0` enables `engine/worktrees/pool.js` to
 recycle worktree dirs across branches.
 
 - **Borrow** in `spawnAgent` only when (a) the new branch doesn't exist on
@@ -107,6 +107,88 @@ that intentionally preserve an existing branch continue using that branch
 unchanged. Warm-pool borrowing keeps its existing fetch-and-checkout of
 `origin/<mainBranch>`.
 
+### Shared remote-ref refresh serialization (W-ms5tb6ha016fda10)
+
+`refs/remotes/origin/<mainBranch>` is shared by every dispatch in a clone. Two
+fresh-worktree preparations in the same tick both run `git fetch origin
+<mainBranch>`, git takes a `<ref>.lock` for the update, and the loser dies with:
+
+```
+error: cannot lock ref 'refs/remotes/origin/main': is at <sha> but expected <sha>
+```
+
+That is a **local serialization defect**, not connectivity. Two mitigations:
+
+1. **`shared.withSerializedRemoteRefRefresh(repoPath, fn)`** — an in-process,
+   per-repository async mutex that every shared-ref fetch runs inside
+   (`_fetchWithTransientRetry`, reuse-sync, pool borrow, pool return, dependency
+   pre-merge). The engine daemon issues the concurrent fetches, so chaining them
+   per repo removes the race at its source. Deliberately **not** a file lock — a
+   file lock would be held across a network round-trip, which the concurrency
+   rules forbid. Keyed on the resolved repo path, so unrelated projects still
+   fetch in parallel, and it wraps **only** the fetch (never the follow-up merge,
+   checkout, or worktree add).
+2. **`FAILURE_CLASS.GIT_REF_LOCK`** — classification for the residual
+   cross-process case (an operator's own `git fetch`, the dashboard, the CLI).
+   Previously these classified as `NETWORK_ERROR`, which hid the real cause
+   behind a "flaky network" story and burned dispatch retries. `GIT_REF_LOCK` is
+   **retryable** (the lock clears in seconds) and
+   `_fetchWithTransientRetry` now retries once on it before surfacing.
+
+`shared.isGitRefLockError(err)` is the predicate; it inspects `message`,
+`stderr`, and `stdout` (exec rejections put the actionable git text on stderr)
+and deliberately does not match genuine network/auth failures.
+
+## Control-plane dispatch: the cheapest worktree is the one never created
+
+Some scheduled maintenance work is 100% Minions/REST + `gh`/`az` and never reads
+or writes repository files — "merge approved PRs", "sweep the PR tracker", "CI
+health check". Allocating a checkout for it is pure cost, and it was also
+*harmful*: those dispatches raced the shared-ref fetch above and failed jobs that
+needed no checkout at all.
+
+A work item (or schedule) may declare `meta.executionSurface: "control-plane"`.
+`shared.resolveSpawnPaths` then short-circuits **before** live-checkout mode and
+before the read-only / mutating split, returning
+`{ cwd: MINIONS_DIR, worktreeRootDir: null, controlPlane: true }`. No worktree is
+created, no branch is derived, and the branch mutex, live-mode concurrency cap,
+operator branch-hold probe, and worktree free-space preflight are all skipped.
+
+The opt-in is **structured and fail-closed** — never inferred from prose:
+
+| | |
+|---|---|
+| Field | `meta.executionSurface` (`"repo"` \| `"control-plane"`; unset ⇒ `"repo"`) |
+| Eligible types | `shared.CONTROL_PLANE_ELIGIBLE_TYPES` — `setup`, `ask`, `explore`, `meeting` |
+| Rejection class | `FAILURE_CLASS.INVALID_EXECUTION_SURFACE` (non-retryable) |
+| Validated at | `POST /api/work-items` (create + update final-state guard), `validateScheduleDefinition`, and again in `spawnAgent` |
+
+Eligibility is an **allowlist**, not a denylist: a repo-reading/mutating type
+(`fix`, `implement`, `review`, `test`, `verify`, `decompose`, `docs`,
+`build-fix-complex`) can never declare `control-plane`, an unknown surface string
+is rejected rather than defaulted, and an invalid declaration degrades to a
+normal repo dispatch rather than silently skipping the checkout.
+
+Project context is preserved without a checkout: the rendered prompt still
+carries `project_name` / `repo_name` / `repo_host` / `main_branch`, plus an
+injected `## Execution surface: control-plane` section
+(`engine/agents/playbook.js#buildControlPlaneHint`) telling the agent the repo is not on
+disk, to work through the API and platform CLIs, not to clone, and to stop and
+report if the task turns out to need repository files.
+
+Schedules carry the declaration through `createScheduledWorkItem`:
+
+```json
+{
+  "id": "merge-approved-prs",
+  "cron": "0 */4 *",
+  "title": "Every 4h: merge approved PRs",
+  "type": "setup",
+  "project": "minions-opg",
+  "executionSurface": "control-plane"
+}
+```
+
 ## Creation integrity gate (#905)
 
 `engine.js#runWorktreeAdd` treats every nonzero `git worktree add` result as a
@@ -154,6 +236,11 @@ post-reset re-verify) and the dirty-files prompt-injection site
 A same-named file nested in a subdir is NOT the marker and stays dirty;
 the marker filter never masks a genuine user/agent edit beside it.
 
+Agent playbook health checks use the same boundary: an untracked root-level
+marker is expected orchestration metadata and must be preserved, while every
+staged, modified, deleted, or other untracked entry remains source dirt that
+stops the agent before it edits the worktree.
+
 When the dirt is real and the worktree can't be auto-healed, the engine
 still quarantines and fails non-retryably, and the quarantine
 auto-recovery loop re-queues the item once in a fresh worktree (see
@@ -171,11 +258,29 @@ quarantine fallback; transient probe, fetch, auth, and push failures leave clean
 ahead commits in place for retry. Recoverable
 `refs/minions/quarantine/*` and `refs/minions/quarantine-wip/*` refs are listed
 on Dashboard → Engine under **Quarantined Work**.
-Periodic cleanup deletes these recovery refs after 10 days. This is intentionally
+Periodic cleanup (`engine/orchestration/cleanup.js`, every 60 ticks) deletes these recovery refs
+after 30 days — no operator schedule or manual `minions cleanup` run is required.
+The window is defined once as `QUARANTINE_REF_RETENTION_DAYS` in
+`engine/core/shared.js` (re-exported by `engine/worktrees/quarantine-refs.js`), and
+`ENGINE_DEFAULTS.worktreeProtectedQuarantineRetentionMs` is derived from it so
+the directory and ref windows cannot drift apart. This is intentionally
 longer than the 24-hour quarantine-directory retention window: the refs preserve
 committed HEAD and WIP snapshots after the directory itself is removed, while
 still preventing recovery refs from growing without bound.
 
+The sweep fails safe. Age comes only from the canonical `/<epoch-ms>` segment
+encoded in the ref name; a ref whose trailing segment is not a plain positive
+integer is **preserved and logged**, never deleted speculatively. Only the
+`refs/minions/quarantine/*` and `refs/minions/quarantine-wip/*` namespaces are
+touched — branches, tags, and near-miss namespaces are ignored — and every delete
+is SHA-guarded (`git update-ref -d <ref> <sha>`) so a concurrently rewritten ref
+is refused rather than clobbered. Commit and WIP refs expire independently on
+their own timestamps, so a pair created by the same quarantine event expires
+together. Each configured project is swept independently: a missing or
+unavailable repository is reported and skipped without aborting the sweep for the
+remaining projects, and repeated sweeps are idempotent. Cleanup logs the removed,
+retained, malformed, and failed counts per run.
+
 At dispatch close, an error result is inspected before orphan GC. A clean branch
 that is aligned with origin (or successfully pushed there) remains removable.
 Dirty files, unpushed commits that cannot be pushed, and indeterminate branch
@@ -363,6 +468,36 @@ After `worktreeStuckThreshold` consecutive failures the path is
 `removeWorktree`** — you'll bypass the retry, escalation, and metrics
 layers.
 
+### Removal is verified, never assumed (W-ms9q3dql00vxd4a8)
+
+Every stage of `removeWorktree` clears the worktree's **contents first** —
+including `.git` and the `.minions-worktree` ownership marker — and only then
+unlinks the top directory. On Windows a handle on the directory itself can make
+that final unlink fail while the stage still reports success (`rd /s /q` in
+particular can exit 0 having deleted everything but the root). The husk left
+behind carries no marker, no `gitdir:` pointer and no registration, so every
+safe GC correctly classifies it as `foreignUnmanaged` and keeps it **forever**.
+That is how an install accumulates a hundred empty `W-*` directories nothing is
+ever allowed to reclaim.
+
+So `removeWorktree` verifies the outcome instead of trusting the exit code.
+Each success path funnels through `_verifyWorktreeRemoved`, which:
+
+1. returns success when the path is genuinely gone;
+2. **finishes the job** with a plain `rmdir` when the leftover is empty — an
+   empty directory has nothing left to lose;
+3. otherwise **re-stamps the ownership marker** (a restore of prior ownership,
+   never a claim over a path that never had one) and reports the removal as the
+   failure it was, so the caller retries and the ordinary orphan sweep can still
+   classify and reclaim the residue at its retention boundary.
+
+The corollary for the GC: the **marker is the proof of ownership, not `.git`**.
+Owned residue whose `.git` pointer is missing entirely or dangles at pruned
+admin metadata still expires normally. Residue with *no* marker stays foreign
+regardless of age, emptiness, or how engine-shaped its name looks — an operator
+may have created that directory a moment ago and be about to `git worktree add`
+into it.
+
 ## Periodic prune (W-mq5o6bvy000x7191)
 
 `pruneWorktreesPeriodic` runs `reconcileStaleWorktreeDirectories`
@@ -389,9 +524,49 @@ remain protected. Deletion requires either `.minions-worktree` or the narrow
 legacy proof used for old Windows residue: a `W-*` basename plus a `.git` file
 pointing inside one of the project's common `.git/worktrees` directories.
 Ordinary unregistered residue is eligible after 2h; unprotected
-`-quarantine-<epoch-ms>` residue after 24h. Protected quarantine directories
-remain until manual recovery because they are the only durable copy of work
-whose backup ref could not be created.
+`-quarantine-<epoch-ms>` residue after 24h. Protected quarantine directories get
+the longest — but no longer unbounded — window: they expire after
+`engine.worktreeProtectedQuarantineRetentionMs` (default **30 days**), whose
+default is *derived* from `engine/core/shared.js#QUARANTINE_REF_RETENTION_MS` — the
+same constant `engine/worktrees/quarantine-refs.js` prunes recovery refs at — so a
+quarantine DIRECTORY and its recovery REFS expire together instead of one
+outliving the other. Setting the value to `0` restores the previous indefinite
+retention; the operator range is 0–365 days.
+
+> **Why the default is derived, not duplicated (W-ms87lj8j018j2572).** The two
+> windows were originally independent literals that both happened to read 10 days.
+> When the ref window moved to 30 days they silently drifted, main went red, and
+> `GET /api/worktree-quarantine-refs` would have listed refs whose directories had
+> already been swept. `ENGINE_DEFAULTS.worktreeProtectedQuarantineRetentionMs` now
+> references the shared constant, and a regression test asserts the structural
+> link rather than mere numeric equality.
+
+> **Why the OVERRIDE is resolved through one seam too (W-ms87rbe601a66d13).**
+> Deriving the two *defaults* from one constant closes the drift at build time
+> but not at runtime. `worktreeProtectedQuarantineRetentionMs` is an operator
+> knob, and the recovery-ref sweep used to ignore it entirely — `cleanup.js`
+> called `pruneExpiredQuarantineRefs(projects)` with no retention argument, so it
+> stayed pinned to the built-in 30 days. An operator who shortened retention to,
+> say, 3 days therefore shortened only the DIRECTORY sweep and reintroduced the
+> exact divergence the shared constant removed. Both sweeps now resolve the
+> window through `shared.resolveQuarantineRetentionMs(config)`:
+> `engine/worktrees/gc.js` for directories and
+> `engine/orchestration/cleanup.js` → `pruneExpiredQuarantineRefs(projects,
+> { retentionMs })` for refs. A non-positive value means **indefinite retention
+> on both sides** — `pruneExpiredQuarantineRefs` treats it as "expire nothing",
+> never as "everything is already expired". `GET /api/worktree-inventory`
+> reports `recoverableRefs.retentionMs` at the resolved window, so the API never
+> advertises a 30-day recovery story on an install that shortened it.
+
+> **Why this changed (W-ms5tb6ha016fda10).** Protected quarantines — the ones
+> whose dirty WIP could NOT be snapshotted to a durable
+> `refs/minions/quarantine-wip/*` ref — were kept forever by *both*
+> `reconcileStaleWorktreeDirectories` and `pruneOrphanWorktrees`. That is how a
+> busy install accumulates quarantine residue with no expiry (20 such dirs
+> observed on 2026-07-29). Every existing protection still applies first: a live
+> SQL claim, a pool member, a keep_processes / managed_spawn cwd anchor, a
+> detected CWD holder, or a missing ownership marker all keep the directory
+> regardless of age, and `isWorktreePathLive` still fails OPEN.
 
 `runCleanup` delegates its configured-root orphan pass to the same reconciler;
 the former fail-open duplicate scan and generic age/count-cap deletion are
@@ -406,6 +581,110 @@ an expected `skipped` outcome, not a failed removal, and is retried on the next
 sweep after the dispatch becomes terminal. Aggregate logs report mutually
 meaningful `removed`, `skipped`, `preserved`, and `failed` counts.
 
+### Inventory / diagnostics — `GET /api/worktree-inventory`
+
+The sweeps above tell you what they *did*. `engine/worktrees/inventory.js`
+answers the question an operator actually asks when a worktree root has 142
+directories and 6 registrations: **what are all these, and which will the engine
+ever reclaim?** It is strictly read-only — no git mutation, no filesystem writes,
+no removal — and classifies every directory under each project's worktree root
+plus every out-of-root git registration into disjoint buckets:
+
+| Bucket | Meaning | Engine-removable? |
+|---|---|---|
+| `active` | a non-terminal dispatch claims the path (`isWorktreePathLive`) | never |
+| `pooled` | warm worktree-pool entry | never (while pooled) |
+| `anchored` | held by a live `keep_processes` / `managed_spawn` cwd | never |
+| `registered` | known to `git worktree list`, not otherwise claimed | only when unlocked + unclaimed |
+| `ownedOrphan` | carries `.minions-worktree`, unregistered, not live | **yes — this is the reclaimable backlog** |
+| `quarantined` (+ `quarantinedProtected`) | `-quarantine-<ts>` recovery artifact | after its retention window |
+| `foreignUnmanaged` | no ownership marker | **never** |
+
+The response also carries `recoverableRefs` (`refs/minions/quarantine{,-wip}/*`
+counts + their retention window, so operators can see what is still recoverable)
+and `avoidedAllocations` — how many dispatches ran with
+`executionSurface: "control-plane"` and therefore never allocated a checkout.
+That count is **derived from the dispatch records themselves**, so there is no
+counter to migrate, reset, or drift out of sync.
+
+Every probe fails safe in the direction of *not* reporting reclaimable work: an
+`isWorktreePathLive` throw classifies as `active`, an unreadable git registry
+sets `registryReadable: false` and reports owned dirs as `registered` (never
+`ownedOrphan`), and both surface an entry in `errors[]`.
+
+Liveness is resolved **once per build** via `shared.listLiveWorktreePaths`
+rather than one dispatch-table scan per directory. That helper is read-only and
+deliberately separate from `isWorktreePathLive`: destructive wipe sites keep the
+per-path guard with its `excludeDispatchId` / `excludeSpawnLeaseId` semantics.
+Its failure contract is identical — `{ ok: false }` means "unknown", and the
+inventory then treats every path as `active`.
+
+#### The count contract — `engine.worktreeCount` (W-ms9q3dql00vxd4a8)
+
+The buckets roll up into a `lifecycle` split that is the **only** supported
+source for a headline "how many worktrees does Minions have?" number:
+
+| Field | Meaning |
+|---|---|
+| `lifecycle.owned` | everything the engine's lifecycle created and is responsible for: `active` + `pooled` + `anchored` + `registered` + `ownedOrphan` + `quarantined` |
+| `lifecycle.foreignUnmanaged` | operator scratch checkouts and unclassifiable residue — reported, never deleted, never folded into the headline |
+| `lifecycle.total` | every directory scanned |
+
+`GET /api/status` publishes `engine.worktreeCount` (`lifecycle.owned`) and
+`engine.worktreeForeignCount` (`lifecycle.foreignUnmanaged`) from this same
+classifier, so the dashboard chip and the inventory endpoint can never disagree.
+The previous metric was a raw filesystem walk that counted any directory
+containing a `.git` entry, which reported an operator's hand-made review scratch
+checkout as an active Minions worktree (observed: 18 "worktrees", 3 engine-owned)
+while missing engine residue whose `.git` pointer was already gone.
+**Do not reintroduce a second classifier** — extend `inventory.js` instead.
+
+Directories are also de-duplicated **across** projects, so two projects whose
+`worktreeRoot` resolves to the same directory classify it once. The second
+project reports `sharedRoot: true`.
+
+#### The scan must not block the dashboard event loop
+
+`_scanWorktreeCount` runs inside the dashboard process every
+`worktreeCountCacheTtl` (30 s), and `/api/status` must not block the event loop
+for more than ~100 ms or it starves Command Center's SSE stream. Classification
+is a tight loop over N directories, so **one synchronous probe per directory is
+enough to blow that budget** — and N is dominated by `foreignUnmanaged` residue
+the engine deliberately never reclaims, so it grows without bound.
+
+`buildWorktreeInventoryAsync` is therefore the only entry point a server may
+call. It pays **every** cost up front and asynchronously — the
+`git worktree list` shell-out per project, the worktree-root directory
+listings, the per-directory ownership / protected-quarantine marker probes, and
+the symlink resolution of both the candidate paths and the `keep_processes` /
+`managed_spawn` anchor cwds — then hands the results to the same synchronous
+classifier, which does pure in-memory lookups. `buildWorktreeInventory` stays
+synchronous and directly unit-testable; when a pre-read is absent or a single
+probe failed, it transparently falls back to the real synchronous helper, so a
+partial pre-read can only make the scan cheaper, never change a classification.
+
+Two footguns this encodes:
+
+- **Never call `shared.isPathInsideOrEqual` inside a per-directory loop.** It
+  symlink-resolves *both* sides on every call, so anchor matching used to cost
+  O(directories × anchors) `realpath` syscalls. Measured on a 150-directory root
+  with three live process anchors: 105–120 ms of contiguous synchronous work and
+  130 ms of event-loop stall per refresh. Resolve once with
+  `shared.pathForComparisonAsync` and compare via
+  `shared.isResolvedPathInsideOrEqual`.
+- **Resolve both sides of a comparison with the same twin.**
+  `fs.realpathSync.native` can canonicalize case on a case-insensitive volume
+  where the promises API does not, so mixing the sync and async resolvers could
+  make an already-resolved path miscompare on macOS.
+
+`test/unit/worktree-lifecycle-bounded-gc.test.js` guards this behaviourally: it
+counts real synchronous `fs` calls during a live
+`buildWorktreeInventoryAsync` run and asserts zero, and asserts the async and
+synchronous paths classify an identical tree identically. A source-string
+assertion is not sufficient here — the original guard was silently neutralised
+by re-anchoring it onto a thin delegating wrapper, where a "no sync filesystem
+calls" assertion is vacuously true.
+
 ### Scratch-dir skip and bounded reaper (PR #388)
 
 The worktree root (`<localPath>/../worktrees/`) is a **shared namespace**: it
@@ -423,7 +702,7 @@ infra / scratch, never a managed worktree. Engine-created worktree dir names
 never start with `.`. The predicate `shared.isWorktreeRootInfraEntry(name)`
 (returns `true` when `name` starts with `.`) is now applied at **both**
 on-disk enumeration sites — `cleanup.js` worktree scan and
-`worktree-gc.js#pruneOrphanWorktrees` — so the sweep is O(1) on scratch size
+`engine/worktrees/gc.js#pruneOrphanWorktrees` — so the sweep is O(1) on scratch size
 regardless of backlog.
 
 **Bounded TTL reaper (`cleanup.js#reapAgentScratch`).** Skipping scratch
@@ -467,7 +746,7 @@ Previously unmarked paths are never marked by this recovery.
 
 `engine.worktreeMinFreeSpaceBytes` defaults to 10 GiB (`0` disables) and is
 editable in Dashboard -> Settings -> Worker Pool & Worktrees. Allocation and a
-second spawn-time race check call `engine/worktree-preflight.js` before any
+second spawn-time race check call `engine/worktrees/preflight.js` before any
 `git worktree add`. The probe uses the nearest existing ancestor when the
 configured worktree root has not been created yet.
 
@@ -481,7 +760,7 @@ on every allocation tick.
 
 ### Locked-`initializing` missing-dir reclaim (W-mqifblkf00149df5)
 
-A third pruner, `worktree-gc.js#reclaimMissingDirWorktrees`, closes a
+A third pruner, `engine/worktrees/gc.js#reclaimMissingDirWorktrees`, closes a
 branch-bricking gap the two scanners above are blind to. An interrupted
 `git worktree add` (engine crash / kill during git's `initializing` lock
 phase) leaves a `.git/worktrees/<id>/` ADMIN entry behind — often carrying
@@ -552,8 +831,8 @@ Windows-only; three layers, each fail-open:
   dispatch that owned this worktree path (discovered via
   `_findTerminalWorktreeOwners`, which reads the dispatch store's non-live
   sections), look up its recorded root PID from the PID file
-  (`shared.findDispatchPidFile`) and kill that PID plus its full descendant
-  tree (`shared.listProcessDescendants` → `shared.killImmediate`). Because it
+  (`shared.findDispatchPidFile`) and terminate that PID plus its full descendant
+  tree through the registered ownership-aware gateway. Because it
   only ever kills a tree the engine itself spawned for THIS path, it can never
   touch a live sibling.
 - **Layer 2 — CWD-scan reap (fallback for orphans the map / PID files no
@@ -585,7 +864,7 @@ Windows-only; three layers, each fail-open:
 
 **Two call sites (W-mqinlicl §6).** `_reapWorktreeHolders` runs both BEFORE the
 quarantine rename (`_quarantineDirtyWorktree`) AND at **dispatch-end GC**
-(`worktree-gc.js#gcDispatchWorktreeIfOrphan`, via the injected `reapHolders`
+(`engine/worktrees/gc.js#gcDispatchWorktreeIfOrphan`, via the injected `reapHolders`
 hook) right before `shared.removeWorktree`. The dispatch-end call passes the
 ending dispatch's id as `ownerDispatchId` (terminal-owner discovery misses it —
 the owner is still `active`) plus `excludeDispatchId`, so a dispatch always

package/engine/README.md

@@ -0,0 +1,46 @@
+# Engine Source Layout
+
+`engine/` contains the Minions orchestration implementation and the live
+installation's generated runtime files. Tracked JavaScript belongs to a source
+domain below; generated databases, logs, PID files, diagnostics, completions,
+and QA artifacts are runtime state and must remain ignored.
+
+## Source domains
+
+| Directory | Responsibility |
+|---|---|
+| `agents/` | Agent spawning, worker pools, LLM/model integration, playbooks, steering, and harness context |
+| `ado/` | Azure DevOps polling, auth, comments, status, and Git credentials |
+| `api/` | API validation, settings validation, and Constellation bridge handling |
+| `api-contracts/` | Dashboard route request/response contracts |
+| `core/` | Shared constants/utilities, queries, feature resolution, safe expressions, and trust fences |
+| `db/` | SQLite connection, schema, migrations, and DB utilities |
+| `memory/` | Consolidation, KB sweeps, retrieval, review learning, and promotion |
+| `observability/` | Memory diagnostics, engine logs, and metrics stores |
+| `operations/` | CLI commands, status checks, and distribution metadata |
+| `orchestration/` | Dispatch, lifecycle, scheduling, routing, pipelines, meetings, cleanup, and timeouts |
+| `persistence/` | Runtime-state stores, DB events, small-state stores, and state operations |
+| `planning/` | Projects, PRDs, work items, issues, validation, and area resolution |
+| `processes/` | Managed/kept processes, launchers, sweeps, and cross-platform process utilities |
+| `providers/` | GitHub and provider-neutral PR comments, reconciliation, cloning, patching, and fix targeting |
+| `qa/` | QA sessions, runs, runbooks, PRD bridge, and process validation |
+| `qa-runners/` | QA runner registry and built-in Playwright/Maestro adapters |
+| `recovery/` | Restart health verification, in-process supervision, and OS-scheduled watchdog recovery |
+| `runtimes/` | Versioned Claude, Copilot, and Codex runtime adapters |
+| `watches/` | Watch registry, action handlers, and SQL store |
+| `worktrees/` | Worktree creation, live checkout, pool, preflight, quarantine, and GC |
+
+No tracked JavaScript files belong directly under `engine/`; source modules
+must live in the domain directories above.
+
+## Rules for physical moves
+
+1. Do not add compatibility wrappers unless an external import contract is
+   known. Track any wrapper in `docs/deprecated.json` with a removal gate.
+2. Update source comments, docs, package contents, direct unit imports, and
+   source-inspection assertions in the same change.
+3. Audit every executable path used by spawn/task-scheduler code on Windows,
+   Linux, and macOS.
+4. Never move generated runtime state as part of a source-layout refactor.
+5. Validate domain tests, syntax/lint, package contents, and the broader unit
+   suite after any future physical move.

package/engine/{ado-comment.js → ado/comment.js}

@@ -1,9 +1,9 @@
 /**
- * engine/ado-comment.js — Azure DevOps mirror of engine/gh-comment.js.
+ * engine/ado/comment.js — Azure DevOps mirror of engine/providers/gh-comment.js.
  *
  * Posts a minions PR-comment thread to ADO with the same hidden marker, the
  * same deterministic brand link as the GitHub path — because both are produced by the ONE neutral
- * builder `engine/comment-format.js#buildMinionsCommentBody`. This module owns
+ * builder `engine/providers/comment-format.js#buildMinionsCommentBody`. This module owns
  * only the ADO transport (REST `…/threads` POST) and ADO-specific input
  * validation (orgBase / project / repositoryId / prNumber).
  *
@@ -15,7 +15,7 @@
  * brand trailer hyperlinked.
  *
  * Auth: a bearer token from `az account get-access-token` (via
- * engine/ado-token.js#acquireAdoToken), threaded as `Authorization: Bearer`.
+ * engine/ado/token.js#acquireAdoToken), threaded as `Authorization: Bearer`.
  * Both the token acquisition and the HTTP transport are injectable so the unit
  * tests run without network or `az`.
  *
@@ -33,8 +33,8 @@
  * changes) stay `active` so they block/notify the author.
  */
 
-const { buildMinionsCommentBody } = require('./comment-format');
-const { acquireAdoToken } = require('./ado-token');
+const { buildMinionsCommentBody } = require('../providers/comment-format');
+const { acquireAdoToken } = require('./token');
 
 const ADO_API_VERSION = '7.1';
 
@@ -83,7 +83,7 @@ function buildThreadsUrl({ orgBase, project, repositoryId, prNumber }) {
 }
 
 /**
- * Default token acquisition — thin wrapper over engine/ado-token.js so callers
+ * Default token acquisition — thin wrapper over engine/ado/token.js so callers
  * can inject a stub in tests. Returns the bearer token string or throws.
  */
 async function _defaultAcquireToken() {
@@ -94,7 +94,7 @@ async function _defaultAcquireToken() {
 /**
  * Post a minions PR comment to an Azure DevOps pull request.
  *
- * Mirrors engine/gh-comment.js#postPrComment: builds the marked +
+ * Mirrors engine/providers/gh-comment.js#postPrComment: builds the marked +
  * brand-linked body via the shared builder, then POSTs a new active thread.
  *
  * @param {object} args
@@ -181,7 +181,7 @@ module.exports = {
   ADO_THREAD_STATUS_ACTIVE,
   ADO_THREAD_STATUS_CLOSED,
   // Re-export the neutral builder so ADO callers have a single import surface,
-  // mirroring engine/gh-comment.js.
+  // mirroring engine/providers/gh-comment.js.
   buildMinionsCommentBody,
   // Internal validators exported for tests.
   _validateOrgBase,

package/engine/{ado-git-auth.js → ado/git-auth.js}

@@ -1,5 +1,5 @@
 /**
- * engine/ado-git-auth.js — W-mpcuc8i80003a7b3
+ * engine/ado/git-auth.js — W-mpcuc8i80003a7b3
  *
  * Inject a per-invocation Authorization: Bearer <token> http.extraHeader into
  * git ops that touch an ADO origin. Solves the headless-dispatch failure mode
@@ -8,7 +8,7 @@
  *
  *   fatal: could not read Username for 'https://office.visualstudio.com'
  *
- * The token is sourced via the existing `engine/ado-token.js#acquireAdoTokenSync`
+ * The token is sourced via `engine/ado/token.js#acquireAdoTokenSync`
  * helper (az CLI first, azureauth fallback). We cache it for 30 minutes and
  * back off acquisition retries for 10 minutes on failure so we don't hammer
  * az / azureauth on every git op during an outage.
@@ -39,8 +39,8 @@
  * Migrate to env-form if argv exposure becomes a real concern.
  */
 
-const shared = require('./shared');
-const adoToken = require('./ado-token');
+const shared = require('../core/shared');
+const adoToken = require('./token');
 const { log } = shared;
 
 const TOKEN_TTL_MS = 30 * 60 * 1000;     // 30 min — matches gh-token cadence