@yemi33/minions 0.1.2448 → 0.1.2449
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/cli-api-client.js +1 -1
- package/bin/install-internal-minions.js +1382 -44
- package/bin/install-layout.js +150 -0
- package/bin/minions.js +460 -167
- package/dashboard/docs/typography.md +65 -12
- package/dashboard/js/command-center.js +58 -4
- package/dashboard/js/detail-panel.js +36 -0
- package/dashboard/js/memory-panel.js +59 -12
- package/dashboard/js/qa.js +179 -20
- package/dashboard/js/refresh.js +148 -12
- package/dashboard/js/render-dispatch.js +3 -4
- package/dashboard/js/render-inbox.js +2 -2
- package/dashboard/js/render-other.js +3 -3
- package/dashboard/js/render-pipelines.js +14 -0
- package/dashboard/js/render-plans.js +57 -9
- package/dashboard/js/render-prd.js +132 -23
- package/dashboard/js/render-prs.js +195 -166
- package/dashboard/js/render-schedules.js +63 -3
- package/dashboard/js/render-utils.js +3 -3
- package/dashboard/js/render-watches.js +19 -3
- package/dashboard/js/render-work-items.js +238 -30
- package/dashboard/js/settings.js +205 -54
- package/dashboard/js/utils.js +51 -1
- package/dashboard/pages/home.html +1 -1
- package/dashboard/pages/qa.html +1 -16
- package/dashboard/pages/work.html +40 -0
- package/dashboard/shared/cc-limits.js +79 -0
- package/dashboard/shared/pr-filters.js +21 -38
- package/dashboard/shared/project-git-summary.js +1 -1
- package/dashboard/shared/record-filters.js +169 -0
- package/dashboard/shared/watches-source.js +1 -1
- package/dashboard/shared/welcome-popup.js +1 -1
- package/dashboard/shared/wi-filters.js +302 -0
- package/dashboard/slim/body.html +1 -0
- package/dashboard/slim/js/command-send.js +26 -0
- package/dashboard/slim/js/modals-tiles.js +380 -39
- package/dashboard/slim/js/status.js +13 -21
- package/dashboard/slim/layout.html +1 -0
- package/dashboard/slim/panel-bootstrap.js +6 -2
- package/dashboard/slim/styles.css +38 -0
- package/dashboard/styles.css +159 -55
- package/dashboard-build.js +13 -2
- package/dashboard.js +956 -423
- package/docs/README.md +11 -6
- package/docs/api-errors.md +2 -2
- package/docs/architecture-review-2026-07-09.md +1 -1
- package/docs/architecture.excalidraw +2 -2
- package/docs/auto-discovery.md +18 -9
- package/docs/branch-derivation.md +4 -4
- package/docs/capture-demos.js +39 -2
- package/docs/ci-runner-canary.md +123 -0
- package/docs/claude-md-propagation.md +2 -2
- package/docs/cloud-agent-dispatch.md +204 -0
- package/docs/command-center.md +7 -7
- package/docs/completion-reports.md +43 -20
- package/docs/constants.md +10 -3
- package/docs/constellation-bridge.md +134 -6
- package/docs/constellation-style-telemetry.md +4 -4
- package/docs/contracts/capability-protocol.v1.json +165 -0
- package/docs/cooldown-merge-semantics.md +12 -12
- package/docs/copilot-cli-schema.md +7 -7
- package/docs/cross-repo-plans.md +10 -10
- package/docs/dead-code-audit-retractions.md +5 -5
- package/docs/default-branch-ci.md +173 -0
- package/docs/deprecated.json +31 -31
- package/docs/design-inbox-entries-schema.md +3 -3
- package/docs/design-language.md +1051 -0
- package/docs/design-state-storage.md +11 -11
- package/docs/diagnostics-crash-reports.md +9 -9
- package/docs/diagnostics-memory.md +5 -5
- package/docs/documentation-audit-2026-07-09.md +7 -7
- package/docs/engine-restart.md +90 -5
- package/docs/harness-mode.md +1 -1
- package/docs/internal-install.md +338 -39
- package/docs/kb-dedup-duplicate-pair-investigation.md +5 -5
- package/docs/kb-pr3223-cascade-archiving.md +1 -1
- package/docs/kb-pr696-merge-conflict-docs.md +6 -6
- package/docs/kb-sweep.md +35 -35
- package/docs/keep-processes.md +1 -1
- package/docs/live-checkout-mode.md +30 -30
- package/docs/managed-spawn.md +18 -14
- package/docs/named-agents.md +7 -7
- package/docs/plan-lifecycle.md +69 -2
- package/docs/pr-author-identity.md +114 -0
- package/docs/pr-auto-fix-dispatch.md +19 -4
- package/docs/pr-comment-followup.md +6 -6
- package/docs/pr-review-fix-loop.md +59 -10
- package/docs/process-termination.md +40 -0
- package/docs/proposals/repo-pool-for-live-checkout.md +13 -13
- package/docs/qa-runbook-lifecycle.md +367 -17
- package/docs/qa-runbooks.md +3 -3
- package/docs/rfc-completion-json.md +18 -18
- package/docs/runtime-adapters.md +26 -21
- package/docs/security.md +6 -6
- package/docs/self-improvement.md +4 -4
- package/docs/shared-lifecycle-module-map.md +473 -472
- package/docs/skills.md +52 -3
- package/docs/slim-ux/concepts.md +121 -116
- package/docs/specs/agent-configurability.md +18 -18
- package/docs/specs/agent-rename.md +18 -18
- package/docs/team-memory.md +38 -21
- package/docs/timeouts-and-liveness.md +118 -10
- package/docs/tutorials/01-install-and-connect.md +1 -1
- package/docs/watches.md +40 -39
- package/docs/workspace-manifests.md +4 -4
- package/docs/worktree-lifecycle.md +293 -14
- package/engine/README.md +46 -0
- package/engine/{ado-comment.js → ado/comment.js} +8 -8
- package/engine/{ado-git-auth.js → ado/git-auth.js} +4 -4
- package/engine/{ado.js → ado/index.js} +417 -63
- package/engine/{ado-status.js → ado/status.js} +6 -8
- package/engine/{ado-token.js → ado/token.js} +1 -1
- package/engine/{acp-transport.js → agents/acp-transport.js} +62 -22
- package/engine/{agent-worker-pool.js → agents/agent-worker-pool.js} +17 -8
- package/engine/{cc-worker-pool.js → agents/cc-worker-pool.js} +16 -6
- package/engine/{claude-md-context.js → agents/claude-md-context.js} +5 -5
- package/engine/{harness-context.js → agents/harness-context.js} +5 -5
- package/engine/{harness.js → agents/harness.js} +3 -3
- package/engine/{llm.js → agents/llm.js} +18 -14
- package/engine/{model-discovery.js → agents/model-discovery.js} +2 -2
- package/engine/{playbook.js → agents/playbook.js} +155 -22
- package/engine/{pooled-agent-process.js → agents/pooled-agent-process.js} +14 -12
- package/engine/{preflight.js → agents/preflight.js} +29 -10
- package/engine/{spawn-agent.js → agents/spawn-agent.js} +25 -14
- package/engine/{spawn-phase-watchdog.js → agents/spawn-phase-watchdog.js} +16 -7
- package/engine/{steering.js → agents/steering.js} +5 -5
- package/engine/{tools-inventory.js → agents/tools-inventory.js} +2 -2
- package/engine/{agent-api-validation.js → api/agent-api-validation.js} +2 -2
- package/engine/{api-validation.js → api/api-validation.js} +1 -1
- package/engine/api/bridge.js +787 -0
- package/engine/{cc-api-validation.js → api/cc-api-validation.js} +1 -1
- package/engine/api/companion.js +560 -0
- package/engine/{content-api-validation.js → api/content-api-validation.js} +2 -2
- package/engine/{pr-issue-validation.js → api/pr-issue-validation.js} +33 -6
- package/engine/{settings-validation.js → api/settings-validation.js} +32 -4
- package/engine/api-contracts/agent-content.js +4 -4
- package/engine/api-contracts/capability-manifest.js +236 -0
- package/engine/api-contracts/capability-protocol.js +333 -0
- package/engine/api-contracts/cc-ops.js +1 -1
- package/engine/api-contracts/config-runtime.js +5 -0
- package/engine/api-contracts/core.js +28 -1
- package/engine/api-contracts/index.js +100 -0
- package/engine/api-contracts/orchestration.js +18 -5
- package/engine/api-contracts/pull-requests.js +37 -6
- package/engine/api-contracts/qa-process.js +29 -6
- package/engine/api-contracts/work-plan-prd.js +21 -1
- package/engine/cloud/contract.js +212 -0
- package/engine/cloud/index.js +159 -0
- package/engine/{execution-model.js → core/execution-model.js} +1 -1
- package/engine/{features.js → core/features.js} +4 -4
- package/engine/{operator-identity.js → core/operator-identity.js} +1 -1
- package/engine/{queries.js → core/queries.js} +201 -36
- package/engine/{safe-expr.js → core/safe-expr.js} +1 -1
- package/engine/{shared.js → core/shared.js} +1637 -175
- package/engine/{stdio-timestamps.js → core/stdio-timestamps.js} +1 -1
- package/engine/{untrusted-fence.js → core/untrusted-fence.js} +3 -3
- package/engine/db/index.js +11 -2
- package/engine/db/migrations/002-dispatches.js +3 -3
- package/engine/db/migrations/003-work-items.js +1 -1
- package/engine/db/migrations/004-pull-requests.js +1 -1
- package/engine/db/migrations/006-metrics.js +1 -1
- package/engine/db/migrations/007-watches.js +2 -2
- package/engine/db/migrations/008-small-state.js +1 -1
- package/engine/db/migrations/009-qa.js +1 -1
- package/engine/db/migrations/010-pr-links.js +1 -1
- package/engine/db/migrations/011-remaining-state.js +1 -1
- package/engine/db/migrations/012-steering-deliveries.js +2 -2
- package/engine/db/migrations/013-backfill-broken-note-links.js +1 -1
- package/engine/db/migrations/014-pr-fix-target-prefs.js +2 -2
- package/engine/db/migrations/015-plans-prds.js +0 -0
- package/engine/db/migrations/018-sql-only-cutover.js +2 -2
- package/engine/db/migrations/021-archived-work-items.js +1 -1
- package/engine/db/migrations/022-global-cc-session.js +1 -1
- package/engine/db/migrations/023-engine-state.js +1 -1
- package/engine/db/migrations/025-malformed-work-item-phantoms.js +1 -1
- package/engine/db/migrations/027-review-learning-lifecycle.js +1 -1
- package/engine/db/migrations/029-repair-reused-versions.js +20 -0
- package/engine/db/migrations/031-pr-author-identity.js +137 -0
- package/engine/{consolidation.js → memory/consolidation.js} +6 -6
- package/engine/{kb-sweep-runner.js → memory/kb-sweep-runner.js} +2 -2
- package/engine/{kb-sweep.js → memory/kb-sweep.js} +9 -7
- package/engine/{memory-retrieval.js → memory/memory-retrieval.js} +46 -4
- package/engine/{memory-store.js → memory/memory-store.js} +3 -3
- package/engine/{promotion.js → memory/promotion.js} +3 -3
- package/engine/{review-learning-backfill.js → memory/review-learning-backfill.js} +6 -6
- package/engine/{review-learning.js → memory/review-learning.js} +10 -5
- package/engine/{diagnostics-memory.js → observability/diagnostics-memory.js} +1 -1
- package/engine/{logs-store.js → observability/logs-store.js} +5 -5
- package/engine/{metrics-store.js → observability/metrics-store.js} +4 -4
- package/engine/{check-status.js → operations/check-status.js} +3 -3
- package/engine/{cli.js → operations/cli.js} +271 -113
- package/engine/{distribution.js → operations/distribution.js} +5 -6
- package/engine/{cleanup.js → orchestration/cleanup.js} +72 -45
- package/engine/{cooldown.js → orchestration/cooldown.js} +5 -5
- package/engine/{dispatch-events.js → orchestration/dispatch-events.js} +2 -2
- package/engine/{dispatch.js → orchestration/dispatch.js} +129 -36
- package/engine/orchestration/failed-scheduled-cleanup.js +274 -0
- package/engine/{lifecycle.js → orchestration/lifecycle.js} +198 -90
- package/engine/{meeting.js → orchestration/meeting.js} +6 -16
- package/engine/{pipeline.js → orchestration/pipeline.js} +12 -12
- package/engine/{pre-dispatch-eval.js → orchestration/pre-dispatch-eval.js} +10 -9
- package/engine/{routing.js → orchestration/routing.js} +3 -3
- package/engine/{schedule-bootstrap.js → orchestration/schedule-bootstrap.js} +4 -4
- package/engine/{scheduler.js → orchestration/scheduler.js} +38 -8
- package/engine/{timeout.js → orchestration/timeout.js} +158 -109
- package/engine/{db-events.js → persistence/db-events.js} +2 -2
- package/engine/{dispatch-store.js → persistence/dispatch-store.js} +7 -7
- package/engine/{inbox-store.js → persistence/inbox-store.js} +2 -2
- package/engine/{note-link-backfill.js → persistence/note-link-backfill.js} +4 -4
- package/engine/{pr-fix-target-store.js → persistence/pr-fix-target-store.js} +8 -8
- package/engine/{pull-requests-store.js → persistence/pull-requests-store.js} +21 -7
- package/engine/{small-state-store.js → persistence/small-state-store.js} +31 -31
- package/engine/persistence/state-operations.js +350 -0
- package/engine/{steering-store.js → persistence/steering-store.js} +6 -6
- package/engine/{issues.js → planning/issues.js} +2 -2
- package/engine/{plan-prd-validation.js → planning/plan-prd-validation.js} +8 -2
- package/engine/planning/prd-result-sidecar.js +190 -0
- package/engine/{prd-store.js → planning/prd-store.js} +17 -17
- package/engine/{project-discovery.js → planning/project-discovery.js} +5 -5
- package/engine/{projects.js → planning/projects.js} +10 -10
- package/engine/{resolve-area.js → planning/resolve-area.js} +1 -1
- package/engine/{work-item-validation.js → planning/work-item-validation.js} +39 -3
- package/engine/{work-items-store.js → planning/work-items-store.js} +29 -21
- package/engine/{keep-process-sweep.js → processes/keep-process-sweep.js} +57 -17
- package/engine/{managed-spawn-launcher.js → processes/managed-spawn-launcher.js} +3 -3
- package/engine/{managed-spawn.js → processes/managed-spawn.js} +97 -46
- package/engine/{process-utils.js → processes/process-utils.js} +599 -55
- package/engine/{abandoned-pr-reconciliation.js → providers/abandoned-pr-reconciliation.js} +17 -7
- package/engine/{comment-classifier.js → providers/comment-classifier.js} +85 -17
- package/engine/{comment-format.js → providers/comment-format.js} +5 -5
- package/engine/{gh-comment.js → providers/gh-comment.js} +15 -15
- package/engine/{gh-token.js → providers/gh-token.js} +4 -4
- package/engine/{github.js → providers/github.js} +131 -54
- package/engine/{pr-action.js → providers/pr-action.js} +13 -12
- package/engine/{pr-clone-keep.js → providers/pr-clone-keep.js} +7 -7
- package/engine/{pr-devbox.js → providers/pr-devbox.js} +6 -6
- package/engine/{pr-fix-target.js → providers/pr-fix-target.js} +13 -13
- package/engine/{pr-remote-patch.js → providers/pr-remote-patch.js} +4 -4
- package/engine/{pr-resolve.js → providers/pr-resolve.js} +7 -7
- package/engine/{pr-temp-clone.js → providers/pr-temp-clone.js} +5 -5
- package/engine/{pr-track.js → providers/pr-track.js} +11 -13
- package/engine/{shared-branch-pr-reconcile.js → providers/shared-branch-pr-reconcile.js} +4 -4
- package/engine/qa/auto-prd-qa.js +313 -0
- package/engine/{qa-from-prd.js → qa/from-prd.js} +42 -12
- package/engine/qa/prd-session.js +240 -0
- package/engine/{qa-process-validation.js → qa/process-validation.js} +14 -9
- package/engine/{qa-runbooks.js → qa/runbooks.js} +1 -1
- package/engine/{qa-runs.js → qa/runs.js} +286 -15
- package/engine/{qa-sessions.js → qa/sessions.js} +595 -49
- package/engine/qa/visual-journey.js +654 -0
- package/engine/{qa-runners.js → qa-runners/index.js} +7 -7
- package/engine/qa-runners/maestro.js +3 -3
- package/engine/qa-runners/playwright.js +2 -2
- package/engine/{restart-health.js → recovery/restart-health.js} +48 -4
- package/engine/recovery/stop-stack.js +607 -0
- package/engine/{supervisor.js → recovery/supervisor.js} +105 -175
- package/engine/{watchdog.js → recovery/watchdog.js} +136 -13
- package/engine/runtimes/claude.js +14 -12
- package/engine/runtimes/codex.js +8 -6
- package/engine/runtimes/copilot.js +17 -16
- package/engine/{watch-actions.js → watches/actions.js} +13 -13
- package/engine/{watches.js → watches/index.js} +43 -32
- package/engine/{watches-store.js → watches/store.js} +4 -4
- package/engine/{create-pr-worktree.js → worktrees/create-pr.js} +1 -1
- package/engine/{worktree-gc.js → worktrees/gc.js} +70 -22
- package/engine/worktrees/inventory.js +671 -0
- package/engine/{live-checkout.js → worktrees/live-checkout.js} +4 -4
- package/engine/{worktree-pool.js → worktrees/pool.js} +2 -2
- package/engine/{worktree-preflight.js → worktrees/preflight.js} +1 -0
- package/engine/worktrees/quarantine-refs.js +173 -0
- package/engine.js +1137 -208
- package/minions.js +147 -77
- package/package.json +10 -6
- package/playbooks/_pr-description-audit.md +110 -78
- package/playbooks/build-fix-complex.md +2 -0
- package/playbooks/fix.md +16 -12
- package/playbooks/implement-shared.md +2 -0
- package/playbooks/implement.md +19 -20
- package/playbooks/plan-to-prd.md +18 -3
- package/playbooks/qa-session-draft.md +136 -1
- package/playbooks/qa-session-execute.md +80 -2
- package/playbooks/qa-session-setup.md +17 -1
- package/playbooks/qa-validate.md +1 -1
- package/playbooks/setup.md +2 -0
- package/playbooks/shared-rules.md +25 -32
- package/playbooks/templates/followup-dispatch.md +4 -3
- package/playbooks/verify.md +1 -1
- package/prompts/cc-system.md +19 -27
- package/watch-plugins/README.md +92 -0
- package/watch-plugins/ado-author-prs.js +336 -0
- package/watch-plugins/gh-author-prs.js +375 -0
- package/watch-plugins/http.js +474 -0
- package/watch-plugins/teams-channel.js +869 -0
- package/docs/dev-composite-workflow.md +0 -101
- package/docs/pr-screenshots/pr-886/after-single-header.png +0 -0
- package/docs/pr-screenshots/pr-886/before-duplicate-header.png +0 -0
- package/docs/pr-screenshots/pr-895/01-cancellation-reason-detail.png +0 -0
- package/docs/pr-screenshots/pr-899/worker-pool-worktrees-AFTER.png +0 -0
- package/docs/pr-screenshots/pr-899/worker-pool-worktrees-BEFORE.png +0 -0
- package/docs/pr-screenshots/pr-901/projects-tab-default.png +0 -0
- package/docs/pr-screenshots/pr-901/projects-tab-fmf-selected.png +0 -0
- package/docs/pr-screenshots/pr-916/model-picker-AFTER-crop.png +0 -0
- package/docs/pr-screenshots/pr-916/model-picker-AFTER.png +0 -0
- package/docs/pr-screenshots/pr-916/model-picker-BEFORE-crop.png +0 -0
- package/docs/pr-screenshots/pr-916/model-picker-BEFORE.png +0 -0
- package/docs/pr-screenshots/pr-916/model-picker-dropdown-AFTER.png +0 -0
- package/docs/pr-screenshots/pr-979/auto-fix-pane-AFTER.png +0 -0
- package/docs/pr-screenshots/pr-979/auto-fix-pane-BEFORE.png +0 -0
- package/docs/pr-screenshots/pr-985/pr-column-em-dash-AFTER.png +0 -0
- package/docs/pr-screenshots/pr-985/pr-column-em-dash-BEFORE.png +0 -0
- package/docs/visual-evidence-ci.md +0 -103
- package/engine/bridge.js +0 -379
- package/engine/quarantine-refs.js +0 -103
- package/engine/state-operations.js +0 -178
- /package/engine/{steering-constraints.js → agents/steering-constraints.js} +0 -0
package/docs/kb-sweep.md
CHANGED
|
@@ -4,7 +4,7 @@ How the `knowledge/` directory stays curated, deduplicated, and small enough to
|
|
|
4
4
|
|
|
5
5
|
## What the Sweep Does
|
|
6
6
|
|
|
7
|
-
The KB sweep is a multi-pass cycle implemented in [`engine/kb-sweep.js`](../engine/kb-sweep.js) that prunes duplicate, boilerplate, stale, and oversized entries from `knowledge/`. It runs asynchronously, archives (rather than deletes) what it removes, and writes a `_swept` frontmatter flag so the expensive rewrite pass is idempotent.
|
|
7
|
+
The KB sweep is a multi-pass cycle implemented in [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) that prunes duplicate, boilerplate, stale, and oversized entries from `knowledge/`. It runs asynchronously, archives (rather than deletes) what it removes, and writes a `_swept` frontmatter flag so the expensive rewrite pass is idempotent.
|
|
8
8
|
|
|
9
9
|
```
|
|
10
10
|
Entries in knowledge/<category>/*.md
|
|
@@ -25,7 +25,7 @@ Entries in knowledge/<category>/*.md
|
|
|
25
25
|
|
|
26
26
|
Cheap, deterministic. No LLM call.
|
|
27
27
|
|
|
28
|
-
For every entry, compute `sha256(normalize(content).slice(0,500) + ':' + content.length)` (source: [`engine/kb-sweep.js
|
|
28
|
+
For every entry, compute `sha256(normalize(content).slice(0,500) + ':' + content.length)` (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_hashEntry`). Group by hash. When two or more entries collide, keep the most recent (by `date:` frontmatter, then mtime) and archive the rest into `knowledge/_swept/` with a `<!-- swept: ... | reason: hash-duplicate of ... -->` header via `_archiveKbFile()` (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_archiveKbFile`, called from `_hashDedup`).
|
|
29
29
|
|
|
30
30
|
This catches near-identical re-postings across batches that the LLM pass might miss because they land in different batches.
|
|
31
31
|
|
|
@@ -35,9 +35,9 @@ Cheap, deterministic. No LLM call. This is the pass that actually stops `knowled
|
|
|
35
35
|
|
|
36
36
|
The highest-volume category (`project-notes`) is dominated by near-identical boilerplate — e.g. hundreds of `# Agent Failed: …` dumps that differ only by work-item id, date, and dispatch id (a single agent/failure-class pair like `dallas` / `live-checkout-failed` can account for hundreds of files). Hash dedup can't collapse them (the leading bytes differ) and the LLM batch pass can't either (they're scattered across dozens of 30-entry batches). Left alone they survive every other pass forever.
|
|
37
37
|
|
|
38
|
-
Pass 1.5 groups eligible entries by a **structural signature** and rolls any group of at least `minGroupSize` notes into a single digest, archiving the originals to `knowledge/_swept/` (source: [`engine/kb-sweep.js`](../engine/kb-sweep.js) `_structuralConsolidate`).
|
|
38
|
+
Pass 1.5 groups eligible entries by a **structural signature** and rolls any group of at least `minGroupSize` notes into a single digest, archiving the originals to `knowledge/_swept/` (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_structuralConsolidate`).
|
|
39
39
|
|
|
40
|
-
- **Signature** = `category :: agent :: kind :: time-bucket`. `kind` is derived from the note (source: [`engine/kb-sweep.js`](../engine/kb-sweep.js) `_consolidationKind`):
|
|
40
|
+
- **Signature** = `category :: agent :: kind :: time-bucket`. `kind` is derived from the note (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_consolidationKind`):
|
|
41
41
|
- `failure-<failureClass>` when the note carries a `failureClass:` (or `result: error`) — the agent-failure dumps.
|
|
42
42
|
- `noop` / `merge-conflict` when the title matches a no-op or merge-conflict fix report.
|
|
43
43
|
- Any note that doesn't match a boilerplate kind returns `null` and is **left untouched** — unique, valuable notes are never grouped.
|
|
@@ -58,9 +58,9 @@ Because originals are archived to `_swept/` (same 30-day retention as every othe
|
|
|
58
58
|
|
|
59
59
|
### Pass 2 — LLM Batch Sweep
|
|
60
60
|
|
|
61
|
-
Survivors are split into **changed** and **unchanged** entries before anything is sent to the LLM (source: [`engine/kb-sweep.js`](../engine/kb-sweep.js) `_classifyLlmScanFreshness`). An entry is `unchanged` only if it carries the `_llmScannedAt` frontmatter marker (`LLM_SCAN_FLAG_KEY`) AND its `mtime` hasn't advanced past that timestamp — the same mtime-gated pattern Pass 3 uses for `_swept`. Everything else (never scanned, or edited/reclassified since its last scan) is `changed`.
|
|
61
|
+
Survivors are split into **changed** and **unchanged** entries before anything is sent to the LLM (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_classifyLlmScanFreshness`). An entry is `unchanged` only if it carries the `_llmScannedAt` frontmatter marker (`LLM_SCAN_FLAG_KEY`) AND its `mtime` hasn't advanced past that timestamp — the same mtime-gated pattern Pass 3 uses for `_swept`. Everything else (never scanned, or edited/reclassified since its last scan) is `changed`.
|
|
62
62
|
|
|
63
|
-
Only the `changed` entries are batched to Claude Haiku in groups of `LLM_BATCH_SIZE = 30` (source: [`engine/kb-sweep.js
|
|
63
|
+
Only the `changed` entries are batched to Claude Haiku in groups of `LLM_BATCH_SIZE = 30` (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `LLM_BATCH_SIZE`) with **full content**. `unchanged` entries are still appended to every prompt as a lightweight roster (title/date/category only, no content) so cross-entry duplicate detection against the whole KB stays correct — a changed entry can still be flagged as a duplicate of an older, already-scanned one — without re-spending tokens re-analyzing entries nothing has touched. If there are zero `changed` entries, the pass makes no LLM calls at all. This fixed an unbounded-cost bug where every 4h sweep cycle re-sent the *entire* KB manifest regardless of what had changed since the prior cycle (source: PR #738).
|
|
64
64
|
|
|
65
65
|
Each batch prompt asks for three lists in JSON:
|
|
66
66
|
|
|
@@ -70,23 +70,23 @@ Each batch prompt asks for three lists in JSON:
|
|
|
70
70
|
| `reclassify` | Entries in the wrong category. Returned as `{index, from, to, reason}`. |
|
|
71
71
|
| `remove` | Stale or empty entries (boilerplate, "no changes needed", bail-out notes). Returned as `{index, reason}`. |
|
|
72
72
|
|
|
73
|
-
Each action archives the file via the same `_archiveKbFile()` helper used by Pass 1; reclassification rewrites the `category:` frontmatter line and moves the file into the new category directory (source: [`engine/kb-sweep.js
|
|
73
|
+
Each action archives the file via the same `_archiveKbFile()` helper used by Pass 1; reclassification rewrites the `category:` frontmatter line and moves the file into the new category directory (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_applyLlmPlan`).
|
|
74
74
|
|
|
75
|
-
**changedIdx guard.** All three actions are only actioned against indices the LLM actually content-analyzed this batch (i.e. members of `changedIdx`). `remove` and `reclassify` require their own referenced `index` to be in `changedIdx`; a `duplicates` pair is actioned if the pair references at least one index in `changedIdx` (a pair where every referenced index is roster-only, `unchangedIdx`, is skipped). Roster-only entries are sent to the LLM with title/date/category only (no content — see the "existing entries" roster section), so an index outside `changedIdx` cannot have been genuinely compared/analyzed this batch; acting on it would mean acting on a hallucinated or stale LLM reference rather than a real comparison (source: [`engine/kb-sweep.js`](../engine/kb-sweep.js) `_applyLlmPlan`, PR #739, W-mrc9y5y000063033).
|
|
75
|
+
**changedIdx guard.** All three actions are only actioned against indices the LLM actually content-analyzed this batch (i.e. members of `changedIdx`). `remove` and `reclassify` require their own referenced `index` to be in `changedIdx`; a `duplicates` pair is actioned if the pair references at least one index in `changedIdx` (a pair where every referenced index is roster-only, `unchangedIdx`, is skipped). Roster-only entries are sent to the LLM with title/date/category only (no content — see the "existing entries" roster section), so an index outside `changedIdx` cannot have been genuinely compared/analyzed this batch; acting on it would mean acting on a hallucinated or stale LLM reference rather than a real comparison (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_applyLlmPlan`, PR #739, W-mrc9y5y000063033).
|
|
76
76
|
|
|
77
|
-
Reclassification targets are validated against `shared.KB_CATEGORIES` (`architecture`, `conventions`, `project-notes`, `build-reports`, `reviews` — source: [`engine/shared.js`](../engine/shared.js) `KB_CATEGORIES`); unknown categories are silently dropped.
|
|
77
|
+
Reclassification targets are validated against `shared.KB_CATEGORIES` (`architecture`, `conventions`, `project-notes`, `build-reports`, `reviews` — source: [`engine/core/shared.js`](../engine/core/shared.js) `KB_CATEGORIES`); unknown categories are silently dropped.
|
|
78
78
|
|
|
79
|
-
If a batch returns invalid JSON or the runtime is unavailable, that batch is skipped with a warning and the rest of the sweep continues (source: [`engine/kb-sweep.js
|
|
79
|
+
If a batch returns invalid JSON or the runtime is unavailable, that batch is skipped with a warning and the rest of the sweep continues (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_llmBatchSweep`).
|
|
80
80
|
|
|
81
|
-
`_llmScannedAt` is stamped on every analyzed entry only at the very end of the whole sweep — after Pass 3's rewrite — to avoid a same-run mtime race with the rewrite pass's own content edit (source: [`engine/kb-sweep.js`](../engine/kb-sweep.js), stamping step called from `runKbSweep`).
|
|
81
|
+
`_llmScannedAt` is stamped on every analyzed entry only at the very end of the whole sweep — after Pass 3's rewrite — to avoid a same-run mtime race with the rewrite pass's own content edit (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js), stamping step called from `runKbSweep`).
|
|
82
82
|
|
|
83
83
|
### Pass 2.5 — Age-Based TTL Expiry (secondary safety net)
|
|
84
84
|
|
|
85
85
|
Cheap, deterministic. No LLM call. With Pass 1.5 now collapsing the bulk of the boilerplate structurally, this pass is a **secondary safety net**: it archives genuinely stale one-off notes that never clustered into a large enough group to be consolidated (e.g. a lone failure class that only occurred twice in a week). The dedup and rewrite passes can only reclaim *duplicate* or *empty* entries, so an age cutoff is still the backstop for long-tail one-offs.
|
|
86
86
|
|
|
87
|
-
For every surviving entry, the pass computes the entry's age and archives it to `knowledge/_swept/` when it exceeds that category's TTL (source: [`engine/kb-sweep.js`](../engine/kb-sweep.js) `_expireByAge`). Two safety properties:
|
|
87
|
+
For every surviving entry, the pass computes the entry's age and archives it to `knowledge/_swept/` when it exceeds that category's TTL (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_expireByAge`). Two safety properties:
|
|
88
88
|
|
|
89
|
-
- **Age is taken from the authored date** (`YYYY-MM-DD-` filename prefix / `date:` frontmatter), *not* mtime — so the Pass 3 rewrite touching a file does not reset its clock and let a stale entry live forever. When no date is parseable it falls back to mtime; when neither exists the entry is treated as fresh and never age-expired (source: [`engine/kb-sweep.js`](../engine/kb-sweep.js) `_entryAgeMs`).
|
|
89
|
+
- **Age is taken from the authored date** (`YYYY-MM-DD-` filename prefix / `date:` frontmatter), *not* mtime — so the Pass 3 rewrite touching a file does not reset its clock and let a stale entry live forever. When no date is parseable it falls back to mtime; when neither exists the entry is treated as fresh and never age-expired (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_entryAgeMs`).
|
|
90
90
|
- **Only categories with a positive TTL are eligible.** Durable categories (`architecture`, `conventions`) are absent from the default map and are never age-expired. Per-agent memory under `knowledge/agents/` is never even scanned into the manifest (it's not in `KB_CATEGORIES`), and pinned entries are excluded upstream — so none of them can be aged out.
|
|
91
91
|
|
|
92
92
|
The TTL map is `ENGINE_DEFAULTS.kbCategoryTtlDays`, overridable per-fleet via `config.engine.kbCategoryTtlDays` (merged over the defaults; set a category to `0` to disable expiry for it):
|
|
@@ -102,34 +102,34 @@ Because expiry archives to `_swept/` (subject to the same 30-day retention as ev
|
|
|
102
102
|
|
|
103
103
|
### Pass 3 — Per-Entry Rewrite
|
|
104
104
|
|
|
105
|
-
Survivors of Passes 1, 1.5, and 2 that are still on disk get rewritten one entry at a time, with up to `NORMALIZE_CONCURRENCY = 5` parallel workers (source: [`engine/kb-sweep.js
|
|
105
|
+
Survivors of Passes 1, 1.5, and 2 that are still on disk get rewritten one entry at a time, with up to `NORMALIZE_CONCURRENCY = 5` parallel workers (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `NORMALIZE_CONCURRENCY`). Consolidation digests are excluded (rewriting would destroy their roll-up table). Each remaining entry is sent to Haiku with a fixed template prompt that:
|
|
106
106
|
|
|
107
107
|
- Caps output at ~800 words.
|
|
108
108
|
- Forces the structure `## Summary` → `## Key Findings` → `## Action Items` (omit if none) → `## References` (omit if none).
|
|
109
109
|
- Preserves all `file:line` references and code snippets.
|
|
110
110
|
- Drops boilerplate (full dates, agent IDs in the body, narrative scaffolding).
|
|
111
111
|
|
|
112
|
-
After a successful rewrite, the entry's frontmatter gains a `_swept: <ISO timestamp>` key (source: [`engine/kb-sweep.js
|
|
112
|
+
After a successful rewrite, the entry's frontmatter gains a `_swept: <ISO timestamp>` key (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_rewritePass`).
|
|
113
113
|
|
|
114
114
|
## The `_llmScannedAt` Frontmatter Flag
|
|
115
115
|
|
|
116
|
-
Mirrors `_swept` but gates Pass 2 instead of Pass 3: an entry with `_llmScannedAt` set and an mtime that hasn't advanced past it is treated as `unchanged` and skipped from full-content LLM analysis on the next sweep (roster-only context still applies — see Pass 2 above). Editing an entry (or a category reclassify) bumps its mtime past the stamp, so the next sweep re-analyzes it. The key is exported as `LLM_SCAN_FLAG_KEY` (source: [`engine/kb-sweep.js
|
|
116
|
+
Mirrors `_swept` but gates Pass 2 instead of Pass 3: an entry with `_llmScannedAt` set and an mtime that hasn't advanced past it is treated as `unchanged` and skipped from full-content LLM analysis on the next sweep (roster-only context still applies — see Pass 2 above). Editing an entry (or a category reclassify) bumps its mtime past the stamp, so the next sweep re-analyzes it. The key is exported as `LLM_SCAN_FLAG_KEY` (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `LLM_SCAN_FLAG_KEY`).
|
|
117
117
|
|
|
118
118
|
## The `_swept` Frontmatter Flag
|
|
119
119
|
|
|
120
120
|
`_swept` makes Pass 3 idempotent across runs. Semantics:
|
|
121
121
|
|
|
122
|
-
- **Set** to `new Date().toISOString()` whenever the rewrite pass successfully rewrites the body (source: [`engine/kb-sweep.js
|
|
123
|
-
- **Skipped** by Pass 3 on subsequent sweeps as long as the file's `mtimeMs` is `<= sweptAt + 1000` ms (1 s grace for FS timestamp jitter — source: [`engine/kb-sweep.js
|
|
122
|
+
- **Set** to `new Date().toISOString()` whenever the rewrite pass successfully rewrites the body (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_rewritePass`).
|
|
123
|
+
- **Skipped** by Pass 3 on subsequent sweeps as long as the file's `mtimeMs` is `<= sweptAt + 1000` ms (1 s grace for FS timestamp jitter — source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_rewritePass`).
|
|
124
124
|
- **Re-processed** if the file is edited after the sweep flag was set (mtime exceeds the grace window).
|
|
125
125
|
|
|
126
126
|
Hash dedup (Pass 1) and the LLM sweep (Pass 2) ignore `_swept` — those passes act on metadata and similarity, not on whether the entry was previously rewritten.
|
|
127
127
|
|
|
128
|
-
The flag key is exported as `SWEPT_FLAG_KEY` for tests (source:
|
|
128
|
+
The flag key is exported as `SWEPT_FLAG_KEY` for tests (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `SWEPT_FLAG_KEY`, re-exported from `module.exports`).
|
|
129
129
|
|
|
130
130
|
## 30-Day Archive Retention
|
|
131
131
|
|
|
132
|
-
Archived entries land in `knowledge/_swept/` with their original filename (deduped via `shared.uniquePath()` if it collides). They're not deleted immediately — `_pruneOldSwept()` runs at the end of every sweep and removes any file whose `mtimeMs` is older than `SWEPT_RETENTION_MS = 30 * 24 * 60 * 60 * 1000` (30 days — source: [`engine/kb-sweep.js
|
|
132
|
+
Archived entries land in `knowledge/_swept/` with their original filename (deduped via `shared.uniquePath()` if it collides). They're not deleted immediately — `_pruneOldSwept()` runs at the end of every sweep and removes any file whose `mtimeMs` is older than `SWEPT_RETENTION_MS = 30 * 24 * 60 * 60 * 1000` (30 days — source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `SWEPT_RETENTION_MS`, pruning function `_pruneOldSwept`).
|
|
133
133
|
|
|
134
134
|
This gives humans a recovery window: if a sweep archives something useful, you have ~30 days to copy it back out of `_swept/` before it's permanently pruned.
|
|
135
135
|
|
|
@@ -139,10 +139,10 @@ The dashboard exposes two endpoints:
|
|
|
139
139
|
|
|
140
140
|
| Method | Path | Purpose |
|
|
141
141
|
|--------|------|---------|
|
|
142
|
-
| `POST` | `/api/knowledge/sweep` | Start a sweep in the background. Returns `202 { ok: true, started: true }` (source: [`dashboard.js
|
|
143
|
-
| `GET` | `/api/knowledge/sweep/status` | Poll `{ inFlight, startedAt, lastResult, lastCompletedAt }` (source: [`dashboard.js
|
|
142
|
+
| `POST` | `/api/knowledge/sweep` | Start a sweep in the background. Returns `202 { ok: true, started: true }` (source: [`dashboard.js`](../dashboard.js) route registry, handler `handleKnowledgeSweep`). |
|
|
143
|
+
| `GET` | `/api/knowledge/sweep/status` | Poll `{ inFlight, startedAt, lastResult, lastCompletedAt }` (source: [`dashboard.js`](../dashboard.js) route registry, handler `handleKnowledgeSweepStatus`). |
|
|
144
144
|
|
|
145
|
-
POST body is optional. Pass `{ "pinnedKeys": ["knowledge/conventions/foo.md", ...] }` to add request-level pins on top of `pinned.md`. Pinned entries are excluded from every pass (source: [`engine/kb-sweep.js
|
|
145
|
+
POST body is optional. Pass `{ "pinnedKeys": ["knowledge/conventions/foo.md", ...] }` to add request-level pins on top of `pinned.md`. Pinned entries are excluded from every pass (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `_runKbSweepImpl`).
|
|
146
146
|
|
|
147
147
|
```bash
|
|
148
148
|
# Trigger a sweep
|
|
@@ -156,9 +156,9 @@ curl http://localhost:7331/api/knowledge/sweep/status
|
|
|
156
156
|
|
|
157
157
|
### Concurrency Guard
|
|
158
158
|
|
|
159
|
-
Only one sweep runs at a time. The POST handler refuses to start a second sweep with `{ ok: true, alreadyRunning: true, startedAt }` while one is in flight (source: [`dashboard.js
|
|
159
|
+
Only one sweep runs at a time. The POST handler refuses to start a second sweep with `{ ok: true, alreadyRunning: true, startedAt }` while one is in flight (source: [`dashboard.js`](../dashboard.js) `handleKnowledgeSweep`).
|
|
160
160
|
|
|
161
|
-
The guard auto-releases when the runner PID is dead or the sweep exceeds `staleGuardMs(entryCount)` — `max(30 min, 1 s × entryCount)` (source: [`engine/kb-sweep.js
|
|
161
|
+
The guard auto-releases when the runner PID is dead or the sweep exceeds `staleGuardMs(entryCount)` — `max(30 min, 1 s × entryCount)` (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `staleGuardMs` + `readSweepLiveness`, consumed by [`dashboard.js`](../dashboard.js) `handleKnowledgeSweep`).
|
|
162
162
|
|
|
163
163
|
### Persistent State
|
|
164
164
|
|
|
@@ -175,13 +175,13 @@ Sweep liveness is coordinated through `engine/kb-sweep-state.json` so the dashbo
|
|
|
175
175
|
{ "status": "failed", "startedAt": ..., "completedAt": ..., "error": "..." }
|
|
176
176
|
```
|
|
177
177
|
|
|
178
|
-
The state file plus runner PID liveness is the source of truth; the detached runner has no dashboard-owned in-memory lifecycle (source: [`engine/kb-sweep.js
|
|
178
|
+
The state file plus runner PID liveness is the source of truth; the detached runner has no dashboard-owned in-memory lifecycle (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `readSweepLiveness` + `_writeSweepState`, [`dashboard.js`](../dashboard.js) `handleKnowledgeSweep`). A separate `engine/kb-swept.json` is written after each successful non-dry-run sweep with the human-readable summary shown by the dashboard's "swept N days ago" badge (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `runKbSweep` → `shared.mutateKbSwept`).
|
|
179
179
|
|
|
180
180
|
## Automatic Periodic Sweep
|
|
181
181
|
|
|
182
|
-
The engine tick loop can also auto-spawn the KB sweep without dashboard interaction. Gated by `engine.autoConsolidateMemory` (default `true`; set `false` to disable — source: [`engine/shared.js`](../engine/shared.js) `ENGINE_DEFAULTS.autoConsolidateMemory`):
|
|
182
|
+
The engine tick loop can also auto-spawn the KB sweep without dashboard interaction. Gated by `engine.autoConsolidateMemory` (default `true`; set `false` to disable — source: [`engine/core/shared.js`](../engine/core/shared.js) `ENGINE_DEFAULTS.autoConsolidateMemory`):
|
|
183
183
|
|
|
184
|
-
- When `engine.autoConsolidateMemory: true`, every tick the engine consults `shouldAutoSweep()` from [`engine/kb-sweep.js`](../engine/kb-sweep.js) and, when the 4-hour cadence has elapsed since the last completion, calls `spawnSweepRunnerDetached()` to fire-and-forget a fresh `engine/kb-sweep-runner.js` process (source: [`engine.js`](../engine.js) tick step 2.1).
|
|
184
|
+
- When `engine.autoConsolidateMemory: true`, every tick the engine consults `shouldAutoSweep()` from [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) and, when the 4-hour cadence has elapsed since the last completion, calls `spawnSweepRunnerDetached()` to fire-and-forget a fresh `engine/memory/kb-sweep-runner.js` process (source: [`engine.js`](../engine.js) tick step 2.1).
|
|
185
185
|
- The inbox→`notes.md` consolidation runs every tick *regardless* of this flag via `consolidateInbox()`; `autoConsolidateMemory` controls **only** the heavier `knowledge/` sweep.
|
|
186
186
|
- The detached runner survives `minions restart` (same pattern used by the manual trigger), and the in-flight guard above prevents overlap with manual sweeps.
|
|
187
187
|
|
|
@@ -189,12 +189,12 @@ The engine tick loop can also auto-spawn the KB sweep without dashboard interact
|
|
|
189
189
|
|
|
190
190
|
The KB page surfaces sweep state in two places:
|
|
191
191
|
|
|
192
|
-
1. **Trigger button** (`kbSweep()` in [`dashboard/js/render-kb.js
|
|
193
|
-
2. **`#kb-swept-time` indicator** renders `swept N days ago · now sweeping (Xm)` (source: [`dashboard/js/render-kb.js
|
|
192
|
+
1. **Trigger button** (`kbSweep()` in [`dashboard/js/render-kb.js`](../dashboard/js/render-kb.js)) posts to `/api/knowledge/sweep` with the user's pinned keys and shows an immediate "queued" toast. If the response says `alreadyRunning`, the toast switches to "already running (Xm elapsed) — let it finish first".
|
|
193
|
+
2. **`#kb-swept-time` indicator** renders `swept N days ago · now sweeping (Xm)` (source: [`dashboard/js/render-kb.js`](../dashboard/js/render-kb.js), `kb-swept-time` element). The `sweepInFlight` and `sweepStartedAt` fields come from `GET /api/knowledge` (source: [`dashboard.js`](../dashboard.js) `handleKnowledgeList`).
|
|
194
194
|
|
|
195
195
|
### Polling
|
|
196
196
|
|
|
197
|
-
The KB page is refreshed on every dashboard status cycle (~4 s) in [`dashboard/js/refresh.js
|
|
197
|
+
The KB page is refreshed on every dashboard status cycle (~4 s) via `refreshKnowledgeBase()` in [`dashboard/js/refresh.js`](../dashboard/js/refresh.js). There is no separate sweep-status polling timer and no auto-stop — the in-flight badge keeps refreshing for as long as the user has the dashboard open and the sweep is running. Sweeps can take many minutes for a large KB; the persistent `engine/kb-sweep-state.json` flag plus the indefinite refresh loop are what let the indicator survive dashboard restarts and arbitrarily long sweeps.
|
|
198
198
|
|
|
199
199
|
## Result Summary
|
|
200
200
|
|
|
@@ -221,7 +221,7 @@ The KB page is refreshed on every dashboard status cycle (~4 s) in [`dashboard/j
|
|
|
221
221
|
}
|
|
222
222
|
```
|
|
223
223
|
|
|
224
|
-
After a non-dry-run sweep, `queries.invalidateKnowledgeBaseCache()` is called so the next `/api/knowledge` read sees the new state (source: [`engine/kb-sweep.js
|
|
224
|
+
After a non-dry-run sweep, `queries.invalidateKnowledgeBaseCache()` is called so the next `/api/knowledge` read sees the new state (source: [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) `runKbSweep`).
|
|
225
225
|
|
|
226
226
|
## What NOT to Do
|
|
227
227
|
|
|
@@ -240,9 +240,9 @@ The archive directory `knowledge/_swept/` is also off-limits to manual edits —
|
|
|
240
240
|
|
|
241
241
|
## Related Files
|
|
242
242
|
|
|
243
|
-
- [`engine/kb-sweep.js`](../engine/kb-sweep.js) — implementation
|
|
244
|
-
- [`engine/queries.js`](../engine/queries.js) — `getKnowledgeBaseEntries()`, `invalidateKnowledgeBaseCache()`
|
|
243
|
+
- [`engine/memory/kb-sweep.js`](../engine/memory/kb-sweep.js) — implementation
|
|
244
|
+
- [`engine/core/queries.js`](../engine/core/queries.js) — `getKnowledgeBaseEntries()`, `invalidateKnowledgeBaseCache()`
|
|
245
245
|
- [`dashboard.js`](../dashboard.js) — `handleKnowledgeSweep`, `handleKnowledgeSweepStatus`, `handleKnowledgeList`
|
|
246
246
|
- [`dashboard/js/render-kb.js`](../dashboard/js/render-kb.js) — UI trigger and indicator
|
|
247
247
|
- [`dashboard/js/refresh.js`](../dashboard/js/refresh.js) — refresh cadence (every status cycle)
|
|
248
|
-
- [`engine/shared.js`](../engine/shared.js) — `KB_CATEGORIES`, `getPinnedItems()`, `uniquePath()`
|
|
248
|
+
- [`engine/core/shared.js`](../engine/core/shared.js) — `KB_CATEGORIES`, `getPinnedItems()`, `uniquePath()`
|
package/docs/keep-processes.md
CHANGED
|
@@ -35,7 +35,7 @@ Set `meta.keep_processes: true` on the WI; agent writes `agents/<id>/keep-pids.j
|
|
|
35
35
|
## Lifecycle
|
|
36
36
|
|
|
37
37
|
- **Validation:** sidecar shape + workdir checked in `onAgentClose`. Failure → non-retryable `failure_class: invalid-keep-processes-{workdir,schema}`.
|
|
38
|
-
- **Sweep:** [`engine/keep-process-sweep.js`](../engine/keep-process-sweep.js) reaps at boot and every 180 ticks. Dead PIDs are pruned; expired TTL
|
|
38
|
+
- **Sweep:** [`engine/processes/keep-process-sweep.js`](../engine/processes/keep-process-sweep.js) reaps at boot and every 180 ticks. Dead PIDs are pruned; expired TTL routes through the identity-verified `engine.keep-process.ttl` termination site.
|
|
39
39
|
- **No healthcheck:** unlike managed-spawn, `keep_processes` does not gate WI completion on first-healthy. If the child crashes immediately the agent still succeeds — by design (ad-hoc helper semantics).
|
|
40
40
|
|
|
41
41
|
## Caps
|
|
@@ -34,9 +34,9 @@ Implementation: `engine.js` builds a `liveProjectsInUse` set from the active dis
|
|
|
34
34
|
|
|
35
35
|
The in-spawn check below remains as a **defense-in-depth fallback** for the race window between the allocation-time probe and the actual git operations inside `spawnAgent` (e.g. the tree goes dirty in the few seconds between the pre-check and the spawn) — it is now the *exception* path, not the common path.
|
|
36
36
|
|
|
37
|
-
Before spawning, `engine/live-checkout.js#prepareLiveCheckout` first refuses an in-progress repository operation or detached HEAD, captures the original ref, then runs `git status --porcelain` from `project.localPath`. Any dirty output (staged, unstaged, untracked) **returns** an explicit `{ ok:false, reason:'dirty', dirtyFiles:[…] }` result unless explicit reset recovery is requested. `spawnAgent` uses that result to run auto-stash first and optional auto-reset second. Only when configured recovery is unavailable or fails does the confirmed-dirty result become a dispatch refusal:
|
|
37
|
+
Before spawning, `engine/worktrees/live-checkout.js#prepareLiveCheckout` first refuses an in-progress repository operation or detached HEAD, captures the original ref, then runs `git status --porcelain` from `project.localPath`. Any dirty output (staged, unstaged, untracked) **returns** an explicit `{ ok:false, reason:'dirty', dirtyFiles:[…] }` result unless explicit reset recovery is requested. `spawnAgent` uses that result to run auto-stash first and optional auto-reset second. Only when configured recovery is unavailable or fails does the confirmed-dirty result become a dispatch refusal:
|
|
38
38
|
|
|
39
|
-
- Non-retryable `FAILURE_CLASS.LIVE_CHECKOUT_DIRTY` (added to `engine/dispatch.js`'s `neverRetry` set so the dispatcher never re-spawns mechanically). **Reserved for this confirmed-dirty result only** — a thrown helper/git error is `LIVE_CHECKOUT_FAILED`, see below (#305). **Auto-cleanup exception (W-mqzmkoqt000hbca2, #582):** when `liveCheckoutAutoReset` or `liveCheckoutAutoStash` is enabled (per-project or fleet-wide), `spawnAgent` treats the dirty refusal as retryable instead — the engine re-cleans the tree on every attempt, so the two-strike cap below is skipped and the dispatch keeps retrying up to the normal `maxRetries` cap. The `dispatch.js` `neverRetry` entry is kept only as a defensive fallback for callers that omit the explicit `agentRetryable` override.
|
|
39
|
+
- Non-retryable `FAILURE_CLASS.LIVE_CHECKOUT_DIRTY` (added to `engine/orchestration/dispatch.js`'s `neverRetry` set so the dispatcher never re-spawns mechanically). **Reserved for this confirmed-dirty result only** — a thrown helper/git error is `LIVE_CHECKOUT_FAILED`, see below (#305). **Auto-cleanup exception (W-mqzmkoqt000hbca2, #582):** when `liveCheckoutAutoReset` or `liveCheckoutAutoStash` is enabled (per-project or fleet-wide), `spawnAgent` treats the dirty refusal as retryable instead — the engine re-cleans the tree on every attempt, so the two-strike cap below is skipped and the dispatch keeps retrying up to the normal `maxRetries` cap. The `dispatch.js` `neverRetry` entry is kept only as a defensive fallback for callers that omit the explicit `agentRetryable` override.
|
|
40
40
|
- Inbox alert written via `dispatch.writeInboxAlert('live-checkout-dirty-<wi-id>', body)`. The body lists the dirty files verbatim from `git status --porcelain`.
|
|
41
41
|
- Work item stamped with `_pendingReason: 'live_checkout_dirty'` so the dashboard surfaces the block.
|
|
42
42
|
- Completion summary: `live-checkout refused: N dirty file(s) in <localPath>`.
|
|
@@ -48,14 +48,14 @@ Outside the explicit auto-stash and auto-reset policies, the engine never resets
|
|
|
48
48
|
Auto-stash is **ON by default**. The engine stashes dirty changes so dispatch can proceed without manual intervention; operators can opt out per project or fleet-wide:
|
|
49
49
|
|
|
50
50
|
- Configure via per-project `project.liveCheckoutAutoStash` (takes priority) or fleet-wide `engine.liveCheckoutAutoStash` (default `true`). Both are surfaced in Settings.
|
|
51
|
-
- When enabled and the tree is dirty, `spawnAgent` delegates the whole flow to `engine/live-checkout.js#applyLiveCheckoutAutoStash`, which calls `performLiveCheckoutAutoStash` to run `git stash push --include-untracked -m "minions-auto-stash-<dispatchId>-<timestamp>"` in `project.localPath` (`--include-untracked` so the `??` files that `git status --porcelain` counts as dirty are parked too). The stash git command runs **outside any file lock**.
|
|
51
|
+
- When enabled and the tree is dirty, `spawnAgent` delegates the whole flow to `engine/worktrees/live-checkout.js#applyLiveCheckoutAutoStash`, which calls `performLiveCheckoutAutoStash` to run `git stash push --include-untracked -m "minions-auto-stash-<dispatchId>-<timestamp>"` in `project.localPath` (`--include-untracked` so the `??` files that `git status --porcelain` counts as dirty are parked too). The stash git command runs **outside any file lock**.
|
|
52
52
|
- On stash **success** the helper re-runs `prepareLiveCheckout` (the tree is now clean), clears any stale `_pendingReason: 'live_checkout_dirty'` stamp (via the injected `clearDirtyStamp` callback), writes a `live-checkout-autostash-<wi-id>` inbox note with the stash name + manual-pop guidance, and returns `{ outcome:'stashed', liveResult }` so dispatch proceeds. If the re-preflight throws, it returns `{ outcome:'threw', error }` and `spawnAgent` fails the dispatch as `LIVE_CHECKOUT_FAILED`.
|
|
53
53
|
- On stash **failure** the error is surfaced (logged, never swallowed), the helper returns `{ outcome:'unchanged', liveResult }`, and the dispatch falls through to the normal retry-once-then-fail dirty path above.
|
|
54
54
|
- The engine **never pops the stash** automatically — that is the operator's choice. The stash name is logged and noted so the operator can `git stash pop` (or `git stash apply`) in `project.localPath` manually.
|
|
55
55
|
|
|
56
56
|
#### 2a. Thrown pre-spawn failures are retryable, NOT dirty (#305)
|
|
57
57
|
|
|
58
|
-
A thrown error from `prepareLiveCheckout` — a required-arg/ref-validation guard, or a transient `git status`/`rev-parse`/`checkout`/`symbolic-ref` failure — is **not** proof that the operator tree is dirty. `spawnAgent`'s `catch` block therefore completes the dispatch with the **separate** `FAILURE_CLASS.LIVE_CHECKOUT_FAILED` (`'live-checkout-failed'`), which is deliberately **excluded** from `engine/dispatch.js`'s `neverRetry` set. The dispatcher auto-retries with bounded backoff up to `ENGINE_DEFAULTS.maxRetries`, so racy branch-lock handoff, a just-finished sibling dispatch, or transient git state recovers on the next attempt **without a manual `/api/work-items/retry`**. Genuinely terminal underlying reasons (auth, validation) still short-circuit via the reason-string check in `isRetryableFailureReason`. No dirty-files inbox alert is written and no `_pendingReason: 'live_checkout_dirty'` stamp is applied for this path — those belong to the confirmed-dirty result above.
|
|
58
|
+
A thrown error from `prepareLiveCheckout` — a required-arg/ref-validation guard, or a transient `git status`/`rev-parse`/`checkout`/`symbolic-ref` failure — is **not** proof that the operator tree is dirty. `spawnAgent`'s `catch` block therefore completes the dispatch with the **separate** `FAILURE_CLASS.LIVE_CHECKOUT_FAILED` (`'live-checkout-failed'`), which is deliberately **excluded** from `engine/orchestration/dispatch.js`'s `neverRetry` set. The dispatcher auto-retries with bounded backoff up to `ENGINE_DEFAULTS.maxRetries`, so racy branch-lock handoff, a just-finished sibling dispatch, or transient git state recovers on the next attempt **without a manual `/api/work-items/retry`**. Genuinely terminal underlying reasons (auth, validation) still short-circuit via the reason-string check in `isRetryableFailureReason`. No dirty-files inbox alert is written and no `_pendingReason: 'live_checkout_dirty'` stamp is applied for this path — those belong to the confirmed-dirty result above.
|
|
59
59
|
|
|
60
60
|
> **`git status --porcelain` maxBuffer (W-mqvejug6000eeb20).** All read-only git probes in `prepareLiveCheckout` / `restoreLiveCheckoutAtDispatchEnd` run with a 50 MB `maxBuffer` (`LIVE_CHECKOUT_GIT_MAX_BUFFER`, threaded through `baseOpts`). A live tree with thousands of dirty/untracked paths could otherwise overflow `execFile`'s 1 MB default and reject with `ERR_CHILD_PROCESS_STDOUT_MAXBUFFER` — a *thrown* error that mis-classified as a transient retryable `LIVE_CHECKOUT_FAILED` (per 2a) and retry-stormed to the cap instead of being surfaced as the dirty tree it actually was. A caller-supplied `gitOpts.maxBuffer` still wins (spread after the default).
|
|
61
61
|
|
|
@@ -121,7 +121,7 @@ Like the blob-fetch case this is **structural, not transient** — it does *not*
|
|
|
121
121
|
|
|
122
122
|
Live mode shares one checkout per project. There is no pool to recycle, no quarantine directory, no per-WI subdirectory under the project root. The mutating-concurrency cap (Guarantee 1) is the only isolation mechanism: agents take turns in the same directory.
|
|
123
123
|
|
|
124
|
-
Pool short-circuits in `engine.js` and `engine/cleanup.js` gate on `worktreePath` truthiness and `!liveMode`, so borrow / return / orphan-GC execute zero git commands for live projects.
|
|
124
|
+
Pool short-circuits in `engine.js` and `engine/orchestration/cleanup.js` gate on `worktreePath` truthiness and `!liveMode`, so borrow / return / orphan-GC execute zero git commands for live projects.
|
|
125
125
|
|
|
126
126
|
### 5. Refuse on mid-operation / detached HEAD (engine never aborts the operator's in-progress op)
|
|
127
127
|
|
|
@@ -130,11 +130,11 @@ Pool short-circuits in `engine.js` and `engine/cleanup.js` gate on `worktreePath
|
|
|
130
130
|
- **In-progress git operation.** The git dir is resolved robustly via `git rev-parse --git-dir` (so submodule / gitdir-file / `repo`-managed trees — the setups that motivate live mode — are covered; a *failure* to resolve the git dir now throws → retryable `LIVE_CHECKOUT_FAILED` rather than silently probing a fabricated `<localPath>/.git` and missing the operation). Sentinel paths under it are probed: `MERGE_HEAD` → merge, `rebase-merge/` & `rebase-apply/` → rebase, `CHERRY_PICK_HEAD` → cherry-pick, `REVERT_HEAD` → revert, **`BISECT_LOG` → bisect**. First hit returns `{ ok:false, reason:'mid-operation', op, details }`.
|
|
131
131
|
- **Detached HEAD.** `git symbolic-ref -q HEAD` exiting with **code 1** returns `{ ok:false, reason:'detached-head', sha }` (sha from `git rev-parse HEAD`). Branching off a detached HEAD would strand the operator's anonymous commits. A *transient* `symbolic-ref` failure (spawn error / timeout, no exit-1) is **not** treated as a detached HEAD — it rethrows → retryable `LIVE_CHECKOUT_FAILED` — so an on-a-branch tree that hit a hiccup is never permanently refused.
|
|
132
132
|
|
|
133
|
-
Either condition fails the dispatch non-retryably with `FAILURE_CLASS.LIVE_CHECKOUT_MID_OPERATION` (`'live-checkout-mid-operation'`; added to `engine/dispatch.js`'s neverRetry set alongside `LIVE_CHECKOUT_DIRTY`). `spawnAgent` writes a `live-checkout-blocked-<wi-id>` inbox alert and stamps the work item `_pendingReason: 'live_checkout_mid_operation'` (or `'live_checkout_detached_head'` for the detached case). The recovery guidance tells the operator to finish or abort the in-progress op with their own commands (`git <op> --continue` / `git <op> --abort`), or checkout a branch, then re-dispatch. The engine never runs `git reset`, `git clean`, `git stash`, `git rebase --abort`, or moves HEAD on the operator's behalf.
|
|
133
|
+
Either condition fails the dispatch non-retryably with `FAILURE_CLASS.LIVE_CHECKOUT_MID_OPERATION` (`'live-checkout-mid-operation'`; added to `engine/orchestration/dispatch.js`'s neverRetry set alongside `LIVE_CHECKOUT_DIRTY`). `spawnAgent` writes a `live-checkout-blocked-<wi-id>` inbox alert and stamps the work item `_pendingReason: 'live_checkout_mid_operation'` (or `'live_checkout_detached_head'` for the detached case). The recovery guidance tells the operator to finish or abort the in-progress op with their own commands (`git <op> --continue` / `git <op> --abort`), or checkout a branch, then re-dispatch. The engine never runs `git reset`, `git clean`, `git stash`, `git rebase --abort`, or moves HEAD on the operator's behalf.
|
|
134
134
|
|
|
135
135
|
### 6. STALE-BASE GUARD — refuse to fork off a diverged local base (W-mr98op8w000ma4ad)
|
|
136
136
|
|
|
137
|
-
**W-mrcifup2000c218a — detected at ALLOCATION time first.** Same as Guarantee 2, the stale-base condition is now checked BEFORE dispatch by `engine/live-checkout.js#checkLiveCheckoutStaleBase` (a cheap, read-only, fetch-free `git rev-list --count origin/<mainRef>..<mainRef>` probe, mirroring the in-spawn check below) in the dispatch-existing-pending loop. A stale hit stamps `_pendingReason: 'live_checkout_stale_base'` and leaves the item pending — no retry consumed, no terminal failure — re-checked every tick, instead of the pre-fix behavior of failing non-retryably on the very first hit. A `live-checkout-stale-base-<wi-id>` inbox alert is written (deduped per day) so the condition is still surfaced to a human. The in-spawn refusal documented below is now a defense-in-depth fallback for the race window between the pre-check and the actual `git checkout -b`, not the common path — it still fails non-retryably when it fires, because a stale base found there is still deterministic and un-recoverable by a bare retry.
|
|
137
|
+
**W-mrcifup2000c218a — detected at ALLOCATION time first.** Same as Guarantee 2, the stale-base condition is now checked BEFORE dispatch by `engine/worktrees/live-checkout.js#checkLiveCheckoutStaleBase` (a cheap, read-only, fetch-free `git rev-list --count origin/<mainRef>..<mainRef>` probe, mirroring the in-spawn check below) in the dispatch-existing-pending loop. A stale hit stamps `_pendingReason: 'live_checkout_stale_base'` and leaves the item pending — no retry consumed, no terminal failure — re-checked every tick, instead of the pre-fix behavior of failing non-retryably on the very first hit. A `live-checkout-stale-base-<wi-id>` inbox alert is written (deduped per day) so the condition is still surfaced to a human. The in-spawn refusal documented below is now a defense-in-depth fallback for the race window between the pre-check and the actual `git checkout -b`, not the common path — it still fails non-retryably when it fires, because a stale base found there is still deterministic and un-recoverable by a bare retry.
|
|
138
138
|
|
|
139
139
|
A **clean** tree can still hide **committed** contamination. When `prepareLiveCheckout` is about to create a **new** branch (`git checkout -b <branch>`) off the operator's current HEAD, and HEAD is sitting on the project base ref (`mainRef`, the common case), the local `mainRef` branch may carry commits that were never pushed to `origin/<mainRef>` — e.g. leftover in-progress work from a prior dispatch on a live-mode project (capped to one mutating dispatch, but shared across many sequential dispatches), an interrupted process, or a bad rebase. Forking a new branch off that base silently inherits those commits into the branch **and its PR**. This is exactly the scope-contamination class first seen on ADO PR 5411214 and recurring on AB#12016662 / ADO PR 5419146 (~20 unrelated OCM files baked into an otherwise-clean 2-file a11y fix).
|
|
140
140
|
|
|
@@ -191,7 +191,7 @@ Immediately before the new-branch `git checkout -b`, `prepareLiveCheckout` compa
|
|
|
191
191
|
- **The base is per-project, NOT a hardcoded `'main'`.** `spawnAgent` resolves `mainRef` via `shared.resolveMainBranch(cwd, project.mainBranch)` (`engine.js`), which honors the project's configured `mainBranch`, else auto-detects `refs/remotes/origin/HEAD`, else falls back to `main`. Repos whose default branch is **not** `main` — e.g. Office Android development, whose harness (`claude.md`) documents a `lkg/main/android` base — are handled by setting `project.mainBranch: "lkg/main/android"` (or by having `origin/HEAD` point at it); the guard then keys on that branch. The word "main" in this guarantee is shorthand for *"the project's resolved base branch,"* never the literal string.
|
|
192
192
|
- **HEAD already on `mainRef`** → forks normally; the guard never probes the base ref.
|
|
193
193
|
- **HEAD off-base, local `refs/heads/<mainRef>` exists** → PLAIN `git checkout <mainRef>` (no fetch, no reset, no `--force` — the tree was verified clean at Guarantee 2), then forks off it. No auth-less GVFS blob fetch is forced in the common case.
|
|
194
|
-
- **HEAD off-base, no usable local `mainRef`** (fresh-clone edge case) — or the local checkout itself fails — → **fail closed** with `{ ok:false, reason:'wrong-base' }` **unless auto-base-repair is enabled (see §6a).** The caller refuses **non-retryably** with `FAILURE_CLASS.LIVE_CHECKOUT_WRONG_BASE` (`'live-checkout-wrong-base'`; added to `engine/dispatch.js`'s neverRetry set), writes an inbox alert naming the off-base HEAD, the expected base, and the exact `git checkout <mainRef>` recovery command, and stamps the work item `_pendingReason`. This mirrors the dirty / mid-operation philosophy of refusing on ambiguous operator state rather than silently forking off the wrong base.
|
|
194
|
+
- **HEAD off-base, no usable local `mainRef`** (fresh-clone edge case) — or the local checkout itself fails — → **fail closed** with `{ ok:false, reason:'wrong-base' }` **unless auto-base-repair is enabled (see §6a).** The caller refuses **non-retryably** with `FAILURE_CLASS.LIVE_CHECKOUT_WRONG_BASE` (`'live-checkout-wrong-base'`; added to `engine/orchestration/dispatch.js`'s neverRetry set), writes an inbox alert naming the off-base HEAD, the expected base, and the exact `git checkout <mainRef>` recovery command, and stamps the work item `_pendingReason`. This mirrors the dirty / mid-operation philosophy of refusing on ambiguous operator state rather than silently forking off the wrong base.
|
|
195
195
|
|
|
196
196
|
#### 7a. Opt-in auto-base-repair (`liveCheckoutAutoBaseRepair`, W-mr3lunnq000o9f41)
|
|
197
197
|
|
|
@@ -221,7 +221,7 @@ Live-mode agents run **in-place** in the operator's checkout, so when a dispatch
|
|
|
221
221
|
- **Retry-with-backoff on the WIP auto-save commit (#608).** The `git add -A` + `git commit` above (and the equivalent self-heal commit described below) go through `_commitAgentWipWithRetry`: a bounded retry (3 attempts, linear backoff) that fires only when the commit fails with an `index.lock` / "another git process" message — i.e. a transient collision with a concurrent git invocation — never on a genuine failure like "nothing to commit". Previously a single failed attempt (e.g. a `.git/index.lock` race) silently aborted the auto-save, leaving the tree dirty on the agent branch and letting the pollution below take hold.
|
|
222
222
|
- **Orphaned-agent-branch self-heal at dispatch start (#608).** If dispatch-end restore is ever skipped or interrupted (crash, forced kill, an engine restart racing the restore), the *next* dispatch for that project can start with HEAD already sitting on a stray `work/<wi-id>` branch from the previous run — and, before this fix, `prepareLiveCheckout`'s dirty-tree check ran *before* `originalRef` was even captured, so the function returned `{ reason: 'dirty' }` immediately and the branch was never noticed or cleaned, silently repeating for every subsequent dispatch. `prepareLiveCheckout` now recognizes this case: if the tree is still dirty after the existing auto-clean-artifacts recheck, and the current branch (read from the already-fetched `git status --porcelain -b` header, no extra git call) matches the `work/<id>` naming convention (`_looksLikeAgentBranch`) and differs from this dispatch's own target branch, the WIP is committed onto that stray branch (again via `_commitAgentWipWithRetry`) and HEAD is switched back to `mainRef` before the normal flow continues. This only ever touches a branch matching the engine's own naming convention — an operator's own feature branch (e.g. `feature/x`) is never mistaken for engine litter and is left untouched.
|
|
223
223
|
- **AUTO-RESTORE (best-effort, never `--force`).** At dispatch-end `restoreLiveCheckoutAtDispatchEnd` issues a **plain** `git checkout <originalRef>` — no `--force`, no `-B`, no reset, no clean, no stash. It no-ops when there is nothing to restore: no captured `originalRef`, the agent branch *is* the original ref, or HEAD already sits on the original ref (matched against the branch name *or* the raw sha so the detached-HEAD case is recognized). It is strictly best-effort: every error is swallowed and logged, and a restore never alters the dispatch result.
|
|
224
|
-
- **Read-site trust invariant (P-mrb8yg9x001hfa9f audit).** `restoreLiveCheckoutAtDispatchEnd` never re-derives or re-validates `originalRef` itself — it does not call `_looksLikeAgentBranch` or otherwise probe for contamination; it simply checks out whatever ref it is handed. This is safe *only* because `prepareLiveCheckout`'s own originalRef-contamination invariant (W-mrb6an8300058fd8, §"Original-ref capture" above) already corrects `originalRef` to `mainRef` at capture time whenever HEAD was on a stray engine-managed branch, for both the existing-branch (4b) and new-branch/STALE-BASE-GUARD (4d) code paths — and `engine.js` persists that corrected value verbatim (`engine.js:3106-3122`) for every restore call site (`onAgentClose`, the `engine/cli.js` restart re-attach path, and the `engine/timeout.js` reaping paths) to consume as-is. If the write-side correction in `prepareLiveCheckout` ever regressed, restore would silently check the operator back out onto the contaminated branch, since it has no independent check of its own to catch that. Characterized end-to-end in `test/unit/prepare-live-checkout.test.js` (chains `prepareLiveCheckout` → `restoreLiveCheckoutAtDispatchEnd` for both the 4b and 4d paths).
|
|
224
|
+
- **Read-site trust invariant (P-mrb8yg9x001hfa9f audit).** `restoreLiveCheckoutAtDispatchEnd` never re-derives or re-validates `originalRef` itself — it does not call `_looksLikeAgentBranch` or otherwise probe for contamination; it simply checks out whatever ref it is handed. This is safe *only* because `prepareLiveCheckout`'s own originalRef-contamination invariant (W-mrb6an8300058fd8, §"Original-ref capture" above) already corrects `originalRef` to `mainRef` at capture time whenever HEAD was on a stray engine-managed branch, for both the existing-branch (4b) and new-branch/STALE-BASE-GUARD (4d) code paths — and `engine.js` persists that corrected value verbatim (`engine.js:3106-3122`) for every restore call site (`onAgentClose`, the `engine/operations/cli.js` restart re-attach path, and the `engine/orchestration/timeout.js` reaping paths) to consume as-is. If the write-side correction in `prepareLiveCheckout` ever regressed, restore would silently check the operator back out onto the contaminated branch, since it has no independent check of its own to catch that. Characterized end-to-end in `test/unit/prepare-live-checkout.test.js` (chains `prepareLiveCheckout` → `restoreLiveCheckoutAtDispatchEnd` for both the 4b and 4d paths).
|
|
225
225
|
- **Fallback notify (only when a safe switch is impossible).** If git declines the plain checkout — most likely because the agent left uncommitted changes a checkout would overwrite — the refusal is **honored**: the tree is left exactly as the agent left it and a deduped `live-checkout-branch-<dispatchId>` inbox alert tells the operator how to switch back manually (`git -C <localPath> checkout <originalRef>`). The engine never forces the switch. An **unexpected** restore error (git missing, repo corruption, a GVFS blob fetch on the switch-back) now also writes this alert, so a non-refusal failure never silently strands the tree.
|
|
226
226
|
- **Terminal-failure alert.** When the dispatch ends in a non-success terminal state, a deduped `live-checkout-failed-<dispatchId>` inbox alert is written so the operator knows a live-mode run failed inside their own checkout (where any partial work is visible). This is independent of the restore and fires even when the restore itself succeeds.
|
|
227
227
|
|
|
@@ -360,38 +360,38 @@ Live-checkout mode is deliberately small. These are NOT supported and will not b
|
|
|
360
360
|
- **No per-WI subdirectory isolation.** Live mode is one-checkout-per-project by design. If you need isolation, use `checkoutMode: 'worktree'` (the default).
|
|
361
361
|
- **No per-WI override.** `checkoutMode` is per-project only. There is no `meta.checkoutMode` on a work item that overrides the project setting.
|
|
362
362
|
- **No auto-pull / no fast-forward on existing branches.** See Guarantee 3. If a PR branch is checked out locally at a different SHA than `origin/<branch>`, the operator resolves it manually.
|
|
363
|
-
- **No special timeout / kill handling.** Live-mode dispatches are killed by PID exactly like isolated-mode dispatches (`engine/timeout.js` header comment). The engine sends SIGTERM/SIGKILL to the tracked process and never touches the working tree to deliver the kill. (Dispatch-end auto-restore still runs afterward — a plain branch switch back to the operator's original ref, never a reset/clean/stash; see [Dispatch-end auto-restore](#dispatch-end-auto-restore).)
|
|
363
|
+
- **No special timeout / kill handling.** Live-mode dispatches are killed by PID exactly like isolated-mode dispatches (`engine/orchestration/timeout.js` header comment). The engine sends SIGTERM/SIGKILL to the tracked process and never touches the working tree to deliver the kill. (Dispatch-end auto-restore still runs afterward — a plain branch switch back to the operator's original ref, never a reset/clean/stash; see [Dispatch-end auto-restore](#dispatch-end-auto-restore).)
|
|
364
364
|
|
|
365
365
|
## Related code
|
|
366
366
|
|
|
367
367
|
| File | Purpose |
|
|
368
368
|
|---|---|
|
|
369
|
-
| `engine/shared.js` — `CHECKOUT_MODES`, `validateCheckoutMode`, `resolveCheckoutMode`, `isLiveCheckoutProject` | Enum + validator + back-compat resolver (P-a3f9b201; consolidated W-mqiaw974). |
|
|
370
|
-
| `engine/shared.js` — `resolveLiveCheckoutAutoReset` + `ENGINE_DEFAULTS.liveCheckoutAutoReset` | Pure precedence resolver (per-project boolean > fleet-wide engine default > false) + the fleet-wide default (OFF as of W-mrawgw4q000a6bff — was ON as of W-mqzbbhn2). Gates the dirty-tree auto-reset in `prepareLiveCheckout` (W-mqvejug6000eeb20); in `engine.js spawnAgent` only fires as a fallback after auto-stash fails/is disabled (W-mrawgw4q000a6bff). |
|
|
371
|
-
| `engine/shared.js` — `resolveSpawnPaths` | Returns `{ cwd: localPath, worktreeRootDir: null, liveMode: true }` for live dispatches; project-bound worktree-mode read-only types request detached worktree placement. |
|
|
372
|
-
| `engine/live-checkout.js` — `prepareLiveCheckout` | Pure helper by default: pre-mutation mid-operation / detached-HEAD preflight, original-ref capture, dirty check, **already-on-branch fast path**, `refs/heads/<branch>` existence check, branch resolution from HEAD (no fetch — issue #226), **no-half-switch + `blob-fetch`/`worktree-conflict` classification**, and opt-in destructive WIP cleanup (`reset --hard HEAD` + `clean -fd` + re-check + audit note) that preserves the branch and committed history. Git probes use a **50 MB maxBuffer**. |
|
|
373
|
-
| `engine/live-checkout.js` — `restoreLiveCheckoutAtDispatchEnd` | Dispatch-end auto-restore (plain `git checkout <originalRef>`, never `--force`/reset/clean/stash, best-effort) + **self-healing dirty recovery** (auto-commit agent WIP onto the agent branch) + `live-checkout-failed-<dispatchId>` terminal-failure alert + `live-checkout-branch-<dispatchId>` fallback notify (now also on unexpected restore errors) (P-d9e6b2c4; self-heal PL-live-checkout-reliability-hardening). |
|
|
374
|
-
| `engine/live-checkout.js` — `resolveLiveCheckoutAutoStash`, `performLiveCheckoutAutoStash`, `applyLiveCheckoutAutoStash` | Auto-stash resolver (per-project boolean, then fleet setting, then default true) + `git stash push --include-untracked` runner + stash/re-preflight orchestration. Engine never auto-pops (W-mqtvnnj1000357fa). |
|
|
375
|
-
| `engine/live-checkout.js` — `maybeRestoreLiveCheckoutFromRecord` | Shared persisted-record restore wrapper; requires explicit live provenance plus current live configuration and fails closed for worktree mode. |
|
|
369
|
+
| `engine/core/shared.js` — `CHECKOUT_MODES`, `validateCheckoutMode`, `resolveCheckoutMode`, `isLiveCheckoutProject` | Enum + validator + back-compat resolver (P-a3f9b201; consolidated W-mqiaw974). |
|
|
370
|
+
| `engine/core/shared.js` — `resolveLiveCheckoutAutoReset` + `ENGINE_DEFAULTS.liveCheckoutAutoReset` | Pure precedence resolver (per-project boolean > fleet-wide engine default > false) + the fleet-wide default (OFF as of W-mrawgw4q000a6bff — was ON as of W-mqzbbhn2). Gates the dirty-tree auto-reset in `prepareLiveCheckout` (W-mqvejug6000eeb20); in `engine.js spawnAgent` only fires as a fallback after auto-stash fails/is disabled (W-mrawgw4q000a6bff). |
|
|
371
|
+
| `engine/core/shared.js` — `resolveSpawnPaths` | Returns `{ cwd: localPath, worktreeRootDir: null, liveMode: true }` for live dispatches; project-bound worktree-mode read-only types request detached worktree placement. |
|
|
372
|
+
| `engine/worktrees/live-checkout.js` — `prepareLiveCheckout` | Pure helper by default: pre-mutation mid-operation / detached-HEAD preflight, original-ref capture, dirty check, **already-on-branch fast path**, `refs/heads/<branch>` existence check, branch resolution from HEAD (no fetch — issue #226), **no-half-switch + `blob-fetch`/`worktree-conflict` classification**, and opt-in destructive WIP cleanup (`reset --hard HEAD` + `clean -fd` + re-check + audit note) that preserves the branch and committed history. Git probes use a **50 MB maxBuffer**. |
|
|
373
|
+
| `engine/worktrees/live-checkout.js` — `restoreLiveCheckoutAtDispatchEnd` | Dispatch-end auto-restore (plain `git checkout <originalRef>`, never `--force`/reset/clean/stash, best-effort) + **self-healing dirty recovery** (auto-commit agent WIP onto the agent branch) + `live-checkout-failed-<dispatchId>` terminal-failure alert + `live-checkout-branch-<dispatchId>` fallback notify (now also on unexpected restore errors) (P-d9e6b2c4; self-heal PL-live-checkout-reliability-hardening). |
|
|
374
|
+
| `engine/worktrees/live-checkout.js` — `resolveLiveCheckoutAutoStash`, `performLiveCheckoutAutoStash`, `applyLiveCheckoutAutoStash` | Auto-stash resolver (per-project boolean, then fleet setting, then default true) + `git stash push --include-untracked` runner + stash/re-preflight orchestration. Engine never auto-pops (W-mqtvnnj1000357fa). |
|
|
375
|
+
| `engine/worktrees/live-checkout.js` — `maybeRestoreLiveCheckoutFromRecord` | Shared persisted-record restore wrapper; requires explicit live provenance plus current live configuration and fails closed for worktree mode. |
|
|
376
376
|
| `engine.js` — `spawnAgent` live-mode block | Calls `prepareLiveCheckout`, delegates opt-in auto-stash to `applyLiveCheckoutAutoStash` on a dirty tree (W-mqtvnnj1000357fa), handles dirty / throw branches, gates `git worktree add` on `!liveMode` (P-a3f9b204). |
|
|
377
377
|
| `engine.js` — `spawnAgent` mid-op / detached-HEAD refusal block | Emits `LIVE_CHECKOUT_MID_OPERATION`, writes `live-checkout-blocked-<wi-id>` alert, stamps `_pendingReason: 'live_checkout_mid_operation'` / `'live_checkout_detached_head'` (P-c5a1f3b8). |
|
|
378
378
|
| `engine.js` — `spawnAgent` originalRef persistence | Persists `originalRef` / `originalRefType` onto the dispatch record via `mutateDispatch` so restore survives an engine restart (P-c5a1f3b8). |
|
|
379
379
|
| `engine.js` — `onAgentClose` live-mode restore wiring | Calls `restoreLiveCheckoutAtDispatchEnd` on every terminal result (P-d9e6b2c4). |
|
|
380
|
-
| `engine/cli.js` — orphan / reattach restore | Fires the dispatch-end restore (via `maybeRestoreLiveCheckoutFromRecord`) from the persisted record on the engine-restart completion path (P-d9e6b2c4; unified PL-live-checkout-reliability-hardening). |
|
|
381
|
-
| `engine/timeout.js` — `completeFromOutput` + orphan-sweep restore | Fires `maybeRestoreLiveCheckoutFromRecord` when a re-attached live dispatch is reaped post-restart — closes the gap where these paths had no restore wiring (PL-live-checkout-reliability-hardening). |
|
|
382
|
-
| `engine/lifecycle.js` — `autoDispatchLiveValidationWi` | Creates PR-targeted validation WIs and transactionally deduplicates them by source WI plus canonical PR identity/head across central and project scopes. |
|
|
380
|
+
| `engine/operations/cli.js` — orphan / reattach restore | Fires the dispatch-end restore (via `maybeRestoreLiveCheckoutFromRecord`) from the persisted record on the engine-restart completion path (P-d9e6b2c4; unified PL-live-checkout-reliability-hardening). |
|
|
381
|
+
| `engine/orchestration/timeout.js` — `completeFromOutput` + orphan-sweep restore | Fires `maybeRestoreLiveCheckoutFromRecord` when a re-attached live dispatch is reaped post-restart — closes the gap where these paths had no restore wiring (PL-live-checkout-reliability-hardening). |
|
|
382
|
+
| `engine/orchestration/lifecycle.js` — `autoDispatchLiveValidationWi` | Creates PR-targeted validation WIs and transactionally deduplicates them by source WI plus canonical PR identity/head across central and project scopes. |
|
|
383
383
|
| `engine.js` — dispatcher `liveProjectsInUse` set | Per-project mutating-concurrency cap (P-a3f9b205). |
|
|
384
384
|
| `engine.js` — worktree-pool / orphan-GC short-circuits | `worktreePath===null` no-ops in live mode (P-a3f9b206). |
|
|
385
|
-
| `engine/cleanup.js` — `runPeriodicWorktreeSweep` live filter | Excludes live-checkout projects from the registry-derived periodic worktree GC so the operator's primary checkout never enters the GC decision surface (PL-live-checkout-reliability-hardening). |
|
|
386
|
-
| `engine/create-pr
|
|
387
|
-
| `engine/pr-action.js` — `buildCreatePrFollowups({ …, mainBranch })` live-mode message | CC "Create PR" chip instruction. In live mode the PR is ALWAYS built on a fresh branch off `origin/<mainBranch>` (stash → fetch → `checkout -B` → pop → commit → push → PR → restore), never reusing the current branch as the base; stop-on-stash-pop-conflict. `dashboard.js` `POST /api/pr-action/offer-create-pr` resolves `mainBranch` via `shared.resolveMainBranch` and threads it in (W-mr3jbpru). |
|
|
385
|
+
| `engine/orchestration/cleanup.js` — `runPeriodicWorktreeSweep` live filter | Excludes live-checkout projects from the registry-derived periodic worktree GC so the operator's primary checkout never enters the GC decision surface (PL-live-checkout-reliability-hardening). |
|
|
386
|
+
| `engine/worktrees/create-pr.js` — `prepareCreatePrWorktree` | Copies pre-existing operator changes into an isolated worktree and returns `sourceCheckoutPreserved: true`; never resets, cleans, checks out, or deletes source files. |
|
|
387
|
+
| `engine/providers/pr-action.js` — `buildCreatePrFollowups({ …, mainBranch })` live-mode message | CC "Create PR" chip instruction. In live mode the PR is ALWAYS built on a fresh branch off `origin/<mainBranch>` (stash → fetch → `checkout -B` → pop → commit → push → PR → restore), never reusing the current branch as the base; stop-on-stash-pop-conflict. `dashboard.js` `POST /api/pr-action/offer-create-pr` resolves `mainBranch` via `shared.resolveMainBranch` and threads it in (W-mr3jbpru). |
|
|
388
388
|
| `dashboard/js/settings.js` — checkoutMode dropdown + chip + `set-liveCheckoutAutoReset` fleet toggle | Operator-facing UI; the fleet-wide auto-reset toggle persists to `engine.liveCheckoutAutoReset` (per-project UI deferred — config.json only) (P-a3f9b207; auto-reset toggle W-mqvejug6000eeb20). |
|
|
389
389
|
| `test/unit/{resolve-spawn-paths-live-mode,prepare-live-checkout,spawn-agent-live-mode-wiring,live-checkout-auto-stash,live-checkout-stash-before-reset}.test.js` | Wiring and contract tests (P-a3f9b208; auto-stash helpers W-mqtvnnj1000357fa; stash-before-reset ordering W-mrawgw4q000a6bff). |
|
|
390
|
-
| `engine/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_DIRTY` | Non-retryable refusal class. |
|
|
391
|
-
| `engine/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_MID_OPERATION` | Non-retryable refusal class for a mid-operation / detached-HEAD operator tree (in-progress merge/rebase/cherry-pick/revert/bisect or detached HEAD), distinct from the dirty-tree class. Emitted by `spawnAgent`'s mid-op / detached-HEAD refusal block (P-a7f3c1d9; wired P-c5a1f3b8). |
|
|
392
|
-
| `engine/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_BLOB_FETCH` | Non-retryable refusal class for an existing-branch checkout that could not hydrate the tree on a blobless GVFS partial clone (auth-less cache fetch, headless) — deterministic, so excluded from mechanical retry (PL-live-checkout-reliability-hardening). |
|
|
393
|
-
| `engine/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_WORKTREE_CONFLICT` | Non-retryable refusal class for an existing-branch checkout that refused because the branch is already checked out in another worktree — structural, so excluded from mechanical retry (W-mr28h2j2000y0de1). |
|
|
394
|
-
| `engine/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_STALE_BASE` | Non-retryable refusal class for a new-branch fork whose local `mainRef` base has diverged from `origin/<mainRef>` with unpushed commits (STALE-BASE GUARD) — detected fetch-free via `git rev-list --count origin/<mainRef>..<mainRef>`; deterministic, so excluded from mechanical retry (W-mr98op8w000ma4ad). |
|
|
390
|
+
| `engine/core/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_DIRTY` | Non-retryable refusal class. |
|
|
391
|
+
| `engine/core/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_MID_OPERATION` | Non-retryable refusal class for a mid-operation / detached-HEAD operator tree (in-progress merge/rebase/cherry-pick/revert/bisect or detached HEAD), distinct from the dirty-tree class. Emitted by `spawnAgent`'s mid-op / detached-HEAD refusal block (P-a7f3c1d9; wired P-c5a1f3b8). |
|
|
392
|
+
| `engine/core/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_BLOB_FETCH` | Non-retryable refusal class for an existing-branch checkout that could not hydrate the tree on a blobless GVFS partial clone (auth-less cache fetch, headless) — deterministic, so excluded from mechanical retry (PL-live-checkout-reliability-hardening). |
|
|
393
|
+
| `engine/core/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_WORKTREE_CONFLICT` | Non-retryable refusal class for an existing-branch checkout that refused because the branch is already checked out in another worktree — structural, so excluded from mechanical retry (W-mr28h2j2000y0de1). |
|
|
394
|
+
| `engine/core/shared.js` — `FAILURE_CLASS.LIVE_CHECKOUT_STALE_BASE` | Non-retryable refusal class for a new-branch fork whose local `mainRef` base has diverged from `origin/<mainRef>` with unpushed commits (STALE-BASE GUARD) — detected fetch-free via `git rev-list --count origin/<mainRef>..<mainRef>`; deterministic, so excluded from mechanical retry (W-mr98op8w000ma4ad). |
|
|
395
395
|
| `engine.js` — `_liveCheckoutDirtyAttempts` counter | Dedicated two-strike dirty memory (survives the discovery + retry `_pendingReason` scrubs that defeated #434); first dirty failure retries once, second fails non-retryably (PL-live-checkout-reliability-hardening) — **unless** `liveCheckoutAutoReset`/`liveCheckoutAutoStash` is enabled, in which case the two-strike cap is skipped entirely and the dirty failure stays retryable up to `maxRetries` (W-mqzmkoqt000hbca2, #582). |
|
|
396
|
-
| `engine/dispatch.js` — `isRetryableFailureReason` neverRetry | Excludes `LIVE_CHECKOUT_DIRTY`, `LIVE_CHECKOUT_MID_OPERATION`, `LIVE_CHECKOUT_BLOB_FETCH`, and `LIVE_CHECKOUT_WORKTREE_CONFLICT` from mechanical retry. |
|
|
397
|
-
| `engine/timeout.js` header comment | Confirms no special live-mode kill handling. |
|
|
396
|
+
| `engine/orchestration/dispatch.js` — `isRetryableFailureReason` neverRetry | Excludes `LIVE_CHECKOUT_DIRTY`, `LIVE_CHECKOUT_MID_OPERATION`, `LIVE_CHECKOUT_BLOB_FETCH`, and `LIVE_CHECKOUT_WORKTREE_CONFLICT` from mechanical retry. |
|
|
397
|
+
| `engine/orchestration/timeout.js` header comment | Confirms no special live-mode kill handling. |
|
package/docs/managed-spawn.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Managed-spawn primitive
|
|
2
2
|
|
|
3
3
|
> Engine-owned long-running services with sidecar declaration, healthcheck-gated dispatch SUCCESS, and dashboard discovery.
|
|
4
|
-
> Module: [`engine/managed-spawn.js`](../engine/managed-spawn.js) · Dashboard panel: `/engine` → "Managed Processes".
|
|
4
|
+
> Module: [`engine/processes/managed-spawn.js`](../engine/processes/managed-spawn.js) · Dashboard panel: `/engine` → "Managed Processes".
|
|
5
5
|
|
|
6
6
|
## Why this exists
|
|
7
7
|
|
|
@@ -57,7 +57,7 @@ The sidecar lives at `<MINIONS_DIR>/agents/<agentId>/managed-spawn.json` and is
|
|
|
57
57
|
}
|
|
58
58
|
```
|
|
59
59
|
|
|
60
|
-
The renderer in [`buildManagedSpawnHint`](../engine/managed-spawn.js) emits this exact shape (with allowlist + cap reminders) into the agent's prompt whenever the work item has `meta.managed_spawn: true`. Treat the rendered hint as the source of truth — if this doc and the hint drift, the hint wins.
|
|
60
|
+
The renderer in [`buildManagedSpawnHint`](../engine/processes/managed-spawn.js) emits this exact shape (with allowlist + cap reminders) into the agent's prompt whenever the work item has `meta.managed_spawn: true`. Treat the rendered hint as the source of truth — if this doc and the hint drift, the hint wins.
|
|
61
61
|
|
|
62
62
|
For an opted-in dispatch, the sidecar is mandatory. Completion acceptance reads
|
|
63
63
|
only the canonical running agent's directory and requires `written_by` and
|
|
@@ -147,7 +147,7 @@ Not every managed-spawn target is a TCP-bound HTTP dev server. Native mobile/des
|
|
|
147
147
|
}
|
|
148
148
|
```
|
|
149
149
|
|
|
150
|
-
Known device-farm tooling: `adb` and `emulator` are both on the managed-spawn executable allowlist (device/emulator shell + install + logcat; `emulator -list-avds` discovers configured AVDs before picking one). `maestro` is a declarative mobile UI-flow runner already wired in as a built-in QA runner adapter (`engine/qa-runners
|
|
150
|
+
Known device-farm tooling: `adb` and `emulator` are both on the managed-spawn executable allowlist (device/emulator shell + install + logcat; `emulator -list-avds` discovers configured AVDs before picking one). `maestro` is a declarative mobile UI-flow runner already wired in as a built-in QA runner adapter (`engine/qa-runners/`) for the DRAFT/EXECUTE phases — it is not itself on the managed-spawn `cmd`/healthcheck allowlist. The full worked example — including `ttl_minutes`, `attrs`, `written_by`/`wi_id` — is inlined in [`buildManagedSpawnHint`](../engine/processes/managed-spawn.js).
|
|
151
151
|
|
|
152
152
|
### Non-PATH native launcher binaries (e.g. Android SDK's `emulator.exe` / `adb.exe`)
|
|
153
153
|
|
|
@@ -219,7 +219,7 @@ Single quotes (`'…'`) work the same way. No `powershell -Command "& '<path>' <
|
|
|
219
219
|
Background work the engine does without further agent involvement:
|
|
220
220
|
- **Per-tick** (cadence `engine.managedSpawn.sweepEvery: 30`, ~30 min): `sweepManagedSpawn()` drops dead-PID rows, kills + unlinks TTL-expired specs, rotates `managed-logs/<name>.log` past `logRotateBytes` (10MB).
|
|
221
221
|
- **Per-engine-boot**: `bootReconcileManagedSpawn()` (timeout-bounded by `bootReconcileMaxMs: 2000` ms) drops dead PIDs, kills expired, runs one immediate healthcheck per survivor.
|
|
222
|
-
- **Per-project-removal**: `removeManagedSpecsForProject(name)` (called from `engine/projects.js removeProject`)
|
|
222
|
+
- **Per-project-removal**: `removeManagedSpecsForProject(name)` (called from `engine/planning/projects.js removeProject`) identity-verifies each matching process before terminating it; records whose identity cannot be observed are preserved and reported instead of risking a recycled-PID kill, while a PID whose live identity conclusively proves it is *not* the recorded child (it started after that child did) has its stale record dropped without any kill.
|
|
223
223
|
|
|
224
224
|
## WI 1 (build) → WI 2 (test) chained-validation pattern
|
|
225
225
|
|
|
@@ -235,7 +235,7 @@ The canonical use case: split a build + smoke-test workflow across two work item
|
|
|
235
235
|
**WI 2 — test against the live service:**
|
|
236
236
|
- `dependencies: [WI-1-id]` (engine won't dispatch until WI 1 is SUCCESS)
|
|
237
237
|
- `meta.managed_spawn: false` (or omit — no new specs to write)
|
|
238
|
-
- Playbook auto-inject (from `engine/playbook.js`) ships a `## Live managed processes for project <name>` block in WI 2's prompt with WI 1's specs' `name`, `pid`, `attrs`, `log_path`, `ttl_expires_at`, and base URL.
|
|
238
|
+
- Playbook auto-inject (from `engine/agents/playbook.js`) ships a `## Live managed processes for project <name>` block in WI 2's prompt with WI 1's specs' `name`, `pid`, `attrs`, `log_path`, `ttl_expires_at`, and base URL.
|
|
239
239
|
- WI 2 reads `attrs.base_url` from its prompt context, hits the live service, runs whatever test it needs to.
|
|
240
240
|
|
|
241
241
|
Outcome: if WI 2 fails, the spec stays alive (TTL-managed) and the next dispatch of WI 2 reuses the same instance — no rebuild churn. If WI 1 fails healthcheck, WI 2 never dispatches; you fix WI 1 and retry.
|
|
@@ -266,8 +266,12 @@ single integer from 1 byte through 1 MiB (default 64 KiB).
|
|
|
266
266
|
|
|
267
267
|
Both controls bind the request to the selected row's `pid` + `started_at`
|
|
268
268
|
generation and verify the live OS process start identity before killing. A
|
|
269
|
-
concurrent replacement,
|
|
270
|
-
deleting the surviving generation
|
|
269
|
+
concurrent replacement, an unobservable identity, or a failed kill returns `409`
|
|
270
|
+
without deleting the surviving generation. A **recycled** PID is different: when
|
|
271
|
+
the live identity conclusively shows the process started after our recorded
|
|
272
|
+
child, the row is stale rather than live, so it is dropped (kill returns
|
|
273
|
+
`killed: false`) and the foreign PID is never signalled; restart also reserves
|
|
274
|
+
the name before
|
|
271
275
|
spawning so concurrent calls cannot orphan duplicate generations. Engine batch
|
|
272
276
|
spawns use the same reservation protocol, first-health and boot-probe writes
|
|
273
277
|
must match the reserved generation, and failed rollback kills remain as
|
|
@@ -281,7 +285,7 @@ Killing a spec from outside Minions (raw `Stop-Process`) leaves a stale SQL row
|
|
|
281
285
|
|
|
282
286
|
## Configuration
|
|
283
287
|
|
|
284
|
-
All knobs live under `engine.managedSpawn` in `engine/shared.js` (`ENGINE_DEFAULTS.managedSpawn`). Override per install via `config.json`:
|
|
288
|
+
All knobs live under `engine.managedSpawn` in `engine/core/shared.js` (`ENGINE_DEFAULTS.managedSpawn`). Override per install via `config.json`:
|
|
285
289
|
|
|
286
290
|
| Key | Default | Notes |
|
|
287
291
|
|---|---|---|
|
|
@@ -308,7 +312,7 @@ All knobs live under `engine.managedSpawn` in `engine/shared.js` (`ENGINE_DEFAUL
|
|
|
308
312
|
| Dispatch ERROR `failure_class: invalid-managed-spawn` | Sidecar schema/allowlist violation | Read inbox alert; the validator includes a precise reason. Non-retryable — fix and re-dispatch. |
|
|
309
313
|
| Dispatch ERROR `failure_class: managed-spawn-healthcheck-failed` | The target exited before first health, or `timeout_s` elapsed before it became healthy | Check the reported code/signal and bounded `engine/managed-logs/<name>.log` tail. Sibling spawns are left alive. |
|
|
310
314
|
| WI shows yellow `⚠ managed-spawn: N/M healthy` chip on dashboard | Partial healthcheck failure: ≥1 spec passed, ≥1 failed. WI keeps `status: done` (the agent's primary work — getting an accepted sidecar — succeeded); annotation `_managedSpawnPartial = { healthy, failed[{name, reason, log_tail}], evaluated_at }` lives on the WI. Click the row to see per-spec failure reasons + last 20 log lines. Dispatch is still recorded ERROR with `failure_class: managed-spawn-healthcheck-failed`. Restart the failing spec via `POST /api/managed-processes/restart` once you've fixed the root cause (often: workspace deps not installed; see smoke-test rule in the hint). W-mpbpexrg00110661. |
|
|
311
|
-
| Spec gone after `minions restart` | The tracked Node launcher died or boot reconciliation dropped its PID | Inspect the managed log and verify `engine/managed-spawn-launcher.js` remained the process-tree root until the target exited. |
|
|
315
|
+
| Spec gone after `minions restart` | The tracked Node launcher died or boot reconciliation dropped its PID | Inspect the managed log and verify `engine/processes/managed-spawn-launcher.js` remained the process-tree root until the target exited. |
|
|
312
316
|
| Spec listed `alive: true, healthy: false` for >30s | Healthcheck loop self-detected service degradation | The spec did not pass a subsequent healthcheck. Inspect the service; restart via API once recovered. |
|
|
313
317
|
| Stale row sticks around with dead PID | Spec killed outside Minions | Wait one sweep cycle (~30 min) or call `POST /api/managed-processes/kill` manually. |
|
|
314
318
|
|
|
@@ -323,15 +327,15 @@ All knobs live under `engine.managedSpawn` in `engine/shared.js` (`ENGINE_DEFAUL
|
|
|
323
327
|
|
|
324
328
|
| File | Purpose |
|
|
325
329
|
|---|---|
|
|
326
|
-
| `engine/managed-spawn.js` | Schema, validator, spawn, healthcheck, sweep, state-file I/O. All pure helpers; no side effects on import. |
|
|
330
|
+
| `engine/processes/managed-spawn.js` | Schema, validator, spawn, healthcheck, sweep, state-file I/O. All pure helpers; no side effects on import. |
|
|
327
331
|
| `engine.js` `onAgentClose` (line ~2247) | Acceptance gate (item 2) + healthcheck gate (item 3) wired into dispatch result. |
|
|
328
332
|
| `engine.js` `tickInner` (line ~5687) | Per-30-tick `sweepManagedSpawn()` invocation. |
|
|
329
|
-
| `engine/cli.js` | Boot-path `bootReconcileManagedSpawn()` invocation. |
|
|
330
|
-
| `engine/playbook.js` | Auto-inject `## Live managed processes` block + `managed_spawn` hint section. |
|
|
331
|
-
| `engine/projects.js` `removeProject` | `removeManagedSpecsForProject` cleanup hook. |
|
|
333
|
+
| `engine/operations/cli.js` | Boot-path `bootReconcileManagedSpawn()` invocation. |
|
|
334
|
+
| `engine/agents/playbook.js` | Auto-inject `## Live managed processes` block + `managed_spawn` hint section. |
|
|
335
|
+
| `engine/planning/projects.js` `removeProject` | `removeManagedSpecsForProject` cleanup hook. |
|
|
332
336
|
| `dashboard.js` | 5 routes (`/api/managed-processes`, `…/by-name/<n>`, `…/kill`, `…/restart`, `…/log-stream/<n>`). |
|
|
333
337
|
| `dashboard/js/render-managed.js` + `dashboard/pages/engine.html` | "Managed Processes" panel + log-viewer modal. |
|
|
334
|
-
| `engine/shared.js` `ENGINE_DEFAULTS.managedSpawn` | All configurable knobs in one place. |
|
|
338
|
+
| `engine/core/shared.js` `ENGINE_DEFAULTS.managedSpawn` | All configurable knobs in one place. |
|
|
335
339
|
|
|
336
340
|
## Plan items
|
|
337
341
|
|
package/docs/named-agents.md
CHANGED
|
@@ -48,7 +48,7 @@ The full per-agent config shape supported under `config.agents.<id>`:
|
|
|
48
48
|
### Three-tier resolution chain
|
|
49
49
|
|
|
50
50
|
Most knobs resolve **per-agent override → `engine.*` fleet default → runtime default**.
|
|
51
|
-
The resolver helpers live in [`engine/shared.js`](../engine/shared.js):
|
|
51
|
+
The resolver helpers live in [`engine/core/shared.js`](../engine/core/shared.js):
|
|
52
52
|
|
|
53
53
|
| Helper | Chain |
|
|
54
54
|
|--------|-------|
|
|
@@ -73,7 +73,7 @@ but only as the middle tier.
|
|
|
73
73
|
### Workspace manifest (optional)
|
|
74
74
|
|
|
75
75
|
`agents.<id>.workspace_manifest` declares declarative permission scoping. Shape and
|
|
76
|
-
defaults (`WORKSPACE_MANIFEST_DEFAULTS` in `engine/shared.js`):
|
|
76
|
+
defaults (`WORKSPACE_MANIFEST_DEFAULTS` in `engine/core/shared.js`):
|
|
77
77
|
|
|
78
78
|
| Key | Default | Effect |
|
|
79
79
|
|-----|---------|--------|
|
|
@@ -82,7 +82,7 @@ defaults (`WORKSPACE_MANIFEST_DEFAULTS` in `engine/shared.js`):
|
|
|
82
82
|
| `allowed_external_urls` | `null` (permissive) | URL allow-list (supports `*.example.com` wildcards). |
|
|
83
83
|
| `memory_scope` | `'shared'` | One of `private`, `shared`, `read-only-shared`. |
|
|
84
84
|
|
|
85
|
-
Enforcement helpers (also in `engine/shared.js`): `validateWorkspaceManifest`,
|
|
85
|
+
Enforcement helpers (also in `engine/core/shared.js`): `validateWorkspaceManifest`,
|
|
86
86
|
`resolveAgentManifest`, `agentCanUseRepo`, and `mergeManifestAllowedTools`. Today the
|
|
87
87
|
engine actively enforces (a) the
|
|
88
88
|
**repo gate** at dispatch time in `engine.js spawnAgent`
|
|
@@ -146,7 +146,7 @@ alternate is available it stays with the same agent and dedups an engine inbox n
|
|
|
146
146
|
`agentLock: true` (or `hardAgent: true`) hard-pins the agent and bypasses reassignment —
|
|
147
147
|
operator intent wins. The counter is cleared on successful completion alongside
|
|
148
148
|
`_retryCount`. Helpers: `bumpAgentRetryCount`, `getAgentRetryCount`,
|
|
149
|
-
`resolveMaxRetriesPerAgent` in `engine/shared.js`.
|
|
149
|
+
`resolveMaxRetriesPerAgent` in `engine/core/shared.js`.
|
|
150
150
|
|
|
151
151
|
## 4. How skills are defined
|
|
152
152
|
|
|
@@ -163,7 +163,7 @@ an `expertise ?? skills` fallback — see [`docs/deprecated.json`](deprecated.js
|
|
|
163
163
|
**(b) Real runtime skills** — `SKILL.md` files the runtime CLI actually loads. These are
|
|
164
164
|
auto-extracted from agent output: when an agent emits a fenced ```` ```skill ```` block
|
|
165
165
|
(with YAML frontmatter carrying at least a `name`), `extractSkillsFromOutput` in
|
|
166
|
-
[`engine/lifecycle.js`](../engine/lifecycle.js) writes it out. The engine also scans
|
|
166
|
+
[`engine/orchestration/lifecycle.js`](../engine/orchestration/lifecycle.js) writes it out. The engine also scans
|
|
167
167
|
`notes/inbox/` notes for skill blocks, since agents often write them there instead of stdout.
|
|
168
168
|
|
|
169
169
|
- **`scope: minions`** (the default when `scope` is omitted) → written to the runtime's
|
|
@@ -198,7 +198,7 @@ convention is documented in
|
|
|
198
198
|
[`knowledge/agents/README.md`](../knowledge/agents/README.md).
|
|
199
199
|
|
|
200
200
|
**Written exclusively by the consolidation pipeline.** `appendToAgentMemory` in
|
|
201
|
-
[`engine/consolidation.js`](../engine/consolidation.js) routes each `notes/inbox/` finding
|
|
201
|
+
[`engine/memory/consolidation.js`](../engine/memory/consolidation.js) routes each `notes/inbox/` finding
|
|
202
202
|
to its author's file. Authorship comes from the inbox note's YAML frontmatter `agent:`
|
|
203
203
|
field, with the **filename prefix** (`<agent>-…md`) as fallback (`extractInboxAgent`). Key
|
|
204
204
|
properties:
|
|
@@ -213,7 +213,7 @@ properties:
|
|
|
213
213
|
set; `temp-*` ids are skipped. This is a strict superset of broadcast consolidation: the
|
|
214
214
|
shared `notes.md` digest still happens; per-agent routing is *in addition*.
|
|
215
215
|
|
|
216
|
-
**Prompt memory modes.** [`engine/playbook.js`](../engine/playbook.js) always
|
|
216
|
+
**Prompt memory modes.** [`engine/agents/playbook.js`](../engine/agents/playbook.js) always
|
|
217
217
|
places non-empty `pinned.md` first. Default active retrieval then injects a
|
|
218
218
|
bounded `Relevant Memory` pack when it finds evidence. If retrieval is disabled,
|
|
219
219
|
shadowed, empty, or fails, the engine instead injects bounded `notes.md` followed
|