@voybio/ace-swarm 2.4.4 → 2.4.7

package/CHANGELOG.md

@@ -1,5 +1,74 @@
 # Changelog
 
+## [Unreleased]
+
+## [2.4.6] - 2026-07-15
+
+### Added
+
+- `ace ls` / `ace read` (aliases `get`, `cat`) — human read surface for ACEPACK-only artifacts.
+- `ace fsck` (+ `ace doctor` integration): header/index/length-prefix/CRC/event-section validation, orphan detection, `--repair`.
+- Per-chunk CRC32 in the ACEPACK index; `AcePackedStoreCorruptionError`; corrupt-store quarantine before any rewrite.
+
+### Changed
+
+- Store-first by default: all generated `agent-state/**` artifacts are ACEPACK-only unless on the projection allowlist (`.agents/ACE/tasks/todo.md`). `ACE_STORE_FIRST=0` keeps legacy disk read precedence for one release.
+- `ace compact` now also sweeps loose generated artifacts. `ace cache` is **deprecated** and is *not* an alias for it: `cache` ingests + cleans without compacting.
+- `ace goal start` refuses an objective whose compiled plan carries no machine-checkable obligation, with reason code `contract_uncheckable`. Declare an expected artifact, or add a `verify: <shell command>` success criterion.
+- `ace goal resume` now un-parks a `--plan-only` goal (previously the compiled plan was unreachable), and refuses to resume a `contract_uncheckable` goal, which re-running cannot change.
+- `ace eval`'s default battery ships at `assets/eval-battery/battery.json`; `--battery` is no longer needed.
+- `ace eval --adapt --epochs N` now re-runs the battery once per epoch against the updated playbook instead of re-reflecting a single cached trace N times.
+- `ace help` lists `goal`, `playbook`, `eval`, `compact`, `migrate`, `repair`, and `add skills`.
+- Handoff payloads, source-edit events, playbook data, and scheduler/context-snapshot projections are store-backed; root `tasks/SWARM_HANDOFF.*.json` is no longer written.
+
+### Fixed
+
+- Preserved the configured provider profile for isolated `ace eval` runs while keeping task writes and traces inside disposable workspaces.
+- Restricted goal verification directives so model-authored or operator prose containing `verification:` cannot become shell commands.
+- Refreshed hosted-provider defaults and setup guidance, including Claude Opus 4.8 and the Anthropic OpenAI-compatible endpoint.
+- Unified mutation-intent completion checks, accepted `--flag=value` in eval/playbook commands, scoped concurrent Hermes subprocess interruption per turn, and bounded large diffs to changed regions.
+
+- Silent truncation / cross-section bleed on corrupt-index store reads; stale snapshot-cache serving; event-section trust of corrupt counts.
+- **Lifecycle-sweep deadlock:** `ProjectionManager.projectAfterCommit()` awaited the workspace sweep from inside `withStoreWriteCoordinator`; on a dirty workspace (>=1 loose file older than 5 min) the sweep's own coordinated write re-entered the non-reentrant write queue and hung forever. The sweep is now detached (fire-and-forget, throttled to once per 60s) and exposed via `waitForPendingLifecycleSweep()` for CLI/test flows to drain.
+- **Uncoordinated source-edit event writes:** `appendSourceEditEvent` wrote the store directly via `writeStoreBlobSync` with no lease, risking lost updates under concurrent writers and unbounded NDJSON growth. Events are now buffered and flushed through a coordinated writer, with a 2000-line rotation cap into `state/source-edit-events-archive.ndjson`.
+- **`ace fsck --repair` deleting live write leases:** lock directories are now classified (`active`/`stale`/`unreadable`) before repair; only stale or unreadable leases are removed, and an active lease surfaces as a passing `workspace:write-lock` check instead of a false orphan.
+- **fsck false failures on v1 stores:** `header:regions` and chunk-bounds checks now branch on store version instead of assuming v2 header fields (v1 has `evtOffset=0` and chunks starting at `HEADER_SIZE_V1`, not `HEADER_SIZE`).
+- `ace read --out <file> <path>` no longer misparses the flag's value as the target path; `ace ls` reads sizes from the store index instead of decoding every blob.
+- **`ace compact` / `ace cache` destroyed all goal state (data loss):** the sweep ingested `agent-state/goals/**` into ACEPACK and deleted it from disk, but `GoalRepository` reads plain files — every goal became `Unknown goal_id` and any detached `ace goal run` was orphaned. The sweep now excludes that directory, and `ace repair` restores goals stranded by a pre-fix compact.
+- **Success contract could be satisfied vacuously:** `checkSuccessContract` returned `ok` when a contract had zero enforceable obligations, so a model reached `status: "done"` on prose alone. Contracts with no required artifact, no path-like evidence ref, and no verification command are now refused at `startGoal` and again at verify time. `required_evidence_refs` are enforced; non-path refs (e.g. `reviewer sign-off`) route to `uncompilable_criteria` instead of being silently unsatisfiable.
+- **`ace eval` could not run from a published install:** the default battery resolved under `test/`, which `package.json` `files[]` excludes. `loadBattery` now fails with an actionable message rather than a raw `ENOENT`, and `--adapt` validates the battery before creating `agent-state/eval-reports/`.
+- **Flag values leaked into positional text:** `ace goal start "Ship it" --budget-hours 2` stored the objective `"Ship it 2"`, and `ace playbook add --section verification` created a bullet whose content was literally `"--section verification"`. Both now parse `--flag value` and `--flag=value` correctly and honour `--`.
+- `ace playbook tag <unknown-id>` exits 1 instead of reporting success; `ace goal run|resume|stop <unknown-id>` print `Unknown goal: <id>` instead of dumping a raw stack trace.
+- `ace eval --adapt`'s baseline pass no longer mutates the playbook it is a control for (`ACE_PLAYBOOK_REFLECT=0` during every measured run), so `playbook_stats` counts exactly the harness's own curations.
+
+## [2.4.5] - 2026-07-09
+
+Mitigations from `REPORT_ACE_DEEP_INVESTIGATION_2026-07-01.md` (WS-A/B/C/D/E/G plus the WS-H telemetry seam), with code-review tightening of the already-landed trenches.
+
+### Fixed
+
+- **D1 / WS-A — Hermes zero-tool shadow:** `defaultToolScopeForPolicy` is now total (never returns `undefined`/`[]`); undefined scopes resolve to the role's `ROLE_TOOL_DEFAULTS` (with a new `builder` entry) and unknown roles to orchestrator defaults minus mutation tools. The Hermes executor and `ace mcp-shadow --tools ""` now hard-fail with `reason_code=hermes_empty_tool_scope`. Each Hermes-local turn records a capability snapshot. `ace doctor --hermes` now asserts a computed non-coders (orchestrator) scope is actually served (>=3 tools), not just `get_all_agents_summary`.
+- **D2 / WS-B — fake green:** no-model-runtime orchestrator steps now return `blocked`/`no_model_runtime` instead of `completed`; the supervisor maps `blocked` bridge results to blocked steps; `run_orchestrator` short-circuits to a plan envelope (`execution_backend:"none"`) and gained an explicit `mode:"plan_only"`.
+- **D6 (partial) / WS-A step 6 — Hermes secret leak & context:** the Python worker env is now allowlisted (`PATH`/`HOME`/`HERMES_*`/`ACE_*`/provider base-URL vars) instead of forwarding the full `process.env`; the hard-coded 65536 context length is replaced by a probed-or-8192 conservative default.
+- **D14 — promotion predicate:** structural-edit `promotable` now keys on the unified-diff result, not the set-membership `diffContents` (display-only).
+- **D5 / WS-G step 3 — version truth:** the MCP server now reports the real `package.json` version instead of the stale `0.2.1`.
+- **D11 / §14 — hygiene:** removed the stray `src/.vericify/` dogfood state and git-ignored dogfood/staging litter paths.
+
+### Added
+
+- **WS-J — Experience Playbook (experimental):** added fs-backed playbook bullet storage under `agent-state/playbook/`, deterministic reflection/curation, `ACE_PLAYBOOK`-gated prompt injection, direct-lane attribution, adaptation reports via `ace eval --adapt`, and `ace playbook` CLI/projection support. The legacy `get_lessons`/`add_lesson` MCP tools now read/write playbook bullets instead of appending a second markdown memory family. Injection remains off by default pending adaptation evidence; no paper-magnitude gains are claimed.
+- **WS-C — forward-pacing gate:** the completion gate is now conjunctive — a `completed` claim is demoted to `blocked`/`forward_pacing_incomplete` unless every declared `expected_artifact` and `required_evidence_ref` is satisfied (unstructured steps unchanged). `run_orchestrator` threads both into the bridge. Added a tool doom-loop brake (identical tool+result 3x warns, 4x blocks with a native blocker post; polling with changing results is exempt) and an inter-turn text-loop brake for envelopes that execute no tool (e.g. repeated empty `tool_calls`). Per-step `PLAN_UPDATE` status events carry `{plan_id, step_id, from, to, in_progress_step_id}`. Coders/builder guardrails forbid `status:complete` while artifacts/evidence are missing.
+- **WS-H (seam) — token truth:** provider usage (`prompt_eval_count`/`eval_count`, OpenAI usage) is threaded into `BridgeResult.usage_totals`; a context-overflow sentinel blocks and emits `MODEL_CONTEXT_OVERFLOW_SUSPECTED` at >=98% of `num_ctx`; an output cap (`num_predict`/`max_tokens`) bounds runaway generation; retries are restricted to transport-class errors (5xx/408/429/network), never deterministic 4xx; envelope-required roles request JSON mode (`format:"json"` / `response_format`); a per-run NDJSON bridge trace is written under `agent-state/bridge-traces/`.
+- **WS-E — surgical-edit cost & concurrency:** full-workspace staging only when validation is requested; staging prunes `node_modules`/`.git` at any depth (not just top-level); `expected_base_hash` fail-closed precondition; a source-mutation lease coordinator with `SOURCE_EDIT_*` events; backups moved under `.ace-staging/`; `gcStaging` GC wired into `ace mcp`/`ace cache`.
+- **WS-G — premium-host profile & governance throttle:** `ace mcp --profile host-min|full` (host-min = 22 curated tools, ~22 KB vs 113/~68 KB); governance mutation guidance is now delta-only (append on digest change or every 10th call) and the bootstrap nudge is throttled per session; `buildAceDigest` is memoized with a 60s TTL to stop per-call file I/O.
+- **D3 / WS-D — in-process job worker & heartbeat watchdog:** the scheduler was a pure planner — `dispatch_jobs` moved jobs to `running` and grabbed locks but nothing ever executed a job body or called `complete_job`, and `dispatchJobs` re-stamped every running lock's heartbeat each tick, which *masked* a dead worker and left jobs stuck `running` forever. Dispatch no longer renews running-job heartbeats: only a live worker (`heartbeat_job`) or a fresh start proves liveness, so a missing heartbeat reliably signals a dead worker. A heartbeat watchdog (`DEFAULT_HEARTBEAT_STALE_MS`, default 120 s; tunable per tick via `heartbeat_stale_seconds`) now runs inside every `dispatch_jobs` tick and as a standalone `reap_stale_jobs` tool, moving stale running jobs into `unknown_recovery` and releasing their locks (`SCHEDULER_WATCHDOG` events, `reason_code=heartbeat_stale`) — unlike lease recovery this fires even under the same owner with a valid lease. New `src/job-worker.ts` closes the execution loop: a `kind`→handler registry plus `runJobWorkerOnce` that dispatches, executes each running job's handler (heartbeating throughout, never holding the scheduler lock across a handler), and finalizes via `complete_job`. New MCP tools `heartbeat_job` and `reap_stale_jobs` (full surface now 115 tools). Prerequisite for the WS-F Goal Engine.
+
+## [2.4.4] - 2026-06-25
+
+### Fixed
+
+- Fixed the Hermes bridge session manager dropping `final`/`error` frame semantics when the final NDJSON frame arrived without a trailing newline (recovered only by the post-close tail-flush). A newline-less `final` no longer reports `turns=0` with an empty summary, and a newline-less `error` frame is no longer silently reported as `completed`. The per-frame extraction is now shared by the live stream handler and the close-flush path so they cannot drift.
+
 ## [2.4.3] - 2026-05-23
 
 ### Changed