@voybio/ace-swarm 2.4.3 → 2.4.6

package/docs/hermes.md

@@ -0,0 +1,129 @@
+# Hermes Companion Engine
+
+> Companion to the [Local Models & Hermes section in the README](../README.md#local-models--hermes). That section explains why the bridge exists; this document covers how to run and operate it.
+
+Hermes is an optional, user-supplied companion execution substrate for local models. ACE does not install Hermes, clone it, or make it a package dependency. When the operator explicitly passes `--hermes` or `--engine hermes_local`, Hermes handles local provider quirks and tool-call framing. The local model provider is still Ollama or llama.cpp. ACE remains the system of record — state, evidence, verification, scheduler — and exports a **scoped shadow MCP tool surface** to Hermes so the local model only sees the tools it needs for a given task scope.
+
+Hermes-local is opt-in. Without `--hermes` or `--engine hermes_local`, the TUI uses the direct provider path and does not require Python, uv, or Hermes.
+
+## Architecture
+
+```
+ACE TUI  (ace tui --hermes / --engine hermes_local)
+        │  user input → ChatSession.sendMessage();
+        │  ACE owns session identity, policy, state
+        ▼
+runLocalModelTask()                    ← src/local-model-runtime.ts
+        │  ACE dispatches, ACE controls the turn
+        ▼
+HermesSubprocessExecutor               ← Node.js class, src/hermes/session-manager.ts
+        │  spawns Python worker; ACE controls lifecycle
+        ├── writes hermesHome/config.yaml (provider, model, user-supplied Hermes path, shadow MCP entry)
+        ├── pipes ace mcp-shadow --tools <csv>  ← scoped tools-only shadow server (stdio)
+        │         (ACE-owned; local model sees only allowed tools, not public prompts/resources)
+        ▼
+hermes_bridge_worker.py                ← Python subprocess
+        │  reads config.yaml; calls shadow MCP & local model
+        ├── streams framed NDJSON events → HermesBridgeFrameDecoder
+        └── calls local model (ollama / llama.cpp)
+        │
+        ▼  bridge frames validated and applied by ACE
+ACE store (ace-state.ace)              ← authoritative output; ACE-owned
+        evidence, continuity, run ledger persisted here
+```
+
+## Bridge protocol (one-shot)
+
+Each turn starts one worker process. Node writes one JSON request to stdin and closes stdin; the worker emits newline-delimited JSON events on stdout and exits. This is not a persistent or bidirectional session protocol. Events are decoded as they arrive, but the current TUI displays the result after process exit rather than rendering token deltas live.
+
+The request contains the session and turn IDs, task, role, provider, model, base URL, maximum turns, and system prompt. Worker events carry a `type` field:
+
+| Frame type | Direction | Purpose |
+|---|---|---|
+| `session_open` | H→N | worker opened the requested session |
+| `turn_run` | H→N | worker began the task turn |
+| `session_close` | H→N | worker closed the session |
+| `status` | H→N | heartbeat / progress |
+| `delta` | H→N | streaming token chunk |
+| `reasoning` | H→N | chain-of-thought (optional) |
+| `assistant_interim` | H→N | partial assistant message |
+| `tool_start` | H→N | tool call initiated |
+| `tool_progress` | H→N | tool call in flight |
+| `tool_complete` | H→N | tool call result |
+| `final` | H→N | complete assistant response |
+| `error` | H→N | fatal bridge error |
+
+Frame decoding is handled by `HermesBridgeFrameDecoder` in `src/hermes/bridge-protocol.ts`. The decoder strips control bytes, sanitizes the line, and validates against the event schema before handing frames to the session manager.
+
+Exactly one `final` event is required. A missing or duplicate terminal event, malformed frame, child error/nonzero exit, timeout, or output-limit breach fails the turn. `final.completed=false` is preserved as a blocked result. The executor owns cancellation: TUI Ctrl+C sends `SIGTERM`, followed by `SIGKILL` if the worker does not stop. Default limits are 120 seconds and 4 MiB combined stdout/stderr per turn.
+
+## Running with Hermes
+
+The operator owns the Hermes and Python setup. ACE only discovers and launches it when asked.
+
+```bash
+# Doctor check with user-supplied Hermes/Python
+ACE_HERMES_ROOT=/path/to/hermes-agent \
+HERMES_PYTHON=/path/to/python \
+ace doctor --llm llama.cpp --base-url http://127.0.0.1:8080/v1 --model gemma-3-4b-it --hermes
+
+# TUI session with Hermes
+ACE_HERMES_ROOT=/path/to/hermes-agent \
+HERMES_PYTHON=/path/to/python \
+ace tui --provider llama.cpp --base-url http://127.0.0.1:8080/v1 --model gemma-3-4b-it --hermes
+
+# Optional uv-managed Hermes project, still operator supplied
+ACE_HERMES_ROOT=/path/to/hermes-agent \
+ACE_HERMES_UV_PROJECT=/path/to/hermes-agent \
+ace tui --provider llama.cpp --base-url http://127.0.0.1:8080/v1 --model gemma-3-4b-it --hermes
+```
+
+Launch precedence is explicit CLI options, environment, the store-backed `state/runtime/hermes_launch_profile`, then development defaults. Supported environment configuration is:
+
+- `ACE_HERMES_ROOT`: operator-supplied Hermes checkout.
+- `HERMES_PYTHON`: Python executable containing Hermes dependencies. If omitted, launch resolution uses `python3`.
+- `ACE_HERMES_UV_PROJECT`: run `uv run --project <path> python`.
+- `ACE_HERMES_COMMAND_JSON`: advanced command override as a JSON string array, for example `["uv","run","python"]`; shell strings are rejected.
+
+For llama.cpp, use its OpenAI-compatible API root ending in `/v1` (for example `http://127.0.0.1:8080/v1`). Ollama’s native root (for example `http://127.0.0.1:11434`) is normalized to `/v1` for Hermes.
+
+Or select the engine explicitly:
+
+```bash
+ace tui --provider llama.cpp --base-url http://localhost:8080 --engine hermes_local
+# shorthand
+ace tui --provider ollama --model qwen3:8b --hermes
+```
+
+When Hermes is selected, ACE launches the user-supplied Hermes/Python setup for that session and exposes only the scoped shadow MCP tool surface. Tool evidence, not model prose, determines executable mutation.
+
+## Doctor checks
+
+`ace doctor` surfaces Hermes-local readiness gates only when `--hermes` is passed:
+
+```bash
+ACE_HERMES_ROOT=/path/to/hermes-agent HERMES_PYTHON=/path/to/python \
+  ace doctor --llm llama.cpp --model gemma-3-4b-it --base-url http://localhost:8080 --hermes
+```
+
+Checks performed:
+
+- Hermes bridge worker packaged (`scripts/hermes_bridge_worker.py` present)
+- Hermes checkout supplied by the operator (`ACE_HERMES_ROOT` or `--hermes-root`)
+- Python supplied by the operator (`HERMES_PYTHON` or `--hermes-python`)
+- Optional uv project supplied by the operator (`ACE_HERMES_UV_PROJECT` or `--hermes-uv-project`)
+- Shadow MCP command available for scoped tools-only transport
+- Local runtime reachable and model available
+
+```bash
+# Auto-scan common Ollama + llama.cpp endpoints
+ace doctor --scan
+```
+
+## Evidence model
+
+ACE persists tool evidence when the model invokes `write_workspace_file` successfully through the shadow MCP surface. Completion is authorized by tool evidence, not model prose: Hermes results pass through the same forward-pacing verifier as direct results. A mutation-intent `completed` claim is demoted to `blocked` unless persisted evidence exists; callers may additionally supply declared artifacts and required evidence references.
+
+The Hermes-final artifact materialization path (`materialized_from_hermes_final`) is provided only as a development diagnostic (`scripts/dev/run_hermes_crispr_live.mjs`); it is not part of the standard Hermes-local runtime.
+
+Which local model runs is chosen by the operator at `ace connect` or through the runtime profile.

package/docs/library-api.md

@@ -0,0 +1,31 @@
+# Package API Boundary
+
+`@voybio/ace-swarm` is currently an executable MCP server and CLI package, not a supported JavaScript library SDK.
+
+## Supported entrypoints
+
+- Binaries: `ace`, `ace-swarm`, `ace-mcp-server`, and `ace-add`
+- MCP transport: `ace mcp` or `ace serve`
+- Operator CLI: commands documented in [CLI Reference](cli-reference.md)
+- Programmatic protocol: MCP tools, resources, templates, and prompts documented in [MCP Tool Reference](mcp-tool-reference.md)
+
+Importing the package root executes the MCP stdio server because `dist/index.js` calls `startStdioServer()` immediately. It does not expose a stable function/class API. Internal files under `dist/` are implementation details and are not package-exported subpaths.
+
+```js
+// Do not use this as a library import: it starts the stdio server.
+import "@voybio/ace-swarm";
+```
+
+## Integration choices
+
+Use one of these stable boundaries:
+
+1. Configure the MCP server in a supported host with `ace mcp-config`.
+2. Spawn the `ace` CLI and consume its documented stdout/exit contract.
+3. Use an MCP client and the registered tool/resource schemas.
+
+Do not import `dist/store/*`, `dist/tools-*`, or other internal modules from a consumer package. Their paths, types, and side effects may change without a semver API guarantee.
+
+## Future SDK contract
+
+A supported library API would require explicit side-effect-free exports, documented lifecycle ownership, stable error/result types, and package-level integration tests. Until such exports appear in `package.json`, the CLI and MCP protocol are the public APIs.

package/docs/mcp-tool-reference.md

@@ -0,0 +1,290 @@
+# MCP Tool Reference
+
+> Companion to the [README](../README.md). The full MCP surface spans **119 registered tools** across 13 domain modules, plus resources and prompts. For premium hosts (Claude Code, Codex), the curated `ace mcp --profile host-min` exposes 22 high-value tools; the rest remain reachable via `run_orchestrator` delegation or `--profile full`.
+
+ACE exposes tools through a domain-modular registration system (`src/tools.ts` delegates to 13 domain modules). Tools can be allow-listed at startup; `ace mcp-shadow` uses this to give Hermes a scoped tools-only subset. Public `ace mcp` always exposes the full prompts and resources surface; its visible tool schema depends on `--profile` (119 tools for `full`, 22 for `host-min`). `ace mcp-shadow` instead exposes only its explicitly allow-listed tools and omits prompts and resources.
+
+## Agent & Runtime
+
+| Tool | Purpose |
+|---|---|
+| `get_agent_instructions` | Read instruction content for any ACE agent role |
+| `get_agent_manifest` | Read the manifest for a given agent role |
+| `get_all_agents_summary` | List all registered agents with role metadata |
+| `get_kernel_artifact` | Read a named kernel artifact from the store |
+| `get_runtime_profile` | Read the active runtime profile (provider, model, base URL) |
+| `validate_runtime_profile` | Schema-validate the stored runtime profile |
+| `get_transition_log` | Read recent agent state-transition history |
+| `get_capability_snapshot` | Read one session/turn capability snapshot (`session_id`, `turn_number`) |
+| `get_runtime_tool_specs` | Read the tool-spec registry for the active runtime |
+| `validate_runtime_tool_specs` | Schema-validate the tool-spec registry |
+| `list_runtime_tool_specs` | List all registered tool specs |
+| `execute_runtime_tool` | Execute a named runtime tool by spec |
+
+## Orchestration & Planning
+
+| Tool | Purpose |
+|---|---|
+| `propose_plan` | Generate a multi-step task plan proposal |
+| `validate_plan` | Validate a plan proposal against acceptance contracts |
+| `run_local_model` | Dispatch a task to the configured local model runtime |
+| `run_orchestrator` | Run an ACE orchestrator session for a given task |
+| `route_task` | Route an incoming task to the appropriate agent role |
+| `bootstrap_state` | Seed or repair the ACE store for the current workspace |
+| `get_framework_status` | Read aggregate framework health and readiness |
+| `validate_framework` | Run the built-in framework self-checks (manifest gates are run by `execute_gates`) |
+| `audit_public_surface` | Audit exact registered tools/resources/prompts, emitted-event registry parity, and parse every bound JSON schema |
+| `audit_store_authority` | Verify store authority and projection consistency |
+| `execute_gates` | Run explicit gate manifest IDs, or the active stage's gates when IDs are omitted. Shipped gates: `gate-autonomy`, `gate-completeness`, `gate-operability`, `gate-security` |
+
+## Handoffs
+
+| Tool | Purpose |
+|---|---|
+| `create_handoff` | Create a typed, validated handoff record |
+| `validate_handoff` | Validate a handoff payload against the schema |
+| `get_handoff_template` | Retrieve the single canonical handoff template (no arguments) |
+| `get_handoff_status` | Read the status of a specific handoff |
+| `ack_handoff` | Acknowledge a handoff as received and accepted |
+
+Example — creating a handoff from orchestrator to qa:
+
+```text
+# via any MCP client
+create_handoff {
+  "from": "orchestrator",
+  "to": "qa",
+  "title": "Review auth module test coverage",
+  "task_type": "engineering",
+  "priority": "P1",
+  "directive": "Auth module implementation complete; review and extend test coverage",
+  "acceptance_criteria": ["All /api/auth endpoints covered by integration tests"],
+  "context_pointers": { "engineering_constraint": "src/auth.ts:1" }
+}
+```
+
+## Scheduler
+
+| Tool | Purpose |
+|---|---|
+| `enqueue_job` | Add a job to the queue with priority, deps, and resource locks |
+| `dispatch_jobs` | Tick the scheduler and dispatch ready jobs (runs the heartbeat watchdog) |
+| `dispatch_job_now` | Immediately start a single ready job if deps and resources are clear |
+| `ack_job` | Acknowledge or update job lifecycle (accept/reject/cancel/resume) |
+| `complete_job` | Mark a job done or failed |
+| `heartbeat_job` | Prove worker liveness for a running job so the watchdog does not reap it |
+| `reap_stale_jobs` | Run the heartbeat watchdog once: recover stale running jobs, release locks |
+| `list_jobs` | Read the current job queue and lock-table snapshot |
+| `get_lock_table` | Read active resource locks |
+| `get_scheduler_lease` | Read the current scheduler lease |
+
+Example — queuing a blocked job:
+
+```text
+enqueue_job {
+  "kind": "handoff.lifecycle",
+  "priority": "P1",
+  "dependencies": [{ "kind": "handoff", "ref": "HND-20260710-abc12345", "expected": "accepted" }],
+  "resource_requirements": ["agent-state/handoff-registry.json"],
+  "run_after": "2030-05-08T18:00:00Z"
+}
+```
+
+## Goals
+
+| Tool | Purpose |
+|---|---|
+| `start_goal` | Compile an objective into a validated plan and open a goal record |
+| `get_goal_status` | Read goal status, tick/replan counts, ready steps, and escalations |
+| `resume_goal` | Resume a stopped or escalated goal from its persisted state |
+| `stop_goal` | Operator stop for a running goal |
+
+A goal reaches `done` only through evidence-gated success; it otherwise leaves `executing` through budget exhaustion, operator stop, or an escalated blocker. The same loop is available from the CLI as `ace goal start|status|run|resume|stop`.
+
+## Files & Structural Editing
+
+| Tool | Purpose |
+|---|---|
+| `read_workspace_file` | Read any file in the workspace |
+| `write_workspace_file` | Write or overwrite a file (evidence-tracked) |
+| `list_workspace` | List one workspace directory (`path`; no glob filtering) |
+| `safe_edit_file` | Edit a file with collision-safe write semantics |
+| `diff_files` | Produce a sequence-aware unified diff; set `mode` explicitly to `content` or `path` |
+| `read_file_lines` | Read a line range from a file |
+| `outline_file` | Extract code declarations using language-oriented declaration patterns (not Markdown headings) |
+| `astgrep_query` | Execute ast-grep directly against a workspace directory |
+| `astgrep_locate` | Locate a specific symbol or construct with ast-grep |
+| `compile_structural_edit` | Compile an ast-grep rewrite to a patch |
+| `preview_structural_edit` | Preview a structural edit without applying |
+| `astgrep_rewrite` | Apply a rewrite to one promoted target; multi-file rewrites are refused |
+| `apply_patch` | Apply a unified diff patch to the workspace |
+
+## Change Intelligence & Drift
+
+| Tool | Purpose |
+|---|---|
+| `snapshot_state` | Take a semantic snapshot of the current workspace state |
+| `semantic_diff` | Diff two semantic snapshots |
+| `drift_report` | Produce a drift report between baseline and current state |
+| `list_snapshots` | List available workspace snapshots |
+| `rewrite_targets` | Identify files that are candidates for rewrite based on drift |
+
+## Discovery & Indexing
+
+| Tool | Purpose |
+|---|---|
+| `refresh_astgrep_index` | Rebuild the ast-grep structural index |
+| `scan_workspace_delta` | Scan for file changes since the last index pass |
+| `web_research_packet` | Package a web research context for agent consumption |
+| `mcp_ingestion_probe` | Perform a bounded MCP initialize over stdio/Streamable HTTP and list tools, resources, resource templates, and prompts |
+| `codemunch_snapshot` | Take a full codemunch snapshot of the workspace |
+| `codemunch_index` | Index a codemunch snapshot for search |
+
+## Git
+
+| Tool | Purpose |
+|---|---|
+| `git_status` | Read working-tree status |
+| `git_commit` | Stage and commit changes with a message |
+| `git_diff` | Diff staged or unstaged changes, optionally restricted by `path` |
+| `git_log` | Read recent commit history |
+
+## Memory & Continuity
+
+| Tool | Purpose |
+|---|---|
+| `context_snapshot` | Capture a structured context snapshot for the current session |
+| `build_continuity_packet` | Build a continuity packet for cross-session handoff |
+| `recall_context` | Recall an exact snapshot name, or list all snapshots when no name is supplied |
+
+Example — before switching models mid-task:
+
+```text
+build_continuity_packet {
+  "snapshot_name": "auth-module-pre-model-switch",
+  "source_paths": ["src/auth.ts", "src/auth.test.ts"],
+  "max_status_events": 20
+}
+```
+
+## Lifecycle & Status Events
+
+| Tool | Purpose |
+|---|---|
+| `emit_status_event` | Append a typed lifecycle event to `STATUS_EVENTS.ndjson` |
+| `list_status_events` | Read recent lifecycle events with optional filters |
+| `open_circuit_breaker` | Trip a named circuit breaker to halt a failing upstream path |
+| `close_circuit_breaker` | Reset a tripped circuit breaker |
+| `generate_kanban_ui` | Rebuild the kanban board from current state |
+
+Example:
+
+```text
+emit_status_event {
+  "source_module": "capability-ops",
+  "event_type": "TASK_COMPLETE",
+  "status": "pass",
+  "summary": "Auth integration tests green across all providers"
+}
+```
+
+## Todo & Run Ledger
+
+| Tool | Purpose |
+|---|---|
+| `get_todo` | Read the current todo surface |
+| `get_todo_state` | Read structured todo state from the store |
+| `update_todo` | Update one or more todo items |
+| `get_run_ledger` | Read the run ledger (history of major tool invocations) |
+
+## Vericify (Verification)
+
+| Tool | Purpose |
+|---|---|
+| `get_vericify_context` | Get the current verification context packet |
+| `get_vericify_delta` | Get the incremental verification delta |
+| `get_vericify_bridge` | Read the vericify bridge snapshot |
+| `refresh_vericify_bridge` | Refresh the bridge snapshot from test runner output |
+| `validate_vericify_bridge` | Schema-validate the bridge snapshot |
+| `get_vericify_process_posts` | Read the vericify process post log |
+| `append_vericify_process_post` | Append a test result post to the process log |
+| `validate_vericify_process_post_log` | Schema-validate the process post log |
+
+## Tracker Integration
+
+| Tool | Purpose |
+|---|---|
+| `get_tracker_snapshot` | Read the cached tracker snapshot (GitHub Issues, Linear, etc.) |
+| `validate_tracker_snapshot` | Schema-validate the tracker snapshot |
+| `list_tracker_adapter_kinds` | List available tracker adapters |
+| `get_tracker_adapter_health` | Check health of the active tracker adapter |
+| `list_tracker_items` | List items from the tracker |
+| `create_tracker_comment` | Post a comment to a tracker item |
+| `update_tracker_item_state` | Update the state of a tracker item |
+| `refresh_tracker_snapshot` | Pull fresh data from the tracker into the snapshot |
+
+## Skills & Tasks
+
+| Tool | Purpose |
+|---|---|
+| `skill_catalog_summary` | List all available skills with brief descriptions |
+| `skill_audit` | Audit skill freshness and coverage |
+| `problem_triage` | Triage a problem description against known skills |
+| `list_skills` | List skills in the store |
+| `get_skill_instructions` | Read full instructions for a named skill |
+| `get_task_pack` | Read a named task pack |
+| `get_lessons` | Read lessons (backed by the experience playbook store) |
+| `add_lesson` | Append a lesson as a playbook bullet |
+
+## Workspace Sessions
+
+| Tool | Purpose |
+|---|---|
+| `list_workspace_sessions` | List managed workspace sessions |
+| `create_workspace_session` | Create a new managed workspace session |
+| `remove_workspace_session` | Remove a workspace session |
+| `validate_managed_workspace_path` | Validate a workspace path against the session registry |
+
+## Unattended Sessions
+
+| Tool | Purpose |
+|---|---|
+| `list_unattended_sessions` | List registered unattended agent sessions |
+| `get_unattended_session` | Read details of a specific unattended session |
+| `start_unattended_session` | Launch an unattended agent session |
+| `stop_unattended_session` | Stop a running unattended session |
+| `wait_for_unattended_session` | Block until a session reaches a terminal state |
+| `validate_unattended_session_registry` | Schema-validate the session registry |
+
+## Framework & Config
+
+| Tool | Purpose |
+|---|---|
+| `get_mcp_config` | Read the baked MCP config for a named client |
+| `run_tests` | Auto-detect and run the consumer workspace test suite, or run the supplied command |
+| `get_cli_split` | Read the CLI split configuration |
+
+## Resource and Prompt Catalog
+
+The release surface is audited from the compiled registrations: **70 fixed resources**, **2 resource templates**, and **19 prompts**. `audit_public_surface` writes the complete, description-bearing catalog to `agent-state/PUBLIC_SURFACE_REPORT.md` and fails on count drift, an empty production event scan, unregistered emitted events, duplicate registrations, or invalid bound JSON schemas.
+
+Fixed resource families:
+
+- `ace://agents/{astgrep,builder,coders,docs,eval,memory,observability,ops,orchestrator,planner,qa,release,research,security,skeptic,spec,ui,vos}/{instructions,manifest}` (36)
+- Handoffs: `ace://handoffs/{handoff_example,handoff_example_ui_coders,handoff_example_vos_ui,handoff_template,status}`
+- Discovery and UI: `ace://index`, `ace://index/fingerprints`, `ace://kanban`, `ace://kanban/html`
+- Kernel and execution: `ace://kernel/{agent_registry,directive_kernel}`, `ace://run-ledger`, `ace://runtime/{continuity-packet,executor-sessions,profile,prompt-template,tool-specs,tracker-snapshot,vericify-bridge,vericify-process-posts,workspaces}`
+- Scheduler and skills: `ace://scheduler/{lease,locks,queue}`, `ace://skills`
+- Structural state: `ace://state/{astgrep-commands,astgrep-index/json,astgrep-index/md}`, `ace://status/events`
+- Task state: `ace://tasks/{cli_work_split,lessons,readme,role_tasks,todo}`
+
+Resource templates:
+
+- `ace-skill-instructions` — read a skill's `SKILL.md` by skill name.
+- `ace-workspace-file` — read a workspace-relative file.
+
+Prompts:
+
+`ace-astgrep`, `ace-codemunch`, `ace-coders`, `ace-codesnipe`, `ace-eval`, `ace-genesis`, `ace-handoff`, `ace-initiate`, `ace-lessons`, `ace-memory`, `ace-observability`, `ace-orchestrator`, `ace-release`, `ace-runtime-executor`, `ace-security`, `ace-skill`, `ace-skills-index`, `ace-ui`, and `ace-vos`.
+
+For schema-exact tool inputs, use MCP `tools/list`; for the release-bound schema files and their surface mappings, run `audit_public_surface`. This avoids maintaining a second hand-copied schema that can silently diverge from the registered Zod contracts.

package/docs/provider-security.md

@@ -0,0 +1,85 @@
+# Provider, Credential, and Trust Boundaries
+
+ACE supports local runtimes and hosted/OpenAI-compatible providers. A discovered provider is only a configuration hint; it is not proof of authentication, reachability, model availability, or permission to mutate the workspace.
+
+## Provider configuration
+
+Prefer explicit CLI flags for one-off diagnosis and the stored runtime profile for repeatable operation:
+
+```bash
+ace connect ollama --model qwen3:8b --base-url http://127.0.0.1:11434
+ace doctor --llm ollama --model qwen3:8b --base-url http://127.0.0.1:11434
+
+ace connect llama-server --model local-model --base-url http://127.0.0.1:8080/v1
+ace doctor --llm llama.cpp --model local-model --base-url http://127.0.0.1:8080/v1
+```
+
+Hosted/OpenAI-compatible providers conventionally use `<PROVIDER>_API_KEY` and `<PROVIDER>_BASE_URL`. Codex/OpenAI discovery also recognizes `OPENAI_API_KEY`; Copilot can use authenticated `gh` state or `GITHUB_TOKEN`. Run `ace doctor` rather than assuming an environment variable is sufficient.
+
+### Hosted provider quick setup
+
+Use provider-specific credentials so switching providers does not accidentally reuse a key. ACE supplies the standard OpenAI endpoint for Codex, the Anthropic OpenAI-compatible endpoint for Claude, and the GitHub Models endpoint for Copilot. Gemini requires its OpenAI-compatible endpoint explicitly.
+
+| Provider | Default model | Credential and endpoint |
+|---|---|---|
+| Codex | `gpt-5` | `CODEX_API_KEY`; endpoint defaults to `https://api.openai.com/v1` |
+| Claude | `claude-opus-4-8` | `CLAUDE_API_KEY`; endpoint defaults to `https://api.anthropic.com/v1/` |
+| Gemini | `gemini-2.5-pro` | `GEMINI_API_KEY` and `GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/` |
+| Copilot | `copilot/gpt-5-mini` | `GITHUB_TOKEN` or `GH_TOKEN` (or authenticated `gh`); endpoint is `https://models.github.com/v1` |
+
+Preflight the exact model before opening the TUI:
+
+```bash
+CODEX_API_KEY=... ace doctor --llm codex --model gpt-5
+CODEX_API_KEY=... ace tui --provider codex --model gpt-5
+
+CLAUDE_API_KEY=... ace doctor --llm claude --model claude-opus-4-8
+CLAUDE_API_KEY=... ace tui --provider claude --model claude-opus-4-8
+
+GEMINI_API_KEY=... GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/ \
+  ace doctor --llm gemini --model gemini-2.5-pro
+GEMINI_API_KEY=... GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/ \
+  ace tui --provider gemini --model gemini-2.5-pro
+
+GITHUB_TOKEN=... ace doctor --llm copilot --model copilot/gpt-5-mini
+GITHUB_TOKEN=... ace tui --provider copilot --model copilot/gpt-5-mini
+```
+
+Provider discovery and a successful credential check do not prove that the account can use the requested model. Treat the doctor's served-model result as the preflight gate and keep an explicit `--model` when provider aliases or catalogs can change.
+
+## Secret handling
+
+- Do not put API keys, bearer tokens, cookies, passwords, or credentials in `TASK.md`, prompts, handoffs, evidence logs, runtime tool arguments, or MCP configuration committed to a repository.
+- Pass secrets through the provider's supported environment or operating-system credential store.
+- `mcp_ingestion_probe` rejects secret-like environment keys and records only allowed configuration metadata.
+- Treat `ace mcp-config` output as executable configuration. Review command paths, working directories, and environment forwarding before installing it globally.
+- Store files and projections can contain task context, tool results, paths, and model output. Protect workspace backups accordingly.
+
+## Local endpoint boundary
+
+Ollama and llama.cpp commonly listen on loopback. Do not expose them on a public interface without an authenticated reverse proxy, request limits, and a reviewed model/tool policy. A local model connected to `mcp-shadow` can invoke only its allowed tool scope, but those tools may still read or mutate the workspace.
+
+## MCP trust boundary
+
+- `ace mcp --profile host-min` reduces the host-visible tool schema; `--profile full` exposes the complete tool surface.
+- Hermes uses `mcp-shadow --tools <csv>` and refuses an empty scope.
+- A successful configuration validation is not a successful MCP handshake. Use probe results that include protocol initialization plus tool/resource enumeration.
+- External MCP servers are separate trust domains. Review their executable command, URL, arguments, and data-retention policy before connecting.
+
+## Tracker boundary
+
+Tracker tools may read items and can create comments or change item state. Keep adapters disabled unless configured intentionally, use least-privilege credentials, and test with read operations before enabling mutations.
+
+## Model output and completion
+
+Model prose does not authorize completion or mutation. Mutation-capable runs must be backed by successful tool results and any declared artifacts, evidence refs, and verification commands. A provider returning HTTP 200 or a process exiting 0 is transport evidence only.
+
+## Incident response
+
+If a credential may have entered an artifact or trace:
+
+1. Stop active sessions.
+2. Revoke or rotate the credential at the provider.
+3. Inspect packed and loose artifacts before sharing or backing up the workspace.
+4. Record sanitized evidence; do not copy the secret into the incident log.
+5. Re-run provider doctor checks with the replacement credential.

package/docs/state-model.md

@@ -0,0 +1,113 @@
+# ACEPACK State Model
+
+> Companion to the [ACEPACK summary in the README](../README.md#acepack-state-model). This document describes the implemented v2 format, its authority boundary, and safe maintenance behavior.
+
+ACE's canonical generated workflow state is stored in `agent-state/ace-state.ace`. Older workspaces may still contain `.agents/ACE/ace-state.ace`; that path is a read fallback, not a second authority. If both files exist, `ace fsck` reports a dual-store conflict so the operator can migrate deliberately instead of guessing which file to delete.
+
+One intentional exception remains: active goal records and plans under `agent-state/goals/` are authoritative loose files so detached goal processes can coordinate without rewriting the packed store. A complete backup therefore includes both `agent-state/ace-state.ace` and `agent-state/goals/` when that directory exists.
+
+## On-disk regions
+
+ACEPACK v2 has four regions:
+
+```text
+ace-state.ace
+├── Header (128 bytes, big-endian)
+│     magic · version · flags
+│     kv_index_offset · kv_index_length · kv_chunk_end
+│     evt_offset · evt_length · evt_count · evt_base_id
+├── KV Chunk Region
+│     knowledge/agents/{name}/{file}  → agent instruction text
+│     knowledge/skills/{name}/{file}  → skill content
+│     topology/{kind}                 → swarm registry, route tables
+│     state/{...}                     → handoffs, todo, ledger, scheduler
+│     meta/{...}                      → schema version, project name
+│     each chunk: [uint32 length][bytes]
+├── Columnar Event Section (rewritten on commit)
+│     uint32    event count
+│     int64[N]  timestamps      (epoch ms, big-endian)
+│     uint8[N]  kinds           (EntityKind enum)
+│     uint8[N]  sources         (ContentSource enum)
+│     uint8[N]  flags           (0x01 = deleted)
+│     padding to 4-byte alignment
+│     uint32[N] payload offsets (relative to payload pool)
+│     uint32[N] payload lengths
+│     uint32    pool length
+│     bytes[M]  payload pool    (UTF-8 JSON blobs)
+└── KV Index (JSON, rewritten on every store write)
+      key → { offset, length, crc? }
+```
+
+The header field definitions live in `src/store/types.ts`; serialization and bounds checks live in `src/store/ace-packed-store.ts` and `src/store/fsck.ts`.
+
+## KV reads and writes
+
+The KV index makes a single-key read an indexed seek plus the selected chunk read: approximately `O(chunk_length)` after the index is loaded. CRC-bearing entries are verified on read, and `ace fsck` also verifies region ordering, length prefixes, bounds, and CRCs.
+
+The writer is append-structured, but the file is **not appended in place**:
+
+- Synchronous operational writes retain old KV chunks, append replacement chunks in the new KV region, copy the current event section, write a new index, and atomically rename a complete temporary file.
+- Async `AcePackedStore.commit()` rebuilds the live KV region, serializes the complete hot event set, writes a complete temporary file, and atomically renames it.
+- `compact()` similarly rewrites the complete container with only live KV chunks.
+
+Consequently, filesystem write cost is proportional to the retained KV bytes plus hot event bytes and the index—not `O(1)`. Atomic rename prevents readers from observing a partially promoted file on supported local filesystems, but it also changes the inode. Writers are serialized through the store write coordinator; bypassing that coordinator is unsupported.
+
+## Columnar events
+
+The physical event layout separates fixed-width columns from JSON payloads and uses about 19 fixed bytes per hot event plus payload bytes. The current JavaScript reader eagerly loads and parses hot-event payloads when opening/scanning a store, so the layout's theoretical timestamp/kind-only scan is not yet exposed by the production API. Do not size current scans as 9 bytes per event.
+
+Events survive `compact()`. `archive/events/*` may contain older contiguous event batches compressed with BGZF and indexed by `archive/events/__index`; `AcePackedStore.archiveColdEvents()` implements this tier, while the normal CLI does not invoke it automatically.
+
+## Authority and projections
+
+The packed store is authoritative for generated operational artifacts, agent instructions, skills, topology, schemas, scheduler state, handoffs, todos, run ledger, and event history. Host-facing files are materialized only through explicit projection allowlists.
+
+Exceptions and boundaries:
+
+- `agent-state/goals/*.json`, plan JSON, and detached goal logs are authoritative loose goal state.
+- Source code and user-authored project files remain ordinary workspace files.
+- `.agents/ACE/ace-state.ace` is legacy fallback state and should not coexist indefinitely with the canonical file.
+- `ACE_STORE_FIRST=0` is a legacy diagnostic escape hatch; it can expose stale disk projections and is not the normal precedence.
+
+## Maintenance and recovery
+
+### Inspect before changing
+
+```bash
+ace fsck --json
+ace ls agent-state
+ace read agent-state/STATUS.md --source
+```
+
+`ace fsck` diagnoses binary structure, CRCs, stale write leases, orphan temporary files, and dual-store conflicts. `--repair` removes only repairable stale/orphan coordination artifacts; it does not reconstruct corrupt application data.
+
+### Compact
+
+```bash
+ace compact --no-sweep  # store rewrite only
+ace compact             # store rewrite plus loose generated-artifact sweep
+```
+
+The default command runs a sweep after compaction: eligible loose generated ACE artifacts are ingested into the store and removed from disk. Use `--no-sweep` when only the container rewrite is intended. Back up the store and active goal directory before maintenance on a critical workspace.
+
+### Repair
+
+`ace repair` re-bakes packaged assets, repairs known store-backed projections, and restores host files. It is not a general corrupt-ACEPACK byte repair tool. Review every warning and rerun `ace fsck`; recover unrecoverable corruption from a known-good backup or a quarantined pre-write copy.
+
+When a write refuses corrupt source bytes, ACE leaves the original in place and writes a timestamped `.corrupt-*` quarantine copy. Do not delete either copy until the cause and recovery source are established.
+
+### Migrate
+
+`ace migrate [store-path]` imports the supported legacy ACE state families: handoffs, todos, run-ledger entries, status events, and Vericify posts. It is not an arbitrary workspace-file importer. Verify the resulting canonical store with `ace fsck`, then archive the legacy store rather than leaving both authorities active.
+
+## Backup contract
+
+For a recoverable snapshot:
+
+1. Stop active ACE writers and detached goal workers.
+2. Copy `agent-state/ace-state.ace`.
+3. Copy `agent-state/goals/` if present.
+4. Record `ace fsck --json` output and the store checksum.
+5. Preserve any `.corrupt-*` quarantine until recovery is verified.
+
+Restoring only the `.ace` file while omitting active loose goal records is not a complete workspace-state restore.