smol-symphony 0.1.0 → 0.3.0

package/PRODUCT.md

@@ -23,7 +23,8 @@ process is logging.
 
 smol-symphony is a small TypeScript orchestrator that takes Markdown issues off a
 local tracker, prepares per-issue workspaces, and runs coding agents
-(Claude Code, Codex, OpenCode) inside isolated smolvm microVMs over ACP. The
+(Claude Code, Codex, OpenCode) inside isolated per-issue Gondolin microVMs over
+ACP. The
 HTTP dashboard exists to do two things well:
 
 1. **Dispatch** — create issues into the tracker without dropping back to the

package/README.md

@@ -1,77 +1,180 @@
 # smol-symphony
 
+> **Disclaimer:** this project is written using AI — the orchestrator
+> dispatches AI coding agents at itself, and the bulk of the code in this
+> repository was authored by those agents under human review.
+
 A small TypeScript orchestrator that reads issues off a local Markdown tracker,
 prepares per-issue workspaces, and runs coding agents (Claude Code, Codex,
-OpenCode) inside isolated [smolvm](https://smolmachines.com/) microVMs over the
-[Agent Client Protocol](https://agentclientprotocol.com).
+OpenCode) inside isolated per-issue [Gondolin](https://github.com/earendil-works/gondolin)
+microVMs over the [Agent Client Protocol](https://agentclientprotocol.com).
 
-The agent signals completion through an injected MCP server (`mark_done`,
-`request_human_steering`); the orchestrator handles state, retry, concurrency,
-and produces either a pull request or a `git format-patch` bundle per issue.
+The agent signals progress through an injected MCP server (`transition`,
+`request_human_steering`, `propose_issue`); the orchestrator handles state,
+retry, concurrency, and opens a pull request per issue when configured for
+a GitHub remote. In local-only mode the per-issue branch is left in the
+workspace for review.
 
 ```
 ┌──────────────────────────────────────────────────────────────────────────┐
 │  symphony (node host)                                                    │
 │                                                                          │
-│   issues/<state>/*.md  ─┐                                                │
-│      WORKFLOW.md       ─┼──▶  orchestrator  ──▶  agent runner            │
-│      WORKFLOW.template.md │   (poll → reconcile → dispatch)              │
-│                          │                          │                    │
-│                          │                          ▼  ACP (JSON-RPC)    │
-│                          │                  ┌──────────────────────────┐ │
-│                          │                  │ smolvm (per-issue VM)    │ │
-│                          │                  │   adapter binary         │ │
-│                          │                  │   workspace mount        │ │
-│                          │                  │   mcp client ←───────────┼─┼─▶ symphony MCP
-│                          │                  └──────────────────────────┘ │     mark_done
-│                          ▼                                               │     request_human_steering
-│   HTTP dashboard (HTMX):  /                                              │
-│     attention · sessions · on disk · new issue · totals                  │
+│    ./issues/<state>/*.md     ──┐                                         │
+│    ./WORKFLOW.yaml             ├──▶  orchestrator  ──▶  agent runner     │
+│    ./WORKFLOW.template.yaml  ──┘     poll · reconcile · dispatch         │
+│                                                          │               │
+│                                                          ▼  ACP/RPC      │
+│                                       ┌───────────────────────────────┐  │
+│                                       │ Gondolin  (per-issue VM)      │  │
+│                                       │   adapter (claude / codex)    │  │
+│                                       │   workspace mount             │  │
+│                                       │   mcp client  ────────────────┼─┐│
+│                                       └───────────────────────────────┘ ││
+│                                                                         ││
+│              symphony MCP server  ◀─────────────────────────────────────┘│
+│      ( transition · request_human_steering · propose_issue )             │
+│                                                                          │
+│    HTTP dashboard (HTMX):  /                                             │
+│      attention · sessions · on disk · new issue · totals                 │
 └──────────────────────────────────────────────────────────────────────────┘
 ```
 
+`SPEC.md` documents the contracts this repo's code references; for the
+original architectural narrative, see
+[openai/symphony/SPEC.md](https://github.com/openai/symphony/blob/main/SPEC.md).
+
 ## Quick start
 
-Prerequisites:
+The golden path from zero to your first dispatched issue. The details behind
+each step live in [Prerequisites in detail](#prerequisites-in-detail) below — you
+should not need them to get here.
 
-- Node.js ≥ 20.
-- A `smolvm` binary on `$PATH` with the server reachable on the configured
-  endpoint (e.g. `smolvm serve start --listen unix:///run/user/$UID/smolvm.sock`).
-- A packed VM image (one-time): `bash scripts/build-vm.sh` produces
-  `.vm/symphony.smolmachine.smolmachine` (~1.1 GB; ships `claude-agent-acp`,
-  `codex-acp`, and `opencode` on the guest `$PATH`).
-- For the default `acp.adapter: claude`: a credentials file at
-  `~/.claude/.credentials.json` on the host (symphony reads and stages it; the
-  host directory is **not** bind-mounted into the VM).
+```bash
+# 1. Build the agent rootfs image once. This ships only `mise` (jdx/mise) — node +
+#    the coding-agent CLIs are mise-installed at dispatch, the launcher is injected —
+#    and tags the result `symphony-agents:latest` — the tag the scaffolded
+#    WORKFLOW.yaml pins, so a first run needs no edit.
+git clone https://github.com/dizk/smol-symphony.git
+cd smol-symphony && npm install && npm run build:image
+
+# 2. In your project, scaffold the minimal WORKFLOW.yaml. With no workflow file
+#    present, symphony offers to write a ~30-line starter (every key required,
+#    no power features), then exits so you can glance at it.
+cd /path/to/your/project
+npx smol-symphony WORKFLOW.yaml
+
+# 3. Verify the whole setup in one pass before the first dispatch. `symphony
+#    doctor` runs every adoption-prerequisite probe — Node version, the
+#    `gondolin.image` pin, each configured adapter's host credential, a
+#    writable `tracker.root`, and a bindable dashboard port — and prints a
+#    PASS/FAIL line per check with a one-line fix for each failure, exiting 0
+#    iff every check passes. Unlike startup (which bails on the first failure),
+#    it runs all probes so you see the whole picture at once.
+npx smol-symphony doctor WORKFLOW.yaml
+
+# 4. Drop one issue into the tracker.
+mkdir -p issues/Todo
+printf -- '---\ntitle: "Say hi"\n---\nAdd a hi() that returns "hi".\n' > issues/Todo/1.md
+
+# 5. Run symphony and watch it dispatch.
+npx smol-symphony WORKFLOW.yaml --port 8787
+# Open http://127.0.0.1:8787/ — issue 1 moves from "on disk" into a running
+# session as symphony boots a per-issue microVM and dispatches the agent.
+```
 
-Run:
+The same first-run scaffold also installs an issue-filing skill at
+`.claude/skills/symphony-issues/SKILL.md` (best-effort — it never blocks the
+workflow scaffold), so a coding agent in the freshly-onboarded repo can file
+well-formed tracker issues and decompose a large task into a `blocked_by` DAG
+from day one.
+
+The scaffolded file ([WORKFLOW.minimal.yaml](./WORKFLOW.minimal.yaml) is the shipped
+copy) runs as-is once `symphony-agents:latest` exists; the only edits a different
+setup needs are `gondolin.image` (if you pinned a build id) and the adapter
+credential for your chosen agent. When you want a power feature it omits — typed
+action DAGs, PR autopilot, per-state adapters, egress allowlists — graduate to
+the annotated [WORKFLOW.template.yaml](./WORKFLOW.template.yaml).
+
+(Or `npm i -g smol-symphony` and then `symphony doctor` / `symphony WORKFLOW.yaml`
+to skip the `npx` fetch — both invoke the `symphony` bin shipped in this
+package.)
+
+### Prerequisites in detail
+
+- **Node.js ≥ 20.**
+- **The agent rootfs image** (step 1). Gondolin is the in-process microVM
+  substrate (`@earendil-works/gondolin`), so there is no separate VM daemon to
+  run. The image is built **once** from `images/agents/` via
+  `npm run build:image`: it starts from `node:24-bookworm-slim`, installs base
+  CLI tooling, and ships only the `mise` (jdx/mise) binary. node + every
+  ACP-capable coding agent (`claude-agent-acp`, `codex-acp`, …) are
+  NO LONGER baked — they come from a mise SYSTEM config symphony stages into the
+  guest at dispatch (`assets/symphony-mise.system.toml`) and installs via
+  `mise install` into a warm-cached data dir; a consuming repo's own `mise.toml`
+  adds project toolchains (rust, go, kotlin+gradle, …) on top. The in-VM stdio
+  launcher is likewise injected at dispatch, not baked. So bumping an agent CLI is
+  a one-line edit to the staged mise config + a restart — no image rebuild + repin.
+  The build prints a content-addressed build id; pin it (or the
+  `symphony-agents:latest` tag) in `WORKFLOW.yaml`'s `gondolin.image`. See
+  [images/agents/README.md](./images/agents/README.md) for the build steps and
+  requirements, and `gondolin.mise` in
+  [WORKFLOW.template.yaml](./WORKFLOW.template.yaml) to tune the data dir or opt out.
+- **An adapter credential.** For the default `acp.adapter: claude`: a
+  credentials file at `~/.claude/.credentials.json` on the host. Symphony reads
+  it only on the host side: the guest holds only a token-shaped placeholder, and
+  the host substitutes the real OAuth access token into the outbound request at
+  Gondolin egress (TLS-MITM). The credential file itself is never staged into
+  the VM. (For `acp.adapter: codex`, see [WORKFLOW.template.yaml](./WORKFLOW.template.yaml).)
+- **Skipping the `npx` fetch.** `npm i -g smol-symphony` installs the `symphony`
+  bin globally so `symphony WORKFLOW.yaml` works without re-fetching; the image
+  build still needs a checkout (it lives in `images/agents/`).
+- **Console output.** With the default log file configured, symphony prints a
+  one-line-per-field startup banner (workflow, tracker root, dashboard URL,
+  log-file path) and routes the structured `key=value` stream to
+  `.symphony/logs/symphony.log` only. `tail -f` that file to follow the detail.
+  Pass `--verbose` (alias `--foreground`) to mirror the structured stream back
+  to the console for interactive debugging. With no log file configured (the
+  `SYMPHONY_LOG_FILE=""` override), the structured stream stays on stderr.
+
+### From a checkout
+
+If you're hacking on symphony itself:
 
 ```bash
+git clone https://github.com/dizk/smol-symphony.git
+cd smol-symphony
 npm install
 npm run build
-npx symphony WORKFLOW.md
+npx symphony WORKFLOW.yaml     # the local bin
 ```
 
-Open the dashboard at `http://127.0.0.1:8787/`. Drop issues into
-`issues/Todo/` from the filesystem or the dashboard's `new issue` form;
-symphony dispatches them on the next poll.
+`npm run dev` (via `tsx watch`) reruns on source edits.
 
 ## Local Markdown tracker
 
 Issues live as `.md` files under `tracker.root`. The parent directory is the
-issue state.
+issue state; the set of valid state directories comes from the `states:` block
+in `WORKFLOW.yaml` (see below) and is auto-mkdir'd on startup.
 
 ```
 issues/
 ├── Todo/
-│   ├── ABC-1.md
-│   └── ABC-2.md
-├── In Progress/
-│   └── ABC-3.md
-└── Done/
-    └── ABC-4.md
+│   ├── 1.md
+│   └── 2.md
+├── Review/
+│   └── 3.md
+├── Done/
+│   └── 4.md
+└── Triage/
+    └── 5.md
 ```
 
+The basename is the issue identifier. When a caller (dashboard form, MCP
+`propose_issue`) omits an explicit identifier, the tracker picks the next free
+positive integer by scanning every state directory under `tracker.root`.
+Operator-supplied identifiers (e.g. `CACHE-7.md`) pass through unchanged and
+coexist with the numeric ones.
+
 Each file has YAML front matter and an optional body:
 
 ```markdown
@@ -79,28 +182,40 @@ Each file has YAML front matter and an optional body:
 title: "Fix the login bug"
 priority: 2
 labels: [bug, auth]
-blocked_by: [ABC-5]
+blocked_by: [5]
 ---
 Long-form description in the body.
 ```
 
 State comparison is case-insensitive. Moving the file between state
 directories is the canonical state transition; the orchestrator does this
-itself in response to `mark_done`. The agent inside the VM does **not** have
-filesystem access to the tracker root: it signals completion through the
-MCP server and the orchestrator does the file move.
-
-## WORKFLOW.md
-
-`WORKFLOW.md` is a YAML front matter block (orchestrator config) plus a
-[Liquid](https://liquidjs.com/)-templated prompt body. The shipped file in
-this repo is the canonical project workflow; see
-[WORKFLOW.template.md](./WORKFLOW.template.md) for the annotated reference
+itself in response to `symphony.transition`. The agent inside the VM does
+**not** have filesystem access to the tracker root: it signals progress
+through the MCP server and the orchestrator does the file move.
+
+## WORKFLOW.yaml
+
+`WORKFLOW.yaml` is a pure YAML config file (the orchestrator config). Each
+active state's prompt lives in its own `prompt_file` — a
+[Liquid](https://liquidjs.com/)-templated file under `prompts/`, wrapped in the
+shared `prompt.preamble_file`/`footer_file`. The shipped file in this repo is
+the canonical project workflow; see
+[WORKFLOW.template.yaml](./WORKFLOW.template.yaml) for the annotated reference
 covering every supported option, its type, default, and example.
 
-Symphony watches the file and re-applies poll interval, concurrency, hooks,
-prompt body, smolvm settings, etc. on change without restart. In-flight runs
-keep the settings they started with.
+The workflow is a **state machine**. A required top-level `states:` block
+declares every state an issue can occupy, its `role` (`active` — dispatched;
+`terminal` — triggers cleanup and handoff; `holding` — sits outside the
+dispatch loop, e.g. `Triage`), and optional per-state `adapter`, `model`,
+`max_turns`, and `allowed_transitions` overrides. A single issue can travel
+through any number of states with distinct adapters and instructions; each
+active state names its own `prompt_file`, so the agent in each state sees only
+that state's instructions plus the shared preamble/footer. The shipped workflow
+uses a two-stage `Todo → Review → Done` flow (Claude implements, Codex reviews).
+
+Symphony watches the file and its prompt files, and re-applies poll interval,
+concurrency, typed actions, prompts, gondolin settings, etc. on change without
+restart. In-flight runs keep the settings they started with.
 
 ## Dashboard
 
@@ -130,37 +245,47 @@ API clients. CSRF-relevant content types (`text/plain`,
 
 Symphony injects an MCP server into each ACP session at
 `http://<host>:<bound-port>/api/v1/issues/<id>/mcp`, gated by a per-dispatch
-bearer token. Two tools:
-
-- **`symphony.mark_done({ title, summary })`** — call once at end of a
-  successful run. `title` is a single-line imperative summary (≤72 chars);
-  `summary` is a one- to three-paragraph narrative. The orchestrator
-  atomically moves the issue file to the terminal state and stops
-  dispatching. The pair lands in
-  `<workspace>/.git/symphony-runtime/mark_done.md` (or
-  `<workspace>/.symphony-runtime/mark_done.md` when the workspace doesn't
-  have its own `.git/`) for the `after_run` hook to consume.
+bearer token. Three tools:
+
+- **`symphony.transition({ to_state, notes? })`** — canonical (and only)
+  exit verb. Moves the issue into another declared state, optionally
+  appending markdown `notes` to the issue body before the move so the next
+  agent (in `to_state`) reads them as part of `issue.description`. A
+  transition into a `role: terminal` state ends the run and triggers
+  workspace cleanup; transitions between active/holding states preserve the
+  workspace so the same `agent/<id>` git branch survives the handoff.
+  Rejected transitions (unknown target, disallowed by
+  `allowed_transitions`) return MCP tool-result errors the agent can read
+  and retry.
 - **`symphony.request_human_steering({ question, context? })`** — call
   when blocked on something only a human can answer. The turn ends
   immediately; the human's reply arrives as the prompt for the next turn.
   Steering-reply turns don't count against `agent.max_turns`.
-
-In smolvm, the VM's `127.0.0.1` transparently reaches the host's
+- **`symphony.propose_issue({ title, description?, labels?, priority? })`** —
+  call when the agent notices work that is out of scope for its current
+  task. The proposal lands in the `Triage/` state directory, which the
+  orchestrator does **not** dispatch; the operator approves (→ first active
+  state) or discards (→ first terminal state, prefers `Cancelled`) from the
+  dashboard. The calling issue's identifier and a timestamp are stamped into
+  the proposal's front-matter as `proposed_by` / `proposed_at` so provenance
+  is visible.
+
+In Gondolin, the VM's `127.0.0.1` transparently reaches the host's
 `127.0.0.1` (verified empirically), so the agent reaches the orchestrator
 without any mount or special host alias.
 
 ## ACP — adapter registry
 
 One ACP client (symphony's `agent/acp.ts`), two shipped adapter profiles.
-Each profile encodes the binary symphony launches and the host credential
-file it stages into the workspace before exec:
+Each profile encodes the binary symphony launches and the credential path
+the adapter reaches for inside the VM:
 
-| Adapter   | Binary             | Host credential file              |
+| Adapter   | Binary             | Credential surface                |
 | --------- | ------------------ | --------------------------------- |
-| `claude`  | `claude-agent-acp` | `~/.claude/.credentials.json`     |
-| `codex`   | `codex-acp`        | `~/.codex/auth.json`              |
+| `claude`  | `claude-agent-acp` | placeholder + host egress swap    |
+| `codex`   | `codex-acp`        | placeholder + host egress swap    |
 
-`WORKFLOW.md`:
+`WORKFLOW.yaml`:
 
 ```yaml
 acp:
@@ -171,44 +296,64 @@ acp:
   stall_timeout_ms: 300000
 ```
 
-Selecting an adapter is enough — symphony auto-derives the launch command
-that stages the credential into the workspace's runtime dir and copies it
-into the adapter's expected guest path before exec. Set `command:` only to
-override (testing a forked adapter, a non-standard binary path); doing so
-opts out of automatic credential staging.
-
-Credentials are **never bind-mounted from the host**. Symphony copies the
-single credential file into a per-workspace location (under `.git/` when
-the workspace has its own clone, else `.symphony-runtime/`) and refuses to
-operate on workspaces inside the credential file's ancestor repo.
-
-## After-run handoff: PR or patch
-
-`WORKFLOW.md`'s `after_run` hook ships in two modes:
-
-- **Pull request mode.** Triggered when `SYMPHONY_REPO=<owner>/<repo>` is
-  exported. The hook pushes the per-issue branch to GitHub and runs
-  `gh pr create --base $SYMPHONY_BASE_BRANCH ...`. Requires `gh auth status`
-  to be clean on the host. The token never enters the VM.
-- **Patch bundle mode** (default). Writes
-  `.symphony/patches/<branch>.patch` via `git format-patch` so you can
-  review and apply with `git am`. No remote required.
-
-The agent's `mark_done.md` provides the PR title/body or commit message; the
-hook reads it from the workspace's runtime dir.
-
-See [AGENTS.md](./AGENTS.md) for the env-var contract and switch-over
-commands.
+Selecting an adapter is enough — symphony auto-derives the launch command.
+For `claude`, a per-VM identity file (organization + account UUIDs only,
+no tokens) is staged into the workspace runtime dir and copied to
+`~/.claude.json` inside the VM; the guest holds only a token-shaped
+placeholder, and the host substitutes the real access token into the
+outbound request at Gondolin egress. For `codex`, no identity file is
+staged (OpenAI ships no third-party fingerprint check); the VM is launched
+with `OPENAI_API_KEY=<placeholder>`, and the host substitutes the real
+OpenAI credential at egress. Set `command:` only to override (testing a
+forked adapter, a non-standard binary path).
+
+Credentials **never enter the VM** for either adapter. For `claude`, the
+host's `~/.claude/.credentials.json` is read only on the host side; the VM
+sees `~/.claude.json` (identity-only) plus the placeholder value in its
+`Authorization` header, which the host swaps for the real token at egress.
+For `codex`, the host's `~/.codex/auth.json` (access token /
+`OPENAI_API_KEY`, **never** the refresh token) is read only host-side; the
+real `OPENAI_API_KEY` is stripped from the forwarded VM boot env, so the VM
+holds only the placeholder.
+
+## After-run handoff: pull request
+
+On transition into the Done state the orchestrator runs that state's typed
+`actions:` block — two records, `push_branch` then `create_pr_if_missing` —
+which push the per-issue branch and open a PR when a GitHub repo is configured:
+set `workspace.github_repo: <owner>/<repo>` in `WORKFLOW.yaml` (or export
+`SYMPHONY_REPO=<owner>/<repo>`, which overrides the file). Each action's
+`if: $repo` predicate short-circuits to a no-op in local-only mode (neither
+set), so the branch is simply left in the workspace until cleanup; pick the
+commits up with `git log agent/<id>` against your local clone. The PR is opened with `gh pr create --base
+$base_branch ...`, which requires `gh auth status` to be clean on the host; the
+token never enters the VM.
+
+The orchestrator stages the action context — `$branch`, `$base_branch`,
+`$pr_title` (already id-prefixed), and `$pr_body_file` (a temp file holding the
+current issue body) — before the block fires, so the PR description carries
+every `symphony.transition` notes block accumulated across the run, the full
+handoff thread from implementer through reviewer to approval. Per-action retry
+and snapshot plumbing replaces the old opaque shell-exit surface: on a
+rate-limit the `create_pr_if_missing` action surfaces "retrying in 60s" on the
+dashboard rather than failing silently.
+
+See `states.Done.actions` in [WORKFLOW.yaml](./WORKFLOW.yaml) for the canonical
+record pair and [WORKFLOW.template.yaml](./WORKFLOW.template.yaml) for the
+typed-action reference.
 
 ## Trust posture
 
-Sandbox isolation comes from running each agent inside a smolvm microVM.
-The VM has no network credentials (only the agent's API key is forwarded
-via `smolvm.forward_env`), no tracker filesystem access (the tracker is
-reached only through the MCP server), and stripped git remotes (set by
-`after_create`).
+Sandbox isolation comes from running each agent inside a Gondolin microVM.
+The VM has no OAuth refresh tokens or long-lived access tokens: for both
+`claude` and `codex`, the guest holds only a token-shaped placeholder and
+the host substitutes the real credential into the outbound request at
+Gondolin egress, so no real Anthropic or OpenAI credential is present in
+the VM. The VM has no tracker filesystem access (the tracker is reached
+only through the MCP server) and stripped git remotes (set by the
+orchestrator's `setupWorkspaceDir`).
 
-Within the ACP session, the orchestrator follows SPEC §10.5's "high-trust"
+Within the ACP session, the orchestrator follows SPEC §6.1's "high-trust"
 posture:
 
 - Command execution and file change approvals: auto-approve.
@@ -220,12 +365,16 @@ posture:
 
 ```bash
 npm run typecheck    # tsc --noEmit
-npm test             # 67 tests across workflow, tracker, prompt, workspace,
-                     # adapters, http, and mcp surfaces
+npm test             # 170 tests across workflow, tracker, prompt, workspace,
+                     # adapters, http, mcp, acp-bridge, orchestrator, run log,
+                     # runner state resolution, and tool-call summary surfaces
 npm run build        # tsc emit to dist/
 ```
 
-An end-to-end smoke run needs a real smolvm + VM image.
+An end-to-end smoke run needs the built agent image (see `images/agents/`).
+
+See [CHANGELOG.md](./CHANGELOG.md) for operator-visible changes between
+releases.
 
 ## License