syncmill 0.2.0__tar.gz → 0.3.0__tar.gz

{syncmill-0.2.0 → syncmill-0.3.0}/CHANGELOG.md

@@ -4,6 +4,142 @@ All notable changes to SyncMill are documented here.
 
 ## Unreleased
 
+## 0.3.0 — 2026-10-31
+
+### Added
+
+- `syncmill start --trial`: a walkthrough of the whole loop that needs no
+  login. Two fake agents propose fixes for a failing test in a throwaway
+  repository; both are verified with `python -m unittest`, one passes and one
+  does not, the passing change is shown as a diff and applied only after you
+  confirm. It runs the real CLI in a temporary folder with its own HOME,
+  state and an allowlisted environment, and removes the folder afterwards
+  unless `--keep`. `syncmill start` alone asks whether to try it or work on
+  your project (#171).
+- `syncmill start` on your project: checks which agents are ready (doctor's
+  check), asks for the task, suggests verification commands read from
+  well-known files when `[gate]` is unset (saved only after you agree), shows
+  agents and timeouts, the most agent calls, the check, the repository with
+  any uncommitted changes, the memory location and an "unknown" cost before
+  running, then offers the diff and asks before applying. `--dry-run` shows
+  that screen only; without a terminal it needs `--task` and `--yes`. To
+  compare, it runs one Room round over two agents, lists the candidates with
+  their verification result, and walks through the diff, the selection and
+  the apply; Rooms are turned on in `syncmill.toml` only after you agree,
+  otherwise it runs one agent. `start` runs from the repository root (#171).
+- Reviewable workflow plans: `syncmill workflow validate`, `create`,
+  `export`, `revise` and `lineage`. A plan can now be a schema-v1 JSON
+  document instead of planner output. `validate` checks it against the
+  project's agents and limits and writes nothing; `create` stores it without
+  calling a provider; `export` writes any workflow's plan in a canonical form
+  that `create` and `revise` read back to the same digest; `revise` stores an
+  edited plan as a new workflow in the original's lineage with the next
+  version number, leaving the original untouched. Documents are data only:
+  strict UTF-8 JSON up to 8 MiB, with duplicate keys, `NaN` and any
+  `schema_version` but `1` refused. A document plan's call budget counts node
+  attempts only, since no planning call was made (#220).
+- `[gate] env_allowlist` closes a quality gate's environment to basic OS
+  variables plus exact names, for runs, rooms and `apply --verify` alike.
+  Unset, gates keep inheriting the agents' environment (#223).
+- `[board] transitions_retain_seconds` trims the work-board's transitions
+  audit automatically after each `board run`, folding its stats first so
+  `board stats` is unchanged. `0`, the default, keeps it off; `board compact`
+  remains the manual path (#122).
+- `syncmill doctor` prints one line each for the quality gate, run reports
+  and the work-board: on with its setting, or off with the setting that
+  turns it on. Informational only; an off feature never fails `doctor`,
+  `--strict` included. It opens no store.
+
+### Changed
+
+- Later phases of `pipeline`, `council` and `decompose` now receive every
+  earlier result of the same run directly in their prompt, instead of
+  whatever a relevance search over `shared` returned. Pipeline stage N gets
+  stages 1..N-1 in order; each council critique round and the synthesis get
+  every proposal plus the latest round's critiques; decompose synthesis gets
+  every subtask in plan order, including ones that did not complete, marked
+  as gaps. Long results are shortened head and tail within a 32 KiB budget,
+  and nothing is dropped. The handoff survives a failed memory backend; the
+  `shared` writes and tag-filtered recall are unchanged (#125).
+
+- `syncmill apply --verify` and `syncmill room apply --verify` refuse when no
+  `[gate] commands` are configured, instead of applying the patch unverified
+  without a word. They refuse before the source, the run or room state, or the
+  run artifacts change, foreground and `--detach` alike; leave `--verify` off
+  to apply without a check. A round that ran a gate, applied after the gate
+  commands were removed, is refused the same way. Scripts that passed
+  `--verify` with no gate configured now fail (#235).
+- A workflow's final-node quality gate no longer sees the full parent
+  environment. It runs code the final agent wrote, so it now gets basic OS
+  variables, that agent's `workflow_env_allowlist` and `[gate] env_allowlist`:
+  by default no more than the agent got, and no provider credential unless
+  one of those lists names it. A workflow gate that relied on any other
+  inherited variable must list it (#223).
+- A workflow created with no team keeps running without one. `workflow run`,
+  `resume` and `revise` applied whatever `default_team` was configured at
+  the time, so a later default team could refuse the plan's agents or run it
+  under a different lineup. An explicit `--team` still overrides. A workflow
+  an older release stored with no team follows the current `default_team` as
+  before, since it cannot tell "no team" from "not recorded". A failed read
+  of the workflow's team is now an error rather than a silent fall back to
+  the default (#224).
+
+### Fixed
+
+- A room round no longer logs "Retaining recovery workspace" for the
+  workspace it keeps for later rounds; the notice is now reserved for a
+  workspace kept after a failure.
+- `syncmill room apply --verify` says when the round's recorded gate settings
+  do not show it ran the current `[gate]`: a `[syncmill] verify:` line names
+  the differing commands, timeout or environment, or says the round recorded
+  no settings, no environment (provenance v1) or no gate. Verify still runs
+  the current `[gate]`, as every `--verify` does; the line is printed before
+  any verify error, also to a detached apply's job log, and never blocks the
+  apply (#233).
+- `syncmill apply --verify` of a workflow's final artifact runs in the same
+  closed environment as that workflow's final-node gate (basic OS variables,
+  the final agent's `workflow_env_allowlist` and `[gate] env_allowlist`)
+  instead of inheriting the full environment when `[gate] env_allowlist` is
+  unset. The artifact records its final agent, so this holds after
+  `workflow prune`; an artifact from an earlier release gets only
+  `[gate] env_allowlist`, and verify says so. The environment is decided from
+  the artifact as the locked apply reads it (#230).
+- Room rounds record the gate's environment allowlist with the rest of their
+  gate settings (round provenance v2), and `room compare`, `room export` and
+  the TUI show it: inherited, common variables only, or common variables plus
+  the listed names. Rounds recorded earlier read "environment not recorded",
+  and `--all-rounds` no longer calls them `same settings` as a round whose
+  environment is recorded; equal older rounds group as `same commands and
+  timeout`. The comparison JSON gains `env_allowlist_status` and
+  `env_allowlist` in `gate_settings` and `cross_round_settings` (still
+  `schema_version` 1). A release before this one reports a round started by
+  this release as an unreadable provenance record (#229).
+- Running the TUI job manager directly (`python -m syncmill.tui_jobs`) on an
+  unsupported platform is refused before anything is written, as every CLI
+  command already was; the worker used to leave a `state.json` behind (#137).
+- Attempt spans of the concurrent strategies sit where each execution ran.
+  Compete attempts, council propose, critique and synthesis, and decompose
+  subtasks and synthesis were all anchored at the end of their phase, so an
+  attempt that finished in one second beside a sixty-second sibling appeared
+  at second 59. Each span now starts at the instant its agent started the
+  measured run, recorded per invocation as workflow nodes already were; a
+  runner that records no start keeps the previous placement (#160).
+- `syncmill workflow plan` reports a planner reply carrying an unpaired
+  surrogate escape (in a node id, a dependency, any value or the prose around
+  the plan) as an invalid workflow instead of a `UnicodeEncodeError`
+  traceback, and no longer leaves a half-written workflow directory (#222).
+- Plan documents nested deeper than 64 arrays and objects are refused as
+  "nested too deeply" on every Python version and platform. On Python 3.14 a
+  deeply nested `schema_version` used to escape as `RecursionError`, and a
+  deep but truncated document was reported as invalid JSON (#225).
+
+### Documentation
+
+- The README and design notes no longer say snapshot capture leaves the source
+  index untouched. The source check refreshes the index's file-stat cache, as
+  `git status` does; staged contents, branches and objects are still not
+  changed (#183).
+
 ## 0.2.0 — 2026-10-21
 
 ### Added

{syncmill-0.2.0 → syncmill-0.3.0}/CLAUDE.md

@@ -10,6 +10,8 @@ syncmill orchestrates multiple CLI coding agents (Claude Code, Codex CLI, Kimi C
 
 ```bash
 uv sync                              # install deps + create .venv (uses uv, Python 3.12)
+uv run syncmill start --trial         # offline walkthrough (trial.py, #171): fake codex/kimi in a temp sandbox, room round → compare → diff → apply --verify; --keep keeps the folder; bare `start` menu (no TTY → explains instead of waiting)
+uv run syncmill start --task "<task>" [-a AGENT] [--dry-run | --yes]   # project path: agent_readiness → confirm screen (start.py render_start_plan; gate discovery reads files only, saved via teamfile.set_gate_commands after a yes) → ctx.invoke(run) → diff/apply offer; run leaves its id in ctx.meta[_START_RUN_META]; --compare / 2×-a = one Room round via RoomCoordinator → start.room_candidates over comparison_to_dict → room diff/select/apply (rooms off → ask, teamfile.set_rooms_enabled, else single); repo root only
 uv run syncmill init                 # initialize a project
 uv run syncmill doctor               # validate repo, agents, and stores
 uv run syncmill run --dry-run "<prompt>"   # print bounded RunPlan; no agent starts
@@ -25,7 +27,7 @@ uv run syncmill room compare <room-id> [--round N | --all-rounds] [--json]   # r
 uv run syncmill room export <room-id> [--round N | --all-rounds] [--format markdown|json] [--output FILE]   # body-free review summary of the same read model: json == `room compare --json`; markdown fences all stored free text (render_review_markdown); --output is O_EXCL 0600, no overwrite
 uv run syncmill room export <room-id> --include patch:<cand> [--include output:|gate-output:<cand>|prompt:<round> ...] (--preview | --confirm TOKEN)   # opt-in raw content (room_raw.py, #189): preview prints sizes/digests/first lines + a token over every item descriptor; confirm re-collects and writes only on an equal token; reads bound to the comparison's room/round/digests; gate-output accepted only when room_compare._gate_verdict accepts the record; markdown = escaped display rendering, json adds `raw_content` (base64, ensure_ascii); never redacted
 # room rounds also record each member's conversation (session cold/resume/fork, checkpoint candidate ids + omitted, memory delivery) in rounds.conversation_json, written once under the lease before the fan-out (rooms.round_conversation_status binds it to SQLite rows); compare shows per-candidate facts and round counts only
-# room rounds record their effective starting tree + gate settings in rounds.provenance_json (rooms.round_provenance_status, writer and readers share it; NULL = legacy); compare/export/TUI show them as starting tree / gate settings beside the report-derived gate profile, whose JSON fields are unchanged
+# room rounds record their effective starting tree + gate settings (v2 adds the gate's env_allowlist: null = inherited, [] distinct) in rounds.provenance_json (rooms.round_provenance_status, writer and readers share it, readers accept v1+v2; NULL = legacy); compare/export/TUI show them as starting tree / gate settings beside the report-derived gate profile, whose JSON fields are unchanged; a v1 round's environment reads not recorded and never groups as same settings; `room apply --verify` still runs the current `[gate]` and prints a `verify:` notice (`rooms.verify_settings_notice`) unless the round's v2 record equals it
 uv run syncmill room select <room-id> <cand> [--expected-revision N --expected-digest SHA256 --rationale TEXT]   # verifies the artifact, records a bound selection (rooms.selection_json); apply refuses an inconsistent record
 uv run syncmill conversations list
 uv run syncmill conversations show <run-id>
@@ -92,7 +94,10 @@ Request flow (`CLI → Supervisor.run`, in `supervisor.py`):
 
 1. Resolve a strict Git root. `capture_repository_snapshot` blocks unsafe states and
    captures HEAD + dirty tracked + untracked (ignored excluded) in a private control
-   clone without changing the source index/branch/object database.
+   clone without changing the source branch, object database or staged index
+   contents. `repository_fingerprint` (snapshot capture, room rounds, every apply,
+   `room compare`) does refresh the source index's stat cache, as `git status` does
+   (#183).
 2. `Supervisor` builds an `AgentRegistry` and project-scoped memory for initialized projects.
 3. Per run: build a snapshot-backed `WorktreePool`; every strategy worktree starts
    at the same synthetic snapshot commit. A pending continuation seeds the parent patch.
@@ -134,12 +139,13 @@ Pluggable seams: memory backends, agents, strategies, quality gates and the circ
 - Implement the `Strategy` Protocol and register it in `_STRATEGIES` (`strategies/__init__.py`); `_make_strategy` hard-fails on an unknown name — never default silently.
 - `strategies/base.py` holds the Protocol and every cross-strategy helper (`_enrich_prompt`, `_gate_accepts`, `_breaker_*`, failure builders, `_select_winner`, `_collect_files_changed` → `worktree.capture_changes`); diff collection stays below the strategy modules to keep imports acyclic. One module per strategy.
 - Each strategy gets a `WorktreePool` and owns its topology (Route/Compete/Council: one worktree per agent, Council reusing it across phases; Pipeline: one shared; Decompose: planner + each subtask + each synthesizer) and sets `files_changed`/`patch` via `_collect_files_changed` (mutates the result for a COMPLETED git worktree; the underlying `capture_changes` returns `CapturedChanges` or raises `PatchCaptureError`).
-- Route = priority fallback; Pipeline = sequential stages handing off via `shared`, fail-fast; Compete = concurrent, converge via `_select_winner`; Council = propose → critique ×`council.rounds` → synthesize, deliberating only via `shared`; Decompose = `runnable[0]` plans via sync `execute` (a bad plan degrades to one whole-task subtask), subtasks round-robin concurrently via `aexecute_each`, every runnable agent synthesizes in a fresh worktree; capped by `decompose.max_subtasks`.
+- Route = priority fallback; Pipeline = sequential stages, each given every earlier stage's result, fail-fast; Compete = concurrent, converge via `_select_winner`; Council = propose → critique ×`council.rounds` → synthesize, each critique round and synthesis given every proposal plus the latest round's critiques; Decompose = `runnable[0]` plans via sync `execute` (a bad plan degrades to one whole-task subtask), subtasks round-robin concurrently via `aexecute_each`, every runnable agent synthesizes in a fresh worktree from every subtask's result; capped by `decompose.max_subtasks`.
+- **Intra-run handoff is total (#125).** A later phase's prompt carries the earlier results the strategy already holds — `_handoff_item` per result, rendered by `_merge_handoff` and placed by `_enrich_prompt(prompt, context, handoff)` between the recalled context and the task. Every item keeps its label (a result that did not complete is a visible gap); only bodies are shortened, water-filled within `_HANDOFF_MAX_BYTES`, and the budget shrinks so the composed prompt stays under `_PROMPT_MAX_BYTES` (argv). Never searched back from memory, so it survives a dead backend; the `shared` writes and tag-filtered recall stay for cross-run value and human notes. Results are never merged as trees.
 - `_select_winner` under `config.selection.mode`: `verified` (default; first gate-passing COMPLETED in priority order), `priority` (checks only the first completed), `judge` (ranks gate-passing candidates, falls back to verified on failure). Never by wall-clock.
 - No strategy applies the winner's diff; capability re-ordering happens once in the Supervisor, never per strategy. Tests stub `syncmill.strategies.base.capture_changes`; capture/index tests use real Git repos.
 
 **Quality gates** (`gates.py`)
-- `QualityGate.evaluate(worktree) -> GateReport` runs after the agent finishes, on its worktree — never a pre-run probe. `CommandGate` runs each command via `shlex.split` without a shell (add entries, not `&&`), passes iff all exit 0, in the agents' isolated environment (no ambient `GIT_DIR`/`GIT_CONFIG_PARAMETERS`/hooks). `build_gate` → `None` when no commands are configured (the default).
+- `QualityGate.evaluate(worktree) -> GateReport` runs after the agent finishes, on its worktree — never a pre-run probe. `CommandGate` runs each command via `shlex.split` without a shell (add entries, not `&&`), passes iff all exit 0, in the agents' isolated environment (no ambient `GIT_DIR`/`GIT_CONFIG_PARAMETERS`/hooks). By default a gate is no wider than the agent it judges: a set `gate.env_allowlist` narrows it (and `apply --verify`) to the workflow common variables + those exact names, applied after isolation; a workflow gate is always narrowed (unset = `[]`, plus the final agent's `workflow_env_allowlist`), and so is `apply --verify` of its artifact (final agent from the manifest's `plan.final_agent`, decided under the apply lock). `build_gate` → `None` when no commands are configured (the default). `--verify` with no commands is refused (`artifacts.VERIFY_NEEDS_GATE`) before any apply-state change: in the CLI (and before a detached worker launches), in `RoomCoordinator.apply` before `begin_apply`, and in `ArtifactStore._apply_locked` as the backstop (`verify_commands=[]` refuses; `None` = not requested).
 - Strategy layer only; agents never see it. `_gate_accepts`: fail → FAILED with `result.gate`. Route falls back; Pipeline gates only the final cumulative tree; Compete downranks in `_select_winner`; Council and Decompose gate only synthesis. A gate-rejected result is never shared.
 
 **Circuit breaker** (`breaker.py`)
@@ -192,7 +198,7 @@ Rules that must hold:
   | Council first critique | `council-proposal` |
   | Council later critiques + synthesis | `council-proposal,council-critique` |
 
-- These are relevance searches, not delivery guarantees (prior runs compete; row/byte limits apply).
+- These are relevance searches, not delivery guarantees (prior runs compete; row/byte limits apply). The current run's own earlier results do not depend on them: they arrive through the total handoff above.
 - Collaboration is knowledge, not file merges: interactive edits land in the real tree, agents in isolated worktrees. The human is not a compete candidate (future work).
 
 ### Read-only MCP gateway (`gateway.py`)
@@ -246,6 +252,7 @@ Rules that must hold:
 - **Atomic claim + lease fencing.** Every claim path (`claim(id)` and `next_runnable`) is a `BEGIN IMMEDIATE` CAS `planned→running` (autocommit `isolation_level=None`, WAL, `busy_timeout`). Automatic selection (`next_runnable`) additionally requires: not `human_required`, every `depends_on` item `done` (a missing dependency is unsatisfied), FIFO `created_at, rowid`, optional exact `goal`. An explicit `claim(id)` checks only id + `planned` — it is the operator override. The lease is a fencing token: `apply_update(run_id=)` and `_block_on_crash` CAS on `state='running' AND lease_run_id=?` and drop a 0-row match. A same-state annotate keeps the lease; a no-`--state` annotate resolves its target inside the transaction. Every transactional method rolls back on **any** exception. `recover_stale_running()` runs at the top of every `run_item`.
 - **Memory boundary.** Items run with `SyncmillMemory.scoped(run_id)`, collapsing shared and per-agent namespaces to `board-run:{run_id}`; scoped reads also union `shared` (D4), writes never leave the scope. Afterwards one best-effort sanitized summary goes to global `shared` via `share_board_result` (output with the board block stripped, never the enriched prompt) — **only if the fenced `apply_update` applied**; a run that lost its lease publishes nothing. Decisions (`active_decisions(goal=item.goal)`: active global decisions plus that exact goal's) and the handoff baton are injected deterministically, never via top-k recall. Scratch namespaces are GC'd via `collectible_board_run_ids`, which reads ownership and liveness in one SQLite snapshot (a claim commits its lease and ledger row atomically) and excludes active leases and reclaimed runs still within one lease timeout of grace; the sweep runs under a cross-process lock, tombstones only acknowledged deletions, and never touches `shared` or `agent-runtime:*`.
 - **Crash vs failure.** A crash (`BaseException`) moves the item to `blocked` + `human_required` and re-raises; a FAILED result is not a crash (`apply_update` records `blocked`).
+- **Audit compaction.** `board compact` and the opt-in `board.transitions_retain_seconds` (default `0` = off) share `compact_transitions`: fold stats aggregates, then delete, in one transaction, so `stats()` is unchanged. The automatic call runs only at `run_item`'s housekeeping points (after the outcome, or on an empty claim; never on the claim or crash path) and degrades to a warning.
 - **Phase 2.** Decisions live in their own table; supersede is a same-scope CAS, and injection stays in `board.py::render_item_prompt` (a `strategies/base.py` helper would create a `board → strategies` cycle); a failed decisions read degrades. `Board.stats` derives everything from rows + the audit table. The gateway's `board_*` tools are read-only and always registered (exact-set guard = 6 tools); a bad `state` is a client argument error, not "unavailable"; uuid validation lives in `Board.get`.
 - `Evidence`/`WorkItem`/`WorkState`/`Decision`/`BoardStats` live in `board.py`, not `models.py`.
 - **Tests:** `tests/test_board.py` drives the real `Board` on `tmp_path` with an injected `now`; `conftest.py` provides `FakeBoard` and `FakeMemory.scoped`/`share_board_result` (scoped shares never reach the parent's `.shared`).
@@ -325,7 +332,7 @@ OTLP-JSON trace export (`trace.otlp.json` beside the envelope) that tracegraph i
 Rules that must hold:
 
 - **Disabled is the default** (`build_tracer` → `None`, no filesystem touch). `dir=""` means the report `runs_dir` (`fallback_dir`). `otlp-http` keeps the durable file and additionally streams body-free spans to Phoenix; stream failures degrade to warnings/receipts. `include_output` is warned about and ignored.
-- **Retrospective recorder.** Strategies append completed `SpanRecord`s after each attempt/fan-out returns (never live spans inside the agent layer); builders live in `strategies/base.py` (`_attempt_span`, `_gate_span`, `_select_span`) so semantics cannot drift between strategies.
+- **Retrospective recorder.** Strategies append completed `SpanRecord`s after each attempt/fan-out returns (never live spans inside the agent layer); builders live in `strategies/base.py` (`_attempt_span`, `_gate_span`, `_select_span`) so semantics cannot drift between strategies. A concurrent phase passes `started_at=` to `aexecute_isolated`/`aexecute_each` and hands each attempt its recorded `start` (span = `start` → `start + duration`); without one (sequential steps, runners that bypass `CLIAgent.aexecute`) the span falls back to `end − duration`.
 - **Span `name` is a stable structural label** (agent ids, phases, indices — never uuids or timestamps); tracegraph diffs on it. Causality rides `graph.node.parent_id` and span links, never `parentSpanId` (flat under the run root) or timestamps.
 - **Attributes:** builders put bare-suffix keys in `attrs`; `_span_attributes` prefixes `syncmill.`, drops anything outside `_ALLOWLIST`, and injects `run_id`/`schema_version` (toolgraph evidence adds its own `toolgraph.*` keys). Error text is bounded (`timeout` | `exit_code=N` | `gate: <summary>`), never raw output.
 - **Attempt spans carry the RAW pre-gate status** (snapshot before `_gate_accepts`/`_select_winner` mutate results; council/decompose also snapshot duration). Gate spans exist only for candidates actually gated; the `select` span parents on the winning attempt and links the other raw-COMPLETED candidates. An early aggregate failure leaves no select span.

{syncmill-0.2.0 → syncmill-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: syncmill
-Version: 0.2.0
+Version: 0.3.0
 Summary: Multi-agent orchestration with shared memory — grind tasks through competing and collaborating AI agents
 Project-URL: Homepage, https://github.com/memtomem/syncmill
 Project-URL: Repository, https://github.com/memtomem/syncmill
@@ -76,6 +76,7 @@ to orchestrate, installed and authenticated.
 ```bash
 uv tool install syncmill      # or: pip install syncmill
 syncmill --version
+syncmill start --trial        # a five-second walkthrough with fake agents, no login
 ```
 
 Optional backends are extras: `syncmill[mem0]`, `syncmill[langgraph]`,
@@ -155,6 +156,9 @@ uv sync --project "$SYNCMILL_SRC" --extra phoenix
 # Run these examples from the target repository. --project selects the
 # Syncmill source checkout without changing the target repository.
 
+# Try the whole loop offline first: two fake agents, verify, compare, diff, apply
+uv run --project "$SYNCMILL_SRC" syncmill start --trial
+
 # One-time project setup and readiness check
 uv run --project "$SYNCMILL_SRC" syncmill init
 uv run --project "$SYNCMILL_SRC" syncmill doctor              # one ready provider is enough
@@ -202,6 +206,10 @@ uv run --project "$SYNCMILL_SRC" syncmill workflow run <workflow-id>
 # Or detach the same durable execution and follow it from another shell / Runs tab.
 uv run --project "$SYNCMILL_SRC" syncmill workflow run <workflow-id> --detach
 uv run --project "$SYNCMILL_SRC" syncmill workflow status <workflow-id> --watch
+# Or start from a JSON plan you wrote or exported; revise keeps every version.
+uv run --project "$SYNCMILL_SRC" syncmill workflow export <workflow-id> --output plan.json
+uv run --project "$SYNCMILL_SRC" syncmill workflow revise <workflow-id> --file plan.json
+uv run --project "$SYNCMILL_SRC" syncmill workflow lineage <workflow-id>
 
 # With Toolgraph control-plan preflight enabled, MillFlow freezes a body-free
 # DAG at planning and checks those exact bytes before every run/resume. Each
@@ -451,6 +459,8 @@ max_changed_files = 512
 # Quality gates (#10): each entry is one command, all must exit 0.
 [gate]
 commands = ["pytest -q", "ruff check ."]
+# Optional: close the gate's environment to basic OS variables + these names.
+# env_allowlist = ["UV_INDEX_URL"]
 
 # Circuit breaker (#8): off by default.
 [breaker]
@@ -584,8 +594,9 @@ User ─▶ Supervisor ─┬─▶ (read) memory backend  → inject context in
 - **Workspaces** live under `~/.syncmill/workspaces/<run-id>/`. syncmill refuses
   non-Git repositories, unborn repositories, unresolved operations,
   submodules (including clean/initialized ones), and sparse checkouts instead of silently running in an empty temp
-  directory. Source branches, the source index, and the source object database
-  are not changed while the snapshot is made. A killed run's marked workspace is
+  directory. Source branches, the staged contents of the source index, and the
+  source object database are not changed while the snapshot is made; the source
+  check refreshes the index's file-stat cache the way `git status` does. A killed run's marked workspace is
   reclaimed by the next normal run once its owner process is gone.
   Recovery-marked workspaces are preserved; PID reuse can delay dead-owner
   cleanup until the one-week safety backstop.

{syncmill-0.2.0 → syncmill-0.3.0}/README.md

@@ -39,6 +39,7 @@ to orchestrate, installed and authenticated.
 ```bash
 uv tool install syncmill      # or: pip install syncmill
 syncmill --version
+syncmill start --trial        # a five-second walkthrough with fake agents, no login
 ```
 
 Optional backends are extras: `syncmill[mem0]`, `syncmill[langgraph]`,
@@ -118,6 +119,9 @@ uv sync --project "$SYNCMILL_SRC" --extra phoenix
 # Run these examples from the target repository. --project selects the
 # Syncmill source checkout without changing the target repository.
 
+# Try the whole loop offline first: two fake agents, verify, compare, diff, apply
+uv run --project "$SYNCMILL_SRC" syncmill start --trial
+
 # One-time project setup and readiness check
 uv run --project "$SYNCMILL_SRC" syncmill init
 uv run --project "$SYNCMILL_SRC" syncmill doctor              # one ready provider is enough
@@ -165,6 +169,10 @@ uv run --project "$SYNCMILL_SRC" syncmill workflow run <workflow-id>
 # Or detach the same durable execution and follow it from another shell / Runs tab.
 uv run --project "$SYNCMILL_SRC" syncmill workflow run <workflow-id> --detach
 uv run --project "$SYNCMILL_SRC" syncmill workflow status <workflow-id> --watch
+# Or start from a JSON plan you wrote or exported; revise keeps every version.
+uv run --project "$SYNCMILL_SRC" syncmill workflow export <workflow-id> --output plan.json
+uv run --project "$SYNCMILL_SRC" syncmill workflow revise <workflow-id> --file plan.json
+uv run --project "$SYNCMILL_SRC" syncmill workflow lineage <workflow-id>
 
 # With Toolgraph control-plan preflight enabled, MillFlow freezes a body-free
 # DAG at planning and checks those exact bytes before every run/resume. Each
@@ -414,6 +422,8 @@ max_changed_files = 512
 # Quality gates (#10): each entry is one command, all must exit 0.
 [gate]
 commands = ["pytest -q", "ruff check ."]
+# Optional: close the gate's environment to basic OS variables + these names.
+# env_allowlist = ["UV_INDEX_URL"]
 
 # Circuit breaker (#8): off by default.
 [breaker]
@@ -547,8 +557,9 @@ User ─▶ Supervisor ─┬─▶ (read) memory backend  → inject context in
 - **Workspaces** live under `~/.syncmill/workspaces/<run-id>/`. syncmill refuses
   non-Git repositories, unborn repositories, unresolved operations,
   submodules (including clean/initialized ones), and sparse checkouts instead of silently running in an empty temp
-  directory. Source branches, the source index, and the source object database
-  are not changed while the snapshot is made. A killed run's marked workspace is
+  directory. Source branches, the staged contents of the source index, and the
+  source object database are not changed while the snapshot is made; the source
+  check refreshes the index's file-stat cache the way `git status` does. A killed run's marked workspace is
   reclaimed by the next normal run once its owner process is gone.
   Recovery-marked workspaces are preserved; PID reuse can delay dead-owner
   cleanup until the one-week safety backstop.

{syncmill-0.2.0 → syncmill-0.3.0}/docs/architecture/board.md

@@ -14,6 +14,8 @@ The `board` command group turns syncmill from a single-run, stateless orchestrat
 
 `run_item` flow: `_require_board` (lazy build; clear error if disabled) → `recover_stale_running()` → atomically claim (`claim(id)` or `next_runnable(run_id)`; `None` stops a `--loop` caller) → `Task(prompt=render_item_prompt(item))` → `run(memory=scoped)` → `parse_board_update(result.output)` → `board.apply_update(...)` [advisory] → `share_board_result(...)` to global shared. Wrapped in `try/except BaseException` → a genuine CRASH transitions the item `→ blocked` (`_block_on_crash`, best-effort, sets `human_required`) and re-raises; a FAILED result is **not** a crash (`apply_update` records it as `blocked`). **`--strategy` is NOT a `run_item` param** — the board CLI mutates `config.default_strategy` BEFORE constructing the `Supervisor` (matching how `run` selects a strategy in `__init__`). The CLI `board` group mirrors `session`: `_require_board` (mirror `_require_tmux`) hard-errors when disabled; `add TASK [--goal --depends-on(repeatable) --human-required]` (no `--owner` — `owner_agent` is RECORD-ONLY in v1, stamped from `result.agent_id`), `list [--state --goal]`, `show ID`, `run [--id --loop --strategy --agent --team --goal]` (Phase 4: `--goal <g>` scopes the atomic claim to that goal via `next_runnable(goal=)` — mutually exclusive with `--id` — and auto-applies the team declaring `g`; `--team` overlays a profile pre-`Supervisor` and beats the goal mapping), `update ID [--state --evidence --sign-off --note --clear-human-required]` (the human path; `done` REQUIRES `--evidence`/`--sign-off`). **Hermetic tests:** `tests/test_board.py` drives the REAL `Board` against `tmp_path/"board.db"` with an injected `now` (like `test_breaker.py`) — CRUD/audit, atomic dedup on BOTH paths (two `Board`s/one db → exactly one claims), dep/skip, stale recovery, done-gate, gate→Evidence, advisory-done-clamps, corrupt+unopenable db→`BoardError`, parse/strip tables; `conftest.py` adds a duck-typed `FakeBoard` + `FakeMemory.scoped`/`share_board_result` (the scoped child's shares stay off the parent's global `.shared` — the leak guard); `test_supervisor.py` covers `run_item` (board-scoped-memory leak guard, crash→blocked, lazy build never touched by a plain `run`, `close()` closes the board); `test_cli.py` exercises the `board` subcommands via `CliRunner` with a tmp db. **`Evidence`/`WorkItem`/`WorkState`/`Decision`/`BoardStats` live in `board.py`, not `models.py`** (they never flow through a strategy — the `breaker._AgentState` rule).
 
+**Audit compaction (#118, #122).** The `transitions` audit is append-only. `Board.compact_transitions(older_than_seconds=)` folds what `stats()` derives from old rows (per-item `planned→running` claim counts into `transition_rollups`, stale reclaims into `transition_counters`) and deletes them in ONE `BEGIN IMMEDIATE` transaction, after `_backfill_run_ledger` (which scans those same claim rows); `stats()` reads live rows plus the rollups, so its numbers are identical across a compaction. `board compact` is the explicit path. `BoardConfig.transitions_retain_seconds` (default `0` = off — not the manual command's "trim everything") opts into the same call from `Supervisor._compact_board_transitions` at `run_item`'s housekeeping points (after the outcome is recorded, and on an empty claim — never on the claim or crash path), best-effort: any failure rolls back in the store and degrades to a warning, never blocking dispatch. Concurrent `--loop` workers serialize on the write lock, and a second compaction folds only the rows the first left, so nothing is double-counted. `idx_transitions_at` keeps the age-bounded delete an index search.
+
 **Phase 2 — Collaboration depth + read API** (4 deliverables, one PR; the design grounded against the merged Phase-1 code, which shifted two plan details — see below):
 - **D1 Decisions (never-re-litigate).** A separate `decisions` table in the same db (orthogonal to work items — no state machine, no lease): `Board.add_decision(text, *, scope="", supersedes=None)` (atomically retires the superseded one **of the same scope** — a missing/already-retired/cross-scope target raises `BoardError` (same-scope CAS: `WHERE id=? AND superseded_by IS NULL AND scope=?`, so superseding can never silently retire a different goal's decision), and the new decision is NOT inserted; empty text rejected) + `Board.active_decisions(goal=None)` (non-superseded, oldest-first; `goal=None` → all, a goal string → global `scope=""` + that-goal-scoped). `run_item` fetches `active_decisions(goal=item.goal)` and injects the texts via the EXISTING `render_item_prompt(item, active_decisions=)` seam — **deterministic, in full, never top-k**. **Grounded divergence from the plan:** the plan put a `_enrich_with_decisions` helper in `strategies/base.py`, but Phase 1 already shipped the injection seam *inside* `board.py::render_item_prompt`, and a `base.py` helper would force a `board → strategies` import that violates the acyclicity invariant — so the seam stays in `board.py` (no `base.py` helper). The decisions read is **best-effort in `run_item`** (a `BoardError` degrades to no-decisions and logs, never blocks the item — decision injection is advisory prompt enrichment, not load-bearing dispatch). CLI: `board decide "<text>" [--scope --supersedes]`, `board decisions [--goal]`.
 - **D2 `board stats`.** `Board.stats(*, now=None, stall_seconds=None) → BoardStats` computed entirely from existing rows + the append-only audit (no new write-path instrumentation): `false_done` (DONE with no passing `Evidence` — an integrity alarm, impossible via the gate), `redispatched` (items with >1 `planned→running` audit row = work a stale reclaim forced a redo — the *materialised* duplicate-work signal; a purely concurrent claim the CAS averts leaves no trace by design and need not be counted), `reclaimed_stale` (count of `recover_stale_running` reclaims, matched by the shared `_STALE_RECLAIM_NOTE`), and aged `blocked`/`human_required`/`dropped_handoffs` (untouched past `stall_seconds`). `BoardConfig.stall_seconds` (≥1, default 3600) is the aging window (`SYNCMILL_BOARD__STALL_SECONDS`); `build_board` threads it into `Board`. CLI: `board stats`; renderer `render_board_stats`.

{syncmill-0.2.0 → syncmill-0.3.0}/docs/architecture/extension-points.md

@@ -13,8 +13,8 @@
 
 - **Memory backends** (`memory_backend.py`): `MemoryBackend` is the async, raise-freely storage contract; `SyncmillMemory` owns every best-effort/degradation path. `MemtomemBackend` remains the default and the only runtime module allowed to reach memtomem private internals; the migration-only exception is documented below. Optional adapters are `Mem0Backend` (`syncmill[mem0]`), restricted to Qdrant for exact count/delete semantics, and `LangGraphBackend` (`syncmill[langgraph]`), backed by persistent `AsyncSqliteStore`; both use local FastEmbed defaults, no sessions, and verbatim storage. Backend construction goes through `build_backend(config.memory)`, which refuses non-verbatim storage. The contract also carries the multi-agent surface — `search_multi` (merged multi-namespace search), `add_returning_id`, `share_chunk` (provenance copy), `register_namespace` (metadata upsert) — with `BackendCapabilities.native_multi_namespace_search`/`share_provenance_links` documenting the memtomem-native vs adapter-fallback split (see the Memory section). Tests fake each SDK boundary; opt-in real smokes use temporary stores.
 - **Agents** (`agents.py`): subclass `CLIAgent`, implement `_build_command`, and override `_parse_output` / `_is_error_output` for that CLI's output format — you get both the sync `execute` and the async `aexecute` paths for free from the base class. Register the class in `_AGENT_CONSTRUCTORS`, add a matching `AgentConfig` field in `config.py` (an id with a hyphen — e.g. `kimi-code` — uses the underscore field name `kimi_code`, which `SyncmillConfig.agent_config` maps for you), and add the id to `agent_priority`. Output formats differ per CLI: Claude extracts the `result` key from its JSON, Codex joins `agent_message` items from its JSONL, and Kimi Code (`kimi`) keeps the `role:"assistant"` text lines from its `stream-json` JSONL. Only Claude exits 0 on an in-band error (hence its `_is_error_output`); the others signal failure with a non-zero exit.
-- **Strategies** (`strategies/`): implement the `Strategy` Protocol and register it in `_STRATEGIES`, then select it via `default_strategy` (or the `--strategy/-s` CLI flag). `_make_strategy` raises on an unknown name — mirror that hard-fail rather than silently defaulting. **Package layout** (extracted from `supervisor.py` in #12, once all four strategies made that module unwieldy — the demand the "don't pre-abstract" invariant gated on): `strategies/base.py` holds the `Strategy` Protocol plus every cross-strategy helper (`_enrich_prompt`; the gate helper `_gate_accepts`; the breaker helpers `_breaker_runnable`/`_breaker_record`/`_breaker_all_open_failure`; the failure builders `_no_stages_failure`/`_aggregate_failure`; the convergence seam `_select_winner`; and the per-worktree diff collection `_collect_files_changed`, which delegates to `worktree.capture_changes`) — diff collection lives *below* the strategy modules here, not in `supervisor.py`, precisely to keep the import graph acyclic (the Supervisor imports the registry, so it sits *above* the strategies). Each strategy is then its own module (`route.py`, `pipeline.py` with its private `_pipeline_failure`, `compete.py`, `council.py` with the `_council_*` prompt builders, `decompose.py` with the `_decompose_*` prompt builders + `_parse_subtasks`), and `strategies/__init__.py` owns the registry (`_STRATEGIES`, `available_strategies`, `_make_strategy`) because it needs every concrete class. The Supervisor imports `Strategy`/`_make_strategy` from the package and keeps only the run lifecycle + `_matched_order` (capability re-ordering is applied once at the Supervisor, never per strategy). Strategy tests stub `syncmill.strategies.base.capture_changes`; mandatory collection returns `CapturedChanges` or raises `PatchCaptureError`. Use real Git repositories for capture/index-preservation tests. Five strategies exist: `RouteStrategy` (try agents in priority order, fall back on failure, aggregate errors if all fail), `PipelineStrategy` (run agents as sequential stages — each reads prior stages' results from `shared` and writes its own there, fail-fast on a broken stage; the prompt enricher `_enrich_prompt` is shared by all), `CompeteStrategy` (run every agent **concurrently**, each in its own worktree, then keep a winner), `CouncilStrategy` (multi-round deliberation; see below), and `DecomposeStrategy` (split the task into subtasks, run them across agents concurrently, then synthesize; see below). Each strategy is handed a `WorktreePool` and owns its worktree topology — Route/Compete/Council acquire one isolated worktree per agent, Pipeline a single shared one, Decompose one per planner + per subtask + per synthesizer — and sets per-worktree `files_changed` (via `_collect_files_changed`) on the result it returns. `CompeteStrategy` is syncmill's first *concurrent* strategy: it bridges async via `run_async` over the `aexecute_isolated` fan-out (per-agent worktree + per-agent timeout), and converges through `_select_winner` under `config.selection.mode` (never wall-clock/`duration_seconds`). The default `verified` mode selects the first gate-passing COMPLETED result in priority order; `priority` checks only the first completed candidate; `judge` ranks all gate-passing candidates and falls back to verified priority if judging fails. It aggregates failures like Route when none qualify. `_select_winner` is the quality-gate (#10) and selection-policy seam (see the quality-gates extension point below); it does **not** auto-apply the winner's diff (worktrees stay disposable — the Supervisor shares the winner to `shared` and normally tears down the run worktrees; recovery retention is exempt). `CouncilStrategy` (#13, v0.3) is the **multi-round** strategy: each agent gets one worktree reused across three kinds of *concurrent* phase — **propose** (every runnable agent drafts a solution; the completers become the council *members* and each proposal is published to `shared`), **critique** (`config.council.rounds` times — each member re-reads the council's proposals + prior critiques from `shared` and refines its own worktree, publishing each critique back; a failed critique is tolerated, not dropped), then **synthesize** (every member produces a final solution from the whole deliberation, and the council converges by **reusing `_select_winner`** — so a council is "deliberate over rounds, then compete on the synthesized candidates"). Deliberation flows **only** through `shared` (relevance-ranked, best-effort — the same handoff Pipeline uses), never agent-to-agent, so the synthesized winner reflects more than one agent's input by construction. `members` is just `(agent, worktree)` pairs — no per-member result state — because synthesis re-runs everyone and `_select_winner` is the single convergence point; the returned winner's `duration_seconds` is the council wall-clock (sum of each concurrent phase's slowest agent). Like Compete it neither cleans worktrees nor applies the winner's diff — the Supervisor shares the winner and normally tears down the run worktrees. `DecomposeStrategy` (#14, v0.3) is syncmill's first task-**splitting** strategy — where every other strategy runs agents on the *same* task, decompose runs them on *different pieces* (the maintainer chose the concurrent/isolated integration model over a sequential shared-tree one: collaborate via shared *knowledge*, not file merges). Three phases: **decompose** — the highest-priority runnable agent (`runnable[0]`) plans the breakdown in a scratch worktree via the **sync** `execute` path (a single sequential step needs no fan-out, like route/pipeline), returning a JSON array parsed by `_parse_subtasks` (robust to fences/prose; a failed/empty/unparseable plan **degrades** to a single whole-task subtask, so a flaky planner never aborts — best-effort like memory); **subtasks** — the subtasks are distributed **round-robin** over the runnable agents (`runnable[i % len(runnable)]` — "across agents") and run **concurrently**, each in its own worktree via the new `aexecute_each` fan-out (the per-item-prompt counterpart to `aexecute_isolated`'s broadcast), each completed subtask's summary published to `shared` (a failed subtask is tolerated, not shared; **none** completing → aggregate failure, like council with no proposal); **synthesize** — every runnable agent reads the subtask summaries from `shared` and produces the integrated solution in a **fresh** worktree (broadcast via `aexecute_isolated`), converging through the shared `_select_winner` seam under the same selection policy. All memory I/O sits strictly between fan-outs (the shared-loop rule); the winner's `duration_seconds` is the decompose wall-clock (planner + slowest subtask + slowest synthesizer). Like Compete/Council it neither cleans worktrees nor applies a diff — integration is by shared knowledge (the subtask worktrees are torn down, their code discarded, exactly as the losers' trees are). `config.decompose.max_subtasks` (≥1, default 8) caps the breakdown.
-- **Quality gates** (`gates.py`, #10): a gate validates an agent's work **after it finishes**, against the diff in that agent's worktree — reactive, never a pre-run liveness probe (respects "no `health_check`"). Implement the `QualityGate` Protocol (`evaluate(worktree) -> GateReport`); the shipped `CommandGate` runs each configured command (via `shlex.split`, no shell — chain checks by adding entries, not `&&`) and passes iff all exit 0, in the **same isolated environment the agents get** (`isolated_git_env`) — a gate check reads the repository, so an ambient `GIT_DIR` would have it validate a different tree than the worktree under judgement, and an ambient `GIT_CONFIG_PARAMETERS`/`core.hooksPath` would execute someone else's script inside an agent-writable worktree. `build_gate(config.gate)` returns the gate or **`None` when no commands are configured** — the disabled default, so behavior is unchanged out of the box (set `SYNCMILL_GATE__COMMANDS='["pytest -q", ...]'`). The gate is plumbed through the **strategy layer only** — agents stay pure `(prompt, workdir, timeout)` functions and never see it. The Supervisor builds it per-run and injects it into `Strategy.execute(..., gate=)`; the shared `_gate_accepts` helper means "passed the gate" is one thing everywhere: pass → accept; fail → downgrade the result to FAILED with a gate error (attached as `result.gate`) so the rejection flows through the existing failure plumbing. Each strategy applies that verdict per its topology — **Route** falls back to the next agent, **Pipeline** fails the chain (gating only the final cumulative tree, since intermediate stages may legitimately be broken mid-chain), **Compete** downranks failers in `_select_winner`, **Council** gates only the synthesis phase (reusing `_select_winner` — propose/critique are ungated deliberation, only the converged final solution must clear the gate), and **Decompose** likewise gates only the synthesis phase (the planner and the individual subtasks are ungated — a subtask is a partial piece that needn't pass alone — and only the integrated synthesis, again via `_select_winner`, must clear the gate). A gate-rejected result is never shared to `shared` (it's a FAILED result, which the Supervisor already skips).
+- **Strategies** (`strategies/`): implement the `Strategy` Protocol and register it in `_STRATEGIES`, then select it via `default_strategy` (or the `--strategy/-s` CLI flag). `_make_strategy` raises on an unknown name — mirror that hard-fail rather than silently defaulting. **Package layout** (extracted from `supervisor.py` in #12, once all four strategies made that module unwieldy — the demand the "don't pre-abstract" invariant gated on): `strategies/base.py` holds the `Strategy` Protocol plus every cross-strategy helper (`_enrich_prompt`; the gate helper `_gate_accepts`; the breaker helpers `_breaker_runnable`/`_breaker_record`/`_breaker_all_open_failure`; the failure builders `_no_stages_failure`/`_aggregate_failure`; the convergence seam `_select_winner`; and the per-worktree diff collection `_collect_files_changed`, which delegates to `worktree.capture_changes`) — diff collection lives *below* the strategy modules here, not in `supervisor.py`, precisely to keep the import graph acyclic (the Supervisor imports the registry, so it sits *above* the strategies). Each strategy is then its own module (`route.py`, `pipeline.py` with its private `_pipeline_failure`, `compete.py`, `council.py` with the `_council_*` prompt builders, `decompose.py` with the `_decompose_*` prompt builders + `_parse_subtasks`), and `strategies/__init__.py` owns the registry (`_STRATEGIES`, `available_strategies`, `_make_strategy`) because it needs every concrete class. The Supervisor imports `Strategy`/`_make_strategy` from the package and keeps only the run lifecycle + `_matched_order` (capability re-ordering is applied once at the Supervisor, never per strategy). Strategy tests stub `syncmill.strategies.base.capture_changes`; mandatory collection returns `CapturedChanges` or raises `PatchCaptureError`. Use real Git repositories for capture/index-preservation tests. Five strategies exist: `RouteStrategy` (try agents in priority order, fall back on failure, aggregate errors if all fail), `PipelineStrategy` (run agents as sequential stages — each later stage gets every earlier stage's result directly (#125) and also writes its own to `shared`, fail-fast on a broken stage; the prompt enricher `_enrich_prompt` is shared by all), `CompeteStrategy` (run every agent **concurrently**, each in its own worktree, then keep a winner), `CouncilStrategy` (multi-round deliberation; see below), and `DecomposeStrategy` (split the task into subtasks, run them across agents concurrently, then synthesize; see below). Each strategy is handed a `WorktreePool` and owns its worktree topology — Route/Compete/Council acquire one isolated worktree per agent, Pipeline a single shared one, Decompose one per planner + per subtask + per synthesizer — and sets per-worktree `files_changed` (via `_collect_files_changed`) on the result it returns. `CompeteStrategy` is syncmill's first *concurrent* strategy: it bridges async via `run_async` over the `aexecute_isolated` fan-out (per-agent worktree + per-agent timeout), and converges through `_select_winner` under `config.selection.mode` (never wall-clock/`duration_seconds`). The default `verified` mode selects the first gate-passing COMPLETED result in priority order; `priority` checks only the first completed candidate; `judge` ranks all gate-passing candidates and falls back to verified priority if judging fails. It aggregates failures like Route when none qualify. `_select_winner` is the quality-gate (#10) and selection-policy seam (see the quality-gates extension point below); it does **not** auto-apply the winner's diff (worktrees stay disposable — the Supervisor shares the winner to `shared` and normally tears down the run worktrees; recovery retention is exempt). `CouncilStrategy` (#13, v0.3) is the **multi-round** strategy: each agent gets one worktree reused across three kinds of *concurrent* phase — **propose** (every runnable agent drafts a solution; the completers become the council *members* and each proposal is published to `shared`), **critique** (`config.council.rounds` times — each member re-reads the council's proposals + prior critiques from `shared` and refines its own worktree, publishing each critique back; a failed critique is tolerated, not dropped), then **synthesize** (every member produces a final solution from the whole deliberation, and the council converges by **reusing `_select_winner`** — so a council is "deliberate over rounds, then compete on the synthesized candidates"). Deliberation flows **only** through `shared` (relevance-ranked, best-effort — the same handoff Pipeline uses), never agent-to-agent, so the synthesized winner reflects more than one agent's input by construction. `members` is just `(agent, worktree)` pairs — no per-member result state — because synthesis re-runs everyone and `_select_winner` is the single convergence point; the returned winner's `duration_seconds` is the council wall-clock (sum of each concurrent phase's slowest agent). Like Compete it neither cleans worktrees nor applies the winner's diff — the Supervisor shares the winner and normally tears down the run worktrees. `DecomposeStrategy` (#14, v0.3) is syncmill's first task-**splitting** strategy — where every other strategy runs agents on the *same* task, decompose runs them on *different pieces* (the maintainer chose the concurrent/isolated integration model over a sequential shared-tree one: collaborate via shared *knowledge*, not file merges). Three phases: **decompose** — the highest-priority runnable agent (`runnable[0]`) plans the breakdown in a scratch worktree via the **sync** `execute` path (a single sequential step needs no fan-out, like route/pipeline), returning a JSON array parsed by `_parse_subtasks` (robust to fences/prose; a failed/empty/unparseable plan **degrades** to a single whole-task subtask, so a flaky planner never aborts — best-effort like memory); **subtasks** — the subtasks are distributed **round-robin** over the runnable agents (`runnable[i % len(runnable)]` — "across agents") and run **concurrently**, each in its own worktree via the new `aexecute_each` fan-out (the per-item-prompt counterpart to `aexecute_isolated`'s broadcast), each completed subtask's summary published to `shared` (a failed subtask is tolerated, not shared; **none** completing → aggregate failure, like council with no proposal); **synthesize** — every runnable agent reads the subtask summaries from `shared` and produces the integrated solution in a **fresh** worktree (broadcast via `aexecute_isolated`), converging through the shared `_select_winner` seam under the same selection policy. All memory I/O sits strictly between fan-outs (the shared-loop rule); the winner's `duration_seconds` is the decompose wall-clock (planner + slowest subtask + slowest synthesizer). Like Compete/Council it neither cleans worktrees nor applies a diff — integration is by shared knowledge (the subtask worktrees are torn down, their code discarded, exactly as the losers' trees are). `config.decompose.max_subtasks` (≥1, default 8) caps the breakdown.
+- **Quality gates** (`gates.py`, #10): a gate validates an agent's work **after it finishes**, against the diff in that agent's worktree — reactive, never a pre-run liveness probe (respects "no `health_check`"). Implement the `QualityGate` Protocol (`evaluate(worktree) -> GateReport`); the shipped `CommandGate` runs each configured command (via `shlex.split`, no shell — chain checks by adding entries, not `&&`) and passes iff all exit 0, in the **same isolated environment the agents get** (`isolated_git_env`; with `gate.env_allowlist` set, narrowed after isolation to the workflow common variables plus those exact names, and always narrowed for a workflow's final-node gate to also the final agent's `workflow_env_allowlist` — a gate runs agent-written code, so by default it gets no wider environment than that agent; names an operator lists are the exception) — a gate check reads the repository, so an ambient `GIT_DIR` would have it validate a different tree than the worktree under judgement, and an ambient `GIT_CONFIG_PARAMETERS`/`core.hooksPath` would execute someone else's script inside an agent-writable worktree. `build_gate(config.gate)` returns the gate or **`None` when no commands are configured** — the disabled default, so behavior is unchanged out of the box (set `SYNCMILL_GATE__COMMANDS='["pytest -q", ...]'`). The gate is plumbed through the **strategy layer only** — agents stay pure `(prompt, workdir, timeout)` functions and never see it. The Supervisor builds it per-run and injects it into `Strategy.execute(..., gate=)`; the shared `_gate_accepts` helper means "passed the gate" is one thing everywhere: pass → accept; fail → downgrade the result to FAILED with a gate error (attached as `result.gate`) so the rejection flows through the existing failure plumbing. Each strategy applies that verdict per its topology — **Route** falls back to the next agent, **Pipeline** fails the chain (gating only the final cumulative tree, since intermediate stages may legitimately be broken mid-chain), **Compete** downranks failers in `_select_winner`, **Council** gates only the synthesis phase (reusing `_select_winner` — propose/critique are ungated deliberation, only the converged final solution must clear the gate), and **Decompose** likewise gates only the synthesis phase (the planner and the individual subtasks are ungated — a subtask is a partial piece that needn't pass alone — and only the integrated synthesis, again via `_select_winner`, must clear the gate). A gate-rejected result is never shared to `shared` (it's a FAILED result, which the Supervisor already skips).
 - **Circuit breaker** (`breaker.py`, #8): reactive per-agent back-off from *repeatedly* failing agents — `RouteStrategy` already falls back on one failure, the breaker remembers failures **across runs**. `CircuitBreaker` tracks consecutive failures per agent: after `failure_threshold` in a row the circuit **opens** (the agent is skipped) for `cooldown_seconds`, then goes **half-open** (one trial), and any success **closes** it (resets the count). The three states are derived from `(failures, opened_at)`, never stored. **Reactive, not a `health_check`:** `allows()` decides purely from *recorded past outcomes* — it never calls an agent to test liveness; skipping an OPEN agent is "it kept failing when we ran it," not a probe. State **persists** to `state_path` (JSON, atomic temp+`os.replace`) loaded on construction — this is mandatory, since each `syncmill run` runs an agent at most once, so "N in a row" only accumulates across separate invocations. `build_breaker(config.breaker)` returns **`None` when `enabled=False`** (the default), so the disabled path never touches the filesystem and behavior is unchanged. Like the gate it's plumbed through the **strategy layer only** (agents never see it); the Supervisor builds it once (it carries cross-run state + a `degraded` flag the CLI surfaces, unlike the stateless per-run gate) and injects it into `Strategy.execute(..., breaker=)`. Shared helpers keep it uniform: `_breaker_runnable` drops OPEN agents (all five strategies skip them; Route falls back, Pipeline chains the remaining stages, Compete, Council, and Decompose fan out only the survivors — all return `_breaker_all_open_failure` if the breaker tripped everyone), and `_breaker_record` records each run agent's **RAW execution status taken *before* the gate** — so a gate rejection (a quality verdict on a *working* agent) does **not** trip the breaker; only an agent that fails to *run* does. **CouncilStrategy is the one deliberate exception to "one record per run."** A council runs each member up to `2 + rounds` times (propose + critique rounds + synthesize), but records the breaker **exactly once per agent, at the propose phase** — the unique phase where every runnable agent executes once, from identical input, so it is the fair "can this agent run?" probe. The threshold is denominated in *runs*; recording every council execution would make council trip the breaker ~`(2 + rounds)`× faster than the other strategies (a strategy-dependent regression) and would conflate execution-reliability with a working member struggling on the harder critique/synthesize prompts — the same reason a gate rejection never trips it. So council's critique/synthesize re-runs of an already-recorded member are deliberately **not** recorded, keeping council's contribution to the cross-run "N in a row" count one-per-agent-per-run, identical to the other strategies. **DecomposeStrategy preserves that same one-record-per-agent-per-run invariant by a different mechanism: it records each agent's RAW outcome at its FIRST execution this run** (a `recorded` set), deduping all later runs. Decompose has no council-style uniform-and-easy phase to record at: the planner is one agent on a planning prompt, the subtasks are a non-uniform round-robin, and synthesis (the only phase where *every* runnable agent runs) is the **hardest** prompt — recording there would over-penalize a working agent struggling on integration, the very reason council records at propose, not synthesize. So "first execution" records the planner at its short/early (typically easy) planning step and every other agent at its first subtask (or, failing that, its synthesis), probing every agent that runs exactly once — and, as everywhere, the pre-gate status, so a synthesis gate rejection never trips it. Two "agent didn't run" cases are deliberately different: a **degraded** breaker (unreadable/malformed state file) is best-effort/advisory like memory — it degrades *permissive* (`allows()` returns True for everyone, surfaced as a yellow CLI notice) so a broken breaker never blocks or aborts work; an **OPEN** circuit (recorded failures, still within cooldown) is *intentionally* skipped, and if that empties the candidate pool the strategy fails fast with a clear "all circuit-open, cooling down — retry or disable" result (`_breaker_all_open_failure`, parallel to the gate failing the run when every output is rejected). That fail-fast is the back-off doing its job — respecting the cooldown the user opted into — not the breaker aborting healthy work; "advisory / never aborts" describes only the degraded path.
 - **Capability matching** (`matcher.py`, #9): declarative, config-driven re-ordering of the candidate agents by *suitability* for a task, layered **over** the `agent_priority` baseline. Each agent declares capability tokens in config (`SYNCMILL_CAPABILITIES__AGENTS='{"codex": ["python"], "kimi-code": ["rust"]}'` — a map keyed by the **real** agent id, so hyphenated `kimi-code` is a verbatim key, unlike the underscore `AgentConfig` fields); a task carries a hint in its `Task.tags` (the previously-unused field, reused — set via the repeatable `--capability/-c` CLI flag). `DeclarativeMatcher.order` scores each agent by the **set-intersection size** of its declared capabilities with the (normalized, case-insensitive) hint and **stable-sorts descending**, so a matching agent moves ahead of raw priority while equal-fit agents keep priority order (the tie-break is sort stability over the input). It **re-orders, never drops** — the output is a permutation, so nothing is stranded and the two acceptance criteria hold by construction: a matching agent sorts ahead (score ≥ 1 > 0), and *nothing matching* (or no hint) is an all-zero stable sort = identity = priority order. **DECLARATIVE, never a probe** — a pure dict-lookup + set-intersection of static config with `task.tags`, strictly more passive than the breaker (respects "no `health_check`"). `build_matcher(config.capabilities, config.agent_priority)` returns **`None` when no agent declares a capability** (the disabled default → identity ordering, behavior unchanged); the optional `known_agent_ids` only **advisory-warns** (never raises) on a capability declared for an id not in the lineup (a config typo), while free-form *tokens* are never validated (an unmatched token is the graceful fallback). Unlike the gate/breaker — whose verdicts mean different things per strategy, so they're threaded **into** `Strategy.execute` — capability matching has **no topology-specific semantics**: it produces exactly one artifact, a re-ordering of the flat candidate list every strategy already consumes as a priority sequence. So the Supervisor applies it **once** in `run` (`_matched_order`, the `else`/non-forced path only — a forced `-a` agent has no ordering to influence), **before** dispatching to the strategy and **before** the breaker's per-strategy OPEN-agent filter (preference first, then availability) — making **all five** strategies capability-aware with no `Strategy` protocol change: Route tries the best-matched agent first, Pipeline leads its chain with it, Compete and Council make it the winner-priority (Council's members preserve the matched order, so the best-matched agent leads the synthesis selection), and Decompose makes it the planner + leads its subtask round-robin and synthesis-winner priority. This is the deliberate, uniform reading of #9's "layered over the existing priority order" (#9 only mandates Route; Pipeline/Compete/Council/Decompose are the free consequence of re-ordering at the source). Built once in `Supervisor.__init__` (stateless/config-derived, no filesystem — unlike the breaker); the CLI rebuilds a display-only matcher so the `Agents: …` banner shows the **actual** matched order rather than raw priority.
 - **Model routing** (`model_routing.py`, PR 1 of `docs/difficulty-model-routing-plan.md` — the canonical contract; read it before touching this): difficulty-based selection of the **provider-internal model** — never the agent lineup, order, or strategy semantics. Default `off` is byte-identical legacy behavior (no classifier, no decision, no ContextVar scope, no plan key). `[model_routing]` config (`ModelRoutingConfig`, the ONE nested model with `extra="forbid"` — a typo inside a cost-control table must be an error; `mode` rejects `enforce` as reserved) maps tier→provider→selector with **verbatim real-id keys** (`kimi-code`); selectors are shape-validated only (one argv token, no control chars) — an unavailable selector stays an ordinary provider failure (route falls back; the breaker records it like any failure). ONE immutable `ModelRoutingDecision` per top-level run, computed in `Supervisor.run` right after `_matched_order` + the no-agents guard and **projected onto the eligible lineup only**; `run_item` passes `enable_routing=False` (Board is a PR-2 surface — observe must never classify the ENRICHED board prompt), and a continuation (`parent_run_id` set) NEVER classifies — it inherits via `inherit_decision` (strict v1 validation, `ValueError("corrupt model routing metadata")` fails closed; a legacy parent = no flag even under observe). The classifier `prompt-complexity-v1` is deterministic and self-contained (EN = letter/digit-boundary phrase match, **KO = plain substring** — agglutination makes boundary matching near-zero-recall; URL tokens excluded from `path_targets`; inline code spans never trigger `multi_target`); a `ClassifierError` **degrades** in observe (warn + no decision — `Supervisor.last_routing_warning` → yellow CLI notice) but the explicit path fails closed (`AllUnmappedError` when NO eligible agent has a mapping). Execution rides `agents.py`'s `_MODEL_SELECTORS` ContextVar + `model_routing_scope(selectors)` (the scope carries the decision's immutable `effective_selectors` mapping, NOT the decision — keeps `agents → model_routing` un-imported/acyclic); the Supervisor opens exactly one scope around the strategy execution (`nullcontext()` when no decision — off opens NO scope) and each builder reads only its own id (Claude appends `--model`, Kimi's `-m` LEADS the argv, Codex's `-m` precedes the positionals in both fresh and resume shapes — pinned by exact-argv tests + a fake-provider E2E). Serialization is single-sourced: `decision.as_dict()` → `RunPlan.model_routing` (as_dict **omits** the key when None — off-mode plan JSON is golden-tested) → the manifest `plan` block; `TaskResult`/result.json/envelope/trace deliberately carry nothing. `continue` reads `manifest["plan"]["model_routing"]` (isinstance-guarded) → `run(inherited_routing=)` = **selector affinity** (stored selector wins over current config; observe/legacy parents resume with no flag). `--difficulty` forwards in `_inner_argv` (the `--team` rule: config selection); the compact banner (`render_banner`) prints only when a decision exists. Hermetic tests: `test_model_routing.py` (the §10.1 boundary matrix incl. multi-process determinism), plus routing sections in `test_config.py`/`test_agents.py` (a `reset_model_selectors` autouse fixture mirrors `reset_output_sink`)/`test_supervisor.py`/`test_cli.py`/`test_v2_workflow.py` (E2E affinity). PR 2 (enforce/Board/Rooms/TUI) is **calibration-gated** on the plan doc's §8 — don't build it without that evidence.

{syncmill-0.2.0 → syncmill-0.3.0}/docs/architecture/interactive-collaboration.md

@@ -25,7 +25,9 @@ be eligible there. Untagged notes, rows tagged only `task-result`, and
 | Council later critiques and synthesis | `council-proposal,council-critique` |
 
 These are relevance searches, not delivery guarantees: prior runs of the same
-kind still compete, and row/byte limits apply. Boards union-read both their run
+kind still compete, and row/byte limits apply. A run's own earlier-phase
+results do not depend on them — the strategy passes those to the next phase
+directly, in full (#125). Boards union-read both their run
 namespace and global `shared`, applying the same filter to both. Backends that
 cannot filter return unfiltered context with a warning and degraded status.
 

{syncmill-0.2.0 → syncmill-0.3.0}/docs/architecture/tracer.md

@@ -6,6 +6,7 @@
 > **Known stale statements** — checked against the code on 2026-10-21. The text below
 > is kept verbatim; where it disagrees with these notes, the code (and `CLAUDE.md`) win.
 >
+> - "with `start ≈ end − duration_seconds`" — concurrent phases (compete, council, decompose subtasks and synthesis) now place each attempt at the start its fan-out recorded (`started_at=`); `end − duration` is only the fallback when no start was recorded (#160).
 > - Attributes are not only the three tracegraph keys plus the `syncmill.*` allowlist: `_span_attributes` also emits `toolgraph.preflight.*` and `toolgraph.graph_generation` from toolgraph evidence.
 
 `tracer.py` is the flat module for the **OTLP-JSON trace exporter** — spans tracegraph ingests with zero adapter changes (`tracegraph ingest-otlp <file>`), because syncmill emits the exact three attributes its `OTLPSpanAdapter` consumes (`openinference.span.kind`, `graph.node.id`, `graph.node.parent_id`) plus an additive `syncmill.*` **allowlist** tracegraph ignores. The emitted document shape is **pinned byte-level by tracegraph's T1 generators** (`examples/syncmill_otlp_traces.py` + its goldens) — protobuf-JSON attr encoding (`intValue` carries int64 as a *string*; bool checked before int), span `name` as a **stable structural label** (agent ids/phases/indices, never uuids/timestamps — tracegraph `diff`/AHU labels on `name`, so an unstable name makes every diff a divergence), flat `parentSpanId` containment under the run root (causality rides `graph.node.parent_id` + span `links`, never `parentSpanId` and never timestamps), and bounded error messages (`timeout` | `exit_code=N` | `gate: <summary>` — raw output is forbidden). **Retrospective recorder, not live spans:** strategies append completed `SpanRecord`s right after each attempt/fan-out returns (the same shared-loop boundary as `_breaker_record` — a live tracer would penetrate the agent layer), with `start ≈ end − duration_seconds` (safe: tracegraph never promotes timestamps to causality). **All five strategies are instrumented** (route + compete in S3′; pipeline/council/decompose in S6). The span builders live in `strategies/base.py` (the single generalized `_attempt_span` — `phase=`/`name=`/`round_no=`/`links=` params cover stage/propose/critique/synthesize/plan/subtask — plus `_gate_span`/`_select_span` beside the `_monitor_*` helpers) so link/naming semantics can't drift between strategies; builders put **bare-suffix keys** in `attrs` and `FileTracer.flush()` prefixes `syncmill.` + enforces the allowlist + injects `run_id`/`schema_version`, so a builder can never leak a body-shaped attribute. Semantics locked by the T1 goldens + the design's §2.3 span table: route attempt `node_id=attempt:{agent}:{i}` with parent = the **previous failed attempt** (fallback = caused by the prior failure); compete attempts `attempt:{agent}` all parent on `run` (completion order is never causal), `worktree_slot` = fan-out index; **attempt spans carry the RAW pre-gate status** (snapshot taken before `_gate_accepts`/`_select_winner` mutate results in place — a gate rejection is a quality verdict, same reason the breaker records raw; council/decompose snapshot duration too, since the winner's `duration_seconds` is mutated to the phase-summed wall-clock); gate spans (`TOOL`, parent = their attempt) exist only for candidates actually gated (the lazy `_select_winner` prefix); the `select` span parents on the **winning attempt** and links **every other raw-COMPLETED candidate** (the golden reading of "examined" — computable without changing lazy `_select_winner`; a timed-out sibling is never linked; a gate-rejected raw-completer is). **S6 semantics:** pipeline stages `stage:{i}:{agent}` chain on the previous stage (the chain IS the causality; a broken stage leaves its span and stops; the gate-accepted final stage carries `winner`/`files_changed_count` with its OWN pre-sum duration, its verdict on a `gate:{agent}:{i}` span; no slot, no select). Council: `propose:{agent}` fans out from `run` (the one fresh-worktree phase → slot); `critique:{agent}:{r}` (attr `round`) and `synthesize:{agent}` parent on that member's OWN previous-phase node (**worktree continuity** — a FAILED critique still becomes the next parent) while `links` carry the previous phase's **PUBLISHED** (COMPLETED → shared) nodes — declared availability, never claimed consumption (the design's finding-6 approximation: the handoff is relevance-ranked best-effort search, so actual consumption is unknowable). Decompose: `plan` is the hub (name `plan:{planner}`, recorded even FAILED — the degraded whole-task subtask still parents on it); `subtask:{i}` parents on `plan` (name `subtask:{i}:{agent}` — the round-robin repeats agent ids, so the diff-stable label needs the index; slot=i); `synthesize:{agent}` parents on `plan` linking the COMPLETED subtask nodes (fresh worktrees → slot). Council/decompose reuse compete's select-span semantics on the synthesize candidates; an **early aggregate-failure** (no council members / no completed subtasks) leaves **no select span** — selection never ran, distinct from ran-but-nobody-won (select parents `run`). `syncmill.status` is derived (`TaskStatus` has no TIMEOUT: `error.startswith("Timed out after")` is the only timeout signal; `cancelled` is contract vocabulary but unreachable — a cancelled agent re-raises, never becomes a TaskResult); `syncmill.exit_code` only when `>= 0` (the `-1` sentinel is noise); `files_changed_count`/`winner` stamped on the winner only (losers' worktrees are never diffed — a deliberate divergence from the synthetic goldens, which are a shape contract, not an attr-value contract). **Wiring:** `build_tracer(config.integration.trace, run_id=, fallback_dir=config.integration.report.runs_dir)` returns **`None` when disabled** (the default — byte-identical off path); `fallback_dir` is a deliberate signature addition (`dir=""` means "the report runs_dir" and only the Supervisor sees `ReportConfig`); `otlp-http` keeps the durable file and additionally streams completed body-free spans to Phoenix through the optional exporter; streaming failures and queue drops degrade to warnings/receipts without failing the run. `include_output` remains warn+ignore in `build_tracer` (once per run), not a config validator (which would fire on every construction). The Supervisor records the `syncmill.run` root span in its `finally` (final status known there) and flushes **before `reporter.finish`** (which digests the artifact into the envelope `trace` block: `{status:"ok", protocol:"otlp-json-file", trace_id, artifact:"trace.otlp.json", digest}`) — both guarded so nothing trace-shaped can skip the memory-session close or `pool.cleanup_all()`; a flush failure yields `trace:{status:"error"}` + `"trace"` in the envelope `degraded` array. `FileTracer.flush()` is **all-or-nothing** (buffer + mkstemp/`os.replace` — `OTLPSpanAdapter` raises on partial documents, dangling parents, duplicate spanIds; a dangling link/duplicate node_id degrades with a warning rather than emitting a rejectable file). **Trace-only retention** (closing the v1 residue): trace enabled + report disabled used to leave `{runs_dir}/{run_id}` dirs holding only trace files un-swept forever, since retention rode `reporter.start` and `build_reporter` returns `None`. The Supervisor now calls `report.sweep_runs_dir` under `report.keep_last` when `reporter is None ∧ tracer is not None ∧ not trace.dir` — one `keep_last` bounds SyncMill-owned run directories whichever producer fills them. Placement is load-bearing: **FIRST statement inside `run`'s `try`**, not before it, so a `KeyboardInterrupt`/`SystemExit` mid-sweep still enters finalization — the `except BaseException` recovery path and the `finally`'s trace flush — instead of escaping the lifecycle entirely (note that path *retains* the workspace via `pool.retain()`, making `cleanup_all` a deliberate no-op; the point is that finalization runs, not that worktrees are removed); and being inside the try (after the snapshot-capture and artifact-init early returns at supervisor.py:505/:557) means an early return can never prune old runs while producing no trace of its own. `tracer is not None` — not `trace.enabled` — so a degraded `build_tracer` (unresolvable `~`) also keeps the both-off path at zero filesystem touch. The strict-preflight branch needs no sweep: `_validate_strict_reporting` rejects `toolgraph.mode=strict` without `report.enabled`, so it is always reporter-backed. **Remaining deliberate residue:** a custom `trace.dir` is NEVER swept — a user-pointed directory is user-owned and may hold more than run dirs, so syncmill won't delete inside it. Hermetic tests never import tracegraph: a **pure-python mini-validator** (`test_supervisor._mini_validate_otlp`) mirrors the adapter's raise conditions, `test_tracer.py` pins the document shape against the T1 contract, and the strategy tests drive `FakeTracer` (conftest, the FakeMonitor template) through all five `_run_*` helpers (plus a real-FileTracer decompose E2E covering the round-robin node-uniqueness risk).