@giamme/forge 0.0.0-stage → 0.9.0

package/docs/safety.md

@@ -0,0 +1,55 @@
+# Safety
+
+[← README](../README.md) · [All docs](README.md)
+
+What forge never does to your branch and working tree, what it refuses to run on, and how it keeps reviewed work and merged work the same.
+
+## Safety guarantees
+
+- Solo runs leave source changes uncommitted. Forge creates private review commits, but
+  does not commit on the solo source branch, push, or reset the user tree.
+- In a decomposed run, commits and merges happen **only** on the `forge/<run-id>/*` task
+  branches and the `forge/<run-id>-integration` branch — branches forge created for itself.
+  Never your branch, never a push. Your branch and working tree are untouched for the whole
+  run.
+- Bypass flags are never added on forge's own initiative. `--yolo-dwarf` / `--yolo-qa` are the
+  only route to an unsandboxed run, so that choice is always explicit and visible.
+- **One dwarf per working tree, always.** Two agents editing one tree interleave their edits
+  into a diff neither of them wrote and qa cannot untangle. Parallelism comes from separate
+  worktrees, which is why worktrees are mandatory there rather than an optimization.
+- A failed task's branch and worktree are **never** deleted — they're the only record of what
+  went wrong and the only thing to retry from. A successful task's worktree is removed only
+  after `results.tsv` is written and read back.
+- QA runs in a disposable repository. Mutations to the source or review snapshot invalidate
+  acceptance. Disabled editing tools alone do not prevent writes through Bash.
+
+## Preconditions
+
+- Not a git repo → decomposed runs refuse. Worktrees are impossible and there's no honest
+  fallback.
+- Uncommitted changes → **warned prominently.** Worktrees branch from a commit, so your local
+  edits are invisible to every dwarf. This is the likeliest route to a confusingly wrong
+  result.
+
+## Execution integrity
+
+Forge now captures solo changes against a recorded starting snapshot, including staged edits,
+without altering the user's index. Existing work is recorded separately in `existing.diff`.
+QA receives the full implementation requirements and reviews a disposable repository; source
+or snapshot changes invalidate the result. Only the recorded reviewed task commit can merge.
+
+Parallel execution rechecks dependencies before each task and retry. Failed prerequisites leave
+consumers BLOCKED. Directory and child file declarations are serialized.
+
+`forge-dispatch.sh doctor --spec sol --role dwarf` validates a selected model recipe using local
+CLI help. Solo and parallel runners preflight the whole selected pipeline before implementation.
+These checks do not establish authentication, billing or live model availability.
+
+Use `forge-parallel.sh run <plan> --verify 'your check command'` or put the command in
+`.forge/verify` to verify the combined result. Output and exit status are saved with the plan.
+Integration checks a candidate containing the current user branch before updating it. A missing
+command is reported as UNVERIFIED; a failing configured check blocks integration. The project's
+own test suites that the command does not run are named as warnings (advisory), and a failing
+command is rerun once so a flaky lane is reported rather than failing the run — see
+[verification coverage and flaky reruns](reviews.md#verification-coverage-and-flaky-reruns).
+`integrate` also names any task that is not merged and therefore not part of what it delivers.

package/docs/troubleshooting.md

@@ -0,0 +1,54 @@
+# Troubleshooting
+
+[← README](../README.md) · [All docs](README.md)
+
+Symptoms, causes and fixes, from a skill a harness cannot see to a task that will not merge.
+
+| Symptom | Cause / fix |
+|---|---|
+| skill missing in **codex** only | frontmatter isn't strict YAML. Codex refuses to load and logs `invalid YAML`; the other four are lenient and load it fine. Note codex's line numbers exclude the opening `---`, so its "line 3" is the file's line 4 |
+| skill missing in **antigravity** | `agy` ignores `~/.gemini/skills` entirely — skills load only inside a plugin. Re-run `forge-install.sh` |
+| antigravity runs an **old version** | `agy plugin install` copies. Re-run `forge-install.sh` after every edit |
+| duplicate entry in **opencode** | opencode auto-scans `~/.claude/`; a `~/.config/opencode/command/forge.md` on top is redundant |
+| agy: `Find command timed out` | missing `--add-dir <repo>` — forge passes it; a hand-built command must too |
+| agy dwarf ends with an error and an empty diff | it tried to run a shell command without yolo. A single denial aborts the run — use `--yolo-dwarf` |
+| dwarf produced no changes | usually an ambiguous prompt it couldn't resolve headlessly. Nobody can answer a question mid-run |
+| task status `ERROR` | worktree creation or a dispatch failed — read `tasks/<id>/dwarf.out` / `qa.out` |
+| task status `UNKNOWN` | QA never emitted a verdict line; read `tasks/<id>/qa.last`. `retry` (or `run --retry-failed N`) asks QA again on the unchanged work — no dwarf runs, no attempt is spent |
+| task status `INFRA`, `run` exits `8` | a quota/usage limit, auth problem, rate limit, network error or empty answer stopped a dispatch — not a task failure, and no attempt was spent. Read `tasks/<id>/infra.txt` (`class=`, `retry_after=`), fix the cause, run the same command again. `run --infra-retries M` waits it out instead. Dependents stay `PENDING`, never `BLOCKED` |
+| a task merged although its reviewer said FAIL | `--qa-threshold` accepted it: every CONFIRMED finding was labelled and milder than the threshold. See `tasks/<id>/known_issues.md` and `qa.gate`; nothing is accepted without an explicit threshold |
+| `accept` exits `3` "changed since it was reviewed" | the branch or worktree moved after QA, so accepting would sign off on unreviewed code. `retry` for a fresh review |
+| `combine` / `split` refuses | the message names why: unmerged task left behind, duplicate ids, locked or dirty plan, a Fractal-backed plan, or a selection that strands a dependent. Nothing was changed |
+| `integrate --final-review` stops with exit `5` | the whole-run reviewer found a blocking defect (or never reached a verdict); findings are in `<plan>/…/final-review/qa.last`. Your branch is untouched. Exit `8` means the reviewer hit a quota/auth stop: run it again |
+| `FLAKY` in the summary, verification `PASS` | the verify command failed once and passed on rerun. Read `verification.log` (the first run) — an intermittent failure can be a real bug. `--verify-retries 0` restores the strict single run |
+| `verify coverage: suite '…' is not run by the verify command` | the project defines a suite your `--verify` skips. Advisory only; add it to the command (or `.forge/verify`) if it should gate the merge |
+| implementer guard notes in the summary | the dwarf committed, switched branch, left processes running, or promised later work. Recorded in `tasks/<id>/guard.txt`; the diff and the reviewer still decide. Leftover processes were stopped |
+| task status `CONFLICT` | QA passed but the merge conflicted — `files` was understated somewhere. Check the run's scope-drift list: it usually names the file |
+| task status `TIMEOUT` | that role exceeded the dispatch timeout and was killed. Raise `--timeout`, or `retry` — the partial edit is still in the worktree |
+| a dispatch never returns | it can't any more: every backend is killed at its timeout (`--timeout`, else `FORGE_TIMEOUT`, else the registry's `timeout` column, else 45 min — 60 min for the default `xhigh` QA/planner effort) |
+| a task won't re-run: "already exists" | a worktree directory was deleted by hand. `run` and `retry` prune stale registrations first, so re-run rather than deleting the branch |
+| a failed task is expensive to redo | it isn't — `forge-parallel.sh retry <dir> <id> [--dwarf <spec>]` re-runs that one task with its reviewer's findings, in the worktree it already has |
+| an interrupted run seems lost | run `run` again; merged tasks are skipped, not re-dispatched |
+| every wave has exactly one task | `files` sets overlap across most tasks; the decomposition isn't actually parallel |
+| dwarves reimplement already-merged work | `goal.txt` or `files` missing, so the capsule carries no useful status |
+| several tasks report "produced no changes" | the decomposition made tasks that weren't independently actionable — use a coarser `--decompose-level`, not more retries |
+| `retry` says "produced no new changes" | the dwarf returned a byte-identical diff, so no reviewer was paid to read the same code twice. Escalate with `--dwarf`, or fix the prompt |
+| a new file's code never reached QA | fixed: untracked files are diffed against `/dev/null`. A hand-built capture using only `git status --porcelain` shows the reviewer a filename and no content |
+| claude dwarf ends "I need your permission to edit" | a permission mode that prompts for Edit. forge uses `acceptEdits` plus an explicit Bash allowance, because `auto` allows shell but prompts for Edit, and `acceptEdits` alone allows Edit but prompts for real shell commands |
+| `.forge/` turns up in a review | a diff capture is missing `':(exclude).forge'` |
+| memory stays empty | nothing durable was learned — the common, correct case. The ledger still has a row per dispatch |
+| a wrong fact keeps reappearing | `memory.md` is rebuilt from the ledger; delete the ledger rows, not the memory line |
+| dwarves build incompatible interfaces | the seam was never agreed — plan the approach, or use `--planner` |
+| Fractal was installed but a new unattended run uses ordinary execution | installation never activates it; pass `--fractal` explicitly |
+| `Fractal selected but unavailable` | run `forge fractal doctor`, then the suggested managed install; resume retains the selected backend |
+| existing run rejects a new backend, pool or limit | selection and bounds are recorded; create a new run directory for changed settings |
+| Fractal reports completion but Forge has not accepted the work | inspect QA, verdict and verification; execution completion does not establish acceptance |
+| Fractal import stops with `source_drift` | the source changed after its snapshot; preserve both versions and inspect the candidate instead of forcing import |
+| Fractal work was paused or its coordinator exited | inspect `status` and `logs`, then use explicit `resume RUN_ID`; retry starts a new attempt |
+| browser dashboard closed while work is running | execution is independent; reopen with `forge fractal open RUN_ID` |
+| costs are unknown or a dollar cap is rejected | the bridge only reports available usage; v1 cannot enforce a spending cap |
+
+For per-harness invocation details, effort ladders and known CLI failure modes, see
+[`references/harnesses.md`](../references/harnesses.md). For decomposition mechanics, see
+[`references/decompose.md`](../references/decompose.md). For memory internals, see
+[`references/memory.md`](../references/memory.md).

package/forge

@@ -0,0 +1,79 @@
+#!/usr/bin/env bash
+set -uo pipefail
+# npm runs this file through a symlink in its bin dir, usually a relative one. Follow the
+# links so ROOT is the package and not the bin dir; older macOS has no readlink -f.
+SELF="${BASH_SOURCE[0]}"
+while [ -L "$SELF" ]; do
+  LINK="$(readlink "$SELF")"
+  case "$LINK" in
+    /*) SELF="$LINK" ;;
+    *) SELF="$(dirname "$SELF")/$LINK" ;;
+  esac
+done
+ROOT="$(cd -P "$(dirname "$SELF")" && pwd)"
+
+usage() {
+  printf '%s\n' \
+    'usage: forge <command> [args]' \
+    '' \
+    'commands:' \
+    '  install   link forge into each agent harness on this machine; --dry-run previews' \
+    '  fractal   opt-in Fractal execution: install, doctor, runs, status, tree, logs, open, report, ...' \
+    '  jev       optional Jev advice: setup, status, doctor, enable, disable, backtest' \
+    '  dataset   every run'"'"'s inputs and outcomes: status, dump, import, export, outcomes, ...' \
+    '  ui        forge-ui, the run viewer: open [DIR], status, stop' \
+    '  runs      this repo'"'"'s recent runs in the terminal: state, verdicts and run dirs' \
+    '  update    fast-forward this checkout and refresh every harness installation' \
+    '  version   the installed version, and whether a newer release is out' \
+    '' \
+    'Run `forge <command> --help` for its options.' \
+    'Runs start from your agent harness (/forge "<goal>" --dwarf <model>); see README.md.'
+}
+
+case "${1:-}" in
+  install)
+    shift
+    exec /bin/bash "$ROOT/scripts/forge-install.sh" "$@"
+    ;;
+  fractal)
+    shift
+    exec python3 "$ROOT/scripts/forge-fractal.py" "$@"
+    ;;
+  jev)
+    shift
+    exec python3 "$ROOT/scripts/forge-jev.py" "$@"
+    ;;
+  dataset)
+    shift
+    exec python3 "$ROOT/scripts/forge-dataset.py" "$@"
+    ;;
+  ui)
+    shift
+    exec python3 "$ROOT/scripts/forge-ui.py" "$@"
+    ;;
+  runs)
+    shift
+    exec python3 "$ROOT/scripts/forge-runs.py" "$@"
+    ;;
+  update)
+    shift
+    exec /bin/bash "$ROOT/scripts/forge-update.sh" "$@"
+    ;;
+  version)
+    shift
+    exec python3 "$ROOT/scripts/forge-version.py" "$@"
+    ;;
+  -h|--help|help)
+    usage
+    exit 0
+    ;;
+  '')
+    usage >&2
+    exit 2
+    ;;
+  *)
+    printf 'forge: unknown command: %s\n\n' "$1" >&2
+    usage >&2
+    exit 2
+    ;;
+esac

package/package.json

@@ -1,6 +1,29 @@
 {
   "name": "@giamme/forge",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.9.0",
+  "description": "Multi-model coding with a reviewer that reads the code: plan, build in parallel worktrees, review every real diff, merge only what passes.",
+  "keywords": ["claude-code", "codex", "opencode", "agent-skill", "code-review", "multi-model"],
+  "license": "MIT",
+  "author": "Giamme",
+  "repository": { "type": "git", "url": "git+https://github.com/Giamme/forge.git" },
+  "bin": { "forge": "forge" },
+  "os": ["darwin", "linux"],
+  "files": [
+    "forge",
+    "SKILL.md",
+    "VERSION",
+    "CHANGELOG.md",
+    "README.md",
+    "docs/",
+    "LICENSE",
+    "registry.tsv",
+    "references/",
+    "scripts/",
+    "ui/forge_ui/",
+    "ui/README.md",
+    "ui/THIRD_PARTY_NOTICES.md",
+    "ui/docs/api.md",
+    "!**/__pycache__"
+  ],
+  "publishConfig": { "access": "public" }
+}

package/references/dataset.md

@@ -0,0 +1,329 @@
+# Run dataset
+
+Every forge run records its inputs and outcomes in one global store outside every repo,
+so a decision model (Jev, Laya or any other) can later be trained or calibrated on what
+forge actually did. Run and plan dirs hold nearly all of this, but they live in `/tmp`
+and scratchpads and get deleted. `.forge/ledger.tsv` survives, but it keeps only spec
+strings and verdicts. It has no inputs, no resolved models, no attempt numbers and no
+tokens.
+
+```
+${FORGE_DATA_HOME:-${XDG_DATA_HOME:-~/.local/share}/forge}/dataset/
+  dataset.sqlite        events (rid-keyed, append-only) and repos
+  blobs/<aa>/<sha>.gz   prompts, diffs, messages, logs: capped, gzipped, deduplicated
+  errors.log            capture failures
+```
+
+Capture is on by default and costs a few milliseconds per execution. `FORGE_DATASET=off`
+turns it off for one invocation.
+
+## Raw events, not training examples
+
+The store holds **what happened, when it happened**: the state each decision saw and the
+outcomes that followed. It never holds (state, question, label) rows. Those are derived
+per decision by an exporter, so a question nobody has asked yet can still be answered
+from old runs without re-running anything. Labels come from folding a run's events in
+time order: an attempt that PASSed, then a merge that CONFLICTed, then a human `accept`.
+
+| kind | one per | what it holds |
+|---|---|---|
+| `start` | execution | a receipt written by `begin()`. A start with no matching end is an execution that was killed or is still running |
+| `dispatch` | model call | role, alias, harness, **resolved** model, effort after clamping, timeout and its source, delivery (custom or native review), phase timings, exit, timeout, infra class, tokens, parsed verdict and findings, `FORGE_LEARNING` lines. Blobs: the delivered prompt and the final message |
+| `attempt` | task attempt or solo run | status, attempt number (with refund and resume flags), trigger (`initial`, `auto`, `human`), dwarf and qa spec, declared vs changed files, numstat, drift, guard notes, gate line, patch key, the files this execution wrote and the ones it only carried. Blobs: the diff, retry findings, memory slices, capsule, Jev task files |
+| `verification` / `verify_run` | verify, and each command execution | command and its source (flag, `.forge/verify`, Jev), status, flaky, Jev triage. One `verify_run` per execution, each with its own log |
+| `plan` | `plan` | goal, original request, decompose level, exploratory routing decisions (`tasks[id].route`, with their odds), author (orchestrator or planner), per-task declared vs executed difficulty and who set it, pools, waves, Jev routing and coupling. Blobs: both task tables, prompts, approaches, excerpts of the declared files at plan time, the file inventory, `memory.md` |
+| `run_end` | `run` | results per task, the accepted integration commit, `memory.md` |
+| `invocation` | user-facing `forge-parallel.sh` subcommand | argv, exit, duration, plus final review, accept, integrate and split/combine payloads |
+| `event` | point event | `merge` (MERGED, CONFLICT with the files, INVALIDATED and why), `integrate_advanced`, `integrate_conflict`, `combine_conflict`, `sched_terminal`, `subset_run` (every outcome), `memory_record`, `memory_prune` |
+| `event` `escalation` | `retry --dwarf` | the model a retry escalated from and to, recorded at the rewrite, before the retried attempt begins |
+| `event` `shadow_review` | shadow review | how an opt-in second review ended (`done`, `invalid`, `overtime`, `orphaned`, ...), with the reviewed commit and both specs. Its dispatch is recorded with role `qa_shadow` |
+| `jev` | Jev request | site, answers, usage, latency, mode. Blob: the **full request** (state and questions). Its original sha is `jev.jsonl`'s `key` |
+| `fractal_step` / `fractal_node` | Fractal step, node state | decomposition decision, iteration, resolved model, cost, tokens, candidate diff |
+| `ledger` | ledger row | imported `.forge/ledger.tsv` rows |
+| `outcome` | integrated task, per horizon | written by `forge dataset outcomes`, never by a run: how many of the lines a task added survive N days after `integrate`, and what reverted it |
+
+Each blob reference carries `sha` (the stored bytes), `original_sha`, `n` (the original
+size), `kept`, and `cut`. A cut keeps the head and the tail, because a QA prompt ends with
+the diff and a log ends with the failure. Caps: prompt 1 MiB, diff 2 MiB, message 256 KiB,
+log 272 KiB. Transcripts (`*.log`) and `.cmd` (which some harnesses use to carry the
+prompt in argv) are never stored. A harness without structured output has its final
+message copied from the whole log, so for those harnesses only the tail is kept, flagged
+`final_source: log_copy`.
+
+## How it is captured
+
+All capture goes through a handful of hooks. Every one of them is fail-open: errors go
+to `errors.log`, nothing is printed, and nothing changes an exit code.
+
+- **`forge-runtime.py begin()/finish()`.** Every dispatch, task attempt, solo run,
+  verification and parallel run already passes through these. `finish()` captures from a
+  `finally:` block, so an extraction error cannot lose the record. This is the only
+  hook that sees every exit path: an empty diff, INVALIDATED, a timeout, an infra stop
+  or a `die`.
+- **Provenance travels with the execution.** `begin()` writes
+  `attempts/<role>-<uuid>/dataset.json` with the run id, task, starting attempt counter,
+  trigger and a signature (inode, mtime, size) of every task file. `finish()` credits a
+  file to this execution only when its signature changed. A dwarf that fails returns
+  before `changes.diff` is refreshed, and without this check it would inherit the
+  previous attempt's diff and verdict.
+- **Point events** (`forge_dataset_event`) record facts that disappear before any EXIT
+  trap could read them: a merge that gets aborted, a combine that rolls back, a user
+  branch that moves just before a cleanup that may still fail.
+- **An EXIT trap in `forge-parallel.sh`** records every user-facing subcommand, `die`
+  included.
+- **`forge_jev/client.py`** keeps the canonical request bytes of every live Jev call,
+  including calls made without a run dir. It skips fixture replays, the setup probe and
+  backtests.
+
+Run identity is a uuid in `<plan-or-run-dir>/dataset.uid`, created once (`O_EXCL`) by
+`plan` or by the first `begin()`. Split and combine copy attempt dirs unchanged, so a
+copied execution keeps its original identity and collapses to one event by rid.
+
+## Which forge produced an event
+
+Outcomes depend on the prompts, contracts and registry forge ran with, and the skill is
+often a live dev checkout that changes between releases. So every event carries four
+provenance columns:
+
+| column | what it identifies |
+|---|---|
+| `forge_version` | the release in `VERSION` |
+| `registry_sha` | the effective `registry.tsv` (or `FORGE_REGISTRY`) |
+| `code_sha` | the content of every file that shapes a run: `scripts/`, `references/`, `SKILL.md`, `VERSION`, `forge` and the effective registry. Two runs share it only if they ran the same code, uncommitted edits included |
+| `forge_rev` | the skill checkout's `HEAD` (`git rev-parse`, read-only), or null for a copied skill dir |
+
+An execution is stamped with what `begin()` saw, because its prompt was built then.
+`finish()` runs in another process; if the code changed in between, the event carries
+`code_changed: true` and can be left out of any per-regime analysis.
+
+An imported run was produced by whatever forge ran it, so the importer stamps nothing,
+unless a live `begin()` left its provenance in the attempt dir. Rows imported before
+schema 2 carry the importer's version by mistake, so readers treat `forge_version` and
+`registry_sha` on `src = 'import'` rows without a `code_sha` as unknown. The exporter
+gives those a `forge_rev_estimate`: the last forge commit before the run started, which is a
+guess and is labeled as one. Schema 2 added `code_sha` and `forge_rev`. The first
+capture after an upgrade adds them in one `BEGIN IMMEDIATE` transaction, and an older
+forge still running keeps writing.
+
+## Commands
+
+```sh
+./forge dataset status [-v] [--json]    # where, how much, and what looks incomplete
+./forge dataset dump [--kind K] [--run R] [--task T] [--since UNIX]   # events as JSON lines
+./forge dataset blob <sha>              # print one stored blob
+./forge dataset import <dir|archive|legacy-dir>... [--registry] [--fractal] [--ledger REPO] [--dry-run]
+./forge dataset export [--slice S]... [--limit N] [--seed N] [--out DIR]   # labeled decision cases
+./forge dataset outcomes [--horizon-days 14] [--repo PATH] [--dry-run]     # what integrated work became
+./forge dataset routing [--repo PATH] [--since DAYS] [--json]   # pass rate and runtime per tier and model
+```
+
+`status` runs a different completeness check for each mode, because each mode
+legitimately lacks different things. A solo run has no separate verification, and a
+final review's QA has no task attempt. The checks are:
+
+- executions that started but never ended
+- task QA dispatches with no task attempt
+- parallel PASSes with no merge outcome in a finished run
+- parallel runs with no end
+- blob references with no stored blob
+
+## Exporting labeled decisions
+
+`export` folds the events into cases in the System One wire format (one JSON object per
+line with `id`, `state`, `questions` and `expected`), the format decision-model
+benchmarks and fine-tuning recipes take. Each case is a decision Jev makes in forge,
+with its state built the way `forge_jev` builds it for that site and its label taken
+from what happened afterwards:
+
+| slice | question | label |
+|---|---|---|
+| `routing` | will the first attempt at this task pass review? (state adds the dwarf and reviewer models) | the final status of the first real attempt (see below) |
+| `routing` | which difficulty tier is this task? | the tier the orchestrator or planner declared. This is agreement, not an outcome |
+| `retry` | will this retry pass, given the review findings it was handed? (state adds the attempt number and the previous review) | the final status of that attempt (see below) |
+| `review` | will the independent reviewer pass this diff? (state names the reviewer's model, effort and harness) | the QA verdict. A diff whose reviews disagreed is dropped |
+| `review` | will most of the lines this diff adds stay in the code, unrewritten and unreverted? | from `outcomes`: at least 80% of the added lines still blamed to the task's commits at the horizon, and no revert. Only the review of the diff that landed gets it |
+| `review` | what is the most severe problem the reviewer will find? (`none`, `P3` to `P0`, the contract's scale) | the worst CONFIRMED finding, only when every CONFIRMED finding carries a severity and none is UNSTATED. A review whose findings cannot be fully read gets no label rather than a wrong `none` |
+| `drift` | is this undeclared file likely to need editing? | whether the passing attempt changed it. Negatives are undeclared files near the task, plus random ones |
+| `citation` | do the places this review cites appear in this diff? | the review paired with its own diff, or with another task's diff from the same repo that shares no file |
+| `memory` | is this recorded fact worth including for this task? | the fact's anchor is a file the task edited (true), or lies in an area it never touched (false). Anything in between is left out |
+
+The reviewer is part of the review state because strictness differs by reviewer: the
+same kind of diff passes far more often under one reviewer than another, so a label
+means little without knowing who gave it. Routing and retry states also carry a `size`
+block (`prompt_chars`, `declared_files`, `plan_tasks`, `deps`, `decompose_level`, from
+`forge_jev/features.py`), since task size moves the first-pass rate more than anything
+else in the history.
+
+An attempt is labeled by the last execution that carries its number, because a QA-only
+rerun keeps the number. Refunded and infra-stopped executions spent no attempt. PASS is
+true and FAIL is false. TIMEOUT is false only when the implementer's own dispatch ran out
+of time. Everything else says nothing about the implementer and is left out, counted
+under `censored` in `manifest.json`:
+- a reviewer's TIMEOUT or ERROR
+- a harness crash
+- forge's own failures between dispatches
+- UNKNOWN, INVALIDATED, INTERRUPTED
+
+A later success is never labeled as the first attempt's.
+
+Decisions with no outcome to label them are left out rather than labeled by opinion.
+These are QA effort sizing (recorded until 2026-10-17), the five routing rubrics, failure triage, and coupling, since
+no merge has conflicted yet.
+
+Routing cases of tasks routed with `plan --explore` add `route` to their metadata: the
+candidates, the pool's own pick, the choice, its propensity, and `followed`, which says
+whether attempt 1 really ran the chosen spec (an approval-gate edit or `retry --dwarf` can
+replace it). `assignments.jsonl` lists every such decision with its first attempt's outcome
+or censoring reason, including tasks no case was exported for. It is the randomized cohort,
+free of the export's own filters.
+
+Review cases whose attempt also had a shadow review (`plan --shadow-qa`) add `shadow` (its
+verdict, reviewer and worst finding) and `consensus` to their metadata. Only a shadow that
+finished on an untouched snapshot and reviewed the same commit counts. The primary
+labels never change, because `qa_shadow` dispatches are not reviews of record.
+`./forge dataset status` prints the agreement rate, overall and per model pair.
+
+`meta.jsonl` records each case's slice, run, repo and label source. It also holds hosted
+Jev's own answers where forge recorded them, as baselines that are never part of a case.
+Routing and retry cases add:
+- `decision_provenance` (the plan's) and `outcome_provenance` (the attempt's)
+- `code_changed`
+- the plan `options`
+- `started`
+
+Routing cases also add how the task went:
+- `attempts_observed`
+- `raw_verdict`
+- `gate_pass`, a reviewer FAIL the severity gate accepted
+- `first_raw_pass_attempt`
+- `human_accept`
+- `stop_reason`: `passed`, `accepted`, `noretry`, `interrupted` or `no_pass_observed`
+
+A final FAIL is not proof a task cannot pass: the retry budget ended the observation.
+The output defaults to `<data>/exports/forge-decisions/` (`0700`, files `0600`). It holds
+prompts, diffs and reviews from your repos, so keep it out of public repos and pass it
+only to local models. Builds are deterministic for a given store and `--seed`.
+
+## Routing
+
+`forge dataset routing` answers "which model should this tier use?" from the store, in two
+tables kept apart because they answer different questions.
+
+**Observational.** Every task's first attempt, by declared tier and the model and effort that
+ran it (resolved, so `sol` and `gpt-6.1-sol:medium:codex` share a row):
+- first-pass rate with a Wilson 95% interval
+- the share that passed review at some attempt, or that a human accepted
+- unfinished tasks
+- attempts per task
+- the implementer's median runtime
+
+Routing chose who ran what, so a model sent the hard tasks looks worse here than it is.
+Fractal runs are left out, since their dispatches have identities of their own.
+
+**Randomized.** The `plan --explore` draws, one row per (tier, pool pick, neighbour) pair.
+Round-robin pools make the pair differ between rows, and a model has no chance at all
+outside its own pairs, so pairs are never pooled. For each arm:
+- assigned tasks
+- first-pass as a Hajek (self-normalized inverse-propensity) mean
+- Kish's effective n
+- attrition: tasks not run as assigned, and tasks with no outcome
+- runtime
+
+Every task stays in the arm it was *assigned* (intent to treat), whatever the approval gate
+or a retry did afterwards. A task is bound to the last draw recorded before its first attempt
+started, so a later re-plan never relabels it. Draws from before the two-candidate rule
+(a pick among every spec of the plan) have no pair and are counted, not compared.
+
+**The decision rule** is fixed in advance and only advises. A pair is judged once, at the
+first of 30, 60 or 120 neighbour assignments it reaches, on exactly the assignments up to
+that point. The interval is a run-cluster bootstrap, shown only with at least 10 neighbour
+assignments across 3 runs.
+- For a **low** pair the neighbour is the stronger model: switching is advised only when the
+  interval is above 0.
+- For **medium** and **high** pairs it is the cheaper one: it is called non-inferior when the
+  interval's lower end is above -0.10 and its runtime is lower.
+
+At the default `--explore 0.2` a row changes model 10% of the time, so a pair needs weeks of
+normal use to reach its first window.
+
+## Outcomes after merge
+
+Every other label comes from a reviewer. `forge dataset outcomes` adds one that does not,
+read from the repo's history after the fact. `integrate` merges the run branch with
+`--no-ff` and fast-forwards your branch to that merge, never squashing, so each task's
+attempt commits land as they are and `git blame` still attributes a task's lines to them.
+For every integration, and every task it brought in, the harvester records:
+
+- **Survival:** of the text lines the task added (`base_ref..reviewed_commit`), how many
+  are still attributed to the task's commits at the horizon, per file. A deleted file
+  counts as none surviving. Binary files are skipped, and a task that only deleted lines
+  is `no_additions`.
+- **Reverts:** later commits whose message says `This reverts commit <sha>` for one of the
+  task's commits, its merge or the integration.
+- **The horizon:** the last commit on your branch's first-parent chain, after the
+  integration, dated before `integrate` + N days. The branch is resolved once, and the
+  chain is walked by position, so commit dates out of order cannot skip a commit.
+
+Two cases leave a task without a survival count:
+- **Pending:** the horizon has not been reached and no later commit has passed it yet.
+  Nothing is written, and the next harvest tries again.
+- **Lost:** a rebase, squash or reset took the integration off the branch. This means
+  unobservable, not failed.
+
+The rid names the integration, the reviewed commit and the horizon, so harvesting again
+adds nothing, and two horizons are two outcomes. Survival describes the diff that landed,
+with every retry in it, not the first model the task was routed to. That is why the
+exporter attaches it to that diff's review.
+
+Each verification event also records the commit it verified (`verification.commit`, named
+before any early return), so a FAIL or UNVERIFIED result can be joined to its code too.
+
+## Importing old runs
+
+`import` reads run dirs that predate live capture through the same readers. An execution
+seen both live and imported collapses to one event. Old dirs have no begin-time snapshot,
+so the importer will not guess which execution wrote a mutable file:
+
+- The **newest** task attempt gets the task dir's current files, flagged `legacy`.
+- **Earlier** attempts get only what is immutable:
+  - their own dispatch dirs
+  - the diff QA was shown (the last `` ```diff `` block of the QA prompt)
+  - the status that `attempts.tsv` logged, or failing that, the QA verdict, with
+    `status_source` saying which
+- Unfinished executions (still running, or killed) are skipped. The first write of a rid
+  wins, so importing one now would block the real outcome that live capture records when
+  it ends. Re-import the dir afterwards to fill anything live capture missed.
+- Repos under temp dirs (test runs) are skipped unless `--include-temp` is given.
+- Archives are extracted to a temp dir and mapped back to their original path. The legacy
+  run id hashes that original path, so the archive and the original dir agree on it.
+
+`~/.local/share/forge/legacy/` holds archives of run dirs taken before live capture, with a
+`MANIFEST.tsv`. Passing that dir to `import` reads every archive, plus the ledger copies
+listed in `ledgers/REPOS.tsv`.
+
+## Invariants
+
+- **Never inside the repo.** Capture refuses, silently and before any write, when the store
+  resolves inside the run's repo or a forge worktree. `integrate` refuses a dirty tree.
+- **Never read by a prompt builder.** Memory inject, the capsule and Ripwire must not read
+  the store. A reviewer that could see earlier runs' verdicts would not be independent.
+  `tests/test_dataset.py` enforces that outside its own package, the dataset is only
+  imported for capture.
+- **Read-only in your repos.** Capture and the harvester run only object-store git queries
+  in a repo: `rev-parse`, `rev-list`, `log`, `show`, `ls-tree`, `cat-file`, `diff` between
+  commits and `blame` at a commit. All run with `GIT_OPTIONAL_LOCKS=0`. None takes the
+  working tree or refreshes the index. `outcomes` reads the store through a read-only
+  connection and writes only what it found.
+- **Private.** Prompts, diffs and logs can contain secrets. The dir is `0700` and its files
+  `0600`. Do not sync it. Redaction belongs to whatever exports a training set.
+
+## Not captured
+
+- **Token counts for openclaude, opencode and antigravity.** Those harnesses expose none.
+  They are recorded as null.
+- **Dispatches refused before `begin()`** (an unknown alias, bad flags). No model ran.
+- **The orchestrator's own reasoning.** Only its outputs are recorded: the task table,
+  `request.md` and the approval edits visible between the planner's message and the
+  declared table.
+- **Counterfactuals.** Each task runs on one model, so nothing here says whether another
+  model would have done better. That needs deliberate head-to-head runs.