@giamme/forge 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +56 -0
- package/SKILL.md +22 -3
- package/VERSION +1 -1
- package/docs/artifacts.md +28 -2
- package/docs/commands.md +55 -0
- package/docs/dataset.md +22 -0
- package/docs/decomposed-runs.md +54 -2
- package/docs/forge-ui.md +48 -7
- package/docs/memory.md +4 -2
- package/docs/reviews.md +87 -2
- package/forge +31 -1
- package/package.json +1 -1
- package/references/dataset.md +103 -4
- package/references/decompose.md +41 -5
- package/references/harnesses.md +43 -1
- package/references/ui.md +31 -6
- package/scripts/forge-config.py +350 -0
- package/scripts/forge-control.py +649 -0
- package/scripts/forge-dispatch.sh +235 -121
- package/scripts/forge-issues.py +1134 -0
- package/scripts/forge-lib-plans.sh +6 -2
- package/scripts/forge-lib-review.sh +158 -3
- package/scripts/forge-lib-task.sh +358 -41
- package/scripts/forge-parallel-options.sh +3 -1
- package/scripts/forge-parallel.sh +99 -13
- package/scripts/forge-prompt.py +61 -4
- package/scripts/forge-route.py +245 -28
- package/scripts/forge-runs-rm.py +840 -0
- package/scripts/forge-runs.py +19 -1
- package/scripts/forge-runtime.py +47 -1
- package/scripts/forge-schedule.py +227 -11
- package/scripts/forge-ui.py +85 -19
- package/scripts/forge_dataset/__init__.py +84 -8
- package/scripts/forge_dataset/capture.py +5 -2
- package/scripts/forge_dataset/cli.py +44 -1
- package/scripts/forge_dataset/export.py +36 -3
- package/scripts/forge_dataset/forecast.py +355 -0
- package/scripts/forge_dataset/harvest.py +45 -2
- package/scripts/forge_dataset/importer.py +431 -17
- package/scripts/forge_dataset/reviewers.py +145 -0
- package/scripts/forge_dataset/routing.py +1 -0
- package/scripts/forge_jev/__init__.py +7 -3
- package/scripts/forge_registry.py +183 -26
- package/ui/README.md +41 -7
- package/ui/docs/api.md +259 -21
- package/ui/forge_ui/__main__.py +10 -2
- package/ui/forge_ui/control.py +587 -0
- package/ui/forge_ui/export.py +17 -3
- package/ui/forge_ui/reader.py +210 -12
- package/ui/forge_ui/server.py +125 -4
- package/ui/forge_ui/static/app.js +104 -18
- package/ui/forge_ui/static/cloud3d.js +4 -1
- package/ui/forge_ui/static/cloudmodel.js +1 -1
- package/ui/forge_ui/static/cloudui.js +13 -4
- package/ui/forge_ui/static/controls.js +619 -0
- package/ui/forge_ui/static/findingsview.js +438 -0
- package/ui/forge_ui/static/graph.js +6 -5
- package/ui/forge_ui/static/index.html +7 -1
- package/ui/forge_ui/static/leaderboard.js +18 -0
- package/ui/forge_ui/static/modelstats.js +19 -1
- package/ui/forge_ui/static/panel.js +76 -14
- package/ui/forge_ui/static/rundelete.js +388 -0
- package/ui/forge_ui/static/runmodel.js +128 -13
- package/ui/forge_ui/static/scene3d.js +32 -15
- package/ui/forge_ui/static/settings.js +399 -0
- package/ui/forge_ui/static/style.css +184 -5
- package/ui/forge_ui/static/uikit.js +161 -0
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,62 @@ All notable changes to this project are documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and versions follow the format `{major}.{minor}[.{patch}]` ([...] parts appear only when non-zero).
|
|
7
7
|
|
|
8
|
+
## [0.11] - 2026-10-26
|
|
9
|
+
|
|
10
|
+
Reviews now remember the last round: a retry's review starts from the previous findings, and a repeated whole-run review from the last report. forge dataset can forecast a plan before it runs and grade how often each reviewer's PASS held, and forge-ui shows a superseded final review as OUTDATED instead of FAIL.
|
|
11
|
+
|
|
12
|
+
### Highlights
|
|
13
|
+
|
|
14
|
+
- `forge dataset reviewers`: a report card for reviewer models. For each reviewer, how often work it PASSed was later reverted or mostly rewritten within the outcome horizon (14 days by default), with a 95% interval, or "not enough data" below 10 outcomes. forge-ui's Models view shows it as the Reviewers table's Did not hold column. Advisory only: it never changes routing.
|
|
15
|
+
- `forge dataset forecast <plan>` estimates a plan from past tasks at the same tier and model: median and p90 tokens and time, and the first-try PASS rate with a 95% interval, per task and in total, flagging tasks unlikely to pass first time. `forge-parallel.sh run` prints it before dispatching and records it for later checks (`--no-forecast` turns it off). It is advisory only.
|
|
16
|
+
- On a retry, the per-task reviewer now first checks the previous round's findings, sees the diff between the two attempts and reads the implementer's reply, instead of starting cold. This applies on every harness, so a finding that was fixed is confirmed as fixed and one that was not is raised again.
|
|
17
|
+
- A final review that later merges superseded now shows as OUTDATED in the forge-ui run view and on the cloud map, not as the run's verdict, and tasks merged with `accept` show as accepted instead of failed. `forge issues` and the Open-findings view mark the findings of an outdated review, and accepted findings get their own origin and issue text.
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
|
|
21
|
+
- `plan --reviewer-session` is an opt-in experiment that continues the reviewer's own session across rounds on claude, openclaude and codex, and falls back to a fresh session whenever it cannot. Those reviews are tagged in the dataset and kept out of the reviewer statistics, so the experiment does not skew the independence numbers.
|
|
22
|
+
|
|
23
|
+
### Changed
|
|
24
|
+
|
|
25
|
+
- A mistyped command now says which one you most likely meant, for example `forge rnus` answers "Did you mean `forge runs`?" above the usage.
|
|
26
|
+
- A repeated whole-run review now carries the previous final review's report and the diff since it, so it checks what the earlier review found rather than reviewing the run from scratch.
|
|
27
|
+
|
|
28
|
+
### Fixed
|
|
29
|
+
|
|
30
|
+
- forge-ui started directly (`python3 -m forge_ui`) now finds forge's run registry the way forge writes it: an `XDG_CONFIG_HOME` that starts with `~` lists your runs instead of none.
|
|
31
|
+
- A grok dwarf or reviewer that runs its tests in the background no longer ends as an error after 10 minutes: grok now waits for that work as long as forge's own time limit allows, and `forge-dispatch.sh doctor` checks that the installed grok supports this.
|
|
32
|
+
- A relative `XDG_CONFIG_HOME` now counts as unset, as the XDG spec says, so forge registers runs and keeps `jev.json` in `~/.config` instead of recording nothing. Before, `forge ui open` printed a link to a run that forge-ui then did not list.
|
|
33
|
+
- The dataset now imports the traces from the archive of a run whose directory name starts with `review-`; before, nothing from it was imported.
|
|
34
|
+
- An escalation pin, or any escalation, no longer writes through a symlinked `escalations.tsv`.
|
|
35
|
+
- The run summary, and `escalation.next` in the router's `--check`, no longer say a pin waits for a QA-only re-run when the next retry will really be a full attempt.
|
|
36
|
+
- A relative `XDG_DATA_HOME` now counts as unset, as the XDG spec says, so the run dataset, run archives and exports stay in `~/.local/share/forge` instead of a folder inside whichever repo forge ran from. `forge config show` reports the same place.
|
|
37
|
+
|
|
38
|
+
## [0.10] - 2026-10-23
|
|
39
|
+
|
|
40
|
+
forge-ui can now change a run: live QA threshold, retries and timeout, a one-shot model pin, open findings filed as GitHub issues, deleting runs while the dataset keeps them, and every config source in one view. Each control is also a forge CLI: forge control, forge runs rm, forge config and new forge issues modes.
|
|
41
|
+
|
|
42
|
+
### Highlights
|
|
43
|
+
|
|
44
|
+
- After a run, review findings nobody would act on can become GitHub issues: forge offers once to file the findings `--qa-threshold` tolerated, the PLAUSIBLE ones and whatever a whole-run review left open. Where the gh-proman skill is set up for the repository they go on the project board through it; anywhere else forge files them itself with plain `gh` (`forge issues <plan-dir>`). A finding is never filed twice. `--file-issues auto` files them without asking, `--file-issues off` never offers.
|
|
45
|
+
- `forge control show|set|pin|unpin <plan-dir>` changes a plan while it runs: the QA threshold (read once per attempt, per task too, and `none` exempts a task), the timeout, `--escalate-after`, verify retries, and, while a scheduler is live, the retry and infrastructure-retry budgets. The budgets keep their per-invocation rule: a live scheduler publishes them for the length of the run and re-reads them at every decision. Every change is validated, logged in `control.log` and recorded in the dataset as a `control` event.
|
|
46
|
+
- forge-ui has an Open findings view: the findings a run, or every run of its repository, left open, with their filed state. Select some to file as GitHub issues or dismiss ones that need no work. `forge issues` gains `--offline`, `--only`, `--all-runs`, `--dismiss`/`--undismiss` (with `--reason`), `--include-dismissed` and a `findings` list in its JSON; the whole-run review's findings are now collected, and filing and dismissal take a repository lock (exit 3 when busy).
|
|
47
|
+
- `forge runs rm <run-dir>` deletes a run while the dataset keeps what it taught: a strict import, a self-describing archive (`FORGE_ARCHIVE.json`), and `run_delete_started` and `run_deleted` events, all under the run's own lock. `--worktrees` and `--branches` also remove the run's clean worktrees and its unchecked-out branches (foreign and dirty worktrees and UNMERGED branches are reported); `--dry-run` previews. forge-ui has a matching delete dialog. The dataset also imports `escalations.tsv` and `control.log`.
|
|
48
|
+
- forge-ui is no longer read-only: it gains write endpoints and Controls panels for a run and for each task (QA threshold, timeout, escalate-after, verify retries, budgets, threshold override, retry budget, model pin). It only ever calls forge's own CLIs, and every change must carry a per-launch token and pass the Host and Origin checks. `--read-only` (or `FORGE_UI_READONLY=1`) turns writing off, and `forge ui open` never reuses a server running in the other mode.
|
|
49
|
+
|
|
50
|
+
### Added
|
|
51
|
+
|
|
52
|
+
- `forge control pin <plan-dir> <task> <spec>` sets the model a task's next full retry uses, once: the spec is checked with `forge-dispatch.sh doctor` (no quota), applied at the retry and written to `tasks.tsv`, and `escalations.tsv` gains a `source` column (`policy`, `retry` or `pin`) saying who decided. `forge control unpin` drops it; Fractal plans never apply pins.
|
|
53
|
+
- `forge config show|set|unset` shows every config source in one view (the registry, Jev, a repository's `.forge/verify`, `setup` and `verify-subset`, the dataset and `FORGE_*` variables with secrets hidden) and edits the per-repo files and `jev.enabled`. forge-ui shows the same view as Settings.
|
|
54
|
+
|
|
55
|
+
### Changed
|
|
56
|
+
|
|
57
|
+
- A repeated `forge ui open` for a run that already has a browser tab no longer looks like it did nothing: it prints the link as before and says on stderr that no new tab was opened, so you know to use the link.
|
|
58
|
+
|
|
59
|
+
### Fixed
|
|
60
|
+
|
|
61
|
+
- `forge ui open DIR` now reads only the first line of the run's `repo` file, as the runners do, so a run whose `repo` file has extra lines is listed under its real repo instead of under a path that does not exist.
|
|
62
|
+
- A forge-ui export no longer leaks a local path spelled another way: the run, repository and home paths are also scrubbed in their real form (on macOS, `/private/var/...` and `/var/...`).
|
|
63
|
+
|
|
8
64
|
## [0.9] - 2026-10-21
|
|
9
65
|
|
|
10
66
|
forge-ui has a new look and layout, a 3D stage that marks every review finding on its task, and each dwarf's and reviewer's conversation, which runs on claude, openclaude and opencode now keep in full. Also new: `forge runs` and a first screen that says how to get runs; npm installations now hear about new releases.
|
package/SKILL.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: forge
|
|
3
3
|
description: Dispatch implementation and independent diff review to user-selected models through local agent CLIs. Use for /forge or explicit requests to delegate coding or review; explanation-only questions do not authorize dispatch.
|
|
4
|
-
argument-hint: '"<goal>" --dwarf <alias>[:effort[:harness]] [--qa <alias>] [--planner <alias>] [--yolo-dwarf] [--yolo-qa] [--decompose-level low|medium|high] [--fractal|--no-fractal|--fractal-auto-decompose] [--fractal-planner <spec>] [--jev|--no-jev] [--jev-shadow] [--no-memory] [--no-ripwire] [--ui|--no-ui] [--timeout <seconds>] [--retry-failed <n>] [--escalate-after <n>] [--qa-threshold P0|P1|P2|P3] [--shadow-qa <spec>[,<spec>]] [--shadow-rate <0..1>] [--explore <0..0.5>] [--fail-first]'
|
|
4
|
+
argument-hint: '"<goal>" --dwarf <alias>[:effort[:harness]] [--qa <alias>] [--planner <alias>] [--yolo-dwarf] [--yolo-qa] [--decompose-level low|medium|high] [--fractal|--no-fractal|--fractal-auto-decompose] [--fractal-planner <spec>] [--jev|--no-jev] [--jev-shadow] [--no-memory] [--no-ripwire] [--ui|--no-ui] [--timeout <seconds>] [--retry-failed <n>] [--escalate-after <n>] [--qa-threshold P0|P1|P2|P3] [--file-issues ask|auto|off] [--shadow-qa <spec>[,<spec>]] [--shadow-rate <0..1>] [--reviewer-session] [--explore <0..0.5>] [--fail-first]'
|
|
5
5
|
allowed-tools: [Bash, Read]
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -43,8 +43,9 @@ For explanation requests, explain the workflow without spending model quota.
|
|
|
43
43
|
separate authorization and `integrate --approved`. Never push automatically.
|
|
44
44
|
- Show the decomposed task/model table and wait for approval before dispatch. Retry only
|
|
45
45
|
when requested; preserve failed task branches and worktrees. `run --retry-failed N`,
|
|
46
|
-
`--infra-retries M`, `--qa-threshold Pn
|
|
47
|
-
sampled share of attempts, never read, to measure reviewer agreement)
|
|
46
|
+
`--infra-retries M`, `--qa-threshold Pn`, `plan --shadow-qa` (a second review of a
|
|
47
|
+
sampled share of attempts, never read, to measure reviewer agreement) and
|
|
48
|
+
`plan --reviewer-session` (continue the reviewer's session across rounds) are opt-in: offer
|
|
48
49
|
them, never add them unprompted. So is `--fail-first` (solo or `plan`): the change's new
|
|
49
50
|
tests also run without its non-test changes, and the reviewer is asked to report any that
|
|
50
51
|
still pass as P2; it needs `.forge/verify-subset` and never changes a result by itself. `plan` explores by default (`--explore 0.2`: now and then
|
|
@@ -55,10 +56,28 @@ For explanation requests, explain the workflow without spending model quota.
|
|
|
55
56
|
(`--escalate-after N`, `0` = off); an explicit `retry --dwarf` overrides it. `accept`
|
|
56
57
|
overrides a reviewer, so run it (with the user's `--reason` and
|
|
57
58
|
`--approved`) only on the user's explicit instruction; never self-accept.
|
|
59
|
+
- When you report a run, read `<plan>/control.log` and mention any human change made mid-run
|
|
60
|
+
(a threshold, a budget, a model pin; `forge control show <plan>` lists them). Never set a
|
|
61
|
+
control, pin or delete a run with `forge control` / `forge runs rm` unless the user asks.
|
|
58
62
|
- Exit `8` / status `INFRA` is a quota, auth or network stop, not a task failure and no
|
|
59
63
|
attempt was spent: report it, say what to fix, and re-run only when told. For a multi-task
|
|
60
64
|
plan, offer the opt-in whole-run review (`review`, or `integrate --final-review <qa-spec>`);
|
|
61
65
|
it spends QA quota, so never run it unprompted.
|
|
66
|
+
- **Leftover findings become issues.** A run can finish with review findings nobody will act on:
|
|
67
|
+
- CONFIRMED findings that `--qa-threshold` tolerated (`tasks/<id>/known_issues.md`);
|
|
68
|
+
- PLAUSIBLE findings in a passed task's `qa.last`;
|
|
69
|
+
- whatever a whole-run review left open.
|
|
70
|
+
|
|
71
|
+
When there are any, act on the user's `--file-issues` choice with one filer:
|
|
72
|
+
- **gh-proman**, only when it is installed and set up for the repository: `gh-proman -C <repo> config show` exits 0. It also puts the issues on the team's board with a priority. Run `gh-proman findings --from <plan or run dir> --include-plausible --json`, then follow the gh-proman skill's review-findings steps.
|
|
73
|
+
- **forge's own filer** otherwise (gh-proman missing, or not configured for this repo; never configure it yourself): `python3 <skill_dir>/scripts/forge-issues.py <plan or run dir> --include-plausible --json`, plain `gh issue create`. Pass `--lang` matching the repo's existing issues, and `--label bug` when the repo has that label. If `gh` is missing, unauthenticated or the repo has no GitHub remote, list the findings in the report and say why none were filed.
|
|
74
|
+
|
|
75
|
+
The choice:
|
|
76
|
+
- `ask` (the default): show the dry-run summary (count by severity, titles, anything already filed) and ask once: **"File these N findings as GitHub issues? [all / CONFIRMED only / none]"**. Then re-run with `--apply`, adding `--include-plausible` only for "all".
|
|
77
|
+
- `auto`: file them without asking (`--apply`, PLAUSIBLE included as "to verify" issues) and report what was created and skipped.
|
|
78
|
+
- `off`: never offer.
|
|
79
|
+
|
|
80
|
+
`--file-issues` is an instruction to you: never forward it to a runner. Skip INFRA, interrupted and still-failing runs. Both filers mark each issue with the same finding fingerprint and skip a finding either one already filed, so offering again after a retry only brings up new ones. A draft flagged as a possible duplicate is skipped on `--apply`: show it, and file it (`--confirm-new`) only if the user says it is different.
|
|
62
81
|
- Jev is optional, off unless configured and enabled, and always advisory: it never
|
|
63
82
|
changes a task's tier, never selects a model, and never substitutes for QA acceptance
|
|
64
83
|
or repository verification (`--jev-act` is still accepted and changes nothing). Read
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.
|
|
1
|
+
0.11
|
package/docs/artifacts.md
CHANGED
|
@@ -17,6 +17,8 @@ as if the dwarf had written it.
|
|
|
17
17
|
<role>.last the final message — read this first
|
|
18
18
|
<role>.infra only after an infrastructure failure: class, retry_after, detail
|
|
19
19
|
<role>.orphans only when processes were left running after the harness exited
|
|
20
|
+
<role>.session session id reported by claude, openclaude or codex; absent when the dispatch published none
|
|
21
|
+
<role>.continue only when the dispatch was asked to continue a reviewer session: requested=, mode=continued|failed, reason=
|
|
20
22
|
changes.diff the dwarf's real diff, which is qa's input
|
|
21
23
|
```
|
|
22
24
|
|
|
@@ -48,9 +50,25 @@ A decomposed run adds:
|
|
|
48
50
|
tasks/<id>/known_issues.md, qa.gate findings tolerated by --qa-threshold, and the gate's decision
|
|
49
51
|
tasks/<id>/guard.txt what forge observed about the implementer (self-commit, orphans, promises)
|
|
50
52
|
tasks/<id>/infra.txt, resume, noretry infrastructure stop record; QA-only resume marker; no-progress marker
|
|
51
|
-
tasks/<id>/qa_threshold optional per-task blocking threshold
|
|
53
|
+
tasks/<id>/qa_threshold optional per-task blocking threshold: P0-P3, or none (exempt from the plan's)
|
|
54
|
+
tasks/<id>/escalations.tsv epoch, attempt, from-spec, to-spec, source (policy|retry|pin); older rows have 4 columns
|
|
55
|
+
tasks/<id>/escalation.pin a human's one-shot model for the next full retry: one dwarf spec on one line
|
|
56
|
+
tasks/<id>/escalation.pin.applying the pin while a retry has claimed it; removed once the retry is recorded
|
|
57
|
+
tasks/<id>/retry_findings.commit commit the carried review judged
|
|
58
|
+
tasks/<id>/retry_findings.session session id of that review, when it recorded one
|
|
59
|
+
tasks/<id>/retry_findings.resolved that review's resolution (harness=, model=, effort=)
|
|
60
|
+
tasks/<id>/changes.since.diff diff from that commit to the one under review; omitted when there is no such range
|
|
61
|
+
tasks/<id>/changes.since.range the two commits, full hashes
|
|
62
|
+
tasks/<id>/changes.since.stat diff --stat of that range
|
|
63
|
+
tasks/<id>/review_sessions.tsv one row per review round while plan --reviewer-session is on: round, attempt, mode, reason, session id
|
|
64
|
+
tasks/<id>/retry_budget, retries.used live per-task retry budget (a number); retries spent (kept after the run)
|
|
52
65
|
results.tsv id status branch
|
|
53
|
-
timeout, qa_threshold, verify_retries options remembered from run/retry flags
|
|
66
|
+
timeout, qa_threshold, verify_retries options remembered from run/retry flags (also escalate_after)
|
|
67
|
+
reviewer_session holds on; present while plan --reviewer-session is on
|
|
68
|
+
<final-review-dir>/review.meta final-review-<epoch>-<pid>/ or integrate-*/final-review/; base=, tip=, source=, spec=, threshold=, integration=, previous=, since=; session= and session_reason= while reviewer_session is set
|
|
69
|
+
control.log one JSON line per human change made with forge control
|
|
70
|
+
scheduler.live, control.lock only while a scheduler runs: {"pid", "started"}; the lock serialises forge control
|
|
71
|
+
retry_budget, infra_budget only while a scheduler runs: its live budgets (a number each)
|
|
54
72
|
integration.base commit the integration branch started from
|
|
55
73
|
verification.{status,exit,log,command,fingerprint} the combined verification
|
|
56
74
|
verification.{flaky,retry.log} only when a failing command passed on rerun
|
|
@@ -65,5 +83,13 @@ forge/<run-id>/<task-id> task branch
|
|
|
65
83
|
forge/<run-id>-integration integration branch
|
|
66
84
|
```
|
|
67
85
|
|
|
86
|
+
`control.log` lines look like
|
|
87
|
+
`{"at": 1761200000.5, "actor": "ui", "action": "set", "task": null, "key": "qa-threshold", "value": "P2", "previous": "P1"}`
|
|
88
|
+
(`actor` is `cli` or `ui`; `action` is `set`, `pin` or `unpin`; `key` is the control's name, or `pin`; `task` is null for a plan-wide
|
|
89
|
+
control). `scheduler.live`, `retry_budget`, `infra_budget` and the per-task `retry_budget` exist
|
|
90
|
+
only while the scheduler is live: it publishes the flags it was given when it starts, re-reads them
|
|
91
|
+
at every decision and removes them when it stops, which is why `forge control` refuses a budget
|
|
92
|
+
change outside a live run. See [live controls](decomposed-runs.md#changing-a-run-while-it-works).
|
|
93
|
+
|
|
68
94
|
The worktree root is a **sibling of the repo, never `TMPDIR`**. Isolated worktrees are never
|
|
69
95
|
pushed, so a temp root that gets cleaned takes the only copy of that work with it.
|
package/docs/commands.md
CHANGED
|
@@ -16,11 +16,65 @@ harness (`/forge`); this command installs forge and drives its optional tools.
|
|
|
16
16
|
| `forge version [--check]` | the installed version, and whether a newer release is out |
|
|
17
17
|
| `forge fractal …` | opt-in Fractal execution; see [the command group](fractal.md#forge-fractal-command-group) |
|
|
18
18
|
| `forge runs [--repo DIR \| --all] [--limit N] [--json]` | this repo's recent runs, newest first: run id, kind, age, task states, verification and final review, goal, and the run dir. Read-only; the same summaries forge-ui shows |
|
|
19
|
+
| `forge control show\|set\|pin\|unpin PLAN_DIR …` | change a live plan: `set` takes `qa-threshold`, `timeout`, `escalate-after`, `verify-retries`, `retry-budget` or `infra-retries` (the last two only while a scheduler runs; `--task ID` for `qa-threshold` and `retry-budget`, value `inherit` drops the override); `pin PLAN_DIR TASK SPEC` sets the model the task's next full retry uses once, `unpin` drops it; `--json`. Every change is logged in `control.log`. See [live controls](decomposed-runs.md#changing-a-run-while-it-works) |
|
|
20
|
+
| `forge config show\|set\|unset [--repo DIR] [--json]` | one view of every config source (registry, Jev, the repo's `.forge/verify`, `setup` and `verify-subset`, dataset, `FORGE_*` variables with secrets hidden); `set KEY VALUE` / `unset KEY` edit `repo.verify`, `repo.setup`, `repo.verify-subset` (must contain `{files}`) and, for `set`, `jev.enabled true\|false`. `-` as a value reads stdin. Exit 2 on invalid input |
|
|
21
|
+
| `forge runs rm RUN_DIR [--dry-run] [--yes] [--no-archive] [--worktrees] [--branches] [--json]` | delete a run directory but keep what it taught the dataset; see [below](#forge-runs-rm) |
|
|
22
|
+
| `forge issues RUN [--apply] …` | file the review findings a run left open as GitHub issues; see [below](#forge-issues) |
|
|
19
23
|
| `forge ui open\|status\|stop` | forge-ui, the run viewer; see [the command group](forge-ui.md#forge-ui-command-group) |
|
|
20
24
|
| `forge dataset …` | every run's inputs and outcomes; see [run dataset](dataset.md) |
|
|
21
25
|
| `forge jev …` | optional Jev advice: setup, status, doctor, enable, disable, backtest; see [Jev](add-ons.md#jev) |
|
|
22
26
|
| `forge --help` | list the commands; `forge <command> --help` shows each one's options |
|
|
23
27
|
|
|
28
|
+
### `forge runs rm`
|
|
29
|
+
|
|
30
|
+
Deletes a run directory (a plan dir or a solo run dir) without losing its traces. It holds the
|
|
31
|
+
run's own lock from the first step to the last (`schedule.lock` of a plan, `pipeline.lock` of a
|
|
32
|
+
solo run), so a run started meanwhile fails to start instead of racing the delete. In order:
|
|
33
|
+
|
|
34
|
+
1. **preserve**: a strict dataset import of the run; every event must be stored, or nothing is deleted;
|
|
35
|
+
2. **archive**: a self-describing `.tar.gz` under `<data home>/archive/` (`--no-archive` skips it);
|
|
36
|
+
3. **intent**: the dataset event `run_delete_started`;
|
|
37
|
+
4. `--worktrees` removes the run's clean worktrees; `--branches` runs `git branch -D` on its
|
|
38
|
+
branches that no worktree has checked out;
|
|
39
|
+
5. the run directory is removed, `run_deleted` is recorded and the run leaves the registry.
|
|
40
|
+
|
|
41
|
+
What counts as the run's is decided from the run id, not read from the run directory: a worktree
|
|
42
|
+
under `<repo>/../.forge-worktrees/<run id>/` on `forge/<run id>/<task>` or
|
|
43
|
+
`forge/<run id>-integration`. Any other worktree there is `foreign`, one with uncommitted changes
|
|
44
|
+
is `dirty`; neither is touched, and a dirty one keeps the run directory too. Branches are reported
|
|
45
|
+
`merged` or `UNMERGED`, so you see what deleting them would lose. `--dry-run` changes nothing and
|
|
46
|
+
reports; `--yes` skips the prompt for the run id (needed without a terminal); `--json` prints one
|
|
47
|
+
object, on failure too. Exit 0 ok, 2 refused input, 3 busy or the run is active, 4 failed during
|
|
48
|
+
deletion or the run directory was kept (the same command finishes the job).
|
|
49
|
+
|
|
50
|
+
### `forge issues`
|
|
51
|
+
|
|
52
|
+
`forge issues <plan dir, task dir, review dir, known_issues.md or qa.last>` drafts one GitHub
|
|
53
|
+
issue per finding the run left open (`--group task|run` for fewer) and files them only with
|
|
54
|
+
`--apply`. The findings are the CONFIRMED ones `--qa-threshold` tolerated (origin
|
|
55
|
+
`tolerated`), the CONFIRMED ones a person accepted with `accept` (origin `accepted`, with
|
|
56
|
+
their own issue text: a person accepted the work over the reviewer's verdict), PLAUSIBLE
|
|
57
|
+
ones with `--include-plausible`, and what the whole-run review left open. A whole-run
|
|
58
|
+
review the integration branch has moved past is outdated: the issue says so and names the
|
|
59
|
+
tasks that merged after it. Each issue carries a hidden
|
|
60
|
+
fingerprint, so a finding is never filed twice, by this command or by gh-proman. Also:
|
|
61
|
+
`--min-severity`, `--repo`, `--lang`, `--label`, `--confirm-new`, `--max-issues` (default 10; pass
|
|
62
|
+
the size of a `--only` selection).
|
|
63
|
+
|
|
64
|
+
| Flag | Does |
|
|
65
|
+
|---|---|
|
|
66
|
+
| `--json` | machine-readable output; `findings` lists each one with its fingerprint, severity, origin (`tolerated`, `accepted`, `review` or `plausible`), task, `review`, `filed` state and `dismissed` mark. A whole-run finding's `review` is `{name, outdated, merged_since}` (`outdated` is null when the review has no verdict or the plan has no accepted integration commit; `merged_since` is empty unless the review is outdated). Every other finding has `review` null |
|
|
67
|
+
| `--offline` | never run `gh`: no filed state, commit publication judged from local refs |
|
|
68
|
+
| `--only FP` | keep only these 12-hex fingerprints (repeatable or comma-separated) |
|
|
69
|
+
| `--all-runs` | read `RUN` as a repository (or a plan dir naming one) and collect every run in its registry; the newest plan wins for a finding in several |
|
|
70
|
+
| `--dismiss FP`, `--undismiss FP`, `--reason TEXT` | mark findings as needing no work (or take it back), then exit; no network. The marks live outside the repo, in `${XDG_STATE_HOME:-~/.local/state}/forge/findings/<repo key>.json` |
|
|
71
|
+
| `--include-dismissed` | draft dismissed findings too |
|
|
72
|
+
|
|
73
|
+
Two filers for one repository at once would both see a finding as unfiled, so `--apply` takes a
|
|
74
|
+
repository lock from the issue lookup to the last `gh issue create`, and each dismissal takes it
|
|
75
|
+
around its write. A busy lock is retried for 10 s, then exit 3. Exit 0 ok, 2 usage or a `gh` or
|
|
76
|
+
dismissal-store error, 3 busy.
|
|
77
|
+
|
|
24
78
|
## The `/forge` slash command
|
|
25
79
|
|
|
26
80
|
```
|
|
@@ -270,6 +324,7 @@ Bash 3.2 entrypoints use a Python standard-library coordinator to schedule eligi
|
|
|
270
324
|
```
|
|
271
325
|
forge-parallel.sh plan <plan-dir> --repo <dir> [routing flags] [--planner <spec>] [--no-memory]
|
|
272
326
|
[--decompose-level L] [--shadow-qa <spec>[,..]|none] [--shadow-rate R]
|
|
327
|
+
[--reviewer-session | --no-reviewer-session]
|
|
273
328
|
[--explore E] [--ui | --no-ui] [--fail-first | --no-fail-first]
|
|
274
329
|
forge-parallel.sh run <plan-dir> [--max-parallel N] [--yolo-dwarf] [--yolo-qa] [--dry-run]
|
|
275
330
|
[--verify <cmd>] [--setup <cmd>] [--timeout S]
|
package/docs/dataset.md
CHANGED
|
@@ -14,7 +14,29 @@ forge dataset status # size, counts, and anything that looks incomple
|
|
|
14
14
|
forge dataset import ~/.local/share/forge/legacy # runs that predate live capture
|
|
15
15
|
forge dataset export # labeled decision cases for benchmarks and fine-tuning
|
|
16
16
|
forge dataset outcomes # which integrated lines survived, read from git history
|
|
17
|
+
forge dataset reviewers # how often work each reviewer PASSed was later reverted or rewritten
|
|
18
|
+
forge dataset forecast <plan> # likely tokens, time and first-try PASS rate for a plan's tasks
|
|
17
19
|
```
|
|
18
20
|
|
|
21
|
+
`forge dataset reviewers` is a report card for reviewer models, built on those outcomes. It is
|
|
22
|
+
advisory only and never changes routing, and it shows "not enough data" below 10 outcomes per
|
|
23
|
+
reviewer rather than a rate a handful of tasks would make up. See
|
|
24
|
+
[the reviewer report card](../references/dataset.md#reviewer-report-card).
|
|
25
|
+
|
|
26
|
+
`forge dataset forecast <plan>` estimates a plan from past tasks at the same tier and model:
|
|
27
|
+
median and p90 tokens and time, and the first-try PASS rate with a 95% interval, per task and in
|
|
28
|
+
total. It flags tasks below 50% and says "not enough history" under 5 past tasks. `run` prints
|
|
29
|
+
the same block before it dispatches (`--no-forecast` turns it off) and records it, so its accuracy
|
|
30
|
+
can be checked later. It is advisory only. See [plan forecast](../references/dataset.md#plan-forecast).
|
|
31
|
+
|
|
32
|
+
Deleting a run keeps what it taught the dataset. `forge runs rm <run-dir>` first imports the run
|
|
33
|
+
**strictly** (every event must be stored, or nothing is deleted), then writes a self-describing
|
|
34
|
+
archive under `<data home>/archive/`: a `.tar.gz` whose `FORGE_ARCHIVE.json` records the run
|
|
35
|
+
dir's original path, run id and dataset uid, so `forge dataset import` maps it back to the same
|
|
36
|
+
run later. It records `run_delete_started` and `run_deleted` events around the removal. The same
|
|
37
|
+
import also reads a plan's `escalations.tsv` (with who decided: `policy`, `retry` or `pin`) and
|
|
38
|
+
`control.log`, so human interventions made with `forge control` are in the dataset as
|
|
39
|
+
`control` events. See [forge runs rm](commands.md#forge-runs-rm).
|
|
40
|
+
|
|
19
41
|
Capture is fail-open and never read by anything that builds a prompt. See
|
|
20
42
|
[references/dataset.md](../references/dataset.md) for the event schema and its invariants.
|
package/docs/decomposed-runs.md
CHANGED
|
@@ -99,8 +99,9 @@ that baseline, and QA sees only baseline membership recorded for that task.
|
|
|
99
99
|
## Fresh session per task, plus a capsule
|
|
100
100
|
|
|
101
101
|
Every dispatch is a **clean slate** — forge never passes `--continue`, `--resume`, or
|
|
102
|
-
`codex exec resume
|
|
103
|
-
|
|
102
|
+
`codex exec resume`, except the opt-in `plan --reviewer-session` experiment (it continues
|
|
103
|
+
a reviewer's session across rounds on claude, openclaude and codex). One task's confused
|
|
104
|
+
turn can't poison another, and each task's context stays small.
|
|
104
105
|
|
|
105
106
|
Clean slates alone would leave every dwarf blind to the wider run, so a short `capsule.md` is
|
|
106
107
|
generated from the task's pinned baseline and preserved for both implementation and QA:
|
|
@@ -181,6 +182,10 @@ bash scripts/forge-parallel.sh run "$PLAN" --max-parallel 4 --yolo-dwarf
|
|
|
181
182
|
bash scripts/forge-parallel.sh integrate "$PLAN" --approved
|
|
182
183
|
```
|
|
183
184
|
|
|
185
|
+
`run` starts by printing a forecast for the plan from the dataset: likely tokens, time and
|
|
186
|
+
first-try PASS rate per task, with tasks unlikely to pass first time flagged. It is advisory only,
|
|
187
|
+
and `--no-forecast` turns it off. See [plan forecast](../references/dataset.md#plan-forecast).
|
|
188
|
+
|
|
184
189
|
`integrate` refuses without `--approved`, and refuses on a dirty working tree.
|
|
185
190
|
For Fractal execution, pass tier pools to `plan` and backend/limit flags to `run`.
|
|
186
191
|
`--max-parallel` limits task pipelines; the shared `--fractal-concurrency` limit includes
|
|
@@ -200,6 +205,53 @@ and pipeline checkpoints. A task/subtree selector resumes only that execution sc
|
|
|
200
205
|
Explicit `retry` retains the backend and frozen pools; a replacement dwarf must already
|
|
201
206
|
belong to that pool. See [recovery](../references/fractal.md#limits-and-lifecycle).
|
|
202
207
|
|
|
208
|
+
## Changing a run while it works
|
|
209
|
+
|
|
210
|
+
`forge control` changes a plan's knobs without stopping it (forge-ui calls the same command):
|
|
211
|
+
|
|
212
|
+
```bash
|
|
213
|
+
forge control show <plan-dir> [--json] # every control, per-task overrides, pins, recent changes
|
|
214
|
+
forge control set <plan-dir> qa-threshold P2 # also --task <id>; P0-P3, none, or inherit (task only)
|
|
215
|
+
forge control set <plan-dir> retry-budget 3 [--task <id>] # automatic retries; live scheduler only
|
|
216
|
+
forge control set <plan-dir> infra-retries 2 # infrastructure waits; live scheduler only
|
|
217
|
+
forge control set <plan-dir> timeout|escalate-after|verify-retries <n>
|
|
218
|
+
forge control pin <plan-dir> <task> sol:high:codex # the model the next full retry uses, once
|
|
219
|
+
forge control unpin <plan-dir> <task>
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
Each change is validated, written under `control.lock`, and says when it applies:
|
|
223
|
+
|
|
224
|
+
- **The QA threshold is read once per attempt**, at the start of the task's next attempt; the
|
|
225
|
+
attempt already running keeps the one it began with, so a verdict is never judged by a bar that
|
|
226
|
+
moved under it. A per-task `none` exempts that task from the plan's threshold; `inherit` drops
|
|
227
|
+
the override. A plan-level `none` clears the threshold.
|
|
228
|
+
- **Budgets stay "per invocation".** `run --retry-failed N` and `--infra-retries M` still apply
|
|
229
|
+
only to the invocation that was given them and are not remembered; the next `run` starts
|
|
230
|
+
from its own flags. A rule that came back from an old file would silently spend money nobody
|
|
231
|
+
asked for. While a scheduler is live it mirrors them into the plan (`retry_budget`,
|
|
232
|
+
`infra_budget`, `tasks/<id>/retry_budget`) and re-reads them at every decision, which is what
|
|
233
|
+
`forge control` edits. When the scheduler stops it removes them; so a budget can only be changed
|
|
234
|
+
while a run is live (exit 3 otherwise), and the change dies with the run.
|
|
235
|
+
`tasks/<id>/retries.used` counts the retries spent and stays after the run for forge-ui.
|
|
236
|
+
- **A pin is one-shot.** `pin` makes the task's next *full* retry use that spec (checked with
|
|
237
|
+
`forge-dispatch.sh doctor`, which spends no quota). A QA-only resume spends no implementer
|
|
238
|
+
attempt and leaves the pin for later. The retry applies it, writes it into `tasks.tsv`, records
|
|
239
|
+
an `escalations.tsv` row with `source` `pin`, clears any `split.suggested` stop and consumes
|
|
240
|
+
it. That row is never written through a symlink: if `escalations.tsv`, or any directory
|
|
241
|
+
from the plan down to it, is one, the pin stays and nothing is retried. The run summary
|
|
242
|
+
says a pin waits (`pinned; after the QA-only re-run`) only when the next entry really
|
|
243
|
+
re-runs QA: a `resume` marker, or an unreviewed verdict, that still matches the
|
|
244
|
+
worktree. A marker the next retry will discard is reported as `pinned`.
|
|
245
|
+
Fractal plans never apply pins, and a MERGED task refuses one.
|
|
246
|
+
- **Who decided an escalation.** The last column of `tasks/<id>/escalations.tsv` is `policy`
|
|
247
|
+
(`--escalate-after`), `retry` (a typed `retry --dwarf`) or `pin`; rows from before it was added
|
|
248
|
+
have four columns and read as `policy`.
|
|
249
|
+
|
|
250
|
+
Every change is appended to `control.log` (one JSON line: `at`, `actor` `cli` or `ui`, `action`,
|
|
251
|
+
`task`, `key`, `value`, `previous`) and recorded in the dataset as a `control` event, so a run's
|
|
252
|
+
human interventions are on the record next to what the policy did. Setting a control to the
|
|
253
|
+
value it already has changes and logs nothing.
|
|
254
|
+
|
|
203
255
|
## Splitting and combining plans
|
|
204
256
|
|
|
205
257
|
`run --retry-failed N` removes most of the reason to split a plan (a failed task no longer holds
|
package/docs/forge-ui.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
[← README](../README.md) · [All docs](README.md)
|
|
4
4
|
|
|
5
|
-
forge-ui is a small
|
|
5
|
+
forge-ui is a small local web page that shows runs live and, when you ask, changes them, as a 3D isometric map
|
|
6
6
|
ported from [archivision](https://github.com/Giamme/archivision). Its **run cloud** puts every
|
|
7
7
|
run in one full-screen view, grouped by project, in five layouts (helix, sphere, galaxy,
|
|
8
8
|
timeline, constellation), filtered down to exactly the runs you want; drag any run anywhere
|
|
@@ -32,7 +32,10 @@ and the cloud remembers it.
|
|
|
32
32
|
|
|
33
33
|
The page reads like a signal box's desk: engraved plates for headings, push buttons with lamps
|
|
34
34
|
for switches, and lamps that keep one meaning everywhere (amber running, green merged, red failed,
|
|
35
|
-
unlit pending).
|
|
35
|
+
unlit pending). Tasks merged with `accept` show as accepted (hollow green) rather than failed.
|
|
36
|
+
A final review superseded by tasks merged into the integration branch shows as OUTDATED
|
|
37
|
+
(hollow unlit) instead of as the run's verdict; changes on the user's own branch after an
|
|
38
|
+
integrate are not tracked. Two layouts share it. *Immersive*, the default, floats the runs, the inspector and the tasks
|
|
36
39
|
over a full-window stage (on a first visit the inspector and the tasks start folded, so a run
|
|
37
40
|
opens on its plate); *Workbench* docks them beside it with splitters. `1`, `2` and
|
|
38
41
|
`3` fold each module away, the header switches layout and theme (system, light or dark), and the
|
|
@@ -52,6 +55,43 @@ following its log as it grows. It is read from the agent CLI's own log, whicheve
|
|
|
52
55
|
wrote it (as plain text for those that log no events yet), or for an older claude run from
|
|
53
56
|
Claude's session file on this machine.
|
|
54
57
|
|
|
58
|
+
### Controls
|
|
59
|
+
|
|
60
|
+
forge-ui reads everything, and it changes things only by calling forge's own commands: it
|
|
61
|
+
never writes a run file, a repo or a config itself. That keeps one set of rules (validation,
|
|
62
|
+
locks, the audit log) whichever way you change something, and a change from the page is
|
|
63
|
+
indistinguishable from the same command typed in a terminal, except that `control.log` says
|
|
64
|
+
`ui` instead of `cli`. Every change asks you to confirm and says when it takes effect.
|
|
65
|
+
|
|
66
|
+
- **Run controls** (a plan's inspector): the QA threshold, the per-attempt timeout, how many
|
|
67
|
+
failures before escalating, verification retries, and, while a scheduler is live, the retry
|
|
68
|
+
and infrastructure-retry budgets. A task has its own panel for a threshold override
|
|
69
|
+
(including *None*, which exempts it from the run's threshold), its own retry budget, and a
|
|
70
|
+
**model pin**: the model its next full retry uses, once. Each is `forge control`; see
|
|
71
|
+
[decomposed runs](decomposed-runs.md#changing-a-run-while-it-works). A control that cannot
|
|
72
|
+
apply (a budget with no live scheduler, a merged task) is disabled and says why.
|
|
73
|
+
- **Open findings:** the review findings a run left open (tolerated by `--qa-threshold`,
|
|
74
|
+
PLAUSIBLE, or left by the whole-run review), for this run or every run of its repository.
|
|
75
|
+
Findings of an outdated review carry an "outdated review" marker, and findings a person
|
|
76
|
+
accepted have origin `accepted`. Select some and file them as GitHub issues, or dismiss the
|
|
77
|
+
ones that need no work (and take that back). It runs `forge issues`, so a finding is never
|
|
78
|
+
filed twice.
|
|
79
|
+
- **Delete a run:** a dialog that first shows a dry run, then deletes the run directory
|
|
80
|
+
through `forge runs rm`. The dataset keeps the run's traces, and by default a `.tar.gz` of
|
|
81
|
+
them too; you can also remove its clean worktrees and its branches that nothing has checked
|
|
82
|
+
out. Anything that is not the run's own, or has uncommitted changes, is listed and left alone.
|
|
83
|
+
- **Settings:** every config source in one view (`forge config show`) and edits for the
|
|
84
|
+
per-repo ones (`repo.verify`, `repo.setup`, `repo.verify-subset`, and Jev on or off). The
|
|
85
|
+
registry stays read-only: the skill directory is often a live symlink to a checkout.
|
|
86
|
+
|
|
87
|
+
Writing is protected, because anything that can reach the server could otherwise change your
|
|
88
|
+
runs. The server makes a random token each time it starts and puts it in the page it serves;
|
|
89
|
+
the page sends it with every change, and the server also checks the `Host` and `Origin`
|
|
90
|
+
headers, so another website open in your browser cannot make your browser change a run. Reading
|
|
91
|
+
needs no token. `--read-only` (or `FORGE_UI_READONLY=1`) turns writing off altogether: no
|
|
92
|
+
token is issued and every change is refused. `FORGE_UI_READONLY=1 forge ui open` starts such a
|
|
93
|
+
server, and the launcher never reuses a server running in the other mode; it replaces it.
|
|
94
|
+
|
|
55
95
|
Also: replay, critical path, Fractal child-task trees, quick jump (Ctrl K, ⌘K on a Mac, or
|
|
56
96
|
`/`), a bell for runs that finish or fail, failed tasks and final-review verdicts (with opt-in
|
|
57
97
|
browser notifications), and a run exported as one HTML file that opens without forge-ui, with
|
|
@@ -68,8 +108,9 @@ forge ui open [DIR] # an existing run dir, or the run list
|
|
|
68
108
|
```
|
|
69
109
|
|
|
70
110
|
All runs share one detached server on `127.0.0.1:8765` (`FORGE_UI_PORT`) that exits after
|
|
71
|
-
four idle hours (`FORGE_UI_IDLE_S`). A UI that cannot start never fails the run.
|
|
72
|
-
token: any local process can read the prompts and diffs of the runs it shows
|
|
111
|
+
four idle hours (`FORGE_UI_IDLE_S`). A UI that cannot start never fails the run. Reading needs
|
|
112
|
+
no token: any local process can read the prompts and diffs of the runs it shows; only changes
|
|
113
|
+
need the token (see [Controls](#controls)). See
|
|
73
114
|
[references/ui.md](../references/ui.md) and [ui/README.md](../ui/README.md).
|
|
74
115
|
|
|
75
116
|
To look around before your first run, point it at made-up runs (the ones in these pictures), from a git checkout:
|
|
@@ -83,10 +124,10 @@ python3 ui/tools/demo_runs.py /tmp/forge-demo
|
|
|
83
124
|
|
|
84
125
|
| Command | Purpose |
|
|
85
126
|
| --- | --- |
|
|
86
|
-
| `open [DIR] [--no-browser] [--port N]` | start or reuse the shared forge-ui server; print `forge-ui: <url>` for the plan or solo run dir `DIR` (the run list without it) and open a browser the first time |
|
|
87
|
-
| `status [--json]` | exit 0 and describe the server when it runs, 1 when not |
|
|
127
|
+
| `open [DIR] [--no-browser] [--port N]` | start or reuse the shared forge-ui server (never one in the other mode); print `forge-ui: <url>` for the plan or solo run dir `DIR` (the run list without it) and open a browser the first time (a later `open` of the same run says on stderr that it opened none) |
|
|
128
|
+
| `status [--json]` | exit 0 and describe the server (and whether it is read-only) when it runs, 1 when not |
|
|
88
129
|
| `stop` | stop the server |
|
|
89
130
|
|
|
90
131
|
`FORGE_UI=off` makes `--ui` on a runner start nothing (it does not stop a running server or block `forge ui open`; use `forge ui stop`), `FORGE_UI_BROWSER=off` never opens a browser,
|
|
91
|
-
`FORGE_UI_PORT` (8765) and `FORGE_UI_IDLE_S` (14400, `0` = never) tune the server. State and
|
|
132
|
+
`FORGE_UI_PORT` (8765) and `FORGE_UI_IDLE_S` (14400, `0` = never) tune the server, and `FORGE_UI_READONLY=1` starts it read-only. State and
|
|
92
133
|
`server.log` are in `${XDG_STATE_HOME:-~/.local/state}/forge/ui/`.
|
package/docs/memory.md
CHANGED
|
@@ -4,8 +4,10 @@
|
|
|
4
4
|
|
|
5
5
|
What forge learns about a repository, where it keeps it, what each role is told, and what forge has spent there.
|
|
6
6
|
|
|
7
|
-
Every forge dispatch is a clean slate
|
|
8
|
-
|
|
7
|
+
Every forge dispatch is a clean slate — the opt-in `plan --reviewer-session` experiment,
|
|
8
|
+
which continues a reviewer's session across rounds, is the exception — so without help the
|
|
9
|
+
tenth run in a repo rediscovers what the first one learned. forge keeps that in the repo,
|
|
10
|
+
in two files:
|
|
9
11
|
|
|
10
12
|
```
|
|
11
13
|
.forge/memory.md small, capped, injected into every prompt
|
package/docs/reviews.md
CHANGED
|
@@ -51,7 +51,20 @@ Two things it deliberately does:
|
|
|
51
51
|
from — not just the fix. Reviewing the fix alone would let the first attempt's code through
|
|
52
52
|
unread.
|
|
53
53
|
- **Refuses to re-review identical code.** If the retry produces a byte-identical diff, the
|
|
54
|
-
task fails again without spending a QA dispatch.
|
|
54
|
+
task fails again without spending a QA dispatch. A retry that only rejects the findings,
|
|
55
|
+
and leaves that diff unchanged, is still this identical retry: it writes
|
|
56
|
+
`tasks/<id>/noretry` and QA is not dispatched.
|
|
57
|
+
|
|
58
|
+
When the diff did change, QA on that retry is handed the previous round as well as the
|
|
59
|
+
cumulative diff. The report goes under **Findings from the previous review round**. The
|
|
60
|
+
diff between the two attempts goes under **Changes since that review**, and is kept as
|
|
61
|
+
`tasks/<id>/changes.since.diff`, `changes.since.range` and `changes.since.stat` (the
|
|
62
|
+
range is omitted when the earlier commit is missing, unknown or not an ancestor, or when
|
|
63
|
+
nothing changed; a diff over 100 KB is replaced in the prompt by its stat). The
|
|
64
|
+
implementer's reply goes under **The implementer's reply (its own account; verify
|
|
65
|
+
every claim against the code)**. The prompt tells the reviewer to restate only findings that are still
|
|
66
|
+
open, as `- [Px][CONFIRMED|PLAUSIBLE]` lines, and to mention a fixed or rejected finding
|
|
67
|
+
only in a plain sentence. The gate and `forge issues` count every labelled line.
|
|
55
68
|
|
|
56
69
|
`retry` is something you run. Forge never loops a dwarf against its own reviewer on its own
|
|
57
70
|
initiative — unless you ask a *run* to with `--retry-failed N`, see
|
|
@@ -120,6 +133,30 @@ lists them. The literal status stays `PASS`, so nothing downstream needed a new
|
|
|
120
133
|
It fails safe: an unlabelled CONFIRMED finding, or a FAIL with no CONFIRMED finding at all,
|
|
121
134
|
is never tolerated, and without a threshold nothing changes.
|
|
122
135
|
|
|
136
|
+
Tolerated findings are recorded, not fixed. After a run, the skill offers once to file them as
|
|
137
|
+
GitHub issues, together with PLAUSIBLE findings of passed tasks and whatever a whole-run review
|
|
138
|
+
left open (`--file-issues ask|auto|off`, `ask` by default). Where the
|
|
139
|
+
[gh-proman](https://github.com/Giamme/gh-proman) skill is set up for the repository, it files
|
|
140
|
+
them on the team's project board; anywhere else forge files them itself with plain `gh`:
|
|
141
|
+
|
|
142
|
+
```sh
|
|
143
|
+
./forge issues <plan-dir> # drafts only: titles, severity, already filed
|
|
144
|
+
./forge issues <plan-dir> --include-plausible --apply --label bug
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Each issue gets the finding, its severity, the task, reviewer and a code link at the merged
|
|
148
|
+
commit. A hidden fingerprint, shared with gh-proman, keeps a finding from being filed twice.
|
|
149
|
+
Findings a person merged with `accept` are origin `accepted`: the CONFIRMED lines under
|
|
150
|
+
`## Accepted by a human despite the reviewer's <status> verdict` in `known_issues.md`.
|
|
151
|
+
Their issue says a person accepted the work over that verdict, and gives the recorded
|
|
152
|
+
reason. Findings above that heading, which a threshold tolerated, stay origin
|
|
153
|
+
`tolerated`. A whole-run review whose integration branch has moved since it is outdated.
|
|
154
|
+
Its findings stay listed, and the issue says the review is outdated and names the tasks
|
|
155
|
+
that merged after it. In `--json` each whole-run finding carries `review`: `name`,
|
|
156
|
+
`outdated` and `merged_since`. `outdated` is null when the review has no verdict or the
|
|
157
|
+
plan has no accepted integration commit, and `merged_since` is empty unless the review
|
|
158
|
+
is outdated. Every other finding has `review` null.
|
|
159
|
+
|
|
123
160
|
## Accepting a task despite its reviewer
|
|
124
161
|
|
|
125
162
|
```bash
|
|
@@ -135,7 +172,9 @@ it on its own. It accepts only a `FAIL` or `UNKNOWN` task whose reviewed work is
|
|
|
135
172
|
the branch is still at the reviewed commit and the worktree still has the saved fingerprint —
|
|
136
173
|
otherwise it exits `3` and says to `retry` for a fresh review (accepting would sign off on code
|
|
137
174
|
nobody reviewed). It writes `tasks/<id>/known_issues.md` (your reason, who and when, and the
|
|
138
|
-
reviewer's reply copied verbatim
|
|
175
|
+
reviewer's reply copied verbatim, under
|
|
176
|
+
`## Accepted by a human despite the reviewer's <status> verdict`) and `accepted.by`,
|
|
177
|
+
merges through the same checks as any other
|
|
139
178
|
task, and puts everything back if the merge conflicts (exit `6`). Dependents are not touched;
|
|
140
179
|
`run` continues them.
|
|
141
180
|
|
|
@@ -165,6 +204,52 @@ the one remembered in `final_qa`, else the single spec every task shares (otherw
|
|
|
165
204
|
Tasks that are not merged are not in the result, and it says so. `integrate` also names unmerged
|
|
166
205
|
tasks even without a review.
|
|
167
206
|
|
|
207
|
+
A later whole-run review carries the previous one's report, under **Findings from the
|
|
208
|
+
previous final review**, and the diff since it, under **Changes since that review**
|
|
209
|
+
(inlined up to 200 KB, otherwise its stat). `review.meta` records `integration=` (the
|
|
210
|
+
accepted integration commit, or `none`), `previous=` (that review's directory, or
|
|
211
|
+
`none`) and `since=` (its tip, when that commit is an ancestor of this review;
|
|
212
|
+
otherwise `none`, and the prompt says the range cannot be shown). When the two reviews
|
|
213
|
+
share a commit, `since=` is that commit and the prompt says nothing changed. The same
|
|
214
|
+
labelled-line rule applies: restate only what is still open, because every labelled line
|
|
215
|
+
is filed.
|
|
216
|
+
|
|
217
|
+
## Reviewer session (opt-in experiment)
|
|
218
|
+
|
|
219
|
+
`plan --reviewer-session` continues the same reviewer's session across rounds. It is off
|
|
220
|
+
unless you pass it; `--no-reviewer-session` turns it off again and removes the plan's
|
|
221
|
+
`reviewer_session` marker. While the marker is present, per-task QA and a repeated
|
|
222
|
+
whole-run review continue the previous round's session on claude, openclaude and codex,
|
|
223
|
+
when the resolved harness, model and effort are unchanged. The approval table names it.
|
|
224
|
+
Dispatch details are in
|
|
225
|
+
[continuing a reviewer session](../references/harnesses.md#continuing-a-reviewer-session).
|
|
226
|
+
|
|
227
|
+
Any round that cannot continue starts a fresh session. That is the first round, a missing
|
|
228
|
+
or malformed session id, a resolution that is missing or different, any harness other
|
|
229
|
+
than those three, a Fractal plan, and a continue that the harness refuses or that fails
|
|
230
|
+
(dispatch exit **9**). Exit 9 means the session was not continued. Forge sends the same
|
|
231
|
+
prompt again as a fresh session, and that dispatch decides the round. One row per round
|
|
232
|
+
is appended to `tasks/<id>/review_sessions.tsv`: round, attempt, `continued` or `fresh`,
|
|
233
|
+
the reason, and the session id the round considered (empty when there was none). The
|
|
234
|
+
row describes the dispatch whose result counts, so a failed continue is a single
|
|
235
|
+
`fresh` row. Reasons are `same-spec`, `first-round`, `no-session`, `invalid-session`,
|
|
236
|
+
`spec-changed`, `harness-unsupported`, `fractal`, `continue-unsupported` and
|
|
237
|
+
`continue-failed`. A final review records the same outcome as `session=` and
|
|
238
|
+
`session_reason=` in `review.meta`.
|
|
239
|
+
|
|
240
|
+
A shadow review never continues a session. It stays a separate fresh review of a copy of
|
|
241
|
+
the prompt.
|
|
242
|
+
|
|
243
|
+
A review that was asked to continue is tagged `session_continue` in the dataset
|
|
244
|
+
(`continued`, or `failed` when the continue did not succeed) and left out of the
|
|
245
|
+
reviewer slices and the agreement rate. The fresh dispatch after exit 9 is not tagged,
|
|
246
|
+
so it still counts. See [the dataset](../references/dataset.md).
|
|
247
|
+
|
|
248
|
+
This is separate from a QA-only resume, the re-run after an infrastructure stop. That
|
|
249
|
+
re-run is the `tasks/<id>/resume` marker: forge revalidates it against the worktree,
|
|
250
|
+
then asks QA again about unchanged work, and no attempt is spent. The reviewer session
|
|
251
|
+
continues the reviewer's conversation into a later round, and only when you opt in.
|
|
252
|
+
|
|
168
253
|
## Shadow reviews: how much of a verdict is the reviewer?
|
|
169
254
|
|
|
170
255
|
`plan --shadow-qa <spec>[,<spec>...] [--shadow-rate R]` (default R = 0.1) sends a sampled
|