@giamme/forge 0.0.0-stage → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +205 -0
- package/LICENSE +21 -0
- package/README.md +194 -2
- package/SKILL.md +123 -0
- package/VERSION +1 -0
- package/docs/README.md +26 -0
- package/docs/add-ons.md +33 -0
- package/docs/artifacts.md +69 -0
- package/docs/commands.md +315 -0
- package/docs/dataset.md +20 -0
- package/docs/decomposed-runs.md +268 -0
- package/docs/development.md +95 -0
- package/docs/forge-ui.md +92 -0
- package/docs/fractal.md +181 -0
- package/docs/install.md +160 -0
- package/docs/memory.md +156 -0
- package/docs/models.md +118 -0
- package/docs/reviews.md +231 -0
- package/docs/safety.md +55 -0
- package/docs/troubleshooting.md +54 -0
- package/forge +79 -0
- package/package.json +27 -4
- package/references/dataset.md +329 -0
- package/references/decompose.md +667 -0
- package/references/fractal-dashboard.md +21 -0
- package/references/fractal.md +307 -0
- package/references/harnesses.md +365 -0
- package/references/jev.md +865 -0
- package/references/memory.md +228 -0
- package/references/planning.md +42 -0
- package/references/ripwire.md +47 -0
- package/references/runtime.md +64 -0
- package/references/solo.md +113 -0
- package/references/ui.md +112 -0
- package/registry.tsv +81 -0
- package/scripts/forge-artifact.sh +78 -0
- package/scripts/forge-contract.py +430 -0
- package/scripts/forge-dataset.py +6 -0
- package/scripts/forge-dispatch.sh +671 -0
- package/scripts/forge-fail-first.py +195 -0
- package/scripts/forge-fractal-dispatch.sh +4 -0
- package/scripts/forge-fractal-options.sh +54 -0
- package/scripts/forge-fractal.py +6 -0
- package/scripts/forge-guard.py +266 -0
- package/scripts/forge-install-fractal.sh +4 -0
- package/scripts/forge-install-ripwire.sh +52 -0
- package/scripts/forge-install.sh +135 -0
- package/scripts/forge-jev-options.sh +259 -0
- package/scripts/forge-jev.py +6 -0
- package/scripts/forge-lib-plans.sh +785 -0
- package/scripts/forge-lib-review.sh +494 -0
- package/scripts/forge-lib-task.sh +302 -0
- package/scripts/forge-lib-verify.sh +30 -0
- package/scripts/forge-memory.sh +575 -0
- package/scripts/forge-metrics.sh +39 -0
- package/scripts/forge-parallel-options.sh +76 -0
- package/scripts/forge-parallel.sh +1717 -0
- package/scripts/forge-plan-check.py +118 -0
- package/scripts/forge-prompt.py +45 -0
- package/scripts/forge-ripwire.py +136 -0
- package/scripts/forge-route.py +344 -0
- package/scripts/forge-runs.py +121 -0
- package/scripts/forge-runtime.py +586 -0
- package/scripts/forge-schedule.py +312 -0
- package/scripts/forge-shadow.py +305 -0
- package/scripts/forge-solo.sh +347 -0
- package/scripts/forge-ui-options.sh +42 -0
- package/scripts/forge-ui.py +435 -0
- package/scripts/forge-update.sh +71 -0
- package/scripts/forge-verify-coverage.py +956 -0
- package/scripts/forge-version.py +240 -0
- package/scripts/forge_dataset/__init__.py +439 -0
- package/scripts/forge_dataset/capture.py +330 -0
- package/scripts/forge_dataset/checks.py +109 -0
- package/scripts/forge_dataset/cli.py +233 -0
- package/scripts/forge_dataset/export.py +1009 -0
- package/scripts/forge_dataset/harvest.py +716 -0
- package/scripts/forge_dataset/importer.py +412 -0
- package/scripts/forge_dataset/outcomes.py +275 -0
- package/scripts/forge_dataset/routing.py +278 -0
- package/scripts/forge_fractal/__init__.py +84 -0
- package/scripts/forge_fractal/bridge.py +46 -0
- package/scripts/forge_fractal/cli.py +202 -0
- package/scripts/forge_fractal/dashboard.css +138 -0
- package/scripts/forge_fractal/dashboard.html +41 -0
- package/scripts/forge_fractal/dashboard.js +687 -0
- package/scripts/forge_fractal/decomposition.py +262 -0
- package/scripts/forge_fractal/execution.py +659 -0
- package/scripts/forge_fractal/inspection.py +261 -0
- package/scripts/forge_fractal/install.py +158 -0
- package/scripts/forge_fractal/invocation.py +94 -0
- package/scripts/forge_fractal/pipeline.py +124 -0
- package/scripts/forge_fractal/selection.py +179 -0
- package/scripts/forge_jev/__init__.py +376 -0
- package/scripts/forge_jev/backtest.py +322 -0
- package/scripts/forge_jev/cli.py +980 -0
- package/scripts/forge_jev/client.py +418 -0
- package/scripts/forge_jev/corpus.py +156 -0
- package/scripts/forge_jev/coupling.py +244 -0
- package/scripts/forge_jev/features.py +24 -0
- package/scripts/forge_jev/gates.py +100 -0
- package/scripts/forge_jev/memory.py +163 -0
- package/scripts/forge_jev/outcome.py +49 -0
- package/scripts/forge_jev/predict.py +204 -0
- package/scripts/forge_jev/questions.py +661 -0
- package/scripts/forge_jev/review.py +73 -0
- package/scripts/forge_jev/routing.py +271 -0
- package/scripts/forge_jev/subset.py +123 -0
- package/scripts/forge_jev/verify.py +453 -0
- package/scripts/forge_registry.py +140 -0
- package/ui/README.md +347 -0
- package/ui/THIRD_PARTY_NOTICES.md +177 -0
- package/ui/docs/api.md +970 -0
- package/ui/forge_ui/__init__.py +20 -0
- package/ui/forge_ui/__main__.py +135 -0
- package/ui/forge_ui/diffs.py +454 -0
- package/ui/forge_ui/executions.py +436 -0
- package/ui/forge_ui/export.py +214 -0
- package/ui/forge_ui/findings.py +132 -0
- package/ui/forge_ui/fractal.py +582 -0
- package/ui/forge_ui/models.py +216 -0
- package/ui/forge_ui/reader.py +1424 -0
- package/ui/forge_ui/server.py +312 -0
- package/ui/forge_ui/static/D-DINCondensed-Bold.woff2 +0 -0
- package/ui/forge_ui/static/app.js +1079 -0
- package/ui/forge_ui/static/cloud3d.js +647 -0
- package/ui/forge_ui/static/cloudmodel.js +375 -0
- package/ui/forge_ui/static/cloudui.js +397 -0
- package/ui/forge_ui/static/convo.js +295 -0
- package/ui/forge_ui/static/datatable.js +232 -0
- package/ui/forge_ui/static/diffview.js +235 -0
- package/ui/forge_ui/static/favicon.svg +1 -0
- package/ui/forge_ui/static/files.js +776 -0
- package/ui/forge_ui/static/findings3d.js +134 -0
- package/ui/forge_ui/static/gantt.js +97 -0
- package/ui/forge_ui/static/graph.js +266 -0
- package/ui/forge_ui/static/icons.js +94 -0
- package/ui/forge_ui/static/index.html +67 -0
- package/ui/forge_ui/static/iso.js +816 -0
- package/ui/forge_ui/static/leaderboard.js +311 -0
- package/ui/forge_ui/static/map3d.js +45 -0
- package/ui/forge_ui/static/modelstats.js +249 -0
- package/ui/forge_ui/static/notify.js +179 -0
- package/ui/forge_ui/static/panel.js +562 -0
- package/ui/forge_ui/static/runmodel.js +667 -0
- package/ui/forge_ui/static/scene3d.js +1471 -0
- package/ui/forge_ui/static/search.js +120 -0
- package/ui/forge_ui/static/shell.js +137 -0
- package/ui/forge_ui/static/stagectl.js +150 -0
- package/ui/forge_ui/static/style.css +1073 -0
- package/ui/forge_ui/static/theme.js +25 -0
- package/ui/forge_ui/transcripts.py +608 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and versions follow the format `{major}.{minor}[.{patch}]` ([...] parts appear only when non-zero).
|
|
7
|
+
|
|
8
|
+
## [0.9] - 2026-10-21
|
|
9
|
+
|
|
10
|
+
forge-ui has a new look and layout, a 3D stage that marks every review finding on its task, and each dwarf's and reviewer's conversation, which runs on claude, openclaude and opencode now keep in full. Also new: `forge runs` and a first screen that says how to get runs; npm installations now hear about new releases.
|
|
11
|
+
|
|
12
|
+
### Highlights
|
|
13
|
+
|
|
14
|
+
- forge-ui shows what each dwarf and reviewer said and did. Every attempt in the inspector links to its conversation, which opens where a diff opens: the agent's messages, its thinking folded, each command it ran with its exit code and output, each file it changed with the patch and a link to that file in the diff its review read, its other tool calls, and the final answer with tokens and cost. Search it with `/`, keep only what it said, ran or changed, and step through its errors with `E`. While the agent works, the conversation follows its log as it grows. It is read from the CLI's own log: codex, grok, opencode and claude streams event by event; for a claude run that logged only its final answer, from Claude's session file for that run if it is still on this machine; any other log as plain text. An export includes conversations only when prompts and texts are included.
|
|
15
|
+
- forge-ui has a new look and better controls. It reads like a signal box's desk: engraved plates for headings, push buttons whose lamp lights when they are on, and lamps that keep one meaning everywhere (amber running, green merged, red failed, unlit pending), equally finished in light and dark. Choose between two layouts: *Immersive* (the default) floats the runs, the inspector and the tasks over a full-window 3D stage (on a first visit the inspector and the tasks start folded, so a run opens on its plate, and choosing a task opens the inspector), and *Workbench* docks them beside it with splitters you can drag. `1`, `2` and `3` fold each one away, the header switches layout and theme (system, light or dark), and the runs list sorts by latest activity, state or repo. Each model shows as one name plate everywhere, a finding's severity is a shape rather than a status colour, and the run's settings, request and verification output moved from the header into the inspector (*Run details*). The task table and the model leaderboard sort by any column, reorder columns by dragging a header (or Alt+← →), and show or hide columns from a *Columns* menu. forge-ui remembers all of it in your browser.
|
|
16
|
+
|
|
17
|
+
### Added
|
|
18
|
+
|
|
19
|
+
- `forge runs` lists the runs registered for the current repo (or `--all` repos), newest first, with each run's status, checks, goal and run directory; `--json` for scripts. Read-only.
|
|
20
|
+
- forge-ui's 3D stage shows more and is easier to steer. Every review finding now stands over its task's block as a mark shaped by its severity (P0 spike, P1 diamond, P2 cone, P3 pin); hover it for its location and summary, click it to open the diff that review read, scrolled to that finding. Choosing a task keeps its route lit, what it waits on and what waits on it (`I` switches this off), and hovering a task shows a card with its status, attempts, busy time, tokens, models and worst finding. Merges, failed attempts, escalations and final reviews play over the plate as they happen and as a replay passes them, and the replay now runs smoothly instead of in steps. The *View* menu switches each part of the drawing on or off (cables, flow, lamps, roof text, readouts, findings, finish line, districts, halos, route). New keys at the stage's edge fit what is shown (`F`), zoom (`+` `−`), switch between perspective, true isometric and top-down (`V`), reset the view (`0`), open a legend of what the stage draws now (`L`), go full screen, and list every key (`?`). A block whose roof is too small to read gets its name on a tag that faces you, and on the map ← → walk the runs, ↑ ↓ the repositories and Enter opens one.
|
|
21
|
+
|
|
22
|
+
### Changed
|
|
23
|
+
|
|
24
|
+
- forge-ui's first screen, with no runs yet, now says how to get one (start a run with `/forge "<goal>" --dwarf <model>`, or `forge ui open <run-dir>` for runs kept elsewhere) instead of an empty 3D plate under filters, and the run list no longer suggests `python3 -m forge_ui --root <dir>`, which only worked from inside forge's ui folder. The map takes over as soon as the first run appears.
|
|
25
|
+
- Runs on claude, openclaude and opencode now keep their whole conversation. forge starts claude and openclaude with `--output-format stream-json --verbose` and opencode with `--format json`, so `<role>.log` holds every message, tool call and result instead of only the final answer (claude) or plain text (openclaude, opencode). forge-ui shows it as the attempt's conversation. The final message, the token counts and the infrastructure checks are read from these streams: opencode now reports its tokens, its answer is the text of its last step, and an opencode `error` event that no answer followed (an expired token, a refused route) is classified like codex's. antigravity keeps its plain output until its stream has been seen on a working login.
|
|
26
|
+
|
|
27
|
+
### Fixed
|
|
28
|
+
|
|
29
|
+
- An npm installation now hears about new releases: the once-a-day notice after a run asks the npm registry (or `npm_config_registry`) and names npm's command, `npm install -g @giamme/forge@latest && forge install`. `forge version` shows that command too, instead of telling npm users to point `forge update --source` at a Git checkout they don't have.
|
|
30
|
+
- With `XDG_STATE_HOME` set but empty, Fractal keeps its state (its runtime and its runs) in `~/.local/state/forge/fractal`, where forge-ui looks, instead of in a `forge/fractal` folder under whatever directory forge was started from. `forge dataset import --fractal` reads the same place.
|
|
31
|
+
|
|
32
|
+
## [0.8] - 2026-10-21
|
|
33
|
+
|
|
34
|
+
forge installs from npm: `npm install -g @giamme/forge`, then `forge install`. The forge command also gains `forge update`, `forge version` and a real `--help`, reviews can ask for opt-in fail-first evidence, and forge is now under the MIT License.
|
|
35
|
+
|
|
36
|
+
### Highlights
|
|
37
|
+
|
|
38
|
+
- forge installs from npm: `npm install -g @giamme/forge`, then `forge install` links it into your agent harnesses, and `npm install -g @giamme/forge@latest && forge install` updates it. The `forge` command works through the link npm puts on your PATH, `forge install` runs the installer from a checkout too, and `forge update` on an npm installation names the npm command instead of asking for a Git checkout. The once-a-day update notice needs a Git checkout, so npm installations don't get it yet.
|
|
39
|
+
|
|
40
|
+
### Added
|
|
41
|
+
|
|
42
|
+
- forge is now released under the MIT License: anyone may use, change and share it as long as the copyright notice stays with it. Until now the repository had no license, which left nobody the right to use it.
|
|
43
|
+
- `forge update` and `forge version` now work from the forge command. `forge version` shows your version, checkout and upstream (`--check` asks for the newest release), and a finished solo run, `plan` or `run` prints one line when a newer forge is out, at most once a day; it stays quiet offline, in CI or with `FORGE_UPDATE_CHECK=off`.
|
|
44
|
+
- New opt-in `--fail-first` (solo runs and `plan`): forge also runs the tests a change adds or edits against the code without its change, using `.forge/verify-subset`, and asks the reviewer to report any test that still passes as a P2 finding. It is advice only and never changes a result by itself; forge-ui shows the evidence under the review findings, and each check is recorded in the run dataset.
|
|
45
|
+
|
|
46
|
+
### Changed
|
|
47
|
+
|
|
48
|
+
- `./forge --help` (or `-h`, or `forge help`) now lists the commands with a line on each and exits successfully, instead of failing with four bare usage lines. An unknown command is named in the error, and `forge fractal --help` and `forge jev --help` call themselves by the name you typed rather than their script file.
|
|
49
|
+
|
|
50
|
+
### Fixed
|
|
51
|
+
|
|
52
|
+
- With `XDG_CONFIG_HOME` set but empty, `forge ui open` wrote its run list into a `forge/` folder in the current directory (usually your repo), where forge-ui could not find it; runs were left off forge-ui's run map, and `forge jev` read `jev.json` from the current directory. An empty value now counts as unset, as the XDG spec says, so all of it lives in `~/.config/forge`.
|
|
53
|
+
- forge-ui now shows a review's verdict exactly as forge records it: only a FORGE_VERDICT line that ends the reply counts. A reply forge treated as having no verdict (and blocked on) no longer shows as passed or failed in forge-ui.
|
|
54
|
+
- `forge-install.sh --help` prints its usage instead of installing, and an option it does not know, such as a mistyped `--dryrun`, is refused rather than ignored, which used to run a real installation.
|
|
55
|
+
|
|
56
|
+
## [0.7.1] - 2026-10-20
|
|
57
|
+
|
|
58
|
+
A patch for forge-ui: orbiting its 3D views no longer stalls, and you can look around it on made-up runs before your first one.
|
|
59
|
+
|
|
60
|
+
### Added
|
|
61
|
+
|
|
62
|
+
- You can look around forge-ui before your first run: `python3 ui/tools/demo_runs.py DIR` writes made-up runs (one still working, and a few weeks of history across five projects), and `python3 -m forge_ui --no-registry --root DIR`, run from `ui/`, shows them.
|
|
63
|
+
|
|
64
|
+
### Fixed
|
|
65
|
+
|
|
66
|
+
- Orbiting forge-ui's 3D views no longer stalls. While the camera moved, every frame asked for two more, so the drawing doubled with each frame: a second of orbiting drew hundreds of times and the page stalled, worst in the run cloud.
|
|
67
|
+
|
|
68
|
+
## [0.7] - 2026-10-20
|
|
69
|
+
|
|
70
|
+
forge-ui grows from a run viewer into a full picture of your runs: a full-screen 3D cloud of every run, grouped by project and filtered down to exactly the runs you want, diffs that show each review finding on the line it names, and a model leaderboard that compares implementers and reviewers on your own history. It also gains 3D run views, quick jump, notifications, a files heatmap and one-file HTML exports.
|
|
71
|
+
|
|
72
|
+
### Highlights
|
|
73
|
+
|
|
74
|
+
- forge-ui has a run cloud (*Run cloud* in the top bar): every run at once, full screen in 3D, grouped by project. Choose how it is laid out (a helix per project, spheres, galaxies, a timeline with one lane per project, or a constellation that pulls together runs sharing models) and whether runs are blocks or orbs. The filter panel narrows the cloud to exactly the runs you want: projects, state, difficulty, tasks per run, the models that implemented or reviewed them, recent activity, verdicts and more. Orbit, zoom and drag any run wherever you want it; the cloud remembers where you put things in that browser, per layout, and *Positions* resets them or exports and imports them as a file. Click a run for a summary card, double-click to open it.
|
|
75
|
+
- forge-ui shows diffs with the review findings in place. From a task, any of its reviews or the final review, open the diff: the task's current, previous or committed changes, or exactly the diff a reviewer read, with every P0–P3 finding shown under the line it points at (and the ones that point outside the diff listed apart). n and p jump between findings. When a run used Fractal, a task also shows the child tasks it was split into, in the inspector and as a small tree floating over the opened task in 3D, each with its decision, cost, output and its own diff.
|
|
76
|
+
- forge-ui has a model leaderboard (*Models* in the top bar): for each implementer model, how often its first attempt passed (with a confidence interval), its retries, escalations away, timeouts, time and tokens; for each reviewer, its verdicts, findings per review and how often shadow reviews agreed with it. Filter by project, difficulty and period, or keep only the tasks where forge picked the model at random, the fairest comparison since routing is otherwise not random. Click a model to see its runs in the run cloud.
|
|
77
|
+
|
|
78
|
+
### Added
|
|
79
|
+
|
|
80
|
+
- forge-ui now draws your runs as a 3D map in archivision's style. The front page shows every run grouped by repo, with a lamp per task and filters that fade what does not match; a run shows its tasks in their planned waves, ending in integration, verification and the whole-run final review, and *Execution* shows the same run on wall-clock time, attempt by attempt, against `--max-parallel`. Click a task to open it to its attempts, reviews and escalations; the inspector beside it shows each step's exact prompt, model, timings, tokens, verdict and P0–P3 findings. Statuses update live, and you can replay a run, light its critical path, and size or colour tasks by duration, tokens, attempts or model. The 3D view loads three.js from cdn.jsdelivr.net the first time it is shown; offline, or with *2D* chosen, the page works without it.
|
|
81
|
+
- forge-ui has a quick jump and notifications. Press Ctrl K (⌘K on a Mac) or `/` anywhere to search every run, task, project and model and jump straight to it; Shift+Enter shows a run in the run cloud. The new bell tells you when a run finishes or ends with failures, when a task fails, and when a final review gives its verdict, with an unread count in the tab title; turn on browser notifications from the bell to hear about them while the tab is in the background.
|
|
82
|
+
- You can export a run from forge-ui as one HTML file to share (*Export…* in the run's header). The file opens in any browser without forge-ui, even offline (in 2D; in 3D when it can reach cdn.jsdelivr.net), with the run's plan, execution, inspector and findings. It asks each time what to include: prompts, reports and diffs are left out unless you tick them, and local paths are hidden unless you untick that.
|
|
83
|
+
- forge-ui shows which tasks touched which files: *Files* in a run's toolbar draws every file the run planned or changed against its tasks, wave by wave, with how many attempts changed each file, the files a task changed without planning to (drift), merge conflicts, and files two tasks of the same wave both worked on. Click a cell to select the task and open that file's diff.
|
|
84
|
+
|
|
85
|
+
### Changed
|
|
86
|
+
|
|
87
|
+
- Every plan, run and solo run now registers its run dir in forge's run registry, not only the ones opened with `--ui`, so forge-ui's map can show all of a repo's runs. While the forge-ui server runs, that means it can show every run on the machine; `forge ui stop` closes it, and `FORGE_RUN_REGISTRY=off` stops runs from registering. `forge-parallel.sh run` also records the `--max-parallel` it was given in the plan dir.
|
|
88
|
+
|
|
89
|
+
### Fixed
|
|
90
|
+
|
|
91
|
+
- forge-ui's 2D dependency graph shows its status colours again; the page's own security policy had been blocking them.
|
|
92
|
+
|
|
93
|
+
## [0.6] - 2026-10-20
|
|
94
|
+
|
|
95
|
+
Forge gets a companion: forge-ui, a read-only local page that watches any run live, now ships inside forge and is offered at the start of every run. Grok 4.7 joins as a sixth harness through xAI's grok CLI, and a grok session the CLI cancels partway through is now recorded instead of passing as finished.
|
|
96
|
+
|
|
97
|
+
### Added
|
|
98
|
+
|
|
99
|
+
- Forge can now dispatch through xAI's `grok` CLI. The `grok` alias runs Grok 4.7 there, at efforts up to `xhigh`, as an implementer or a read-only reviewer: `--dwarf grok:high`. Its answers and token counts are read like Claude's, and an expired login or a rate limit is reported as an infrastructure failure (exit 8). The previous route, Grok 4.6 through opencode, is still available as `grok:high:opencode`.
|
|
100
|
+
- Forge can now show a run live in a local web page. At the start of each new run `/forge` asks "Open forge-ui to watch this run? [y/N]"; answer yes (or pass `--ui`) and forge prints a `forge-ui:` link to the tasks, statuses, models, attempts and reviewer verdicts of a decomposed or solo run, and opens it in your browser. `--no-ui` skips the question. All runs share one local server that stops itself after four idle hours, and `forge ui open|status|stop` manages it by hand; `FORGE_UI=off` stops runs from launching it. A UI that cannot start never fails the run. The page has no login, so while the server runs, any program on your machine can read the prompts and diffs it shows, even for runs started with `--no-ui`; `forge ui stop` closes it. Solo runs now also record `status`, `attempts.tsv`, `dwarf.spec` and `qa.spec`.
|
|
101
|
+
|
|
102
|
+
### Fixed
|
|
103
|
+
|
|
104
|
+
- A grok session that the grok CLI cancels partway through no longer passes as a finished run. This happens when a tool call needs an approval that a headless run can't give, and the CLI still exits 0. forge now records it as `cancelled` in `<role>.infra`, together with the refused command, and says so in the run output. The work done before the cancel is still reviewed as usual.
|
|
105
|
+
- The exploration note under `plan`'s table now appears only when at least one row was actually drawn. It used to say "exploration on" even when every row named its own dwarf, or no tier had a neighbour to try. Its odds are also printed with a decimal point whatever your number locale (it showed "probability 0,1" under an Italian locale).
|
|
106
|
+
|
|
107
|
+
## [0.5] - 2026-10-19
|
|
108
|
+
|
|
109
|
+
Forge now picks models from measured results rather than a decision model's opinion: it explores your own pools by default, compares models and efforts per tier with forge dataset routing, moves a task to a stronger model after three failed attempts, and flags tasks too large to pass review first time. Jev's effort sizing, tier rewriting and calibration are gone, and Cloudflare's hosted Clef can answer chosen Jev decisions instead.
|
|
110
|
+
|
|
111
|
+
### Added
|
|
112
|
+
|
|
113
|
+
- With a Cloudflare provider configured, forge records two predictions it never acts on: whether each task's first attempt will pass review (once per plan, after models are assigned) and whether a retry will pass with the findings it was handed. They are written beside the task (`clef.first_pass`, `clef.retry_pass.<n>`) and into the run dataset, and the export attaches them to its routing and retry cases, so their accuracy on new runs can be measured before anything relies on them. No prompt, verdict, table or retry ever sees them; `FORGE_JEV_OUTCOMES=off` turns them off.
|
|
114
|
+
- Forge can send chosen decision sites to Cloudflare's hosted Clef instead of TypeSafe's Jev: `forge jev setup --provider cloudflare --account-id <id>` validates a Workers AI token (kept in `cloudflare.env` beside `jev.json`, never in argv or logs) and maps the sites a paired replay of forge's own live requests showed Clef no worse on, which today is drift prediction alone; `forge jev sites` shows or changes the map and `FORGE_CLEF=off` sends everything back. Thresholds can now differ per model, and `status` and `doctor` report readiness per site and provider. Without a Cloudflare provider configured nothing changes.
|
|
115
|
+
- After three failed attempts on one model, a decomposed task's next retry goes a tier up in the pools you gave `plan`, instead of repeating a dispatch whose fourth try passes least often. When no stronger model exists, the task stops with a split suggestion, and a plain `retry` refuses until you split the task or name a model with `--dwarf`. Use `--escalate-after N` to change the threshold, or `--escalate-after 0` to turn this off.
|
|
116
|
+
- `plan` now names tasks that are large enough that their first attempt usually fails review (at least 7,000 characters of prompt or 9 declared files; forge's history passes about a third of those), and tasks whose prompt has no `## Acceptance criteria` section. The note is advisory: the table and the exit code are unchanged.
|
|
117
|
+
|
|
118
|
+
### Changed
|
|
119
|
+
|
|
120
|
+
- `plan` now explores by default (`--explore 0.2`): now and then, a row left to forge gets the model one tier away in your own pools (cheaper for medium and high tasks, stronger for low ones), with the odds recorded. The new `forge dataset routing` command compares each pair of models from those draws, and also shows every model's first-pass rate and runtime per tier. Pools that differ only in effort (`--dwarf-low sol --dwarf-high sol:xhigh`) are compared too, so the table also shows whether a higher effort passes more. `--explore 0` turns exploration off, and that choice is kept for later re-plans.
|
|
121
|
+
|
|
122
|
+
### Removed
|
|
123
|
+
|
|
124
|
+
- **Breaking:** Jev no longer suggests efforts or rewrites tiers, because forge's run history showed it could not choose either: its effort suggestions carried no information beyond the task's tier. `forge jev calibrate`, `forge jev parse-spec` and `forge jev qa-effort` are gone, and so are the effort suggestions in Jev's routing lines and Fractal's re-rating of child tasks. `--jev-act` is still accepted but changes nothing. `forge jev doctor` lists the `routing_act` and `calibrate_floor` thresholds as safe to delete from an existing config. Drift checks, memory curation, verification and the other advisory gates are unchanged.
|
|
125
|
+
|
|
126
|
+
### Fixed
|
|
127
|
+
|
|
128
|
+
- The run dataset no longer drops an event when several writers in one process are the first ever to write to a new store: switching the new file to WAL is no longer raced by every connection, and losing that race is harmless.
|
|
129
|
+
- A comma list in `--qa` (or `--qa-high`, `--qa-medium`, `--qa-low`) now rotates through its reviewers task by task, as the docs promise and as the dwarf flags always did. Before, `plan` copied the whole list into every row, so every review failed before reaching a model. A list typed directly into a task's dwarf or qa column is now refused when you run `plan`.
|
|
130
|
+
|
|
131
|
+
### Security
|
|
132
|
+
|
|
133
|
+
- A decision-model API key or token that straddled the 200-character cut of an error message could leave its first characters in `jev.jsonl` and `jev.skip`; error text is now redacted before it is shortened.
|
|
134
|
+
|
|
135
|
+
### For contributors
|
|
136
|
+
|
|
137
|
+
- The test suite now runs every test process against a throwaway forge config home, so no test, including one still being written, can read your real `jev.json` (and its TypeSafe key) or add its temp dirs to your runs registry.
|
|
138
|
+
|
|
139
|
+
## [0.4] - 2026-10-15
|
|
140
|
+
|
|
141
|
+
Every forge run now records its inputs and outcomes in a dataset outside your repos, built for training and calibrating decision models on how forge really performs: it exports labeled decision cases, follows merged work into later history, and can add a second, independent review or an exploratory model pick to measure what a single run cannot.
|
|
142
|
+
|
|
143
|
+
### Highlights
|
|
144
|
+
|
|
145
|
+
- Every forge run now records its inputs and outcomes in a dataset outside your repos (`~/.local/share/forge/dataset/`). The record includes the resolved model of each dispatch, prompts, the diff of each attempt, verdicts and findings, tokens, merge and verification outcomes, and the full request behind each Jev judgment. It survives deleting run directories and exists for training decision models on how forge actually performs. `forge dataset status` shows what has been captured, `forge dataset import` brings in runs from before this change, and `FORGE_DATASET=off` turns capture off.
|
|
146
|
+
|
|
147
|
+
### Added
|
|
148
|
+
|
|
149
|
+
- `forge dataset export` turns the run dataset into labeled decision cases in the System One format that decision-model benchmarks and fine-tuning tools read. Each case is a decision Jev makes in forge, labeled by what happened next: whether a task's first attempt passed review, whether a retry passed given the findings it was handed, whether the reviewer passed a diff (the reviewer's model and effort are part of the case), which undeclared files a task really edited, whether a review describes the diff it was given, and whether a memory fact concerned a file the task went on to edit. Hosted Jev's own answers are kept alongside as baselines. The export stays private in `~/.local/share/forge/exports/`.
|
|
150
|
+
- `forge dataset outcomes` records what integrated work became, from your repo's history alone: for every task an `integrate` brought in, how many of the lines it added are still there 14 days later (`--horizon-days` to change it) and whether a later commit reverted it. It only reads git objects, never your working tree or index, writes nothing for a horizon not yet reached, and adds nothing when run again. The export turns it into the first label no reviewer decides: whether a merged diff survived. Verification results now also name the commit they verified.
|
|
151
|
+
- `plan --shadow-qa <spec>[,<spec>...]` sends a sampled share of reviewed attempts (`--shadow-rate`, default 10%) to a second, independent reviewer of another model family, chosen only from the specs you list. Its verdict is recorded in the run dataset and never read by the run: it cannot change a status, a merge, memory or a retry, and a shadow that runs long is stopped soon after the main review finishes. `forge dataset status` reports how often the two reviewers agree, overall and per model pair, which tells you how much of a review verdict is the diff and how much is who reviewed it. The approval table shows the extra reviews before anything runs.
|
|
152
|
+
- `plan --explore E` (0 to 0.5, or `FORGE_ROUTE_EXPLORE`) now and then gives a task you left to forge another of the models you passed to that plan, instead of its pool's turn, and records each decision with its probability in the run dataset. Until now every outcome was the outcome of the model the router chose, so nothing could say whether another model would have done better; these recorded odds make that estimable later. Forge still never uses a model you did not name, explored rows are marked `*` in the approval table so the spend is visible before you approve, pinned rows are never touched, and without the flag a plan is exactly as before. A model change made by `retry --dwarf` is now recorded too, at the moment it happens, and Jev's routing calibration leaves explored tasks out.
|
|
153
|
+
|
|
154
|
+
### Changed
|
|
155
|
+
|
|
156
|
+
- The run dataset now records which forge produced every event: besides the release, a fingerprint of the exact scripts, references and registry a run used (`code_sha`) and the checkout's commit (`forge_rev`), taken when the execution starts and flagged if the code changed before it ended. Imported runs no longer carry the importer's version. The store upgrades itself on first use. `forge dataset export` uses this to label outcomes more honestly: an attempt that says nothing about the implementer (a reviewer timeout, a harness crash, no verdict) is left out and counted instead of letting a later success count as the first attempt's, reviews gain a severity label (`none` to `P0`) when every finding is graded, routing and retry states gain an explicit task-size block, and each case's metadata says which forge decided and ran it and how the task went (attempts, raw verdict, gate pass, human accept, why it stopped). The orchestrator now records the decompose level it split at even when the user named none.
|
|
157
|
+
|
|
158
|
+
### For contributors
|
|
159
|
+
|
|
160
|
+
- The fake agent CLI in the test suite numbers its calls under a lock, so two tasks sharing a scenario key no longer replay the same call twice. That race made `test_accept_that_cannot_merge_exits_6_and_records_nothing` fail intermittently under load.
|
|
161
|
+
|
|
162
|
+
## [0.3] - 2026-10-09
|
|
163
|
+
|
|
164
|
+
Forge's GPT aliases now run the GPT-6 generation, with a new astra alias for GPT-6 Astra, and Fractal can decide on its own when to split work further, with a redesigned run inspector to follow it.
|
|
165
|
+
|
|
166
|
+
### Highlights
|
|
167
|
+
|
|
168
|
+
- `sol` and `luna` now run GPT-6.1 Sol and GPT-6 Luna on codex and openclaude, and a new `astra` alias runs GPT-6 Astra, the frontier model. `terra` stays on GPT-5.6 Terra, which has no GPT-6 successor. Claude models run through opencode move to Opus 5.5, Sonnet 5.5, Fable 5.1 and Haiku 5.5.
|
|
169
|
+
|
|
170
|
+
### Added
|
|
171
|
+
|
|
172
|
+
- Fractal can now decide at every eligible node whether to split the work or implement it directly: `--fractal-auto-decompose` turns it on (opt-in; without it Fractal behaves as before) and `--fractal-planner <spec>` picks a separate model for those decisions. The planning decisions show up in reports.
|
|
173
|
+
|
|
174
|
+
### Changed
|
|
175
|
+
|
|
176
|
+
- The Fractal run dashboard and offline HTML reports are redesigned as a run inspector: it opens on progress and problems, lets you drill into task QA and history or a node's decisions, logs and changes, and refreshes in place every couple of seconds without losing your search, selection or scroll position.
|
|
177
|
+
|
|
178
|
+
## [0.2] - 2026-10-06
|
|
179
|
+
|
|
180
|
+
A failed review no longer stalls a run: tasks retry inside it, quota and auth stops pause without spending an attempt, and an optional whole-run review checks the combined result before it merges.
|
|
181
|
+
|
|
182
|
+
### Highlights
|
|
183
|
+
|
|
184
|
+
- Quota, auth, network and empty-output failures no longer cost an attempt or fail a task. The run pauses with the new status `INFRA` and exit code 8 (details in `tasks/<id>/infra.txt`), and `--infra-retries M` waits the limit out and carries on. A task whose review was cut off re-runs only the reviewer, not the implementer.
|
|
185
|
+
- A task whose reviewer says FAIL can now be retried inside the same run: `run --retry-failed N` re-queues it with the reviewer's findings while the other tasks keep going, instead of waiting for the whole run to end. It is off unless you pass it.
|
|
186
|
+
- An optional whole-run review looks at the combined result of all tasks, where per-task reviewers cannot see cross-task problems: `review <plan>` is read-only, and `integrate --final-review <spec>` blocks the merge on a FAIL. It spends reviewer quota only when you ask for it.
|
|
187
|
+
|
|
188
|
+
### Added
|
|
189
|
+
|
|
190
|
+
- `--qa-threshold Pn` accepts a reviewer FAIL made only of labelled findings milder than Pn, merges the work and records the findings in `known_issues.md`; unlabelled findings always block. `accept <plan> <task> --reason "..." --approved` makes the same decision for a single task by hand.
|
|
191
|
+
- Forge now warns before spending when the project defines test suites that `--verify` does not run, and re-runs a failing verify command once if the source did not change. A pass on the rerun is reported as FLAKY. `--verify-retries 0` turns the rerun off.
|
|
192
|
+
- Forge now records when an implementer commits on its own, switches branch, leaves background processes running (they are cleaned up) or promises later work. The notes go to `guard.txt`, the reviewer prompt and the run summary; they are warnings and never change a task's status.
|
|
193
|
+
- `split` moves tasks, with their attempts, findings and reviewed work, out of a plan into a new one, and `combine` joins plans whose tasks are all merged. Both refuse unsafe cases without changing anything and roll back on interrupt or conflict.
|
|
194
|
+
|
|
195
|
+
### Changed
|
|
196
|
+
|
|
197
|
+
- Default timeouts now scale with the model's effort (the reviewer and planner default of 3600s is up from 2700s), `registry.tsv` takes an optional per-model timeout column, and `--timeout` now also applies to decomposed runs.
|
|
198
|
+
|
|
199
|
+
### Fixed
|
|
200
|
+
|
|
201
|
+
- A reviewer verdict followed by a carriage return, or a memory note placed after the verdict line, no longer turns a PASS into UNKNOWN.
|
|
202
|
+
|
|
203
|
+
### For contributors
|
|
204
|
+
|
|
205
|
+
- `forge jev test-subset` now reports a usage error for a missing `{files}` placeholder or repo directory before asking for an API key.
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Giamme
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,195 @@
|
|
|
1
|
-
|
|
1
|
+
<h1 align="center">forge</h1>
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
<p align="center">
|
|
4
|
+
<b>Multi-model coding with a reviewer that reads the code: plan, build in parallel worktrees, review every real diff, merge only what passes.</b>
|
|
5
|
+
</p>
|
|
6
|
+
|
|
7
|
+
<p align="center">A skill for Claude Code, Codex, OpenCode, OpenClaude, Antigravity and Grok.</p>
|
|
8
|
+
|
|
9
|
+
<p align="center">
|
|
10
|
+
<a href="https://www.npmjs.com/package/@giamme/forge"><img src="https://img.shields.io/npm/v/@giamme/forge?label=npm" alt="npm version"></a>
|
|
11
|
+
<a href="https://github.com/Giamme/forge/releases"><img src="https://img.shields.io/github/v/release/Giamme/forge?label=release" alt="Latest release"></a>
|
|
12
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="License: MIT"></a>
|
|
13
|
+
<a href="CHANGELOG.md"><img src="https://img.shields.io/badge/changelog-what's%20new-blue" alt="Changelog"></a>
|
|
14
|
+
</p>
|
|
15
|
+
|
|
16
|
+
<p align="center">
|
|
17
|
+
<a href="#install">Install</a> ·
|
|
18
|
+
<a href="#quick-start">Quick start</a> ·
|
|
19
|
+
<a href="#how-it-works">How it works</a> ·
|
|
20
|
+
<a href="#forge-ui">forge-ui</a> ·
|
|
21
|
+
<a href="docs/README.md">Docs</a> ·
|
|
22
|
+
<a href="CHANGELOG.md">Changelog</a>
|
|
23
|
+
</p>
|
|
24
|
+
|
|
25
|
+
<p align="center">
|
|
26
|
+
<img src="ui/docs/media/hero.webp" width="880" alt="A decomposed run in forge-ui's 3D view: eight tasks on a plate in their planned waves, five merged in green and two running in amber, with a mark over each block for every review finding, then the same run on wall-clock time, where two first attempts that failed review show in red beside the retries that passed.">
|
|
27
|
+
</p>
|
|
28
|
+
<p align="center"><sub>The run from <a href="#quick-start">Quick start</a> in <a href="#forge-ui">forge-ui</a>: its tasks in the waves the plan gave them, then as they really ran, the failed attempts and their retries included. Demo data.</sub></p>
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
You hand forge a coding goal and pick the models. **Dwarves** implement it. A **qa** reviewer
|
|
33
|
+
reads each dwarf's real diff, never the dwarf's summary of its own work. A model grading its own
|
|
34
|
+
homework isn't a check: the second stage is worth having because it sees the code, not the claim.
|
|
35
|
+
|
|
36
|
+
- **Any model, any CLI.** `--dwarf sol:xhigh --qa opus` runs GPT through Codex and Claude Opus
|
|
37
|
+
through Claude Code in one command, from whichever CLI you are in. → [Models](docs/models.md)
|
|
38
|
+
- **Parallel when you want it.** `--decompose-level` splits the goal into tasks, routes each to a
|
|
39
|
+
model by difficulty, and runs them concurrently in isolated git worktrees.
|
|
40
|
+
→ [Decomposed runs](docs/decomposed-runs.md)
|
|
41
|
+
- **Only reviewed work merges.** Every task is reviewed on its own diff. A failed task keeps its
|
|
42
|
+
branch for a retry with the reviewer's findings, and only passing tasks merge.
|
|
43
|
+
→ [Reviews](docs/reviews.md)
|
|
44
|
+
- **You decide what gets spent and what lands.** A decomposed run shows you the task table and
|
|
45
|
+
stops before any dispatch. Solo changes stay uncommitted, decomposed work lands on your branch
|
|
46
|
+
only through `integrate --approved`, and forge never pushes. → [Safety](docs/safety.md)
|
|
47
|
+
- **It learns your repo.** What a run discovers goes into a small `.forge/memory.md`, and every
|
|
48
|
+
later run is given it. → [Project memory](docs/memory.md)
|
|
49
|
+
- **You can watch it live.** forge-ui draws every run in 3D, with diffs, findings and a model
|
|
50
|
+
leaderboard. → [forge-ui](docs/forge-ui.md)
|
|
51
|
+
|
|
52
|
+
## Install
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
npm install -g @giamme/forge
|
|
56
|
+
forge install # links forge into every agent CLI it finds
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Update with `npm install -g @giamme/forge@latest && forge install`. Or install from a git
|
|
60
|
+
checkout:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
git clone https://github.com/Giamme/forge.git ~/.claude/skills/forge
|
|
64
|
+
bash ~/.claude/skills/forge/scripts/forge-install.sh
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
forge runs on macOS and Linux. It needs bash, git, Python 3.9 or later, and the agent CLIs
|
|
68
|
+
you route models through; [the doctor](docs/install.md#check-what-forge-can-reach) shows which
|
|
69
|
+
ones it can reach. Foundry's `forge` shares the name: see
|
|
70
|
+
[the Foundry note](docs/install.md#from-npm).
|
|
71
|
+
|
|
72
|
+
→ [Install and update](docs/install.md)
|
|
73
|
+
|
|
74
|
+
## Quick start
|
|
75
|
+
|
|
76
|
+
Inside any agent CLI where forge is installed:
|
|
77
|
+
|
|
78
|
+
```text
|
|
79
|
+
/forge "add retry with exponential backoff to the HTTP client" --dwarf sol:high
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
That's the whole minimum. `sol` builds it at `high` effort on Codex, `opus` reviews the diff at
|
|
83
|
+
`xhigh` (the default reviewer), and you get back a summary of the change plus qa's findings.
|
|
84
|
+
|
|
85
|
+
```text
|
|
86
|
+
# pick both sides
|
|
87
|
+
/forge "fix the off-by-one in the pagination cursor" --dwarf luna:max --qa sol:xhigh
|
|
88
|
+
|
|
89
|
+
# the same model, through a different CLI
|
|
90
|
+
/forge "port the config loader to TOML" --dwarf sol:high:openclaude --qa opus
|
|
91
|
+
|
|
92
|
+
# let the dwarf leave the sandbox to install deps and run the suite
|
|
93
|
+
/forge "add integration tests against a real Postgres container" --dwarf sol:xhigh --yolo-dwarf
|
|
94
|
+
|
|
95
|
+
# several dwarves in parallel, routed by difficulty
|
|
96
|
+
/forge "add auth: middleware, token model, login route, and docs" \
|
|
97
|
+
--decompose-level medium \
|
|
98
|
+
--dwarf-high sol:xhigh --dwarf-low gemini:low:antigravity --dwarf luna:high
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
A decomposed run plans the tasks, rates and routes each one, then **shows you the table and
|
|
102
|
+
stops** before spending anything. Before an interactive run, forge also asks whether to use
|
|
103
|
+
[Fractal](docs/fractal.md) and whether to open [forge-ui](docs/forge-ui.md).
|
|
104
|
+
|
|
105
|
+
→ [Every flag, with more examples](docs/commands.md)
|
|
106
|
+
|
|
107
|
+
## How it works
|
|
108
|
+
|
|
109
|
+
```mermaid
|
|
110
|
+
flowchart LR
|
|
111
|
+
goal(["your goal"]) --> plan["plan: tasks and models,<br/>then you read the table"] --> dwarf["each task: a dwarf<br/>implements it in<br/>its own worktree"] --> qa["qa reads<br/>the real diff"]
|
|
112
|
+
qa -. "FAIL: retry with<br/>the findings" .-> dwarf
|
|
113
|
+
qa -- PASS --> merge["merged into<br/>integration"] --> deliver(["verification, final review,<br/>integrate --approved"])
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Without `--decompose-level`, the whole goal is one task: one dwarf, and one review of its diff.
|
|
117
|
+
With optional [Fractal execution](docs/fractal.md), an implementer can delegate bounded child
|
|
118
|
+
tasks inside the paths it owns. forge still reviews the combined diff.
|
|
119
|
+
|
|
120
|
+
Every model is named by one spec, `alias:effort:harness`. Each field after the alias is optional:
|
|
121
|
+
|
|
122
|
+
| Spec | Means |
|
|
123
|
+
|---|---|
|
|
124
|
+
| `sol` | `sol` at the role's default effort, on its default CLI (codex) |
|
|
125
|
+
| `sol:xhigh` | the same model at `xhigh` effort |
|
|
126
|
+
| `sol:xhigh:openclaude` | the same model and effort, run through `openclaude` |
|
|
127
|
+
|
|
128
|
+
Either role can run through any of six agent CLIs:
|
|
129
|
+
|
|
130
|
+
| | codex | claude | openclaude | opencode | antigravity | grok |
|
|
131
|
+
|---|---|---|---|---|---|---|
|
|
132
|
+
| binary | `codex` | `claude` | `openclaude` | `opencode` | `agy` | `grok` |
|
|
133
|
+
|
|
134
|
+
→ [Effort ladders, clamping, sandboxes and adding a model](docs/models.md)
|
|
135
|
+
|
|
136
|
+
## forge-ui
|
|
137
|
+
|
|
138
|
+
forge-ui is an opt-in, read-only local page that watches any run live, drawn as a 3D map. It
|
|
139
|
+
shows every run on one plate, tasks in their waves and as they really ran, and the prompt, model,
|
|
140
|
+
timings and P0–P3 findings behind every step. forge offers it at the start of every new run and
|
|
141
|
+
never opens it without a yes.
|
|
142
|
+
|
|
143
|
+
<p align="center">
|
|
144
|
+
<img src="ui/docs/media/cloud.webp" width="880" alt="The run cloud: about seventy runs as blocks in five project spirals, orbited, then laid out as galaxies and as a timeline with one lane per project.">
|
|
145
|
+
</p>
|
|
146
|
+
|
|
147
|
+
<table>
|
|
148
|
+
<tr>
|
|
149
|
+
<td width="50%" valign="top"><img src="ui/docs/media/run.png" alt="forge-ui's window: the runs list, a run's eight tasks on a 3D plate in their planned waves with dependency cables, finding marks and the finish nodes, and the run's inspector."><br><b>A run, as planned.</b> Tasks in their waves with dependency cables, ending in integration, verification and the final review. Click a task to open its attempts, dwarves and reviews.</td>
|
|
150
|
+
<td width="50%" valign="top"><img src="ui/docs/media/execution.png" alt="The same run on wall-clock time: one block per attempt, failed attempts in red with their finding marks, a plane marking now, the concurrency strip, and the inspector's timeline and critical path."><br><b>And as it ran.</b> One block per attempt in the lane it ran in, failed ones in red, against <code>--max-parallel</code>; the inspector names the critical path.</td>
|
|
151
|
+
</tr>
|
|
152
|
+
<tr>
|
|
153
|
+
<td valign="top"><img src="ui/docs/media/diff.png" alt="The diff a reviewer read, with a P1 finding pinned under the line comparing token digests and a P2 finding under the revoke function."><br><b>Findings on their lines.</b> Any diff of a task, or exactly the diff one reviewer read, with each P0–P3 finding under the line it names.</td>
|
|
154
|
+
<td valign="top"><img src="ui/docs/media/models.png" alt="The model leaderboard: per implementer, tasks, first-attempt pass rate with a 95% interval bar, retries, escalations, timeouts, accepts, median model time and tokens."><br><b>Models, measured on your runs.</b> First-attempt pass rate with a confidence interval, retries, escalations, time and tokens; reviewers' verdicts and shadow agreement.</td>
|
|
155
|
+
</tr>
|
|
156
|
+
<tr>
|
|
157
|
+
<td valign="top"><img src="ui/docs/media/files.png" alt="The files heatmap: files grouped by directory against tasks grouped by wave, with planned, touched, drift and same-wave overlap marks."><br><b>Who touched which file.</b> Planned, changed, drifted into or conflicted on, wave by wave; a cell opens that file's diff.</td>
|
|
158
|
+
<td valign="top"><img src="ui/docs/media/map.png" alt="The map of every run: one block per run in a district per repo, green when done, amber while running, red with failures."><br><b>Every run, by repo.</b> One block per run with a lamp per task, filtered by state, repo, model, age or review verdict.</td>
|
|
159
|
+
</tr>
|
|
160
|
+
</table>
|
|
161
|
+
|
|
162
|
+
To look around before your first run, open it on made-up runs (the ones pictured here) from a
|
|
163
|
+
git checkout:
|
|
164
|
+
|
|
165
|
+
```sh
|
|
166
|
+
python3 ui/tools/demo_runs.py /tmp/forge-demo
|
|
167
|
+
(cd ui && python3 -m forge_ui --no-registry --root /tmp/forge-demo --open)
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
→ [forge-ui](docs/forge-ui.md)
|
|
171
|
+
|
|
172
|
+
## Documentation
|
|
173
|
+
|
|
174
|
+
| Page | What's in it |
|
|
175
|
+
|---|---|
|
|
176
|
+
| [Install and update](docs/install.md) | npm or a git checkout, the doctor, what the installer does per CLI, updating |
|
|
177
|
+
| [Command reference](docs/commands.md) | every `/forge` flag with examples, the `forge` command, the runner scripts and their exit codes |
|
|
178
|
+
| [Models, efforts and harnesses](docs/models.md) | the `alias:effort:harness` spec, effort ladders and clamping, yolo mode, the model registry |
|
|
179
|
+
| [Decomposed runs](docs/decomposed-runs.md) | decompose levels and difficulty, routing, waves, capsules, planning, resuming, split and combine, `tasks.tsv` |
|
|
180
|
+
| [Reviews, retries and recovery](docs/reviews.md) | verdicts, retries, infrastructure stops, severity thresholds, accepting, whole-run and shadow reviews, guards, verification, fail-first |
|
|
181
|
+
| [Fractal execution](docs/fractal.md) | the optional nested-execution backend: install, run, inspect, bounds and recovery |
|
|
182
|
+
| [Project memory](docs/memory.md) | `.forge/memory.md` and the ledger, what each role is told, whether to commit it, spend |
|
|
183
|
+
| [forge-ui](docs/forge-ui.md) | the run viewer: commands, the shared server, environment variables, privacy |
|
|
184
|
+
| [Run dataset](docs/dataset.md) | every run's inputs and outcomes, kept for training decision models |
|
|
185
|
+
| [Run artifacts](docs/artifacts.md) | the run directory, a plan's files, worktrees and branches |
|
|
186
|
+
| [Safety](docs/safety.md) | what forge never does to your branch, preconditions, execution integrity |
|
|
187
|
+
| [Ripwire and Jev](docs/add-ons.md) | optional repository context and advisory judgments |
|
|
188
|
+
| [Troubleshooting](docs/troubleshooting.md) | symptoms, causes and fixes |
|
|
189
|
+
| [Development](docs/development.md) | repository layout, offline checks, runtime measurement |
|
|
190
|
+
|
|
191
|
+
## License
|
|
192
|
+
|
|
193
|
+
forge is released under the [MIT License](LICENSE). forge-ui's 3D view ports code from
|
|
194
|
+
archivision, iso-glow and iso-figure, all MIT-licensed; their notices are in
|
|
195
|
+
[ui/THIRD_PARTY_NOTICES.md](ui/THIRD_PARTY_NOTICES.md).
|
package/SKILL.md
ADDED
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: forge
|
|
3
|
+
description: Dispatch implementation and independent diff review to user-selected models through local agent CLIs. Use for /forge or explicit requests to delegate coding or review; explanation-only questions do not authorize dispatch.
|
|
4
|
+
argument-hint: '"<goal>" --dwarf <alias>[:effort[:harness]] [--qa <alias>] [--planner <alias>] [--yolo-dwarf] [--yolo-qa] [--decompose-level low|medium|high] [--fractal|--no-fractal|--fractal-auto-decompose] [--fractal-planner <spec>] [--jev|--no-jev] [--jev-shadow] [--no-memory] [--no-ripwire] [--ui|--no-ui] [--timeout <seconds>] [--retry-failed <n>] [--escalate-after <n>] [--qa-threshold P0|P1|P2|P3] [--shadow-qa <spec>[,<spec>]] [--shadow-rate <0..1>] [--explore <0..0.5>] [--fail-first]'
|
|
5
|
+
allowed-tools: [Bash, Read]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Forge
|
|
9
|
+
|
|
10
|
+
Run an implementer (**dwarf**) followed by an independent **QA** review of its actual
|
|
11
|
+
changes. Use the scripts for dispatch and capture; do not reconstruct harness commands.
|
|
12
|
+
For explanation requests, explain the workflow without spending model quota.
|
|
13
|
+
|
|
14
|
+
## Choices and boundaries
|
|
15
|
+
|
|
16
|
+
- A dwarf model must be chosen by the user. Ask when it is missing; never invent one.
|
|
17
|
+
- QA defaults to `opus`; effort defaults to `medium` for dwarf and `xhigh` for QA/planner.
|
|
18
|
+
- Specs are `alias[:effort[:harness]]`, e.g. `sol:xhigh:openclaude`. Resolve using
|
|
19
|
+
[registry.tsv](registry.tsv); report any effort clamp. Literal model IDs require a harness.
|
|
20
|
+
- Bypass flags apply only when explicitly requested for that role (`--yolo-dwarf`,
|
|
21
|
+
`--yolo-qa`). Bash access can mutate files despite disabled editing tools.
|
|
22
|
+
- Ask **“Use Fractal for this run? [y/N]”** once before an interactive run, unless
|
|
23
|
+
the user supplied `--fractal`, `--no-fractal`, or `--fractal-auto-decompose`. Pass the answer explicitly to
|
|
24
|
+
the runner. Unattended runs default off. Installation never activates Fractal.
|
|
25
|
+
Resume and explicit retry retain the recorded choice. Read [Fractal](references/fractal.md)
|
|
26
|
+
when selected; Fractal completion never substitutes for Forge QA acceptance.
|
|
27
|
+
- Ask **“Open forge-ui to watch this run? [y/N]”** once at the start of every new
|
|
28
|
+
interactive run, unless the user supplied `--ui` or `--no-ui`. Pass the answer to `plan`
|
|
29
|
+
or `forge-solo.sh` and relay the `forge-ui:` URL they print. Never ask on resume or retry;
|
|
30
|
+
unattended runs leave it off. A UI failure never changes the run. Read [forge-ui](references/ui.md).
|
|
31
|
+
- A finished solo run, `plan` or `run` may end with one stderr line, `forge X is available
|
|
32
|
+
(you have Y): <command>` (checked at most once a day; the command is `forge update`, or npm's
|
|
33
|
+
for an npm installation). Relay it once; run the update only when the user asks (that
|
|
34
|
+
command, then restart the harness session). `FORGE_UPDATE_CHECK=off` silences it.
|
|
35
|
+
- `--fractal-auto-decompose` enables automatic split-or-atomic decisions before each
|
|
36
|
+
eligible node implements. Include the effective planner and bounds in the approval
|
|
37
|
+
preview; child decisions need no additional user approval inside those bounds.
|
|
38
|
+
Forward `--fractal-planner <spec>` to the runner. When omitted, forward a selected
|
|
39
|
+
top-level `--planner` as `--fractal-planner`; otherwise each task's initial root model
|
|
40
|
+
plans all its descendants. Ordinary `--fractal` keeps worker-requested decomposition.
|
|
41
|
+
- One implementer per tree. Solo edits remain uncommitted. Decomposed runs may commit
|
|
42
|
+
and merge on Forge task and integration branches. Updating the user's branch requires
|
|
43
|
+
separate authorization and `integrate --approved`. Never push automatically.
|
|
44
|
+
- Show the decomposed task/model table and wait for approval before dispatch. Retry only
|
|
45
|
+
when requested; preserve failed task branches and worktrees. `run --retry-failed N`,
|
|
46
|
+
`--infra-retries M`, `--qa-threshold Pn` and `plan --shadow-qa` (a second review of a
|
|
47
|
+
sampled share of attempts, never read, to measure reviewer agreement) are opt-in: offer
|
|
48
|
+
them, never add them unprompted. So is `--fail-first` (solo or `plan`): the change's new
|
|
49
|
+
tests also run without its non-test changes, and the reviewer is asked to report any that
|
|
50
|
+
still pass as P2; it needs `.forge/verify-subset` and never changes a result by itself. `plan` explores by default (`--explore 0.2`: now and then
|
|
51
|
+
a row left to forge gets its neighbouring tier's model from the user's pools, odds
|
|
52
|
+
recorded for `forge dataset routing`); say so with the table and pass `--explore 0` when
|
|
53
|
+
the user declines. Retries follow a policy: after three failed attempts on one model the
|
|
54
|
+
next goes a tier up in the user's pools, or the task stops with a split suggestion
|
|
55
|
+
(`--escalate-after N`, `0` = off); an explicit `retry --dwarf` overrides it. `accept`
|
|
56
|
+
overrides a reviewer, so run it (with the user's `--reason` and
|
|
57
|
+
`--approved`) only on the user's explicit instruction; never self-accept.
|
|
58
|
+
- Exit `8` / status `INFRA` is a quota, auth or network stop, not a task failure and no
|
|
59
|
+
attempt was spent: report it, say what to fix, and re-run only when told. For a multi-task
|
|
60
|
+
plan, offer the opt-in whole-run review (`review`, or `integrate --final-review <qa-spec>`);
|
|
61
|
+
it spends QA quota, so never run it unprompted.
|
|
62
|
+
- Jev is optional, off unless configured and enabled, and always advisory: it never
|
|
63
|
+
changes a task's tier, never selects a model, and never substitutes for QA acceptance
|
|
64
|
+
or repository verification (`--jev-act` is still accepted and changes nothing). Read
|
|
65
|
+
[Jev](references/jev.md) when selected.
|
|
66
|
+
|
|
67
|
+
## Common workflow
|
|
68
|
+
|
|
69
|
+
1. Inspect the repository and existing work. State the goal, selected models and any bypass
|
|
70
|
+
flags. Keep complete requirements in `prompt.md`; short titles are insufficient for QA.
|
|
71
|
+
2. Preflight every selected role before the first model call, including an optional planner:
|
|
72
|
+
`bash <skill_dir>/scripts/forge-dispatch.sh doctor --spec <spec> --role <role>`.
|
|
73
|
+
This checks resolution, executable and advertised flags without model quota. Authentication,
|
|
74
|
+
quota and live model availability remain unverified; a live probe must be explicitly requested.
|
|
75
|
+
3. Use [solo runs](references/solo.md) for one implementation. For `--decompose-level
|
|
76
|
+
low|medium|high`, read [decomposition](references/decompose.md), prepare the task table,
|
|
77
|
+
obtain approval, then use `forge-parallel.sh plan/run`. Write the user's request verbatim
|
|
78
|
+
to `request.md` in the run or plan dir, and pass `--decompose-level` to `plan` with the
|
|
79
|
+
level you actually split at, even when the user named none. Keep each task small and end
|
|
80
|
+
its `prompt.md` with an `## Acceptance criteria` section of checks the reviewer can verify;
|
|
81
|
+
`plan` names tasks over 7K characters or 9 files, where first attempts mostly fail review.
|
|
82
|
+
Pass the recorded Fractal choice and requested limits to the execution runner; tier
|
|
83
|
+
pools go to solo directly or decomposed `plan`. Fractal does not replace planner dispatch.
|
|
84
|
+
`--timeout` applies to decomposed runs too (kept in the plan).
|
|
85
|
+
4. Runners default to summary output. Read each `dwarf.last` and `qa.last` once; retrieve raw
|
|
86
|
+
logs only when needed. Report findings faithfully, and identify verification
|
|
87
|
+
limits. Source or review mutations invalidate acceptance. Missing verdicts are UNKNOWN.
|
|
88
|
+
An absent repository verification command means UNVERIFIED, even when QA passes.
|
|
89
|
+
|
|
90
|
+
For a Fractal run, report its managed run ID with the Forge result. Use
|
|
91
|
+
`<skill_dir>/forge fractal runs --repo <repo> --json` and `tree RUN_ID --json` to discover
|
|
92
|
+
run/task/node IDs; use `status`, `logs` or `open` for inspection. Run-level `resume`
|
|
93
|
+
recovers the recorded pipeline; scoped controls affect only the selected execution scope.
|
|
94
|
+
Preserve failed work and state the distinction between execution completion, QA acceptance
|
|
95
|
+
and verification. Read the Fractal reference before retrying, changing limits or exporting
|
|
96
|
+
a portable report.
|
|
97
|
+
|
|
98
|
+
Ripwire context is automatic when compatible; `--no-ripwire` or `FORGE_RIPWIRE=off`
|
|
99
|
+
disables it. Pass opt-out to every role, including a separately dispatched planner.
|
|
100
|
+
Runners offer installation once before agents; dispatch never prompts. See
|
|
101
|
+
[Ripwire](references/ripwire.md) for installation, limits and fallback behavior.
|
|
102
|
+
|
|
103
|
+
## Conditional references
|
|
104
|
+
|
|
105
|
+
- [Planning](references/planning.md): read for `--planner` or an agreed approach.
|
|
106
|
+
- [Solo](references/solo.md): command, snapshots, artifacts, exit codes and native-review limits.
|
|
107
|
+
- [Decomposition](references/decompose.md): task schema, routing, waves, retries and integration.
|
|
108
|
+
- [Harnesses](references/harnesses.md): invocation troubleshooting or adding a harness.
|
|
109
|
+
- [Fractal](references/fractal.md): opt-in nested execution, installation, limits,
|
|
110
|
+
recovery, managed run commands, read-only dashboard and portable reports.
|
|
111
|
+
- [Memory](references/memory.md): `.forge/` learning and spend records; `--no-memory` disables
|
|
112
|
+
them. Memory is written to the user's repository but excluded from implementation diffs.
|
|
113
|
+
- [forge-ui](references/ui.md): the opt-in local run viewer; `forge ui open|status|stop`,
|
|
114
|
+
`FORGE_UI=off`, one shared server, privacy.
|
|
115
|
+
- [Dataset](references/dataset.md): every run's inputs and outcomes, kept outside the repo
|
|
116
|
+
for training decision models; `forge dataset status|import|export|outcomes`, `FORGE_DATASET=off`.
|
|
117
|
+
- [Jev](references/jev.md): optional TypeSafe System One judgments for routing, test
|
|
118
|
+
selection and memory curation; off by default. Chosen sites can go to Cloudflare's hosted
|
|
119
|
+
Clef instead (`forge jev setup --provider cloudflare`, `forge jev sites`, `FORGE_CLEF=off`).
|
|
120
|
+
|
|
121
|
+
The common invocation is `/forge "<goal>" --dwarf <spec> [--qa <spec>]`. Advanced flags and
|
|
122
|
+
commands live with their mode reference. Use [offline tests and evaluation cases](tests/README.md)
|
|
123
|
+
when maintaining this skill.
|
package/VERSION
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
0.9
|
package/docs/README.md
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# forge documentation
|
|
2
|
+
|
|
3
|
+
[← README](../README.md)
|
|
4
|
+
|
|
5
|
+
The README is the short overview. These pages hold the rest, one topic each.
|
|
6
|
+
|
|
7
|
+
| Page | What's in it |
|
|
8
|
+
|---|---|
|
|
9
|
+
| [Install and update](install.md) | npm or a git checkout, the doctor, what the installer does per CLI, updating |
|
|
10
|
+
| [Command reference](commands.md) | every `/forge` flag with examples, the `forge` command, the runner scripts and their exit codes |
|
|
11
|
+
| [Models, efforts and harnesses](models.md) | the `alias:effort:harness` spec, effort ladders and clamping, yolo mode, the model registry |
|
|
12
|
+
| [Decomposed runs](decomposed-runs.md) | decompose levels and difficulty, routing, waves, capsules, planning, resuming, split and combine, `tasks.tsv` |
|
|
13
|
+
| [Reviews, retries and recovery](reviews.md) | verdicts, retries, infrastructure stops, severity thresholds, accepting, whole-run and shadow reviews, guards, verification, fail-first |
|
|
14
|
+
| [Fractal execution](fractal.md) | the optional nested-execution backend: install, run, inspect, bounds and recovery |
|
|
15
|
+
| [Project memory](memory.md) | `.forge/memory.md` and the ledger, what each role is told, whether to commit it, spend |
|
|
16
|
+
| [forge-ui](forge-ui.md) | the run viewer: commands, the shared server, environment variables, privacy |
|
|
17
|
+
| [Run dataset](dataset.md) | every run's inputs and outcomes, kept for training decision models |
|
|
18
|
+
| [Run artifacts](artifacts.md) | the run directory, a plan's files, worktrees and branches |
|
|
19
|
+
| [Safety](safety.md) | what forge never does to your branch, preconditions, execution integrity |
|
|
20
|
+
| [Ripwire and Jev](add-ons.md) | optional repository context and advisory judgments |
|
|
21
|
+
| [Troubleshooting](troubleshooting.md) | symptoms, causes and fixes |
|
|
22
|
+
| [Development](development.md) | repository layout, offline checks, runtime measurement |
|
|
23
|
+
|
|
24
|
+
Deeper material that the skill itself reads lives in [`references/`](../references/): the
|
|
25
|
+
per-harness details, the decomposition and Fractal contracts, memory internals, Jev and the
|
|
26
|
+
dataset schema.
|