@giamme/forge 0.8.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +50 -0
- package/README.md +106 -1766
- package/SKILL.md +22 -3
- package/VERSION +1 -1
- package/docs/README.md +26 -0
- package/docs/add-ons.md +33 -0
- package/docs/artifacts.md +84 -0
- package/docs/commands.md +365 -0
- package/docs/dataset.md +29 -0
- package/docs/decomposed-runs.md +310 -0
- package/docs/development.md +95 -0
- package/docs/forge-ui.md +128 -0
- package/docs/fractal.md +181 -0
- package/docs/install.md +160 -0
- package/docs/memory.md +156 -0
- package/docs/models.md +118 -0
- package/docs/reviews.md +245 -0
- package/docs/safety.md +55 -0
- package/docs/troubleshooting.md +54 -0
- package/forge +20 -0
- package/package.json +3 -2
- package/references/dataset.md +18 -1
- package/references/decompose.md +20 -2
- package/references/fractal.md +1 -1
- package/references/harnesses.md +18 -6
- package/references/solo.md +1 -1
- package/references/ui.md +27 -5
- package/scripts/forge-config.py +347 -0
- package/scripts/forge-control.py +645 -0
- package/scripts/forge-dispatch.sh +7 -1
- package/scripts/forge-issues.py +806 -0
- package/scripts/forge-lib-task.sh +234 -35
- package/scripts/forge-parallel-options.sh +3 -1
- package/scripts/forge-parallel.sh +3 -2
- package/scripts/forge-route.py +184 -27
- package/scripts/forge-runs-rm.py +840 -0
- package/scripts/forge-runs.py +139 -0
- package/scripts/forge-runtime.py +59 -6
- package/scripts/forge-schedule.py +227 -11
- package/scripts/forge-ui.py +85 -19
- package/scripts/forge-version.py +54 -12
- package/scripts/forge_dataset/__init__.py +77 -6
- package/scripts/forge_dataset/capture.py +5 -2
- package/scripts/forge_dataset/harvest.py +23 -1
- package/scripts/forge_dataset/importer.py +420 -18
- package/scripts/forge_fractal/__init__.py +2 -1
- package/scripts/forge_registry.py +168 -16
- package/ui/README.md +116 -23
- package/ui/THIRD_PARTY_NOTICES.md +108 -0
- package/ui/docs/api.md +283 -23
- package/ui/forge_ui/__main__.py +10 -2
- package/ui/forge_ui/control.py +575 -0
- package/ui/forge_ui/export.py +33 -9
- package/ui/forge_ui/fractal.py +1 -2
- package/ui/forge_ui/reader.py +121 -8
- package/ui/forge_ui/server.py +135 -5
- package/ui/forge_ui/static/D-DINCondensed-Bold.woff2 +0 -0
- package/ui/forge_ui/static/app.js +358 -131
- package/ui/forge_ui/static/cloud3d.js +3 -2
- package/ui/forge_ui/static/cloudui.js +8 -5
- package/ui/forge_ui/static/controls.js +619 -0
- package/ui/forge_ui/static/convo.js +295 -0
- package/ui/forge_ui/static/datatable.js +232 -0
- package/ui/forge_ui/static/diffview.js +29 -10
- package/ui/forge_ui/static/findings3d.js +134 -0
- package/ui/forge_ui/static/findingsview.js +412 -0
- package/ui/forge_ui/static/graph.js +3 -3
- package/ui/forge_ui/static/icons.js +94 -0
- package/ui/forge_ui/static/index.html +31 -13
- package/ui/forge_ui/static/iso.js +70 -19
- package/ui/forge_ui/static/leaderboard.js +44 -48
- package/ui/forge_ui/static/panel.js +92 -36
- package/ui/forge_ui/static/rundelete.js +388 -0
- package/ui/forge_ui/static/runmodel.js +17 -1
- package/ui/forge_ui/static/scene3d.js +518 -43
- package/ui/forge_ui/static/search.js +2 -1
- package/ui/forge_ui/static/settings.js +399 -0
- package/ui/forge_ui/static/shell.js +137 -0
- package/ui/forge_ui/static/stagectl.js +150 -0
- package/ui/forge_ui/static/style.css +1099 -469
- package/ui/forge_ui/static/theme.js +25 -0
- package/ui/forge_ui/static/uikit.js +161 -0
- package/ui/forge_ui/transcripts.py +608 -0
|
@@ -0,0 +1,310 @@
|
|
|
1
|
+
# Decomposed runs
|
|
2
|
+
|
|
3
|
+
[← README](../README.md) · [All docs](README.md)
|
|
4
|
+
|
|
5
|
+
Split one goal into tasks, route each to a model by difficulty, and run them in parallel in isolated worktrees, merging only the tasks that pass review.
|
|
6
|
+
|
|
7
|
+
`--decompose-level low|medium|high` turns one dwarf into many. Without the flag, nothing
|
|
8
|
+
changes — one dwarf, one QA pass.
|
|
9
|
+
|
|
10
|
+
## Two independent axes
|
|
11
|
+
|
|
12
|
+
Easy to conflate, so: **decompose-level** is how finely the *goal* is split. **difficulty**
|
|
13
|
+
is how hard each resulting *task* is.
|
|
14
|
+
|
|
15
|
+
| `--decompose-level` | Split |
|
|
16
|
+
|---|---|
|
|
17
|
+
| `low` | only where the goal has obviously independent pieces; 2–3 coarse tasks |
|
|
18
|
+
| `medium` | one task per coherent unit (module, feature slice); typically 4–6 |
|
|
19
|
+
| `high` | finest split where each task is still independently reviewable *and* independently correct; typically 7–12 |
|
|
20
|
+
|
|
21
|
+
| `difficulty` | Means |
|
|
22
|
+
|---|---|
|
|
23
|
+
| `low` | mechanical and local: rename, move, docs, a test for behaviour that already exists |
|
|
24
|
+
| `medium` | self-contained implementation against a clear spec; bounded blast radius |
|
|
25
|
+
| `high` | needs design judgment, cross-cutting behaviour, subtle edge cases, state or concurrency |
|
|
26
|
+
|
|
27
|
+
Difficulty is about **judgment required, not diff size**. A five-hundred-line mechanical
|
|
28
|
+
rename is `low`. A ten-line concurrency fix is `high`. Getting this backwards sends the cheap
|
|
29
|
+
model at the subtle problem and the expensive one at the boilerplate — worse than not routing
|
|
30
|
+
at all.
|
|
31
|
+
|
|
32
|
+
## Routing dwarves to tasks
|
|
33
|
+
|
|
34
|
+
Per-task assignment can't be expressed when you type the command, because the tasks don't
|
|
35
|
+
exist yet. Naming a dwarf per task would mean guessing how the goal will split. So you map
|
|
36
|
+
**model tiers to difficulty tiers**, once, in advance — three layers, each optional:
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
# 1. one dwarf for everything
|
|
40
|
+
--dwarf sol:xhigh
|
|
41
|
+
|
|
42
|
+
# 2. per-tier routing, with plain --dwarf as the fallback for any tier not named
|
|
43
|
+
--dwarf-high sol:xhigh --dwarf-medium luna:high --dwarf-low gemini:low:antigravity
|
|
44
|
+
--dwarf sonnet:medium
|
|
45
|
+
|
|
46
|
+
# 3. pool several models within one tier — round-robined across tasks
|
|
47
|
+
--dwarf-high sol:xhigh,terra:xhigh
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
`--qa`, `--qa-high`, `--qa-medium`, `--qa-low` mirror it exactly. QA has no tier rules by
|
|
51
|
+
default and falls back to `opus`.
|
|
52
|
+
|
|
53
|
+
Resolution order per task: **explicit column value → the tier's rule → plain `--dwarf` →
|
|
54
|
+
`UNASSIGNED`**. An `UNASSIGNED` task makes `plan` exit `2` and `run` refuse outright. forge
|
|
55
|
+
does not invent a model when you haven't said which one should spend your quota.
|
|
56
|
+
|
|
57
|
+
Layer three is **the approval gate**: after decomposition, before any dispatch, you get the
|
|
58
|
+
task table with each task's difficulty and resolved dwarf/qa. Retier a task ("2 is actually
|
|
59
|
+
medium") and it re-routes per your rules, or override a single row's dwarf outright. This is
|
|
60
|
+
also the cost gate — `high` on a large goal is 20+ dispatches, which should be visible before
|
|
61
|
+
it's spent.
|
|
62
|
+
|
|
63
|
+
**Exploration is on by default** (`--explore 0.2`):
|
|
64
|
+
- **Which rows:** only rows left to forge, never one whose dwarf column you filled.
|
|
65
|
+
- **What changes:** now and then such a row gets its *neighbouring* tier's spec from your own
|
|
66
|
+
pools: the next cheaper tier for medium and high rows, the next stronger for low rows.
|
|
67
|
+
Pools that differ only in effort (`--dwarf-low sol --dwarf-high sol:xhigh`) compare the
|
|
68
|
+
two efforts.
|
|
69
|
+
- **How often:** about one row in ten at the default.
|
|
70
|
+
- **What you see:** each explored row is marked `*` in the table, before anything is spent.
|
|
71
|
+
- **What it's for:** `forge dataset routing` reads the draws back as pass rates per model and
|
|
72
|
+
effort. Pass `--explore 0` to turn it off.
|
|
73
|
+
|
|
74
|
+
See [decompose.md](../references/decompose.md#exploratory-routing-on-by-default).
|
|
75
|
+
|
|
76
|
+
## Waves, and what actually bounds parallelism
|
|
77
|
+
|
|
78
|
+
A **wave** is the largest set of tasks whose dependencies are all satisfied by earlier waves
|
|
79
|
+
**and** whose declared `files` sets are pairwise disjoint.
|
|
80
|
+
|
|
81
|
+
Disjointness — not the task count — is the real bound. Two dwarves editing one file produce a
|
|
82
|
+
conflict no reviewer can untangle and no merge can resolve honestly, so they land in different
|
|
83
|
+
waves however the goal was decomposed. When that happens forge says so:
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
deferrals (serialized to protect the merge):
|
|
87
|
+
wave 1: lint deferred (file overlap with a task already in this wave)
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
That line matters. A decomposition that looks parallel but serializes at run time should
|
|
91
|
+
announce it, rather than leaving you wondering why eight tasks took eight rounds. When most
|
|
92
|
+
tasks overlap, the fix is a better split — not more parallelism.
|
|
93
|
+
|
|
94
|
+
Waves are a planning preview. A Python 3 standard-library coordinator starts eligible tasks
|
|
95
|
+
as prerequisites merge and capacity becomes available, while excluding overlapping active
|
|
96
|
+
paths. Merges are serialized. Each task pins the integration commit at dispatch; retries keep
|
|
97
|
+
that baseline, and QA sees only baseline membership recorded for that task.
|
|
98
|
+
|
|
99
|
+
## Fresh session per task, plus a capsule
|
|
100
|
+
|
|
101
|
+
Every dispatch is a **clean slate** — forge never passes `--continue`, `--resume`, or
|
|
102
|
+
`codex exec resume`. One task's confused turn can't poison another, and each task's context
|
|
103
|
+
stays small.
|
|
104
|
+
|
|
105
|
+
Clean slates alone would leave every dwarf blind to the wider run, so a short `capsule.md` is
|
|
106
|
+
generated from the task's pinned baseline and preserved for both implementation and QA:
|
|
107
|
+
|
|
108
|
+
```
|
|
109
|
+
# forge run <run-id>
|
|
110
|
+
Goal: <overall goal>
|
|
111
|
+
You are doing ONE task of <N>. Do not do the others' work.
|
|
112
|
+
|
|
113
|
+
## Your task
|
|
114
|
+
tokens — Add the token model difficulty: medium
|
|
115
|
+
Files you own: src/auth/token.py
|
|
116
|
+
|
|
117
|
+
## Run status
|
|
118
|
+
id diff status files
|
|
119
|
+
middleware high MERGED src/auth/middleware.py
|
|
120
|
+
tokens medium THIS ONE src/auth/token.py
|
|
121
|
+
docs low NOT IN BASE README.md
|
|
122
|
+
|
|
123
|
+
## Ground rules
|
|
124
|
+
- Your branch already contains every MERGED task's work — do not reimplement it.
|
|
125
|
+
- Files owned by other tasks are off limits; another dwarf is editing them now.
|
|
126
|
+
- If your task truly needs a change in someone else's file, say so in your final
|
|
127
|
+
message instead of making it.
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
It's written **per task**, not once per run — tasks in a wave execute concurrently, so a
|
|
131
|
+
single shared capsule file is a race in which each task overwrites it and reads back whichever
|
|
132
|
+
sibling wrote last, handing a dwarf someone else's task description.
|
|
133
|
+
|
|
134
|
+
For QA the capsule additionally scopes the review: code from `MERGED` tasks is already in the
|
|
135
|
+
base and is not this task's bug. Without that, a wave-2 reviewer reports wave-1 code as
|
|
136
|
+
defects in the diff it was handed.
|
|
137
|
+
|
|
138
|
+
## Planning
|
|
139
|
+
|
|
140
|
+
A dwarf that is handed only an objective plans its own approach, silently, inside its own
|
|
141
|
+
run. Usually fine — but in a decomposed run the capsule prevents two dwarves editing one
|
|
142
|
+
*file* and does nothing about them inventing incompatible *interfaces* at a shared seam.
|
|
143
|
+
Both stay in their lane, both pass their own QA, and the mismatch surfaces at integration.
|
|
144
|
+
|
|
145
|
+
So forge plans before it dispatches. By default the orchestrator does it and writes the
|
|
146
|
+
approach into each task's prompt — no extra dispatch. `--planner <spec>` hands that stage to
|
|
147
|
+
a named model instead:
|
|
148
|
+
|
|
149
|
+
```bash
|
|
150
|
+
/forge "migrate the API layer to gRPC" --decompose-level medium \
|
|
151
|
+
--planner sol:xhigh --dwarf-high sol:high --dwarf luna:high
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
**One planner dispatch per run, never one per task.** That is the whole point: a single mind
|
|
155
|
+
designs both sides of every seam, where N independent planners would recreate the problem.
|
|
156
|
+
Effort defaults to `xhigh`, since a bad plan is executed at full price by every dwarf
|
|
157
|
+
downstream of it. The planner runs with QA's permission profile — it reads the repo and
|
|
158
|
+
writes nothing to it.
|
|
159
|
+
|
|
160
|
+
The approach shows up in the approval table under each task, which is the last moment it can
|
|
161
|
+
be changed for free, and it is handed to QA as the *intended* approach — so a review can
|
|
162
|
+
finally say "this works, but it abandons the planned shape" rather than only judging against
|
|
163
|
+
the objective.
|
|
164
|
+
|
|
165
|
+
## Plan, run, integrate
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
PLAN=/tmp/forge-auth
|
|
169
|
+
|
|
170
|
+
# 1. plan and inspect the routing
|
|
171
|
+
bash scripts/forge-parallel.sh plan "$PLAN" --repo ~/dev/api \
|
|
172
|
+
--dwarf-high sol:xhigh --dwarf-low gemini:low:antigravity --dwarf luna:high
|
|
173
|
+
|
|
174
|
+
# 2. see exactly what will be dispatched, without dispatching
|
|
175
|
+
bash scripts/forge-parallel.sh run "$PLAN" --dry-run
|
|
176
|
+
|
|
177
|
+
# 3. run it, four dwarves at a time, implementers unsandboxed
|
|
178
|
+
bash scripts/forge-parallel.sh run "$PLAN" --max-parallel 4 --yolo-dwarf
|
|
179
|
+
|
|
180
|
+
# 4. after reading the results, deliver the passing work
|
|
181
|
+
bash scripts/forge-parallel.sh integrate "$PLAN" --approved
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
`integrate` refuses without `--approved`, and refuses on a dirty working tree.
|
|
185
|
+
For Fractal execution, pass tier pools to `plan` and backend/limit flags to `run`.
|
|
186
|
+
`--max-parallel` limits task pipelines; the shared `--fractal-concurrency` limit includes
|
|
187
|
+
all their child invocations and QA. See [decomposed Fractal execution](../references/decompose.md#fractal-within-decomposed-tasks).
|
|
188
|
+
|
|
189
|
+
## Resuming an interrupted run
|
|
190
|
+
|
|
191
|
+
Run `run` again to skip MERGED tasks, merge saved PASS results after fingerprint checks,
|
|
192
|
+
and dispatch eligible pending tasks. Tasks paused on an infrastructure failure (`INFRA`) are
|
|
193
|
+
dispatched again without any flag, since nothing was wrong with the work. Failed and
|
|
194
|
+
interrupted tasks need an explicit `retry` (or `run --retry-failed N` for rejected ones).
|
|
195
|
+
Their worktrees and pinned baselines remain available. Concurrent run/retry/integrate
|
|
196
|
+
operations for the same plan are excluded by an advisory lock.
|
|
197
|
+
|
|
198
|
+
For a Fractal-backed run, use `forge fractal resume RUN_ID` to recover recorded execution
|
|
199
|
+
and pipeline checkpoints. A task/subtree selector resumes only that execution scope.
|
|
200
|
+
Explicit `retry` retains the backend and frozen pools; a replacement dwarf must already
|
|
201
|
+
belong to that pool. See [recovery](../references/fractal.md#limits-and-lifecycle).
|
|
202
|
+
|
|
203
|
+
## Changing a run while it works
|
|
204
|
+
|
|
205
|
+
`forge control` changes a plan's knobs without stopping it (forge-ui calls the same command):
|
|
206
|
+
|
|
207
|
+
```bash
|
|
208
|
+
forge control show <plan-dir> [--json] # every control, per-task overrides, pins, recent changes
|
|
209
|
+
forge control set <plan-dir> qa-threshold P2 # also --task <id>; P0-P3, none, or inherit (task only)
|
|
210
|
+
forge control set <plan-dir> retry-budget 3 [--task <id>] # automatic retries; live scheduler only
|
|
211
|
+
forge control set <plan-dir> infra-retries 2 # infrastructure waits; live scheduler only
|
|
212
|
+
forge control set <plan-dir> timeout|escalate-after|verify-retries <n>
|
|
213
|
+
forge control pin <plan-dir> <task> sol:high:codex # the model the next full retry uses, once
|
|
214
|
+
forge control unpin <plan-dir> <task>
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
Each change is validated, written under `control.lock`, and says when it applies:
|
|
218
|
+
|
|
219
|
+
- **The QA threshold is read once per attempt**, at the start of the task's next attempt; the
|
|
220
|
+
attempt already running keeps the one it began with, so a verdict is never judged by a bar that
|
|
221
|
+
moved under it. A per-task `none` exempts that task from the plan's threshold; `inherit` drops
|
|
222
|
+
the override. A plan-level `none` clears the threshold.
|
|
223
|
+
- **Budgets stay "per invocation".** `run --retry-failed N` and `--infra-retries M` still apply
|
|
224
|
+
only to the invocation that was given them and are not remembered; the next `run` starts
|
|
225
|
+
from its own flags. A rule that came back from an old file would silently spend money nobody
|
|
226
|
+
asked for. While a scheduler is live it mirrors them into the plan (`retry_budget`,
|
|
227
|
+
`infra_budget`, `tasks/<id>/retry_budget`) and re-reads them at every decision, which is what
|
|
228
|
+
`forge control` edits. When the scheduler stops it removes them; so a budget can only be changed
|
|
229
|
+
while a run is live (exit 3 otherwise), and the change dies with the run.
|
|
230
|
+
`tasks/<id>/retries.used` counts the retries spent and stays after the run for forge-ui.
|
|
231
|
+
- **A pin is one-shot.** `pin` makes the task's next *full* retry use that spec (checked with
|
|
232
|
+
`forge-dispatch.sh doctor`, which spends no quota). A QA-only resume spends no implementer
|
|
233
|
+
attempt and leaves the pin for later. The retry applies it, writes it into `tasks.tsv`, records
|
|
234
|
+
an `escalations.tsv` row with `source` `pin`, clears any `split.suggested` stop and consumes
|
|
235
|
+
it. Fractal plans never apply pins, and a MERGED task refuses one.
|
|
236
|
+
- **Who decided an escalation.** The last column of `tasks/<id>/escalations.tsv` is `policy`
|
|
237
|
+
(`--escalate-after`), `retry` (a typed `retry --dwarf`) or `pin`; rows from before it was added
|
|
238
|
+
have four columns and read as `policy`.
|
|
239
|
+
|
|
240
|
+
Every change is appended to `control.log` (one JSON line: `at`, `actor` `cli` or `ui`, `action`,
|
|
241
|
+
`task`, `key`, `value`, `previous`) and recorded in the dataset as a `control` event, so a run's
|
|
242
|
+
human interventions are on the record next to what the policy did. Setting a control to the
|
|
243
|
+
value it already has changes and logs nothing.
|
|
244
|
+
|
|
245
|
+
## Splitting and combining plans
|
|
246
|
+
|
|
247
|
+
`run --retry-failed N` removes most of the reason to split a plan (a failed task no longer holds
|
|
248
|
+
the rest of a run). When you still want independent work in separate plans — separate locks,
|
|
249
|
+
separate integration branches, one verified at a time — these two make it a supported operation
|
|
250
|
+
instead of hand-built git plumbing:
|
|
251
|
+
|
|
252
|
+
- `split <plan> <new-plan> --tasks a,b` moves tasks **with their saved state** (attempts,
|
|
253
|
+
findings, reviewed commit, status) into a new plan: their branches and worktrees are renamed to
|
|
254
|
+
the new run (`forge/<new>/<id>`), the new integration branch starts at the source's accepted
|
|
255
|
+
tip, dependencies on already-merged tasks are dropped, and the source forgets the moved rows
|
|
256
|
+
(`split-<new>.txt` records them). It refuses — changing nothing — for a Fractal-backed plan, a
|
|
257
|
+
MERGED or RUNNING task, a selection that leaves a dependent behind, or a dependency that is
|
|
258
|
+
neither moving nor merged. Copy first, delete last; an interrupted split rolls back.
|
|
259
|
+
- `combine <new-plan> <plan-a> <plan-b>…` joins plans whose tasks are **all `MERGED`** into one
|
|
260
|
+
plan: a new integration branch merges each source's, the per-task records are copied, and
|
|
261
|
+
`run <new-plan>` then only verifies (no model call) so `integrate` works unchanged. It refuses
|
|
262
|
+
different repositories, locked or dirty sources, duplicate task ids, and any task left behind;
|
|
263
|
+
a merge conflict exits `6` after removing everything it created. `--verify` sets the combined
|
|
264
|
+
plan's check; `--dry-run` previews and predicts conflicts. Task branches keep their original
|
|
265
|
+
names.
|
|
266
|
+
|
|
267
|
+
## Scope drift
|
|
268
|
+
|
|
269
|
+
After each dwarf, the runner compares what the task **declared** in `files` against what it
|
|
270
|
+
actually touched (a declared directory covers the paths beneath it). Undeclared paths appear
|
|
271
|
+
in the summary and in the QA prompt:
|
|
272
|
+
|
|
273
|
+
```
|
|
274
|
+
scope drift (touched files the task did not declare):
|
|
275
|
+
auth src/db.py
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
Never a failure — a dwarf that genuinely needed one more file did the right thing. What it
|
|
279
|
+
buys is that a merge `CONFLICT` two waves later arrives with its cause already named.
|
|
280
|
+
|
|
281
|
+
## `tasks.tsv`
|
|
282
|
+
|
|
283
|
+
Tab-separated, one row per task, written during decomposition:
|
|
284
|
+
|
|
285
|
+
```
|
|
286
|
+
# id deps difficulty files dwarf qa title
|
|
287
|
+
middleware - high src/auth/middleware.py - - Auth middleware
|
|
288
|
+
tokens - medium src/auth/token.py - - Token model
|
|
289
|
+
route middleware,tokens medium src/routes/login.py - - Login route
|
|
290
|
+
docs - low README.md - - Document the auth flow
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
| Column | Meaning |
|
|
294
|
+
|---|---|
|
|
295
|
+
| `id` | short unique slug; also the worktree and branch name |
|
|
296
|
+
| `deps` | comma-list of task ids that must land first, or `-` |
|
|
297
|
+
| `difficulty` | `low` \| `medium` \| `high` |
|
|
298
|
+
| `files` | expected touch set, comma-separated — **drives disjointness** |
|
|
299
|
+
| `dwarf` / `qa` | `-` to resolve from the routing rules; an explicit spec always wins |
|
|
300
|
+
| `title` | one line, shown in the approval table and the capsule |
|
|
301
|
+
|
|
302
|
+
Each task also needs `tasks/<id>/prompt.md`, written the same way a single-task forge goal is.
|
|
303
|
+
|
|
304
|
+
`files` is a **promise, not a prediction**: it's what the wave planner trusts when deciding
|
|
305
|
+
what may run concurrently, and what the capsule tells other dwarves not to touch. An
|
|
306
|
+
understated `files` is how two dwarves end up in the same file.
|
|
307
|
+
|
|
308
|
+
`plan` writes resolved specs back into the `dwarf`/`qa` columns, so `run` never consults the
|
|
309
|
+
routing rules — and an explicit value survives a re-plan, which is what makes a gate override
|
|
310
|
+
durable. (`UNASSIGNED` is treated as a leftover marker and re-resolved.)
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# Development
|
|
2
|
+
|
|
3
|
+
[← README](../README.md) · [All docs](README.md)
|
|
4
|
+
|
|
5
|
+
The repository layout, the offline checks, and how forge measures its own runtime cost.
|
|
6
|
+
|
|
7
|
+
## Repository layout
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
forge/
|
|
11
|
+
├── SKILL.md the skill itself — what the orchestrating model reads
|
|
12
|
+
├── registry.tsv alias → (harness, model, effort). Add a row to teach forge a model
|
|
13
|
+
├── README.md the short overview
|
|
14
|
+
├── docs/ the full documentation, one page per topic
|
|
15
|
+
├── LICENSE MIT
|
|
16
|
+
├── package.json the npm package: its name, version and the files it ships
|
|
17
|
+
├── forge launcher, on PATH from npm: `forge install`, `fractal`, `jev`, `dataset` and `ui`
|
|
18
|
+
├── scripts/
|
|
19
|
+
│ ├── forge-dispatch.sh resolve a spec → run one role on one harness
|
|
20
|
+
│ ├── forge-solo.sh the whole single-task run: dwarf → diff → qa
|
|
21
|
+
│ ├── forge-parallel.sh plan / run / retry / integrate for decomposed runs
|
|
22
|
+
│ ├── forge-parallel-options.sh flags shared by several forge-parallel.sh subcommands
|
|
23
|
+
│ ├── forge-lib-task.sh attempt log, infra stops, QA-only resume, guards, severity gate, report blocks
|
|
24
|
+
│ ├── forge-lib-verify.sh verify command lookup and the coverage report
|
|
25
|
+
│ ├── forge-lib-plans.sh accept / split / combine
|
|
26
|
+
│ ├── forge-lib-review.sh the opt-in whole-run review (review, integrate --final-review)
|
|
27
|
+
│ ├── forge-schedule.py readiness scheduler: in-run retries, infrastructure pauses
|
|
28
|
+
│ ├── forge-contract.py the shared QA/dwarf prompt contract and the severity gate
|
|
29
|
+
│ ├── forge-guard.py implementer guards: session wrapper, orphan reaping, promise scan
|
|
30
|
+
│ ├── forge-verify-coverage.py suites the project defines that the verify command skips
|
|
31
|
+
│ ├── forge-memory.sh inject / note / record / spend project memory
|
|
32
|
+
│ ├── forge-fractal.py Fractal CLI and internal adapter entrypoint
|
|
33
|
+
│ ├── forge-fractal-options.sh runner selection, limit flags and help
|
|
34
|
+
│ ├── forge-fractal-dispatch.sh bridge entrypoint for existing pipelines
|
|
35
|
+
│ ├── forge-install-fractal.sh isolated optional runtime installer
|
|
36
|
+
│ ├── forge_fractal/ nested execution, recovery, inspection and dashboard
|
|
37
|
+
│ ├── forge-ui.py `forge ui open|status|stop`: the one shared forge-ui server
|
|
38
|
+
│ ├── forge-ui-options.sh `--ui|--no-ui` and the fail-open launch, sourced by the runners
|
|
39
|
+
│ ├── forge_registry.py the run registry: where run dirs are noted, read by forge-ui
|
|
40
|
+
│ ├── forge-dataset.py `forge dataset` entry point
|
|
41
|
+
│ ├── forge_dataset/ the run dataset: capture hooks, readers, importer, status checks
|
|
42
|
+
│ └── forge-install.sh install into every harness that needs it (opencode and grok find it themselves)
|
|
43
|
+
├── ui/ forge-ui: read-only local run viewer (`forge_ui` package, tests, API contract)
|
|
44
|
+
└── references/
|
|
45
|
+
├── ui.md forge-ui: commands, env vars, shared server, state, privacy
|
|
46
|
+
├── harnesses.md per-harness invocation, ladders, failure modes
|
|
47
|
+
├── decompose.md tasks.tsv schema, difficulty criteria, wave algorithm
|
|
48
|
+
├── fractal.md opt-in backend, limits, recovery and observability
|
|
49
|
+
├── memory.md FORGE_LEARNING grammar, promotion and pruning rules
|
|
50
|
+
└── dataset.md the run dataset: what is captured, where, and its invariants
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## Offline checks
|
|
54
|
+
|
|
55
|
+
Run `bash tests/check.sh` for offline regression checks. See [tests](../tests/README.md) for skill
|
|
56
|
+
invocation scenarios and verification limits. There is currently no CI workflow.
|
|
57
|
+
|
|
58
|
+
## Runtime efficiency and measurement
|
|
59
|
+
|
|
60
|
+
Shell entrypoints require Python 3 (standard library only) in addition to Bash and Git.
|
|
61
|
+
The [runtime helper decision](../references/runtime.md) records the module boundaries and tradeoffs.
|
|
62
|
+
Solo and parallel runners default to `--output summary`; use `--output full` to include raw
|
|
63
|
+
backend output. Direct dispatch keeps its full-output default. Read each `dwarf.last` and
|
|
64
|
+
`qa.last` once, and retrieve `.log` or `.out` only when investigating a problem.
|
|
65
|
+
|
|
66
|
+
QA receives complete requirements, approach, ownership, retry findings and distinct memory
|
|
67
|
+
facts once, with review instructions separate from implementation instructions. Prompts and
|
|
68
|
+
full binary diffs remain on disk. Review repositories receive self-contained tree object packs,
|
|
69
|
+
create baseline/final commits, and check out the final tree once. Exact tree comparisons and
|
|
70
|
+
all source, review and integration fingerprint checks remain acceptance boundaries.
|
|
71
|
+
|
|
72
|
+
`FORGE_CAPABILITY_CACHE` optionally selects an offline help cache. Runners share a run-local
|
|
73
|
+
`capabilities/` cache across preflights and dispatches. Keys cover the executable's resolved
|
|
74
|
+
path, device/inode, size and modification/change timestamps, help command mode, registry
|
|
75
|
+
contents and dispatch recipe. Every selected spec, effort and permission recipe is still
|
|
76
|
+
validated; cached help never proves authentication or live model availability.
|
|
77
|
+
|
|
78
|
+
Run `bash scripts/forge-dispatch.sh report <run-dir>` for JSON attempt records, including
|
|
79
|
+
failures and timeouts, with project memory enabled or disabled. `attempts/<role>-<id>/`
|
|
80
|
+
retains prompts, final responses, raw logs, resolution, command and `metrics.json` per dispatch.
|
|
81
|
+
Pipeline records capture preparation, preflight, snapshot, verification and total time;
|
|
82
|
+
dispatch records capture preparation, preflight, model and total time. Nested pipeline
|
|
83
|
+
`dispatch` time includes dispatch overhead: do not add nested totals together. Timing uses
|
|
84
|
+
monotonic seconds. A still-running or forcibly killed record has a null exit code and may
|
|
85
|
+
have incomplete timings. A dispatch that stopped on an infrastructure failure records its class
|
|
86
|
+
(`quota`, `auth`, `rate_limit`, `network`, `empty`) as `infra_class`, and keeps the `.infra` and
|
|
87
|
+
`.orphans` files beside the other attempt artifacts.
|
|
88
|
+
|
|
89
|
+
Prompt bytes are separate from native input, cached-input and output token counts. Codex
|
|
90
|
+
uses `turn.completed` usage; Claude uses result usage (input includes uncached, cache-read
|
|
91
|
+
and cache-creation tokens). Unsupported or absent usage stays null, never estimated from
|
|
92
|
+
bytes. See [measurement and live evaluation](../tests/README.md).
|
|
93
|
+
|
|
94
|
+
Measured results, including the live comparison's higher total token use and latency, are
|
|
95
|
+
recorded in [the September 7 evaluation](../tests/benchmark-2026-09-07.md).
|
package/docs/forge-ui.md
ADDED
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
# forge-ui
|
|
2
|
+
|
|
3
|
+
[← README](../README.md) · [All docs](README.md)
|
|
4
|
+
|
|
5
|
+
forge-ui is a small local web page that shows runs live and, when you ask, changes them, as a 3D isometric map
|
|
6
|
+
ported from [archivision](https://github.com/Giamme/archivision). Its **run cloud** puts every
|
|
7
|
+
run in one full-screen view, grouped by project, in five layouts (helix, sphere, galaxy,
|
|
8
|
+
timeline, constellation), filtered down to exactly the runs you want; drag any run anywhere
|
|
9
|
+
and the cloud remembers it.
|
|
10
|
+
|
|
11
|
+
<p align="center">
|
|
12
|
+
<img src="../ui/docs/media/cloud.webp" width="880" alt="The run cloud: about seventy runs as blocks in five project spirals, orbited, then laid out as galaxies and as a timeline with one lane per project.">
|
|
13
|
+
</p>
|
|
14
|
+
|
|
15
|
+
<table>
|
|
16
|
+
<tr>
|
|
17
|
+
<td width="50%" valign="top"><img src="../ui/docs/media/run.png" alt="forge-ui's window: the runs list, a run's eight tasks on a 3D plate in their planned waves with dependency cables, finding marks and the finish nodes, and the run's inspector."><br><b>A run, as planned.</b> Tasks in their waves with dependency cables, ending in integration, verification and the final review. Click a task to open its attempts, dwarves and reviews.</td>
|
|
18
|
+
<td width="50%" valign="top"><img src="../ui/docs/media/execution.png" alt="The same run on wall-clock time: one block per attempt, failed attempts in red with their finding marks, a plane marking now, the concurrency strip, and the inspector's timeline and critical path."><br><b>And as it ran.</b> One block per attempt in the lane it ran in, failed ones in red, against <code>--max-parallel</code>; the inspector names the critical path.</td>
|
|
19
|
+
</tr>
|
|
20
|
+
<tr>
|
|
21
|
+
<td valign="top"><img src="../ui/docs/media/diff.png" alt="The diff a reviewer read, with a P1 finding pinned under the line comparing token digests and a P2 finding under the revoke function."><br><b>Findings on their lines.</b> Any diff of a task, or exactly the diff one reviewer read, with each P0–P3 finding under the line it names.</td>
|
|
22
|
+
<td valign="top"><img src="../ui/docs/media/conversation.png" alt="A dwarf's conversation: its thinking, a search, the files it wrote, a test run that failed with exit 1 and the run that passed after the fix."><br><b>What each agent said and ran.</b> A dwarf's or reviewer's conversation from its CLI's log: commands with their exit and output, edits linked to the diff, its thinking; searchable, errors one key apart, live while it runs.</td>
|
|
23
|
+
</tr>
|
|
24
|
+
<tr>
|
|
25
|
+
<td valign="top"><img src="../ui/docs/media/models.png" alt="The model leaderboard: per implementer, tasks, first-attempt pass rate with a 95% interval bar, retries, escalations, timeouts, accepts, median model time and tokens."><br><b>Models, measured on your runs.</b> First-attempt pass rate with a confidence interval, retries, escalations, time and tokens; reviewers' verdicts and shadow agreement.</td>
|
|
26
|
+
<td valign="top"><img src="../ui/docs/media/files.png" alt="The files heatmap: files grouped by directory against tasks grouped by wave, with planned, touched, drift and same-wave overlap marks."><br><b>Who touched which file.</b> Planned, changed, drifted into or conflicted on, wave by wave; a cell opens that file's diff.</td>
|
|
27
|
+
</tr>
|
|
28
|
+
<tr>
|
|
29
|
+
<td colspan="2" valign="top" align="center"><img width="50%" src="../ui/docs/media/map.png" alt="The map of every run: one block per run in a district per repo, green when done, amber while running, red with failures."><br><b>Every run, by repo.</b> One block per run with a lamp per task, filtered by state, repo, model, age or review verdict.</td>
|
|
30
|
+
</tr>
|
|
31
|
+
</table>
|
|
32
|
+
|
|
33
|
+
The page reads like a signal box's desk: engraved plates for headings, push buttons with lamps
|
|
34
|
+
for switches, and lamps that keep one meaning everywhere (amber running, green merged, red failed,
|
|
35
|
+
unlit pending). Two layouts share it. *Immersive*, the default, floats the runs, the inspector and the tasks
|
|
36
|
+
over a full-window stage (on a first visit the inspector and the tasks start folded, so a run
|
|
37
|
+
opens on its plate); *Workbench* docks them beside it with splitters. `1`, `2` and
|
|
38
|
+
`3` fold each module away, the header switches layout and theme (system, light or dark), and the
|
|
39
|
+
task and model tables sort, reorder and hide columns. forge-ui remembers all of it in your browser.
|
|
40
|
+
|
|
41
|
+
On the 3D stage every review finding stands over its block as a mark shaped by its severity, and
|
|
42
|
+
a click opens the diff at that finding. A chosen task keeps its route lit, a hover shows its card
|
|
43
|
+
(attempts, time, tokens, worst finding), and merges, failed attempts and escalations play over the
|
|
44
|
+
plate as they happen or as a replay passes them. The *View* menu switches each part of the drawing
|
|
45
|
+
on or off; the keys at the stage's edge fit, zoom, switch between perspective, isometric and
|
|
46
|
+
top-down, open a legend of what is drawn, and `?` lists every key.
|
|
47
|
+
|
|
48
|
+
Each dwarf's and reviewer's conversation opens from its attempt: what it said, what it ran and
|
|
49
|
+
with what exit, what it changed (with a link into the diff), its thinking folded, and the final
|
|
50
|
+
answer with tokens and cost, searchable, with its errors one key apart and, while it runs,
|
|
51
|
+
following its log as it grows. It is read from the agent CLI's own log, whichever harness
|
|
52
|
+
wrote it (as plain text for those that log no events yet), or for an older claude run from
|
|
53
|
+
Claude's session file on this machine.
|
|
54
|
+
|
|
55
|
+
### Controls
|
|
56
|
+
|
|
57
|
+
forge-ui reads everything, and it changes things only by calling forge's own commands: it
|
|
58
|
+
never writes a run file, a repo or a config itself. That keeps one set of rules (validation,
|
|
59
|
+
locks, the audit log) whichever way you change something, and a change from the page is
|
|
60
|
+
indistinguishable from the same command typed in a terminal, except that `control.log` says
|
|
61
|
+
`ui` instead of `cli`. Every change asks you to confirm and says when it takes effect.
|
|
62
|
+
|
|
63
|
+
- **Run controls** (a plan's inspector): the QA threshold, the per-attempt timeout, how many
|
|
64
|
+
failures before escalating, verification retries, and, while a scheduler is live, the retry
|
|
65
|
+
and infrastructure-retry budgets. A task has its own panel for a threshold override
|
|
66
|
+
(including *None*, which exempts it from the run's threshold), its own retry budget, and a
|
|
67
|
+
**model pin**: the model its next full retry uses, once. Each is `forge control`; see
|
|
68
|
+
[decomposed runs](decomposed-runs.md#changing-a-run-while-it-works). A control that cannot
|
|
69
|
+
apply (a budget with no live scheduler, a merged task) is disabled and says why.
|
|
70
|
+
- **Open findings:** the review findings a run left open (tolerated by `--qa-threshold`,
|
|
71
|
+
PLAUSIBLE, or left by the whole-run review), for this run or every run of its repository.
|
|
72
|
+
Select some and file them as GitHub issues, or dismiss the ones that need no work (and take
|
|
73
|
+
that back). It runs `forge issues`, so a finding is never filed twice.
|
|
74
|
+
- **Delete a run:** a dialog that first shows a dry run, then deletes the run directory
|
|
75
|
+
through `forge runs rm`. The dataset keeps the run's traces, and by default a `.tar.gz` of
|
|
76
|
+
them too; you can also remove its clean worktrees and its branches that nothing has checked
|
|
77
|
+
out. Anything that is not the run's own, or has uncommitted changes, is listed and left alone.
|
|
78
|
+
- **Settings:** every config source in one view (`forge config show`) and edits for the
|
|
79
|
+
per-repo ones (`repo.verify`, `repo.setup`, `repo.verify-subset`, and Jev on or off). The
|
|
80
|
+
registry stays read-only: the skill directory is often a live symlink to a checkout.
|
|
81
|
+
|
|
82
|
+
Writing is protected, because anything that can reach the server could otherwise change your
|
|
83
|
+
runs. The server makes a random token each time it starts and puts it in the page it serves;
|
|
84
|
+
the page sends it with every change, and the server also checks the `Host` and `Origin`
|
|
85
|
+
headers, so another website open in your browser cannot make your browser change a run. Reading
|
|
86
|
+
needs no token. `--read-only` (or `FORGE_UI_READONLY=1`) turns writing off altogether: no
|
|
87
|
+
token is issued and every change is refused. `FORGE_UI_READONLY=1 forge ui open` starts such a
|
|
88
|
+
server, and the launcher never reuses a server running in the other mode; it replaces it.
|
|
89
|
+
|
|
90
|
+
Also: replay, critical path, Fractal child-task trees, quick jump (Ctrl K, ⌘K on a Mac, or
|
|
91
|
+
`/`), a bell for runs that finish or fail, failed tasks and final-review verdicts (with opt-in
|
|
92
|
+
browser notifications), and a run exported as one HTML file that opens without forge-ui, with
|
|
93
|
+
prompts, diffs and local paths left out unless you choose them.
|
|
94
|
+
|
|
95
|
+
Offline (it loads three.js from a CDN for the 3D view) the same page works in 2D. It is opt-in.
|
|
96
|
+
The `/forge` skill asks "Open forge-ui to watch this run? [y/N]" at the start of every new
|
|
97
|
+
interactive run, or you pass it yourself:
|
|
98
|
+
|
|
99
|
+
```sh
|
|
100
|
+
bash scripts/forge-solo.sh "$RUN" --repo "$REPO" --dwarf sol --ui
|
|
101
|
+
bash scripts/forge-parallel.sh plan "$PLAN" --repo "$REPO" --decompose-level medium --ui
|
|
102
|
+
forge ui open [DIR] # an existing run dir, or the run list
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
All runs share one detached server on `127.0.0.1:8765` (`FORGE_UI_PORT`) that exits after
|
|
106
|
+
four idle hours (`FORGE_UI_IDLE_S`). A UI that cannot start never fails the run. Reading needs
|
|
107
|
+
no token: any local process can read the prompts and diffs of the runs it shows; only changes
|
|
108
|
+
need the token (see [Controls](#controls)). See
|
|
109
|
+
[references/ui.md](../references/ui.md) and [ui/README.md](../ui/README.md).
|
|
110
|
+
|
|
111
|
+
To look around before your first run, point it at made-up runs (the ones in these pictures), from a git checkout:
|
|
112
|
+
|
|
113
|
+
```sh
|
|
114
|
+
python3 ui/tools/demo_runs.py /tmp/forge-demo
|
|
115
|
+
(cd ui && python3 -m forge_ui --no-registry --root /tmp/forge-demo --open)
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
## `forge ui` command group
|
|
119
|
+
|
|
120
|
+
| Command | Purpose |
|
|
121
|
+
| --- | --- |
|
|
122
|
+
| `open [DIR] [--no-browser] [--port N]` | start or reuse the shared forge-ui server (never one in the other mode); print `forge-ui: <url>` for the plan or solo run dir `DIR` (the run list without it) and open a browser the first time (a later `open` of the same run says on stderr that it opened none) |
|
|
123
|
+
| `status [--json]` | exit 0 and describe the server (and whether it is read-only) when it runs, 1 when not |
|
|
124
|
+
| `stop` | stop the server |
|
|
125
|
+
|
|
126
|
+
`FORGE_UI=off` makes `--ui` on a runner start nothing (it does not stop a running server or block `forge ui open`; use `forge ui stop`), `FORGE_UI_BROWSER=off` never opens a browser,
|
|
127
|
+
`FORGE_UI_PORT` (8765) and `FORGE_UI_IDLE_S` (14400, `0` = never) tune the server, and `FORGE_UI_READONLY=1` starts it read-only. State and
|
|
128
|
+
`server.log` are in `${XDG_STATE_HOME:-~/.local/state}/forge/ui/`.
|