@giamme/forge 0.8.0 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (83) hide show
  1. package/CHANGELOG.md +50 -0
  2. package/README.md +106 -1766
  3. package/SKILL.md +22 -3
  4. package/VERSION +1 -1
  5. package/docs/README.md +26 -0
  6. package/docs/add-ons.md +33 -0
  7. package/docs/artifacts.md +84 -0
  8. package/docs/commands.md +365 -0
  9. package/docs/dataset.md +29 -0
  10. package/docs/decomposed-runs.md +310 -0
  11. package/docs/development.md +95 -0
  12. package/docs/forge-ui.md +128 -0
  13. package/docs/fractal.md +181 -0
  14. package/docs/install.md +160 -0
  15. package/docs/memory.md +156 -0
  16. package/docs/models.md +118 -0
  17. package/docs/reviews.md +245 -0
  18. package/docs/safety.md +55 -0
  19. package/docs/troubleshooting.md +54 -0
  20. package/forge +20 -0
  21. package/package.json +3 -2
  22. package/references/dataset.md +18 -1
  23. package/references/decompose.md +20 -2
  24. package/references/fractal.md +1 -1
  25. package/references/harnesses.md +18 -6
  26. package/references/solo.md +1 -1
  27. package/references/ui.md +27 -5
  28. package/scripts/forge-config.py +347 -0
  29. package/scripts/forge-control.py +645 -0
  30. package/scripts/forge-dispatch.sh +7 -1
  31. package/scripts/forge-issues.py +806 -0
  32. package/scripts/forge-lib-task.sh +234 -35
  33. package/scripts/forge-parallel-options.sh +3 -1
  34. package/scripts/forge-parallel.sh +3 -2
  35. package/scripts/forge-route.py +184 -27
  36. package/scripts/forge-runs-rm.py +840 -0
  37. package/scripts/forge-runs.py +139 -0
  38. package/scripts/forge-runtime.py +59 -6
  39. package/scripts/forge-schedule.py +227 -11
  40. package/scripts/forge-ui.py +85 -19
  41. package/scripts/forge-version.py +54 -12
  42. package/scripts/forge_dataset/__init__.py +77 -6
  43. package/scripts/forge_dataset/capture.py +5 -2
  44. package/scripts/forge_dataset/harvest.py +23 -1
  45. package/scripts/forge_dataset/importer.py +420 -18
  46. package/scripts/forge_fractal/__init__.py +2 -1
  47. package/scripts/forge_registry.py +168 -16
  48. package/ui/README.md +116 -23
  49. package/ui/THIRD_PARTY_NOTICES.md +108 -0
  50. package/ui/docs/api.md +283 -23
  51. package/ui/forge_ui/__main__.py +10 -2
  52. package/ui/forge_ui/control.py +575 -0
  53. package/ui/forge_ui/export.py +33 -9
  54. package/ui/forge_ui/fractal.py +1 -2
  55. package/ui/forge_ui/reader.py +121 -8
  56. package/ui/forge_ui/server.py +135 -5
  57. package/ui/forge_ui/static/D-DINCondensed-Bold.woff2 +0 -0
  58. package/ui/forge_ui/static/app.js +358 -131
  59. package/ui/forge_ui/static/cloud3d.js +3 -2
  60. package/ui/forge_ui/static/cloudui.js +8 -5
  61. package/ui/forge_ui/static/controls.js +619 -0
  62. package/ui/forge_ui/static/convo.js +295 -0
  63. package/ui/forge_ui/static/datatable.js +232 -0
  64. package/ui/forge_ui/static/diffview.js +29 -10
  65. package/ui/forge_ui/static/findings3d.js +134 -0
  66. package/ui/forge_ui/static/findingsview.js +412 -0
  67. package/ui/forge_ui/static/graph.js +3 -3
  68. package/ui/forge_ui/static/icons.js +94 -0
  69. package/ui/forge_ui/static/index.html +31 -13
  70. package/ui/forge_ui/static/iso.js +70 -19
  71. package/ui/forge_ui/static/leaderboard.js +44 -48
  72. package/ui/forge_ui/static/panel.js +92 -36
  73. package/ui/forge_ui/static/rundelete.js +388 -0
  74. package/ui/forge_ui/static/runmodel.js +17 -1
  75. package/ui/forge_ui/static/scene3d.js +518 -43
  76. package/ui/forge_ui/static/search.js +2 -1
  77. package/ui/forge_ui/static/settings.js +399 -0
  78. package/ui/forge_ui/static/shell.js +137 -0
  79. package/ui/forge_ui/static/stagectl.js +150 -0
  80. package/ui/forge_ui/static/style.css +1099 -469
  81. package/ui/forge_ui/static/theme.js +25 -0
  82. package/ui/forge_ui/static/uikit.js +161 -0
  83. package/ui/forge_ui/transcripts.py +608 -0
@@ -0,0 +1,310 @@
1
+ # Decomposed runs
2
+
3
+ [← README](../README.md) · [All docs](README.md)
4
+
5
+ Split one goal into tasks, route each to a model by difficulty, and run them in parallel in isolated worktrees, merging only the tasks that pass review.
6
+
7
+ `--decompose-level low|medium|high` turns one dwarf into many. Without the flag, nothing
8
+ changes — one dwarf, one QA pass.
9
+
10
+ ## Two independent axes
11
+
12
+ Easy to conflate, so: **decompose-level** is how finely the *goal* is split. **difficulty**
13
+ is how hard each resulting *task* is.
14
+
15
+ | `--decompose-level` | Split |
16
+ |---|---|
17
+ | `low` | only where the goal has obviously independent pieces; 2–3 coarse tasks |
18
+ | `medium` | one task per coherent unit (module, feature slice); typically 4–6 |
19
+ | `high` | finest split where each task is still independently reviewable *and* independently correct; typically 7–12 |
20
+
21
+ | `difficulty` | Means |
22
+ |---|---|
23
+ | `low` | mechanical and local: rename, move, docs, a test for behaviour that already exists |
24
+ | `medium` | self-contained implementation against a clear spec; bounded blast radius |
25
+ | `high` | needs design judgment, cross-cutting behaviour, subtle edge cases, state or concurrency |
26
+
27
+ Difficulty is about **judgment required, not diff size**. A five-hundred-line mechanical
28
+ rename is `low`. A ten-line concurrency fix is `high`. Getting this backwards sends the cheap
29
+ model at the subtle problem and the expensive one at the boilerplate — worse than not routing
30
+ at all.
31
+
32
+ ## Routing dwarves to tasks
33
+
34
+ Per-task assignment can't be expressed when you type the command, because the tasks don't
35
+ exist yet. Naming a dwarf per task would mean guessing how the goal will split. So you map
36
+ **model tiers to difficulty tiers**, once, in advance — three layers, each optional:
37
+
38
+ ```bash
39
+ # 1. one dwarf for everything
40
+ --dwarf sol:xhigh
41
+
42
+ # 2. per-tier routing, with plain --dwarf as the fallback for any tier not named
43
+ --dwarf-high sol:xhigh --dwarf-medium luna:high --dwarf-low gemini:low:antigravity
44
+ --dwarf sonnet:medium
45
+
46
+ # 3. pool several models within one tier — round-robined across tasks
47
+ --dwarf-high sol:xhigh,terra:xhigh
48
+ ```
49
+
50
+ `--qa`, `--qa-high`, `--qa-medium`, `--qa-low` mirror it exactly. QA has no tier rules by
51
+ default and falls back to `opus`.
52
+
53
+ Resolution order per task: **explicit column value → the tier's rule → plain `--dwarf` →
54
+ `UNASSIGNED`**. An `UNASSIGNED` task makes `plan` exit `2` and `run` refuse outright. forge
55
+ does not invent a model when you haven't said which one should spend your quota.
56
+
57
+ Layer three is **the approval gate**: after decomposition, before any dispatch, you get the
58
+ task table with each task's difficulty and resolved dwarf/qa. Retier a task ("2 is actually
59
+ medium") and it re-routes per your rules, or override a single row's dwarf outright. This is
60
+ also the cost gate — `high` on a large goal is 20+ dispatches, which should be visible before
61
+ it's spent.
62
+
63
+ **Exploration is on by default** (`--explore 0.2`):
64
+ - **Which rows:** only rows left to forge, never one whose dwarf column you filled.
65
+ - **What changes:** now and then such a row gets its *neighbouring* tier's spec from your own
66
+ pools: the next cheaper tier for medium and high rows, the next stronger for low rows.
67
+ Pools that differ only in effort (`--dwarf-low sol --dwarf-high sol:xhigh`) compare the
68
+ two efforts.
69
+ - **How often:** about one row in ten at the default.
70
+ - **What you see:** each explored row is marked `*` in the table, before anything is spent.
71
+ - **What it's for:** `forge dataset routing` reads the draws back as pass rates per model and
72
+ effort. Pass `--explore 0` to turn it off.
73
+
74
+ See [decompose.md](../references/decompose.md#exploratory-routing-on-by-default).
75
+
76
+ ## Waves, and what actually bounds parallelism
77
+
78
+ A **wave** is the largest set of tasks whose dependencies are all satisfied by earlier waves
79
+ **and** whose declared `files` sets are pairwise disjoint.
80
+
81
+ Disjointness — not the task count — is the real bound. Two dwarves editing one file produce a
82
+ conflict no reviewer can untangle and no merge can resolve honestly, so they land in different
83
+ waves however the goal was decomposed. When that happens forge says so:
84
+
85
+ ```
86
+ deferrals (serialized to protect the merge):
87
+ wave 1: lint deferred (file overlap with a task already in this wave)
88
+ ```
89
+
90
+ That line matters. A decomposition that looks parallel but serializes at run time should
91
+ announce it, rather than leaving you wondering why eight tasks took eight rounds. When most
92
+ tasks overlap, the fix is a better split — not more parallelism.
93
+
94
+ Waves are a planning preview. A Python 3 standard-library coordinator starts eligible tasks
95
+ as prerequisites merge and capacity becomes available, while excluding overlapping active
96
+ paths. Merges are serialized. Each task pins the integration commit at dispatch; retries keep
97
+ that baseline, and QA sees only baseline membership recorded for that task.
98
+
99
+ ## Fresh session per task, plus a capsule
100
+
101
+ Every dispatch is a **clean slate** — forge never passes `--continue`, `--resume`, or
102
+ `codex exec resume`. One task's confused turn can't poison another, and each task's context
103
+ stays small.
104
+
105
+ Clean slates alone would leave every dwarf blind to the wider run, so a short `capsule.md` is
106
+ generated from the task's pinned baseline and preserved for both implementation and QA:
107
+
108
+ ```
109
+ # forge run <run-id>
110
+ Goal: <overall goal>
111
+ You are doing ONE task of <N>. Do not do the others' work.
112
+
113
+ ## Your task
114
+ tokens — Add the token model difficulty: medium
115
+ Files you own: src/auth/token.py
116
+
117
+ ## Run status
118
+ id diff status files
119
+ middleware high MERGED src/auth/middleware.py
120
+ tokens medium THIS ONE src/auth/token.py
121
+ docs low NOT IN BASE README.md
122
+
123
+ ## Ground rules
124
+ - Your branch already contains every MERGED task's work — do not reimplement it.
125
+ - Files owned by other tasks are off limits; another dwarf is editing them now.
126
+ - If your task truly needs a change in someone else's file, say so in your final
127
+ message instead of making it.
128
+ ```
129
+
130
+ It's written **per task**, not once per run — tasks in a wave execute concurrently, so a
131
+ single shared capsule file is a race in which each task overwrites it and reads back whichever
132
+ sibling wrote last, handing a dwarf someone else's task description.
133
+
134
+ For QA the capsule additionally scopes the review: code from `MERGED` tasks is already in the
135
+ base and is not this task's bug. Without that, a wave-2 reviewer reports wave-1 code as
136
+ defects in the diff it was handed.
137
+
138
+ ## Planning
139
+
140
+ A dwarf that is handed only an objective plans its own approach, silently, inside its own
141
+ run. Usually fine — but in a decomposed run the capsule prevents two dwarves editing one
142
+ *file* and does nothing about them inventing incompatible *interfaces* at a shared seam.
143
+ Both stay in their lane, both pass their own QA, and the mismatch surfaces at integration.
144
+
145
+ So forge plans before it dispatches. By default the orchestrator does it and writes the
146
+ approach into each task's prompt — no extra dispatch. `--planner <spec>` hands that stage to
147
+ a named model instead:
148
+
149
+ ```bash
150
+ /forge "migrate the API layer to gRPC" --decompose-level medium \
151
+ --planner sol:xhigh --dwarf-high sol:high --dwarf luna:high
152
+ ```
153
+
154
+ **One planner dispatch per run, never one per task.** That is the whole point: a single mind
155
+ designs both sides of every seam, where N independent planners would recreate the problem.
156
+ Effort defaults to `xhigh`, since a bad plan is executed at full price by every dwarf
157
+ downstream of it. The planner runs with QA's permission profile — it reads the repo and
158
+ writes nothing to it.
159
+
160
+ The approach shows up in the approval table under each task, which is the last moment it can
161
+ be changed for free, and it is handed to QA as the *intended* approach — so a review can
162
+ finally say "this works, but it abandons the planned shape" rather than only judging against
163
+ the objective.
164
+
165
+ ## Plan, run, integrate
166
+
167
+ ```bash
168
+ PLAN=/tmp/forge-auth
169
+
170
+ # 1. plan and inspect the routing
171
+ bash scripts/forge-parallel.sh plan "$PLAN" --repo ~/dev/api \
172
+ --dwarf-high sol:xhigh --dwarf-low gemini:low:antigravity --dwarf luna:high
173
+
174
+ # 2. see exactly what will be dispatched, without dispatching
175
+ bash scripts/forge-parallel.sh run "$PLAN" --dry-run
176
+
177
+ # 3. run it, four dwarves at a time, implementers unsandboxed
178
+ bash scripts/forge-parallel.sh run "$PLAN" --max-parallel 4 --yolo-dwarf
179
+
180
+ # 4. after reading the results, deliver the passing work
181
+ bash scripts/forge-parallel.sh integrate "$PLAN" --approved
182
+ ```
183
+
184
+ `integrate` refuses without `--approved`, and refuses on a dirty working tree.
185
+ For Fractal execution, pass tier pools to `plan` and backend/limit flags to `run`.
186
+ `--max-parallel` limits task pipelines; the shared `--fractal-concurrency` limit includes
187
+ all their child invocations and QA. See [decomposed Fractal execution](../references/decompose.md#fractal-within-decomposed-tasks).
188
+
189
+ ## Resuming an interrupted run
190
+
191
+ Run `run` again to skip MERGED tasks, merge saved PASS results after fingerprint checks,
192
+ and dispatch eligible pending tasks. Tasks paused on an infrastructure failure (`INFRA`) are
193
+ dispatched again without any flag, since nothing was wrong with the work. Failed and
194
+ interrupted tasks need an explicit `retry` (or `run --retry-failed N` for rejected ones).
195
+ Their worktrees and pinned baselines remain available. Concurrent run/retry/integrate
196
+ operations for the same plan are excluded by an advisory lock.
197
+
198
+ For a Fractal-backed run, use `forge fractal resume RUN_ID` to recover recorded execution
199
+ and pipeline checkpoints. A task/subtree selector resumes only that execution scope.
200
+ Explicit `retry` retains the backend and frozen pools; a replacement dwarf must already
201
+ belong to that pool. See [recovery](../references/fractal.md#limits-and-lifecycle).
202
+
203
+ ## Changing a run while it works
204
+
205
+ `forge control` changes a plan's knobs without stopping it (forge-ui calls the same command):
206
+
207
+ ```bash
208
+ forge control show <plan-dir> [--json] # every control, per-task overrides, pins, recent changes
209
+ forge control set <plan-dir> qa-threshold P2 # also --task <id>; P0-P3, none, or inherit (task only)
210
+ forge control set <plan-dir> retry-budget 3 [--task <id>] # automatic retries; live scheduler only
211
+ forge control set <plan-dir> infra-retries 2 # infrastructure waits; live scheduler only
212
+ forge control set <plan-dir> timeout|escalate-after|verify-retries <n>
213
+ forge control pin <plan-dir> <task> sol:high:codex # the model the next full retry uses, once
214
+ forge control unpin <plan-dir> <task>
215
+ ```
216
+
217
+ Each change is validated, written under `control.lock`, and says when it applies:
218
+
219
+ - **The QA threshold is read once per attempt**, at the start of the task's next attempt; the
220
+ attempt already running keeps the one it began with, so a verdict is never judged by a bar that
221
+ moved under it. A per-task `none` exempts that task from the plan's threshold; `inherit` drops
222
+ the override. A plan-level `none` clears the threshold.
223
+ - **Budgets stay "per invocation".** `run --retry-failed N` and `--infra-retries M` still apply
224
+ only to the invocation that was given them and are not remembered; the next `run` starts
225
+ from its own flags. A rule that came back from an old file would silently spend money nobody
226
+ asked for. While a scheduler is live it mirrors them into the plan (`retry_budget`,
227
+ `infra_budget`, `tasks/<id>/retry_budget`) and re-reads them at every decision, which is what
228
+ `forge control` edits. When the scheduler stops it removes them; so a budget can only be changed
229
+ while a run is live (exit 3 otherwise), and the change dies with the run.
230
+ `tasks/<id>/retries.used` counts the retries spent and stays after the run for forge-ui.
231
+ - **A pin is one-shot.** `pin` makes the task's next *full* retry use that spec (checked with
232
+ `forge-dispatch.sh doctor`, which spends no quota). A QA-only resume spends no implementer
233
+ attempt and leaves the pin for later. The retry applies it, writes it into `tasks.tsv`, records
234
+ an `escalations.tsv` row with `source` `pin`, clears any `split.suggested` stop and consumes
235
+ it. Fractal plans never apply pins, and a MERGED task refuses one.
236
+ - **Who decided an escalation.** The last column of `tasks/<id>/escalations.tsv` is `policy`
237
+ (`--escalate-after`), `retry` (a typed `retry --dwarf`) or `pin`; rows from before it was added
238
+ have four columns and read as `policy`.
239
+
240
+ Every change is appended to `control.log` (one JSON line: `at`, `actor` `cli` or `ui`, `action`,
241
+ `task`, `key`, `value`, `previous`) and recorded in the dataset as a `control` event, so a run's
242
+ human interventions are on the record next to what the policy did. Setting a control to the
243
+ value it already has changes and logs nothing.
244
+
245
+ ## Splitting and combining plans
246
+
247
+ `run --retry-failed N` removes most of the reason to split a plan (a failed task no longer holds
248
+ the rest of a run). When you still want independent work in separate plans — separate locks,
249
+ separate integration branches, one verified at a time — these two make it a supported operation
250
+ instead of hand-built git plumbing:
251
+
252
+ - `split <plan> <new-plan> --tasks a,b` moves tasks **with their saved state** (attempts,
253
+ findings, reviewed commit, status) into a new plan: their branches and worktrees are renamed to
254
+ the new run (`forge/<new>/<id>`), the new integration branch starts at the source's accepted
255
+ tip, dependencies on already-merged tasks are dropped, and the source forgets the moved rows
256
+ (`split-<new>.txt` records them). It refuses — changing nothing — for a Fractal-backed plan, a
257
+ MERGED or RUNNING task, a selection that leaves a dependent behind, or a dependency that is
258
+ neither moving nor merged. Copy first, delete last; an interrupted split rolls back.
259
+ - `combine <new-plan> <plan-a> <plan-b>…` joins plans whose tasks are **all `MERGED`** into one
260
+ plan: a new integration branch merges each source's, the per-task records are copied, and
261
+ `run <new-plan>` then only verifies (no model call) so `integrate` works unchanged. It refuses
262
+ different repositories, locked or dirty sources, duplicate task ids, and any task left behind;
263
+ a merge conflict exits `6` after removing everything it created. `--verify` sets the combined
264
+ plan's check; `--dry-run` previews and predicts conflicts. Task branches keep their original
265
+ names.
266
+
267
+ ## Scope drift
268
+
269
+ After each dwarf, the runner compares what the task **declared** in `files` against what it
270
+ actually touched (a declared directory covers the paths beneath it). Undeclared paths appear
271
+ in the summary and in the QA prompt:
272
+
273
+ ```
274
+ scope drift (touched files the task did not declare):
275
+ auth src/db.py
276
+ ```
277
+
278
+ Never a failure — a dwarf that genuinely needed one more file did the right thing. What it
279
+ buys is that a merge `CONFLICT` two waves later arrives with its cause already named.
280
+
281
+ ## `tasks.tsv`
282
+
283
+ Tab-separated, one row per task, written during decomposition:
284
+
285
+ ```
286
+ # id deps difficulty files dwarf qa title
287
+ middleware - high src/auth/middleware.py - - Auth middleware
288
+ tokens - medium src/auth/token.py - - Token model
289
+ route middleware,tokens medium src/routes/login.py - - Login route
290
+ docs - low README.md - - Document the auth flow
291
+ ```
292
+
293
+ | Column | Meaning |
294
+ |---|---|
295
+ | `id` | short unique slug; also the worktree and branch name |
296
+ | `deps` | comma-list of task ids that must land first, or `-` |
297
+ | `difficulty` | `low` \| `medium` \| `high` |
298
+ | `files` | expected touch set, comma-separated — **drives disjointness** |
299
+ | `dwarf` / `qa` | `-` to resolve from the routing rules; an explicit spec always wins |
300
+ | `title` | one line, shown in the approval table and the capsule |
301
+
302
+ Each task also needs `tasks/<id>/prompt.md`, written the same way a single-task forge goal is.
303
+
304
+ `files` is a **promise, not a prediction**: it's what the wave planner trusts when deciding
305
+ what may run concurrently, and what the capsule tells other dwarves not to touch. An
306
+ understated `files` is how two dwarves end up in the same file.
307
+
308
+ `plan` writes resolved specs back into the `dwarf`/`qa` columns, so `run` never consults the
309
+ routing rules — and an explicit value survives a re-plan, which is what makes a gate override
310
+ durable. (`UNASSIGNED` is treated as a leftover marker and re-resolved.)
@@ -0,0 +1,95 @@
1
+ # Development
2
+
3
+ [← README](../README.md) · [All docs](README.md)
4
+
5
+ The repository layout, the offline checks, and how forge measures its own runtime cost.
6
+
7
+ ## Repository layout
8
+
9
+ ```
10
+ forge/
11
+ ├── SKILL.md the skill itself — what the orchestrating model reads
12
+ ├── registry.tsv alias → (harness, model, effort). Add a row to teach forge a model
13
+ ├── README.md the short overview
14
+ ├── docs/ the full documentation, one page per topic
15
+ ├── LICENSE MIT
16
+ ├── package.json the npm package: its name, version and the files it ships
17
+ ├── forge launcher, on PATH from npm: `forge install`, `fractal`, `jev`, `dataset` and `ui`
18
+ ├── scripts/
19
+ │ ├── forge-dispatch.sh resolve a spec → run one role on one harness
20
+ │ ├── forge-solo.sh the whole single-task run: dwarf → diff → qa
21
+ │ ├── forge-parallel.sh plan / run / retry / integrate for decomposed runs
22
+ │ ├── forge-parallel-options.sh flags shared by several forge-parallel.sh subcommands
23
+ │ ├── forge-lib-task.sh attempt log, infra stops, QA-only resume, guards, severity gate, report blocks
24
+ │ ├── forge-lib-verify.sh verify command lookup and the coverage report
25
+ │ ├── forge-lib-plans.sh accept / split / combine
26
+ │ ├── forge-lib-review.sh the opt-in whole-run review (review, integrate --final-review)
27
+ │ ├── forge-schedule.py readiness scheduler: in-run retries, infrastructure pauses
28
+ │ ├── forge-contract.py the shared QA/dwarf prompt contract and the severity gate
29
+ │ ├── forge-guard.py implementer guards: session wrapper, orphan reaping, promise scan
30
+ │ ├── forge-verify-coverage.py suites the project defines that the verify command skips
31
+ │ ├── forge-memory.sh inject / note / record / spend project memory
32
+ │ ├── forge-fractal.py Fractal CLI and internal adapter entrypoint
33
+ │ ├── forge-fractal-options.sh runner selection, limit flags and help
34
+ │ ├── forge-fractal-dispatch.sh bridge entrypoint for existing pipelines
35
+ │ ├── forge-install-fractal.sh isolated optional runtime installer
36
+ │ ├── forge_fractal/ nested execution, recovery, inspection and dashboard
37
+ │ ├── forge-ui.py `forge ui open|status|stop`: the one shared forge-ui server
38
+ │ ├── forge-ui-options.sh `--ui|--no-ui` and the fail-open launch, sourced by the runners
39
+ │ ├── forge_registry.py the run registry: where run dirs are noted, read by forge-ui
40
+ │ ├── forge-dataset.py `forge dataset` entry point
41
+ │ ├── forge_dataset/ the run dataset: capture hooks, readers, importer, status checks
42
+ │ └── forge-install.sh install into every harness that needs it (opencode and grok find it themselves)
43
+ ├── ui/ forge-ui: read-only local run viewer (`forge_ui` package, tests, API contract)
44
+ └── references/
45
+ ├── ui.md forge-ui: commands, env vars, shared server, state, privacy
46
+ ├── harnesses.md per-harness invocation, ladders, failure modes
47
+ ├── decompose.md tasks.tsv schema, difficulty criteria, wave algorithm
48
+ ├── fractal.md opt-in backend, limits, recovery and observability
49
+ ├── memory.md FORGE_LEARNING grammar, promotion and pruning rules
50
+ └── dataset.md the run dataset: what is captured, where, and its invariants
51
+ ```
52
+
53
+ ## Offline checks
54
+
55
+ Run `bash tests/check.sh` for offline regression checks. See [tests](../tests/README.md) for skill
56
+ invocation scenarios and verification limits. There is currently no CI workflow.
57
+
58
+ ## Runtime efficiency and measurement
59
+
60
+ Shell entrypoints require Python 3 (standard library only) in addition to Bash and Git.
61
+ The [runtime helper decision](../references/runtime.md) records the module boundaries and tradeoffs.
62
+ Solo and parallel runners default to `--output summary`; use `--output full` to include raw
63
+ backend output. Direct dispatch keeps its full-output default. Read each `dwarf.last` and
64
+ `qa.last` once, and retrieve `.log` or `.out` only when investigating a problem.
65
+
66
+ QA receives complete requirements, approach, ownership, retry findings and distinct memory
67
+ facts once, with review instructions separate from implementation instructions. Prompts and
68
+ full binary diffs remain on disk. Review repositories receive self-contained tree object packs,
69
+ create baseline/final commits, and check out the final tree once. Exact tree comparisons and
70
+ all source, review and integration fingerprint checks remain acceptance boundaries.
71
+
72
+ `FORGE_CAPABILITY_CACHE` optionally selects an offline help cache. Runners share a run-local
73
+ `capabilities/` cache across preflights and dispatches. Keys cover the executable's resolved
74
+ path, device/inode, size and modification/change timestamps, help command mode, registry
75
+ contents and dispatch recipe. Every selected spec, effort and permission recipe is still
76
+ validated; cached help never proves authentication or live model availability.
77
+
78
+ Run `bash scripts/forge-dispatch.sh report <run-dir>` for JSON attempt records, including
79
+ failures and timeouts, with project memory enabled or disabled. `attempts/<role>-<id>/`
80
+ retains prompts, final responses, raw logs, resolution, command and `metrics.json` per dispatch.
81
+ Pipeline records capture preparation, preflight, snapshot, verification and total time;
82
+ dispatch records capture preparation, preflight, model and total time. Nested pipeline
83
+ `dispatch` time includes dispatch overhead: do not add nested totals together. Timing uses
84
+ monotonic seconds. A still-running or forcibly killed record has a null exit code and may
85
+ have incomplete timings. A dispatch that stopped on an infrastructure failure records its class
86
+ (`quota`, `auth`, `rate_limit`, `network`, `empty`) as `infra_class`, and keeps the `.infra` and
87
+ `.orphans` files beside the other attempt artifacts.
88
+
89
+ Prompt bytes are separate from native input, cached-input and output token counts. Codex
90
+ uses `turn.completed` usage; Claude uses result usage (input includes uncached, cache-read
91
+ and cache-creation tokens). Unsupported or absent usage stays null, never estimated from
92
+ bytes. See [measurement and live evaluation](../tests/README.md).
93
+
94
+ Measured results, including the live comparison's higher total token use and latency, are
95
+ recorded in [the September 7 evaluation](../tests/benchmark-2026-09-07.md).
@@ -0,0 +1,128 @@
1
+ # forge-ui
2
+
3
+ [← README](../README.md) · [All docs](README.md)
4
+
5
+ forge-ui is a small local web page that shows runs live and, when you ask, changes them, as a 3D isometric map
6
+ ported from [archivision](https://github.com/Giamme/archivision). Its **run cloud** puts every
7
+ run in one full-screen view, grouped by project, in five layouts (helix, sphere, galaxy,
8
+ timeline, constellation), filtered down to exactly the runs you want; drag any run anywhere
9
+ and the cloud remembers it.
10
+
11
+ <p align="center">
12
+ <img src="../ui/docs/media/cloud.webp" width="880" alt="The run cloud: about seventy runs as blocks in five project spirals, orbited, then laid out as galaxies and as a timeline with one lane per project.">
13
+ </p>
14
+
15
+ <table>
16
+ <tr>
17
+ <td width="50%" valign="top"><img src="../ui/docs/media/run.png" alt="forge-ui's window: the runs list, a run's eight tasks on a 3D plate in their planned waves with dependency cables, finding marks and the finish nodes, and the run's inspector."><br><b>A run, as planned.</b> Tasks in their waves with dependency cables, ending in integration, verification and the final review. Click a task to open its attempts, dwarves and reviews.</td>
18
+ <td width="50%" valign="top"><img src="../ui/docs/media/execution.png" alt="The same run on wall-clock time: one block per attempt, failed attempts in red with their finding marks, a plane marking now, the concurrency strip, and the inspector's timeline and critical path."><br><b>And as it ran.</b> One block per attempt in the lane it ran in, failed ones in red, against <code>--max-parallel</code>; the inspector names the critical path.</td>
19
+ </tr>
20
+ <tr>
21
+ <td valign="top"><img src="../ui/docs/media/diff.png" alt="The diff a reviewer read, with a P1 finding pinned under the line comparing token digests and a P2 finding under the revoke function."><br><b>Findings on their lines.</b> Any diff of a task, or exactly the diff one reviewer read, with each P0–P3 finding under the line it names.</td>
22
+ <td valign="top"><img src="../ui/docs/media/conversation.png" alt="A dwarf's conversation: its thinking, a search, the files it wrote, a test run that failed with exit 1 and the run that passed after the fix."><br><b>What each agent said and ran.</b> A dwarf's or reviewer's conversation from its CLI's log: commands with their exit and output, edits linked to the diff, its thinking; searchable, errors one key apart, live while it runs.</td>
23
+ </tr>
24
+ <tr>
25
+ <td valign="top"><img src="../ui/docs/media/models.png" alt="The model leaderboard: per implementer, tasks, first-attempt pass rate with a 95% interval bar, retries, escalations, timeouts, accepts, median model time and tokens."><br><b>Models, measured on your runs.</b> First-attempt pass rate with a confidence interval, retries, escalations, time and tokens; reviewers' verdicts and shadow agreement.</td>
26
+ <td valign="top"><img src="../ui/docs/media/files.png" alt="The files heatmap: files grouped by directory against tasks grouped by wave, with planned, touched, drift and same-wave overlap marks."><br><b>Who touched which file.</b> Planned, changed, drifted into or conflicted on, wave by wave; a cell opens that file's diff.</td>
27
+ </tr>
28
+ <tr>
29
+ <td colspan="2" valign="top" align="center"><img width="50%" src="../ui/docs/media/map.png" alt="The map of every run: one block per run in a district per repo, green when done, amber while running, red with failures."><br><b>Every run, by repo.</b> One block per run with a lamp per task, filtered by state, repo, model, age or review verdict.</td>
30
+ </tr>
31
+ </table>
32
+
33
+ The page reads like a signal box's desk: engraved plates for headings, push buttons with lamps
34
+ for switches, and lamps that keep one meaning everywhere (amber running, green merged, red failed,
35
+ unlit pending). Two layouts share it. *Immersive*, the default, floats the runs, the inspector and the tasks
36
+ over a full-window stage (on a first visit the inspector and the tasks start folded, so a run
37
+ opens on its plate); *Workbench* docks them beside it with splitters. `1`, `2` and
38
+ `3` fold each module away, the header switches layout and theme (system, light or dark), and the
39
+ task and model tables sort, reorder and hide columns. forge-ui remembers all of it in your browser.
40
+
41
+ On the 3D stage every review finding stands over its block as a mark shaped by its severity, and
42
+ a click opens the diff at that finding. A chosen task keeps its route lit, a hover shows its card
43
+ (attempts, time, tokens, worst finding), and merges, failed attempts and escalations play over the
44
+ plate as they happen or as a replay passes them. The *View* menu switches each part of the drawing
45
+ on or off; the keys at the stage's edge fit, zoom, switch between perspective, isometric and
46
+ top-down, open a legend of what is drawn, and `?` lists every key.
47
+
48
+ Each dwarf's and reviewer's conversation opens from its attempt: what it said, what it ran and
49
+ with what exit, what it changed (with a link into the diff), its thinking folded, and the final
50
+ answer with tokens and cost, searchable, with its errors one key apart and, while it runs,
51
+ following its log as it grows. It is read from the agent CLI's own log, whichever harness
52
+ wrote it (as plain text for those that log no events yet), or for an older claude run from
53
+ Claude's session file on this machine.
54
+
55
+ ### Controls
56
+
57
+ forge-ui reads everything, and it changes things only by calling forge's own commands: it
58
+ never writes a run file, a repo or a config itself. That keeps one set of rules (validation,
59
+ locks, the audit log) whichever way you change something, and a change from the page is
60
+ indistinguishable from the same command typed in a terminal, except that `control.log` says
61
+ `ui` instead of `cli`. Every change asks you to confirm and says when it takes effect.
62
+
63
+ - **Run controls** (a plan's inspector): the QA threshold, the per-attempt timeout, how many
64
+ failures before escalating, verification retries, and, while a scheduler is live, the retry
65
+ and infrastructure-retry budgets. A task has its own panel for a threshold override
66
+ (including *None*, which exempts it from the run's threshold), its own retry budget, and a
67
+ **model pin**: the model its next full retry uses, once. Each is `forge control`; see
68
+ [decomposed runs](decomposed-runs.md#changing-a-run-while-it-works). A control that cannot
69
+ apply (a budget with no live scheduler, a merged task) is disabled and says why.
70
+ - **Open findings:** the review findings a run left open (tolerated by `--qa-threshold`,
71
+ PLAUSIBLE, or left by the whole-run review), for this run or every run of its repository.
72
+ Select some and file them as GitHub issues, or dismiss the ones that need no work (and take
73
+ that back). It runs `forge issues`, so a finding is never filed twice.
74
+ - **Delete a run:** a dialog that first shows a dry run, then deletes the run directory
75
+ through `forge runs rm`. The dataset keeps the run's traces, and by default a `.tar.gz` of
76
+ them too; you can also remove its clean worktrees and its branches that nothing has checked
77
+ out. Anything that is not the run's own, or has uncommitted changes, is listed and left alone.
78
+ - **Settings:** every config source in one view (`forge config show`) and edits for the
79
+ per-repo ones (`repo.verify`, `repo.setup`, `repo.verify-subset`, and Jev on or off). The
80
+ registry stays read-only: the skill directory is often a live symlink to a checkout.
81
+
82
+ Writing is protected, because anything that can reach the server could otherwise change your
83
+ runs. The server makes a random token each time it starts and puts it in the page it serves;
84
+ the page sends it with every change, and the server also checks the `Host` and `Origin`
85
+ headers, so another website open in your browser cannot make your browser change a run. Reading
86
+ needs no token. `--read-only` (or `FORGE_UI_READONLY=1`) turns writing off altogether: no
87
+ token is issued and every change is refused. `FORGE_UI_READONLY=1 forge ui open` starts such a
88
+ server, and the launcher never reuses a server running in the other mode; it replaces it.
89
+
90
+ Also: replay, critical path, Fractal child-task trees, quick jump (Ctrl K, ⌘K on a Mac, or
91
+ `/`), a bell for runs that finish or fail, failed tasks and final-review verdicts (with opt-in
92
+ browser notifications), and a run exported as one HTML file that opens without forge-ui, with
93
+ prompts, diffs and local paths left out unless you choose them.
94
+
95
+ Offline (it loads three.js from a CDN for the 3D view) the same page works in 2D. It is opt-in.
96
+ The `/forge` skill asks "Open forge-ui to watch this run? [y/N]" at the start of every new
97
+ interactive run, or you pass it yourself:
98
+
99
+ ```sh
100
+ bash scripts/forge-solo.sh "$RUN" --repo "$REPO" --dwarf sol --ui
101
+ bash scripts/forge-parallel.sh plan "$PLAN" --repo "$REPO" --decompose-level medium --ui
102
+ forge ui open [DIR] # an existing run dir, or the run list
103
+ ```
104
+
105
+ All runs share one detached server on `127.0.0.1:8765` (`FORGE_UI_PORT`) that exits after
106
+ four idle hours (`FORGE_UI_IDLE_S`). A UI that cannot start never fails the run. Reading needs
107
+ no token: any local process can read the prompts and diffs of the runs it shows; only changes
108
+ need the token (see [Controls](#controls)). See
109
+ [references/ui.md](../references/ui.md) and [ui/README.md](../ui/README.md).
110
+
111
+ To look around before your first run, point it at made-up runs (the ones in these pictures), from a git checkout:
112
+
113
+ ```sh
114
+ python3 ui/tools/demo_runs.py /tmp/forge-demo
115
+ (cd ui && python3 -m forge_ui --no-registry --root /tmp/forge-demo --open)
116
+ ```
117
+
118
+ ## `forge ui` command group
119
+
120
+ | Command | Purpose |
121
+ | --- | --- |
122
+ | `open [DIR] [--no-browser] [--port N]` | start or reuse the shared forge-ui server (never one in the other mode); print `forge-ui: <url>` for the plan or solo run dir `DIR` (the run list without it) and open a browser the first time (a later `open` of the same run says on stderr that it opened none) |
123
+ | `status [--json]` | exit 0 and describe the server (and whether it is read-only) when it runs, 1 when not |
124
+ | `stop` | stop the server |
125
+
126
+ `FORGE_UI=off` makes `--ui` on a runner start nothing (it does not stop a running server or block `forge ui open`; use `forge ui stop`), `FORGE_UI_BROWSER=off` never opens a browser,
127
+ `FORGE_UI_PORT` (8765) and `FORGE_UI_IDLE_S` (14400, `0` = never) tune the server, and `FORGE_UI_READONLY=1` starts it read-only. State and
128
+ `server.log` are in `${XDG_STATE_HOME:-~/.local/state}/forge/ui/`.