@hominis/fireforge 0.44.2 → 0.44.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/dist/build-info.json +4 -4
- package/dist/src/commands/token.js +3 -1
- package/dist/src/core/token-variant.d.ts +12 -2
- package/dist/src/core/token-variant.js +106 -17
- package/docs/README.md +35 -0
- package/docs/configuration.md +116 -0
- package/docs/exit-codes.md +63 -0
- package/docs/furnace.md +57 -0
- package/docs/lifecycle-invariants.md +309 -0
- package/docs/machine-output.md +94 -0
- package/docs/patch-workflow.md +119 -0
- package/docs/run-logs.md +60 -0
- package/docs/testing.md +92 -0
- package/docs/verification-trees.md +48 -0
- package/package.json +2 -1
|
@@ -0,0 +1,309 @@
|
|
|
1
|
+
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
|
2
|
+
|
|
3
|
+
# Mutation lifecycle invariants (internal)
|
|
4
|
+
|
|
5
|
+
Internal architecture notes for contributors. FireForge's safety behavior
|
|
6
|
+
around destructive operations is deliberately spread across small modules;
|
|
7
|
+
this document is the one place that states the invariants those modules
|
|
8
|
+
uphold **together**, and which module owns which piece. If you add a new
|
|
9
|
+
mutating command, read the decision table at the bottom first.
|
|
10
|
+
|
|
11
|
+
## The invariants
|
|
12
|
+
|
|
13
|
+
1. **Only `bin/fireforge.ts` terminates the process.** Library code
|
|
14
|
+
propagates `CommandError` / `FireForgeError`; it never calls
|
|
15
|
+
`process.exit`. Enforced by the process-boundary test.
|
|
16
|
+
2. **At most one real engine/patch mutation runs at a time** per resource:
|
|
17
|
+
CLI build/test/export/re-export/furnace mutations first take the
|
|
18
|
+
`.fireforge/engine-session.lock`, then subsystem locks such as build,
|
|
19
|
+
furnace, or patch-directory locks. Furnace mutations serialize on a
|
|
20
|
+
per-root lock, patch directory mutations (filename allocation + manifest
|
|
21
|
+
writes) on a per-directory lock. Dry-runs hold the engine-session lock
|
|
22
|
+
too — a `re-export --dry-run` over an untracked binary legally touches
|
|
23
|
+
the git index, and holding the lock serializes even "read-only"
|
|
24
|
+
previews against real writers.
|
|
25
|
+
3. **A furnace mutation that does not finish restores the engine or says so.**
|
|
26
|
+
Both a signal (SIGINT/SIGTERM) and a thrown error trigger a journal-based
|
|
27
|
+
rollback; if rollback is incomplete or uncertain, a `pendingRepair` marker
|
|
28
|
+
blocks all further furnace mutations until
|
|
29
|
+
`fireforge doctor --repair-furnace` reconciles.
|
|
30
|
+
|
|
31
|
+
The thrown-error half landed in 0.43.0. Before it, `runFurnaceMutation`'s
|
|
32
|
+
`finally` only deregistered the operation — and because it also set
|
|
33
|
+
`completed`, a body that threw removed itself from the signal handler's
|
|
34
|
+
view as well, so a throw outside a body's own catch left the checkout
|
|
35
|
+
torn with no marker. `applyAllComponents` was the live instance: it rolls
|
|
36
|
+
back on collected `result.errors`/`stepErrors`, but a throw from its
|
|
37
|
+
pre/post-loop work escaped both. The wrapper now runs cleanups, restores
|
|
38
|
+
the journal, and rethrows the **original** error (a rollback failure
|
|
39
|
+
becomes a marker plus a warning, never a replacement error).
|
|
40
|
+
|
|
41
|
+
Both paths race for the same journal, so each operation carries a
|
|
42
|
+
`rollbackState` claimed synchronously before either path's first await;
|
|
43
|
+
whichever gets there first restores and the other skips. Bodies that
|
|
44
|
+
already restore on their own — the eight `furnace/*` command bodies,
|
|
45
|
+
`furnace-apply`, `preview` — call `ctx.markRolledBack()` so the wrapper
|
|
46
|
+
keeps its hands off. One deliberate asymmetry: on the throw path a body
|
|
47
|
+
that never registered a journal writes **no** marker, because a
|
|
48
|
+
pre-flight refusal would otherwise block every later mutation behind a
|
|
49
|
+
repair with nothing to reconcile; the signal path still writes one there,
|
|
50
|
+
since an interrupt genuinely may have landed mid-write.
|
|
51
|
+
|
|
52
|
+
4. **A signal during a compound "apply + persist state" pair never tears the
|
|
53
|
+
pair.** Rebase-style operations intentionally leave the engine mutated,
|
|
54
|
+
so rollback is the wrong tool; instead the exit is _held_ (bounded) until
|
|
55
|
+
the bookkeeping write lands, so `--continue` never sees stale progress.
|
|
56
|
+
5. **No lock outlives the process it belongs to** (best effort, two layers):
|
|
57
|
+
the signal pipeline force-releases furnace locks before exiting, and
|
|
58
|
+
stale-lock recovery (PID-first, then age) unblocks the next command for
|
|
59
|
+
anything that escaped.
|
|
60
|
+
6. **Destructive patch operations are gated and audited**: explicit change
|
|
61
|
+
summary, confirmation (or `--yes`), `--dry-run`, hard refusal on
|
|
62
|
+
structural conflicts (`--force-unsafe` to override), and a JSONL history
|
|
63
|
+
entry appended only after the mutation succeeded. This covers metadata
|
|
64
|
+
mutations too: `patch tier` and `patch lint-ignore` route through
|
|
65
|
+
`confirmDestructive` (they used to accept `--yes` without ever
|
|
66
|
+
prompting, so the flag only appeared in the history record). Enforced
|
|
67
|
+
by `patch-tier-and-lint-ignore.integration.test.ts`.
|
|
68
|
+
7. **A verification tree never mutates shared state**. Trees
|
|
69
|
+
are full project-root CoW snapshots under `.fireforge/trees/<name>`
|
|
70
|
+
with NO merge-back model. Enforcement is layered: the `.fireforge/
|
|
71
|
+
tree.json` marker plus a default-deny command-verdict table
|
|
72
|
+
(`src/core/tree-guard.ts`) refuse every mutating command through one
|
|
73
|
+
commander `preAction` hook in `cli.ts` — a newly added command is
|
|
74
|
+
refused in trees until classified (drift test). Because a tree is a
|
|
75
|
+
full project root, its build and engine-session locks key on the TREE
|
|
76
|
+
root, so in-tree reads never contend with the primary. `tree create`
|
|
77
|
+
snapshots under the PRIMARY engine-session lock (never captures a
|
|
78
|
+
mid-mutation state) — and with `--with-objdir` additionally under the
|
|
79
|
+
primary BUILD lock, under which the objdir is resolved and its build
|
|
80
|
+
artifacts validated immediately before cloning (any pre-lock probe is
|
|
81
|
+
a fast-fail courtesy whose result is discarded, never the
|
|
82
|
+
authoritative answer), so a concurrent `fireforge build` can neither
|
|
83
|
+
tear the obj-\* snapshot nor swap it between preflight and clone.
|
|
84
|
+
A symlinked (or engine-escaping) primary obj-\* is refused BEFORE any
|
|
85
|
+
copying: every `cp` mode the clone uses preserves symlinks, so the
|
|
86
|
+
clone would carry a link back to the original build and the mozinfo
|
|
87
|
+
rewrite plus `_virtualenvs` removal would mutate that original
|
|
88
|
+
through it — `assertCloneSafeObjdir` (tree-store.ts) checks lstat and
|
|
89
|
+
realpath containment (lexical `resolve()` cannot see either) on the
|
|
90
|
+
primary objdir and again on the tree's copy before any write goes
|
|
91
|
+
through it, pinned by `tree-store.integration.test.ts` and
|
|
92
|
+
`tree.integration.test.ts`;
|
|
93
|
+
tree create/remove serialise on a primary-side
|
|
94
|
+
`.fireforge/trees.lock`; `tree remove` refuses while a live PID holds
|
|
95
|
+
a tree lock and path-contains its `rm -rf` target.
|
|
96
|
+
|
|
97
|
+
In-tree `test` (0.41.0) is the one conditional carve-out, and it is
|
|
98
|
+
marker-gated: mach objdirs embed absolute `topsrcdir`/`topobjdir`
|
|
99
|
+
paths, so `tree create --with-objdir` rewrites the cloned objdir's
|
|
100
|
+
`mozinfo.json` to the tree via the same safe-relocation rewriter
|
|
101
|
+
`build --rewrite-mozinfo` uses, scrubs the cloned `_virtualenvs`
|
|
102
|
+
(primary shebangs; mach rebuilds them in-tree), runs `mach configure`
|
|
103
|
+
inside the tree so the configure-generated root files are regenerated
|
|
104
|
+
against the tree's paths rather than retaining the primary's — and
|
|
105
|
+
verifies exactly that set afterwards (config.status, backend.mk,
|
|
106
|
+
Makefile, config/autoconf.mk, via `findObjdirRelocationViolation`,
|
|
107
|
+
pinned by `mach-objdir-relocation.test.ts`; nested Makefiles are
|
|
108
|
+
products of the verified config.status, and `.deps` build products
|
|
109
|
+
are out of scope as read-only staleness corrected by the first
|
|
110
|
+
in-tree rebuild) — and copies
|
|
111
|
+
`last-build.json` so the in-tree stale-build gates anchor correctly.
|
|
112
|
+
Both relocation steps are fail-closed — a rewrite refusal or a failed
|
|
113
|
+
in-tree configure aborts the create and removes the partial tree —
|
|
114
|
+
and only after both succeed is the marker's `clonedObjdir` field
|
|
115
|
+
written, which is what the guard's `test` predicate consults. At run
|
|
116
|
+
time, `test` additionally refuses when the objdir its preflight found
|
|
117
|
+
is not the one the marker vouches for
|
|
118
|
+
(`assertObjdirMatchesTreeMarker`), so an objdir that appeared through
|
|
119
|
+
any path other than `tree create --with-objdir` is never run against.
|
|
120
|
+
`test --build` stays refused (a tree is a snapshot with no merge-back
|
|
121
|
+
model), and a corrupt marker refuses `test` like every conditional
|
|
122
|
+
verdict.
|
|
123
|
+
|
|
124
|
+
**Both halves of that enforcement fail CLOSED on an unreadable state,
|
|
125
|
+
because "we cannot tell" is not "there is nothing there."** A marker
|
|
126
|
+
that exists but does not parse reports `corrupt`, not `absent`, and
|
|
127
|
+
refuses the command — reading it as absent granted a snapshot the
|
|
128
|
+
full mutating command set on the strength of a file that failed to
|
|
129
|
+
parse. A lock directory whose owner cannot be identified reports
|
|
130
|
+
`unknown`, not free, and refuses removal — `withFileLock` creates the
|
|
131
|
+
lock directory before writing its owner record and treats that write
|
|
132
|
+
as non-fatal, so a live holder legitimately produces a lock with no
|
|
133
|
+
readable PID (a lock directory that vanished between the probe and the
|
|
134
|
+
owner read is the one exception: that is a clean release, and reads as
|
|
135
|
+
free). Each has one named escape (`--ignore-corrupt-tree-marker`,
|
|
136
|
+
`tree remove --force`); neither has a silent one. The corrupt-marker
|
|
137
|
+
refusal spares the unconditionally-**allowed** read-only commands
|
|
138
|
+
(`status`, `lint`, `typecheck`, `verify`): their verdict never consults
|
|
139
|
+
marker fields, so an unreadable marker cannot change their answer, and
|
|
140
|
+
blocking them left the operator unable to diagnose the very tree that
|
|
141
|
+
needs inspecting. `conditional` verdicts can write and stay refused.
|
|
142
|
+
Enforced by
|
|
143
|
+
`tree-guard.test.ts` (verdict drift gate + corrupt-marker refusal),
|
|
144
|
+
`tree-store.integration.test.ts` (marker tri-state, unknown-owner
|
|
145
|
+
refusal, rewrite-then-reconfigure-then-marker ordering),
|
|
146
|
+
`tree.test.ts` (under-lock objdir re-validation ordering) and
|
|
147
|
+
`tree.integration.test.ts` (real-program refusal).
|
|
148
|
+
|
|
149
|
+
8. **A verdict is only reported when it was actually verified.** `fireforge
|
|
150
|
+
test` brackets the harness run with `snapshotEngineGeneration` and
|
|
151
|
+
fails the run if `engine/` changed underneath it. The probe can
|
|
152
|
+
legitimately fail — `engine/` need not be a git checkout — so a probe
|
|
153
|
+
that fails on **both** sides warns and proceeds; a one-sided
|
|
154
|
+
transition throws, because either the second probe measured nothing
|
|
155
|
+
about a checkout that had something to measure, or there was no
|
|
156
|
+
baseline to compare against. Enforced by
|
|
157
|
+
`engine-session-lock.test.ts`.
|
|
158
|
+
|
|
159
|
+
A probe that fails on a **transient** `.git/index.lock` — another
|
|
160
|
+
writer held the index for the instant we looked — is retried on a
|
|
161
|
+
bounded budget rather than latched as "unmeasurable";
|
|
162
|
+
a durable failure (a non-git `engine/`, an unreadable `.git`) still
|
|
163
|
+
reports immediately. Enforced by
|
|
164
|
+
`engine-generation-probe-retry.test.ts`.
|
|
165
|
+
|
|
166
|
+
The same invariant reaches across the `tree exec` boundary.
|
|
167
|
+
`tree exec` hands stdout to the child (`stdio: 'inherit'`), so the
|
|
168
|
+
child's `FIREFORGE-VERDICT:` line reaches the caller byte-for-byte; the
|
|
169
|
+
parent therefore seals stdout the moment the spawn settles, so its own
|
|
170
|
+
"exited with code N" refusal cannot print after the verdict and displace
|
|
171
|
+
it. A refusal raised BEFORE the child spawns emits no verdict at all —
|
|
172
|
+
consumers must read a missing verdict as a failed step, never a pass.
|
|
173
|
+
|
|
174
|
+
9. **FireForge's own read-only commands are not writers of the primary
|
|
175
|
+
engine checkout**. `verify`, `lint --per-patch`, and
|
|
176
|
+
`typecheck` are read-only to the operator but not to git: `git status`
|
|
177
|
+
and `git diff HEAD` refresh and rewrite `.git/index`, and the
|
|
178
|
+
untracked-binary diff path stages and unstages for real. Each of those
|
|
179
|
+
commands therefore runs its whole body inside `withPrivateGitIndex`,
|
|
180
|
+
which seeds a temp `GIT_INDEX_FILE` from the repository's own index so
|
|
181
|
+
every refresh lands there and is discarded; `typecheck` additionally
|
|
182
|
+
forces `incremental: false` and drops `tsBuildInfoFile`, so no sidecar
|
|
183
|
+
is written under `engine/`. The scope is fail-open — an unresolvable
|
|
184
|
+
git dir simply does not install it — and non-reentrant. This exists so
|
|
185
|
+
invariant 8 stays evidential: a concurrent `fireforge test` must not be
|
|
186
|
+
invalidated by FireForge's own read-only lane. Enforced by
|
|
187
|
+
`git-readonly-index.test.ts` (real repository, real index mtime).
|
|
188
|
+
|
|
189
|
+
10. **Lock queues are observable, and leaving one is as reliable as
|
|
190
|
+
joining it**. A waiter that contends registers a file in
|
|
191
|
+
`<lock>.waiters/` named `<startedAtMs>-<pid>-<uuid>` and deregisters
|
|
192
|
+
in a `finally` that covers both outcomes — acquired the lock, or timed
|
|
193
|
+
out. Depth counts only entries whose PID is still alive, so a killed
|
|
194
|
+
waiter cannot inflate it forever, and the registry is advisory:
|
|
195
|
+
failures to register are logged verbosely and never fail an
|
|
196
|
+
acquisition. `fireforge status --lock` and the `--wait-lock` progress
|
|
197
|
+
line read it. Enforced by `lock-visibility.test.ts` (real concurrent
|
|
198
|
+
waiter) and `engine-session-lock.test.ts` (queue position in the wait
|
|
199
|
+
line).
|
|
200
|
+
|
|
201
|
+
11. **A manifest repair preserves what it cannot recompute, or refuses.**
|
|
202
|
+
`doctor --repair-patches-manifest` is a MERGE: only `filesAffected` and
|
|
203
|
+
`order` are derived from the patch files, and every other field on an
|
|
204
|
+
existing entry is carried forward by spreading it, so a field a `.patch`
|
|
205
|
+
body cannot express — `stagedDependencies`, `lintIgnore`, `tier`, and
|
|
206
|
+
whatever is added to `PatchMetadata` next — survives by construction
|
|
207
|
+
rather than by being listed. The one case where nothing can be carried
|
|
208
|
+
forward is a `patches.json` that exists but does not parse; the rebuild
|
|
209
|
+
refuses it, naming what would be reinvented, unless
|
|
210
|
+
`--allow-metadata-loss` is passed. `doctor --repair-files-affected` is
|
|
211
|
+
the narrow repair for drift that is only in the derived list: it goes
|
|
212
|
+
through `mutatePatchRowsInManifest`, so untouched rows keep their exact
|
|
213
|
+
JSON. Both repairs take the patch-directory lock (invariant 2), both
|
|
214
|
+
support `--dry-run`, and a repair that wrote is reported before the
|
|
215
|
+
doctor summary in every branch — the run can still exit non-zero on an
|
|
216
|
+
unrelated check, and a non-zero exit otherwise reads as "nothing
|
|
217
|
+
happened". Enforced by `patch-manifest-repair.test.ts` (real temp
|
|
218
|
+
queues) and the repair cases in `doctor.test.ts`.
|
|
219
|
+
|
|
220
|
+
## Lock release (0.44.0)
|
|
221
|
+
|
|
222
|
+
`releaseLockIfOwned(lockPath)` in `src/core/file-lock.ts` is the ownership-
|
|
223
|
+
checked removal for callers that hold no acquisition token — currently the
|
|
224
|
+
signal-time furnace sweeper (`forceReleaseFurnaceLocksForActiveOperations`).
|
|
225
|
+
It verifies the PID before removing, which is the half of `releaseLock`'s
|
|
226
|
+
guard a sweeper can still perform: the sweeper knows which lock paths its own
|
|
227
|
+
operations opened but not the per-acquisition UUID they minted. The bare `rm`
|
|
228
|
+
it replaced could delete a lock a DIFFERENT process had acquired in the window
|
|
229
|
+
between our operation dying and the sweep running.
|
|
230
|
+
|
|
231
|
+
`completeJournalRollback` in `src/core/furnace-operation.ts` owns the
|
|
232
|
+
mutation-body rollback sequence that eight furnace commands previously
|
|
233
|
+
duplicated verbatim. The ordering is the invariant: `markRolledBack()` runs
|
|
234
|
+
BEFORE the restore, so a restore that itself fails does not leave the
|
|
235
|
+
lifecycle wrapper replaying the same journal on the way out; and the ORIGINAL
|
|
236
|
+
error is rethrown unless the rollback is what failed.
|
|
237
|
+
|
|
238
|
+
## Who owns what
|
|
239
|
+
|
|
240
|
+
| Module | Owns |
|
|
241
|
+
| --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
242
|
+
| `bin/fireforge.ts` | The only `process.exit` call sites; SIGINT/SIGTERM pipeline; re-entrancy guard (a second Ctrl+C exits immediately); exit codes 130/143. |
|
|
243
|
+
| `src/core/file-lock.ts` | Generic directory-as-lock (`mkdir` atomicity), owner PID **+ per-acquisition token** file, stale recovery via **rename-aside reaping** (dead PID unblocks immediately; ageless fallback after 5 min; the probe re-runs every ~5 s while waiting, so a holder dying mid-wait cannot strand waiters until `timeoutMs`), poll + timeout. Release verifies the lock still carries this acquisition's token before removing — a reaper-replaced lock is never deleted by its previous owner. Enforced by `file-lock.test.ts` (two-waiter reap race, mid-wait holder death, foreign-owner release). Releases in `finally` — which never runs across `process.exit`, hence invariant 5's other layer. Opt-in wait ergonomics: `pollMaxMs` (poll interval doubles from `pollMs` up to the cap) and `waitProgressMs`/`onWaitProgress` (periodic progress callback carrying the holder's PID, liveness, and owner-metadata lines from the pid file). All three are inert when unset — existing callers keep byte-identical behavior. |
|
|
244
|
+
| `src/core/engine-session-lock.ts` | Project-wide engine-session lock under `.fireforge/`, stamped with PID, command, and start time metadata. CLI `build`, `test`, `export`, `re-export`, `furnace apply`, `furnace deploy`, and `furnace sync` take it before subsystem locks — unconditionally, dry-runs included: a `re-export --dry-run` over an untracked binary legally touches the git index (temporary `--intent-to-add` staging), so holding the lock serializes even "read-only" previews against real writers. Dry-run purity is separately enforced at runtime by `withDryRunPurityGuard` (`re-export-bulk-scan.ts`), which fingerprints the engine tree and every patch artifact around the dry-run and fails hard on divergence. Contention fails fast after ~1 s by default with the fixed message "Another FireForge engine-mutating command is already running. Wait for it to finish, then retry `<command>`." — that message contract is unchanged and also closes the bounded wait. Each of those commands accepts `--wait-lock [seconds]` (bare flag = 60, valid 1..3600) for a bounded operator wait instead: polling backs off exponentially 100 ms → 2 s, and roughly every 5 s a progress line identifies the holder from the lock's owner-metadata lines (`Waiting for the FireForge engine lock held by PID 12345 (command=build, started=…) — 10s of up to 60s.`). The budget is not the engine lock's alone: it covers EVERY lock the command takes, which for the three furnace commands means the per-root furnace lock as well (see the `furnace-operation.ts` row). `FIREFORGE_WAIT_LOCK=<seconds>` sets the same budget for a whole session, parsed by the same bounds as the flag so the two cannot drift; an explicit budget at a call site always wins. Commands that take no lock accept `--wait-lock` and ignore it, so a scripted sequence can blanket-append the flag without a usage error killing the run. `fireforge test` also snapshots the engine git generation before and after the harness run and fails the verdict as invalid if the tree changes mid-run. |
|
|
245
|
+
| `src/core/furnace-operation.ts` | `runFurnaceMutation` lifecycle wrapper: per-root furnace lock, `pendingRepair` pre-flight refusal, in-flight operation registry (journal + cleanups + `completed` flag), signal rollback with bounded timeouts, `pendingRepair` persistence, furnace-lock force-release sweep. The lock's wait budget is `options.lockTimeoutMs` (threaded from `--wait-lock` by `waitLockMutationOptions`, which does the seconds→ms conversion once) falling back to `sessionWaitLockMs()` (`FIREFORGE_WAIT_LOCK`) and then to `DEFAULT_LOCK_TIMEOUT_MS`. The fallback is what covers the twelve furnace mutators that declare no flag (`create`, `remove`, `rename`, `refresh`, `scan`, `override`, `chrome-doc`, …). A contended refusal here names the FURNACE lock, says it is the second of two, and points at `--wait-lock`/`FIREFORGE_WAIT_LOCK` — a budget that is paid in full and then refuses anyway is the worst shape a budget can have, so the refusal must not read as the engine lock's. |
|
|
246
|
+
| `src/core/furnace-rollback.ts` | The rollback journal itself: file snapshots and their restoration. |
|
|
247
|
+
| `src/utils/assert.ts` | The runtime-assertion primitive (`assert`, `assertDefined`, `expectDefined`) and, through `InternalInvariantError`, exit code 11. Checks state FireForge itself established — journal-before-mutation, lock-held-before-write, manifest order well-formedness — and is deliberately NOT input validation: anything derived from user input, the filesystem, or a subprocess stays a typed error from `src/errors/`. Every check routes through the helper rather than an inline `if (!c) throw` so the failure branch is covered once here instead of uncovered at every call site. |
|
|
248
|
+
| `src/core/signal-critical.ts` | Registry of signal-deferred critical sections (`runInSignalCriticalSection`). Pure registry — installs no handlers; the bin pipeline drains it with a bounded wait. |
|
|
249
|
+
| `src/core/patch-lock.ts` | `withPatchDirectoryLock` — serializes patch filename allocation and manifest read-modify-writes. **Not reentrant**: callers must not already hold it. |
|
|
250
|
+
| `src/core/destructive.ts` | The destructive-operation contract: `confirmDestructive` (summary, prompt, `--yes`, `--dry-run`, conflict refusal, `--force-unsafe`) and the `.fireforge-history.jsonl` audit log. |
|
|
251
|
+
| `src/core/tree-store.ts` | Verification-tree lifecycle: CoW snapshot contents/exclusions, the objdir clone relocation (mozinfo rewrite + caller-supplied in-tree `mach configure`, both before the marker records `clonedObjdir`; `assertObjdirMatchesTreeMarker` re-checks at `test` time), the `.fireforge/tree.json` marker and its absent/valid/**corrupt** tri-state, staleness fingerprints, the primary-side `trees.lock`, and removal that refuses both a live holder and a lock of **unknown** ownership, with rm-target containment. |
|
|
252
|
+
| `src/core/tree-guard.ts` | Read-only enforcement inside trees: the default-deny per-command verdict table and the `preAction` hook body `runTreeGuardHook` installed by `createProgram()` (invariant 7). |
|
|
253
|
+
|
|
254
|
+
## The signal pipeline, end to end
|
|
255
|
+
|
|
256
|
+
On SIGINT/SIGTERM, `bin/fireforge.ts`:
|
|
257
|
+
|
|
258
|
+
1. If a rollback is already in flight (second Ctrl+C), exits immediately.
|
|
259
|
+
2. Otherwise runs **in parallel** and waits for both:
|
|
260
|
+
- `rollbackActiveOperationsForSignal(signal)` — per active furnace
|
|
261
|
+
operation: cleanup callbacks (15 s bound each), journal restore (15 s
|
|
262
|
+
bound), `pendingRepair` marker if anything failed or no journal was
|
|
263
|
+
registered yet. Operations whose body already completed are skipped, so
|
|
264
|
+
a signal in the finally-window can't roll back a committed mutation.
|
|
265
|
+
- `waitForActiveCriticalSections(5 s)` — holds exit for in-flight
|
|
266
|
+
"apply + persist" pairs (invariant 4), bounded so a stuck write cannot
|
|
267
|
+
postpone the exit the user asked for.
|
|
268
|
+
3. Force-releases the furnace lock directories of active operations
|
|
269
|
+
(`withFileLock`'s `finally` will never run past `process.exit`).
|
|
270
|
+
4. `process.exit(130 | 143)`.
|
|
271
|
+
|
|
272
|
+
## Choosing a primitive for a new mutation
|
|
273
|
+
|
|
274
|
+
| Your operation… | Use |
|
|
275
|
+
| -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
|
|
276
|
+
| Mutates the engine and should be _undone_ on Ctrl+C (apply, deploy, preview, remove) | `runFurnaceMutation` + `ctx.registerJournal` (plus `ctx.registerCleanup` for extra teardown) |
|
|
277
|
+
| Intentionally leaves the engine mutated but must finish a paired bookkeeping write (rebase-style `--continue` state) | `runInSignalCriticalSection` around the apply + persist pair — keep the body short |
|
|
278
|
+
| Touches the patch directory / manifest | `withPatchDirectoryLock` (never nested) |
|
|
279
|
+
| Deletes or rewrites user-owned patch state | The `confirmDestructive` contract, history appended only on success |
|
|
280
|
+
| Only reads | Nothing — dry-runs must not block or be blocked |
|
|
281
|
+
| Runs read verification concurrently beside a busy primary | A `fireforge tree` (CoW snapshot; mutation refused inside — invariant 7) |
|
|
282
|
+
|
|
283
|
+
## Where this is tested
|
|
284
|
+
|
|
285
|
+
- `src/core/__tests__/file-lock.test.ts` — lock acquisition, staleness, PID
|
|
286
|
+
recovery.
|
|
287
|
+
- `src/core/__tests__/furnace-operation.test.ts` — wrapper lifecycle, signal
|
|
288
|
+
rollback outcomes, `pendingRepair` semantics, lock force-release.
|
|
289
|
+
- `src/core/__tests__/signal-critical.test.ts` — the critical-section
|
|
290
|
+
registry contract in isolation.
|
|
291
|
+
- `src/core/__tests__/signal-compound-mutation-scenario.test.ts` — the
|
|
292
|
+
composed bin-handler pipeline: exit held for a compound mutation, bounded
|
|
293
|
+
hold for a stuck section, rollback + drain + lock release together.
|
|
294
|
+
- `src/core/__tests__/destructive.test.ts` — the destructive-op contract.
|
|
295
|
+
- `src/core/__tests__/tree-guard.test.ts` — invariant 7's verdict table
|
|
296
|
+
(default-deny drift gate, conditional predicates, corrupt-marker refusal).
|
|
297
|
+
- `src/core/__tests__/tree-store.integration.test.ts` — clone contents,
|
|
298
|
+
objdir rewrite/reconfigure/marker ordering, marker tri-state,
|
|
299
|
+
unknown-owner removal refusal.
|
|
300
|
+
- `src/commands/__tests__/tree.test.ts` — create/remove command flow,
|
|
301
|
+
including objdir re-validation under the primary build lock, and the
|
|
302
|
+
`tree exec` stdout seal (invariant 8).
|
|
303
|
+
- `src/__tests__/tree-exec-verdict.test.ts` — invariant 8 across a real
|
|
304
|
+
process boundary: exactly one verdict line, last on stdout, refusal on
|
|
305
|
+
stderr.
|
|
306
|
+
- `src/commands/__tests__/tree.integration.test.ts` — real-program in-tree
|
|
307
|
+
refusals and `--with-objdir` end-to-end against a synthetic objdir.
|
|
308
|
+
- `src/__tests__/process-boundary.test.ts` — invariant 1 (`process.exit`
|
|
309
|
+
only in bin).
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
|
2
|
+
|
|
3
|
+
# Machine-readable output contract (internal)
|
|
4
|
+
|
|
5
|
+
The `--json` surface is a contract with scripted consumers. Before 0.44.0 it
|
|
6
|
+
was four unrelated shapes: `status --json` and `tree list --json` shared a
|
|
7
|
+
`schemaVersion: 1` envelope but disagreed on failure, `test` emitted a
|
|
8
|
+
`FIREFORGE-VERDICT:` line, and `lint --per-patch --report` wrote a versioned
|
|
9
|
+
file nothing ever reads back. This file is the single statement of what a
|
|
10
|
+
`--json` command must do, so the next one added does not invent a fifth.
|
|
11
|
+
|
|
12
|
+
## The rules
|
|
13
|
+
|
|
14
|
+
1. **Success writes exactly one JSON document to stdout**, with
|
|
15
|
+
`schemaVersion` as its first key.
|
|
16
|
+
2. **Failure writes exactly one error document to stdout**, of the shape
|
|
17
|
+
`{ "schemaVersion": 1, "error": "<human message>", "code": "<tag>" }`,
|
|
18
|
+
and then exits non-zero. The `error` text is the same sentence the
|
|
19
|
+
non-JSON mode would print; `code` is a stable machine-readable tag
|
|
20
|
+
(`engine-missing`, `engine-not-git`, `tree-list-failed`, …) that consumers
|
|
21
|
+
may branch on.
|
|
22
|
+
3. **Machine mode is engaged before any output.** `setMachineOutputMode(true)`
|
|
23
|
+
must run before the first write, so every diagnostic — including one
|
|
24
|
+
rendered later by `withErrorHandling` — routes to stderr and cannot
|
|
25
|
+
interleave with the payload.
|
|
26
|
+
4. **stdout carries nothing else.** No banners, no progress, no warnings.
|
|
27
|
+
`logger.ts` enforces this through a single `routeToStderr()` gate that all
|
|
28
|
+
its helpers consult, including `spinner()`, `note()` and `intro`/`outro`.
|
|
29
|
+
|
|
30
|
+
## Implementation
|
|
31
|
+
|
|
32
|
+
Use `emitMachineError` from `src/utils/machine-output.ts` for rule 2. It
|
|
33
|
+
writes the envelope and throws `CommandError`, which the CLI boundary treats
|
|
34
|
+
as already-rendered — so the boundary adds no second rendering on stderr.
|
|
35
|
+
|
|
36
|
+
```ts
|
|
37
|
+
if (options.json === true) {
|
|
38
|
+
setMachineOutputMode(true);
|
|
39
|
+
try {
|
|
40
|
+
const payload = await collect();
|
|
41
|
+
process.stdout.write(
|
|
42
|
+
`${JSON.stringify({ schemaVersion: MACHINE_OUTPUT_SCHEMA_VERSION, ...payload }, null, 2)}\n`
|
|
43
|
+
);
|
|
44
|
+
} catch (error: unknown) {
|
|
45
|
+
emitMachineError('collect-failed', toError(error).message);
|
|
46
|
+
} finally {
|
|
47
|
+
setMachineOutputMode(false);
|
|
48
|
+
}
|
|
49
|
+
return;
|
|
50
|
+
}
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## Current surfaces
|
|
54
|
+
|
|
55
|
+
| Surface | Shape | Notes |
|
|
56
|
+
| ---------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
57
|
+
| `status --json` | `{ schemaVersion: 1, … }` | Error envelope on three engine preconditions |
|
|
58
|
+
| `tree list --json` | `{ schemaVersion: 1, trees }` | Error envelope added in 0.44.0 |
|
|
59
|
+
| `test` verdict line | `FIREFORGE-VERDICT: PASS\|FAIL reason=… [log=<path>]` | Not JSON. A single final stdout line, sealed by `setStdoutSealed` so nothing can displace it. `log=` names the run's own complete output under `.fireforge/logs/`, so a piped or truncated run still leaves a re-readable artifact |
|
|
60
|
+
| `lint --per-patch --report <path>` | `{ schemaVersion: 1, … }` | Written to a file, never read back by FireForge |
|
|
61
|
+
| `status --raw` | `git status --porcelain` lines | Not JSON, but a machine surface: it engages the same stdout discipline, so rules 3 and 4 apply and rules 1 and 2 do not. A clean tree prints nothing |
|
|
62
|
+
| `build` log announcement | `Full build output: <path>` | Not JSON. `build` prints no verdict line, so it names its run log directly on the way out |
|
|
63
|
+
|
|
64
|
+
`--raw` is why `cli.ts` reads `process.argv` for `--json` **or** `--raw`
|
|
65
|
+
before dispatching: rule 3 is about who owns stdout, and that question is
|
|
66
|
+
settled the same way for a porcelain stream as for a JSON document.
|
|
67
|
+
|
|
68
|
+
`doctor` and `verify` have no `--json` mode. `doctor` already computes a
|
|
69
|
+
serializable `DoctorCheck[]`, so adding one is cheap if a consumer asks —
|
|
70
|
+
follow the rules above rather than inventing a shape.
|
|
71
|
+
|
|
72
|
+
The run logs behind `log=` are specified in [`run-logs.md`](run-logs.md).
|
|
73
|
+
|
|
74
|
+
## Versioning
|
|
75
|
+
|
|
76
|
+
`schemaVersion` is `MACHINE_OUTPUT_SCHEMA_VERSION` in
|
|
77
|
+
`src/utils/machine-output.ts`. Additive changes (a new key) do not bump it.
|
|
78
|
+
Removing or retyping a key does, and needs a note in the changelog naming
|
|
79
|
+
what moved.
|
|
80
|
+
|
|
81
|
+
## `test` verdict reasons
|
|
82
|
+
|
|
83
|
+
`FIREFORGE-VERDICT: FAIL reason=<reason>` uses a closed set
|
|
84
|
+
(`FireforgeVerdictReason`, `src/commands/test-verdict.ts`):
|
|
85
|
+
|
|
86
|
+
| Reason | Meaning |
|
|
87
|
+
| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
88
|
+
| `crash` | The harness or the browser died; no trustworthy suite result. |
|
|
89
|
+
| `no-tests` | The run dispatched but nothing ran. |
|
|
90
|
+
| `test-failures` | The suite ran and reported unexpected results. |
|
|
91
|
+
| `preflight` | The run was refused before the harness was reached. |
|
|
92
|
+
| `inconclusive` | A result exists but `engine/` moved under it, so it was discarded. |
|
|
93
|
+
| `lock-timeout` | The run never started: the engine session lock stayed contended. |
|
|
94
|
+
| `killed` | A signal terminated the run. Written from the signal handler so a log tail is always self-describing — a killed run must never be silent, or "killed", "still running" and "never started" become indistinguishable. |
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
|
2
|
+
|
|
3
|
+
# Patch workflow
|
|
4
|
+
|
|
5
|
+
The day-to-day loop: change code inside `engine/`, export it, keep the queue
|
|
6
|
+
applying cleanly. `fireforge --help` remains the authority on flags; this file
|
|
7
|
+
covers the behaviour that a flag list cannot state.
|
|
8
|
+
|
|
9
|
+
## The loop
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
npx fireforge status
|
|
13
|
+
npx fireforge export browser/base/content/browser.js --name custom-toolbar --category ui
|
|
14
|
+
npx fireforge re-export custom-toolbar
|
|
15
|
+
npx fireforge lint --per-patch
|
|
16
|
+
npx fireforge verify
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## Exporting
|
|
20
|
+
|
|
21
|
+
Patch names are normalized on export and rename: `window-chrome-tests`,
|
|
22
|
+
`ui-window-chrome-tests`, and `235-ui-window-chrome-tests` all resolve to the
|
|
23
|
+
same `<order>-ui-window-chrome-tests.patch` shape.
|
|
24
|
+
|
|
25
|
+
For large generated asset batches, `re-export --scan --scan-files <manifest>`
|
|
26
|
+
assigns files to their owner patches without broad directory scanning. The
|
|
27
|
+
manifest is JSON:
|
|
28
|
+
|
|
29
|
+
```json
|
|
30
|
+
{
|
|
31
|
+
"assignments": [
|
|
32
|
+
{ "patch": "002-branding-runtime-icons.patch", "files": ["browser/branding/hominis/icon.svg"] }
|
|
33
|
+
]
|
|
34
|
+
}
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
The command is dry-runnable, refuses ambiguous ownership, and reports each
|
|
38
|
+
file-to-patch assignment before refreshing the patch.
|
|
39
|
+
|
|
40
|
+
Scan-less `re-export` previews drift; `--refuse-foreign-drift --expect <path>`
|
|
41
|
+
restricts it to intended files.
|
|
42
|
+
|
|
43
|
+
### Deletions
|
|
44
|
+
|
|
45
|
+
A deleted upstream file is captured like any other change: a path that is
|
|
46
|
+
tracked in HEAD but absent from disk stays in the diff scope, and the emitted
|
|
47
|
+
patch carries a `deleted file mode` section. A captured deletion keeps its
|
|
48
|
+
`filesAffected` entry — the patch is what performs the deletion, so pruning
|
|
49
|
+
the entry would leave the body deleting a file the manifest says the patch has
|
|
50
|
+
nothing to do with — and it still requires the confirmation, because the patch
|
|
51
|
+
will remove those files wherever it is applied.
|
|
52
|
+
|
|
53
|
+
An **untracked** absent path is refused rather than warned about: a patch
|
|
54
|
+
cannot express it either way. The `--files` path also refuses when any
|
|
55
|
+
requested path produces no hunk, checked against the full requested list
|
|
56
|
+
rather than the post-filter subset, so nothing dropped upstream can pass as a
|
|
57
|
+
silent success.
|
|
58
|
+
|
|
59
|
+
## Queue maintenance
|
|
60
|
+
|
|
61
|
+
Under `fireforge patch`:
|
|
62
|
+
|
|
63
|
+
- `patch compact` closes ordinal gaps (range-aware when `patchPolicy.ranges`
|
|
64
|
+
is configured).
|
|
65
|
+
- `patch reorder` moves a patch.
|
|
66
|
+
- `patch split <source> --files <paths...> --name <name>` carves files out of
|
|
67
|
+
a patch into a new one as a single transaction — including staged-dependency
|
|
68
|
+
owner rewrites — with `--dry-run` support.
|
|
69
|
+
- `patch rename` also supports `--category` and `--order`.
|
|
70
|
+
|
|
71
|
+
Every queue-mutating patch command accepts `--wait-lock [seconds]`; see
|
|
72
|
+
[`configuration.md`](configuration.md#lock-waits) for how the budget resolves.
|
|
73
|
+
|
|
74
|
+
## Linting the queue
|
|
75
|
+
|
|
76
|
+
`lint --per-patch --patches` accepts repeated flags, comma lists, full
|
|
77
|
+
filenames/stems, manifest names, category-prefixed slugs, and bare slugs.
|
|
78
|
+
|
|
79
|
+
`lint --per-patch --max-warnings 0` is the warning-clean form for a release
|
|
80
|
+
gate.
|
|
81
|
+
|
|
82
|
+
`lint --per-patch --report <path>` writes a machine-readable JSON report with
|
|
83
|
+
each patch's line count, tier, active size thresholds, issues, and
|
|
84
|
+
`lintIgnore`-suppressed issues. The size metrics
|
|
85
|
+
(`countNonBinaryDiffLines`, `resolvePatchSizeTier`, `getPatchSizeThresholds`)
|
|
86
|
+
are also exported on the programmatic API.
|
|
87
|
+
|
|
88
|
+
Per-patch lint type-checks plain Firefox JS against a built-in Firefox-globals
|
|
89
|
+
shim that tracks upstream WebIDL additions per release. See
|
|
90
|
+
[`configuration.md`](configuration.md#patchlint) for the project hooks
|
|
91
|
+
(`checkJsExtraShim`, `checkJsTestFiles`, `checkJsTestShim`).
|
|
92
|
+
|
|
93
|
+
For release whitespace checks, use `npm run whitespace:check`; it still checks
|
|
94
|
+
source diffs while excluding generated `patches/*.patch` diff syntax.
|
|
95
|
+
|
|
96
|
+
## CI enforcement
|
|
97
|
+
|
|
98
|
+
`fireforge status --check` exits non-zero when any unmanaged, drifted, or
|
|
99
|
+
conflicted file exists. `--fail-on <class,...>` tunes the policy set, and
|
|
100
|
+
`--json` composes — the JSON stays parseable and its `files[]` entries name
|
|
101
|
+
the owning `patch`. `--include-ownership` adds ownership rows to JSON status
|
|
102
|
+
output.
|
|
103
|
+
|
|
104
|
+
The JSON shapes are specified in [`machine-output.md`](machine-output.md), and
|
|
105
|
+
the codes a script should branch on in [`exit-codes.md`](exit-codes.md).
|
|
106
|
+
|
|
107
|
+
## Rebasing onto a new Firefox source release
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
npx fireforge source set --version 145.0.0esr --product firefox-esr --sha256 <archive-sha256>
|
|
111
|
+
npx fireforge download --force
|
|
112
|
+
npx fireforge rebase
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
If a patch fails, fix the reject inside `engine/`, then run
|
|
116
|
+
`npx fireforge rebase --continue`. A rebase deliberately leaves the engine
|
|
117
|
+
mutated, so its apply-and-persist pairs are held across a signal rather than
|
|
118
|
+
rolled back — see invariant 4 in
|
|
119
|
+
[`lifecycle-invariants.md`](lifecycle-invariants.md).
|
package/docs/run-logs.md
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
|
2
|
+
|
|
3
|
+
# Run logs (internal)
|
|
4
|
+
|
|
5
|
+
`fireforge test` and `fireforge build` write their own complete output to
|
|
6
|
+
`.fireforge/logs/<command>-<timestamp>.log` as they stream. The module is
|
|
7
|
+
`src/core/run-log.ts`.
|
|
8
|
+
|
|
9
|
+
## Why the tool holds the log, not the operator
|
|
10
|
+
|
|
11
|
+
A run's diagnosis used to exist in exactly one place: the terminal. The
|
|
12
|
+
ergonomic default when output is long — `fireforge test … 2>&1 | tail -40` —
|
|
13
|
+
keeps the summary and discards the `TEST-UNEXPECTED-FAIL` lines that say WHAT
|
|
14
|
+
broke, and it launders the exit code besides. Nothing else held the output:
|
|
15
|
+
`runMachCapture` keeps a 2 MB in-memory tail that dies with the process, and
|
|
16
|
+
the shard retry loop discards earlier attempts outright.
|
|
17
|
+
|
|
18
|
+
The operator rule ("never pipe a run") was written down three times downstream
|
|
19
|
+
and broken after each writing. That is the tell that it is not an operator
|
|
20
|
+
problem — the tool is the only party that can make the log survive the
|
|
21
|
+
mistake.
|
|
22
|
+
|
|
23
|
+
## The contract
|
|
24
|
+
|
|
25
|
+
- **Raw bytes, never the echo filter's collapsed form.** The artifact exists
|
|
26
|
+
to be re-read by whoever is diagnosing the run; shortening it would
|
|
27
|
+
reintroduce the loss it prevents.
|
|
28
|
+
- **Opened before any preflight**, so a refusal is logged too. Those are
|
|
29
|
+
exactly the runs whose only output a `tail` throws away.
|
|
30
|
+
- **Retained 20 deep per command kind.** Per-kind, so a busy `test` loop
|
|
31
|
+
cannot evict the `build` log an operator needs to read beside it. Pruning
|
|
32
|
+
runs before the new file is opened, and orders by the filename's
|
|
33
|
+
timestamp rather than by stat-ing every entry.
|
|
34
|
+
- **Timestamps are filesystem-safe**: the ISO form with `:` and `.` replaced
|
|
35
|
+
by `-`, which Windows accepts and which still sorts chronologically.
|
|
36
|
+
- **Best-effort throughout.** A log that cannot be opened, written or pruned
|
|
37
|
+
degrades to no log. A run must never fail over a diagnostic, so `openRunLog`
|
|
38
|
+
returns `undefined` on failure, a post-open stream error latches `broken`
|
|
39
|
+
and drops every later write, and pruning failures are logged verbosely
|
|
40
|
+
only.
|
|
41
|
+
|
|
42
|
+
## Where the path is announced
|
|
43
|
+
|
|
44
|
+
`test` publishes it as an additive `log=<path>` key on the
|
|
45
|
+
`FIREFORGE-VERDICT:` line. That is forced, not chosen: the verdict must stay
|
|
46
|
+
the run's LAST stdout write ([`machine-output.md`](machine-output.md)), so a
|
|
47
|
+
separate announcement after it would break the contract, and one before it is
|
|
48
|
+
the first thing a `tail` cuts.
|
|
49
|
+
|
|
50
|
+
`build` prints no verdict line, so it announces the path directly
|
|
51
|
+
(`Full build output: <path>`).
|
|
52
|
+
|
|
53
|
+
## Where the tee sits
|
|
54
|
+
|
|
55
|
+
At the two mach capture funnels in `src/core/mach.ts`, not at the commands.
|
|
56
|
+
That covers every dispatch: the three test dispatchers, `package`, and the
|
|
57
|
+
stdio-inheriting path `mach build` uses — which rides the collectors' existing
|
|
58
|
+
`mirror` hook rather than growing a second one. A new dispatcher added at the
|
|
59
|
+
funnels is logged by construction; one added beside them is not, which is the
|
|
60
|
+
reason to keep going through them.
|