@hominis/fireforge 0.49.2 → 0.51.0

package/docs/lifecycle-invariants.md

@@ -21,10 +21,11 @@ command, read the decision table at the bottom first.
    take the `.fireforge/engine-session.lock`, then subsystem locks such as
    build, furnace, or patch-directory locks. Furnace mutations serialize on a
    per-root lock, and patch directory mutations (filename allocation plus
-   manifest writes) on a per-directory lock. Dry runs hold the engine-session
-   lock too, because a `re-export --dry-run` over an untracked binary legally
-   touches the git index, and holding the lock serializes even "read-only"
-   previews against real writers.
+   manifest writes) on a per-directory lock. `re-export --dry-run` is the
+   exception: it writes nothing, runs its git plumbing on a private index
+   (so even the `--intent-to-add` staging an untracked binary needs never
+   reaches the shared `.git/index`), and serializes only against other dry
+   runs, so a preview no longer queues behind a peer's build or test run.
 
 3. **A furnace mutation that does not finish restores the engine or says
    so.** Both a signal (SIGINT or SIGTERM) and a thrown error trigger a
@@ -73,6 +74,17 @@ command, read the decision table at the bottom first.
    prompting, so the flag only showed up in the history record. Enforced by
    `patch-tier-and-lint-ignore.integration.test.ts`.
 
+   The history log is write-only: FireForge appends to it and no command
+   reads it back, so nothing depends on its contents or location. It lives
+   at `.fireforge/history.jsonl` (or `fireforge.json#history.path`), outside
+   the tracked patch queue. Before 0.50.0 it was
+   `patches/.fireforge-history.jsonl`; the first append after upgrading
+   moves that file, and appends its records when both exist rather than
+   overwriting either. Each record carries `invocationId` (one per FireForge
+   process), `pid`, `ppid`, and `sessionId` when `FIREFORGE_SESSION_ID` is
+   set, so a session sharing the checkout can select its own lines.
+   Enforced by `history-log-file.test.ts`.
+
 7. **A verification tree never mutates shared state.** Trees are full
    project-root CoW snapshots under `.fireforge/trees/<name>`, with no
    merge-back model.
@@ -204,16 +216,28 @@ command, read the decision table at the bottom first.
    `git-readonly-index.test.ts`, against a real repository and a real index
    mtime.
 
-10. **Lock queues are observable, and leaving one is as reliable as joining
-    it.** A waiter that contends registers a file in `<lock>.waiters/` named
-    `<startedAtMs>-<pid>-<uuid>` and deregisters in a `finally` that covers
-    both outcomes: acquired the lock, or timed out. Depth counts only entries
-    whose PID is still alive, so a killed waiter cannot inflate it forever,
-    and the registry is advisory: failures to register are logged verbosely
-    and never fail an acquisition. `fireforge status --lock` and the
-    `--wait-lock` progress line read it. Enforced by `lock-visibility.test.ts`
-    (a real concurrent waiter) and `engine-session-lock.test.ts` (queue
-    position in the wait line).
+10. **Lock queues are observable, first come first served, and leaving one
+    is as reliable as joining it.** A waiter that contends registers a ticket
+    in `<lock>.waiters/` named `<startedAtMs>-<pid>-<uuid>` and deregisters in
+    a `finally` that covers both outcomes: acquired the lock, or timed out.
+    A newcomer that finds live tickets takes one before trying the lock at
+    all. Since 0.50.0 the tickets decide the grant: only the waiter with no
+    live ticket ahead of it (arrival time, then ticket name) attempts the
+    lock, so "You are next" is a promise. Before, every waiter raced `mkdir`
+    and a waiter whose poll had backed off lost each free moment to later
+    arrivals. Because a ticket now blocks the queue, a dead one must not: a
+    ticket whose PID is gone (or reused, where the start tick can tell) is
+    skipped and removed, and one its owner has not refreshed for 30 s (every
+    waiter rewrites its ticket every 5 s) is skipped, which covers a stopped
+    waiter and PID reuse elsewhere. A registry that cannot be written falls
+    back to the old unordered contention rather than failing an acquisition.
+    This applies to every `withFileLock` lock: the engine-session, patch,
+    furnace and build locks among them. `fireforge status --lock` and the
+    `--wait-lock` progress lines read it, and the lines name the ordering
+    rule. Enforced by `file-lock-queue.test.ts` (arrival order against faster
+    later pollers, crashed, killed and stopped waiters, timeouts),
+    `lock-visibility.test.ts` (a real concurrent waiter) and
+    `engine-session-lock.test.ts` (queue position in the wait line).
 
 11. **A manifest repair preserves what it cannot recompute, or refuses.**
     `doctor --repair-patches-manifest` is a merge: only `filesAffected` and
@@ -325,13 +349,14 @@ real-procfs `withFileLock` reap of a live-PID stale record.
 The project-wide engine-session lock under `.fireforge/`, stamped with PID,
 command, and start time metadata. CLI `build`, `test`, `export`, `re-export`,
 `furnace apply`, `furnace deploy` and `furnace sync` take it before subsystem
-locks, unconditionally, including dry runs: a `re-export --dry-run` over an
-untracked binary legally touches the git index (temporary `--intent-to-add`
-staging), so holding the lock serializes even "read-only" previews against
-real writers. Dry-run purity is separately enforced at runtime by
-`withDryRunPurityGuard` (`re-export-bulk-scan.ts`), which fingerprints the
-engine tree and every patch artifact around the dry run and fails hard on
-divergence.
+locks. `re-export --dry-run` does not: it runs inside `withPrivateGitIndex`,
+so its temporary `--intent-to-add` staging lands on a throwaway index, and it
+serializes only against other dry runs. Dry-run purity is enforced at runtime
+by `withDryRunPurityGuard` (`re-export-bulk-scan.ts`). For `re-export` it
+fingerprints what the dry run could touch (the engine HEAD and index, the
+selected patches' bodies, their `patches.json` rows and the engine files they
+own) and fails hard on divergence, naming a foreign writer when the history
+log or the engine-session lock shows one in the window (`re-export-purity-scope.ts`).
 
 Contention fails fast after about a second by default, with the fixed message
 "Another FireForge engine-mutating command is already running. Wait for it to
@@ -346,7 +371,10 @@ run and how large the budget is.
 
 The budget is not the engine lock's alone. It covers every lock the command
 takes, which for the three furnace commands means the per-root furnace lock as
-well (see the `furnace-operation.ts` section). `FIREFORGE_WAIT_LOCK=<seconds>`
+well (see the `furnace-operation.ts` section). The furnace leaves that take
+only the furnace lock (`create`, `remove`, `rename`, `refresh`, `scan`,
+`override`, `preview`, `chrome-doc create`/`remove`) honour the flag too, for
+that one lock. `FIREFORGE_WAIT_LOCK=<seconds>`
 sets the same budget for a whole session, parsed with the same bounds as the
 flag so the two cannot drift, and an explicit budget at a call site always
 wins. Commands that take no lock accept `--wait-lock` and ignore it, so a
@@ -364,11 +392,10 @@ cleanups and a `completed` flag), signal rollback with bounded timeouts,
 
 The lock's wait budget is `options.lockTimeoutMs`, threaded from `--wait-lock`
 by `waitLockMutationOptions`, which does the seconds-to-milliseconds
-conversion once. It falls back to `sessionWaitLockMs()`
-(`FIREFORGE_WAIT_LOCK`) and then to `DEFAULT_LOCK_TIMEOUT_MS`. That fallback
-is what covers the twelve furnace mutators that declare no flag (`create`,
-`remove`, `rename`, `refresh`, `scan`, `override`, `chrome-doc`, and the
-rest). A contended refusal here names the furnace lock, says it is the second
+conversion once. Every furnace leaf that takes the lock threads its own
+flag. It falls back to `sessionWaitLockMs()` (`FIREFORGE_WAIT_LOCK`) and then
+to `DEFAULT_LOCK_TIMEOUT_MS`; that fallback is the only budget for callers
+that thread none (the pre-build apply, `doctor --repair-furnace`). A contended refusal here names the furnace lock, says it is the second
 of two, and points at `--wait-lock` and `FIREFORGE_WAIT_LOCK`. A budget that
 is paid in full and then refuses anyway is the worst shape a budget can have,
 so the refusal must not read as the engine lock's.
@@ -404,7 +431,9 @@ hold it.
 
 The destructive-operation contract: `confirmDestructive` (summary, prompt,
 `--yes`, `--dry-run`, conflict refusal, `--force-unsafe`) and the
-`.fireforge-history.jsonl` audit log.
+append-only history log (`src/core/history-log-file.ts` resolves its path,
+migrates a legacy `patches/.fireforge-history.jsonl`, and stamps each
+record's invocation and session).
 
 ### `src/core/tree-store.ts`
 
@@ -444,9 +473,15 @@ On SIGINT or SIGTERM, `bin/fireforge.ts`:
      work: the group sweep and the caller's `postCloseSweep` (the harness
      helper reap in `fireforge test`). Both run before the child's closure
      settles, so a forwarded SIGTERM cannot race `process.exit` past them.
-3. Force-releases the furnace lock directories of active operations, since
+3. Restores any `test --negative-control` mutation still applied
+   (`restoreActiveNegativeControlsForSignal`), after child shutdown and
+   before any lock is released, so the next lock holder never builds or
+   tests the neuter. A restore the run itself already started is awaited.
+   A control a SIGKILL left behind is undone by the next `fireforge test`
+   from its on-disk journal (`src/core/negative-control.ts`).
+4. Force-releases the furnace lock directories of active operations, since
    `withFileLock`'s `finally` will never run past `process.exit`.
-4. Calls `process.exit(130 | 143)`.
+5. Calls `process.exit(130 | 143)`.
 
 A parent that dies without signalling FireForge (SIGKILL on the `npm` above
 it) enters no handler here. `fireforge test` covers that case itself with

package/docs/machine-output.md

@@ -53,27 +53,46 @@ if (options.json === true) {
 
 ## Current surfaces
 
-| Surface                            | Shape                                                                                 | Notes                                                                                                                                                                                                                              |
-| ---------------------------------- | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `status --json`                    | `{ schemaVersion: 1, … }`                                                             | Error envelope on three engine preconditions                                                                                                                                                                                       |
-| `tree list --json`                 | `{ schemaVersion: 1, trees }`                                                         | Error envelope added in 0.44.0                                                                                                                                                                                                     |
-| `token list --json`                | `{ schemaVersion: 1, tokensCssPath, categories }`                                     | Categories in file order, each with its tokens (`name`, `line`, `value`)                                                                                                                                                           |
-| `token show --json`                | `{ schemaVersion: 1, name, category, declarations }`                                  | One entry per declaring block, with its selector trail and line                                                                                                                                                                    |
-| `test` verdict line                | `FIREFORGE-VERDICT: PASS\|FAIL reason=… [note=<class>] [shuffle=<seed>] [log=<path>]` | Not JSON. A single final stdout line, sealed by `setStdoutSealed` so nothing can displace it. `log=` names the run's own complete output under `.fireforge/logs/`, so a piped or truncated run still leaves a re-readable artifact |
-| `lint --per-patch --report <path>` | `{ schemaVersion: 1, … }`                                                             | Written to a file, never read back by FireForge                                                                                                                                                                                    |
-| `status --raw`                     | `git status --porcelain` lines                                                        | Not JSON, but still a machine surface: it engages the same stdout discipline, so rules 3 and 4 apply and rules 1 and 2 do not. A clean tree prints nothing                                                                         |
-| `build` log announcement           | `Full build output: <path>`                                                           | Not JSON. `build` prints no verdict line, so it names its run log directly on the way out                                                                                                                                          |
+| Surface                            | Shape                                                                                                                                                                                                                                                                             | Notes                                                                                                                                                                                                                              |
+| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `status --json`                    | `{ schemaVersion: 1, … }`                                                                                                                                                                                                                                                         | Error envelope on three engine preconditions                                                                                                                                                                                       |
+| `status --json` under `--check`    | adds `check: { enabled, failOn, allow?, failOnUnknown?, failed, offenders }`                                                                                                                                                                                                      | Since 0.50.0 on the full payload too, not only `--summary`. `allow` and `failOnUnknown` appear only when set. An offender failing only through `--fail-on-unknown` carries `unnamed: true`. `--patches` adds `scope: { patches }`  |
+| `furnace validate --json`          | `{ schemaVersion: 1, passed, summary, maxWarnings?, components, issues, excepted, staleExceptions }`                                                                                                                                                                              | `issues` counts after `validationExceptions`, stale-exception warnings included. `excepted` carries each suppressed finding with its `reason`                                                                                      |
+| `check --json`                     | `{ schemaVersion: 1, verdict, reason?, summary, steps }`                                                                                                                                                                                                                          | One entry per step: `name`, `kind`, `command`, `status` (`pass`/`fail`/`skipped`), `exitCode`, `durationMs`, `detail`, and on failure `outputTail` (`timedOut` when killed at its budget). No verdict line under `--json`          |
+| `check` verdict line               | `FIREFORGE-VERDICT: PASS\|FAIL [reason=step-failures\|lock-timeout\|skipped-steps failed=<steps>] steps=<n> skipped=<n> [skipped-steps=<steps>]`                                                                                                                                  | Not JSON. The last stdout line, after the step table, sealed like `test`'s. Its `reason=` set is its own, not `test`'s                                                                                                             |
+| `tree list --json`                 | `{ schemaVersion: 1, trees }`                                                                                                                                                                                                                                                     | Error envelope added in 0.44.0                                                                                                                                                                                                     |
+| `token list --json`                | `{ schemaVersion: 1, tokensCssPath, categories }`                                                                                                                                                                                                                                 | Categories in file order, each with its tokens (`name`, `line`, `value`)                                                                                                                                                           |
+| `token show --json`                | `{ schemaVersion: 1, name, category, declarations }`                                                                                                                                                                                                                              | One entry per declaring block, with its selector trail and line                                                                                                                                                                    |
+| `token lint --json`                | `{ schemaVersion: 1, tokensCssPath, tokenPrefix, filesScanned, findings }`                                                                                                                                                                                                        | Each finding has `check`, `file`, `line`, `token`, `message`. The document is written even when findings exist; the exit is then non-zero, as with `status --check --json`                                                         |
+| `test` verdict line                | `FIREFORGE-VERDICT: PASS\|FAIL reason=… [note=<class>] [started=<n>/<m> first-unstarted=<path> started-source=<src>] [shards=<p>/<t> [failed-shards=<labels>]] [runs=<p>/<n>] [control=<outcome>] [shuffle=<seed>] [furnace-applied=<names>\|furnace-apply=skipped] [log=<path>]` | Not JSON. A single final stdout line, sealed by `setStdoutSealed` so nothing can displace it. `log=` names the run's own complete output under `.fireforge/logs/`, so a piped or truncated run still leaves a re-readable artifact |
+| `lint --per-patch --report <path>` | `{ schemaVersion: 1, patches, queueIssues, totals, … }`                                                                                                                                                                                                                           | Written to a file. `queueIssues` holds the cross-patch findings; `patch staged-dependency --add --from` reads back their `stagedDeclarations`                                                                                      |
+| `status --raw`                     | `git status --porcelain` lines                                                                                                                                                                                                                                                    | Not JSON, but still a machine surface: it engages the same stdout discipline, so rules 3 and 4 apply and rules 1 and 2 do not. A clean tree prints nothing                                                                         |
+| `build` log announcement           | `Full build output: <path>`                                                                                                                                                                                                                                                       | Not JSON. `build` prints no verdict line, so it names its run log directly on the way out                                                                                                                                          |
 
 `--raw` is why `cli.ts` reads `process.argv` for `--json` or `--raw` before
 dispatching. Rule 3 is about who owns stdout, and that question is settled
 the same way for a porcelain stream as for a JSON document.
 
-`doctor` and `verify` have no `--json` mode. `doctor` already computes a
+`doctor` and `verify` have no `--json` mode (`fireforge check` reports
+`verify` as a step with its exit code and output tail). `doctor` already computes a
 serializable `DoctorCheck[]`, so adding one is cheap if a consumer asks for
 it. Follow the rules above rather than inventing a shape.
 
 The run logs behind `log=` are specified in [`run-logs.md`](run-logs.md).
 
+Additive keys since 0.51.0 outside `test`:
+
+- `check --json`: a skipped step carries `skipKind`, `config` when
+  `check.skip` names it and `unavailable` when the project has nothing for
+  it to run. The verdict line adds `skipped-steps=<steps>` whenever a step
+  is skipped, and `reason=skipped-steps` when `check.failOnSkipped` fails a
+  run on an `unavailable` skip alone.
+- `furnace validate --json`: a finding about particular elements carries
+  `tags` (`no-keyboard-handler` names each synthetic element with an
+  `@click`). An exception that matched an error adds a
+  `validation-exception-on-error` error, and a full run under
+  `furnace.json#minComponents` a `component-floor` error on `furnace.json`.
+
 ## Versioning
 
 `schemaVersion` is `MACHINE_OUTPUT_SCHEMA_VERSION` in
@@ -111,7 +130,76 @@ tokenises `key=value` pairs is unaffected, and a gate that names no class
 emits no `note=` at all. The classes are not a closed set and may grow, so
 consumers should treat an unrecognised one as opaque. Current values:
 `browser-busy`, `stale-browser`, `stale-browser-kill-failed`, `marionette-port-busy`, `mochitest-port-busy`,
-`coverage-replaced`, `stale-build`, `stale-components`.
+`coverage-replaced`, `stale-build`, `stale-components`, `no-launchable-binary`, `negative-control` (the
+control diff was refused), `negative-control-pending` (an interrupted run's
+control could not be undone safely). `note=negative-control-restore` rides a
+`reason=inconclusive` line whose control could not be restored byte-exact.
+
+### Multi-run invocations: `FIREFORGE-RUN:`, `runs=` and `control=`
+
+A mixed-harness request, `--repeat <n>` and `--negative-control <diff>` run
+several mach invocations under one lock hold (see
+[`testing.md`](testing.md#several-runs-under-one-lock-hold)). Each sub-run
+writes one raw stdout line before the verdict:
+
+```
+FIREFORGE-RUN: PASS|FAIL [reason=…] [note=…] [checks=<n>] [unexpected=<n>] [shards=<p>/<t>] run=<i>/<n> [harness=xpcshell|mochitest]
+```
+
+Its keys are the ones that sub-run's verdict line would have carried alone.
+A `FIREFORGE-RUN:` line is never a verdict: the invocation still ends with
+exactly one `FIREFORGE-VERDICT:` line. With more than one planned run it
+carries `runs=<passed>/<planned>` and no count keys; a failed aggregate takes
+the most structural reason among its runs (`inconclusive`, `preflight`,
+`crash`, `harness-arguments`, `no-tests`, `test-failures`) and that run's
+`note=`. A run that never started (the invocation stopped at a refusal)
+counts toward `<planned>` but not `<passed>`.
+
+`control=caught|missed|inconclusive` is present only for a negative control.
+`caught` means every repetition went red with `reason=test-failures`, and
+the process exits 0 even though the status is `FAIL`; `missed` means a
+repetition stayed green (exit 5); `inconclusive` covers every other outcome.
+A one-run control keeps its run's full line, counts included, and appends
+the key.
+
+### The additive `started=` and `first-unstarted=` keys
+
+Written together on a `FAIL reason=test-failures` line when a run that mach
+exited 0 on was refused its pass because requested test files never started:
+`started=<n>/<m>` counts the requested test files with a start (or skip)
+marker, and `first-unstarted=<path>` names the first, in dispatch order,
+that has none. A run refused only for a non-zero `unexpected=` count or a
+mid-test shutdown carries neither. Absent on every other line. See
+[`testing.md`](testing.md#the-verdict-line).
+
+`started-source=raw|stream|capture` follows them and names the evidence the
+count came from: the mozlog structured log (a run with `--results` or
+`--shard-size`), the complete streamed output (every other run), or, only
+when nothing was streamed, the last 2 MB of the capture.
+
+### Shards: `FIREFORGE-SHARD:` and `failed-shards=`
+
+A sharded run prints one `FIREFORGE-SHARD:` line per finished shard, before
+the verdict line:
+
+```
+FIREFORGE-SHARD: PASS|FAIL [reason=…] [checks=<n>] [unexpected=<n>] shard=<i>/<n> path=<label> [failed-files=<path>,…]
+```
+
+`path=` is the shard's path argument (whitespace replaced by `_`), or a
+`--shard-size`/`--jobs` chunk's first file plus `+<k>`. `failed-files=`
+appears under `--shard-size` and names the chunk's files that failed. The
+aggregate verdict appends `failed-shards=<label>,…` after `shards=` when any
+shard did not pass; under `--shard-size` `shards=` counts files and
+`failed-shards=` names files. Both are absent on a green run.
+
+### Additive keys of parallel and per-file runs
+
+`jobs=<n>` marks a `--jobs` run, `lock=shared` a run that held the engine
+session lock shared, and `results=<path>` names the `--results` document. A
+run whose engine changed mid-run only in files the running browser cannot
+see keeps its verdict and appends `drift-tolerated=<n>`. A negative control
+on compiled source appends `control-build=compiled`.
 
 ### The additive `shuffle=` key
 
@@ -158,3 +246,23 @@ gains `fireforgeHost: { start, end, powerSourceChanged }`, where each host
 state includes `load`, `power` and an optional `topProcess` with CPU and
 command. Consumers should tolerate this additive object. Unknown power
 state is visible but never treated as a proven transition.
+
+On a perf run (`--perf-samples`), an `engine/` change during the run makes
+the verdict `reason=inconclusive` only when HEAD moved or a changed path is
+packaged by the running build: a source an objdir install manifest names, a
+directory one installs from, or a test file. Edits that no manifest installs
+(compiled sources, build files, docs) leave the sample standing with a
+warning naming them. Other runs are inconclusive only when a changed path is
+one the running browser can see: a file the objdir links to its source, one
+staged at dispatch, or a test file or staging input of the run; anything
+else leaves the verdict standing with `drift-tolerated=<n>`.
+
+### Additive Furnace keys
+
+A `fireforge test --build` or `--build-only` run whose pre-test Furnace apply
+deployed components appends `furnace-applied=<name>,<name>…`: the components
+whose `components/` source changed since their last deploy, which the
+runtime under test therefore includes. `--no-furnace-apply` skips that apply
+and appends `furnace-apply=skipped`. `--furnace-apply reached` appends
+`furnace-skipped=<names>` for components with undeployed edits that no
+requested test reaches. All are absent otherwise.