@hominis/fireforge 0.46.2 → 0.47.1

package/docs/machine-output.md

@@ -53,16 +53,16 @@ if (options.json === true) {
 
 ## Current surfaces
 
-| Surface                            | Shape                                                                | Notes                                                                                                                                                                                                                              |
-| ---------------------------------- | -------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `status --json`                    | `{ schemaVersion: 1, … }`                                            | Error envelope on three engine preconditions                                                                                                                                                                                       |
-| `tree list --json`                 | `{ schemaVersion: 1, trees }`                                        | Error envelope added in 0.44.0                                                                                                                                                                                                     |
-| `token list --json`                | `{ schemaVersion: 1, tokensCssPath, categories }`                    | Categories in file order, each with its tokens (`name`, `line`, `value`)                                                                                                                                                           |
-| `token show --json`                | `{ schemaVersion: 1, name, category, declarations }`                 | One entry per declaring block, with its selector trail and line                                                                                                                                                                    |
-| `test` verdict line                | `FIREFORGE-VERDICT: PASS\|FAIL reason=… [note=<class>] [log=<path>]` | Not JSON. A single final stdout line, sealed by `setStdoutSealed` so nothing can displace it. `log=` names the run's own complete output under `.fireforge/logs/`, so a piped or truncated run still leaves a re-readable artifact |
-| `lint --per-patch --report <path>` | `{ schemaVersion: 1, … }`                                            | Written to a file, never read back by FireForge                                                                                                                                                                                    |
-| `status --raw`                     | `git status --porcelain` lines                                       | Not JSON, but still a machine surface: it engages the same stdout discipline, so rules 3 and 4 apply and rules 1 and 2 do not. A clean tree prints nothing                                                                         |
-| `build` log announcement           | `Full build output: <path>`                                          | Not JSON. `build` prints no verdict line, so it names its run log directly on the way out                                                                                                                                          |
+| Surface                            | Shape                                                                                 | Notes                                                                                                                                                                                                                              |
+| ---------------------------------- | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `status --json`                    | `{ schemaVersion: 1, … }`                                                             | Error envelope on three engine preconditions                                                                                                                                                                                       |
+| `tree list --json`                 | `{ schemaVersion: 1, trees }`                                                         | Error envelope added in 0.44.0                                                                                                                                                                                                     |
+| `token list --json`                | `{ schemaVersion: 1, tokensCssPath, categories }`                                     | Categories in file order, each with its tokens (`name`, `line`, `value`)                                                                                                                                                           |
+| `token show --json`                | `{ schemaVersion: 1, name, category, declarations }`                                  | One entry per declaring block, with its selector trail and line                                                                                                                                                                    |
+| `test` verdict line                | `FIREFORGE-VERDICT: PASS\|FAIL reason=… [note=<class>] [shuffle=<seed>] [log=<path>]` | Not JSON. A single final stdout line, sealed by `setStdoutSealed` so nothing can displace it. `log=` names the run's own complete output under `.fireforge/logs/`, so a piped or truncated run still leaves a re-readable artifact |
+| `lint --per-patch --report <path>` | `{ schemaVersion: 1, … }`                                                             | Written to a file, never read back by FireForge                                                                                                                                                                                    |
+| `status --raw`                     | `git status --porcelain` lines                                                        | Not JSON, but still a machine surface: it engages the same stdout discipline, so rules 3 and 4 apply and rules 1 and 2 do not. A clean tree prints nothing                                                                         |
+| `build` log announcement           | `Full build output: <path>`                                                           | Not JSON. `build` prints no verdict line, so it names its run log directly on the way out                                                                                                                                          |
 
 `--raw` is why `cli.ts` reads `process.argv` for `--json` or `--raw` before
 dispatching. Rule 3 is about who owns stdout, and that question is settled
@@ -86,15 +86,15 @@ moved.
 `FIREFORGE-VERDICT: FAIL reason=<reason>` uses a closed set
 (`FireforgeVerdictReason`, `src/commands/test-verdict.ts`):
 
-| Reason          | Meaning                                                                                                                                                                                                      |
-| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| `crash`         | The harness or the browser died, so there is no trustworthy suite result.                                                                                                                                    |
-| `no-tests`      | The run dispatched but nothing ran.                                                                                                                                                                          |
-| `test-failures` | The suite ran and reported unexpected results.                                                                                                                                                               |
-| `preflight`     | The run was refused before the harness was reached.                                                                                                                                                          |
-| `inconclusive`  | A result exists, but `engine/` moved under it, so the result was discarded.                                                                                                                                  |
-| `lock-timeout`  | The run never started: the engine session lock stayed contended.                                                                                                                                             |
-| `killed`        | A signal terminated the run. Written from the signal handler so that a log tail always describes itself. A killed run must never be silent, or "killed", "still running" and "never started" all look alike. |
+| Reason          | Meaning                                                                                                                                                                                                                                                                                                                                                           |
+| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `crash`         | The harness or the browser died, so there is no trustworthy suite result.                                                                                                                                                                                                                                                                                         |
+| `no-tests`      | The run dispatched but nothing ran.                                                                                                                                                                                                                                                                                                                               |
+| `test-failures` | The suite ran and reported unexpected results.                                                                                                                                                                                                                                                                                                                    |
+| `preflight`     | The run was refused before the harness was reached.                                                                                                                                                                                                                                                                                                               |
+| `inconclusive`  | A result exists, but `engine/` moved under it, so the result was discarded.                                                                                                                                                                                                                                                                                       |
+| `lock-timeout`  | The run never started: the engine session lock stayed contended.                                                                                                                                                                                                                                                                                                  |
+| `killed`        | A signal terminated the run, or FireForge ended it because its parent process vanished. Written from the signal handler (or the parent-exit watchdog) so that a log tail always describes itself. A killed run must never be silent, or "killed", "still running" and "never started" all look alike. `signal=` names the cause: a signal name, or `parent-exit`. |
 
 ### The additive `note=` key
 
@@ -112,6 +112,31 @@ consumers should treat an unrecognised one as opaque. Current values:
 `stale-browser`, `stale-browser-kill-failed`, `marionette-port-busy`,
 `coverage-replaced`, `stale-build`, `stale-components`.
 
+### The additive `shuffle=` key
+
+A `fireforge test --shuffle [seed]` run forwards mach's `--shuffle` (an
+unseeded shuffle of file order in the harness) and exports the seed FireForge
+drew or was given as `FIREFORGE_SHUFFLE_SEED` in the harness environment. The
+seed is appended to the verdict line as `shuffle=<seed>` on every emission
+form, so a red found by shuffling carries its own reproduction
+(`fireforge test --shuffle=<seed> …`) on the one line a pipe keeps. Reordering
+tasks _inside_ a file is harness code that reads the exported seed; FireForge
+owns the seed, the plumbing and the record. Absent when the run did not
+shuffle.
+
+### The additive `orphans-reaped=` key
+
+How many harness processes FireForge terminated itself during the run: at
+preflight, survivors of an earlier run in this objdir under `--reap-orphans`
+or `test.reapOrphans: "reap"`; at teardown, helpers this run's harness
+launched (httpd, websocket server, ssltunnel, process bridge, the browser,
+`moz-http2`) that were still alive after mach exited. Summed across the census, every
+retry attempt and every shard. Absent when nothing was reaped, so a green on
+a quiet machine prints the line it always did, and a green after a reap is
+distinguishable from it. A run ended by SIGTERM or by the parent-exit
+watchdog writes its verdict before the teardown reap runs, so that line never
+carries the key; the reap is reported on stderr instead.
+
 A preflight refusal's own text is written to stdout before the verdict line,
 and into the run log, so the verdict stays the last stdout write while the
 reason survives both a redirect and a `tail`. See

package/docs/testing.md

@@ -87,21 +87,85 @@ a slow build for a silently wrong test.
   earlier run in this project's objdir. It runs at preflight, before this run
   spawns anything, so every hit is necessarily a survivor. A match has to be
   objdir-anchored, since `xpcshell` and `server.js` are far too generic to
-  report on their own. Survivors slow every later run without appearing in
+  report on their own, and structural: the process's executable is a helper
+  binary, or an interpreter whose script argument is a helper script. A
+  process that merely mentions such a path (an editor, a `grep`, the shell
+  FireForge runs under) is not a helper, and FireForge's own ancestors are
+  never candidates. Survivors slow every later run without appearing in
   its output. The usual symptom is a three-second suite taking minutes of
-  wall clock. The census is report-only by default, and `--reap-orphans`
-  terminates what it finds. It never refuses a run. A different shape,
+  wall clock. The census is report-only by default; `--reap-orphans`, or
+  `test.reapOrphans: "reap"` in `fireforge.json`, terminates what it finds
+  (SIGTERM, a 500 ms grace, SIGKILL). It never refuses a run. Every reap is
+  stamped on the verdict line as `orphans-reaped=<n>`. A different shape,
   reparented Python `multiprocessing` workers, is covered by the
   `Orphaned harness workers` doctor check.
+- Every dispatch owns its own kill path. The harness announces what it
+  launches (`runtests.py | Server pid: <n>`, the websocket server, the SSL
+  tunnel, the websocket/process bridge, and the browser as `Application
+pid`); FireForge tracks those pids and, after mach has exited for any
+  reason (a clean finish, a harness crash before a retry, a no-output
+  timeout, a forwarded SIGTERM, the parent-exit watchdog below), terminates
+  any of them still alive. A tracked pid is re-read from `ps` first and must
+  still be anchored to this objdir and still look like the thing that was
+  launched, so a recycled pid is never signalled. One helper is never
+  announced: mozserve starts `moz-http2` in a session of its own and logs no
+  pid, so it is in neither the group nor the announcements. For it, FireForge
+  snapshots the same-objdir helpers before the dispatch spawns and, after
+  mach exits, reaps any helper-shaped process under this checkout's absolute
+  objdir that is new since that snapshot and reparented to launchd. An
+  earlier run's survivor is in the snapshot and stays the census's call. This
+  reap runs inside the exec layer's close path, so the signal handler's
+  bounded child-shutdown wait covers it. The count joins `orphans-reaped=`.
+- FireForge watches its own parent while a dispatch runs. A supervisor that
+  SIGKILLs the process above `fireforge test` (an `npm run` step killed at
+  its bound) signals nothing below it; FireForge notices the reparenting
+  within about two seconds, writes `FIREFORGE-VERDICT: FAIL reason=killed
+signal=parent-exit`, reaps the harness process group and the tracked
+  helpers, and ends the run without classifying or retrying.
 - `--perf-samples <path>` publishes a perf-sample artifact path to the
   harness (exported as `<BINARYNAME>_PERF_SAMPLE_JSON`).
 
+## Process groups and supervisors
+
+`fireforge test` spawns mach as the leader of its **own** process group
+(`detached: true`), so that FireForge can signal the whole harness tree with
+one negative-pid kill and sweep the group after mach exits. Two consequences
+for anything that supervises `fireforge test`:
+
+- A supervisor that kills _its own_ process group (`kill -- -<npm pid>`) does
+  not reach mach: mach is not in that group. Killing the `npm`/`node` chain
+  above FireForge leaves the harness tree running.
+- SIGTERM to FireForge is the right signal. FireForge forwards it to the
+  mach group, escalates to SIGKILL after a grace period, sweeps the group,
+  reaps the tracked helpers, and writes `FAIL reason=killed signal=SIGTERM`.
+  (That line, like the watchdog's `signal=parent-exit` line, is written
+  before the teardown reap runs, so it cannot carry `orphans-reaped=`; the
+  reap reports to stderr.)
+- A supervisor that has to SIGKILL FireForge should pass
+  `--pgid-file <path>`. FireForge rewrites the file with the harness group id
+  on every mach spawn (each retry attempt and shard) and removes it when the
+  run ends under its own control. After a SIGKILL the file is still there:
+  `kill -TERM -- -$(cat <path>)`, then `-KILL`, takes mach and every helper
+  that shares its group. Two processes are outside it: the browser
+  (mozprocess puts it in a group of its own) and `moz-http2` (mozserve
+  starts it in its own session). A SIGTERM to the group gives `runtests.py`
+  the chance to run its own cleanup, which normally takes the browser with
+  it; nothing takes `moz-http2`, so after a SIGKILL of FireForge it stays
+  until the next run's census, which is why that run should carry the reap
+  posture. A browser that does survive is the Marionette preflight's and
+  `--kill-stale-marionette`'s to remove.
+
+The mochitest helpers (httpd, websocket server, ssltunnel, process bridge)
+are plain children of `runtests.py` and share mach's group. The browser and
+`moz-http2` do not, which is why the teardown reap above exists alongside the
+group sweep.
+
 ## The verdict line
 
 Every test run ends with one machine-readable line:
 
 ```
-FIREFORGE-VERDICT: PASS|FAIL reason=… [note=<class>] [log=<path>]
+FIREFORGE-VERDICT: PASS|FAIL reason=… [note=<class>] [shuffle=<seed>] [orphans-reaped=<n>] [log=<path>]
 ```
 
 Automation should branch on this line rather than on the raw process code,
@@ -117,6 +181,18 @@ text is written to stdout before the verdict line and into the run log, so a
 redirected run keeps the reason as well as the verdict. See
 [`run-logs.md`](run-logs.md).
 
+## Seeded shuffle
+
+`fireforge test --shuffle` runs a mochitest selection in a seeded random file
+order and prints the seed; `--shuffle=<seed>` replays it. FireForge forwards
+mach's own `--shuffle`, exports `FIREFORGE_SHUFFLE_SEED=<seed>` to the harness
+so in-file task reordering (harness `head.js` code, not FireForge) can key off
+the same number, and stamps `shuffle=<seed>` on the verdict line. It is
+mochitest-only: the xpcshell harness has no shuffle, and generic `mach test`
+dispatch would hand the flag to every suite. Write the flag as
+`--shuffle=<seed>` or after the paths, because a bare `--shuffle <path>` reads
+the path as the seed.
+
 Exit code 14 (`INCONCLUSIVE`) is not red: it means `engine/` moved while the
 harness ran and the result was thrown away. Exit 15 (`LOCK_TIMEOUT`) means
 the run never started. See [`exit-codes.md`](exit-codes.md).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hominis/fireforge",
-  "version": "0.46.2",
+  "version": "0.47.1",
   "description": "FireForge — a build tool for customizing Firefox",
   "type": "module",
   "main": "./dist/src/index.js",
@@ -66,8 +66,8 @@
   "devDependencies": {
     "@eslint/js": "^10.0.0",
     "@types/estree": "^1.0.8",
-    "@types/node": "^22.0.0",
-    "@vitest/coverage-v8": "^4.1.2",
+    "@types/node": "^26.4.1",
+    "@vitest/coverage-v8": "^5.0.0",
     "dpdm": "4.3.0",
     "eslint": "^10.0.0",
     "eslint-config-prettier": "^10.1.8",
@@ -75,14 +75,14 @@
     "eslint-plugin-simple-import-sort": "^14.0.0",
     "fast-check": "^4.6.0",
     "husky": "^9.1.7",
-    "knip": "6.32.3",
+    "knip": "6.35.1",
     "lint-staged": "^17.0.4",
     "prettier": "3.9.6",
     "tsx": "^4.7.0",
     "typescript": "~6.0.0",
     "typescript-eslint": "^8.0.0",
     "vite": "^8.0.7",
-    "vitest": "^4.0.18"
+    "vitest": "^5.0.0"
   },
   "engines": {
     "node": ">=22.22.2"