@hominis/fireforge 0.47.0 → 0.47.1

package/CHANGELOG.md

@@ -26,8 +26,19 @@
 
 ### Test harness
 
+- `fireforge test` owns the kill path of every mach dispatch. The harness announces what it launches (`runtests.py | Server pid: <n>`, the websocket server, the SSL tunnel, the websocket/process bridge, the browser as `Application pid`); FireForge tracks those pids and, after mach has exited for any reason (clean finish, harness crash before a retry, no-output timeout, forwarded SIGTERM), terminates any still alive, after re-reading each from `ps` and requiring it to still be anchored to this objdir and still be the thing that was launched. One helper is never announced (mozserve starts `moz-http2` in its own session and logs no pid); for it FireForge snapshots the same-objdir helpers before the dispatch and reaps any helper-shaped process under this checkout's absolute objdir that is new since the snapshot and reparented, so an earlier run's survivor stays the census's call. The reap runs inside the exec layer's close path so the signal handler's child-shutdown wait covers it. Before this, a run killed at its bound could leave the mochitest httpd reparented to launchd at 100% CPU for hours (2026-09-13), and every later run on the objdir competed with it. The group sweep already existed but never reached the browser, which mozprocess puts in a group of its own.
+- New parent-exit watchdog. A supervisor that SIGKILLs the process above `fireforge test` signals nothing below it: FireForge's stdout pipe merely breaks, the EPIPE handler swallows that on purpose, and the detached mach tree runs on unsupervised. That was the actual shape of the incident. FireForge now polls its own parent while a dispatch runs; when the parent is gone it writes `FIREFORGE-VERDICT: FAIL reason=killed signal=parent-exit`, reaps the mach process group and the tracked helpers, and ends the run instead of classifying the killed harness as a crash and retrying it. Off on Windows.
+- `test.reapOrphans: "report" | "reap"` in `fireforge.json` sets the repo's posture for the preflight orphan census; `--reap-orphans` remains the per-invocation form. Default `report`, unchanged behaviour. The reaper counts only processes confirmed gone after its SIGTERM → grace → SIGKILL sequence, and warns about one that is still alive after SIGKILL.
+- The census now matches a helper structurally (the executable is a helper binary, or an interpreter whose script argument is a helper script) and never considers an ancestor of FireForge's own process. It used to match the helper names anywhere in a command line, which also caught an editor opened on `server.js`, a `grep` over the objdir, or the operator's own shell when its command text quoted the path; report-only that printed a misleading `kill`, and under the new reap posture it terminated the shell running FireForge (found while proving the posture on 2026-09-13).
+- Fixed `fireforge test --doctor` exiting 0 with no `Marionette preflight:` result and no verdict line. The preflight's waits (the spawn settle window, the 400 ms poll gap, the SIGTERM→SIGKILL escalation gap) were unref'd around a detached browser child, so whenever the child was the only other thing on the event loop Node exited mid-probe. Both the working tree and the pinned 0.47.0 reproduced it; the waits are now ref'd, all of them bounded.
+- New `fireforge test --pgid-file <path>` publishes the harness process-group id (mach is the group leader) on every spawn and removes the file when the run ends under FireForge's control. A supervisor that had to SIGKILL FireForge takes mach and every helper in its group afterwards with `kill -- -$(cat <path>)`. `docs/testing.md` now states what was true but unwritten: mach runs in its own process group, so a supervisor killing its own group never reaches it.
+- New additive verdict key `orphans-reaped=<n>`: the sum of what the preflight census and the teardown reaps terminated, across attempts and shards. Absent when nothing was reaped, so a green on a quiet machine prints the line it always did and a green after a reap is distinguishable from it. A line written by the signal handler or the parent-exit watchdog precedes the teardown reap and cannot carry it; that reap reports on stderr.
 - New `fireforge test --shuffle [seed]` runs a mochitest selection in a seeded random file order: it forwards mach's `--shuffle`, exports `FIREFORGE_SHUFFLE_SEED=<seed>` to the harness (a fresh seed when omitted), prints the seed, and stamps `shuffle=<seed>` on the `FIREFORGE-VERDICT` line as an additive key. Mach's own shuffle is an unseeded `Math.random` pass, so a red found by shuffling could not be replayed. Reordering tasks inside a file is harness code that reads the exported seed. Mochitest-only: an xpcshell-only or generic dispatch is refused.
 
+### Locks
+
+- A contended wait writes its due progress line before it declares the timeout, not after. The deadline check ran first, so on a slow host (a cold Windows CI runner) whose first poll iteration alone outlasted a short budget, the wait timed out having reported nothing about the holder, and an advance that probe would have seen could not extend the deadline it was about to miss. `smoke (windows-latest)` failed on exactly this in `file-lock.test.ts`.
+
 ## 0.46.0
 
 ### Packaging audit

package/dist/bin/fireforge.js

@@ -8,6 +8,7 @@
  * docs/lifecycle-invariants.md for who owns what.
  */
 import { installBrokenPipeHandler, main } from '../src/cli.js';
+import { removeActivePgidFile } from '../src/commands/test-harness-teardown.js';
 import { emitKilledVerdict } from '../src/commands/test-verdict.js';
 import { forceReleaseHeldLocksForSignal } from '../src/core/file-lock.js';
 import { forceReleaseFurnaceLocksForActiveOperations, isSignalRollbackInFlight, rollbackActiveOperationsForSignal, } from '../src/core/furnace-operation.js';
@@ -108,6 +109,11 @@ function installFurnaceSignalHandler(signal, exitCode) {
             // racing this handler and usually loses. Best-effort like the log
             // itself: a diagnostic must never keep the process from exiting.
             .then(() => closeActiveRunLog().catch(() => undefined))
+            // `--pgid-file` names a group this handler has just signalled; after a
+            // normal exit `testCommand`'s finally removes it, but that finally
+            // loses the race to the exit below. Only a kill FireForge never saw
+            // may leave the file behind.
+            .then(() => removeActivePgidFile().catch(() => undefined))
             .finally(() => {
             exitAfterStdioFlush(exitCode);
         });

package/dist/build-info.json

@@ -1,9 +1,9 @@
 {
   "schemaVersion": 1,
-  "version": "0.47.0",
-  "commit": "2bffcb5fee31d398a521874851af09a62134ae10",
-  "shortCommit": "2bffcb5fee31",
+  "version": "0.47.1",
+  "commit": "11b30621b46142df7a8035ae7b9e8a13936787bb",
+  "shortCommit": "11b30621b461",
   "dirty": false,
   "dirtyHash": null,
-  "buildTime": "2026-09-12T14:05:35.561Z"
+  "buildTime": "2026-09-13T19:20:10.389Z"
 }

package/dist/src/commands/test-harness-teardown.d.ts

@@ -0,0 +1,62 @@
+/**
+ * Owns the kill path of one mach test dispatch for `fireforge test`.
+ *
+ * Three things the run must do around every `runMachTestSuite` call, kept
+ * together because each one exists for the same incident (a supervisor
+ * killed the process above FireForge at its bound and the harness httpd it
+ * had launched spun for hours):
+ *
+ *  1. Track the helper pids the harness announces, and after mach has
+ *     exited terminate any that outlived it (`harness-helper-pids.ts`).
+ *  2. Publish the harness process group (`--pgid-file`), so a supervisor
+ *     that can only SIGKILL FireForge can still take the whole tree with
+ *     one `kill -- -<pgid>` afterwards.
+ *  3. Watch FireForge's own parent (`parent-exit-watchdog.ts`). When it
+ *     vanishes mid-run, write the killed verdict, reap the group, and let
+ *     the dispatch loop end the run instead of retrying.
+ *
+ * Every reap adds to the `orphans-reaped=` verdict attribute.
+ */
+import type { MachTestSuiteOptions } from '../core/mach.js';
+/** Inputs for {@link createHarnessTeardown}. */
+export interface HarnessTeardownInputs {
+    /** Absolute objdir of this project, when known: the anchor for every kill. */
+    objDir: string | undefined;
+    /** `--pgid-file`: where to publish the harness process group id. */
+    pgidFile: string | undefined;
+}
+/** One dispatch's teardown: the hooks to spread into `runMachTestSuite`, and its lifecycle. */
+export interface HarnessTeardown {
+    /** Hooks for {@link MachTestSuiteOptions.teardown}. */
+    hooks: NonNullable<MachTestSuiteOptions['teardown']>;
+    /**
+     * Records the same-objdir helpers alive right now, before the dispatch
+     * spawns anything, so the post-close reap can tell this dispatch's
+     * unannounced helpers from an earlier run's survivors. Await it before
+     * `runMachTestSuite`.
+     */
+    prepare(): Promise<void>;
+    /** Stops the parent watchdog. Call after the dispatch resolved or threw. */
+    dispose(): void;
+}
+/**
+ * Builds the teardown for one dispatch (one retry attempt of one shard).
+ * The pid tracker and the group id are per dispatch; the parent watchdog
+ * runs only while that dispatch is in flight.
+ */
+export declare function createHarnessTeardown(inputs: HarnessTeardownInputs): HarnessTeardown;
+/** Records the run's pgid file so {@link removeActivePgidFile} can find it from the signal handler. */
+export declare function setActivePgidFile(path: string | undefined): void;
+/**
+ * Removes the pgid file at the end of a run that ended under FireForge's
+ * control. A file left behind after a normal exit would name a group id
+ * the kernel may have reused; the file must only outlive FireForge when
+ * FireForge itself was killed without notice, which is what it is for.
+ */
+export declare function removePgidFile(path: string | undefined): Promise<void>;
+/**
+ * Signal-pipeline variant of {@link removePgidFile}: removes the run's pgid
+ * file, if one is registered. Called by the bin entry point after the child
+ * shutdown wait, so the group the file names has already been signalled.
+ */
+export declare function removeActivePgidFile(): Promise<void>;

package/dist/src/commands/test-harness-teardown.js

@@ -0,0 +1,128 @@
+// SPDX-License-Identifier: EUPL-1.2
+/**
+ * Owns the kill path of one mach test dispatch for `fireforge test`.
+ *
+ * Three things the run must do around every `runMachTestSuite` call, kept
+ * together because each one exists for the same incident (a supervisor
+ * killed the process above FireForge at its bound and the harness httpd it
+ * had launched spun for hours):
+ *
+ *  1. Track the helper pids the harness announces, and after mach has
+ *     exited terminate any that outlived it (`harness-helper-pids.ts`).
+ *  2. Publish the harness process group (`--pgid-file`), so a supervisor
+ *     that can only SIGKILL FireForge can still take the whole tree with
+ *     one `kill -- -<pgid>` afterwards.
+ *  3. Watch FireForge's own parent (`parent-exit-watchdog.ts`). When it
+ *     vanishes mid-run, write the killed verdict, reap the group, and let
+ *     the dispatch loop end the run instead of retrying.
+ *
+ * Every reap adds to the `orphans-reaped=` verdict attribute.
+ */
+import { unlink, writeFile } from 'node:fs/promises';
+import { createHelperPidTracker, reapTrackedHarnessPids, snapshotObjdirHelpers, } from '../core/harness-helper-pids.js';
+import { startParentExitWatchdog } from '../core/parent-exit-watchdog.js';
+import { toError } from '../utils/errors.js';
+import { verbose, warn } from '../utils/logger.js';
+import { sweepProcessGroup } from '../utils/process-group.js';
+import { addVerdictRunCount, emitKilledVerdict } from './test-verdict.js';
+/**
+ * Builds the teardown for one dispatch (one retry attempt of one shard).
+ * The pid tracker and the group id are per dispatch; the parent watchdog
+ * runs only while that dispatch is in flight.
+ */
+export function createHarnessTeardown(inputs) {
+    const tracker = createHelperPidTracker();
+    let pgid;
+    let baseline;
+    const watchdog = startParentExitWatchdog({
+        onParentExit: ({ originalPpid, currentPpid }) => {
+            // Verdict first, before anything that can stall: the parent that would
+            // have read it is gone, but the run log and any tee of stdout are not.
+            emitKilledVerdict('parent-exit');
+            warn(`Parent process ${originalPpid} exited while the harness was running (this process now ` +
+                `reports parent ${currentPpid}); ending the run and terminating the harness tree so it ` +
+                'cannot run on unsupervised.');
+            if (pgid !== undefined) {
+                void sweepProcessGroup(pgid, undefined, 'after the parent process exited').catch((error) => {
+                    warn(`Reaping harness group ${pgid} after parent exit failed: ${toError(error).message}`);
+                });
+            }
+        },
+    });
+    return {
+        hooks: {
+            onOutputChunk: (stream, chunk) => {
+                tracker.feed(stream, chunk);
+            },
+            onProcessGroup: (id) => {
+                pgid = id;
+                verbose(`Harness process group: ${id} (mach is the group leader).`);
+                if (inputs.pgidFile !== undefined)
+                    void publishPgid(inputs.pgidFile, id);
+            },
+            postCloseSweep: async () => {
+                const reaped = await reapTrackedHarnessPids(tracker.tracked(), inputs.objDir, baseline);
+                addVerdictRunCount('orphans-reaped', reaped);
+            },
+        },
+        prepare: async () => {
+            baseline = await snapshotObjdirHelpers(inputs.objDir);
+        },
+        dispose: () => {
+            watchdog.stop();
+        },
+    };
+}
+/**
+ * Writes `<pgid>\n` to the pgid file. Best-effort: a supervisor that asked
+ * for the file and cannot get it is warned, and the run proceeds, because
+ * the file is a courtesy to the supervisor and not a precondition of the
+ * tests.
+ */
+async function publishPgid(path, pgid) {
+    try {
+        await writeFile(path, `${pgid}\n`, 'utf8');
+    }
+    catch (error) {
+        warn(`Could not write --pgid-file ${path}: ${toError(error).message}`);
+    }
+}
+/**
+ * The pgid file of the run in flight, for the signal pipeline. `testCommand`'s
+ * own `finally` removes the file on every path it controls, but on SIGTERM
+ * that `finally` races the bin handler's `process.exit` and loses, and a
+ * SIGTERM is notice: the file must not outlive a run FireForge itself
+ * ended. Set when the run starts, cleared by {@link removePgidFile}.
+ */
+let activePgidFile;
+/** Records the run's pgid file so {@link removeActivePgidFile} can find it from the signal handler. */
+export function setActivePgidFile(path) {
+    activePgidFile = path;
+}
+/**
+ * Removes the pgid file at the end of a run that ended under FireForge's
+ * control. A file left behind after a normal exit would name a group id
+ * the kernel may have reused; the file must only outlive FireForge when
+ * FireForge itself was killed without notice, which is what it is for.
+ */
+export async function removePgidFile(path) {
+    if (path === undefined)
+        return;
+    if (activePgidFile === path)
+        activePgidFile = undefined;
+    try {
+        await unlink(path);
+    }
+    catch (error) {
+        // ENOENT is the usual case when no dispatch ever spawned.
+        verbose(`--pgid-file ${path} not removed: ${toError(error).message}`);
+    }
+}
+/**
+ * Signal-pipeline variant of {@link removePgidFile}: removes the run's pgid
+ * file, if one is registered. Called by the bin entry point after the child
+ * shutdown wait, so the group the file names has already been signalled.
+ */
+export async function removeActivePgidFile() {
+    await removePgidFile(activePgidFile);
+}

package/dist/src/commands/test-register.js

@@ -72,7 +72,8 @@ export function registerTest(program, { getProjectRoot, withErrorHandling }) {
         .option('--allow-stale-build', 'Allow tests to run even when packageable engine files changed since the last successful FireForge build')
         .option('--allow-stale-components', 'Run tests despite components.conf changes that only a full "fireforge build" compiles in (the packaged child process will resolve the OLD StaticComponents table)')
         .option('--kill-stale-marionette', "Terminate a recognized stale browser from this objdir, one holding the Marionette port, or this checkout's own leftover mochitest httpd before running tests. Never touches another checkout's live harness.")
-        .option('--reap-orphans', 'Terminate harness helper processes (httpd, pywebsocket, ssltunnel, moz-http2) from this objdir that survived an earlier run. The census itself runs on every test dispatch and is report-only; this flag opts into the kill.')
+        .option('--reap-orphans', 'Terminate harness helper processes (httpd, pywebsocket, ssltunnel, moz-http2) from this objdir that survived an earlier run. The census itself runs on every test dispatch and is report-only; this flag (or test.reapOrphans: "reap" in fireforge.json) opts into the kill. Reaps are stamped on the verdict line as orphans-reaped=<n>.')
+        .option('--pgid-file <path>', 'Write the harness process-group id (mach is the group leader) to this file on every spawn, so a supervisor that had to SIGKILL fireforge can still take the whole harness tree with "kill -- -$(cat <path>)". Removed when the run ends normally.')
         .option('--canary [path]', 'Run one short browser-chrome harness canary. Uses test.canaryPath from fireforge.json when no path is supplied.')
         .option('--doctor', 'Run a marionette handshake preflight before tests (exit 1 on FAIL). With no paths, runs the preflight only.')
         .option('--mach-arg <arg>', 'Forward this argument verbatim to `mach test` (repeatable). Escape valve for upstream xpcshell/mochitest flags FireForge does not model.', ...stringListOption())

package/dist/src/commands/test-run.d.ts

@@ -45,6 +45,8 @@ export interface TestRunContext {
      * the coding-agent markers that quiet it. Off by default.
      */
     fullOutput?: boolean;
+    /** `--pgid-file`: where each dispatch publishes its harness process group id. */
+    pgidFile?: string | undefined;
 }
 /** Outcome of one (possibly retried) harness invocation. */
 export interface TestRunOutcome {

package/dist/src/commands/test-run.js

@@ -10,8 +10,10 @@
  * cost startup time but make results reproducible. The combined invocation
  * stays available via `--no-shard`.
  */
+import { join } from 'node:path';
 import { probeDisplaySleepState } from '../core/display-state.js';
 import { runMachTestSuite } from '../core/mach.js';
+import { parentExited } from '../core/parent-exit-watchdog.js';
 import { getActiveRunLogPath } from '../core/run-log.js';
 import { changedPrefNoiseVerdictNote } from '../core/test-changed-prefs.js';
 import { buildHarnessCrashMessage, classifyHarnessRun, headedDisplayAsleepVerdictNote, headedNoOutputTimeoutHint, } from '../core/test-harness-crash.js';
@@ -21,6 +23,7 @@ import { TestFailureError } from '../errors/build.js';
 import { info, note, warn } from '../utils/logger.js';
 import { getPlatform } from '../utils/platform.js';
 import { maybeInjectAppdirArg } from './test-appdir.js';
+import { createHarnessTeardown } from './test-harness-teardown.js';
 import { emitHarnessVerdict } from './test-verdict.js';
 /** Default bounded retry budget for recognized harness crashes. */
 export const DEFAULT_HARNESS_RETRIES = 2;
@@ -61,13 +64,14 @@ async function runTestsWithRetriesInner(ctx, paths, env) {
     let verdict;
     for (;;) {
         attempts += 1;
-        result = await runMachTestSuite(kind, {
-            engineDir: ctx.engineDir,
-            testPaths: paths,
-            args: extraArgs,
-            env,
-            fullOutput: ctx.fullOutput,
-        });
+        result = await runTrackedMachTestSuite(ctx, kind, paths, extraArgs, env);
+        // The parent that was reading this run is gone and the watchdog has
+        // already killed the harness. Classifying the corpse would call it a
+        // harness crash and retry it under nobody's supervision.
+        if (parentExited()) {
+            throw new TestFailureError('The parent process exited while the harness was running; the harness tree was ' +
+                'terminated and the run ended without a result.', `mach ${kind}`, undefined, getActiveRunLogPath());
+        }
         result = await retryAfterXpcshellSymlinkRepair({
             engineDir: ctx.engineDir,
             objDir: ctx.objDir,
@@ -107,6 +111,31 @@ async function runTestsWithRetriesInner(ctx, paths, env) {
     }
     return { result, verdict, attempts, appdirInjectionAttempted, displayState };
 }
+/**
+ * One mach dispatch with its kill path attached: helper-pid tracking and
+ * post-close reaping, the process-group announcement, and the parent-exit
+ * watchdog for exactly as long as mach runs.
+ */
+async function runTrackedMachTestSuite(ctx, kind, paths, extraArgs, env) {
+    const teardown = createHarnessTeardown({
+        objDir: ctx.objDir === undefined ? undefined : join(ctx.engineDir, ctx.objDir),
+        pgidFile: ctx.pgidFile,
+    });
+    try {
+        await teardown.prepare();
+        return await runMachTestSuite(kind, {
+            engineDir: ctx.engineDir,
+            testPaths: paths,
+            args: extraArgs,
+            env,
+            fullOutput: ctx.fullOutput,
+            teardown: teardown.hooks,
+        });
+    }
+    finally {
+        teardown.dispose();
+    }
+}
 /**
  * Probes the display's power state for a headed no-output stall, and only for
  * that shape. Every other outcome (a passing run, a real test failure, a

package/dist/src/commands/test-verdict.d.ts

@@ -12,8 +12,16 @@ import type { HarnessRunVerdict } from '../core/test-harness-crash.js';
  * of the others.
  */
 export type FireforgeVerdictReason = 'crash' | 'no-tests' | 'test-failures' | 'preflight' | 'inconclusive' | 'lock-timeout' | 'killed';
+/** Additive keys a run can stamp on its verdict line. */
+export type VerdictRunAttributeKey = 'shuffle' | 'orphans-reaped';
 /** Records an additive attribute for this run's verdict line. */
-export declare function setVerdictRunAttribute(key: 'shuffle', value: string): void;
+export declare function setVerdictRunAttribute(key: VerdictRunAttributeKey, value: string): void;
+/**
+ * Adds to a counting attribute. The key appears only once the total is
+ * positive, so a run that reaped nothing prints the line it always did.
+ * Sums across the preflight census, every retry attempt and every shard.
+ */
+export declare function addVerdictRunCount(key: 'orphans-reaped', count: number): void;
 /**
  * Re-arms the verdict sink for a new run. Called at `testCommand` entry so
  * repeated programmatic invocations (and tests) each get their own
@@ -49,7 +57,14 @@ export declare function emitPassVerdict(): void;
  */
 export declare function emitFailVerdict(reason: FireforgeVerdictReason, note?: string): void;
 /**
- * Emits the terminal verdict for a run killed by a signal.
+ * What ended a killed run: a signal FireForge received, or `parent-exit`,
+ * the parent-death watchdog's finding that the process above FireForge
+ * vanished (typically SIGKILLed by a supervisor) while a harness ran.
+ */
+export type KilledVerdictCause = NodeJS.Signals | 'parent-exit';
+/**
+ * Emits the terminal verdict for a run killed by a signal, or ended by
+ * FireForge itself because its parent died.
  *
  * Without it a killed run wrote no terminal line, so a log tail could not
  * distinguish "killed" from "still running" from "never started". Recovery
@@ -59,7 +74,7 @@ export declare function emitFailVerdict(reason: FireforgeVerdictReason, note?: s
  * line asserts a test run happened, and an interrupted `status` did not
  * have one.
  *
- * @param signal - The signal that terminated the run
+ * @param signal - The signal that terminated the run, or `parent-exit`
  * @returns True when a line was written
  */
-export declare function emitKilledVerdict(signal: NodeJS.Signals): boolean;
+export declare function emitKilledVerdict(signal: KilledVerdictCause): boolean;

package/dist/src/commands/test-verdict.js

@@ -7,15 +7,32 @@ let emitted = false;
  * Additive `key=value` attributes of the run in flight, appended to
  * whichever verdict line ends it. `shuffle` is the seed a `--shuffle` run
  * exported to the harness, so a red found by shuffling carries its own
- * reproduction on the one line an unattended reader keeps. Cleared by
+ * reproduction on the one line an unattended reader keeps. `orphans-reaped`
+ * is how many harness processes FireForge had to terminate itself, at
+ * preflight (survivors of an earlier run) or at teardown (helpers this run
+ * launched that outlived mach): a green after a reap is not the same
+ * finding as a green on a quiet machine. Cleared by
  * {@link resetVerdictEmission}, so a later run in the same process cannot
- * inherit it.
+ * inherit them.
  */
 const runAttributes = new Map();
 /** Records an additive attribute for this run's verdict line. */
 export function setVerdictRunAttribute(key, value) {
     runAttributes.set(key, value);
 }
+/**
+ * Adds to a counting attribute. The key appears only once the total is
+ * positive, so a run that reaped nothing prints the line it always did.
+ * Sums across the preflight census, every retry attempt and every shard.
+ */
+export function addVerdictRunCount(key, count) {
+    // Written as a negated `> 0` so a NaN (a stubbed census, a bad parse) is
+    // dropped rather than stamped on the line as `orphans-reaped=NaN`.
+    if (!(count > 0))
+        return;
+    const current = Number(runAttributes.get(key) ?? '0');
+    runAttributes.set(key, String(current + count));
+}
 /**
  * True once a test run has begun in this process.
  *
@@ -96,7 +113,8 @@ export function emitFailVerdict(reason, note) {
     writeVerdictLine(`FIREFORGE-VERDICT: FAIL reason=${reason}${suffix}`);
 }
 /**
- * Emits the terminal verdict for a run killed by a signal.
+ * Emits the terminal verdict for a run killed by a signal, or ended by
+ * FireForge itself because its parent died.
  *
  * Without it a killed run wrote no terminal line, so a log tail could not
  * distinguish "killed" from "still running" from "never started". Recovery
@@ -106,7 +124,7 @@ export function emitFailVerdict(reason, note) {
  * line asserts a test run happened, and an interrupted `status` did not
  * have one.
  *
- * @param signal - The signal that terminated the run
+ * @param signal - The signal that terminated the run, or `parent-exit`
  * @returns True when a line was written
  */
 export function emitKilledVerdict(signal) {

package/dist/src/commands/test.js

@@ -22,9 +22,10 @@ import { stripEnginePrefix } from '../utils/paths.js';
 import { runTestBuildPhase } from './test-build-phase.js';
 import { diagnoseShardOutcome, finalizeSingleRunOutcome } from './test-diagnose.js';
 import { buildPerfSampleEnv, mergeHarnessEnv, resolveShuffleSeed } from './test-harness-env.js';
+import { removePgidFile, setActivePgidFile } from './test-harness-teardown.js';
 import { assertPathlessTestMode, assertTestModeCombinations, canaryTimeoutSeconds, classifyBeforeDispatch, reportCanaryOutcome, resolveCanaryPath, } from './test-modes.js';
 import { DEFAULT_HARNESS_RETRIES, finalizeShardedOutcome, runShardedTests, runTestsWithRetries, } from './test-run.js';
-import { emitFailVerdict, emitPassVerdict, resetVerdictEmission, verdictEmitted, } from './test-verdict.js';
+import { addVerdictRunCount, emitFailVerdict, emitPassVerdict, resetVerdictEmission, verdictEmitted, } from './test-verdict.js';
 async function assertTestPathsExist(engineDir, testPaths) {
     const missingPaths = [];
     for (const testPath of testPaths) {
@@ -227,7 +228,7 @@ async function ensureTestMarionettePortAvailable(port, binaryName, options, skip
     }
     await assertMarionettePortAvailable(port, { binaryName });
 }
-async function ensureTestBrowserEnvironment(engineDir, launchablePath, xpcshellOnly, binaryName, options, objDir) {
+async function ensureTestBrowserEnvironment(engineDir, launchablePath, xpcshellOnly, projectConfig, options, objDir) {
     // A timed-out mochitest can leave the built app alive after its Marionette
     // listener has disappeared. The port probe cannot see that case, but the
     // survivor can still steal focus and wedge every later headed run.
@@ -249,14 +250,18 @@ async function ensureTestBrowserEnvironment(engineDir, launchablePath, xpcshellO
     // Helpers that outlived an earlier run (httpd, pywebsocket, ssltunnel,
     // moz-http2) slow every later run without appearing in its output. Both
     // harnesses are affected, so this census is not gated on xpcshellOnly.
-    await reportOrphanedHarnessProcesses(objDir, {
-        reap: options.reapOrphans === true,
+    // Reaping is opt-in per invocation (--reap-orphans) or per repo
+    // (test.reapOrphans: "reap"); a reap is stamped on the verdict line so a
+    // green after one is not mistaken for a green on a quiet machine.
+    const census = await reportOrphanedHarnessProcesses(objDir, {
+        reap: options.reapOrphans === true || projectConfig.test?.reapOrphans === 'reap',
     });
+    addVerdictRunCount('orphans-reaped', census.reaped);
     const forwardedPort = options.machArg
         ? extractForwardedMarionettePort(options.machArg)
         : undefined;
     const effectivePort = options.marionettePort ?? forwardedPort;
-    await ensureTestMarionettePortAvailable(effectivePort, binaryName, options, {
+    await ensureTestMarionettePortAvailable(effectivePort, projectConfig.binaryName, options, {
         xpcshellOnly,
         doctor: options.doctor === true,
     });
@@ -291,6 +296,7 @@ export async function testCommand(projectRoot, testPaths, options = {}) {
     // exactly the runs whose only output a `tail` throws away. Closed in
     // `finally`, after the verdict line has already read the path.
     setActiveRunLog(await openRunLog(projectRoot, 'test'));
+    setActivePgidFile(options.pgidFile);
     try {
         await runTestCommandBody(projectRoot, testPaths, options);
     }
@@ -302,6 +308,9 @@ export async function testCommand(projectRoot, testPaths, options = {}) {
         throw error;
     }
     finally {
+        // A run that ends under FireForge's control owns no harness group any
+        // more; the file only outlives FireForge when FireForge was killed.
+        await removePgidFile(options.pgidFile);
         await closeActiveRunLog();
     }
 }
@@ -405,7 +414,7 @@ async function runTestCommandBody(projectRoot, testPaths, options = {}) {
     // documented `--mach-arg --marionette-port=NNNN` route still hits the
     // wrapper preflight refusing on 2828 before the forwarded arg reaches
     // mach.
-    const { forwardedPort, effectivePort } = await ensureTestBrowserEnvironment(paths.engine, launchablePath, xpcshellOnly, projectConfig.binaryName, options, buildCheck.objDir);
+    const { forwardedPort, effectivePort } = await ensureTestBrowserEnvironment(paths.engine, launchablePath, xpcshellOnly, projectConfig, options, buildCheck.objDir);
     if (options.doctor) {
         const doctorOutcome = await runDoctorPreflight({
             engineDir: paths.engine,
@@ -434,17 +443,15 @@ async function runTestCommandBody(projectRoot, testPaths, options = {}) {
     if (options.auto === true) {
         extraArgs.push('--auto');
     }
-    if (canaryPath !== undefined) {
+    if (canaryPath !== undefined)
         extraArgs.push(`--timeout=${canaryTimeoutSeconds(projectConfig)}`);
-    }
     // --mach-arg is a verbatim passthrough for upstream mach/xpcshell/mochitest
     // flags FireForge does not model directly (see the xpcshell appdir hint
     // above for why). Appended after --headless so mach sees
     // the FireForge-managed flags first and the escape-valve ones last, which
     // keeps the override precedence predictable.
-    if (forwardedMachArgs.length > 0) {
+    if (forwardedMachArgs.length > 0)
         extraArgs.push(...forwardedMachArgs);
-    }
     appendMarionetteForwardingArgs(extraArgs, options, forwardedPort, xpcshellOnly);
     // Directory arguments mean exactly that directory: mozbuild's test
     // resolver matches paths by string prefix, so a bare directory arg
@@ -482,6 +489,7 @@ async function runTestCommandBody(projectRoot, testPaths, options = {}) {
         headless: options.headless === true,
         ...(options.fullOutput === true ? { fullOutput: true } : {}),
         ...(harnessEnv ? { env: harnessEnv } : {}),
+        pgidFile: options.pgidFile,
     };
     const postRebuildContext = options.build
         ? createPostRebuildFailureContext('fireforge test --build', normalizedPaths)

package/dist/src/core/config-paths.d.ts

@@ -7,7 +7,7 @@ export declare const FIREFORGE_DIR = ".fireforge";
 /** Supported top-level fireforge.json keys backed by the current schema. */
 export declare const SUPPORTED_CONFIG_ROOT_KEYS: readonly ["name", "vendor", "appId", "binaryName", "firefox", "build", "buildAudit", "test", "externalToolchains", "license", "wire", "patchLint", "patchPolicy", "typecheck", "markerComment"];
 /** Supported config paths that can be read or set without --force. */
-export declare const SUPPORTED_CONFIG_PATHS: readonly ["name", "vendor", "appId", "binaryName", "license", "firefox", "firefox.version", "firefox.product", "firefox.sha256", "firefox.candidate", "firefox.allowUnverifiedDownload", "build", "build.jobs", "test", "test.canaryPath", "test.canaryTimeoutSeconds", "externalToolchains", "wire", "wire.subscriptDir", "patchLint", "patchLint.checkJs", "patchLint.checkJsStrict", "patchLint.checkJsCompilerOptions", "patchLint.checkJsExtraShim", "patchLint.checkJsTestFiles", "patchLint.checkJsTestShim", "patchLint.rawColorAllowlist", "patchLint.jsdocClassMethods", "patchLint.testAssertionFloor", "patchLint.chromeScriptJsDoc", "patchPolicy", "typecheck", "typecheck.projects", "typecheck.extraShim", "markerComment"];
+export declare const SUPPORTED_CONFIG_PATHS: readonly ["name", "vendor", "appId", "binaryName", "license", "firefox", "firefox.version", "firefox.product", "firefox.sha256", "firefox.candidate", "firefox.allowUnverifiedDownload", "build", "build.jobs", "test", "test.canaryPath", "test.canaryTimeoutSeconds", "test.reapOrphans", "externalToolchains", "wire", "wire.subscriptDir", "patchLint", "patchLint.checkJs", "patchLint.checkJsStrict", "patchLint.checkJsCompilerOptions", "patchLint.checkJsExtraShim", "patchLint.checkJsTestFiles", "patchLint.checkJsTestShim", "patchLint.rawColorAllowlist", "patchLint.jsdocClassMethods", "patchLint.testAssertionFloor", "patchLint.chromeScriptJsDoc", "patchPolicy", "typecheck", "typecheck.projects", "typecheck.extraShim", "markerComment"];
 /**
  * Gets all project paths based on a root directory.
  * @param root - Root directory of the project

package/dist/src/core/config-paths.js

@@ -53,6 +53,7 @@ export const SUPPORTED_CONFIG_PATHS = [
     'test',
     'test.canaryPath',
     'test.canaryTimeoutSeconds',
+    'test.reapOrphans',
     'externalToolchains',
     'wire',
     'wire.subscriptDir',

package/dist/src/core/config-validate-test-toolchains.js

@@ -45,6 +45,13 @@ export function parseTestBlock(rec, config) {
         }
         out.canaryTimeoutSeconds = canaryTimeout;
     }
+    const reapOrphans = testRec.raw('reapOrphans');
+    if (reapOrphans !== undefined) {
+        if (reapOrphans !== 'report' && reapOrphans !== 'reap') {
+            throw new ConfigError('Config field "test.reapOrphans" must be "report" or "reap"');
+        }
+        out.reapOrphans = reapOrphans;
+    }
     config.test = out;
 }
 /** Parses optional project-specific external toolchain doctor probes. */

package/dist/src/core/file-lock.js

@@ -433,15 +433,13 @@ export async function withFileLock(lockPath, operation, options = {}) {
                         continue;
                     }
                 }
-                if (Date.now() >= deadline) {
-                    // The queue is read here, not carried over from the last progress
-                    // probe: the position the wait reached is the one number that says
-                    // whether a larger budget would have helped, and it must not depend
-                    // on a probe having happened to land inside the budget. We are still
-                    // registered at this point. `deregisterWaiter` runs in the `finally`
-                    // below, which is only entered once this refusal has been built.
-                    throw await buildLockTimeoutError(lockPath, options.onTimeoutMessage, error, await readLockQueue(lockPath, startedAt));
-                }
+                // The probe runs before the deadline check, so a due progress line is
+                // written before the refusal that follows it, never lost to it. On a
+                // slow host (a cold Windows CI runner) the first iteration alone, with
+                // its waiter registration and stale probe, can outlast a short budget;
+                // checked the other way round, the wait timed out having reported
+                // nothing, and an advance that probe would have seen could not extend
+                // the deadline it was about to miss.
                 if (probeWanted && Date.now() - lastProbeAt >= probeIntervalMs) {
                     lastProbeAt = Date.now();
                     // Report the budget that is currently in force, not the one
@@ -473,6 +471,15 @@ export async function withFileLock(lockPath, operation, options = {}) {
                         }
                     }
                 }
+                if (Date.now() >= deadline) {
+                    // The queue is read here, not carried over from the last progress
+                    // probe: the position the wait reached is the one number that says
+                    // whether a larger budget would have helped, and it must not depend
+                    // on a probe having happened to land inside the budget. We are still
+                    // registered at this point. `deregisterWaiter` runs in the `finally`
+                    // below, which is only entered once this refusal has been built.
+                    throw await buildLockTimeoutError(lockPath, options.onTimeoutMessage, error, await readLockQueue(lockPath, startedAt));
+                }
                 await sleep(currentPollMs);
                 if (options.pollMaxMs !== undefined) {
                     currentPollMs = Math.min(currentPollMs * 2, options.pollMaxMs);