@hominis/fireforge 0.15.5 → 0.15.6

package/CHANGELOG.md

@@ -43,6 +43,9 @@
 - Same-basename collisions in `dist/` are now disambiguated by trailing-segment overlap: a branding override at `engine/browser/branding/<name>/content/aboutDialog.css` (which ships at `chrome/<area>/content/branding/aboutDialog.css`) no longer gets matched against the unrelated upstream `chrome/<area>/content/browser/aboutDialog.css`. The scorer rewards candidates whose path contains meaningful intermediate segments from the source (e.g. the `branding` segment) so re-rooted artifacts win over coincidentally-named ones. Generic segments like `content` / `chrome` / `bin` do not count toward the bonus to avoid breaking ties on noise.
 - Test sources (anything under `/test(s)/`, plus `browser_*.js` / `test_*.js` / `xpcshell.toml` / `browser.ini`) are now resolved against the `_tests/` tree under the active `obj-*` directory instead of `dist/`. Mochitest and xpcshell harnesses copy registered tests under `_tests/testing/...`, never into the packaged bundle, so the previous dist-only walk false-flagged every registered test as "missing packaged artifact". Misses still warn — but they now point at `_tests/`, directing the operator to `BROWSER_CHROME_MANIFESTS` / `XPCSHELL_TESTS_MANIFESTS` instead of `package-manifest.in`.
 - Files inside an `if CONFIG[…]:` block in their owning `moz.build` are now skipped on hosts where the gate evaluates off (Windows-only stubinstaller CSS on a macOS build, Darwin-only artwork on Linux, etc.). The detection walks up from the source file to the closest `moz.build`, scans for the basename inside a Python-style indented `if CONFIG[…]:` block, and matches the gate against the host platform via `getPlatform()`. Negation expressions (`!= "WINNT"`, `not CONFIG[…]`) are conservatively NOT treated as single-OS gates, so we never wrongly suppress a warning for a file that should ship on the current host. Lives in the new `src/core/build-audit-platform.ts`.
+- Platform-gate detection now also covers subtrees packaged through platform-specific `Makefile.in` recipes that live outside the `moz.build` graph. Paths under `/stubinstaller/` (Windows NSIS stub installer), `browser/installer/windows/`, `browser/installer/macosx/`, and `browser/installer/linux/` count as host-gated by path convention on hosts where the target platform does not match. Previously the audit warned on every touched branding stubinstaller CSS on every non-Windows build because no ancestor `moz.build` wrapped those files in an `if CONFIG[…]:` block — the packaging trigger is `browser/installer/windows/Makefile.in` / `nsis/stub.nsh`, which the scanner does not parse. An explicit moz.build gate still wins if one is present, so fork-specific overrides behave as before. Surfaced as two warnings per macOS build before the fix.
+- Test-path audits are now gated on the `_tests/all-tests.json` marker file that `mach package-tests` writes. Plain `mach build` populates a partial `_tests/` subtree and stops, so every correctly-registered mochitest / xpcshell source was false-flagged as "missing packaged artifact" on the common build-only path. The audit now checks for the marker and silently skips test-path sources when the marker is absent — operators who want green-checked test registrations run `cd engine && ./mach package-tests` or a scoped `fireforge test <name>` after the build. Surfaced as 24 warnings per build (one per registered mochitest / xpcshell source) before the fix.
+- Stale-artifact warnings now require the matched candidate to be structurally related to the source: either the immediate parent directory trail-matches, or a non-generic source segment (`branding`, the fork's own directory name) appears mid-candidate. Same-basename hits in unrelated subtrees — the motivating case: `engine/browser/modules/<name>/test/head.js` matching the upstream `_tests/xpcshell/dom/quota/test/xpcshell/common/head.js` — are now classified as `missing` with a warning that names the unrelated candidate, rather than a `stale` warning that reads as "your build dropped this file" when in fact the match is spurious. The confidence check only kicks in when staleness would otherwise fire, so `updated` classifications keep their current loose matching. Generic directory segments (`test`, `tests`, `unit`, `common`, `xpcshell`, `mochitest` plus the existing list) no longer contribute to the structural-match bonus so trailing-segment spoofs (a shared `test` / `xpcshell` segment on an unrelated file) are rejected.
 - Ends every build with a `Packaged: N updated, M stale, K missing, S skipped` summary so operators can distinguish a fast-because-incremental build from a fast-because-silently-skipped one without a post-build `find`.
 - `fireforge build` auto-runs `mach configure` before the mach build step when any `moz.build`, `moz.configure`, or `Makefile.in` changed since the last successful build. Prevents the stale-backend trap where an incremental build skips work against a recursive-make backend that no longer matches the source tree. Emits a `Backend config changed; running mach configure first...` banner so the extra step is visible, and continues the build even if configure exits non-zero.
 - Mach build failures with known-cryptic mozbuild errors now print actionable hints. First entry in the table: `mozbuild.preprocessor.Preprocessor.Error: no preprocessor directives found` prints `Use JS_PREFERENCE_FILES instead, or add at least one #filter / #expand directive to the file.` The hint table lives in `src/core/mach-error-hints.ts` and is extensible — new cryptic errors we diagnose get one-line hints added without touching the build wrapper.

package/README.md

@@ -398,12 +398,14 @@ The summary line splits counts — e.g. `Lint: 2 introduced error(s), 0 introduc
 
 `fireforge build` is a transactional step: after a successful mach build it audits the dist bundle against engine-relative paths touched since the last successful build, and warns per file that is packageable-by-convention (`.js`/`.mjs`/`.css`/`.ftl`/`.xhtml`/`app/profile/…`) but has no matching artifact or whose dist mtime is older than the source. Ends every build with a `Packaged: N updated, M stale, K missing, S skipped` summary. The audit is warn-only — it never fails a build that mach reported green.
 
-The audit applies four routing rules to suppress false positives that previously trained operators to ignore its warnings:
+The audit applies six routing rules to suppress false positives that previously trained operators to ignore its warnings:
 
 - **Build inputs are excluded.** `jar.mn`, `moz.build`, `moz.configure`, `Makefile.in`, and `mozbuild.in` are consumed by the build to produce chrome registrations / make targets but never themselves ship. They are skipped before the dist lookup, so editing them no longer fires a "missing packaged artifact" warning.
 - **Same-basename collisions in `dist/` are disambiguated by trailing-segment overlap.** A branding override at `engine/browser/branding/<name>/content/aboutDialog.css` ships at `chrome/<area>/content/branding/aboutDialog.css`. A naive basename match would tie that against the unrelated upstream `chrome/<area>/content/browser/aboutDialog.css`; the audit now scores candidates by trailing path-segment match plus a small bonus for non-generic source segments (`branding`, the branding directory name) appearing in the candidate path, so re-rooted artifacts win over coincidentally-named ones.
+- **Unrelated same-basename hits never surface as "stale".** When the best-scoring candidate shares only the basename with the source and no meaningful intermediate segment (common on sparsely-populated `_tests/` trees where an upstream helper like `head.js` is the only same-basename file left from a prior build), the audit classifies the file as `missing` rather than emitting a misleading stale-comparison warning against the unrelated candidate. The warning names the unrelated file so the operator can confirm the mismatch at a glance.
 - **Test sources are looked up under `_tests/`, not `dist/`.** Anything under `/test(s)/` directories, plus `browser_*.js` / `test_*.js` / `xpcshell.toml` / `browser.ini`, is resolved against the `_tests/` tree under the active `obj-*` directory. Mochitest and xpcshell harnesses copy registered tests there, never into the packaged bundle. Misses still warn — but they point at `_tests/`, directing the operator to `BROWSER_CHROME_MANIFESTS` / `XPCSHELL_TESTS_MANIFESTS` instead of `package-manifest.in`.
-- **Files inside an `if CONFIG[…]:` block in their owning `moz.build` are skipped on hosts where the gate is off.** Windows-only stubinstaller CSS on a macOS build, Darwin-only artwork on Linux, etc. The detection walks up to the closest `moz.build`, scans for the basename inside a Python-style indented `if CONFIG[…]:` block, and matches the gate against the host platform. Negation expressions are conservatively NOT treated as single-OS gates so a warning is never wrongly suppressed for a file that should ship on the current host.
+- **Test-path audits are gated on `_tests/all-tests.json`.** Plain `mach build` populates a partial `_tests/` subtree and stops — full test packaging only runs under `mach package-tests` / `mach test <target>` (or `fireforge test <name>`). The audit now checks for the `all-tests.json` marker written by the packaged-tests make target and silently skips test-path sources when the marker is absent, so every registered mochitest / xpcshell source no longer false-flags as "missing" on the common build-only path. Run `cd engine && ./mach package-tests` (or a scoped `fireforge test`) after a build to green-check test registrations.
+- **Files inside an `if CONFIG[…]:` block in their owning `moz.build` are skipped on hosts where the gate is off.** Windows-only stubinstaller CSS on a macOS build, Darwin-only artwork on Linux, etc. The detection walks up to the closest `moz.build`, scans for the basename inside a Python-style indented `if CONFIG[…]:` block, and matches the gate against the host platform. Negation expressions are conservatively NOT treated as single-OS gates so a warning is never wrongly suppressed for a file that should ship on the current host. Subtrees packaged through platform-specific `Makefile.in` recipes that live outside the `moz.build` graph — `/stubinstaller/` (NSIS), `browser/installer/windows/`, `browser/installer/macosx/`, `browser/installer/linux/` — are also gated by path convention so branding stubinstaller CSS no longer warns on every non-Windows build.
 
 The build also auto-runs `mach configure` before the mach build step when any `moz.build`, `moz.configure`, or `Makefile.in` changed since the last successful build. Prevents incremental builds from silently skipping work against a stale recursive-make backend. Emits a `Backend config changed; running mach configure first...` banner when it fires.
 

package/dist/src/core/build-audit-platform.d.ts

@@ -24,7 +24,9 @@ export interface PlatformGateResult {
 export declare function findEnclosingGate(content: string, basename: string): string | undefined;
 /**
  * Determines whether the given source file is gated off on the current
- * host by an enclosing `if CONFIG[...]:` block in its owning moz.build.
+ * host by an enclosing `if CONFIG[...]:` block in its owning moz.build,
+ * OR by a path-convention rule for installer-tree subdirectories that
+ * are packaged via Makefile.in recipes the audit does not parse.
  * Returns `gatedOff: false` and no expression when no gate is found —
  * the file is not platform-restricted, so the caller should audit it
  * normally.

package/dist/src/core/build-audit-platform.js

@@ -10,10 +10,23 @@
  * detection, the audit fires a "missing packaged artifact" warning for
  * every gated file on every off-platform build — pure noise.
  *
+ * Two gate sources are consulted, in order:
+ *   1. Python-style `if CONFIG[...]:` blocks in the owning `moz.build`.
+ *   2. Path-convention gates — certain directory fragments are packaged
+ *      by platform-specific Makefile.in / NSIS recipes that FireForge
+ *      does not parse, so a file living under `browser/installer/windows/`
+ *      or any `/stubinstaller/` subtree is Windows-only regardless of
+ *      what its nearest moz.build says. (The branding stubinstaller CSS
+ *      is the motivating case: referenced from
+ *      `browser/installer/windows/Makefile.in` / `nsis/stub.nsh` with no
+ *      `if CONFIG[…]:` in any moz.build ancestor.)
+ *
  * The detection is intentionally lightweight: we walk up from the
  * source file looking for the closest `moz.build`, scan it for an
  * occurrence of the source basename inside an `if CONFIG[...]:` block,
  * and check whether the gate expression matches the host platform.
+ * The path-convention pass kicks in only when no moz.build gate is
+ * found, so an explicit moz.build gate always wins.
  *
  * This is best-effort. False negatives (we miss a gate and warn anyway)
  * are tolerable — the audit is warn-only. False positives (we wrongly
@@ -128,9 +141,58 @@ export function findEnclosingGate(content, basename) {
     }
     return undefined;
 }
+/**
+ * Path-convention gates: directories whose files are packaged by
+ * platform-specific build recipes (NSIS stub installer, DMG creation,
+ * Linux installer scripts) that live outside the moz.build graph. A
+ * file under any of these fragments is platform-restricted regardless
+ * of what its nearest `moz.build` says.
+ *
+ * `stubinstaller/` is the Windows NSIS stub installer asset tree. It
+ * is referenced from `browser/installer/windows/Makefile.in` (via
+ * `FILES` / `_WIDGET_FILES` lists) and `nsis/stub.nsh`, never through
+ * an `if CONFIG[…]:` block an ancestor moz.build exposes. Without
+ * this path-level gate, the audit warns on every touched branding
+ * stubinstaller CSS on every non-Windows build.
+ */
+const PATH_GATES = [
+    { fragment: '/stubinstaller/', platform: 'win32', label: 'path convention: /stubinstaller/' },
+    {
+        fragment: '/browser/installer/windows/',
+        platform: 'win32',
+        label: 'path convention: browser/installer/windows/',
+    },
+    {
+        fragment: '/browser/installer/macosx/',
+        platform: 'darwin',
+        label: 'path convention: browser/installer/macosx/',
+    },
+    {
+        fragment: '/browser/installer/linux/',
+        platform: 'linux',
+        label: 'path convention: browser/installer/linux/',
+    },
+];
+/**
+ * Returns a path-convention gate for `sourcePath` when one applies.
+ * Leading slash added so `startsWith`-style prefix traps
+ * (`browser/installer/windows/…`) match whether or not the input
+ * starts with a separator.
+ */
+function findPathConventionGate(sourcePath) {
+    const normalised = `/${sourcePath}`.replace(/\/+/g, '/');
+    for (const entry of PATH_GATES) {
+        if (normalised.includes(entry.fragment)) {
+            return { platform: entry.platform, label: entry.label };
+        }
+    }
+    return undefined;
+}
 /**
  * Determines whether the given source file is gated off on the current
- * host by an enclosing `if CONFIG[...]:` block in its owning moz.build.
+ * host by an enclosing `if CONFIG[...]:` block in its owning moz.build,
+ * OR by a path-convention rule for installer-tree subdirectories that
+ * are packaged via Makefile.in recipes the audit does not parse.
  * Returns `gatedOff: false` and no expression when no gate is found —
  * the file is not platform-restricted, so the caller should audit it
  * normally.
@@ -140,31 +202,36 @@ export function findEnclosingGate(content, basename) {
  * @returns Detection result
  */
 export async function detectPlatformGate(engineDir, sourcePath) {
-    const sourceDir = dirname(join(engineDir, sourcePath));
-    const mozBuild = await findOwningMozBuild(engineDir, sourceDir);
-    if (!mozBuild)
-        return { gatedOff: false };
-    let content;
-    try {
-        content = await readText(mozBuild);
-    }
-    catch {
-        return { gatedOff: false };
-    }
-    const sourceBasename = sourcePath.split('/').pop() ?? '';
-    const expression = findEnclosingGate(content, sourceBasename);
-    if (!expression)
-        return { gatedOff: false };
     let host;
     try {
         host = getPlatform();
     }
     catch {
-        return { gatedOff: false };
+        host = undefined;
+    }
+    const sourceDir = dirname(join(engineDir, sourcePath));
+    const mozBuild = await findOwningMozBuild(engineDir, sourceDir);
+    if (mozBuild) {
+        let content;
+        try {
+            content = await readText(mozBuild);
+        }
+        catch {
+            content = '';
+        }
+        const sourceBasename = sourcePath.split('/').pop() ?? '';
+        const expression = findEnclosingGate(content, sourceBasename);
+        if (expression) {
+            if (host && isGateOffHost(expression, host)) {
+                return { gatedOff: true, gateExpression: expression };
+            }
+            return { gatedOff: false, gateExpression: expression };
+        }
     }
-    if (isGateOffHost(expression, host)) {
-        return { gatedOff: true, gateExpression: expression };
+    const pathGate = findPathConventionGate(sourcePath);
+    if (pathGate && host && host !== pathGate.platform) {
+        return { gatedOff: true, gateExpression: pathGate.label };
     }
-    return { gatedOff: false, gateExpression: expression };
+    return { gatedOff: false };
 }
 //# sourceMappingURL=build-audit-platform.js.map

package/dist/src/core/build-audit.js

@@ -39,7 +39,7 @@ import { toError } from '../utils/errors.js';
 import { pathExists } from '../utils/fs.js';
 import { info, verbose, warn } from '../utils/logger.js';
 import { detectPlatformGate } from './build-audit-platform.js';
-import { isTestPath, resolveBestArtifact } from './build-audit-resolve.js';
+import { countTrailingSegmentMatches, isTestPath, resolveBestArtifact, } from './build-audit-resolve.js';
 import { hasChanges, isMissingHeadError } from './git.js';
 import { git } from './git-base.js';
 import { getUntrackedFiles } from './git-status.js';
@@ -190,6 +190,27 @@ async function resolveTestsRoot(engineDir) {
     }
     return undefined;
 }
+/**
+ * Marker file the `package-tests` make target writes after copying the
+ * full test-source tree under `_tests/`. Its presence is the most reliable
+ * signal that test packaging has actually run for the current obj-dir —
+ * plain `mach build` populates a partial `_tests/` subtree and then stops,
+ * so registered tests are absent even when registration is correct.
+ */
+const PACKAGED_TESTS_MARKER = 'all-tests.json';
+/**
+ * Returns true when the full test-package step has actually run for the
+ * active obj-dir. Without this marker the `_tests/` walk produces false
+ * positives for every correctly-registered mochitest / xpcshell source
+ * on the common "built but tests not packaged" path.
+ *
+ * @param testsRoot Absolute path to the obj-*`/_tests/` tree, or undefined.
+ */
+async function hasPackagedTestsMarker(testsRoot) {
+    if (!testsRoot)
+        return false;
+    return pathExists(join(testsRoot, PACKAGED_TESTS_MARKER));
+}
 /**
  * Resolves the search roots an individual source path should be looked
  * up under. Test-shaped paths get `_tests/`; everything else gets `dist/`.
@@ -200,6 +221,68 @@ function searchRootsFor(source, distRoot, testsRoot) {
     }
     return [distRoot];
 }
+/**
+ * Minimum trailing-segment overlap required for a same-basename dist/
+ * candidate to count as "the packaged artifact" of a source. The
+ * basename always trail-matches (count 1), so a threshold of 2 requires
+ * the immediate parent directory to also agree. Candidates that only
+ * share the basename are classified as missing — warning the operator
+ * to check registration — rather than emitting a misleading stale
+ * comparison against an unrelated file of the same name.
+ *
+ * Cross-tree re-rooting cases (e.g. `branding/<name>/content/foo.css`
+ * landing at `chrome/<area>/content/branding/foo.css`) bypass this
+ * floor because `scoreCandidate` awards a non-generic-segment bonus
+ * that lifts the confidence regardless of trailing overlap; those are
+ * detected below in `isConfidentMatch`.
+ */
+const MIN_TRAILING_SEGMENT_OVERLAP = 2;
+/**
+ * Returns true when the chosen artifact is structurally related to the
+ * source path — either its immediate parent directory trail-matches, or
+ * a non-generic intermediate source segment appears in the candidate
+ * path (the branding-re-root signal already used by the scorer).
+ *
+ * Used to avoid emitting `stale` warnings that point at an unrelated
+ * same-basename file picked up by the basename walker — a class of
+ * warning that is worse than `missing` because it reads as "your build
+ * dropped this file" when in fact the match is spurious.
+ */
+function isConfidentMatch(source, candidate) {
+    if (countTrailingSegmentMatches(source, candidate) >= MIN_TRAILING_SEGMENT_OVERLAP) {
+        return true;
+    }
+    const sourceSegs = source.split('/').filter(Boolean);
+    const candSegs = candidate.split('/').filter(Boolean);
+    const generic = new Set([
+        'content',
+        'chrome',
+        'bin',
+        'browser',
+        'toolkit',
+        'modules',
+        'base',
+        'app',
+        'profile',
+        'shared',
+        'themes',
+        'test',
+        'tests',
+        'unit',
+        'common',
+        'xpcshell',
+        'mochitest',
+    ]);
+    // Skip the basename itself (which trail-matches by definition).
+    for (let i = 0; i < sourceSegs.length - 1; i += 1) {
+        const seg = sourceSegs[i];
+        if (!seg || seg.length <= 2 || generic.has(seg))
+            continue;
+        if (candSegs.includes(seg))
+            return true;
+    }
+    return false;
+}
 /**
  * Audits one engine source path and returns its entry. Pure orchestration
  * helper kept separate so `auditBuildArtifacts` stays under the per-function
@@ -211,7 +294,16 @@ async function auditSinglePath(source, ctx) {
     }
     const gate = await detectPlatformGate(ctx.engineDir, source);
     if (gate.gatedOff) {
-        verbose(`Audit: skipping engine/${source} — gated off by moz.build "${gate.gateExpression ?? '?'}".`);
+        verbose(`Audit: skipping engine/${source} — gated off by "${gate.gateExpression ?? '?'}".`);
+        return { source, artifact: undefined, status: 'skipped' };
+    }
+    // Tests only end up under `_tests/` after `mach package-tests` (or a
+    // test-run that invokes the target) has executed. A plain `mach build`
+    // populates a partial subtree and stops, so every correctly-registered
+    // mochitest / xpcshell source appears "missing" on that common path.
+    // Skip audit for test sources when no packaged-tests marker is present.
+    if (isTestPath(source) && !ctx.testsPackaged) {
+        verbose(`Audit: skipping engine/${source} — _tests/${PACKAGED_TESTS_MARKER} not present; full test packaging has not run for this build.`);
         return { source, artifact: undefined, status: 'skipped' };
     }
     const sourcePath = join(ctx.engineDir, source);
@@ -249,6 +341,20 @@ async function auditSinglePath(source, ctx) {
         };
     }
     if (artifactMtime + 1 < sourceMtime) {
+        // Before claiming "stale", verify the match is structurally related.
+        // A same-basename hit in an unrelated subtree (e.g. `head.js` in a
+        // completely different upstream test helper) should be reported as
+        // missing — the operator needs to check registration, not puzzle
+        // over why an unrelated file appears "older than the source".
+        if (!isConfidentMatch(source, artifact)) {
+            const where = isTestPath(source) ? '_tests/' : 'dist/';
+            return {
+                source,
+                artifact: undefined,
+                status: 'missing',
+                warning: `Audit: engine/${source} was touched but no related packaged artifact with basename "${basename(source)}" was found under ${where} (nearest same-basename file ${artifact} lives in an unrelated subtree). Missing moz.build / jar.mn / package-manifest.in registration?`,
+            };
+        }
         return {
             source,
             artifact,
@@ -283,11 +389,12 @@ export async function auditBuildArtifacts(projectRoot, engineDir, baseline) {
         return summary;
     }
     const testsRoot = await resolveTestsRoot(engineDir);
+    const testsPackaged = await hasPackagedTestsMarker(testsRoot);
     const changed = await collectChangedFiles(engineDir, baseline);
     if (changed.length === 0) {
         return summary;
     }
-    const ctx = { engineDir, distRoot, testsRoot };
+    const ctx = { engineDir, distRoot, testsRoot, testsPackaged };
     for (const source of changed) {
         const result = await auditSinglePath(source, ctx);
         summary[result.status] += 1;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hominis/fireforge",
-  "version": "0.15.5",
+  "version": "0.15.6",
   "description": "FireForge — a build tool for customizing Firefox",
   "type": "module",
   "main": "./dist/src/index.js",