@hominis/fireforge 0.15.4 → 0.15.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -2
- package/README.md +9 -0
- package/dist/src/core/build-audit-platform.d.ts +38 -0
- package/dist/src/core/build-audit-platform.js +237 -0
- package/dist/src/core/build-audit-resolve.d.ts +66 -0
- package/dist/src/core/build-audit-resolve.js +209 -0
- package/dist/src/core/build-audit.d.ts +4 -1
- package/dist/src/core/build-audit.js +246 -85
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -38,7 +38,14 @@
|
|
|
38
38
|
|
|
39
39
|
### Build audit
|
|
40
40
|
|
|
41
|
-
- `fireforge build` now runs a warn-only post-build dist-tree audit after a successful mach build. The audit diffs engine-relative paths touched since the last successful build (stored as `.fireforge/last-build.json`) against the dist bundle, and warns per file that is packageable-by-convention (`.js`/`.mjs`/`.css`/`.ftl`/`.xhtml`/`app/profile/…`) but has no matching artifact, or whose dist mtime is older than the source. Surfaces the class of bug where a new pref file or widget is edited but never registered in `moz.build` / `jar.mn` / `package-manifest.in
|
|
41
|
+
- `fireforge build` now runs a warn-only post-build dist-tree audit after a successful mach build. The audit diffs engine-relative paths touched since the last successful build (stored as `.fireforge/last-build.json`) against the dist bundle, and warns per file that is packageable-by-convention (`.js`/`.mjs`/`.css`/`.ftl`/`.xhtml`/`app/profile/…`) but has no matching artifact, or whose dist mtime is older than the source. Surfaces the class of bug where a new pref file or widget is edited but never registered in `moz.build` / `jar.mn` / `package-manifest.in`.
|
|
42
|
+
- Build-system inputs (`jar.mn`, `moz.build`, `moz.configure`, `Makefile.in`, `mozbuild.in`) are now excluded from the audit's "must appear in dist/" check. They are consumed by the build to produce chrome registrations / make targets but never themselves ship, so every edit produced a guaranteed false positive — and worse, when an unrelated upstream `moz.build` coincidentally existed elsewhere in the bundle (e.g. `<App>.app/Contents/moz.build`) the audit reported "source is newer than packaged artifact" against two completely unrelated files. The exclusion list lives next to `PACKAGEABLE_EXTENSIONS` in `src/core/build-audit.ts`.
|
|
43
|
+
- Same-basename collisions in `dist/` are now disambiguated by trailing-segment overlap: a branding override at `engine/browser/branding/<name>/content/aboutDialog.css` (which ships at `chrome/<area>/content/branding/aboutDialog.css`) no longer gets matched against the unrelated upstream `chrome/<area>/content/browser/aboutDialog.css`. The scorer rewards candidates whose path contains meaningful intermediate segments from the source (e.g. the `branding` segment) so re-rooted artifacts win over coincidentally-named ones. Generic segments like `content` / `chrome` / `bin` do not count toward the bonus to avoid breaking ties on noise.
|
|
44
|
+
- Test sources (anything under `/test(s)/`, plus `browser_*.js` / `test_*.js` / `xpcshell.toml` / `browser.ini`) are now resolved against the `_tests/` tree under the active `obj-*` directory instead of `dist/`. Mochitest and xpcshell harnesses copy registered tests under `_tests/testing/...`, never into the packaged bundle, so the previous dist-only walk false-flagged every registered test as "missing packaged artifact". Misses still warn — but they now point at `_tests/`, directing the operator to `BROWSER_CHROME_MANIFESTS` / `XPCSHELL_TESTS_MANIFESTS` instead of `package-manifest.in`.
|
|
45
|
+
- Files inside an `if CONFIG[…]:` block in their owning `moz.build` are now skipped on hosts where the gate evaluates off (Windows-only stubinstaller CSS on a macOS build, Darwin-only artwork on Linux, etc.). The detection walks up from the source file to the closest `moz.build`, scans for the basename inside a Python-style indented `if CONFIG[…]:` block, and matches the gate against the host platform via `getPlatform()`. Negation expressions (`!= "WINNT"`, `not CONFIG[…]`) are conservatively NOT treated as single-OS gates, so we never wrongly suppress a warning for a file that should ship on the current host. Lives in the new `src/core/build-audit-platform.ts`.
|
|
46
|
+
- Platform-gate detection now also covers subtrees packaged through platform-specific `Makefile.in` recipes that live outside the `moz.build` graph. Paths under `/stubinstaller/` (Windows NSIS stub installer), `browser/installer/windows/`, `browser/installer/macosx/`, and `browser/installer/linux/` count as host-gated by path convention on hosts where the target platform does not match. Previously the audit warned on every touched branding stubinstaller CSS on every non-Windows build because no ancestor `moz.build` wrapped those files in an `if CONFIG[…]:` block — the packaging trigger is `browser/installer/windows/Makefile.in` / `nsis/stub.nsh`, which the scanner does not parse. An explicit moz.build gate still wins if one is present, so fork-specific overrides behave as before. Surfaced as two warnings per macOS build before the fix.
|
|
47
|
+
- Test-path audits are now gated on the `_tests/all-tests.json` marker file that `mach package-tests` writes. Plain `mach build` populates a partial `_tests/` subtree and stops, so every correctly-registered mochitest / xpcshell source was false-flagged as "missing packaged artifact" on the common build-only path. The audit now checks for the marker and silently skips test-path sources when the marker is absent — operators who want green-checked test registrations run `cd engine && ./mach package-tests` or a scoped `fireforge test <name>` after the build. Surfaced as 24 warnings per build (one per registered mochitest / xpcshell source) before the fix.
|
|
48
|
+
- Stale-artifact warnings now require the matched candidate to be structurally related to the source: either the immediate parent directory trail-matches, or a non-generic source segment (`branding`, the fork's own directory name) appears mid-candidate. Same-basename hits in unrelated subtrees — the motivating case: `engine/browser/modules/<name>/test/head.js` matching the upstream `_tests/xpcshell/dom/quota/test/xpcshell/common/head.js` — are now classified as `missing` with a warning that names the unrelated candidate, rather than a `stale` warning that reads as "your build dropped this file" when in fact the match is spurious. The confidence check only kicks in when staleness would otherwise fire, so `updated` classifications keep their current loose matching. Generic directory segments (`test`, `tests`, `unit`, `common`, `xpcshell`, `mochitest` plus the existing list) no longer contribute to the structural-match bonus so trailing-segment spoofs (a shared `test` / `xpcshell` segment on an unrelated file) are rejected.
|
|
42
49
|
- Ends every build with a `Packaged: N updated, M stale, K missing, S skipped` summary so operators can distinguish a fast-because-incremental build from a fast-because-silently-skipped one without a post-build `find`.
|
|
43
50
|
- `fireforge build` auto-runs `mach configure` before the mach build step when any `moz.build`, `moz.configure`, or `Makefile.in` changed since the last successful build. Prevents the stale-backend trap where an incremental build skips work against a recursive-make backend that no longer matches the source tree. Emits a `Backend config changed; running mach configure first...` banner so the extra step is visible, and continues the build even if configure exits non-zero.
|
|
44
51
|
- Mach build failures with known-cryptic mozbuild errors now print actionable hints. First entry in the table: `mozbuild.preprocessor.Preprocessor.Error: no preprocessor directives found` prints `Use JS_PREFERENCE_FILES instead, or add at least one #filter / #expand directive to the file.` The hint table lives in `src/core/mach-error-hints.ts` and is extensible — new cryptic errors we diagnose get one-line hints added without touching the build wrapper.
|
|
@@ -70,7 +77,9 @@
|
|
|
70
77
|
- New `src/core/marionette-preflight.ts` owns the `--doctor` probe and its teardown semantics.
|
|
71
78
|
- Test mocks for `furnace-registration.js` now cover the new `addLocaleFtlJarMnEntry` / `removeLocaleFtlJarMnEntry` exports; `config.js` mocks in apply-batch tests now cover `loadConfig` because the apply path reads `markerComment` from fireforge.json.
|
|
72
79
|
- Repo-wide scrub of fork-example mentions (`hominis.xhtml`, `HOMINIS` marker-comment examples, fixture tag names) in favour of a generic `mybrowser` / `MYBROWSER` placeholder. FireForge reads as fork-agnostic in docs and fixtures; the npm identity (`@hominis/fireforge`) is unchanged. Closes a v0.15.0 slip-through (one `@hominis/fireforge` reference remained in `src/core/furnace-operation.ts` as a generic example alongside the npm-identity occurrences; the code example is now fork-neutral).
|
|
73
|
-
-
|
|
80
|
+
- Second pass on the same scrub: residual `hominis.xhtml` test fixtures in `wire.test.ts`, `browser-wire.test.ts`, and `doctor.test.ts` are now `mybrowser-shell.xhtml`; the `hominis.js` reference in the build-audit changelog motivating-case description is now generic. Tests retain a distinct override-target (`mybrowser.xhtml`) to preserve the configured-vs-overridden semantic of the wire `--target` precedence test.
|
|
81
|
+
- New modules landed under coverage-gate protection: `src/core/mach-error-hints.ts`, `src/core/build-audit.ts`, `src/core/build-audit-resolve.ts`, `src/core/build-audit-platform.ts`, `src/core/build-baseline.ts`, `src/core/patch-lint-diff-tag.ts`, `src/commands/furnace/chrome-doc.ts`, `src/commands/furnace/chrome-doc-templates.ts`, `src/commands/furnace/create-mochikit.ts`. Per-module thresholds added to `scripts/check-coverage-thresholds.mjs`.
|
|
82
|
+
- Path resolution and Python-style moz.build gate detection extracted from `build-audit.ts` into the new `build-audit-resolve.ts` (basename collisions, `_tests/` routing, trailing-segment scoring) and `build-audit-platform.ts` (`if CONFIG[…]:` gate parser keyed on host platform). Keeps the orchestrator under the per-file LOC budget after the false-positive fixes landed.
|
|
74
83
|
|
|
75
84
|
## 0.14.0
|
|
76
85
|
|
package/README.md
CHANGED
|
@@ -398,6 +398,15 @@ The summary line splits counts — e.g. `Lint: 2 introduced error(s), 0 introduc
|
|
|
398
398
|
|
|
399
399
|
`fireforge build` is a transactional step: after a successful mach build it audits the dist bundle against engine-relative paths touched since the last successful build, and warns per file that is packageable-by-convention (`.js`/`.mjs`/`.css`/`.ftl`/`.xhtml`/`app/profile/…`) but has no matching artifact or whose dist mtime is older than the source. Ends every build with a `Packaged: N updated, M stale, K missing, S skipped` summary. The audit is warn-only — it never fails a build that mach reported green.
|
|
400
400
|
|
|
401
|
+
The audit applies six routing rules to suppress false positives that previously trained operators to ignore its warnings:
|
|
402
|
+
|
|
403
|
+
- **Build inputs are excluded.** `jar.mn`, `moz.build`, `moz.configure`, `Makefile.in`, and `mozbuild.in` are consumed by the build to produce chrome registrations / make targets but never themselves ship. They are skipped before the dist lookup, so editing them no longer fires a "missing packaged artifact" warning.
|
|
404
|
+
- **Same-basename collisions in `dist/` are disambiguated by trailing-segment overlap.** A branding override at `engine/browser/branding/<name>/content/aboutDialog.css` ships at `chrome/<area>/content/branding/aboutDialog.css`. A naive basename match would tie that against the unrelated upstream `chrome/<area>/content/browser/aboutDialog.css`; the audit now scores candidates by trailing path-segment match plus a small bonus for non-generic source segments (`branding`, the branding directory name) appearing in the candidate path, so re-rooted artifacts win over coincidentally-named ones.
|
|
405
|
+
- **Unrelated same-basename hits never surface as "stale".** When the best-scoring candidate shares only the basename with the source and no meaningful intermediate segment (common on sparsely-populated `_tests/` trees where an upstream helper like `head.js` is the only same-basename file left from a prior build), the audit classifies the file as `missing` rather than emitting a misleading stale-comparison warning against the unrelated candidate. The warning names the unrelated file so the operator can confirm the mismatch at a glance.
|
|
406
|
+
- **Test sources are looked up under `_tests/`, not `dist/`.** Anything under `/test(s)/` directories, plus `browser_*.js` / `test_*.js` / `xpcshell.toml` / `browser.ini`, is resolved against the `_tests/` tree under the active `obj-*` directory. Mochitest and xpcshell harnesses copy registered tests there, never into the packaged bundle. Misses still warn — but they point at `_tests/`, directing the operator to `BROWSER_CHROME_MANIFESTS` / `XPCSHELL_TESTS_MANIFESTS` instead of `package-manifest.in`.
|
|
407
|
+
- **Test-path audits are gated on `_tests/all-tests.json`.** Plain `mach build` populates a partial `_tests/` subtree and stops — full test packaging only runs under `mach package-tests` / `mach test <target>` (or `fireforge test <name>`). The audit now checks for the `all-tests.json` marker written by the packaged-tests make target and silently skips test-path sources when the marker is absent, so every registered mochitest / xpcshell source no longer false-flags as "missing" on the common build-only path. Run `cd engine && ./mach package-tests` (or a scoped `fireforge test`) after a build to green-check test registrations.
|
|
408
|
+
- **Files inside an `if CONFIG[…]:` block in their owning `moz.build` are skipped on hosts where the gate is off.** Windows-only stubinstaller CSS on a macOS build, Darwin-only artwork on Linux, etc. The detection walks up to the closest `moz.build`, scans for the basename inside a Python-style indented `if CONFIG[…]:` block, and matches the gate against the host platform. Negation expressions are conservatively NOT treated as single-OS gates so a warning is never wrongly suppressed for a file that should ship on the current host. Subtrees packaged through platform-specific `Makefile.in` recipes that live outside the `moz.build` graph — `/stubinstaller/` (NSIS), `browser/installer/windows/`, `browser/installer/macosx/`, `browser/installer/linux/` — are also gated by path convention so branding stubinstaller CSS no longer warns on every non-Windows build.
|
|
409
|
+
|
|
401
410
|
The build also auto-runs `mach configure` before the mach build step when any `moz.build`, `moz.configure`, or `Makefile.in` changed since the last successful build. Prevents incremental builds from silently skipping work against a stale recursive-make backend. Emits a `Backend config changed; running mach configure first...` banner when it fires.
|
|
402
411
|
|
|
403
412
|
Mach build failures with known-cryptic mozbuild errors now print actionable hints. Example: a `JS_PREFERENCE_PP_FILES` entry with no `#filter` / `#expand` directives now prints `Hint: ...use JS_PREFERENCE_FILES instead, or add at least one #filter / #expand directive to the file.` alongside the raw mach traceback.
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
/** Outcome of a moz.build platform-gate lookup for a single source file. */
|
|
2
|
+
export interface PlatformGateResult {
|
|
3
|
+
/** True when the file is gated off on the current host. */
|
|
4
|
+
gatedOff: boolean;
|
|
5
|
+
/**
|
|
6
|
+
* The gate expression that excluded the file, if any. Surfaced in
|
|
7
|
+
* verbose output so an operator can confirm the audit's reasoning.
|
|
8
|
+
*/
|
|
9
|
+
gateExpression?: string;
|
|
10
|
+
}
|
|
11
|
+
/**
|
|
12
|
+
* Scans a moz.build file for a basename match inside a conditional
|
|
13
|
+
* block, returning the gate expression when one encloses the match.
|
|
14
|
+
*
|
|
15
|
+
* The scanner uses indentation to track block scope (Python-style):
|
|
16
|
+
* an `if CONFIG[…]:` line opens a scope at indent N+1, and dedenting
|
|
17
|
+
* back to indent N closes it. A basename match inside that scope
|
|
18
|
+
* inherits the gate expression.
|
|
19
|
+
*
|
|
20
|
+
* @param content moz.build file content
|
|
21
|
+
* @param basename Basename of the source file we are auditing
|
|
22
|
+
* @returns The enclosing gate expression, or undefined if none
|
|
23
|
+
*/
|
|
24
|
+
export declare function findEnclosingGate(content: string, basename: string): string | undefined;
|
|
25
|
+
/**
|
|
26
|
+
* Determines whether the given source file is gated off on the current
|
|
27
|
+
* host by an enclosing `if CONFIG[...]:` block in its owning moz.build,
|
|
28
|
+
* OR by a path-convention rule for installer-tree subdirectories that
|
|
29
|
+
* are packaged via Makefile.in recipes the audit does not parse.
|
|
30
|
+
* Returns `gatedOff: false` and no expression when no gate is found —
|
|
31
|
+
* the file is not platform-restricted, so the caller should audit it
|
|
32
|
+
* normally.
|
|
33
|
+
*
|
|
34
|
+
* @param engineDir Absolute path to the engine root
|
|
35
|
+
* @param sourcePath Engine-relative POSIX path of the source file
|
|
36
|
+
* @returns Detection result
|
|
37
|
+
*/
|
|
38
|
+
export declare function detectPlatformGate(engineDir: string, sourcePath: string): Promise<PlatformGateResult>;
|
|
@@ -0,0 +1,237 @@
|
|
|
1
|
+
// SPDX-License-Identifier: EUPL-1.2
|
|
2
|
+
/*
|
|
3
|
+
* Platform-gate detection for the post-build dist-tree audit.
|
|
4
|
+
*
|
|
5
|
+
* `moz.build` files commonly wrap entries in conditional blocks like
|
|
6
|
+
* `if CONFIG["MAKENSISU"]:` (Windows-only stubinstaller) or
|
|
7
|
+
* `if CONFIG["OS_TARGET"] == "Darwin":` (macOS-only artwork). On a host
|
|
8
|
+
* that does not match the gate, the wrapped files are never processed
|
|
9
|
+
* by the build, so they cannot appear under `dist/`. Without this
|
|
10
|
+
* detection, the audit fires a "missing packaged artifact" warning for
|
|
11
|
+
* every gated file on every off-platform build — pure noise.
|
|
12
|
+
*
|
|
13
|
+
* Two gate sources are consulted, in order:
|
|
14
|
+
* 1. Python-style `if CONFIG[...]:` blocks in the owning `moz.build`.
|
|
15
|
+
* 2. Path-convention gates — certain directory fragments are packaged
|
|
16
|
+
* by platform-specific Makefile.in / NSIS recipes that FireForge
|
|
17
|
+
* does not parse, so a file living under `browser/installer/windows/`
|
|
18
|
+
* or any `/stubinstaller/` subtree is Windows-only regardless of
|
|
19
|
+
* what its nearest moz.build says. (The branding stubinstaller CSS
|
|
20
|
+
* is the motivating case: referenced from
|
|
21
|
+
* `browser/installer/windows/Makefile.in` / `nsis/stub.nsh` with no
|
|
22
|
+
* `if CONFIG[…]:` in any moz.build ancestor.)
|
|
23
|
+
*
|
|
24
|
+
* The detection is intentionally lightweight: we walk up from the
|
|
25
|
+
* source file looking for the closest `moz.build`, scan it for an
|
|
26
|
+
* occurrence of the source basename inside an `if CONFIG[...]:` block,
|
|
27
|
+
* and check whether the gate expression matches the host platform.
|
|
28
|
+
* The path-convention pass kicks in only when no moz.build gate is
|
|
29
|
+
* found, so an explicit moz.build gate always wins.
|
|
30
|
+
*
|
|
31
|
+
* This is best-effort. False negatives (we miss a gate and warn anyway)
|
|
32
|
+
* are tolerable — the audit is warn-only. False positives (we wrongly
|
|
33
|
+
* skip a gated file that should ship on this host) are not, so the
|
|
34
|
+
* detection errs toward NOT skipping when uncertain.
|
|
35
|
+
*/
|
|
36
|
+
import { dirname, join } from 'node:path';
|
|
37
|
+
import { pathExists, readText } from '../utils/fs.js';
|
|
38
|
+
import { getPlatform } from '../utils/platform.js';
|
|
39
|
+
/**
|
|
40
|
+
* Tokens that uniquely identify a Windows-only `if CONFIG[...]:` block.
|
|
41
|
+
* `MAKENSISU` is the Windows stubinstaller compiler; `OS_TARGET ==
|
|
42
|
+
* "WINNT"` and `MOZ_WIDGET_TOOLKIT == "windows"` are the conventional
|
|
43
|
+
* platform discriminators.
|
|
44
|
+
*/
|
|
45
|
+
const WINDOWS_ONLY_GATE_TOKENS = ['MAKENSISU', '"WINNT"', "'WINNT'", '"windows"', "'windows'"];
|
|
46
|
+
/** Tokens that mark a macOS-only `if CONFIG[...]:` block. */
|
|
47
|
+
const DARWIN_ONLY_GATE_TOKENS = ['"Darwin"', "'Darwin'", '"cocoa"', "'cocoa'"];
|
|
48
|
+
/** Tokens that mark a Linux-only `if CONFIG[...]:` block. */
|
|
49
|
+
const LINUX_ONLY_GATE_TOKENS = ['"Linux"', "'Linux'", '"gtk"', "'gtk'"];
|
|
50
|
+
/**
|
|
51
|
+
* Returns true when the platform-gate expression includes one of the
|
|
52
|
+
* tokens characteristic of a single OS that is not the current host.
|
|
53
|
+
*/
|
|
54
|
+
function isGateOffHost(expression, host) {
|
|
55
|
+
const matchesWindows = WINDOWS_ONLY_GATE_TOKENS.some((t) => expression.includes(t));
|
|
56
|
+
const matchesDarwin = DARWIN_ONLY_GATE_TOKENS.some((t) => expression.includes(t));
|
|
57
|
+
const matchesLinux = LINUX_ONLY_GATE_TOKENS.some((t) => expression.includes(t));
|
|
58
|
+
// Negation gates (`!= "WINNT"`, `not CONFIG["MAKENSISU"]`) flip the
|
|
59
|
+
// semantics. Keep this conservative — if we cannot confidently parse
|
|
60
|
+
// a negation, return false so we don't wrongly suppress a warning.
|
|
61
|
+
const negated = /\bnot\b|!=/.test(expression);
|
|
62
|
+
if (negated)
|
|
63
|
+
return false;
|
|
64
|
+
if (matchesWindows && host !== 'win32')
|
|
65
|
+
return true;
|
|
66
|
+
if (matchesDarwin && host !== 'darwin')
|
|
67
|
+
return true;
|
|
68
|
+
if (matchesLinux && host !== 'linux')
|
|
69
|
+
return true;
|
|
70
|
+
return false;
|
|
71
|
+
}
|
|
72
|
+
/**
|
|
73
|
+
* Walks from a starting directory up to (but not above) the engine root,
|
|
74
|
+
* yielding the first `moz.build` encountered. Returns undefined when no
|
|
75
|
+
* ancestor has one — typically only for files that live above any
|
|
76
|
+
* moz.build entry point, which would not be packageable anyway.
|
|
77
|
+
*/
|
|
78
|
+
async function findOwningMozBuild(engineDir, sourceDir) {
|
|
79
|
+
let current = sourceDir;
|
|
80
|
+
const root = engineDir.replace(/\/+$/, '');
|
|
81
|
+
while (current.startsWith(root)) {
|
|
82
|
+
const candidate = join(current, 'moz.build');
|
|
83
|
+
if (await pathExists(candidate))
|
|
84
|
+
return candidate;
|
|
85
|
+
const parent = dirname(current);
|
|
86
|
+
if (parent === current)
|
|
87
|
+
break;
|
|
88
|
+
current = parent;
|
|
89
|
+
}
|
|
90
|
+
return undefined;
|
|
91
|
+
}
|
|
92
|
+
/**
|
|
93
|
+
* Matches a `basename` appearing at the tail of a quoted path literal
|
|
94
|
+
* on a single moz.build line. Catches both the bare entry
|
|
95
|
+
* `"installing_page.css"` and the path-prefixed entry
|
|
96
|
+
* `"stubinstaller/installing_page.css"`.
|
|
97
|
+
*/
|
|
98
|
+
function matchesQuotedBasename(line, basename) {
|
|
99
|
+
const escaped = basename.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
100
|
+
return new RegExp(`["'](?:[^"']*\\/)?${escaped}["']`).test(line);
|
|
101
|
+
}
|
|
102
|
+
/**
|
|
103
|
+
* Scans a moz.build file for a basename match inside a conditional
|
|
104
|
+
* block, returning the gate expression when one encloses the match.
|
|
105
|
+
*
|
|
106
|
+
* The scanner uses indentation to track block scope (Python-style):
|
|
107
|
+
* an `if CONFIG[…]:` line opens a scope at indent N+1, and dedenting
|
|
108
|
+
* back to indent N closes it. A basename match inside that scope
|
|
109
|
+
* inherits the gate expression.
|
|
110
|
+
*
|
|
111
|
+
* @param content moz.build file content
|
|
112
|
+
* @param basename Basename of the source file we are auditing
|
|
113
|
+
* @returns The enclosing gate expression, or undefined if none
|
|
114
|
+
*/
|
|
115
|
+
export function findEnclosingGate(content, basename) {
|
|
116
|
+
const lines = content.split('\n');
|
|
117
|
+
const stack = [];
|
|
118
|
+
for (const line of lines) {
|
|
119
|
+
if (line.trim() === '')
|
|
120
|
+
continue;
|
|
121
|
+
const indent = (/^(\s*)/.exec(line)?.[1] ?? '').length;
|
|
122
|
+
while (stack.length > 0) {
|
|
123
|
+
const top = stack[stack.length - 1];
|
|
124
|
+
if (top && indent <= top.indent) {
|
|
125
|
+
stack.pop();
|
|
126
|
+
continue;
|
|
127
|
+
}
|
|
128
|
+
break;
|
|
129
|
+
}
|
|
130
|
+
const ifMatch = /^\s*if\s+(.+?):\s*(?:#.*)?$/.exec(line);
|
|
131
|
+
if (ifMatch?.[1]) {
|
|
132
|
+
stack.push({ indent, expression: ifMatch[1] });
|
|
133
|
+
continue;
|
|
134
|
+
}
|
|
135
|
+
if (matchesQuotedBasename(line, basename)) {
|
|
136
|
+
const top = stack[stack.length - 1];
|
|
137
|
+
if (top)
|
|
138
|
+
return top.expression;
|
|
139
|
+
return undefined;
|
|
140
|
+
}
|
|
141
|
+
}
|
|
142
|
+
return undefined;
|
|
143
|
+
}
|
|
144
|
+
/**
|
|
145
|
+
* Path-convention gates: directories whose files are packaged by
|
|
146
|
+
* platform-specific build recipes (NSIS stub installer, DMG creation,
|
|
147
|
+
* Linux installer scripts) that live outside the moz.build graph. A
|
|
148
|
+
* file under any of these fragments is platform-restricted regardless
|
|
149
|
+
* of what its nearest `moz.build` says.
|
|
150
|
+
*
|
|
151
|
+
* `stubinstaller/` is the Windows NSIS stub installer asset tree. It
|
|
152
|
+
* is referenced from `browser/installer/windows/Makefile.in` (via
|
|
153
|
+
* `FILES` / `_WIDGET_FILES` lists) and `nsis/stub.nsh`, never through
|
|
154
|
+
* an `if CONFIG[…]:` block an ancestor moz.build exposes. Without
|
|
155
|
+
* this path-level gate, the audit warns on every touched branding
|
|
156
|
+
* stubinstaller CSS on every non-Windows build.
|
|
157
|
+
*/
|
|
158
|
+
const PATH_GATES = [
|
|
159
|
+
{ fragment: '/stubinstaller/', platform: 'win32', label: 'path convention: /stubinstaller/' },
|
|
160
|
+
{
|
|
161
|
+
fragment: '/browser/installer/windows/',
|
|
162
|
+
platform: 'win32',
|
|
163
|
+
label: 'path convention: browser/installer/windows/',
|
|
164
|
+
},
|
|
165
|
+
{
|
|
166
|
+
fragment: '/browser/installer/macosx/',
|
|
167
|
+
platform: 'darwin',
|
|
168
|
+
label: 'path convention: browser/installer/macosx/',
|
|
169
|
+
},
|
|
170
|
+
{
|
|
171
|
+
fragment: '/browser/installer/linux/',
|
|
172
|
+
platform: 'linux',
|
|
173
|
+
label: 'path convention: browser/installer/linux/',
|
|
174
|
+
},
|
|
175
|
+
];
|
|
176
|
+
/**
|
|
177
|
+
* Returns a path-convention gate for `sourcePath` when one applies.
|
|
178
|
+
* Leading slash added so `startsWith`-style prefix traps
|
|
179
|
+
* (`browser/installer/windows/…`) match whether or not the input
|
|
180
|
+
* starts with a separator.
|
|
181
|
+
*/
|
|
182
|
+
function findPathConventionGate(sourcePath) {
|
|
183
|
+
const normalised = `/${sourcePath}`.replace(/\/+/g, '/');
|
|
184
|
+
for (const entry of PATH_GATES) {
|
|
185
|
+
if (normalised.includes(entry.fragment)) {
|
|
186
|
+
return { platform: entry.platform, label: entry.label };
|
|
187
|
+
}
|
|
188
|
+
}
|
|
189
|
+
return undefined;
|
|
190
|
+
}
|
|
191
|
+
/**
|
|
192
|
+
* Determines whether the given source file is gated off on the current
|
|
193
|
+
* host by an enclosing `if CONFIG[...]:` block in its owning moz.build,
|
|
194
|
+
* OR by a path-convention rule for installer-tree subdirectories that
|
|
195
|
+
* are packaged via Makefile.in recipes the audit does not parse.
|
|
196
|
+
* Returns `gatedOff: false` and no expression when no gate is found —
|
|
197
|
+
* the file is not platform-restricted, so the caller should audit it
|
|
198
|
+
* normally.
|
|
199
|
+
*
|
|
200
|
+
* @param engineDir Absolute path to the engine root
|
|
201
|
+
* @param sourcePath Engine-relative POSIX path of the source file
|
|
202
|
+
* @returns Detection result
|
|
203
|
+
*/
|
|
204
|
+
export async function detectPlatformGate(engineDir, sourcePath) {
|
|
205
|
+
let host;
|
|
206
|
+
try {
|
|
207
|
+
host = getPlatform();
|
|
208
|
+
}
|
|
209
|
+
catch {
|
|
210
|
+
host = undefined;
|
|
211
|
+
}
|
|
212
|
+
const sourceDir = dirname(join(engineDir, sourcePath));
|
|
213
|
+
const mozBuild = await findOwningMozBuild(engineDir, sourceDir);
|
|
214
|
+
if (mozBuild) {
|
|
215
|
+
let content;
|
|
216
|
+
try {
|
|
217
|
+
content = await readText(mozBuild);
|
|
218
|
+
}
|
|
219
|
+
catch {
|
|
220
|
+
content = '';
|
|
221
|
+
}
|
|
222
|
+
const sourceBasename = sourcePath.split('/').pop() ?? '';
|
|
223
|
+
const expression = findEnclosingGate(content, sourceBasename);
|
|
224
|
+
if (expression) {
|
|
225
|
+
if (host && isGateOffHost(expression, host)) {
|
|
226
|
+
return { gatedOff: true, gateExpression: expression };
|
|
227
|
+
}
|
|
228
|
+
return { gatedOff: false, gateExpression: expression };
|
|
229
|
+
}
|
|
230
|
+
}
|
|
231
|
+
const pathGate = findPathConventionGate(sourcePath);
|
|
232
|
+
if (pathGate && host && host !== pathGate.platform) {
|
|
233
|
+
return { gatedOff: true, gateExpression: pathGate.label };
|
|
234
|
+
}
|
|
235
|
+
return { gatedOff: false };
|
|
236
|
+
}
|
|
237
|
+
//# sourceMappingURL=build-audit-platform.js.map
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Heuristic test for "this looks like a packaged-test source file" — the
|
|
3
|
+
* audit routes such paths to `_tests/` instead of `dist/`. Matches
|
|
4
|
+
* mochitest / xpcshell / browser-chrome conventions: any source under a
|
|
5
|
+
* `/test/` or `/tests/` directory, or with a `browser_` / `test_` prefix
|
|
6
|
+
* on a `.js`/`.toml` basename. Test manifests (`*.toml`, `*.list`,
|
|
7
|
+
* `*.ini`) under those directories also qualify.
|
|
8
|
+
*
|
|
9
|
+
* @param sourcePath Engine-relative POSIX path
|
|
10
|
+
* @returns True when the file belongs to the test tree, not the bundle
|
|
11
|
+
*/
|
|
12
|
+
export declare function isTestPath(sourcePath: string): boolean;
|
|
13
|
+
/**
|
|
14
|
+
* Counts how many trailing segments two paths share. Used to score
|
|
15
|
+
* candidate dist artifacts against a source path so that
|
|
16
|
+
* `branding/<name>/content/aboutDialog.css` prefers a candidate at
|
|
17
|
+
* `…/content/branding/aboutDialog.css` over one at
|
|
18
|
+
* `…/content/browser/aboutDialog.css`.
|
|
19
|
+
*
|
|
20
|
+
* @param a First path
|
|
21
|
+
* @param b Second path
|
|
22
|
+
* @returns Count of matching trailing segments (basename always counts as 1)
|
|
23
|
+
*/
|
|
24
|
+
export declare function countTrailingSegmentMatches(a: string, b: string): number;
|
|
25
|
+
/**
|
|
26
|
+
* Computes a score for `candidatePath` relative to `sourcePath`. Higher
|
|
27
|
+
* scores win. Score = trailing-segment match count, with a bonus when
|
|
28
|
+
* the candidate's path contains a meaningful intermediate segment from
|
|
29
|
+
* the source (e.g. `branding`, the branding dir name itself, etc.).
|
|
30
|
+
*
|
|
31
|
+
* The bonus exists because Firefox packaging often re-roots files: a
|
|
32
|
+
* source `branding/<name>/content/aboutDialog.css` lands at
|
|
33
|
+
* `chrome/browser/content/branding/aboutDialog.css` — only the basename
|
|
34
|
+
* trails-match, but the `branding` segment moved into the middle of
|
|
35
|
+
* the candidate path. Without the bonus, that candidate would tie with
|
|
36
|
+
* the unrelated `chrome/browser/content/browser/aboutDialog.css` and
|
|
37
|
+
* the audit would pick whichever the directory walk hit first.
|
|
38
|
+
*
|
|
39
|
+
* @param sourcePath Engine-relative POSIX path
|
|
40
|
+
* @param candidatePath Absolute path under the dist tree
|
|
41
|
+
* @returns Numeric score; higher means better match
|
|
42
|
+
*/
|
|
43
|
+
export declare function scoreCandidate(sourcePath: string, candidatePath: string): number;
|
|
44
|
+
/**
|
|
45
|
+
* Walks a tree under `root` and returns every file whose basename equals
|
|
46
|
+
* `name`. Skips dotfile / hidden directories so the symlinked
|
|
47
|
+
* `.mozbuild` cache (a full upstream copy) does not dominate the scan
|
|
48
|
+
* on macOS.
|
|
49
|
+
*
|
|
50
|
+
* @param root Tree root to search
|
|
51
|
+
* @param name Basename to match
|
|
52
|
+
* @param maxDepth Optional traversal cap (default 12)
|
|
53
|
+
* @returns All matching absolute paths
|
|
54
|
+
*/
|
|
55
|
+
export declare function findAllByBasename(root: string, name: string, maxDepth?: number): Promise<string[]>;
|
|
56
|
+
/**
|
|
57
|
+
* Resolves the best-matching artifact for a source path under one or
|
|
58
|
+
* more search roots. Returns the highest-scoring candidate by trailing
|
|
59
|
+
* segment overlap; ties go to the first-found path (deterministic via
|
|
60
|
+
* the directory-walk order). Returns undefined when no candidate exists.
|
|
61
|
+
*
|
|
62
|
+
* @param sourcePath Engine-relative POSIX source path
|
|
63
|
+
* @param searchRoots Absolute roots to scan (e.g. dist/, _tests/)
|
|
64
|
+
* @returns Best-matching artifact path, or undefined
|
|
65
|
+
*/
|
|
66
|
+
export declare function resolveBestArtifact(sourcePath: string, searchRoots: readonly string[]): Promise<string | undefined>;
|
|
@@ -0,0 +1,209 @@
|
|
|
1
|
+
// SPDX-License-Identifier: EUPL-1.2
|
|
2
|
+
/*
|
|
3
|
+
* Path-resolution helpers for the post-build dist-tree audit.
|
|
4
|
+
*
|
|
5
|
+
* Resolves the expected on-disk artifact location for a given engine
|
|
6
|
+
* source path. The previous implementation matched purely by basename and
|
|
7
|
+
* suffered three classes of false positive:
|
|
8
|
+
*
|
|
9
|
+
* 1. A branding override (e.g. `engine/browser/branding/<name>/content/aboutDialog.css`)
|
|
10
|
+
* shipped at `chrome/browser/content/branding/aboutDialog.css` would
|
|
11
|
+
* get matched against the unrelated upstream
|
|
12
|
+
* `chrome/browser/content/browser/aboutDialog.css`.
|
|
13
|
+
* 2. Test files (`browser_*.js`, `test_*.js`) live under `_tests/`, not
|
|
14
|
+
* `dist/`, so every registered test was reported as missing.
|
|
15
|
+
* 3. Build inputs (`jar.mn`, `moz.build`, `Makefile.in`, `moz.configure`)
|
|
16
|
+
* never appear under `dist/` — they are consumed, not packaged.
|
|
17
|
+
*
|
|
18
|
+
* The helpers below address (1) by ranking same-basename candidates by
|
|
19
|
+
* how many trailing path segments they share with the source, and (2) by
|
|
20
|
+
* routing test paths to a separate `_tests/`-aware resolver.
|
|
21
|
+
*
|
|
22
|
+
* (3) is handled in `build-audit.ts` via `isPackageablePath`.
|
|
23
|
+
*/
|
|
24
|
+
import { readdir } from 'node:fs/promises';
|
|
25
|
+
import { basename, join } from 'node:path';
|
|
26
|
+
import { pathExists } from '../utils/fs.js';
|
|
27
|
+
/** Maximum directory depth to traverse when scanning a tree root. */
|
|
28
|
+
const MAX_SCAN_DEPTH = 12;
|
|
29
|
+
/**
|
|
30
|
+
* Heuristic test for "this looks like a packaged-test source file" — the
|
|
31
|
+
* audit routes such paths to `_tests/` instead of `dist/`. Matches
|
|
32
|
+
* mochitest / xpcshell / browser-chrome conventions: any source under a
|
|
33
|
+
* `/test/` or `/tests/` directory, or with a `browser_` / `test_` prefix
|
|
34
|
+
* on a `.js`/`.toml` basename. Test manifests (`*.toml`, `*.list`,
|
|
35
|
+
* `*.ini`) under those directories also qualify.
|
|
36
|
+
*
|
|
37
|
+
* @param sourcePath Engine-relative POSIX path
|
|
38
|
+
* @returns True when the file belongs to the test tree, not the bundle
|
|
39
|
+
*/
|
|
40
|
+
export function isTestPath(sourcePath) {
|
|
41
|
+
if (sourcePath.includes('/test/') || sourcePath.includes('/tests/')) {
|
|
42
|
+
return true;
|
|
43
|
+
}
|
|
44
|
+
const name = basename(sourcePath);
|
|
45
|
+
if (/^browser_.+\.(js|toml|ini)$/.test(name))
|
|
46
|
+
return true;
|
|
47
|
+
if (/^test_.+\.(js|toml|ini)$/.test(name))
|
|
48
|
+
return true;
|
|
49
|
+
return false;
|
|
50
|
+
}
|
|
51
|
+
/**
|
|
52
|
+
* Splits a POSIX path into segments, dropping empties.
|
|
53
|
+
* @param path POSIX-separated path
|
|
54
|
+
*/
|
|
55
|
+
function pathSegments(path) {
|
|
56
|
+
return path.split('/').filter(Boolean);
|
|
57
|
+
}
|
|
58
|
+
/**
|
|
59
|
+
* Counts how many trailing segments two paths share. Used to score
|
|
60
|
+
* candidate dist artifacts against a source path so that
|
|
61
|
+
* `branding/<name>/content/aboutDialog.css` prefers a candidate at
|
|
62
|
+
* `…/content/branding/aboutDialog.css` over one at
|
|
63
|
+
* `…/content/browser/aboutDialog.css`.
|
|
64
|
+
*
|
|
65
|
+
* @param a First path
|
|
66
|
+
* @param b Second path
|
|
67
|
+
* @returns Count of matching trailing segments (basename always counts as 1)
|
|
68
|
+
*/
|
|
69
|
+
export function countTrailingSegmentMatches(a, b) {
|
|
70
|
+
const aSegs = pathSegments(a);
|
|
71
|
+
const bSegs = pathSegments(b);
|
|
72
|
+
let matches = 0;
|
|
73
|
+
while (matches < aSegs.length &&
|
|
74
|
+
matches < bSegs.length &&
|
|
75
|
+
aSegs[aSegs.length - 1 - matches] === bSegs[bSegs.length - 1 - matches]) {
|
|
76
|
+
matches += 1;
|
|
77
|
+
}
|
|
78
|
+
return matches;
|
|
79
|
+
}
|
|
80
|
+
/**
|
|
81
|
+
* Computes a score for `candidatePath` relative to `sourcePath`. Higher
|
|
82
|
+
* scores win. Score = trailing-segment match count, with a bonus when
|
|
83
|
+
* the candidate's path contains a meaningful intermediate segment from
|
|
84
|
+
* the source (e.g. `branding`, the branding dir name itself, etc.).
|
|
85
|
+
*
|
|
86
|
+
* The bonus exists because Firefox packaging often re-roots files: a
|
|
87
|
+
* source `branding/<name>/content/aboutDialog.css` lands at
|
|
88
|
+
* `chrome/browser/content/branding/aboutDialog.css` — only the basename
|
|
89
|
+
* trails-match, but the `branding` segment moved into the middle of
|
|
90
|
+
* the candidate path. Without the bonus, that candidate would tie with
|
|
91
|
+
* the unrelated `chrome/browser/content/browser/aboutDialog.css` and
|
|
92
|
+
* the audit would pick whichever the directory walk hit first.
|
|
93
|
+
*
|
|
94
|
+
* @param sourcePath Engine-relative POSIX path
|
|
95
|
+
* @param candidatePath Absolute path under the dist tree
|
|
96
|
+
* @returns Numeric score; higher means better match
|
|
97
|
+
*/
|
|
98
|
+
export function scoreCandidate(sourcePath, candidatePath) {
|
|
99
|
+
const trailing = countTrailingSegmentMatches(sourcePath, candidatePath);
|
|
100
|
+
const sourceSegs = pathSegments(sourcePath);
|
|
101
|
+
const candSegs = pathSegments(candidatePath);
|
|
102
|
+
// Look for source segments that appear anywhere in the candidate path
|
|
103
|
+
// but are not part of the trailing match. Each unique mid-path hit on
|
|
104
|
+
// a meaningful (>2-char, not generic like 'content'/'chrome'/'bin')
|
|
105
|
+
// segment adds 1 to the score.
|
|
106
|
+
const generic = new Set([
|
|
107
|
+
'content',
|
|
108
|
+
'chrome',
|
|
109
|
+
'bin',
|
|
110
|
+
'browser',
|
|
111
|
+
'toolkit',
|
|
112
|
+
'modules',
|
|
113
|
+
'base',
|
|
114
|
+
'app',
|
|
115
|
+
'profile',
|
|
116
|
+
'shared',
|
|
117
|
+
'themes',
|
|
118
|
+
]);
|
|
119
|
+
const trailingSet = new Set(sourceSegs.slice(sourceSegs.length - trailing));
|
|
120
|
+
let bonus = 0;
|
|
121
|
+
for (const seg of sourceSegs) {
|
|
122
|
+
if (seg.length <= 2)
|
|
123
|
+
continue;
|
|
124
|
+
if (generic.has(seg))
|
|
125
|
+
continue;
|
|
126
|
+
if (trailingSet.has(seg))
|
|
127
|
+
continue;
|
|
128
|
+
if (candSegs.includes(seg))
|
|
129
|
+
bonus += 1;
|
|
130
|
+
}
|
|
131
|
+
return trailing * 10 + bonus;
|
|
132
|
+
}
|
|
133
|
+
/**
|
|
134
|
+
* Walks a tree under `root` and returns every file whose basename equals
|
|
135
|
+
* `name`. Skips dotfile / hidden directories so the symlinked
|
|
136
|
+
* `.mozbuild` cache (a full upstream copy) does not dominate the scan
|
|
137
|
+
* on macOS.
|
|
138
|
+
*
|
|
139
|
+
* @param root Tree root to search
|
|
140
|
+
* @param name Basename to match
|
|
141
|
+
* @param maxDepth Optional traversal cap (default 12)
|
|
142
|
+
* @returns All matching absolute paths
|
|
143
|
+
*/
|
|
144
|
+
export async function findAllByBasename(root, name, maxDepth = MAX_SCAN_DEPTH) {
|
|
145
|
+
const results = [];
|
|
146
|
+
if (!(await pathExists(root)))
|
|
147
|
+
return results;
|
|
148
|
+
const stack = [{ dir: root, depth: 0 }];
|
|
149
|
+
while (stack.length > 0) {
|
|
150
|
+
const entry = stack.pop();
|
|
151
|
+
if (!entry)
|
|
152
|
+
break;
|
|
153
|
+
if (entry.depth > maxDepth)
|
|
154
|
+
continue;
|
|
155
|
+
let children;
|
|
156
|
+
try {
|
|
157
|
+
children = await readdir(entry.dir, { withFileTypes: true });
|
|
158
|
+
}
|
|
159
|
+
catch {
|
|
160
|
+
continue;
|
|
161
|
+
}
|
|
162
|
+
for (const child of children) {
|
|
163
|
+
const fullPath = join(entry.dir, child.name);
|
|
164
|
+
if (child.isDirectory()) {
|
|
165
|
+
if (child.name.startsWith('.'))
|
|
166
|
+
continue;
|
|
167
|
+
stack.push({ dir: fullPath, depth: entry.depth + 1 });
|
|
168
|
+
continue;
|
|
169
|
+
}
|
|
170
|
+
if (child.name === name) {
|
|
171
|
+
results.push(fullPath);
|
|
172
|
+
}
|
|
173
|
+
}
|
|
174
|
+
}
|
|
175
|
+
return results;
|
|
176
|
+
}
|
|
177
|
+
/**
|
|
178
|
+
* Resolves the best-matching artifact for a source path under one or
|
|
179
|
+
* more search roots. Returns the highest-scoring candidate by trailing
|
|
180
|
+
* segment overlap; ties go to the first-found path (deterministic via
|
|
181
|
+
* the directory-walk order). Returns undefined when no candidate exists.
|
|
182
|
+
*
|
|
183
|
+
* @param sourcePath Engine-relative POSIX source path
|
|
184
|
+
* @param searchRoots Absolute roots to scan (e.g. dist/, _tests/)
|
|
185
|
+
* @returns Best-matching artifact path, or undefined
|
|
186
|
+
*/
|
|
187
|
+
export async function resolveBestArtifact(sourcePath, searchRoots) {
|
|
188
|
+
const name = basename(sourcePath);
|
|
189
|
+
const allCandidates = [];
|
|
190
|
+
for (const root of searchRoots) {
|
|
191
|
+
const found = await findAllByBasename(root, name);
|
|
192
|
+
allCandidates.push(...found);
|
|
193
|
+
}
|
|
194
|
+
if (allCandidates.length === 0)
|
|
195
|
+
return undefined;
|
|
196
|
+
if (allCandidates.length === 1)
|
|
197
|
+
return allCandidates[0];
|
|
198
|
+
let bestScore = -1;
|
|
199
|
+
let best;
|
|
200
|
+
for (const candidate of allCandidates) {
|
|
201
|
+
const score = scoreCandidate(sourcePath, candidate);
|
|
202
|
+
if (score > bestScore) {
|
|
203
|
+
bestScore = score;
|
|
204
|
+
best = candidate;
|
|
205
|
+
}
|
|
206
|
+
}
|
|
207
|
+
return best;
|
|
208
|
+
}
|
|
209
|
+
//# sourceMappingURL=build-audit-resolve.js.map
|
|
@@ -28,7 +28,10 @@ export interface AuditSummary {
|
|
|
28
28
|
}
|
|
29
29
|
/**
|
|
30
30
|
* Decides whether a source path should be packaged. Returns true for paths
|
|
31
|
-
* whose extension or directory fragment matches a known-packaged pattern
|
|
31
|
+
* whose extension or directory fragment matches a known-packaged pattern,
|
|
32
|
+
* after excluding build inputs (`jar.mn`, `moz.build`, etc.) which are
|
|
33
|
+
* consumed by the build but never themselves packaged.
|
|
34
|
+
*
|
|
32
35
|
* @param sourcePath Engine-relative POSIX path (for example browser/app/profile/pref.js).
|
|
33
36
|
* @returns True when the path implies packaging.
|
|
34
37
|
*/
|
|
@@ -19,12 +19,27 @@
|
|
|
19
19
|
* - False positives are acceptable at this stage: fork-specific packaging
|
|
20
20
|
* tricks FireForge doesn't know about will surface as warnings an
|
|
21
21
|
* operator can investigate. The audit never fails the build.
|
|
22
|
+
*
|
|
23
|
+
* Routing rules:
|
|
24
|
+
* - Build inputs (jar.mn, moz.build, Makefile.in, moz.configure) are
|
|
25
|
+
* skipped; they are consumed, not packaged.
|
|
26
|
+
* - Test sources (anything under /test(s)/, browser_*.js, test_*.js)
|
|
27
|
+
* are looked up under _tests/, not dist/ — that's where mach copies
|
|
28
|
+
* them.
|
|
29
|
+
* - Files inside an `if CONFIG[...]:` block in moz.build that gates
|
|
30
|
+
* off on the current host are skipped (Windows stubinstaller CSS on
|
|
31
|
+
* a macOS build, etc.).
|
|
32
|
+
* - Same-basename collisions in dist/ are disambiguated by trailing-
|
|
33
|
+
* segment overlap so a branding override does not get matched
|
|
34
|
+
* against an unrelated upstream file with the same basename.
|
|
22
35
|
*/
|
|
23
36
|
import { stat } from 'node:fs/promises';
|
|
24
37
|
import { basename, join } from 'node:path';
|
|
25
38
|
import { toError } from '../utils/errors.js';
|
|
26
39
|
import { pathExists } from '../utils/fs.js';
|
|
27
40
|
import { info, verbose, warn } from '../utils/logger.js';
|
|
41
|
+
import { detectPlatformGate } from './build-audit-platform.js';
|
|
42
|
+
import { countTrailingSegmentMatches, isTestPath, resolveBestArtifact, } from './build-audit-resolve.js';
|
|
28
43
|
import { hasChanges, isMissingHeadError } from './git.js';
|
|
29
44
|
import { git } from './git-base.js';
|
|
30
45
|
import { getUntrackedFiles } from './git-status.js';
|
|
@@ -44,48 +59,29 @@ const PACKAGEABLE_EXTENSIONS = [
|
|
|
44
59
|
const PACKAGEABLE_PATH_FRAGMENTS = ['/app/profile/', '/chrome/', '/locales/'];
|
|
45
60
|
/** Directories that are build artifacts, not source — never audited. */
|
|
46
61
|
const IGNORE_PATH_FRAGMENTS = ['obj-', 'node_modules/', '.git/', '.cargo/', '.mozbuild/'];
|
|
47
|
-
|
|
48
|
-
*
|
|
49
|
-
*
|
|
50
|
-
*
|
|
51
|
-
*
|
|
62
|
+
/**
|
|
63
|
+
* Basenames that are build-system inputs, not packaged artifacts.
|
|
64
|
+
* `jar.mn` is consumed to produce chrome registrations; `moz.build` /
|
|
65
|
+
* `moz.configure` / `Makefile.in` feed the build backend; none of them
|
|
66
|
+
* ship in the bundle. Auditing them produced a guaranteed false
|
|
67
|
+
* positive on every edit (nothing under dist/ ever has these names),
|
|
68
|
+
* and a worse failure mode when an unrelated upstream `moz.build`
|
|
69
|
+
* coincidentally exists at e.g. `MyBrowser.app/Contents/moz.build` and
|
|
70
|
+
* gets matched as a "stale artifact" of an entirely different file.
|
|
52
71
|
*/
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
if (entry.depth > maxDepth)
|
|
61
|
-
continue;
|
|
62
|
-
let children;
|
|
63
|
-
try {
|
|
64
|
-
children = await readdir(entry.dir, { withFileTypes: true });
|
|
65
|
-
}
|
|
66
|
-
catch {
|
|
67
|
-
continue;
|
|
68
|
-
}
|
|
69
|
-
for (const child of children) {
|
|
70
|
-
const fullPath = join(entry.dir, child.name);
|
|
71
|
-
if (child.isDirectory()) {
|
|
72
|
-
// Skip the symlinked mozbuild cache tree which contains full copies
|
|
73
|
-
// and would dominate the scan on macOS.
|
|
74
|
-
if (child.name.startsWith('.'))
|
|
75
|
-
continue;
|
|
76
|
-
stack.push({ dir: fullPath, depth: entry.depth + 1 });
|
|
77
|
-
continue;
|
|
78
|
-
}
|
|
79
|
-
if (child.name === name) {
|
|
80
|
-
return fullPath;
|
|
81
|
-
}
|
|
82
|
-
}
|
|
83
|
-
}
|
|
84
|
-
return undefined;
|
|
85
|
-
}
|
|
72
|
+
const BUILD_INPUT_BASENAMES = new Set([
|
|
73
|
+
'jar.mn',
|
|
74
|
+
'moz.build',
|
|
75
|
+
'moz.configure',
|
|
76
|
+
'Makefile.in',
|
|
77
|
+
'mozbuild.in',
|
|
78
|
+
]);
|
|
86
79
|
/**
|
|
87
80
|
* Decides whether a source path should be packaged. Returns true for paths
|
|
88
|
-
* whose extension or directory fragment matches a known-packaged pattern
|
|
81
|
+
* whose extension or directory fragment matches a known-packaged pattern,
|
|
82
|
+
* after excluding build inputs (`jar.mn`, `moz.build`, etc.) which are
|
|
83
|
+
* consumed by the build but never themselves packaged.
|
|
84
|
+
*
|
|
89
85
|
* @param sourcePath Engine-relative POSIX path (for example browser/app/profile/pref.js).
|
|
90
86
|
* @returns True when the path implies packaging.
|
|
91
87
|
*/
|
|
@@ -94,6 +90,8 @@ export function isPackageablePath(sourcePath) {
|
|
|
94
90
|
if (sourcePath.includes(fragment))
|
|
95
91
|
return false;
|
|
96
92
|
}
|
|
93
|
+
if (BUILD_INPUT_BASENAMES.has(basename(sourcePath)))
|
|
94
|
+
return false;
|
|
97
95
|
for (const ext of PACKAGEABLE_EXTENSIONS) {
|
|
98
96
|
if (sourcePath.endsWith(ext))
|
|
99
97
|
return true;
|
|
@@ -169,6 +167,203 @@ async function resolveDistRoot(engineDir) {
|
|
|
169
167
|
}
|
|
170
168
|
return undefined;
|
|
171
169
|
}
|
|
170
|
+
/**
|
|
171
|
+
* Resolves the `_tests/` tree under the active obj-* directory, used as
|
|
172
|
+
* a secondary search root for sources that look like packaged tests.
|
|
173
|
+
* Returns undefined when no obj dir exists yet.
|
|
174
|
+
*/
|
|
175
|
+
async function resolveTestsRoot(engineDir) {
|
|
176
|
+
const { readdir } = await import('node:fs/promises');
|
|
177
|
+
let entries;
|
|
178
|
+
try {
|
|
179
|
+
entries = await readdir(engineDir);
|
|
180
|
+
}
|
|
181
|
+
catch {
|
|
182
|
+
return undefined;
|
|
183
|
+
}
|
|
184
|
+
const objDirs = entries.filter((e) => e.startsWith('obj-'));
|
|
185
|
+
for (const objDir of objDirs) {
|
|
186
|
+
const testsPath = join(engineDir, objDir, '_tests');
|
|
187
|
+
if (await pathExists(testsPath)) {
|
|
188
|
+
return testsPath;
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
return undefined;
|
|
192
|
+
}
|
|
193
|
+
/**
|
|
194
|
+
* Marker file the `package-tests` make target writes after copying the
|
|
195
|
+
* full test-source tree under `_tests/`. Its presence is the most reliable
|
|
196
|
+
* signal that test packaging has actually run for the current obj-dir —
|
|
197
|
+
* plain `mach build` populates a partial `_tests/` subtree and then stops,
|
|
198
|
+
* so registered tests are absent even when registration is correct.
|
|
199
|
+
*/
|
|
200
|
+
const PACKAGED_TESTS_MARKER = 'all-tests.json';
|
|
201
|
+
/**
|
|
202
|
+
* Returns true when the full test-package step has actually run for the
|
|
203
|
+
* active obj-dir. Without this marker the `_tests/` walk produces false
|
|
204
|
+
* positives for every correctly-registered mochitest / xpcshell source
|
|
205
|
+
* on the common "built but tests not packaged" path.
|
|
206
|
+
*
|
|
207
|
+
* @param testsRoot Absolute path to the obj-*`/_tests/` tree, or undefined.
|
|
208
|
+
*/
|
|
209
|
+
async function hasPackagedTestsMarker(testsRoot) {
|
|
210
|
+
if (!testsRoot)
|
|
211
|
+
return false;
|
|
212
|
+
return pathExists(join(testsRoot, PACKAGED_TESTS_MARKER));
|
|
213
|
+
}
|
|
214
|
+
/**
|
|
215
|
+
* Resolves the search roots an individual source path should be looked
|
|
216
|
+
* up under. Test-shaped paths get `_tests/`; everything else gets `dist/`.
|
|
217
|
+
*/
|
|
218
|
+
function searchRootsFor(source, distRoot, testsRoot) {
|
|
219
|
+
if (isTestPath(source)) {
|
|
220
|
+
return testsRoot ? [testsRoot] : [];
|
|
221
|
+
}
|
|
222
|
+
return [distRoot];
|
|
223
|
+
}
|
|
224
|
+
/**
|
|
225
|
+
* Minimum trailing-segment overlap required for a same-basename dist/
|
|
226
|
+
* candidate to count as "the packaged artifact" of a source. The
|
|
227
|
+
* basename always trail-matches (count 1), so a threshold of 2 requires
|
|
228
|
+
* the immediate parent directory to also agree. Candidates that only
|
|
229
|
+
* share the basename are classified as missing — warning the operator
|
|
230
|
+
* to check registration — rather than emitting a misleading stale
|
|
231
|
+
* comparison against an unrelated file of the same name.
|
|
232
|
+
*
|
|
233
|
+
* Cross-tree re-rooting cases (e.g. `branding/<name>/content/foo.css`
|
|
234
|
+
* landing at `chrome/<area>/content/branding/foo.css`) bypass this
|
|
235
|
+
* floor because `scoreCandidate` awards a non-generic-segment bonus
|
|
236
|
+
* that lifts the confidence regardless of trailing overlap; those are
|
|
237
|
+
* detected below in `isConfidentMatch`.
|
|
238
|
+
*/
|
|
239
|
+
const MIN_TRAILING_SEGMENT_OVERLAP = 2;
|
|
240
|
+
/**
|
|
241
|
+
* Returns true when the chosen artifact is structurally related to the
|
|
242
|
+
* source path — either its immediate parent directory trail-matches, or
|
|
243
|
+
* a non-generic intermediate source segment appears in the candidate
|
|
244
|
+
* path (the branding-re-root signal already used by the scorer).
|
|
245
|
+
*
|
|
246
|
+
* Used to avoid emitting `stale` warnings that point at an unrelated
|
|
247
|
+
* same-basename file picked up by the basename walker — a class of
|
|
248
|
+
* warning that is worse than `missing` because it reads as "your build
|
|
249
|
+
* dropped this file" when in fact the match is spurious.
|
|
250
|
+
*/
|
|
251
|
+
function isConfidentMatch(source, candidate) {
|
|
252
|
+
if (countTrailingSegmentMatches(source, candidate) >= MIN_TRAILING_SEGMENT_OVERLAP) {
|
|
253
|
+
return true;
|
|
254
|
+
}
|
|
255
|
+
const sourceSegs = source.split('/').filter(Boolean);
|
|
256
|
+
const candSegs = candidate.split('/').filter(Boolean);
|
|
257
|
+
const generic = new Set([
|
|
258
|
+
'content',
|
|
259
|
+
'chrome',
|
|
260
|
+
'bin',
|
|
261
|
+
'browser',
|
|
262
|
+
'toolkit',
|
|
263
|
+
'modules',
|
|
264
|
+
'base',
|
|
265
|
+
'app',
|
|
266
|
+
'profile',
|
|
267
|
+
'shared',
|
|
268
|
+
'themes',
|
|
269
|
+
'test',
|
|
270
|
+
'tests',
|
|
271
|
+
'unit',
|
|
272
|
+
'common',
|
|
273
|
+
'xpcshell',
|
|
274
|
+
'mochitest',
|
|
275
|
+
]);
|
|
276
|
+
// Skip the basename itself (which trail-matches by definition).
|
|
277
|
+
for (let i = 0; i < sourceSegs.length - 1; i += 1) {
|
|
278
|
+
const seg = sourceSegs[i];
|
|
279
|
+
if (!seg || seg.length <= 2 || generic.has(seg))
|
|
280
|
+
continue;
|
|
281
|
+
if (candSegs.includes(seg))
|
|
282
|
+
return true;
|
|
283
|
+
}
|
|
284
|
+
return false;
|
|
285
|
+
}
|
|
286
|
+
/**
|
|
287
|
+
* Audits one engine source path and returns its entry. Pure orchestration
|
|
288
|
+
* helper kept separate so `auditBuildArtifacts` stays under the per-function
|
|
289
|
+
* line budget.
|
|
290
|
+
*/
|
|
291
|
+
async function auditSinglePath(source, ctx) {
|
|
292
|
+
if (!isPackageablePath(source)) {
|
|
293
|
+
return { source, artifact: undefined, status: 'skipped' };
|
|
294
|
+
}
|
|
295
|
+
const gate = await detectPlatformGate(ctx.engineDir, source);
|
|
296
|
+
if (gate.gatedOff) {
|
|
297
|
+
verbose(`Audit: skipping engine/${source} — gated off by "${gate.gateExpression ?? '?'}".`);
|
|
298
|
+
return { source, artifact: undefined, status: 'skipped' };
|
|
299
|
+
}
|
|
300
|
+
// Tests only end up under `_tests/` after `mach package-tests` (or a
|
|
301
|
+
// test-run that invokes the target) has executed. A plain `mach build`
|
|
302
|
+
// populates a partial subtree and stops, so every correctly-registered
|
|
303
|
+
// mochitest / xpcshell source appears "missing" on that common path.
|
|
304
|
+
// Skip audit for test sources when no packaged-tests marker is present.
|
|
305
|
+
if (isTestPath(source) && !ctx.testsPackaged) {
|
|
306
|
+
verbose(`Audit: skipping engine/${source} — _tests/${PACKAGED_TESTS_MARKER} not present; full test packaging has not run for this build.`);
|
|
307
|
+
return { source, artifact: undefined, status: 'skipped' };
|
|
308
|
+
}
|
|
309
|
+
const sourcePath = join(ctx.engineDir, source);
|
|
310
|
+
let sourceMtime;
|
|
311
|
+
try {
|
|
312
|
+
const sourceStat = await stat(sourcePath);
|
|
313
|
+
sourceMtime = sourceStat.mtimeMs;
|
|
314
|
+
}
|
|
315
|
+
catch {
|
|
316
|
+
// Deletion that didn't propagate — distinct class of bug, not audited yet.
|
|
317
|
+
return { source, artifact: undefined, status: 'skipped' };
|
|
318
|
+
}
|
|
319
|
+
const roots = searchRootsFor(source, ctx.distRoot, ctx.testsRoot);
|
|
320
|
+
const artifact = await resolveBestArtifact(source, roots);
|
|
321
|
+
if (!artifact) {
|
|
322
|
+
const where = isTestPath(source) ? '_tests/' : 'dist/';
|
|
323
|
+
return {
|
|
324
|
+
source,
|
|
325
|
+
artifact: undefined,
|
|
326
|
+
status: 'missing',
|
|
327
|
+
warning: `Audit: engine/${source} was touched but no packaged artifact with basename "${basename(source)}" was found under ${where}. Missing moz.build / jar.mn / package-manifest.in registration?`,
|
|
328
|
+
};
|
|
329
|
+
}
|
|
330
|
+
let artifactMtime;
|
|
331
|
+
try {
|
|
332
|
+
const artifactStat = await stat(artifact);
|
|
333
|
+
artifactMtime = artifactStat.mtimeMs;
|
|
334
|
+
}
|
|
335
|
+
catch {
|
|
336
|
+
return {
|
|
337
|
+
source,
|
|
338
|
+
artifact,
|
|
339
|
+
status: 'missing',
|
|
340
|
+
warning: `Audit: engine/${source} has no readable packaged artifact at ${artifact} (disappeared during audit).`,
|
|
341
|
+
};
|
|
342
|
+
}
|
|
343
|
+
if (artifactMtime + 1 < sourceMtime) {
|
|
344
|
+
// Before claiming "stale", verify the match is structurally related.
|
|
345
|
+
// A same-basename hit in an unrelated subtree (e.g. `head.js` in a
|
|
346
|
+
// completely different upstream test helper) should be reported as
|
|
347
|
+
// missing — the operator needs to check registration, not puzzle
|
|
348
|
+
// over why an unrelated file appears "older than the source".
|
|
349
|
+
if (!isConfidentMatch(source, artifact)) {
|
|
350
|
+
const where = isTestPath(source) ? '_tests/' : 'dist/';
|
|
351
|
+
return {
|
|
352
|
+
source,
|
|
353
|
+
artifact: undefined,
|
|
354
|
+
status: 'missing',
|
|
355
|
+
warning: `Audit: engine/${source} was touched but no related packaged artifact with basename "${basename(source)}" was found under ${where} (nearest same-basename file ${artifact} lives in an unrelated subtree). Missing moz.build / jar.mn / package-manifest.in registration?`,
|
|
356
|
+
};
|
|
357
|
+
}
|
|
358
|
+
return {
|
|
359
|
+
source,
|
|
360
|
+
artifact,
|
|
361
|
+
status: 'stale',
|
|
362
|
+
warning: `Audit: engine/${source} is newer than its packaged artifact ${artifact}. Build reported success but the file's path may not flow through packaging — check moz.build / jar.mn entries.`,
|
|
363
|
+
};
|
|
364
|
+
}
|
|
365
|
+
return { source, artifact, status: 'updated' };
|
|
366
|
+
}
|
|
172
367
|
/**
|
|
173
368
|
* Runs the post-build audit. Emits per-file warnings for missing or
|
|
174
369
|
* stale artifacts and a summary info line at the end. Always returns
|
|
@@ -193,57 +388,23 @@ export async function auditBuildArtifacts(projectRoot, engineDir, baseline) {
|
|
|
193
388
|
verbose('Audit skipped: no dist tree found under obj-*/dist/.');
|
|
194
389
|
return summary;
|
|
195
390
|
}
|
|
391
|
+
const testsRoot = await resolveTestsRoot(engineDir);
|
|
392
|
+
const testsPackaged = await hasPackagedTestsMarker(testsRoot);
|
|
196
393
|
const changed = await collectChangedFiles(engineDir, baseline);
|
|
197
394
|
if (changed.length === 0) {
|
|
198
395
|
return summary;
|
|
199
396
|
}
|
|
397
|
+
const ctx = { engineDir, distRoot, testsRoot, testsPackaged };
|
|
200
398
|
for (const source of changed) {
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
sourceMtime = sourceStat.mtimeMs;
|
|
211
|
-
}
|
|
212
|
-
catch {
|
|
213
|
-
// File was deleted since the diff was computed. Skip — a deletion
|
|
214
|
-
// that didn't propagate to the dist tree is a distinct class of bug
|
|
215
|
-
// we don't audit yet.
|
|
216
|
-
summary.skipped += 1;
|
|
217
|
-
summary.entries.push({ source, artifact: undefined, status: 'skipped' });
|
|
218
|
-
continue;
|
|
219
|
-
}
|
|
220
|
-
const artifact = await findArtifactByBasename(distRoot, basename(source));
|
|
221
|
-
if (!artifact) {
|
|
222
|
-
summary.missing += 1;
|
|
223
|
-
summary.entries.push({ source, artifact: undefined, status: 'missing' });
|
|
224
|
-
warn(`Audit: engine/${source} was touched but no packaged artifact with basename "${basename(source)}" was found under ${distRoot}. Missing moz.build / jar.mn / package-manifest.in registration?`);
|
|
225
|
-
continue;
|
|
226
|
-
}
|
|
227
|
-
let artifactMtime;
|
|
228
|
-
try {
|
|
229
|
-
const artifactStat = await stat(artifact);
|
|
230
|
-
artifactMtime = artifactStat.mtimeMs;
|
|
231
|
-
}
|
|
232
|
-
catch {
|
|
233
|
-
// Disappeared after the directory scan; treat as missing.
|
|
234
|
-
summary.missing += 1;
|
|
235
|
-
summary.entries.push({ source, artifact, status: 'missing' });
|
|
236
|
-
warn(`Audit: engine/${source} has no readable packaged artifact at ${artifact} (disappeared during audit).`);
|
|
237
|
-
continue;
|
|
238
|
-
}
|
|
239
|
-
if (artifactMtime + 1 < sourceMtime) {
|
|
240
|
-
summary.stale += 1;
|
|
241
|
-
summary.entries.push({ source, artifact, status: 'stale' });
|
|
242
|
-
warn(`Audit: engine/${source} is newer than its packaged artifact ${artifact}. Build reported success but the file's path may not flow through packaging — check moz.build / jar.mn entries.`);
|
|
243
|
-
continue;
|
|
244
|
-
}
|
|
245
|
-
summary.updated += 1;
|
|
246
|
-
summary.entries.push({ source, artifact, status: 'updated' });
|
|
399
|
+
const result = await auditSinglePath(source, ctx);
|
|
400
|
+
summary[result.status] += 1;
|
|
401
|
+
summary.entries.push({
|
|
402
|
+
source: result.source,
|
|
403
|
+
artifact: result.artifact,
|
|
404
|
+
status: result.status,
|
|
405
|
+
});
|
|
406
|
+
if (result.warning)
|
|
407
|
+
warn(result.warning);
|
|
247
408
|
}
|
|
248
409
|
info(`Packaged: ${summary.updated} updated, ${summary.stale} stale, ${summary.missing} missing, ${summary.skipped} skipped`);
|
|
249
410
|
return summary;
|