@hominis/fireforge 0.19.6 → 0.21.0

package/CHANGELOG.md

@@ -1,5 +1,49 @@
 # Changelog
 
+## 0.21.0
+
+### Features
+
+- **Chrome-doc previews and cleanup.** `furnace chrome-doc create` now supports `--dry-run`, validating the same target files and jar registrations without writing. New `furnace chrome-doc remove <name>` removes scaffolded chrome-doc files, jar entries, and optional xpcshell packaging-test directories, with `--dry-run` and `--yes` support.
+- **Versioned `status --json` schema.** The JSON output is now an object with `schemaVersion`, `summary`, and `files` instead of a bare array. Error paths also emit versioned JSON objects with `code` and `error`.
+
+### Hardening
+
+- **Override removal demotes back to stock.** Removing a Furnace override restores engine files, deletes the override workspace, clears override checksums, and re-adds the component to `stock` tracking instead of dropping it from `furnace.json`. Optional Furnace config fields, including `platformPrefixes`, are preserved across the write.
+- **Rename updates browser-chrome test bodies.** `furnace rename` now rewrites generated browser-chrome mochitest contents as well as filenames and `browser.toml`, preventing stale `waitForElement("<old>")` references after a component rename.
+- **UI build preflight is stricter.** `fireforge build --ui` now refuses before `mach build faster` when the current objdir lacks a completed launchable bundle, guiding fresh imports and partial builds through a full `fireforge build` first.
+- **Interrupt and diagnostics polish.** Signal-driven Furnace preview teardown has regression coverage for stale lock cleanup, `chrome-doc-rollback` markers round-trip through Furnace state validation, watch-mode permission failures name the macOS privacy remediation, and `test --doctor` prints the probed objdir, binary/app path, port, and elapsed time.
+
+### Documentation
+
+- **README — Storybook first-run and audit posture.** The Furnace preview docs now call out the upstream Storybook npm install and audit output as Firefox Storybook workspace dependency state, not FireForge package dependency state.
+- **README — chrome-doc lifecycle and JSON schema.** The Furnace and status sections document chrome-doc dry-runs/removal, UI-build preconditions, watch privacy guidance, and the new versioned `status --json` object.
+
+## 0.20.0
+
+### Features
+
+- **Pinned Firefox archive verification.** `FirefoxConfig` now accepts optional `firefox.sha256`, and `fireforge config firefox.sha256 <digest>` is a supported config path. The digest must be a 64-character SHA-256 hex string; FireForge verifies both cached and freshly downloaded source archives against it before extraction, so reproducible workflows can fail fast on a mismatched upstream archive instead of discovering the problem after engine mutation has begun.
+- **`fireforge patch compact`.** Patch queues with ordinal gaps after deletes, splits, or manual repair can now be compacted back to contiguous numbering under the existing patch manifest lock. The command supports dry-run previews, confirmation/`--yes`, lock-time recomputation, history entries, and the same two-phase rename/rollback safety as other patch renumbering paths.
+- **Furnace xpcshell scaffolding and rename helpers.** Furnace test scaffolding now covers xpcshell packaging probes for storage/module-loading code paths, and rename logic is split into focused helpers for component filenames and config rewrites so custom and override renames share the same behaviour.
+
+### Hardening
+
+- **`fireforge download` is locked end-to-end.** The command now takes a project-level `.fireforge/` sidecar lock around the full engine mutation sequence: engine existence checks, forced removal, source download/extraction, git init or resume, patch cleanup, and state updates. Parallel invocations queue with explicit timeout/stale-lock messaging instead of racing over `engine/`.
+- **Firefox archive cache mutation is locked per archive.** Cache validation, invalidation, download promotion, and metadata writes now run under a per-archive cache lock. A failed downloader removes only its own unique `.part-*` file unless it already promoted the tarball, so a failing peer can no longer delete another process's valid final cache entry.
+- **Download stream timers clean up on all terminal paths.** The stall detector now clears its timer from `destroy(error, callback)` as well as normal `flush`, avoiding late timer errors and unnecessary event-loop retention after pipeline failures.
+- **Relative import linting is AST-backed.** `fireforge lint` now uses Acorn/ESTree traversal to detect relative ES imports, side-effect imports, dynamic `import()`, relative re-exports, and `ChromeUtils` / `Cu.import` calls, with a narrow stripped-text fallback only when parsing fails. This closes the false negatives left by the old regex pass while keeping malformed-file diagnostics useful.
+- **Cross-platform process and lock probes are stricter.** `findExecutable` now parses Windows `where` output with CRLF-safe per-line trimming and returns the first non-empty candidate. File-lock stale recovery treats only `ESRCH` from `process.kill(pid, 0)` as dead; `EPERM` and unknown errors are treated as live/unknown so FireForge does not reclaim a lock owned by another live process.
+- **Filesystem writes get low-risk durability checks.** `writeFileAtomic` preserves the existing mode-preservation behaviour and now best-effort fsyncs the parent directory after rename where the platform supports directory handles. `pathExistsStrict` is available for user-facing probes that should surface `EACCES` / `EPERM` instead of collapsing them into "missing".
+- **Furnace mutations use fresh locked state.** `furnace create`, `furnace remove`, and `furnace rename` re-read `furnace.json` inside the mutation lock before validation and writeback, preserving sibling entries created by concurrent commands instead of writing back stale outer snapshots. `furnace remove` also shares cleanup state for custom browser-chrome, xpcshell, and MochiKit scaffolds.
+- **Furnace refresh distinguishes conflicts from fatal merge failures.** `git merge-file` result handling now treats normal conflict exits separately from fatal/error output or high exit codes. `furnace refresh --all` continues through later overrides after a per-component failure, reports the failed count, and exits non-zero with the failed override names once the rest of the selection has been attempted.
+- **Shared naming and coverage drift guards.** Export placement now reuses `sanitizeName` from patch export instead of maintaining a duplicate slug helper, and the dedicated coverage-threshold script now enrols newly critical modules including Furnace refresh, patch compact, and Furnace xpcshell rename handling.
+
+### Documentation
+
+- **README — download locks and pinned archive checksums.** The quick-start and configuration sections now describe the project download lock, per-archive cache lock, and `firefox.sha256` workflow.
+- **README — concurrent Furnace mutations.** The Furnace section now documents locked fresh-state writeback for create/remove/rename and the `refresh --all` failure summary behaviour.
+
 ## 0.19.0
 
 ### Features

package/README.md

@@ -36,7 +36,7 @@ Inspired by [fern.js](https://github.com/ghostery/user-agent-desktop) and [Melon
 - **Python 3** (required by Firefox's `mach` build system).
 - **Git**
 - Platform build tools: Xcode on macOS, `build-essential` on Linux, Visual Studio Build Tools on Windows.
-- **Watchman** (optional, only required by `fireforge watch`). Install via `brew install watchman` (macOS), `dnf install watchman` (Fedora), or follow the upstream [Meta docs](https://facebook.github.io/watchman/). `fireforge doctor` surfaces a warning row when it is not on `PATH` so the dependency is visible during the usual onboarding sweep rather than at the watch-mode failure site. `fireforge watch` resolves watchman's absolute path via `which` / `where` and prepends its directory to the subprocess `PATH` it hands mach, so a homebrew-installed watchman at `/opt/homebrew/bin/watchman` (absent from the Node subprocess's default `PATH` on macOS) is still visible to `mach watch` without the operator having to re-export `PATH` manually.
+- **Watchman** (optional, only required by `fireforge watch`). Install via `brew install watchman` (macOS), `dnf install watchman` (Fedora), or follow the upstream [Meta docs](https://facebook.github.io/watchman/). `fireforge doctor` surfaces a warning row when it is not on `PATH` so the dependency is visible during the usual onboarding sweep rather than at the watch-mode failure site. `fireforge watch` resolves watchman's absolute path via `which` / `where` and prepends its directory to the subprocess `PATH` it hands mach, so a homebrew-installed watchman at `/opt/homebrew/bin/watchman` (absent from the Node subprocess's default `PATH` on macOS) is still visible to `mach watch` without the operator having to re-export `PATH` manually. If `mach watch` reports `Operation not permitted` / `EPERM` on macOS, FireForge points at the usual privacy fix: grant Full Disk Access or Files and Folders access to the terminal/Codex app and watchman, then restart watchman with `watchman shutdown-server`.
 
 ### Setup
 
@@ -87,6 +87,8 @@ npx fireforge rebase
 
 `fireforge download` indexes the extracted Firefox source into a fresh git repository — a one-time 1–3 minute pass on a cold SSD, longer on slow or loaded disks. The monolithic `git add -A` is capped at 10 minutes by default and falls back to a per-directory chunked pass (30 minutes per chunk) when the cap hits. If indexing still times out, the command now raises `GitIndexingTimeoutError` with recovery guidance: extend the cap via `FIREFORGE_GIT_ADD_TIMEOUT_MS` (monolithic) and/or `FIREFORGE_GIT_ADD_CHUNK_TIMEOUT_MS` (chunked) in milliseconds, e.g. `FIREFORGE_GIT_ADD_TIMEOUT_MS=1800000 fireforge download --force` for a 30-minute monolithic budget, then re-run `fireforge download --force` — the resume path picks up from the partial git state so the repeat is not wasted work.
 
+`fireforge download` is serialised by a project lock under `.fireforge/`, covering the engine-exists check, forced replacement, extraction, git initialisation/resume, patch cleanup and state update. Archive cache entries are also guarded by per-archive locks, so parallel downloads queue instead of racing over the same `engine/` directory or deleting each other's cache results. For reproducible workflows, set `firefox.sha256` to the expected 64-character archive SHA-256; FireForge verifies both cached and freshly downloaded archives before extraction and refuses a mismatch before touching the engine.
+
 `fireforge rebase --dry-run` refuses when the engine has no baseline commit yet (e.g. the aftermath of an aborted `download --force`), so dry-run and real-run preconditions stay in sync.
 
 ## Patch Workflow
@@ -288,9 +290,11 @@ When a patch queue drifts, e.g. due to overlapping new-file creations, forward i
 fireforge verify                    # fsck: manifest + cross-patch lint
 fireforge lint                      # includes the same cross-patch rules
 fireforge status --ownership        # flat path → owning patch table
-fireforge status --json             # machine-readable classified output
+fireforge status --json             # machine-readable classified object
 ```
 
+`status --json` emits a versioned object: `{ "schemaVersion": 1, "summary": { "total": <n>, "byClassification": { ... } }, "files": [...] }`. Error paths also emit a JSON object with `schemaVersion`, `code`, and `error` before exiting non-zero, so scripts can parse both clean and failing runs.
+
 Then fix with the appropriate primitive:
 
 | Problem                                        | Fix                                                                                                              |
@@ -377,6 +381,9 @@ Custom elements live under `toolkit/content/widgets`, but a fork's top-level chr
 fireforge furnace chrome-doc create mybrowser              # full chrome (titlebar + windowtype)
 fireforge furnace chrome-doc create overlay --no-titlebar  # frameless overlay
 fireforge furnace chrome-doc create mybrowser --with-tests # + xpcshell packaging-verification test
+fireforge furnace chrome-doc create mybrowser --dry-run    # preview without writing
+fireforge furnace chrome-doc remove mybrowser --dry-run    # preview cleanup
+fireforge furnace chrome-doc remove mybrowser --yes        # remove files + registrations
 ```
 
 The command writes:
@@ -388,7 +395,7 @@ The command writes:
 - Appends the corresponding `jar.mn` / `jar.inc.mn` entries. The locales/jar.mn append is suppressed when the fork's existing `engine/browser/locales/jar.mn` already carries a `[localization] (%browser/**/*.ftl)` (or `(%browser/*.ftl)`) wildcard that would already pick up the scaffolded FTL — on those forks a per-file `locale/<name>.ftl` entry would be dead weight at best and an outright build break when the fork has dropped the `% locale browser …` registration the per-file entry depends on. Forks still on the legacy registration get the per-file entry as before.
 - When `--with-tests` is set, also scaffolds an xpcshell test + `xpcshell.toml` under `engine/browser/base/content/test/<binary>-xpcshell/<name>/` that probes the packaged app directory (`Services.dirsvc.get("XCurProcD")/chrome/browser/...`) directly rather than going through `chrome://` URI resolution — see "Platform module compatibility" and the xpcshell chrome-URI note further down for why direct filesystem probing is the reliable way to verify chrome-doc packaging. Registration in `XPCSHELL_TESTS_MANIFESTS` is left to the operator because the owning moz.build depends on the fork layout.
 
-Writes are transactional: a SIGINT mid-scaffold rolls back every touched file. Requires an existing engine — run `fireforge download` first.
+Writes are transactional: a SIGINT mid-scaffold rolls back every touched file. `--dry-run` validates the same paths and registrations without acquiring the mutation lock or writing. `furnace chrome-doc remove <name>` removes the scaffolded source files, jar registrations, and optional xpcshell packaging-test directory; use its `--dry-run` first when cleaning an experimental document. Requires an existing engine — run `fireforge download` first.
 
 #### Platform module compatibility
 
@@ -435,6 +442,10 @@ Three styles are available via `--test-style`:
 
 `furnace create --dry-run` previews the planned file set, test scaffold, and `furnace.json` mutation without writing anything. Every validation the real command runs (tag-name shape, name conflicts, engine pre-existence of the component, `--compose` target existence + cycle detection) fires BEFORE the plan is emitted, so a failed preview matches a failed real run.
 
+`furnace create`, `furnace remove`, and `furnace rename` re-read `furnace.json` inside the mutation lock before writing, so concurrent component edits preserve sibling entries instead of writing back a stale outer snapshot. `furnace refresh --all` continues past per-component refresh failures, reports the failed count, and exits non-zero with the failed override names after finishing the rest of the selection.
+
+`furnace preview` starts Firefox's upstream Storybook workspace. On the first run, `mach storybook` may install roughly a thousand npm packages under `engine/browser/components/storybook/` and may print npm audit counts from Storybook's transitive dependencies. FireForge frames that output as upstream Storybook dependency state; it does not mean FireForge's own package dependencies were installed into the project.
+
 ## Additional Commands
 
 The commands below cover project configuration, patch queue management, build packaging and development utilities. Run `fireforge <command> --help` for full option details.
@@ -448,6 +459,9 @@ fireforge config firefox.version
 # Set a config value
 fireforge config firefox.version 145.0.0esr
 
+# Pin the resolved Firefox source archive checksum
+fireforge config firefox.sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+
 # Set a value at a non-standard path (requires --force)
 fireforge config customKey "value" --force
 ```
@@ -456,6 +470,8 @@ Writes are serialised behind a sidecar lock — two concurrent `fireforge config
 
 Re-setting a key to its current value is a no-op: `fireforge.json` is not rewritten, key ordering is preserved, and the success log surfaces `<key> = <value> (unchanged)` instead of a fresh `Set …` line. This means automation that idempotently runs `fireforge config <key> <value>` no longer produces spurious diffs in `fireforge.json`.
 
+`firefox.sha256` is optional. When set, it must be a 64-character hex SHA-256 for the Firefox source archive resolved from `firefox.product` + `firefox.version`; cached archives and fresh downloads are verified against it before extraction. Omit it to keep the default cache-integrity-only behaviour.
+
 ### Patch queue management
 
 ```bash
@@ -531,6 +547,8 @@ Aggregate patch-size findings (`large-patch-files`, `large-patch-lines`) describ
 
 `fireforge build` is a transactional step: after a successful mach build it audits the dist bundle against engine-relative paths touched since the last successful build, and warns per file that is packageable-by-convention (`.js`/`.mjs`/`.css`/`.ftl`/`.xhtml`/`app/profile/…`) but has no matching artifact or whose dist mtime is older than the source. Ends every build with a `Packaged: N updated, M stale, K missing, S skipped` summary. The audit is warn-only — it never fails a build that mach reported green.
 
+`fireforge build --ui` is intentionally a fast rebuild path, not a bootstrap path. It now refuses before invoking `mach build faster` unless the current objdir has a completed launchable bundle; fresh imports and partial builds should run a full `fireforge build` first.
+
 The audit applies seven routing rules to suppress false positives that previously trained operators to ignore its warnings:
 
 - **jar.mn registrations are authoritative.** When the source under audit is claimed by a `(source)` reference in an ancestor `jar.mn`, the audit walks the registration to compute the expected target path (e.g. `content/browser/mybrowser.js`) and probes the dist tree for a candidate whose absolute path ends with that suffix. Picking the correct artifact from a same-basename collision no longer depends on path-similarity scoring. If the registration target is missing from dist, the warning names the `jar.mn` entry so "registration is intact, packaging dropped the file" is distinguishable from "source is unregistered". This is the fix for the class of false positive where `engine/browser/base/content/<name>.js` (registered in `browser/base/jar.mn`) collided with an unrelated `browser/defaults/preferences/<name>.js` added by a separate patch; the heuristic could not distinguish them, so the audit falsely reported the correctly-packaged chrome resource as missing.
@@ -605,7 +623,7 @@ fireforge test --doctor
 fireforge test --doctor browser/base/content/test/foo/browser_bar.js
 ```
 
-Spawns the built browser headless, waits for a marionette handshake on `127.0.0.1:2828`, and reports PASS/FAIL with the tail of the browser's stderr on FAIL. Distinguishes "marionette wedged" (socket silent) from "mach test discovery failed" — both otherwise surface as a silent 360-second hang followed by `Passed: 0, Failed: 0`. Useful as a prefix on routine `fireforge test` invocations when marionette has been flaky.
+Spawns the built browser headless, waits for a marionette handshake on `127.0.0.1:2828`, and reports PASS/FAIL with the tail of the browser's stderr on FAIL. The success output also names the objdir, binary, app path, port, and elapsed probe time so CI logs show exactly what was probed. Distinguishes "marionette wedged" (socket silent) from "mach test discovery failed" — both otherwise surface as a silent 360-second hang followed by `Passed: 0, Failed: 0`. Useful as a prefix on routine `fireforge test` invocations when marionette has been flaky.
 
 The probe is a cascade of six layered checks — engine-present → mach-available → python-available → profile-creatable → browser-spawns → marionette-handshake. Each failure is tagged `[layer N/6: <name>]` so the first broken layer is surfaced immediately instead of the whole cascade blocking on the final socket poll. When the browser binary crashes at startup (missing dylib, wrong CPU arch, corrupt profile) the cascade fails at layer 5 within the settle window, not after the full socket timeout.
 

package/dist/src/commands/build.js

@@ -5,7 +5,7 @@ import { auditBuildArtifacts } from '../core/build-audit.js';
 import { readBuildBaseline, writeBuildBaseline } from '../core/build-baseline.js';
 import { prepareBuildEnvironment } from '../core/build-prepare.js';
 import { getProjectPaths, loadConfig } from '../core/config.js';
-import { attemptMozinfoRewrite, build, buildArtifactMismatchMessage, buildUI, hasBuildArtifacts, runMach, withBuildLock, } from '../core/mach.js';
+import { attemptMozinfoRewrite, build, buildArtifactMismatchMessage, buildUI, hasBuildArtifacts, hasRunnableBundle, runMach, withBuildLock, } from '../core/mach.js';
 import { GeneralError } from '../errors/base.js';
 import { AmbiguousBuildArtifactsError, BuildError } from '../errors/build.js';
 import { toError } from '../utils/errors.js';
@@ -106,6 +106,24 @@ export async function buildCommand(projectRoot, options) {
             throw new GeneralError(mismatchMessage);
         }
     }
+    if (options.ui) {
+        if (!buildCheck.exists || !buildCheck.objDir) {
+            const detail = buildCheck.objDir
+                ? `Build artifacts incomplete in ${buildCheck.objDir}/`
+                : 'No completed obj-* build artifacts found.';
+            throw new GeneralError(`UI-only builds require a completed full build first. ${detail}\n\n` +
+                'Run "fireforge build" and let it finish, then retry "fireforge build --ui".');
+        }
+        const bundleCheck = await hasRunnableBundle(paths.engine, config.binaryName, buildCheck.objDir);
+        if (!bundleCheck.runnable) {
+            const expectedSuffix = bundleCheck.expectedPath
+                ? ` Expected launchable binary at engine/${bundleCheck.expectedPath}.`
+                : '';
+            throw new GeneralError(`UI-only builds require a completed full build first.${expectedSuffix}\n\n` +
+                'Freshly imported or partially built trees cannot use `mach build faster` yet. ' +
+                'Run "fireforge build" and let it finish, then retry "fireforge build --ui".');
+        }
+    }
     // Log brand info if specified
     if (options.brand) {
         verbose(`Building with brand: ${options.brand}`);

package/dist/src/commands/config.js

@@ -31,6 +31,7 @@ const STRING_TYPED_KEYS = new Set([
     'binaryName',
     'firefox.version',
     'firefox.product',
+    'firefox.sha256',
     'license',
     'wire.subscriptDir',
 ]);