@hominis/fireforge 0.44.2 → 0.44.4

package/docs/patch-workflow.md

@@ -0,0 +1,119 @@
+<!-- SPDX-License-Identifier: EUPL-1.2 -->
+
+# Patch workflow
+
+The day-to-day loop: change code inside `engine/`, export it, keep the queue
+applying cleanly. `fireforge --help` remains the authority on flags; this file
+covers the behaviour that a flag list cannot state.
+
+## The loop
+
+```bash
+npx fireforge status
+npx fireforge export browser/base/content/browser.js --name custom-toolbar --category ui
+npx fireforge re-export custom-toolbar
+npx fireforge lint --per-patch
+npx fireforge verify
+```
+
+## Exporting
+
+Patch names are normalized on export and rename: `window-chrome-tests`,
+`ui-window-chrome-tests`, and `235-ui-window-chrome-tests` all resolve to the
+same `<order>-ui-window-chrome-tests.patch` shape.
+
+For large generated asset batches, `re-export --scan --scan-files <manifest>`
+assigns files to their owner patches without broad directory scanning. The
+manifest is JSON:
+
+```json
+{
+  "assignments": [
+    { "patch": "002-branding-runtime-icons.patch", "files": ["browser/branding/hominis/icon.svg"] }
+  ]
+}
+```
+
+The command is dry-runnable, refuses ambiguous ownership, and reports each
+file-to-patch assignment before refreshing the patch.
+
+Scan-less `re-export` previews drift; `--refuse-foreign-drift --expect <path>`
+restricts it to intended files.
+
+### Deletions
+
+A deleted upstream file is captured like any other change: a path that is
+tracked in HEAD but absent from disk stays in the diff scope, and the emitted
+patch carries a `deleted file mode` section. A captured deletion keeps its
+`filesAffected` entry — the patch is what performs the deletion, so pruning
+the entry would leave the body deleting a file the manifest says the patch has
+nothing to do with — and it still requires the confirmation, because the patch
+will remove those files wherever it is applied.
+
+An **untracked** absent path is refused rather than warned about: a patch
+cannot express it either way. The `--files` path also refuses when any
+requested path produces no hunk, checked against the full requested list
+rather than the post-filter subset, so nothing dropped upstream can pass as a
+silent success.
+
+## Queue maintenance
+
+Under `fireforge patch`:
+
+- `patch compact` closes ordinal gaps (range-aware when `patchPolicy.ranges`
+  is configured).
+- `patch reorder` moves a patch.
+- `patch split <source> --files <paths...> --name <name>` carves files out of
+  a patch into a new one as a single transaction — including staged-dependency
+  owner rewrites — with `--dry-run` support.
+- `patch rename` also supports `--category` and `--order`.
+
+Every queue-mutating patch command accepts `--wait-lock [seconds]`; see
+[`configuration.md`](configuration.md#lock-waits) for how the budget resolves.
+
+## Linting the queue
+
+`lint --per-patch --patches` accepts repeated flags, comma lists, full
+filenames/stems, manifest names, category-prefixed slugs, and bare slugs.
+
+`lint --per-patch --max-warnings 0` is the warning-clean form for a release
+gate.
+
+`lint --per-patch --report <path>` writes a machine-readable JSON report with
+each patch's line count, tier, active size thresholds, issues, and
+`lintIgnore`-suppressed issues. The size metrics
+(`countNonBinaryDiffLines`, `resolvePatchSizeTier`, `getPatchSizeThresholds`)
+are also exported on the programmatic API.
+
+Per-patch lint type-checks plain Firefox JS against a built-in Firefox-globals
+shim that tracks upstream WebIDL additions per release. See
+[`configuration.md`](configuration.md#patchlint) for the project hooks
+(`checkJsExtraShim`, `checkJsTestFiles`, `checkJsTestShim`).
+
+For release whitespace checks, use `npm run whitespace:check`; it still checks
+source diffs while excluding generated `patches/*.patch` diff syntax.
+
+## CI enforcement
+
+`fireforge status --check` exits non-zero when any unmanaged, drifted, or
+conflicted file exists. `--fail-on <class,...>` tunes the policy set, and
+`--json` composes — the JSON stays parseable and its `files[]` entries name
+the owning `patch`. `--include-ownership` adds ownership rows to JSON status
+output.
+
+The JSON shapes are specified in [`machine-output.md`](machine-output.md), and
+the codes a script should branch on in [`exit-codes.md`](exit-codes.md).
+
+## Rebasing onto a new Firefox source release
+
+```bash
+npx fireforge source set --version 145.0.0esr --product firefox-esr --sha256 <archive-sha256>
+npx fireforge download --force
+npx fireforge rebase
+```
+
+If a patch fails, fix the reject inside `engine/`, then run
+`npx fireforge rebase --continue`. A rebase deliberately leaves the engine
+mutated, so its apply-and-persist pairs are held across a signal rather than
+rolled back — see invariant 4 in
+[`lifecycle-invariants.md`](lifecycle-invariants.md).

package/docs/run-logs.md

@@ -0,0 +1,60 @@
+<!-- SPDX-License-Identifier: EUPL-1.2 -->
+
+# Run logs (internal)
+
+`fireforge test` and `fireforge build` write their own complete output to
+`.fireforge/logs/<command>-<timestamp>.log` as they stream. The module is
+`src/core/run-log.ts`.
+
+## Why the tool holds the log, not the operator
+
+A run's diagnosis used to exist in exactly one place: the terminal. The
+ergonomic default when output is long — `fireforge test … 2>&1 | tail -40` —
+keeps the summary and discards the `TEST-UNEXPECTED-FAIL` lines that say WHAT
+broke, and it launders the exit code besides. Nothing else held the output:
+`runMachCapture` keeps a 2 MB in-memory tail that dies with the process, and
+the shard retry loop discards earlier attempts outright.
+
+The operator rule ("never pipe a run") was written down three times downstream
+and broken after each writing. That is the tell that it is not an operator
+problem — the tool is the only party that can make the log survive the
+mistake.
+
+## The contract
+
+- **Raw bytes, never the echo filter's collapsed form.** The artifact exists
+  to be re-read by whoever is diagnosing the run; shortening it would
+  reintroduce the loss it prevents.
+- **Opened before any preflight**, so a refusal is logged too. Those are
+  exactly the runs whose only output a `tail` throws away.
+- **Retained 20 deep per command kind.** Per-kind, so a busy `test` loop
+  cannot evict the `build` log an operator needs to read beside it. Pruning
+  runs before the new file is opened, and orders by the filename's
+  timestamp rather than by stat-ing every entry.
+- **Timestamps are filesystem-safe**: the ISO form with `:` and `.` replaced
+  by `-`, which Windows accepts and which still sorts chronologically.
+- **Best-effort throughout.** A log that cannot be opened, written or pruned
+  degrades to no log. A run must never fail over a diagnostic, so `openRunLog`
+  returns `undefined` on failure, a post-open stream error latches `broken`
+  and drops every later write, and pruning failures are logged verbosely
+  only.
+
+## Where the path is announced
+
+`test` publishes it as an additive `log=<path>` key on the
+`FIREFORGE-VERDICT:` line. That is forced, not chosen: the verdict must stay
+the run's LAST stdout write ([`machine-output.md`](machine-output.md)), so a
+separate announcement after it would break the contract, and one before it is
+the first thing a `tail` cuts.
+
+`build` prints no verdict line, so it announces the path directly
+(`Full build output: <path>`).
+
+## Where the tee sits
+
+At the two mach capture funnels in `src/core/mach.ts`, not at the commands.
+That covers every dispatch: the three test dispatchers, `package`, and the
+stdio-inheriting path `mach build` uses — which rides the collectors' existing
+`mirror` hook rather than growing a second one. A new dispatcher added at the
+funnels is logged by construction; one added beside them is not, which is the
+reason to keep going through them.

package/docs/testing.md

@@ -0,0 +1,123 @@
+<!-- SPDX-License-Identifier: EUPL-1.2 -->
+
+# Running tests
+
+`fireforge test` wraps `mach test`. Most of what follows exists because
+mach's own defaults are wrong for a fork that must know exactly what ran.
+
+## Scope is exact
+
+`fireforge test <directory>` runs exactly that directory: FireForge enumerates
+the directory's test files and passes the explicit file list to mach in one
+invocation, so mach's prefix-based path matching cannot silently sweep in
+sibling directories sharing the name prefix. Excluded siblings are echoed with
+their test-file counts.
+
+Multiple path arguments run as **sequential shards** by default — one browser
+instance per argument, with a directory argument keeping its files together —
+announced with a notice, since isolated instances do not exercise
+cross-argument state. `--no-shard` restores one combined invocation.
+
+## Pathless runs must choose a mode
+
+- `--auto` forwards mach's own auto-selection.
+- `--doctor` runs the Marionette preflight only.
+- `--canary [path]` runs one short browser-chrome canary
+  (`test.canaryPath` / `test.canaryTimeoutSeconds` in `fireforge.json`
+  provide defaults).
+
+## Build freshness
+
+When packageable engine files changed since the last successful FireForge
+build, `test` fails before launching stale artifacts. Use `--build` to
+refresh, or `--allow-stale-build` only for intentional out-of-band rebuilds.
+
+`--build-only` packages mixed-harness paths once, then each harness half can
+run without `--build`. `--extend-coverage` safely retains prior scoped
+coverage instead of replacing it; it is refused when the build anchor moved
+(engine HEAD, `engine/mozconfig`, or a previously fingerprinted packageable
+file).
+
+### What a pre-test build costs
+
+`test --build` runs `mach build faster`, with two escalations decided by
+comparing build inputs against the **last successful build** recorded in
+`.fireforge/last-build.json` — not against engine HEAD, since a fork's
+worktree is permanently dirty (imported patches, Furnace-applied
+components):
+
+- a changed `moz.build` / `moz.configure` / `Makefile.in` runs
+  `mach configure` first;
+- a changed `jar.mn` escalates the whole build to a full `mach build`
+  (minutes rather than seconds), so that new install-manifest destinations
+  exist before the run.
+
+Each escalation is paid once per actual content change: a `jar.mn` that is
+dirty against HEAD but byte-identical to what the last full build consumed
+does not escalate again. `fireforge build --ui` is itself a
+`mach build faster` and never escalates; the `jar.mn` record it writes is
+carried forward from the previous full build, so a registration made
+between a `build --ui` and the next `test --build` still triggers the full
+build once.
+
+Whether a full build is really required for an entry added to an existing
+`jar.mn` under `dist/bin` is an open question, recorded in the 0.44.0
+changelog: mozbuild regenerates its backend from a newer `jar.mn` on any
+`mach build`, `faster` included, and the FasterMake backend consumes
+`dist/bin` jar manifests. The escalation stays until that is demonstrated
+end to end against a real objdir (`build --ui` after a jar-only
+registration, then a plain `test`), because relaxing a stale-artifact guard
+on source reading alone trades a slow build for a silently wrong test.
+
+## Resilience
+
+- Recognized harness crashes retry up to `--harness-retries <n>` times
+  (default 2).
+- `--kill-stale-marionette` terminates recognized stale browsers.
+- `--perf-samples <path>` publishes a perf-sample artifact path to the
+  harness (exported as `<BINARYNAME>_PERF_SAMPLE_JSON`).
+
+## The verdict line
+
+Every test run ends with one machine-readable line:
+
+```
+FIREFORGE-VERDICT: PASS|FAIL reason=… [log=<path>]
+```
+
+Automation should branch on it rather than on the raw process code, and must
+treat a **missing** verdict as failure. The closed set of `reason=` values and
+the stdout rules around the line are in
+[`machine-output.md`](machine-output.md); `log=` names the run's own complete
+output, specified in [`run-logs.md`](run-logs.md).
+
+Exit code 14 (`INCONCLUSIVE`) is not red: it means `engine/` moved while the
+harness ran and the result was thrown away. Exit 15 (`LOCK_TIMEOUT`) means the
+run never started. See [`exit-codes.md`](exit-codes.md).
+
+## Output verbosity
+
+mozbuild quiets terminal output to warnings and errors when it detects a
+coding agent (`is_running_under_coding_agent()` keys on `CLAUDECODE`). The
+build half of that quieting is useful; the test half removes `TEST_START` and
+console INFO — the lines a hang or stall diagnosis needs, and the ones
+FireForge's own classifier reads (`Ran N checks`, `Unexpected results:`,
+`TEST-UNEXPECTED-*`), so suppression pushes a run toward `reason=no-tests`.
+
+`fireforge test --full-output` unsets the marker for **test dispatches only**;
+the build path stays quiet. It is opt-in rather than automatic because it
+changes how much a third party prints, and an operator who wants the quieting
+should keep it.
+
+## Concurrency
+
+A test run holds the engine-session lock and snapshots the engine git
+generation before and after the harness run, failing the verdict as invalid if
+the tree changed mid-run. To verify beside a busy primary checkout instead,
+use a [verification tree](verification-trees.md).
+
+## Dev-build gotcha
+
+In dev builds, files under `obj-*/dist/bin` may be symlinks back into the
+source tree (notably prefs), so edit source prefs directly and keep a backup
+before bisection experiments.

package/docs/verification-trees.md

@@ -0,0 +1,48 @@
+<!-- SPDX-License-Identifier: EUPL-1.2 -->
+
+# Verification trees
+
+For concurrent read-mostly verification beside a busy primary checkout,
+`fireforge tree create <name>` snapshots the project — applied engine tree
+included, `obj-*` excluded — into `.fireforge/trees/<name>` using filesystem
+copy-on-write (APFS `clonefile`, btrfs/XFS reflink). Roughly 3 s for a 156 MB
+applied browser tree.
+
+```bash
+npx fireforge tree create verify
+npx fireforge tree exec verify -- npx fireforge lint --per-patch
+npx fireforge tree list
+npx fireforge tree remove verify
+```
+
+## What works inside a tree
+
+Read-only commands and `export`/`re-export` dry runs. **Mutations are
+refused** — a default-deny verdict table, not a blocklist, so a newly added
+mutating command is refused until it is classified. That is invariant 7 in
+[`lifecycle-invariants.md`](lifecycle-invariants.md).
+
+`--with-objdir` adds a safely relocated build, which makes build-less `test`
+possible inside the tree.
+
+## Lifecycle
+
+- `tree list` reports staleness (`--json`; see
+  [`machine-output.md`](machine-output.md)).
+- `tree remove` deletes, refusing an active lock and a lock of unknown
+  ownership.
+- `tree exec <name> -- <cmd>` runs a command inside a tree, sealing its stdout
+  so exactly one verdict line survives, last.
+
+## Filesystems without CoW
+
+Refused unless `--force-copy` explicitly accepts a full physical copy. The
+refusal is the point: a silent full copy of an applied browser tree is not
+the operation anyone asked for.
+
+## The tree marker
+
+`.fireforge/tree.json` is tri-state: absent (primary), valid (snapshot), or
+**corrupt**. A corrupt marker refuses every command, because an unreadable
+marker leaves it unknown whether this is a snapshot or the primary tree.
+`--ignore-corrupt-tree-marker` overrides that deliberately.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hominis/fireforge",
-  "version": "0.44.2",
+  "version": "0.44.4",
   "description": "FireForge — a build tool for customizing Firefox",
   "type": "module",
   "main": "./dist/src/index.js",
@@ -48,6 +48,7 @@
     "!dist/**/test-utils/",
     "!dist/**/*.js.map",
     "templates/",
+    "docs/",
     "README.md",
     "CHANGELOG.md",
     "LICENSE.md"