@hominis/fireforge 0.55.0 → 0.60.0

package/docs/gate.md

@@ -0,0 +1,205 @@
+<!-- SPDX-License-Identifier: EUPL-1.2 -->
+
+# The gate
+
+`fireforge gate` is the judgement before a push: `check`, a build, and every test lane, stress run, negative control and smoke start `fireforge.json` declares, run on a clean committed tree. When everything passes and every declared acceptance is recorded, the git ref `refs/fireforge/verified` moves to the commit. A pre-push hook can then refuse any push whose tree is not the verified tree.
+
+```sh
+git commit -am "Sidebar: keyboard navigation"
+npx fireforge gate
+```
+
+The gate works the same in both modes. It leases one workspace for the whole run ([workspaces.md](workspaces.md)).
+
+## Configuration
+
+Everything the gate runs besides `check` is declared under `gate` in `fireforge.json`:
+
+```json
+{
+  "gate": {
+    "build": "ui",
+    "documents": ["docs/**", "*.md"],
+    "lanes": [
+      {
+        "id": "sidebar",
+        "files": ["browser/base/content/test/sidebar/browser.toml"],
+        "dispatch": "shards"
+      },
+      {
+        "id": "captures",
+        "files": ["browser/base/content/test/captures/**/browser_*.js"],
+        "headed": true,
+        "uploadDir": true,
+        "env": { "CAPTURE_DIR": "{artifacts}/captures" },
+        "machArgs": ["--setpref=layout.css.devPixelsPerPx=2"],
+        "validate": { "command": ["node", "scripts/check-captures.mjs", "{artifacts}/captures"] }
+      }
+    ],
+    "stress": [
+      {
+        "id": "sidebar-x5",
+        "files": ["browser/base/content/test/sidebar/browser.toml"],
+        "repeat": 5
+      }
+    ],
+    "controls": [
+      {
+        "id": "sidebar-focus",
+        "diff": "controls/sidebar-focus.diff",
+        "test": "browser/base/content/test/sidebar/browser_focus.js",
+        "assertions": ["focus moves to the first item"]
+      }
+    ],
+    "runs": [
+      {
+        "id": "startup",
+        "smokeExit": 20,
+        "headless": true,
+        "consoleAllowFile": "configs/console-allow.txt"
+      }
+    ],
+    "acceptance": [
+      {
+        "id": "visual",
+        "reason": "The sidebar matches the design at 2x",
+        "evidence": ["lane:captures"]
+      }
+    ]
+  }
+}
+```
+
+### `build`
+
+`ui` (the default) or `full`. A workspace that needs a full build gets one either way.
+
+### `documents`
+
+Globs of repository paths that change nothing the gate tests. A pattern without a slash matches a file name at any depth. A commit whose every change since the verified commit matches these needs `check` alone.
+
+### `lanes`
+
+A lane is a group of tests run as declared. It is data, not a shell line.
+
+| Key         | Meaning                                                                                                                                     |
+| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
+| `id`        | Lowercase ID (`a-z`, `0-9`, `.`, `_`, `-`)                                                                                                  |
+| `files`     | Engine-relative globs or literal paths of test files, directories or manifests                                                              |
+| `exclude`   | Tests left out of `files`                                                                                                                   |
+| `dispatch`  | `shards` (a browser per mochitest file, the default), `one-browser`, or `standalone` (one invocation, restarting the browser between files) |
+| `headed`    | Run with a visible window, holding the display lock. Default `false`                                                                        |
+| `env`       | Harness environment. A value starting `{artifacts}` names a path in the lane's output directory                                             |
+| `uploadDir` | Give the harness a `MOZ_UPLOAD_DIR` in the lane's output directory. Default `false`                                                         |
+| `machArgs`  | `--setpref=<name>=<value>` arguments whose value names no path; `marionette.*` prefs are refused                                            |
+| `validate`  | `{ "command": [...] }`, run after the harness exits clean; a nonzero exit reads red                                                         |
+
+`env` refuses names FireForge, the build or the toolchain owns: anything starting `FIREFORGE_`, `MOZ_`, `MACH`, `NODE_`, `GIT_`, `PYTHON`, `CARGO`, `RUST`, `LD_`, `DYLD_`, `XDG_`, `LC_` and similar, and names such as `PATH`, `HOME`, `TMPDIR`, `DISPLAY`, `CC`, `CFLAGS`. A value is one line of at most 4096 characters, and a path in it must stay inside the workspace or start with `{artifacts}`.
+
+`validate.command` is an argument vector run from the project root: `node <script> [arguments]`, or a committed script by its path. `{artifacts}` expands to the lane's output directory, and no argument may name a path outside the workspace. The validator's environment names the project directory (`FIREFORGE_PROJECT_ROOT`) and the engine of the workspace the gate runs in (`FIREFORGE_ENGINE`). With `uploadDir`, what the harness uploaded is in `{artifacts}/upload`.
+
+### `stress`
+
+A lane with `repeat`: it runs that many times, and every repetition must pass.
+
+### `controls`
+
+A negative control proves a test catches what it claims to ([testing.md](testing.md#negative-controls)): `diff` is a committed diff (project-relative) holding a mutation, `test` the engine-relative test that must catch it, and `assertions` the labels of the reds it must produce, every one and no other. A control that goes red anywhere else fails its entry as `wrong-red`, naming the expected assertions and the ones seen.
+
+### `runs`
+
+A smoke start of the built browser, as `fireforge run --smoke-exit` does it.
+
+| Key                | Meaning                                                                    |
+| ------------------ | -------------------------------------------------------------------------- |
+| `id`               | Run ID                                                                     |
+| `smokeExit`        | Seconds the console is watched before the browser is ended                 |
+| `headless`         | Start without a window. Default `false`, which holds the display lock      |
+| `args`             | Arguments for the browser                                                  |
+| `consoleAllowFile` | Committed file of console line patterns the smoke start accepts            |
+| `validate`         | As on a lane; judges what the run wrote, such as `{artifacts}/console.txt` |
+
+### `acceptance`
+
+Decisions only a person can make, such as a visual check. `reason` says what is decided; `evidence` lists the declared entries (`lane:<id>`, `stress:<id>`, `control:<id>`, `run:<id>`) whose outputs the person looks at.
+
+## What a gate run does
+
+`fireforge gate` decides, in this order:
+
+1. **Clean tree.** There must be a commit (`gate-no-commit`, exit 4), and the working tree must equal it (`gate-dirty-tree`, exit 1). Untracked files that are not ignored count.
+2. **Already verified.** When `refs/fireforge/verified` is at HEAD, the result is `verified` at once.
+3. **Fast-forward only.** When the verified commit is not an ancestor of HEAD, the gate refuses (`gate-not-fast-forward`, exit 4). `--allow-rewrite` moves the ref to a commit that does not descend from it.
+4. **Recorded pass.** When this checkout recorded a passing full run for this commit, with the same tree and the same FireForge version, that result stands and nothing runs again.
+5. **Documents only.** When every path changed since the verified commit matches `gate.documents`, only `check` runs.
+6. **Full run.** Otherwise the gate leases a workspace and runs, in order: `check`, component deployment and the build, then every lane, stress run, control and smoke start. A red entry does not stop the ones after it, so one run reports everything. After a red build the rest is skipped. When the run ends, HEAD and the working tree must be unchanged (`gate-tree-moved`, exit 14). The result is recorded for the commit.
+
+Then:
+
+- Anything red: the result is `failed`. Exit 5 when the build or a declared entry is red, exit 1 when only `check` is (`gate-check-failed`).
+- Everything passed, but declared acceptances are not recorded for this tree: the result is `pending`, exit 0, and the ref does not move. The next `gate` reuses the recorded pass. A documents-only commit needs no acceptance.
+- Everything passed and every acceptance is recorded: the ref moves to the commit by compare-and-swap against the value it had when the gate started. If another gate moved it meanwhile, the gate leaves it there (`gate-ref-moved`, exit 4). The result is `verified`.
+
+A gate that exits 0 has not necessarily moved the ref: read `result=` on the verdict line ([machine-output.md](machine-output.md#gate)).
+
+### Feedback runs
+
+```sh
+npx fireforge gate --lane sidebar --lane run:startup
+```
+
+`--lane <id>` (an ID, or `kind:id`; repeat for several) is feedback only: it runs those entries, plus the build, on the working tree as it is, committed or not. It runs no `check`, records nothing, and never moves `refs/fireforge/verified`. The result is `feedback`.
+
+## Accepting
+
+```sh
+npx fireforge gate accept visual --note "Compared captures at 2x with the design, all match"
+```
+
+`gate accept <id>` records who decided, when, and the note, against the tree of the checked-out commit. The working tree must be clean, the ID must be declared, and every entry named in its `evidence` must have passed in a gate run on this commit in this checkout (`gate-evidence-missing`, exit 1): run the gate, look at what it produced under `.fireforge/gate/<commit>/`, then accept. `--by <name>` names who decides; the default is git's `user.name`, else the login name.
+
+Acceptances are stored as git refs, `refs/fireforge/accepted/<tree>`, shared by every worktree of the repository and bound to the tree, so a commit with the same tree keeps them. After the last acceptance, run `fireforge gate` again to move the ref.
+
+## Status
+
+```sh
+npx fireforge gate status
+```
+
+Shows where `refs/fireforge/verified` is and how HEAD relates to it (`verified`, `ahead` by some commits, `diverged`, or `unverified`), whether the tree is clean, what the gate recorded for HEAD in this checkout, which acceptances are recorded, and which are pending.
+
+## Where results live
+
+- `.fireforge/gate/<commit>/` in the checkout: `result.json` and one directory per entry: `lane-<id>/output.log`, `lane-<id>/results.json`, `lane-<id>/upload/`, `run-<id>/console.txt`. The newest five runs are kept. A feedback run writes `.fireforge/gate/feedback/`.
+- `refs/fireforge/verified`: the last commit that passed.
+- `refs/fireforge/accepted/<tree>`: the recorded acceptances.
+
+Push the refs if other machines should see them:
+
+```sh
+git push origin refs/fireforge/verified 'refs/fireforge/accepted/*:refs/fireforge/accepted/*'
+```
+
+## A pre-push hook
+
+This hook refuses to push a branch whose tree is not the verified tree. Save it as `.git/hooks/pre-push` (or `.husky/pre-push`) and make it executable.
+
+```sh
+#!/bin/sh
+# Refuse a push whose tree FireForge has not verified.
+verified=$(git rev-parse -q --verify 'refs/fireforge/verified^{tree}') || {
+  echo "pre-push: nothing is verified yet; run: npx fireforge gate" >&2
+  exit 1
+}
+while read -r local_ref local_sha remote_ref remote_sha; do
+  case "$local_sha" in *[!0]*) ;; *) continue ;; esac # a deletion pushes nothing
+  case "$remote_ref" in refs/heads/*) ;; *) continue ;; esac
+  if [ "$(git rev-parse "$local_sha^{tree}")" != "$verified" ]; then
+    echo "pre-push: $local_ref is not the verified tree; run: npx fireforge gate" >&2
+    exit 1
+  fi
+done
+exit 0
+```
+
+It compares trees, not commits, so rewording the verified commit's message does not invalidate it.

package/docs/getting-started.md

@@ -0,0 +1,165 @@
+<!-- SPDX-License-Identifier: EUPL-1.2 -->
+
+# Getting started
+
+This page takes you from an empty directory to a built, running fork, then shows the loop you repeat every day. It covers both ways a project can store its changes:
+
+- **Patch mode**: you edit Firefox in place under `engine/`, and FireForge stores your changes as a numbered queue of `.patch` files in `patches/`.
+- **Source mode**: your changes are ordinary files under `src/`, tracked by git. `src/<path>` supplies the Firefox file `<path>`. FireForge builds them in a workspace it keeps on the host, and can write a patch queue from them whenever you want one.
+
+Both modes use the same daily commands with the same flags. Pick source mode when several people or several checkouts work at once; pick patch mode for a small fork kept by one person. [migrating.md](migrating.md) covers the previous release; `fireforge migrate` converts a patch-mode project to source mode later.
+
+## Requirements
+
+- Node.js 22.22.2 or later
+- Git
+- Python 3
+- The Firefox platform build tools: the Xcode command line tools on macOS, the `build-essential` packages on Linux, Visual Studio Build Tools on Windows. `fireforge bootstrap` installs the rest.
+- Watchman, only for `fireforge watch`
+- Disk: a built Firefox takes tens of gigabytes. In source mode each further workspace costs that again (see [workspaces.md](workspaces.md)).
+
+## Create the project
+
+Install FireForge into a fresh directory that is a git repository:
+
+```sh
+mkdir mybrowser && cd mybrowser
+git init
+npm init -y
+npm install --save-dev @hominis/fireforge
+```
+
+Then create the project. Every value can also be answered at a prompt when you leave the option out.
+
+Patch mode:
+
+```sh
+npx fireforge setup --name MyBrowser --vendor "My Company" --firefox-version 140.9.0esr --mode patches
+```
+
+Source mode:
+
+```sh
+npx fireforge setup --name MyBrowser --vendor "My Company" --firefox-version 140.9.0esr --mode source
+```
+
+`setup` writes:
+
+- `fireforge.json`: the project configuration ([configuration.md](configuration.md)).
+- `configs/*.mozconfig`: the build configuration templates. FireForge fills in your browser's name and identity and places the object directory itself; a template that sets `MOZ_OBJDIR` is refused.
+- `LICENSE`, and `.gitignore` entries for `engine/` (patch mode) and `.fireforge/`, the project's local state.
+- Source mode only: `src/.gitattributes`, which routes owned files through FireForge's merge driver. Creating a source-mode project also downloads the pinned Firefox, because `fireforge.json` records the exact commit it imports as (`source.pristineCommit`).
+
+Commit the result:
+
+```sh
+git add -A && git commit -m "Create MyBrowser"
+```
+
+## Download, bootstrap, build, run
+
+The same five commands work in both modes.
+
+```sh
+npx fireforge download
+npx fireforge bootstrap
+npx fireforge build
+npx fireforge run
+```
+
+- `download` fetches the Firefox source archive named in `fireforge.json`, checks its SHA-256, and imports it as one commit into a pristine repository under `~/.fireforge/pristine/`. In patch mode it then checks that commit out as `engine/`. A host that already has the pin downloads nothing.
+- `bootstrap` runs `mach bootstrap`, which installs the compilers and libraries Firefox needs. It may prompt and ask for elevated rights.
+- `build` runs a full `mach build`. The first one takes an hour or more.
+- `run` starts the browser you just built. Arguments after `--` go to the browser: `npx fireforge run -- --jsconsole`.
+
+In patch mode, a project that already has a patch queue applies it before the first build:
+
+```sh
+npx fireforge import
+```
+
+In source mode there is no `engine/` in the project. `bootstrap`, `build`, `run` and `test` lease a workspace from the host pool, the first one called `w1`, and bring it to your checkout's `src/` before they start. [workspaces.md](workspaces.md) explains the pool.
+
+## Make a first change
+
+Patch mode: edit the file inside `engine/`.
+
+```sh
+$EDITOR engine/browser/base/content/browser.js
+```
+
+Source mode: a Firefox file is touched only after `override` records which upstream version you started from. It copies the pristine file to `src/` for you to edit:
+
+```sh
+npx fireforge override browser/base/content/browser.js
+$EDITOR src/browser/base/content/browser.js
+```
+
+A file Firefox does not have needs no override; create it under `src/`:
+
+```sh
+mkdir -p src/browser/base/content/mybrowser
+$EDITOR src/browser/base/content/mybrowser/panel.mjs
+```
+
+## The daily loop
+
+The loop is the same in both modes. Only the step that records your work differs.
+
+| Step                   | Patch mode                                                        | Source mode                                                              |
+| ---------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------------ |
+| Edit                   | files under `engine/`                                             | files under `src/` (`fireforge override <path>` first for upstream ones) |
+| See what you own       | `fireforge status`                                                | `fireforge status`                                                       |
+| Quality rules          | `fireforge lint`                                                  | `fireforge lint`                                                         |
+| Types                  | `fireforge typecheck`                                             | `fireforge typecheck`                                                    |
+| Rebuild the front end  | `fireforge build --ui`                                            | `fireforge build --ui`                                                   |
+| Run the tests          | `fireforge test <path>`                                           | `fireforge test <path>`                                                  |
+| Record the work        | `fireforge export <paths> --name <name>` or `fireforge re-export` | `git commit`                                                             |
+| Everything static once | `fireforge check`                                                 | `fireforge check`                                                        |
+
+As commands, patch mode:
+
+```sh
+npx fireforge status
+npx fireforge lint
+npx fireforge typecheck
+npx fireforge build --ui
+npx fireforge test browser/base/content/test/mybrowser/browser_panel.js
+npx fireforge export browser/base/content/mybrowser --name ui-panel -d "The MyBrowser panel"
+npx fireforge check
+git add patches && git commit -m "Add the panel"
+```
+
+Source mode:
+
+```sh
+npx fireforge status
+npx fireforge lint
+npx fireforge typecheck
+npx fireforge build --ui
+npx fireforge test browser/base/content/test/mybrowser/browser_panel.js
+npx fireforge check
+git add src .fireforge-sources && git commit -m "Add the panel"
+```
+
+What each step does:
+
+- `status` lists the owned changes (every path the project adds, modifies or deletes relative to the pinned Firefox) and anything unaccounted for: in patch mode, edits no patch holds yet; in source mode, upstream files edited without an override. `status --check` exits 1 when anything is listed.
+- `lint` runs the quality rules over the owned changes ([quality.md](quality.md)).
+- `typecheck` checks the `jsconfig` projects in `typecheck.projects` and the strict typing of owned files.
+- `build --ui` rebuilds only the front end (`mach build faster`). When a compiled source or a build definition changed since the last build, it runs a full build instead.
+- `test` runs mochitest, xpcshell and reftest files through mach and ends with one `FIREFORGE-VERDICT:` line ([testing.md](testing.md)). It refuses a build older than the source; `--build` builds first.
+- In patch mode, `export` captures edits into a new patch and `re-export` rewrites existing patches from what `engine/` holds ([patch-workflow.md](patch-workflow.md)). In source mode git is the record; `fireforge export` writes a patch queue from `src/` when you want one ([source-workflow.md](source-workflow.md)).
+- `check` runs status, verify, lint, typecheck, component validation, the token checks and your own extra steps as one judgement ([quality.md](quality.md)).
+
+Before you push, `fireforge gate` runs `check`, a build and the test lanes `fireforge.json` declares on the committed tree, and marks the commit verified ([gate.md](gate.md)).
+
+## When something is wrong
+
+```sh
+npx fireforge doctor
+```
+
+`doctor` checks the configuration, git, the pinned source, the engine or the workspace pool, the stored changes, interrupted runs, locks, disk space, generated typing files and the tools `externalToolchains` declares, and says what to run for each problem. `doctor --recover` undoes changes an interrupted command left half-made.
+
+Every failure prints `error[<code>]: <message>` with its stable code, and a `hint:` line saying what to do next. Scripts read the exit code ([exit-codes.md](exit-codes.md)) and `--json` ([machine-output.md](machine-output.md)).