pr-merge-audit 1.2.0__tar.gz

pr_merge_audit-1.2.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

pr_merge_audit-1.2.0/PKG-INFO

@@ -0,0 +1,321 @@
+Metadata-Version: 2.4
+Name: pr-merge-audit
+Version: 1.2.0
+Summary: Merge-test a repository's open pull requests together and report which combine, conflict or break each other's tests.
+License: Apache-2.0
+Classifier: Programming Language :: Python :: 3
+Classifier: Environment :: Console
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# pr-merge-audit
+
+Merge-tests the open pull requests of a repository against its base branch and writes a
+Markdown report of which PRs combine cleanly, which conflict, and which break each other's tests.
+
+No AI or network is involved after the snapshot step: the same snapshot, configuration and
+machine always produce the same report.
+
+The tool is not tied to any project, language or test runner. Everything about the repository
+under test (where it lives, how its tests run, how their output reads) is in a configuration
+file kept with that project, not here. `config.example.json` is a starting point.
+
+## Requirements
+
+- Python 3.8 or later, standard library only
+- git 2.38 or later (`git merge-tree --write-tree`)
+- Linux, macOS or Windows. Each test run gets a private home and temp directory, and the tool
+  stops every process the run started when it ends (a process group on Linux and macOS, a job
+  object on Windows). The target project's own test command must of course run on that platform.
+- For `snapshot` only: GitHub (github.com, or GitHub Enterprise through `api`). Optional:
+  `GITHUB_TOKEN` in the environment (unauthenticated API access is rate-limited).
+
+`snapshot` is the only GitHub-specific step. For another host (GitLab, Gitea, Bitbucket), write
+the two snapshot files yourself and run `build` on them:
+
+- `snapshots/DATE.json`: `{"date", "repo", "base_ref", "base_sha", "prs": [{"number", "title",
+  "head", "draft", "status", "files"}]}`, where `status` is `SUCCESS` when the PR passes your
+  required check and `files` lists the paths it changes against the base.
+- `snapshots/DATE.bundle`: a `git bundle` holding `refs/audit/base` and `refs/audit/pr/N` for
+  each PR.
+
+`build` refuses a snapshot whose `base_sha` or a PR's `head` is not a full commit id, that lists a
+PR number twice, or whose ids differ from the bundle's refs, before any test runs.
+
+## Install
+
+```
+pip install pr-merge-audit
+pr-merge-audit --help
+```
+
+or run `python3 pr_merge_audit.py` from a clone; it has no dependencies. The commands below use
+`python3 pr_merge_audit.py`; `pr-merge-audit` takes the same arguments.
+
+## Usage
+
+```
+python3 pr_merge_audit.py --config PROJECT/config.json snapshot   # freeze PR heads and metadata
+python3 pr_merge_audit.py --config PROJECT/config.json build --snapshot snapshots/DATE.json   # offline
+python3 pr_merge_audit.py --config PROJECT/config.json build --snapshot snapshots/DATE.json \
+    --emit-branch refs/heads/integration --emit-repo PATH/TO/CLONE   # also keep the merged build as a branch
+python3 pr_merge_audit.py report --results results/DATE.json --out reports/DATE.md
+python3 pr_merge_audit.py verify --results results/DATE.json --golden reports/DATE.md
+python3 pr_merge_audit.py --config PROJECT/config.json estimate --snapshot snapshots/DATE.json \
+    --reference results/EARLIER.json                     # rough running time on this machine
+```
+
+Paths are relative to the current directory, so run it from wherever you keep that project's
+snapshots and results. `report` and `verify` need no configuration.
+
+`snapshot` writes `snapshots/DATE.json` (PR numbers, titles, head commits, changed files,
+draft flag, required-status state) and `snapshots/DATE.bundle` (every commit involved), so a
+build can be re-run later even after PR branches move or are deleted.
+
+## Procedure
+
+1. **Classify.** Each open PR is either tried or skipped with a reason, in this order: excluded
+   in the config, required status check not successful, draft, listed as a duplicate, listed as
+   a feature, or changes files without touching the test directory (unless listed in
+   `no_tests_ok`).
+2. **Sequential build.** Tried PRs are merged with `git merge --no-ff` onto the base in the
+   configured order (unlisted PRs follow by number). The test command runs after each merge.
+   A conflict is aborted; a test failure is re-run once and the merge is reset. A failure that
+   passes on re-run is recorded as flaky.
+3. **Swap test.** For each conflicting PR, the merged PRs it conflicts with pairwise
+   (`git merge-tree`) are reverted, the PR is merged in their place, and the tests run.
+4. **Culprit search.** For each PR that fails tests without a text conflict, or still fails after
+   its swap, the merge sequence is bisected for a PR that makes it fail, and that pair is tested
+   alone on the base. Bisection assumes a failure, once it appears, stays as more PRs are merged;
+   a later PR that repairs an earlier interaction can mislead it. The report names a pair as
+   failing only when the pair fails alone; otherwise it lists the PR under Other with the
+   candidate bisection pointed at.
+5. **Probes.** Each file in `probes` is copied into each listed target (the base, or a PR head)
+   and run with the probe's own command.
+
+Merge commits use a fixed identity and timestamp, global and system git config are ignored,
+and tests run with a fixed locale, time zone and hash seed, a private `HOME` and `TMPDIR`, and
+none of the caller's environment variables other than `PATH` (plus proxy and certificate settings)
+and those listed in `pass_env`.
+
+## Keeping the merged build as a branch
+
+`build --emit-branch REF --emit-repo PATH` keeps the result of the sequential build: a new branch
+`REF` at its last commit, which is the base with every PR that merged and passed, merged in the
+configured order. Land it as one pull request instead of landing the PRs one by one.
+
+- **Which repository.** The build runs in a private repository loaded from the snapshot bundle and
+  deleted at the end, so the branch goes to the local git repository at `PATH` (found as
+  `git -C PATH` would), usually your clone of the project. Nothing else receives it.
+- **What is written there.** The commits the branch needs, the one ref `REF`, and that ref's
+  reflog entry, which carries your own git identity and the current time (the commits keep the
+  fixed merge identity). No remote, tag, `FETCH_HEAD` or other ref changes, and nothing is
+  pushed: push the branch yourself when you are ready.
+- **Checked before the build.** The two flags go together. `REF` must be a full branch name
+  (`refs/heads/...`) that is not a symbolic ref, and `PATH` must use the snapshot's object format
+  (SHA-1 or SHA-256). A dry run of the final write, which locks `REF` and lets it go, finds an
+  existing branch of that name and clashes such as `x` beside `x/y`. Any of these stops the build
+  before any test runs, with git's own message where git gave one.
+- **Never overwrites.** The write itself requires that `REF` still does not exist and is not a
+  symbolic ref. If either has changed during the build, the results are written without the
+  branch and the build exits with an error. The write never follows a symbolic ref, so no other
+  branch is created: a symbolic ref to an existing branch makes it fail, and so does one that
+  points nowhere, created in the moment between the last check and the write, with recent git
+  (2.56 refuses it; 2.50 replaces it with `REF` itself).
+- **Manifest.** The results file gains an `emitted` key: the repository's git directory, the
+  ref, the commit, the PRs included (in merge order) and every PR excluded with its reason
+  (`conflict`, `fail` or `flaky` for PRs that were tried, `not tried: ...` for the rest). The
+  report does not read it, so it is byte-identical to that of a build without the flag.
+- **Reproducible.** Merge commits use a fixed identity and timestamp, so the same snapshot,
+  configuration and test outcomes give the same commit id.
+
+Swap tests, culprit searches and probes do not change the branch. A PR that conflicts and
+passes once swapped in is excluded: it conflicts with a PR on the branch, so it cannot join as it is.
+If no PR merged, the branch is the base itself.
+
+## Security
+
+`build` runs the test code of every PR it tries, and also runs each probe on its listed targets
+whatever the filters decided about them; `estimate` runs the base's tests. That code runs as you,
+with access to everything your user can reach. **Run builds on a disposable machine or
+container that holds no credentials, keys or tokens.** Nothing in this tool is a sandbox.
+
+Because a PR's code runs inside the test runner, a PR written to fake a pass can do so. A pass
+needs exit status 0 and, when `count_pattern` is set, at least one test counted. That catches an
+early exit or an empty run, but not output crafted to look like a real one.
+
+What the tool does to limit damage short of that:
+
+- Tests run in a fresh copy of the commit's files, made with `git archive`, with no `.git`
+  directory, in a new temporary directory with its own `HOME` and `TMPDIR`. Test code cannot
+  change the shared repository's config, hooks or refs through its working tree. The commit's
+  own `.gitattributes` cannot drop files from the copy (`export-ignore`) or rewrite them
+  (`export-subst`).
+- No archive entry is written through a link or over an entry already there (on a
+  case-insensitive disk, `t` after a link `T`), and links that resolve outside the copy are
+  removed before the tests run. Probe files are written only if no directory on the way is a
+  symlink and no file of that name exists.
+- Each test run gets its own process group, which is killed when the run ends or times out, so
+  processes the tests leave in the background stop with it (unless they start their own session).
+  On Windows a job object does the same; if Windows refuses to put the run in one, the tool prints
+  a warning and stops only the test process itself, so its children may survive the run.
+- Probe names and the reasons from `exclude`, `duplicates` and status checks are escaped in the
+  report, so they cannot add table cells, list items, links, mentions or HTML.
+- The config is checked when it is loaded: commands must be non-empty lists of strings, `workers`
+  and `test_timeout` positive numbers, and each output pattern a valid regular expression with a
+  capturing group.
+- Git runs with system and global config ignored, hooks pointed at the null device,
+  `core.fsmonitor` off and only `PATH` (plus proxy and certificate settings) taken from your
+  environment. Subprocesses never go through a shell. Test runs also get the variables you list in
+  `pass_env` and `env`; anything listed there is visible to every PR's code.
+- Test output is written to disk and at most 4 MB of it is read back.
+- Text taken from test output is put in the report only inside code spans, with backticks
+  replaced, `|` escaped in tables, control characters removed and a length cap, so it cannot add
+  mentions, links, images or table rows to a comment.
+- The work directory is only ever deleted if it holds the `.audit-workspace` marker the tool
+  wrote there.
+- `GITHUB_TOKEN` is sent only to the configured API host and is not passed on by redirects.
+
+## How long it takes
+
+Almost all the time goes into running the test suite, so the cost of a build is counted in
+test-suite runs rather than seconds. Each results file records how many runs each phase used,
+and the report states them:
+
+- **Sequential build:** one run on the base, one per PR that merges, two per PR that fails
+  (the re-run that checks for flakiness), none for a conflict. These run one at a time.
+- **Swap tests:** one run per conflicting PR, two if it fails.
+- **Culprit searches:** about log2(number of merged PRs) + 3 bisection steps per failing PR,
+  each of which also rebuilds part of the merge sequence.
+- **Probes:** one run per target.
+
+Swap tests, culprit searches and probes run on `workers` parallel jobs. To estimate the
+running time on your machine, time one test-suite run here and scale it by the counts of a
+reference build of the same project:
+
+```
+python3 pr_merge_audit.py --config PROJECT/config.json estimate --snapshot snapshots/DATE.json --reference results/EARLIER.json
+```
+
+The estimate uses the counts in the reference results, scaled by how many PRs the new snapshot
+would try. It is a guide, not a promise: the number of conflicts and failures, and so
+the number of follow-up runs, depends on the PRs.
+
+The results file also records the measured time of every run under `timing`, by phase:
+the count, total, median and maximum seconds, and the wall-clock time of the sequential build
+and of the follow-ups.
+
+### Serial landing, for comparison
+
+The report also counts what landing the same PRs one at a time would take. N is the number of
+PRs tried (merged or not). Landing them serially, every PR still open is re-tested after each
+landing, so the runs are N + (N - 1) + ... + 1 = N(N+1)/2. The report states N, that count and
+the runs this build counted, with their ratio when serial landing needs more. It gives no times,
+so it stays the same from build to build.
+
+`estimate` gives the comparison in hours for the snapshot it estimates: N is the number of
+PRs that snapshot would try, and the suite time is the one test-suite run it just timed on the
+base on this machine. It prints the N(N+1)/2 runs times that time in machine-hours, and N times
+it as the hours to land them, assuming one suite run at a time, since each landing waits for its
+run. The suite usually grows as PRs merge, so treat both as a minimum. Both the report and
+`estimate` assume every PR stays until it lands.
+
+### A measured build
+
+One full build of a Python project with 111 open PRs, on a 14-core laptop
+with `workers: 4`. The tool tried 76 PRs. 53 merged, 18 conflicted and 5 failed.
+The suite grew from 70 tests on the base to 248 after the last merge.
+
+| Phase | Runs | Median run | Longest run |
+| :--- | ---: | ---: | ---: |
+| Sequential build | 64 | 76 s | 139 s |
+| Swap tests | 14 | 163 s | 189 s |
+| Culprit searches (5 PRs) | 57 | 62 s | 148 s |
+| Probes | 3 | 0.5 s | 0.8 s |
+
+Wall clock: 72 min for the sequential build, 38 min for the follow-ups (swaps, culprit
+searches and probes on 4 workers), 110 min in total.
+
+Use these numbers to estimate a build of your own repository:
+
+```
+sequential  ≈ (1 + merged + 2 × failed) × suite time
+follow-ups  ≈ (conflicts + 11 × failed) × suite time ÷ workers   (at least one culprit search, ~11 runs, end to end)
+total       ≈ sequential + follow-ups
+```
+
+Here "suite time" is the average time of one test-suite run across the build. Since every
+merge can add tests, it grows as the build goes on, so time the suite on a branch near the
+end state rather than on the base. In the example the suite ran in 68 s on average, so the
+formula gives (1 + 53 + 10) × 68 s ≈ 73 min, and (18 + 55) × 68 s ÷ 4 ≈ 21 min, which is
+lower than the 38 min measured. The difference is the longest culprit search, which bisects
+one step at a time, and the swap tests, which run the full suite after the whole merge
+sequence. Treat the follow-up figure as a minimum.
+
+## Configuration
+
+| Key | Meaning |
+|---|---|
+| `repo`, `remote`, `api`, `base` | Repository, fetch URL, API root, base branch |
+| `required_status` | Status or check name a PR must pass to be tried (empty: none) |
+| `test_cmd` | Required. Test command as a list of arguments; `{python}` is replaced |
+| `test_prefix`, `test_timeout` | Test directory (default `tests/`), seconds per run (default 1800) |
+| `failure_pattern` | Regex whose group 1 is a failing test's name in the output (unset: none named) |
+| `count_pattern` | Regex whose group 1 is the number of tests run (unset: exit status alone decides) |
+| `skipped_pattern`, `message_pattern` | Regexes for the skipped count and the first error message |
+| `test_files` | Regex for test files, used to decide whether a PR changes tests (default: anything under `test_prefix`) |
+| `python`, `workers` | Interpreter for tests (default: the one running the tool), parallel jobs |
+| `pass_env` | Names of variables copied from your environment into test runs (e.g. `RUSTUP_HOME`, `CARGO_HOME`, `GOPATH`, `VIRTUAL_ENV`) |
+| `env` | Variables set in test runs, applied last (can override `HOME`) |
+| `allow_incomplete_checkout` | Run even though the base uses submodules or Git LFS, which test runs lack (shown in the report) |
+| `setup_cmd` | Command run before `test_cmd` in the same copy (e.g. installing dependencies); if it fails, the run fails as `(setup failed)` |
+| `order` | Merge order; it decides which side of each conflict merged first |
+| `exclude`, `duplicates`, `features` | PR number to reason; these PRs are not tried |
+| `no_tests_ok` | PRs tried even though they change no tests |
+| `probes` | `{name, file, cmd, targets}`, optional `dest`; see below |
+
+A probe's `file` (relative to the configuration file) is written to `dest` in the target, by
+default `test_prefix` + `audit_probe_` + its file name, and `cmd` is run there. In `cmd`,
+`{python}`, `{dest}` and `{module}` (`dest` without its suffix, with `.` for `/`) are replaced.
+Targets are `"base"` or PR numbers. Probe output is read with the same patterns as the tests. `dest`
+must be a relative path inside the target: an absolute path, a drive or a `..` part stops the
+build before any test runs.
+
+## Tests
+
+```
+python3 -m unittest discover -s tests
+```
+
+`tests/test_pr_merge_audit.py` builds small git repositories in temporary directories and runs the real
+tool on them (no mocks): merge, conflict and swap, a cross-PR test failure found by bisection, a
+PR that fails on the base alone, the not-tried reasons, probes, run counts, determinism across
+two builds, test isolation from `.git`, archive and probe containment, hidden and early-exiting
+tests, the emitted branch and its manifest, background processes, report escaping, output capping, the work-directory marker and
+token handling on redirects. It also checks the golden report. The suite takes a few seconds.
+
+## Self-check
+
+`tests/golden/` holds the results of a build of the test fixture in `tests/test_pr_merge_audit.py` and
+the report they render to. Rendering is checked with:
+
+```
+python3 pr_merge_audit.py verify --results tests/golden/results.json --golden tests/golden/report.md
+```
+
+which renders the report twice and compares both renders with the golden copy byte for byte.
+Version 1.1 adds the serial-landing line to the report, so a report written by 1.0.x no longer
+matches its results under `verify`; render it again with `report` first.
+After a deliberate change to the report, rebuild the fixture and replace both files.
+
+## Contributing and security
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for how the project is developed and what is enforced,
+and [SECURITY.md](SECURITY.md) for the threat model and how to report a vulnerability.
+
+## License
+
+Apache License 2.0; see `LICENSE`.