@gfdelarue/testkit 0.4.0 → 0.4.1

package/docs/bug-records.md

@@ -0,0 +1,303 @@
+# Bug records
+
+This page is the reference for what Testkit writes to GitHub or Gitea when it files a bug or reports a CI run. Tools that read Testkit's issues and statuses depend on it.
+
+The format version is `1`. The facts block carries it as `v`.
+
+## Labels
+
+Every issue that Testkit files carries the `testkit` label. Testkit and its readers find Testkit's issues by that label and by nothing else.
+
+| Label               | Colour   | Meaning                                                                    |
+| ------------------- | -------- | -------------------------------------------------------------------------- |
+| `testkit`           | `0f766e` | Filed by Testkit with a regression test.                                   |
+| `severity:critical` | `9f1239` | The product is unsafe to run or to trust.                                  |
+| `severity:high`     | `e11d48` | A core behaviour is broken with no acceptable workaround.                  |
+| `severity:medium`   | `d97706` | A behaviour is broken and a workaround exists, or the reach is limited.    |
+| `severity:low`      | `4d7c0f` | A minor defect with little consequence.                                    |
+| `area:<id>`         | `475569` | The product area that holds the fault. The description is the area's name. |
+
+Testkit creates a label that does not exist yet and never changes one that does.
+
+Severity and area are labels because people re-triage both. A reader takes the bug's severity from the `severity:` label. A reader takes the bug's area id from the `area:` label when one is present and from the facts block otherwise. An issue with several `severity:` labels or several `area:` labels has no severity or no area label for this purpose.
+
+## Issue body
+
+The body has five sections and ends with the facts block.
+
+````markdown
+## Expected behaviour
+
+A refund pays out once.
+
+## Authority
+
+`docs/refunds.md`, "Refunds are idempotent on the order id".
+
+## Observed behaviour
+
+Two overlapping refund requests for one order both pay out.
+
+## Root cause
+
+`refund()` reads the order's refund state and writes the payout without a lock.
+
+## Reproduce
+
+```sh
+npx testkit run --file src/checkout/__testkit__/refund.contract.testkit.ts
+```
+
+Test: `src/checkout/__testkit__/refund.contract.testkit.ts`, "a refund pays out once".
+Found at `4f0c2d1e9b7a4c3d8e6f5a4b3c2d1e0f9a8b7c6d`.
+Fault: `src/checkout/refunds.ts`, lines 41 to 58.
+
+<details>
+<summary>Testkit facts</summary>
+
+<!-- testkit-facts:start -->
+
+```json
+{
+  "v": 1,
+  "cause": "race-condition",
+  "impact": "financial-error",
+  "confidence": { "cause": "high", "impact": "high" },
+  "escape": "no-test",
+  "caughtBy": "campaign",
+  "campaign": "refunds-2026-10",
+  "testType": "contract",
+  "test": {
+    "file": "src/checkout/__testkit__/refund.contract.testkit.ts",
+    "title": "a refund pays out once"
+  },
+  "foundAt": "4f0c2d1e9b7a4c3d8e6f5a4b3c2d1e0f9a8b7c6d",
+  "fault": { "path": "src/checkout/refunds.ts", "lines": [41, 58] },
+  "area": { "id": "checkout", "name": "Checkout", "share": 0.14 },
+  "introducedBy": "8a7b6c5d4e3f2a1b0c9d8e7f6a5b4c3d2e1f0a9b",
+  "introducedMethod": "blame"
+}
+```
+
+<!-- testkit-facts:end -->
+
+</details>
+````
+
+The reproduce command runs from the repository root. When `testkit.config.ts` is in a subdirectory, the command carries `--dir <directory>` and the `--file` path is relative to that directory. The `Test:` and `Fault:` lines always give paths from the repository root.
+
+A fault on one line reads `line 41`.
+
+An issue for a k6 script that fails as a whole covers the whole file. Its facts block has no `test.title`, and its `Test:` line quotes the `test.title` of the bug record as the finder's description of the script.
+
+```markdown
+Test: `src/health/__testkit__/health.int.testkit.ts`, the whole script, recorded as "the health endpoint answers".
+```
+
+Testkit redacts the secrets it recognises in the title, in every section, and in `test.title`. It does so when it records the bug and again when it files the issue. It recognises these forms.
+
+- The value of a `KEY=value` assignment whose key names a secret.
+- The password in a URL.
+- The value of an `Authorization`, `Proxy-Authorization`, `Cookie`, or `Set-Cookie` header.
+- A JSON string value whose key names a secret.
+
+A key names a secret when it contains `PASSWORD`, `TOKEN`, `SECRET`, `PRIVATE_KEY`, `AUTH`, `COOKIE`, `SESSION`, or `API_KEY` in any letter case. Testkit cannot recognise every secret, so the finder must still keep credentials, tokens, and passwords out of the record.
+
+An assigned value ends at the first space, with three exceptions.
+
+- A value that starts with `"` or `'` ends at the next such quote on its line. When the line has no such quote, the value ends with the line.
+- A value that starts with `Basic`, `Bearer`, `Digest`, `Negotiate`, or `Token` and then a space or a tab ends with the line.
+- Under a key that ends in `_URL`, Testkit keeps a value that starts with a scheme, such as `https://`, and redacts the URL's password only.
+
+A URL password is the text between `://user:` and the last `@` before the next `/` or space. The user can be empty, as in `redis://:password@cache`. Testkit does not recognise a password that holds a `/`, or a token that stands alone as the user, as in `https://token@host`.
+
+Testkit does not rewrite a path or an area's name. `testkit bug record` rejects a record whose `test.file` or `fault.path` holds text that Testkit redacts. `testkit bug file` stops when the area's name or the reproduce command holds such text.
+
+Testkit reads the facts back from the body before it files the issue. It files nothing when they do not read back as written. That happens when `test.title`, `test.file`, `fault.path`, or the area's name holds one of the two marker comments.
+
+## Facts block
+
+The facts block is one JSON object in a fenced `json` code block between the two marker comments.
+
+To read it:
+
+1. Find the last `<!-- testkit-facts:start -->` in the body.
+2. Find the first `<!-- testkit-facts:end -->` after it.
+3. Take the first fenced code block between the two markers.
+4. Parse its content as JSON. Accept `\r\n` line endings.
+
+A body with no start marker has no facts. A body with a start marker and no readable object has invalid facts. A reader keeps the issue in both cases and records which case applies.
+
+All paths in the block use `/` and are relative to the repository root.
+
+| Field               | Type                                                     | Written                               | Meaning                                                         |
+| ------------------- | -------------------------------------------------------- | ------------------------------------- | --------------------------------------------------------------- |
+| `v`                 | `1`                                                      | Always                                | The format version.                                             |
+| `cause`             | Cause id                                                 | Always                                | The kind of mistake in the code.                                |
+| `impact`            | Impact id                                                | Always                                | What is at stake for the business.                              |
+| `confidence.cause`  | `high` or `low`                                          | Always                                | The finder's confidence in `cause`.                             |
+| `confidence.impact` | `high` or `low`                                          | Always                                | The finder's confidence in `impact`.                            |
+| `escape`            | Escape id                                                | Always                                | Why the existing tests did not catch the bug.                   |
+| `caughtBy`          | `campaign` or `manual`                                   | Always                                | `campaign` when a campaign was active at record time.           |
+| `campaign`          | Campaign id                                              | With `caughtBy: "campaign"`           | The campaign that caught the bug.                               |
+| `testType`          | `ui`, `e2e`, `int`, `contract`, `data`, `dal`, or `load` | Always                                | The type of the regression test, from its file suffix.          |
+| `test.file`         | Path                                                     | Always                                | The regression test file.                                       |
+| `test.title`        | String                                                   | When the issue covers one test        | The failure's title. See [Known failures](#known-failures).     |
+| `foundAt`           | Full commit id                                           | Always                                | The commit the test failed at.                                  |
+| `fault.path`        | Path                                                     | Always                                | The source file that holds the mistake.                         |
+| `fault.lines`       | `[start, end]`                                           | Always                                | The first and last faulty line at `foundAt`, counted from 1.    |
+| `area.id`           | Area id                                                  | When the area map claims `fault.path` | The area's key in `areas`.                                      |
+| `area.name`         | String                                                   | With `area.id`                        | The area's name at filing time.                                 |
+| `area.share`        | Number from 0 to 1                                       | With `area.id`                        | The area's share of the repository's source bytes at `foundAt`. |
+| `introducedBy`      | Full commit id                                           | When the blame resolves               | The newest commit among the fault lines at `foundAt`.           |
+| `introducedMethod`  | `blame`                                                  | With `introducedBy`                   | How Testkit found `introducedBy`.                               |
+
+Testkit writes every field it can resolve. A reader accepts a version 1 block that lacks `escape`, `campaign`, `test.title`, `fault`, `fault.lines`, `area`, `introducedBy`, or `introducedMethod`, because issues that predate the CLI were given facts by hand. A reader ignores fields it does not know.
+
+A reader rejects a block whose `v` it does not know, and a block where a field that is present has the wrong type or an id outside its list.
+
+## Cause
+
+The question is what kind of mistake is in the code. A bug gets exactly one cause. When a bug holds several mistakes, classify the earliest one in the causal chain. Use `other` when none fits.
+
+| Id                              | Definition                                                                                                                                                                                                                                                                                                  | Boundary rule                                                                                                                                                                                                                                                                                           |
+| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `missing-validation`            | The code never checks an input or a state precondition that the domain rules require: range, format, existence, uniqueness, cross-field consistency, or lifecycle state. Includes a validator that exists and accepts invalid input.                                                                        | Earliest mistake wins. If application code never tested the condition, it is `missing-validation` even when a database constraint or the runtime later throws and the caller sees a server error. If the unchecked value is null, absent, empty, zero, or a default, use `null-or-default-mishandling`. |
+| `mishandled-error`              | A failure was detected and signalled, and the handling is wrong: not caught, translated to the wrong error, swallowed, ignored by the UI, or followed by a wrong retry or recovery.                                                                                                                         | The failure must already be signalled by something that checked. If nothing in the application checked and a lower layer failed, it is `missing-validation`.                                                                                                                                            |
+| `race-condition`                | Correctness depends on how overlapping operations interleave: check-then-act without a lock or constraint, a lost update, or a late asynchronous completion that overwrites newer state.                                                                                                                    | Needs two overlapping operations to reproduce. If one sequential flow reproduces it, it is `stale-state` or `missing-validation`.                                                                                                                                                                       |
+| `unbounded-or-inefficient-work` | Results are correct and the cost is unbounded or needlessly repeated: unpaginated reads, N+1 queries, a missing or redundant index, serial waits, or loading everything and then slicing.                                                                                                                   | Correct results at excessive cost. If the result is wrong, it is `incorrect-query-or-selection`.                                                                                                                                                                                                        |
+| `incorrect-query-or-selection`  | The code identifies the wrong set of records: a missing or wrong predicate, unstable ordering, a truncated page treated as complete, duplicate rows from a join, or a lookup key that omits a distinguishing part.                                                                                          | A missing predicate is `missing-access-control` only when it separates actors or tenants with different rights. The right records with wrong arithmetic on them is `incorrect-logic`.                                                                                                                   |
+| `missing-access-control`        | A permission, tenant, ownership, or separation-of-duties check is absent or wrong.                                                                                                                                                                                                                          | Use it when the missing check is about who the actor is. A missing check about what the data or state is belongs in `missing-validation`.                                                                                                                                                               |
+| `null-or-default-mishandling`   | Null, absent, empty, zero, or default values are conflated or not handled: null treated as omitted, a zero or an empty collection treated as missing, a null dereference, a wrong fallback default, or a caller that omits an argument and silently gets the default.                                       | Wins over `missing-validation` and `incorrect-logic` whenever the triggering value is null, absent, empty, zero, or a default.                                                                                                                                                                          |
+| `stale-state`                   | Held state, such as a cache, local UI state, a derived or denormalised value, or leftover fields, is not refreshed, reset, persisted, or tied to the right identity when the thing it mirrors changes.                                                                                                      | If the omission leaves old values visible where current ones should be, it is `stale-state`. If the missing action leaves nothing stale, it is `omitted-step`. If it reproduces only with overlapping operations, it is `race-condition`.                                                               |
+| `interface-mismatch`            | Data is lost or misread where it crosses between representations or components. One component's mapping drops, overwrites, hardcodes, or mis-assigns fields, or two components that must agree disagree on shape, type, field names, allowed values, semantics, or which copy of the data is authoritative. | Use it when the values were right before the crossing. Wrong computation on values that arrived intact is `incorrect-logic`. A held copy that goes out of date after its source changes is `stale-state`. An omitted argument that falls back to a default is `null-or-default-mishandling`.            |
+| `omitted-step`                  | A required follow-on action is not performed on some path: an audit record, a notification, a dispatch, a finalisation, or an alternate route that bypasses the shared routine.                                                                                                                             | If the missing thing is a check, it is `missing-validation`. If its absence leaves old values on display, it is `stale-state`.                                                                                                                                                                          |
+| `incorrect-logic`               | The algorithm, arithmetic, date or time handling, rounding, boundary, comparison, boolean condition, parsing, or order of operations is wrong.                                                                                                                                                              | The default for wrong logic on valid, present values. If the trigger is a null, empty, zero, or default value, use `null-or-default-mishandling`. If the fault is which records were chosen, use `incorrect-query-or-selection`.                                                                        |
+| `other`                         | No category above describes the mistake.                                                                                                                                                                                                                                                                    |                                                                                                                                                                                                                                                                                                         |
+
+## Impact
+
+The question is what is at stake for the business. A bug gets exactly one impact. The list is in precedence order, and the first type that applies wins. Classify the impact the test demonstrated, not a downstream one you expect. Use `other` when none fits.
+
+| Id                            | Definition                                                                                                                                                    | Boundary rule                                                                                                                                         |
+| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `unauthorized-access`         | Someone can see or change something they are not entitled to, including getting past a separation-of-duties rule.                                             | The actor lacks the right. If an entitled actor skips a mandated control or leaves no evidence, it is `audit-or-compliance-gap`.                      |
+| `financial-error`             | A monetary amount, rate, currency, limit, payment or settlement state, or a money-committing decision is wrong where it is stored, charged, paid, or decided. | A money figure that is only displayed wrongly, with the stored value and the decision intact, is `wrong-information-shown`.                           |
+| `audit-or-compliance-gap`     | Required audit evidence or attribution is missing, duplicated, or false, or a mandated control did not run.                                                   | Use it when the harm is to evidence or to a regulatory or policy control. A skipped ordinary process step is `workflow-breakdown`.                    |
+| `data-loss-or-corruption`     | Stored data, or work the user entered, is destroyed, overwritten, duplicated, or left invalid or inconsistent.                                                | If the action silently did nothing and prior stored state is intact, it is `blocked-operation`.                                                       |
+| `workflow-breakdown`          | A business process, such as a workflow, task, schedule, notification, or gate, is skipped, stalled, duplicated, or advanced wrongly.                          | Use it when the process goes wrong after the triggering request returns. A single user action that fails in front of the user is `blocked-operation`. |
+| `availability-or-performance` | The service is slow, unavailable, or exhausts resources in a way that affects more than the triggering request or grows with data volume.                     | One request that fails for one user is `blocked-operation`.                                                                                           |
+| `blocked-operation`           | A legitimate action fails, returns an unusable or misleading error, or silently does not apply, with prior stored state intact.                               | The user was trying to do something and could not. If they were only reading and what they read was wrong, it is `wrong-information-shown`.           |
+| `wrong-information-shown`     | Displayed or returned information is wrong, out of date, incomplete, or duplicated, with nothing stored wrongly.                                              | Last in precedence. If the wrong value is also persisted, it is `data-loss-or-corruption`.                                                            |
+| `other`                       | No type above describes what is at stake.                                                                                                                     |                                                                                                                                                       |
+
+## Escape reason
+
+The question is why the tests that existed at `foundAt` did not catch the bug. A bug gets exactly one reason. Judge the tests that were there before yours. Use `other` when none fits.
+
+| Id                      | Definition                                                                                                                                     | Boundary rule                                                                                                                   |
+| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
+| `no-test`               | No existing test runs the faulty behaviour.                                                                                                    | If any existing test reaches the faulty code path, use another reason.                                                          |
+| `untested-input`        | A test runs the behaviour and never with the input, state, or data volume that triggers the fault.                                             | One sequential flow with the right input reproduces the bug. If it needs overlapping operations, it is `untested-interleaving`. |
+| `untested-interleaving` | A test runs the behaviour in sequence, and the fault needs overlapping operations or a particular timing.                                      | The triggering input is already covered, and only the overlap is missing.                                                       |
+| `wrong-assertion`       | A test runs the triggering case against the real code and passes, because its assertion is missing, too weak, or asserts the broken behaviour. | The faulty code ran in the test. If a stand-in ran in its place, it is `test-double`.                                           |
+| `test-double`           | A test runs the triggering case against a test double or a fixture that hides the fault in the real component.                                 | The real component would have failed the test.                                                                                  |
+| `other`                 | No reason above applies.                                                                                                                       |                                                                                                                                 |
+
+## Areas
+
+An area id matches `^[a-z0-9]+(?:-[a-z0-9]+)*$` and is at most 40 characters long.
+
+`area.share` is the area's source bytes divided by the repository's source bytes at `foundAt`. A source file is a tracked file with a source extension that is not a test, a migration, a dependency, build output, a `.d.ts`, `.d.cts`, or `.d.mts` declaration file, a file with `.min.` in its name, a dotfile, or a `*.config.*` file at the repository root. Each source file counts toward one area at most. A file that two areas claim counts toward neither until the map is fixed.
+
+## Known failures
+
+A failed file reports its failures in one of three forms.
+
+| Failure         | When                                                                                                                                                                                              |
+| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Named failure   | The engine names the test that failed. One failure for each title.                                                                                                                                |
+| Script failure  | A k6 script fails as a whole: it calls `fail()`, throws while it runs, fails a k6 `check()`, or fails a threshold. One per file.                                                                  |
+| Unnamed failure | The file fails any other way, such as a crash, a timeout, a file that does not load, a Vitest error outside a test, a Playwright error outside a spec, or a k6 script that throws during `setup`. |
+
+A named failure's title comes from the engine.
+
+| Engine     | Title                                                                                                                            |
+| ---------- | -------------------------------------------------------------------------------------------------------------------------------- |
+| Vitest     | The names of the `describe` blocks around the test and the test's own name, joined by single spaces.                             |
+| Playwright | The names of the `describe` blocks around the spec and the spec's own title, joined by single spaces.                            |
+| k6         | The label of a failed assertion from `@gfdelarue/testkit/k6`, or the assertion's request and default title when it has no label. |
+
+A Playwright title holds neither the file path nor the project name. A spec that fails in two projects is one named failure.
+
+A k6 assertion on a response has a default title when the script gives it no label. Those assertions are `expect.status`, `expect.statusOneOf`, `expect.notStatus`, `expect.json`, `expect.jsonPath`, `expect.error.shape`, and `expect.error.message`. Testkit names such a failure by its request and the default title, as in `GET /health > status is 201`. Inside a k6 `group`, the group's name comes first, as in `orders > GET /health > status is 201`. Any other assertion with no label keeps its default title, such as `row count is 2`. A label names its failure as written, even when it reads like a default title. Two failures in one file whose titles are equal are one named failure.
+
+When a file reports named failures, `testkit bug file` rejects a bug record whose `test.title` is not one of their titles. Its message lists the titles a bug record can use.
+
+Testkit files no bug under a title that spans lines or holds text it redacts as a secret, so no issue with a `test.title` covers such a failure. `testkit run` redacts some secrets when it stores a run result and writes `<redacted>` in their place. A title that the run result stores with `<redacted>` is such a title. Rename the test, or give the assertion a label, so that its title is one line and holds no such text. [Issue body](#issue-body) lists what Testkit redacts.
+
+Vitest reports an error outside a test when a hook such as `beforeAll` throws, or when code throws after its test has returned. The file then reports one unnamed failure beside its named failures.
+
+Playwright reports an error outside a spec when code such as a `globalTeardown` throws. The file then reports one unnamed failure beside its named failures.
+
+A file also reports one unnamed failure beside its named failures when the engine fails the file for a reason that its failed tests do not explain.
+
+- A Vitest file runs past its timeout, writes more output than Testkit keeps, or ends on a signal or with an exit code other than 1.
+- A Playwright run is interrupted or runs past its timeout.
+- A Playwright process ends on a signal or with an exit code other than 1. A reporter in the Playwright config that ends the process does that.
+- A Playwright config sets `failOnFlakyTests` and a spec passes only on a retry. A spec that passes on a retry is no named failure, so no issue can cover it.
+- A Playwright spec fails in a file other than the selected one, such as a spec in a project that the selected file's project depends on or a spec in a teardown project. That spec is no named failure of the selected file, so no issue can cover it. A spec's file is the file that Playwright loaded. A compiled file whose source map names the selected file is still another file.
+- Testkit cannot keep a file that a Playwright spec attached, or cannot store the evidence of the Playwright run.
+- A k6 process whose thresholds failed ends on a signal or with an exit code other than 99.
+- A k6 process whose script threw an error ends on a signal or with an exit code other than 0 or 99, or leaves no summary that Testkit can read.
+
+A k6 script reports a script failure and no named failure when it fails only through `fail()`, a thrown error, a k6 `check()`, or a threshold. `testkit bug file` then files one issue for the whole file, and the issue's facts carry no `test.title`.
+
+A k6 script reports its named failures and one script failure when it also fails that way beside the assertions the run names. Testkit tells a k6 `check()` from its own assertions by count. The k6 summary counts every failed check, and each failed assertion from `@gfdelarue/testkit/k6` is one failed check. When the summary's count differs from the number of failed assertions the run names, or the summary carries no count, the file has a script failure.
+
+A failure is known when an open issue with the `testkit` label covers it. The issue's `test.file` must equal the failing file's path from the repository root.
+
+| Failure         | Covered by                                                                                            |
+| --------------- | ----------------------------------------------------------------------------------------------------- |
+| Named failure   | An issue whose `test.title` equals the failure's title, or an issue whose facts have no `test.title`. |
+| Script failure  | An issue whose facts have no `test.title`.                                                            |
+| Unnamed failure | No issue.                                                                                             |
+
+An issue with a `test.title` covers that one title and nothing else. An issue with no `test.title` covers the whole file. That is every named failure in the file and its script failure, and never an unnamed failure. An unnamed failure is always new. Closing an issue makes the next failure it covered new.
+
+When Testkit cannot read the issues, every failure is new.
+
+`bugs.knownFailures` in `testkit.config.ts` decides what a known failure does to a run. With `"fail"`, the default, every failing test fails the run. With `"allow"`, a run whose failures are all known passes, and one new failure fails it. A run that fails outside its tests fails under both settings.
+
+## Commit status
+
+`testkit ci report` posts one commit status on the checked-out commit. When CI checks out a pull request's merge commit, the status goes on the pull request's head commit instead.
+
+| Part        | Value                                                                                                                     |
+| ----------- | ------------------------------------------------------------------------------------------------------------------------- |
+| Context     | `testkit`, or the value of `--context`.                                                                                   |
+| State       | `success` when the run passed, or when `bugs.knownFailures` is `"allow"` and every failure is known. `failure` otherwise. |
+| Target URL  | The CI run's URL, when the CI provider gives one.                                                                         |
+| Description | See below. At most 140 characters.                                                                                        |
+
+The description has one of three forms.
+
+| Run                              | Description                |
+| -------------------------------- | -------------------------- |
+| Every file passed                | `142 files passed`         |
+| Tests failed                     | `3 failed: 1 new, 2 known` |
+| The run failed outside its tests | `run failed: <reason>`     |
+
+The counts are failures. A script failure counts as one failure. An unnamed failure counts as one new failure.
+
+Testkit redacts the secrets it recognises in the reason. [Issue body](#issue-body) lists the forms.
+
+`testkit run` redacts some secrets when it stores the run result, and that redaction can stop before the secret's end. The description therefore ends at the first `<redacted>` in the stored reason. A redacted URL password is the exception. `testkit run` redacts a URL password whole, so the description continues past it.
+
+## Pull request comment
+
+`testkit ci report` keeps one comment on the run's pull request and updates it in place. The comment's first line is `<!-- testkit-ci-report -->`, or `<!-- testkit-ci-report:<context> -->` when the context is not `testkit`. Testkit finds its comment by that line.
+
+The comment names each failure by its file and its title. Testkit redacts the secrets it recognises in both, in the reason for a run that failed outside its tests, and in the reason it could not read the open bugs. [Issue body](#issue-body) lists the forms.
+
+A file, a title, or a run's reason that the run result stores with `<redacted>` ends there in the comment and in the step summary, as it does in the [commit status](#commit-status). A title such as `rejects PASSWORD=<redacted> with 401` shows as `rejects PASSWORD=<redacted>`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@gfdelarue/testkit",
-  "version": "0.4.0",
+  "version": "0.4.1",
   "description": "Test orchestration platform for product acceptance and QA workflows",
   "license": "SEE LICENSE IN LICENSE",
   "type": "module",
@@ -86,6 +86,7 @@
   "files": [
     "bin/",
     "dist/",
+    "docs/bug-records.md",
     "docs/engine-architecture.md",
     "docs/test-types.md",
     "testkit-config-schema.md"

package/testkit-config-schema.md

@@ -156,6 +156,56 @@ Optional `testkit lint` settings. Omit it to run the default rules.
 Enabled-rule findings and syntax errors fail the command. There are no severity
 levels. The repository's `docs/lint.md` describes each rule.
 
+### `areas`
+
+Optional map of the product's areas. `testkit bug file` uses it to name the area
+that holds a fault, and puts an `area:<id>` label on the issue.
+
+```ts
+areas: {
+  checkout: { name: "Checkout", paths: ["src/checkout/**"] },
+  billing: { name: "Billing", paths: ["src/billing/**", "src/invoices/**"] },
+},
+```
+
+- The key is the area id: lowercase letters, digits, and single hyphens, at
+  most 40 characters.
+- `name`: one line of 1 to 60 characters. It becomes the label's description.
+- `paths`: at least one glob. Globs are relative to the repository root and
+  written with `/`. A leading `!` or `/` and a `..` segment are rejected.
+
+Each source file belongs to at most one area. `testkit campaign start` lists
+the source files that no area claims and the files that two areas claim.
+`testkit bug file` stops when no area claims the fault's file, or when two
+areas do, and names the file. Without `areas`, Testkit files bugs with no area.
+
+An area's share is its part of the repository's source bytes at the commit that
+holds the bug. Tests, generated files, vendored code, and migrations do not
+count as source.
+
+### `bugs`
+
+Optional settings for bug filing and for runs that meet a filed bug.
+
+- `knownFailures`: `"fail"` (default) or `"allow"`. A known failure is a
+  failing test that an open issue with the `testkit` label covers.
+  [Known failures](docs/bug-records.md#known-failures) gives the rule. With
+  `"fail"`, every failing test fails the run. With `"allow"`, a run passes when
+  every failure is known, and only a new failure fails it. Closing the issue
+  makes the test count again. When Testkit cannot reach the tracker, it treats
+  every failure as new. With `"allow"`, the run reads the open issues, so the
+  `testkit run` step in CI needs a token.
+  [Pass a run whose failures are all filed](README.md#pass-a-run-whose-failures-are-all-filed)
+  shows the step.
+- `repository`: the repository that holds the bug issues. `testkit bug file`
+  files issues there, and `testkit run` and `testkit ci report` read the open
+  issues from it. Commit statuses and pull request comments always go to the
+  `origin` repository. Write `owner/name` for a repository on the same host as
+  the `origin` remote, or a URL such as `https://host/owner/name`. It defaults
+  to the `origin` remote.
+
+[Bug records](docs/bug-records.md) describes the issues Testkit files.
+
 ### `ui`
 
 Targets for Playwright `ui` files. Read through `createUiSandbox`,