@gfdelarue/testkit 0.6.0 → 0.6.1

package/docs/release-notes.md

@@ -1,3 +1,733 @@
+# 0.6.1
+
+## Each `ui` test's browser traffic is in the run evidence
+
+Testkit now records the requests every `ui` test's pages send
+([gfdelarue/testkit#496](https://github.com/gfdelarue/testkit/issues/496)).
+For each `ui` file whose tests sent requests, the run evidence has
+`browser-network.json` and `browser-network-report.md`, from each test's final
+attempt. Rows group page loads and API calls by service, method, and path, with
+IDs replaced by `:id`. Other requests count toward totals by type, and other
+origins appear by host only. The record holds no query strings, headers, or
+bodies.
+
+The report lists leads: a write sent more than once with the same body, and a
+page load or API call that failed in a test that passed. Leads never fail a
+test. Recording is always on. On a page that sent 400 requests it added no time
+we could measure.
+
+`browserRequests()` from `@gfdelarue/testkit/playwright` returns the test's
+requests so a test can assert on them:
+
+```ts
+const payments = (await browserRequests()).filter(
+  (request) => request.service === "api" && request.method === "POST" && request.path === "/payments",
+);
+expect(payments).toHaveLength(1);
+```
+
+[Check a ui test's browser traffic](../README.md#check-a-ui-tests-browser-traffic)
+describes the record, its limits, and the entry fields.
+
+## A `ui` test can slow its browser's network and CPU
+
+`@gfdelarue/testkit/playwright` exports `emulateNetwork(page, conditions)` and
+`emulateCpu(page, slowdown)`
+([gfdelarue/testkit#495](https://github.com/gfdelarue/testkit/issues/495)).
+They set Chromium's own throttling through the DevTools protocol, so a test can
+fail when a change makes a page unusable on a slow connection or a slow device:
+
+```ts
+test("the dashboard is usable on a slow connection", async ({ page }) => {
+  await emulateNetwork(page, "slow-4g");
+  await page.goto("/dashboard");
+  await expect(page.getByRole("heading", { name: "Orders" })).toBeVisible({ timeout: 10_000 });
+});
+```
+
+`emulateNetwork` takes `"fast-4g"`, `"slow-4g"`, or `"3g"`, with the values of
+DevTools' presets of the same name, or custom
+`{ latencyMs, downloadKbps, uploadKbps }`. It converts kilobits per second to
+the protocol's bytes per second. `emulateCpu` takes a fixed multiplier from 1
+to 100, which slows the page relative to the machine that runs it. The network
+conditions apply to the page you pass in, and the CPU slowdown to the browser
+process that runs it.
+[Slow the browser's network and CPU](../README.md#slow-the-browsers-network-and-cpu)
+lists the profile values, what each setting reaches, and how to set time limits
+that don't fail on a slow CI runner.
+
+Testkit's own Postgres client calls ran with the 30-minute timeout meant for
+long user SQL, and the source schema's replay into a template with the
+60-minute step timeout. A schema-only `pg_dump` that couldn't finish used up
+all of `db schema refresh`'s or `verify`'s time limit, and the error said only
+that a Postgres client was cancelled
+([gfdelarue/testkit#468](https://github.com/gfdelarue/testkit/issues/468)).
+
+Each call now has a cap:
+
+- the server version probe before a source schema dump: 60 seconds;
+- a schema-only `pg_dump`: 5 minutes;
+- the source schema's replay into a template: 10 minutes.
+
+When the probe or a dump reaches its cap, the error names the call and its
+cap, such as `Dockerized Postgres client for the schema-only pg_dump (300 s
+cap) exceeded its managed deadline`.
+
+In `db schema verify` and `refresh`, a `--timeout-seconds` above 600 lengthens
+these caps in proportion, along with the admin statements' `lock_timeout` and
+`statement_timeout` and the role setup's 60-second timeout: a limit of 1200
+seconds doubles them. Elsewhere, including `testkit run`, the caps keep their
+defaults. Configured template steps keep their 60-minute timeout.
+
+## A schema operation that stops waiting for a lock says who holds it
+
+When `db schema verify` or `refresh` ran out of its time limit while another
+operation on the same product held a lock it needed, the error said only which
+service it was preparing
+([gfdelarue/testkit#468](https://github.com/gfdelarue/testkit/issues/468)). It
+now names the lock, how long it waited, the holder's process and host, and the
+step the holder is on and for how long. A holder from an older Testkit, or one
+that recorded no step, is reported with its progress unknown.
+
+A lock whose release failed in Testkit's own process used to block that
+process's later waits until it exited. Testkit now reports it at once.
+
+A configured template step's toolchain installation now counts against the
+operation's limit. Only the step's own run doesn't.
+
+## `db schema verify` and `refresh` have a time limit
+
+`db schema verify` and `db schema refresh` had no overall bound. A Testkit
+statement blocked on another session's lock waited until its own
+`lock_timeout`, and any wait without a limit of its own could hang the
+command. Their cleanup ran with no bound and dropped its errors
+([gfdelarue/testkit#468](https://github.com/gfdelarue/testkit/issues/468)).
+
+Each operation now has a time limit, 600 seconds unless `--timeout-seconds`
+sets it. Your configured template steps don't spend it: they keep their own
+step timeout. When the limit runs out, the error lists the setup steps with
+their times, the service being prepared and the wait it stopped in.
+
+Cleanup gets its own 3-minute allowance. If it can't finish, the error says
+what it left, and the command fails even when the operation's work
+succeeded. `testkit destroy` removes what's left.
+
+A process Testkit stops is awaited for at most 15 seconds after its
+termination begins, or 15 seconds after a local service's shutdown grace.
+After that, Testkit reports it as unsettled instead of waiting.
+
+## A retained local database that won't start is rebuilt once
+
+Testkit keeps each product's local Postgres cluster in a Docker volume between
+runs. If that cluster was damaged, for example by a host crash while its files
+were half written, its server exited while starting, and every later run failed
+the same way until `testkit destroy`.
+
+Testkit now notices when the server exits while it waits for the database to
+become ready. If the cluster was on a volume an earlier run left, and the volume
+carries this product's Testkit labels, Testkit removes the container and the
+volume, says so, and provisions again on a fresh volume:
+
+```text
+[testkit] Testkit is replacing database volume <volume>, because Local database container <container> exited while starting with exit code 1: <the server's last FATAL line, redacted>. The new volume starts with an empty cluster, so its template is rebuilt.
+```
+
+It does this at most once per provisioning. A server that exits on a volume
+created in the same provisioning fails it, with its exit code and its last FATAL
+or PANIC line. A server that is slow to start, or that refuses connections while
+it starts, is left alone, and readiness still times out after 60 seconds.
+
+Testkit also creates new clusters with `initdb --no-sync`, through
+`POSTGRES_INITDB_ARGS`. On Docker Desktop that saves about a second of a cold
+start. A cluster that a host crash catches right after `initdb` is now one of
+the clusters the rebuild replaces.
+
+## Admin statements on local Postgres time out
+
+A statement Testkit runs on its local Postgres container, such as
+`CREATE DATABASE` or `DROP DATABASE`, waited indefinitely when another session
+held a lock it needed. A `psql` client that didn't return after its statement
+kept the command waiting too. Neither printed anything
+([gfdelarue/testkit#468](https://github.com/gfdelarue/testkit/issues/468)).
+
+Each admin statement now runs with a 60-second `lock_timeout` and a 10-minute
+`statement_timeout`. Testkit stops its `docker exec` 30 seconds after the
+statement timeout, then ends the client and its session inside the container,
+as it already did after a cancellation. The error names the statement and its
+limits. When the statement ran long enough to be checked, the error also says
+what it was last seen doing, such as the lock it was waiting on and the session
+that held it.
+
+## The local database keeps its data in Testkit's volume, whatever the image's tag
+
+Testkit chose where to mount the local PostgreSQL container's named volume from
+a major version it read out of the image's tag
+([gfdelarue/testkit#481](https://github.com/gfdelarue/testkit/issues/481)). A tag
+that only looked like a version, such as `team-db:33ab` on a PostgreSQL 16
+image, got PostgreSQL 18's mount, `/var/lib/postgresql`. The cluster then lived
+in an anonymous volume that Docker mounted at the image's
+`/var/lib/postgresql/data`, so a recreated container started empty, and
+removing the container deleted the data. A PostgreSQL 18 image whose tag read
+below 18 got the older mount, and its container never became ready.
+
+Testkit now reads the image's own configuration with `docker image inspect`:
+its `PGDATA` and the volumes it declares. It pulls the image first when it
+isn't local. The named volume takes the place of the deepest declared volume
+that holds `PGDATA`, or mounts at `PGDATA` when no declared volume holds it.
+Testkit stops with an error rather than guess when the image sets no `PGDATA`,
+sets a relative one, or declares a volume inside it:
+
+```text
+Testkit cannot tell where image team-db:33ab keeps its PostgreSQL data: its configuration sets no PGDATA. Use an image built from the official postgres image, or set PGDATA in it.
+```
+
+A container that an earlier version mounted in the wrong place is recreated on
+a fresh named volume, so its database starts empty once.
+
+## A crashed run no longer blocks the next one after Windows reuses its PID
+
+On Windows, once a crashed run's processes had ended and Windows had given one
+of their process IDs to another process, the next run, `testkit cleanup` and
+`testkit run stop` failed:
+
+```text
+Application service process creation identity no longer matches pid 6944; refusing unsafe cleanup
+```
+
+The crashed run's record stayed until the unrelated process exited
+([gfdelarue/testkit#485](https://github.com/gfdelarue/testkit/issues/485)).
+Linux and macOS could hit the same error when the new process led its own
+process group.
+
+A recorded process whose PID now belongs to a different process has ended.
+Testkit leaves the new process alone and clears the crashed run's record. It
+still refuses to signal a live process whose creation identity it can't read.
+
+## The local database is ready when it accepts a connection from the host
+
+Testkit waited for its local Postgres container by running `pg_isready` inside
+it through `docker exec`, up to about once a second. On Docker Desktop each
+exec costs about 0.17 s of the WSL VM's CPU, and a cold start ran seven or
+eight of them.
+
+Testkit now waits by connecting to the container's published port on
+`127.0.0.1`, the address the services use, with the startup, authentication
+and `SELECT 1` of its database checks. It tries every 10 to 100 ms, gives each
+attempt 5 seconds, and still gives up after 60 seconds. A port that accepts the
+connection but doesn't answer as PostgreSQL, as Docker Desktop's port proxy
+does before Postgres listens, and a server that rejects the startup, such as
+one still starting, don't count as ready. When readiness times out, the error
+gives the last check's reason. A server that refuses the configured user or
+password, or has no `postgres` database, fails at once with that reason instead
+of waiting out the 60 seconds.
+
+Testkit waits on the published port only when it reaches Docker through a
+local socket or named pipe, whether `DOCKER_HOST`, `DOCKER_CONTEXT` or the
+configuration's current context names it. A daemon behind a `tcp://` or `ssh://`
+endpoint, at any address, can publish ports in another network namespace, as
+Docker-in-Docker does, so readiness there still runs `pg_isready` in the
+container. Through a local socket, Testkit runs that probe as well, at most once
+a second, when nothing answers on the port: the connection is refused, an
+attempt gets no reply, or what answers isn't PostgreSQL. After 10 seconds it
+probes on any failed attempt. When the probe finds the server ready, readiness
+succeeds. A socket that forwards to a daemon in another namespace works this
+way. As before, the database URL Testkit gives tests names `127.0.0.1` and the
+published port, which such a daemon doesn't serve on this host.
+
+## Managed Chromium runs with its sandbox on
+
+Playwright starts Chromium with `--no-sandbox` unless `chromiumSandbox` is
+`true`, and Testkit didn't set it. Every UI file ran without Chromium's
+sandbox, so a compromised renderer ran with the user's access to the checkout,
+SSH keys and env files
+([gfdelarue/testkit#190](https://github.com/gfdelarue/testkit/issues/190)).
+
+Testkit now sets `chromiumSandbox: true` in the Playwright config it generates
+and in the managed test's launch options. It holds whether a file imports
+`test` from `@playwright/test`, `playwright/test`, or
+`@gfdelarue/testkit/playwright`, when a project or a file sets other
+`launchOptions`, and for browsers that `globalSetup` and `globalTeardown`
+launch. An explicit `chromiumSandbox: false` in `playwright.config.ts` or in a
+file's `test.use` still turns it off.
+
+On a host that can't start the sandbox, such as root in a container, Chromium
+now fails to launch. Testkit reports that as an infrastructure failure and puts
+the opt-out first:
+
+```text
+Testkit starts managed Chromium with its sandbox on, and this host can't start it. To run without the sandbox here, set use.launchOptions.chromiumSandbox to false in playwright.config.ts, or in one file with test.use.
+```
+
+[Browser launch options](engines.md#browser-launch-options) shows how to turn
+it off for one host.
+
+## Managed Chromium no longer crashes at launch under older glibc
+
+Chromium sometimes crashed with `SIGSEGV` in `libdbus` during
+`browserType.launch` on Ubuntu 24.04, and the UI file failed
+([gfdelarue/testkit#453](https://github.com/gfdelarue/testkit/issues/453)).
+At startup, Chromium sets `FC_FONTATIONS` on a worker thread while its D-Bus
+thread reads the environment. glibc before 2.41 can move the environment and
+free the old copy under that reader, unless the distribution backported the
+fix. Ubuntu 24.04's glibc 2.39 hasn't.
+
+Playwright files now run with `DBUS_SESSION_BUS_ADDRESS=disabled:` and
+`FC_FONTATIONS=1`, the values Chromium sets itself. Chromium finds both already
+set and doesn't grow the environment at startup.
+
+- Tests, and any process or browser they start, see both variables.
+- A `launchOptions.env` in `playwright.config.ts` or in a file's `test.use`
+  gets both added, under its own values. A browser that a test or global setup
+  launches directly with its own `env` gets only what that sets.
+- The Playwright engine reserves both keys, so a group's environment can't set
+  them.
+
+## A capture records only requests sent to its url
+
+An HTTP capture recorded a request to any path outside `/__testkit__/` from any
+caller that found its port, whatever its `Host` header
+([gfdelarue/testkit#193](https://github.com/gfdelarue/testkit/issues/193)).
+Another local user, or a web page through DNS rebinding, could add captures and
+push real ones out of the list.
+
+A capture's `url` now holds an unguessable segment, as its `requestsUrl`
+already did, such as `http://127.0.0.1:41000/__testkit__/<segment>/webhook`.
+The capture records a request only when its path starts with that segment, and
+answers any other path with 404. A captured request's `path` leaves the segment
+out, so a test that checks `path` still sees `/webhook`. The capture answers
+403 to a request whose `Host` header isn't `127.0.0.1:<port>`. Testkit keeps
+the new segment out of the run's records. A capture's `path` may no longer
+contain `.` or `..` segments, because a client would resolve them against the
+`url` and drop the segment.
+
+If your product builds a URL from the capture's origin, give it the whole `url`
+and append to that instead. [Handles](../README.md#handles) describes the
+capture.
+
+## Tokens, temporary files, results and Git calls are harder to reach
+
+These changes keep another bug, or another user on a shared host, from reaching
+what Testkit handles
+([gfdelarue/testkit#196](https://github.com/gfdelarue/testkit/issues/196)).
+
+- On Linux and macOS, `.testkit` is now private to its owner. Loading the
+  config and starting a run create it with mode `0700`, and any command that
+  resolves the product removes group and other access from an existing one the
+  user owns. Sealed objects are `0400` instead of `0444`. Another user, such as
+  a host user reading results that a container wrote as root, now needs the
+  owner to share them. On Windows, `.testkit` keeps the access its parent
+  folder grants.
+- Services and setup steps no longer inherit `GITHUB_TOKEN`, `GH_TOKEN` or
+  `GITEA_TOKEN` from Testkit's environment. Testkit reads them itself for
+  known-failure checks and bug filing. A service that needs one declares it in
+  its `env`.
+- Lint's `git ls-files` and bug filing's `git diff` no longer run the command
+  that the repository's `core.fsmonitor` names. Run fingerprinting already
+  turned it off.
+- The `testkit-engine-*` directories that Testkit creates in the system's
+  temporary directory, which hold each browser profile, now record their
+  owner. The first engine invocation of a Testkit process removes those whose
+  owner has exited, such as after a crash. On Linux it only removes those from
+  its own PID namespace, so a container that shares the temporary directory
+  with its host leaves the host's alone, and the reverse. Directories from
+  earlier versions have no record and stay.
+- On Linux, a recorded process identity now includes the boot ID, so after a
+  reboot a run or lock record no longer matches an unrelated process that got
+  the same PID and start time. A record from an earlier Testkit still matches on
+  start time alone, as before.
+
+## A Playwright companion decides a Next app's support files
+
+In a Next application, `testkit typecheck` added `next-env.d.ts`,
+`tests/playwright-fixtures`, and the generated types under `.next` and
+`.next-testkit` to the Playwright program even when a `tsconfig.playwright.json`
+left them out or excluded them
+([gfdelarue/testkit#108](https://github.com/gfdelarue/testkit/issues/108)). A
+fixture the companion excluded, or another service's stale build output, could
+then fail the check. With a companion, the program now holds the discovered
+suites, the Playwright config, the files the companion selects, and what they
+import. List the Next files the suites need in the companion. Without a
+companion, the Next defaults stay.
+
+## `typecheck` checks test files outside the tsconfig's `rootDir`
+
+The generated programs took `rootDir` and `outDir` from the tsconfig they
+extend. A test file outside that `rootDir`, such as `tests/sum.test.ts` beside a
+tsconfig whose `rootDir` is `src`, failed `typecheck` with `TS6059` before any of
+its types were checked. TypeScript 6 also defaults `rootDir` to the directory of
+the tsconfig it compiles, which is Testkit's generated one, so under TypeScript 6
+every test file failed when the tsconfig set `outDir` and no `rootDir`
+([gfdelarue/testkit#407](https://github.com/gfdelarue/testkit/issues/407)).
+
+The generated configs now clear the inherited output paths, `outDir`,
+`outFile`, `declarationDir`, `sourceRoot`, and `mapRoot`, and the test programs'
+configs clear `rootDir` too. TypeScript holds every file to an explicit
+`rootDir`, and when an output path is set without one, TypeScript 6 uses the
+tsconfig's own directory. With both cleared, it holds test files to neither. When the tsconfig sets `incremental` or `composite`,
+Testkit now writes its build info beside the generated config in
+`.testkit/_typecheck`, instead of under the repository's `outDir` or into the
+repository's own build info file.
+
+## A run stops its retained database when it releases it
+
+A run that provisioned the local Postgres container now stops it as part of releasing the run's resources, after it drops the run's task databases. It stops the container by the name in `.testkit/_infra/local-postgres` and lists no Docker resources to do it. A container that is already stopped counts as released.
+
+If that container no longer exists, the release fails, and so does the run, with `Could not stop the retained database container <name>: it no longer exists`. Before, a successful run stopped the container later, during finish-time maintenance.
+
+## A run leaves Docker resources to `testkit cleanup`
+
+A run's start and its post-run maintenance no longer list Docker resources, remove unreferenced containers and volumes, or stop the retained database. Provisioning and release still create, replace, and remove what the run needs, and release stops the retained database by name. `testkit cleanup --resources` removes the product's unreferenced containers and volumes, and stops an idle retained database, as before.
+
+A run's start still removes stale manifests of runs whose processes have ended, and post-run maintenance still removes other runs' stale manifests.
+
+`.testkit/results/maintenance.json` no longer has the `removed`, `skipped` and `stopped` lists, since maintenance no longer removes or stops Docker resources.
+
+## `run.json`'s storage inventory covers only `.testkit/results`
+
+`run.json`'s `storage.testkit` inventory now covers only `.testkit/results`, and its `root` is `.testkit/results`. A run no longer walks the database and graph caches to build it. When Testkit can't read part of `.testkit/results`, `run.json` records `"storage": { "testkit": { "status": "failed", "error": "…" } }` and the run keeps its result.
+
+## Testkit's own clients of the owned database run inside its container
+
+Testkit used to start a client container, from `postgres:16` or
+`TESTKIT_POSTGRES_CLIENT_IMAGE`, to declare roles and take schema snapshots in
+the database it owns. Before a snapshot, it also asked the server for its
+version to choose a `pg_dump` image. Both now run inside the database container
+with `docker exec`, using the database image's own `psql` and `pg_dump`, as
+Testkit's admin queries already did. They no longer start a container
+([gfdelarue/testkit#139](https://github.com/gfdelarue/testkit/issues/139)).
+
+This changes two things:
+
+- `TESTKIT_POSTGRES_CLIENT_IMAGE` no longer applies to those operations. Your
+  SQL, in `sql-file` steps and `database-steps` helpers, still runs in a client
+  container from that image, and so do clients of external databases.
+- The `image` that `database.postgres()` uses must include `psql` and
+  `pg_dump`, and its `pg_dump` must be at least as new as its server.
+  Otherwise the operation fails with a message that names the image:
+
+  ```text
+  pg_dump in the database image <image> is older than its server: pg_dump: error: aborting because of server version mismatch
+  ```
+
+## Redaction recognises more secret names, API key headers, and bearer tokens
+
+Sealing, filed bugs, and error messages kept a secret under a key such as
+`STRIPE_KEY`, `GITHUB_PAT`, `SMTP_PASS`, or `SENTRY_DSN`, and the value of an
+`X-Api-Key` header or an `apiKey` member. They kept a bearer token outside an
+`Authorization` header, and a header that a trace stores as a
+`{"name": …, "value": …}` pair
+([gfdelarue/testkit#188](https://github.com/gfdelarue/testkit/issues/188)).
+
+- A key whose last word is `PASS`, `PASSWD`, `PAT`, `DSN`, `CREDENTIAL`, or
+  `CREDENTIALS` now names a secret, as does a key whose last two words are
+  `API` and `KEY`. An upper-case key whose last word is `KEY` names a secret
+  unless the word before it is `PUBLIC` or `PUBLISHABLE`. A run's environment
+  values under these keys are redacted as other secrets are, so a value under
+  a key such as `CACHE_KEY` is now redacted too. `PATH`, `BYPASS`, and
+  `cacheKey` still name no secret.
+- `Api-Key` and `X-Api-Key` header values are redacted like `Authorization`.
+- The token after `Bearer` is redacted wherever it appears when it is at least
+  eight characters long and holds a digit, so prose such as
+  `Bearer authentication` stays.
+- In a `name` and `value` pair, the value is redacted when the name names a
+  secret, such as `Authorization` or `Cookie`. When `testkit run` stores a
+  result, every value in a `cookies` array is redacted too.
+
+`docs/bug-records.md` lists the rules.
+
+## Redaction recognises more secret forms and fewer ordinary words
+
+Sealing, filed bugs, and error messages kept these secrets
+([gfdelarue/testkit#215](https://github.com/gfdelarue/testkit/issues/215)):
+
+- A `privateKey`, `signingKey`, or `encryptionKey` value.
+- A single-quoted member under any secret name other than `password`, as in
+  `{'token': '…'}`.
+- The strings in a JSON array or object under a secret name, as in
+  `{"tokens":["…"]}`. `session` and `auth` settings stay, and an object with a
+  `name` and a `value` member loses only its `value` and members under a secret
+  name.
+- A value after `password:`. A quoted value is redacted anywhere, and an
+  unquoted one when it is the one word of a line that it starts. A single word
+  of letters stays, so `password: 'Required'` and `password: required!` stay.
+- A URL password that holds a `/`, a URL password in a directory name, such as
+  `postgres:/app:…@db`, and a token of at least 20 characters with a digit that
+  stands alone as the user of an `http` or `https` URL, as in
+  `https://ghp_…@github.com`. A resolved URL's password or token is redacted
+  wherever it appears, too.
+- A secret right after another one with no space between them, as in
+  `PASSWORD=…"Cookie: sid=…`.
+- A URL password that output cut short. The first line of a task's output, as
+  an artifact summary or a failure message, is redacted before Testkit cuts it.
+
+They redacted these values that are not secrets:
+
+- A value under `author`, `authority`, or `GIT_AUTHOR_NAME`, and under
+  `sessionCount`, `SESSION_TIMEOUT`, and other keys where `SESSION` comes before
+  a count or a time. A run's environment no longer redacts every occurrence of
+  `GIT_AUTHOR_NAME`'s value either.
+- A path assigned to a key that ends in `_URL`, such as
+  `AUTH_URL=/oauth/authorize`. A `token=` parameter in the path is still
+  redacted. A JSON member such as `{"AUTH_URL":"/oauth/authorize"}` is still
+  redacted whole.
+- The text after a quoted header value that ends in an escaped backslash.
+
+`testkit bug record` now rejects a record whose reproduce command holds text
+that Testkit redacts, as `testkit bug file` already did. Before, a Vitest
+project name such as `TOKEN=…` passed `bug record` and stopped `bug file`.
+
+## Sealing redacts inside traces and other compressed files
+
+Sealing didn't look inside `.zip` and `.gz` files. A Playwright `trace.zip`
+recorded with `use.trace` kept form values, `Authorization` and `Cookie`
+headers, and session cookies in the sealed run, compressed, while the run's
+own copies of the same output were redacted
+([gfdelarue/testkit#187](https://github.com/gfdelarue/testkit/issues/187)).
+
+Sealing now reads `.zip` and `.gz` files, and files that start like one. It
+redacts each text entry with the same rules as a text file and writes the
+archive again, so the trace is kept and opens in the trace viewer with
+`<redacted>` in place of the secrets. A text entry is valid UTF-8 without binary
+control characters, or UTF-16 with a byte order mark. Entries it doesn't change
+keep their bytes. A ZIP with nothing to redact and no comment or extra fields
+stays byte for byte, and a gzip file is always written again without the name
+and comment in its header.
+
+- The bundle's redaction counts include the redacted archives: each counts as
+  a redacted file, its text entries count as scanned files, and its
+  replacements count too.
+- Sealing refuses an archive it can't redact: one it can't read, such as an
+  encrypted, ZIP64, damaged, or larger than 512 MiB archive, one whose binary
+  entry holds a value the run resolved, in UTF-8 or UTF-16, or one that holds
+  another archive.
+- Sealing removes each file it refuses, an archive or a binary file that holds
+  a resolved value, from the run's results. It fails the run with the reasons,
+  then seals the rest and records them. `testkit stop`, `testkit cleanup` and
+  the next run do the same when they seal a run that a killed process left.
+  Before, a refused run left its results in place, and those commands sealed
+  them without the values the run had resolved, the refused file included.
+- Screenshots and video, in a trace or on their own, aren't redacted.
+
+## Sealed bundles leave out the product and home directories
+
+Sealing removed the product directory and host from `run.json`, but the rest of
+the bundle kept them
+([gfdelarue/testkit#189](https://github.com/gfdelarue/testkit/issues/189)). A
+failing Playwright file's stack in `run.json`, the native report's `sourceFile`
+and attachment paths, and a `trace.zip` held the absolute product path. A trace
+also held the home directory, in the path of the Chromium that Testkit manages.
+On a developer machine those paths name the user.
+
+Sealing now rewrites local paths in the run record, in every text file of the
+bundle, and in every text entry of a compressed file such as `trace.zip`. A
+path under the product directory becomes relative to it, and a path under the
+home directory starts with `~`. When sealing changes a file, by this rewrite or
+by redaction, the file's size and SHA-256 in `run.json` now describe the sealed
+file.
+
+## An exit code set after a plain file's tests fails the run
+
+When a reporter or global teardown set Vitest's exit code to 1 after a file's
+tests failed, Testkit read the run as those failed tests alone
+([gfdelarue/testkit#215](https://github.com/gfdelarue/testkit/issues/215)). A
+run whose failed tests all had issues passed the known-failure gate under
+`"allow"`, though something else had also failed it.
+
+Testkit now notes Vitest's own verdict when the tests end, and fails the run
+with a run error when a reporter or global teardown sets the exit code
+afterwards:
+
+```text
+FAIL run error
+    Vitest failed the run after its tests finished: exit code 1 while Vitest closed
+```
+
+When the tests passed, the run error says "after its tests passed", as it does
+for a coverage threshold, and each file keeps its own outcome. Before, such an
+exit failed every file as an unexplained infrastructure failure. A coverage
+threshold that falls short beside a failed test still counts as part of that
+failure, because a failed test leaves code unrun.
+
+## The known-failure gate stops missing three failures
+
+Three failures could pass the known-failure gate under `"allow"`, or never be
+known at all
+([gfdelarue/testkit#215](https://github.com/gfdelarue/testkit/issues/215)):
+
+- A plain Vitest file whose data-flow evidence was malformed lost that error
+  when one of its tests also failed. The file now fails with the evidence
+  error beside its failed tests, which an issue for the tests doesn't cover.
+- A Playwright spec that failed in another file, such as a dependency or
+  teardown project, now always counts as a failure that names no test, even
+  when the run reports nothing else about it.
+- On macOS and Linux, a test file whose name holds a backslash could not be
+  filed, and its failures were always new. A bug record now keeps the
+  backslash, and treats it as a separator only on Windows.
+
+Testkit reads the file a Playwright spec was loaded from out of its file
+suite's title, resolved against `rootDir`. It now checks, as the run begins,
+that each title names a file. If a Playwright release changes the title so that
+it names no file, the file fails with an error that says so, instead of
+reporting the spec under a path where no file exists.
+
+## Four Playwright failures stop passing as known
+
+Four Playwright failures could pass the known-failure gate under `"allow"`, or
+never be known at all
+([gfdelarue/testkit#215](https://github.com/gfdelarue/testkit/issues/215)):
+
+- A reporter in the Playwright config that ended the process with exit code 1
+  read as Playwright's own exit for failed specs, so an issue for those specs
+  covered the run. Testkit's reporter runs last and marks the run once every
+  reporter's hooks have returned. Without that mark, exit code 1 is a failure
+  that names no test. A `PLAYWRIGHT_SPECS_FAILED` failure carries
+  `reportersFinished` in its details. A run stored by an earlier release has no
+  such mark, so `testkit ci report` reads its exit code 1 the same way. A
+  Playwright config that sets `PW_TEST_REPORTER` now fails, because Playwright
+  runs that reporter after Testkit's.
+- A failed spec in a serial group, or a failed `beforeAll` hook, could stop
+  later specs of the selected file. Playwright reports those specs as skipped,
+  so an issue for the failed spec let the run pass. Each such spec is now a
+  detail of kind `playwright-not-run`, which names no test. A spec that the
+  file skips with `test.skip` or `test.fixme` is not one. A spec that a run
+  stops at `maxFailures` gets the same detail, though that run already failed
+  the gate on the error Playwright reports when it stops.
+- Under `failOnFlakyTests`, a spec that passed only on a retry failed the run
+  as a failure that names no test, so no issue could cover it. It is now a
+  `playwright-spec` detail with its title, and an issue for that title covers
+  it.
+- A file whose every spec came from one helper with another extension, such
+  as a `.ts` file and a `.mjs` helper, ran no specs and failed with
+  `PLAYWRIGHT_NO_TESTS`. Playwright moves the file suite's location to that
+  helper, and Testkit selected the file by location. Testkit now selects it
+  with `--grep` on its file suite's title, the file's path from `rootDir`, so
+  its specs run.
+
+## Tracker tokens go only where they belong
+
+Five gaps in how Testkit talks to GitHub and Gitea are closed
+([gfdelarue/testkit#215](https://github.com/gfdelarue/testkit/issues/215)):
+
+- `GITEA_TOKEN` names no host, so Testkit sent it to whichever Gitea host the
+  checkout's remote named. Outside a Gitea Actions job, Testkit now sends it
+  only to the host that `GITEA_SERVER_URL` names. Set
+  `GITEA_SERVER_URL=https://gitea.example.com` beside `GITEA_TOKEN`, or keep the
+  token in a Git credential helper, which is bound to its host already.
+- Testkit sends a token only over HTTPS. A plain `http://` remote or repository
+  URL now fails with a message that says so, unless it names a loopback host.
+- `testkit ci report` updated the first pull request comment that started with
+  its marker, whoever wrote it. It now updates only a comment written by its
+  token's user. It takes that user from the commit status it has just set, or
+  asks the tracker when the status names no one. When neither names the user,
+  it posts a new comment each time rather than edit one.
+- When the tracker created an issue without its labels, `testkit bug file`
+  failed and left the issue open. It now closes that issue first, and says
+  whether it could.
+- `testkit ci report` now leaves out of the summary and the commit status any CI
+  run or issue link that holds credentials or a secret.
+
+## `bug file` asks the remote for the commit, and closes a duplicate it filed
+
+Two gaps in `testkit bug file` are closed, along with a Gitea problem that the
+second turned up ([gfdelarue/testkit#215](https://github.com/gfdelarue/testkit/issues/215)):
+
+- It took the commit as pushed when a local remote-tracking branch held it,
+  even after the remote rewound or deleted that branch. It now lists the
+  remote's branches with `git ls-remote` and checks their tips, fetching a tip
+  the checkout lacks. A renamed branch still counts. When a remote doesn't
+  answer, or a shallow clone's history stops before Git can tell, Testkit
+  stops and says why, unless another remote has the commit.
+- Two checkouts that filed the same test at once could each create an issue,
+  because neither saw the other's in the open issues it read first. After it
+  creates the issue, Testkit now records it in the draft and reads the open
+  issues again. GitHub can list a new issue late, so it waits up to 7.5 seconds
+  for its own to appear, and on GitHub it then reads again two seconds later.
+  It says so when its issue never appears. When an older open issue covers the
+  same test, Testkit closes its own, comments with the older one's number, and
+  stops as it would have if it had seen that issue first.
+- On Gitea, two checkouts creating the same label at once made two labels of
+  that name, and Gitea's label filter then returned no issue. Testkit now
+  deletes a label it created when an older one has the same name, and lists
+  the open bugs without the filter when the repository already has two labels
+  of one name.
+
+## A failed database drop no longer hides the error before it
+
+When a template step failed, Testkit dropped the half-built template database
+before it reported the step's error. If the drop failed too, the run reported
+only the drop's error. A run interrupted during a template `sql-file` step,
+after its Docker connection had closed, reported only `Cannot connect to the
+Docker daemon`, and nothing about the step or the Postgres client container it
+left running
+([gfdelarue/testkit#400](https://github.com/gfdelarue/testkit/issues/400)). A
+`resetBefore` file whose database clone failed had the same problem.
+
+The run now reports both errors, and names the database Testkit couldn't drop:
+
+```text
+Dockerized Postgres client was cancelled
+Postgres client container cleanup failed: Cannot connect to the Docker daemon at tcp://127.0.0.1:36893. Is the docker daemon running?
+Dropping the template database tk_tpl_api_f14201a8f06684a4 also failed: Cannot connect to the Docker daemon at tcp://127.0.0.1:36893. Is the docker daemon running?
+```
+
+## A failure block prints each message once, without color codes
+
+A failed Playwright assertion printed its message twice in the run's failure
+block, once for the test and once for Playwright's overall failure. Both copies
+kept Playwright's ANSI color codes, even with `NO_COLOR` set, and both were cut
+off mid-word
+([gfdelarue/testkit#220](https://github.com/gfdelarue/testkit/issues/220)).
+
+The block now strips the engine's color codes, prints each distinct message
+once, and says how much of a long message it left out. It cuts at a space when
+one falls within 40 characters of the limit, and otherwise at the limit, without
+splitting a character:
+
+```text
+[1/1] FAIL app ui __testkit__/failure.ui.testkit.ts 3s
+    Error: expect(locator).toHaveText(expected) failed Locator: locator('#status') Expected:
+    "expected marker" Received: "actual" Timeout: 1000ms Call log: - Expect "toHaveText" with
+    timeout 1000ms - waiting for locator('#status') 5 × locator … (69 more characters)
+```
+
+The run's failure details keep the full message. The block's second and later
+messages are now redacted like the first.
+
+## Long command output no longer warns about listeners
+
+Commands write their text output one `log` call at a time, usually a line each,
+and each call added an `error` listener to stdout until its write finished. More
+than ten calls in a row, as in a longer `discover`, `status`, `artifacts`, or
+`workspace` listing, printed `MaxListenersExceededWarning` on stderr
+([gfdelarue/testkit#221](https://github.com/gfdelarue/testkit/issues/221)).
+Testkit now adds one listener.
+
+## Testkit's runtime keys take the `@gfdelarue/testkit` prefix
+
+Three keys that Testkit's modules use to share state inside an engine process
+had kept the old `elench.testkit.*` names
+([gfdelarue/testkit#99](https://github.com/gfdelarue/testkit/issues/99)). They
+are now `@gfdelarue/testkit/node-vitest-runtime-input/v1`,
+`@gfdelarue/testkit/node-vitest-runtime-task/v1` and
+`@gfdelarue/testkit/engine-process-environment-overlay/v1`.
+
+There is nothing to do when you upgrade. Testkit writes the keys afresh for
+every invocation, stored runs don't hold them, and the CLI refuses to run when
+the product's installed Testkit is a different version, so both sides always
+come from one release. Code that read these undocumented globals directly needs
+the new names. The `com.elench.testkit.*` Docker labels are unchanged.
+
+## `bug file --batch` files several bugs in one call
+
+`testkit bug file --batch` files the drafts it is given, or every unfiled draft
+when it is given none
+([gfdelarue/testkit#254](https://github.com/gfdelarue/testkit/issues/254)). It
+checks the checkout once, runs the drafts' tests together, and reports filed,
+already filed, rejected, or not filed for each draft. It exits non-zero when
+any draft is rejected or not filed, and a rerun files only what is left. A
+campaign's pull request summary now suggests it for the bugs that are recorded
+and not filed.
+
 # 0.6.0
 
 0.6.0 removes the `contract` and `data` test types and legacy service