slicetest 0.8.0 → 0.10.0

package/README.md

@@ -2,6 +2,8 @@
 
 [![CI](https://github.com/revo1290/slicetest/actions/workflows/ci.yml/badge.svg)](https://github.com/revo1290/slicetest/actions/workflows/ci.yml) [![npm](https://img.shields.io/npm/v/slicetest)](https://www.npmjs.com/package/slicetest) [![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
 
+[Report a bug / 不具合報告](https://github.com/revo1290/slicetest/issues/new/choose) · [Contributing / 貢献ガイド](CONTRIBUTING.md)
+
 Tests that sit between unit tests and end-to-end tests, for apps written in any language or framework.
 
 slicetest starts your app as a real process, points it at a real Postgres (or MySQL, or SQLite) and at stub servers for the services it calls, and lets you check all three sides in one scenario:
@@ -21,15 +23,37 @@ scenario("creating a poll stores it and notifies Slack", async ({ http, db, stub
 });
 ```
 
-No browser, no mocked database, no hooks inside your app. The app only has to read its port, database URL and outbound base URLs from environment variables.
+No browser, no mocked database, no hooks inside your app. The app only has to read its port, database URL and outbound base URLs from environment variables. (State the app keeps in memory is the one thing that may need your help: [state between scenarios](docs/state-isolation.md).)
+
+## Why, and when not to use it
+
+Testing an API against a real server, a real database and stubbed third parties is an established approach, and most stacks can do it already ([Spring Boot](https://docs.spring.io/spring-boot/reference/testing/index.html) starts a real server in tests and [integrates with Testcontainers](https://docs.spring.io/spring-boot/reference/testing/testcontainers.html); [Playwright has API testing](https://playwright.dev/docs/api-testing); [WireMock](https://wiremock.org/docs/simulating-faults/) stubs HTTP services and simulates faults). slicetest doesn't claim something only it can do. It bundles the parts you'd otherwise assemble per project, for any language the app is written in: starting the app, the database and the stubs, resetting them between scenarios, checking the three boundaries in one scenario, and explaining a failure.
+
+The usual labels mix up three separate questions: is anything mocked, is a real server running, and how much does the test cover. Side by side:
 
-## Why
+| Approach | Mainly checks | Strong at | Limits |
+|---|---|---|---|
+| Function / class tests | Logic, boundaries, exception branches | Control over internal dependencies | Real connections and SQL need a separate check |
+| API integration tests you already have | HTTP, the app, a real DB, calls to external services | Fit with your existing framework | Starting, resetting and diagnosing is yours to build, depending on the setup |
+| slicetest | The same API-level tests, plus environment management and diagnostics | Several checks in one scenario; the same scenarios for any language | Reaching into the app's internals, or a dependency it doesn't support, takes extra design |
+| Browser E2E | From UI actions through the main flows | What happens in the browser | Poor at controlling fine-grained internal state |
+
+What slicetest keeps real: the HTTP server, the SQL and the migrations. What it replaces: services you don't own, with stubs. With OpenAPI specs it also checks that your stubs' replies could come from the real service, within what the spec documents ([what each check does and doesn't guarantee](docs/guarantees.md)).
+
+Whether it's worth adopting depends on what you have:
+
+| Team or use | Verdict |
+|---|---|
+| No test setup for the API, the DB and external calls | A small trial fits |
+| Several languages, and you want one way to test all of them | A comparatively strong candidate |
+| Failures take long to investigate in logs and the DB | Worth measuring what the combined output saves |
+| Spring (or similar) integration tests that already run stably | Don't plan to replace them; measure what slicetest would add |
+| Many branches of internal logic to cover | Unit tests first |
+| The UI, or a real identity provider or external API | Browser E2E, or a check against the real service, alongside |
 
-- **Unit tests** mock the database and the network, so broken SQL, migrations and request payloads slip through.
-- **End-to-end tests** drive a browser against a deployed stack. They are slow and hard to make deterministic.
-- **slicetest** keeps the real HTTP server, the real SQL and the real migrations, and replaces only the things you don't own: third-party APIs. With OpenAPI specs, it also checks that those replacements behave like the real thing.
+Not measured yet: whether it is faster than other setups, cheaper to maintain, or stable over long use on real projects. The one comparison made so far (the same nine checks on a Spring Boot app, written with JUnit + Testcontainers + WireMock and with slicetest; run times, flaky runs, injected faults, and what each failure shows) is in [examples/comparison](examples/comparison/README.md), with its limits: one machine, one small app, both suites by one author. The speeds quoted below are narrow measurements, with their conditions.
 
-The database is reset between scenarios with a single `TRUNCATE ... RESTART IDENTITY CASCADE` (about 1.5 ms). The app keeps its connections, so this works with any driver or ORM. Resetting by dropping and re-creating the database takes about 130 ms, and it crashed some apps when their pooled connections were cut.
+The database is reset between scenarios with `TRUNCATE ... RESTART IDENTITY CASCADE` over every table, and the app keeps its connections, so this works with any driver or ORM. In the spike that chose this (`TRUNCATE polls RESTART IDENTITY`: one table, no `CASCADE`, 20 runs, median, a Postgres container on a laptop) the reset took 1.5 ms; dropping and re-creating the database took about 130 ms and crashed some apps when their pooled connections were cut. That is the reset step alone, not the time of a test run, and it wasn't measured with more tables. Since the table list is now read at every reset, a later check on three small tables (same machine, 300 resets) took 1.4 ms for the `TRUNCATE` alone and 2.0 ms with the list. **Of the app's own state, only what is in the database is reset by default**: whatever the app keeps in memory or does in the background survives into the next scenario unless you say otherwise ([state between scenarios](docs/state-isolation.md)).
 
 ## Quick start
 
@@ -38,19 +62,19 @@ npx slicetest init   # detects your stack, writes slicetest.config.yaml and a fi
 npx slicetest        # starts Postgres, migrates, starts your app, runs scenarios/*.scenario.yaml
 ```
 
-`init` recognises Node (`npm start`, or Bun, pnpm or Yarn from the lockfile), Deno, Phoenix, Django, FastAPI, Flask, Rails, Laravel, Symfony, plain PHP, Spring Boot, ASP.NET Core, Go and Rust apps; Atlas, Prisma, Alembic, Django, Rails, Laravel, Doctrine, EF Core, Ecto, Drizzle, Knex and plain SQL migrations; and an `openapi.yaml`. If there's a `compose.yaml` / `docker-compose.yml`, its database service sets `db.image` (and `db.engine: mysql` for MySQL or MariaDB), and Redis, Valkey, Mongo, Elasticsearch, MinIO, RabbitMQ and other services with a port become [`containers`](#containers-redis-search-s3-and-other-dependencies), with a reset command where one is known and the usual variable (`REDIS_URL`, `S3_ENDPOINT`, …) passed to the app. A mail catcher there (Mailpit, MailHog, MailDev, smtp4dev, …) or a mail library in the dependencies turns on [`mail`](#mail-catch-what-the-app-sends). SQLite is picked up from Prisma's provider, Rails' `database.yml`, Django's settings or a SQLite driver, with `DATABASE_URL` in the form the framework reads (`file:…`, `sqlite3:…`). And third-party API URLs in `.env.example` (`STRIPE_API_BASE=https://api.stripe.com`) become stubs [recorded from that service](#recording-a-real-service), with the variable pointed at the stub, while local addresses, databases and your own URLs are left alone. Token issuer settings there (`OIDC_ISSUER`, `AUTH0_DOMAIN`, `JWKS_URL`, `JWT_AUDIENCE`, …) turn on [`auth`](#auth-a-real-openid-issuer-tokens-with-any-claims) and point at slicetest's issuer instead of becoming stubs. It lists every guess as a comment in the config so you know what to check.
+`init` recognises Node (`npm start`, or Bun, pnpm or Yarn from the lockfile), Deno, Phoenix, Django, FastAPI, Flask, Rails, Laravel, Symfony, plain PHP, Spring Boot, ASP.NET Core, Go (the main package at the root or in `cmd/<name>`) and Rust apps (both compiled once before the workers start); Atlas, Prisma, Alembic, Django, Rails, Laravel, Doctrine, EF Core, Ecto, Drizzle, Knex and plain SQL migrations (golang-migrate, goose, sqlx, Diesel and dbmate layouts in `migrations/`, `db/migrations/` or `sql/migrations/`); and an `openapi.yaml`. If there's a `compose.yaml` / `docker-compose.yml`, its database service sets `db.image` (and `db.engine: mysql` for MySQL or MariaDB), and Redis, Valkey, Mongo, Elasticsearch, MinIO, RabbitMQ and other services with a port become [`containers`](#containers-redis-search-s3-and-other-dependencies), with a reset command where one is known and the usual variable (`REDIS_URL`, `S3_ENDPOINT`, …) passed to the app. A mail catcher there (Mailpit, MailHog, MailDev, smtp4dev, …) or a mail library in the dependencies turns on [`mail`](#mail-catch-what-the-app-sends). SQLite is picked up from Prisma's provider, Rails' `database.yml`, Django's settings or a SQLite driver, with `DATABASE_URL` in the form the framework reads (`file:…`, `sqlite3:…`). And third-party API URLs in `.env.example` (`STRIPE_API_BASE=https://api.stripe.com`) become stubs [recorded from that service](#recording-a-real-service), with the variable pointed at the stub, while local addresses, databases and your own URLs are left alone. Token issuer settings there (`OIDC_ISSUER`, `AUTH0_DOMAIN`, `JWKS_URL`, `JWT_AUDIENCE`, …) turn on [`auth`](#auth-a-real-openid-issuer-tokens-with-any-claims) and point at slicetest's issuer instead of becoming stubs. It lists every guess as a comment in the config so you know what to check.
 
-## What you get that's hard to find elsewhere
+## What it bundles
 
 - **One scenario, three boundaries.** Assert on the HTTP response, the rows in the real database and the calls to third-party APIs in the same test, in any language the app is written in.
 - **`db.changes()`**: a diff of every row the scenario inserted, updated or deleted. `toEqual` on it catches writes you didn't expect.
-- **Stubs that can't lie.** Give a stub the provider's OpenAPI spec, and a canned reply the real service would never send fails the test. The run also lists which of the provider's operations the app depends on, flagging deprecated ones.
+- **Stubs checked against the provider's spec.** Give a stub the provider's OpenAPI spec, and a canned reply that contradicts what the spec documents (a status or body the real service doesn't list) fails the test. The spec has to be right and complete for this to mean much. The run also lists which of the provider's operations the app depends on, flagging deprecated ones.
 - **Whole-scenario snapshots.** `expect(await trace()).toMatchSnapshot()` pins the responses, the outbound calls and the database changes in one reviewable file, with dates and UUIDs masked.
-- **Record the real service once, replay forever.** Point a stub at the real API with `SLICETEST_RECORD=1`, or import a HAR file saved from the browser, commit the YAML it writes, and later runs are offline and deterministic.
-- **OpenAPI coverage** of your own API, per operation and status, across all scenarios, and `slicetest gen --uncovered` to scaffold scenarios for what's missing.
+- **Record the real service once, replay offline.** Point a stub at the real API with `SLICETEST_RECORD=1`, or import a HAR file saved from the browser, commit the YAML it writes, and later runs are offline and deterministic. A replay is the service as it was when recorded: re-record now and then to notice changes.
+- **OpenAPI coverage** of your own API: which documented responses (operation and status) some scenario produced, across all scenarios. It isn't code or business-rule coverage. `slicetest gen --uncovered` scaffolds scenarios for what's missing.
 - **Record instead of write.** `npx slicetest record` puts a proxy in front of the app: click through a flow, press Enter, and get a replayable YAML scenario with the stubs' answers, the responses, captured ids and the database changes.
 - **Readable in CI.** On GitHub Actions, failing YAML steps are annotated in the pull request on the line that failed, and the job summary shows the OpenAPI coverage table and a sequence diagram of each failed scenario.
-- **Diagrams that can't go stale.** `--diagrams docs/flows` writes a Mermaid sequence diagram of every scenario (app, stubs, mail, database), regenerated from what really happened on each run.
+- **Diagrams regenerated from each run.** `--diagrams docs/flows` writes a Mermaid sequence diagram of every scenario (app, stubs, mail, database), regenerated from what really happened on each run.
 - **Races on purpose.** `http.concurrently(10, ...)` and `toHaveStatuses({ 201: 1, 409: 9 })` turn "what if two people click at once" into a test against the real database.
 - **Mail as a fourth boundary.** `mail: true` catches the app's SMTP traffic in-process, decoded, with the links pulled out, so a sign-up test can follow the confirmation link.
 - **Real token verification, any user.** `auth: true` gives the app an OpenID issuer with a JWKS, so JWT checks stay on in tests, and scenarios mint tokens with any claims, including expired or foreign-signed ones.
@@ -61,7 +85,7 @@ npx slicetest        # starts Postgres, migrates, starts your app, runs scenario
 - **Hard-coded APIs, stubbed anyway.** `hosts: [api.github.com]` catches calls to URLs written in the code or built into a framework, over HTTPS, from Node, Python, Go, Ruby or the JVM, with no change to the app. Redirects to those hosts are followed to the stub, so OAuth logins run end to end.
 - **N+1 detection for any stack.** A wire-protocol proxy records the SQL the app runs, so query counts are asserted at the HTTP boundary, whatever the ORM or language.
 - **Postgres, MySQL or SQLite**, with the same scenarios and the same helpers on all three, plus Redis, MinIO or any other `containers` reset between scenarios.
-- **Fast resets.** `TRUNCATE` between scenarios (about 1.5 ms) with the app still running, and a cached migrated template, so the second run skips container start-up and migrations.
+- **Cheap resets.** `TRUNCATE` between scenarios with the app still running (about 1.5 ms in the spike above), and a cached migrated template, so the second run skips container start-up and migrations. For apps that need their process restarted or their in-memory state cleared too: `restart`, `reset` and `idle` ([state between scenarios](docs/state-isolation.md)).
 
 ## Install
 
@@ -131,11 +155,30 @@ export default defineConfig({
 1. **Once per run.** slicetest starts `postgres:17-alpine` and migrates a template database. Locally, the container is kept running and the migrated template is cached by the contents of your migrations, so the next run with unchanged migrations skips both steps (see `db.reuse`).
 2. **Once per worker.** It clones the template into the worker's own database.
 3. **Once per test file.** It starts the stub servers, your `services` and your app.
-4. **Before each scenario.** It truncates every table except migration bookkeeping tables (`atlas_schema_revisions`, `_prisma_migrations`, `alembic_version`, `django_migrations`, …) and extension-owned tables such as PostGIS's `spatial_ref_sys`, re-runs the seed, and clears the stubs, cookies and request history. If the app or a service crashed in the previous scenario, it is restarted.
-5. **After each scenario.** The scenario fails if the app or a service crashed, the app called a stub route you didn't register, or (with `openapi`) any traffic didn't match the spec.
+4. **Before each scenario.** It truncates every table except migration bookkeeping tables (`atlas_schema_revisions`, `_prisma_migrations`, `alembic_version`, `django_migrations`, …) and extension-owned tables such as PostGIS's `spatial_ref_sys`, re-runs the seed, and clears the stubs, cookies and request history. If the app or a service crashed in the previous scenario, or has `restart: scenario`, it is stopped before the reset and started after it.
+5. **After each scenario.** The scenario fails if the app or a service crashed, the app called a stub route you didn't register, or (with `openapi`) any traffic didn't match the spec. With `idle`, it waits for the app's background work to finish before those checks.
 
 Database names are unique per run, so several projects or CI jobs can share one Postgres server via `db.url`.
 
+### What is reset between scenarios, and what isn't
+
+| Reset before every scenario | Kept, unless you configure otherwise |
+|---|---|
+| Database rows (and the seed is re-applied), stubs, recordings, mail, the auth issuer, cookies and request history, containers that have a `reset` command | The app and `services` processes, so **anything the app holds in memory (caches, singletons, timers) and any background work in flight**; containers without `reset`; files, other stores and processes the app uses |
+
+So "the database is reset" does not mean every scenario starts from an identical app. An app that caches a query result serves it after the table was emptied, and a write scheduled by one scenario can land in the next one. Three settings deal with that, on `app` and on each of `services`:
+
+```yaml
+app:
+  restart: scenario                 # stop the process before the reset, start it after
+  reset: { path: /__test/reset }    # or: an endpoint that makes the app drop its caches and timers
+  idle:  { path: /__test/idle }     # and/or: an endpoint that answers 2xx once no background work is left
+```
+
+`reset` and `idle` are requests slicetest sends to the app itself, so the app has to serve them: register those endpoints only in test configuration (an environment variable the test config sets), never in production, since `reset` is an unauthenticated call that wipes state. `restart` costs a process start per scenario, which counts toward the test timeout, and the first scenario of a file restarts the app started moments before.
+
+Which one covers what, the order of everything that happens around a scenario, and what stays your job (files, workers, queues) is in [State between scenarios, and running in parallel](docs/state-isolation.md).
+
 ### When a scenario fails
 
 slicetest prints what happened during that scenario, next to Vitest's own error:
@@ -181,10 +224,10 @@ await http.submit(await http.get("/signup"), { button: "Sign up", fields: { emai
 await http.graphql("query Poll($id: ID!) { poll(id: $id) { title } }", { id: 1 });  // POST /graphql ({ path } for another)
 
 const admin = http.with({ headers: { authorization: `Bearer ${token}` } }); // shares cookies with http
-http.cookies.get("session");                                     // cookies persist within a scenario
+http.cookies.get("session");                                     // cookies persist within a scenario, until their Max-Age / Expires
 ```
 
-Requests may only go to the app under test; absolute URLs to other hosts are rejected. Defaults for every request can be set with `http: { headers, timeout }` in the plugin config. With `timeout` (ms), a request the app doesn't answer in time fails with its method and path instead of the whole test timing out, and a refused or dropped connection says the app isn't listening or closed it. A list in `query` repeats the parameter (`{ tag: ["a", "b"] }` → `?tag=a&tag=b`). With `follow`, cookies set by each redirect are kept (a login answering `302` with `Set-Cookie`), `303` and `301`/`302` turn into a `GET` as in browsers, and a redirect to another host is returned instead of followed. Cookies follow their `Path` as in browsers.
+Requests may only go to the app under test; absolute URLs to other hosts are rejected. Defaults for every request can be set with `http: { headers, timeout }` in the plugin config. With `timeout` (ms), a request the app doesn't answer in time fails with its method and path instead of the whole test timing out, and a refused or dropped connection says the app isn't listening or closed it. A list in `query` repeats the parameter (`{ tag: ["a", "b"] }` → `?tag=a&tag=b`). With `follow`, cookies set by each redirect are kept (a login answering `302` with `Set-Cookie`), a `303` turns the request into a `GET` and a `301`/`302` does so for a `POST` only (as fetch and browsers do; a `PUT` or `DELETE` is repeated with its body), `307`/`308` repeat it, and a redirect to another host is returned instead of followed. Cookies follow their `Path` as in browsers.
 
 #### Forms: `http.submit()`
 
@@ -203,7 +246,7 @@ await http.submit(await http.get("/settings"), {
 
 File inputs take a `File` (`fields: { avatar: new File([bytes], "me.png", { type: "image/png" }) }`, YAML `{ file: me.png }` relative to the scenario), and one left alone is sent as the empty file a browser sends; the form must be `multipart/form-data`. An image button (`<input type="image">`) sends its click position.
 
-`button` matches a submit button's text, `value`, `name` or `id`, and picks its form; its `formaction` / `formmethod` / `formenctype` apply. `fields` replace what the page had and must name existing fields, so a typo fails. When nothing matches, the error lists the page's forms and their buttons. In YAML, `submit:` uses the page the previous step requested.
+`button` matches a submit button's text, `value`, `name` or `id`, and picks its form; its `formaction` / `formmethod` / `formenctype` apply. `fields` replace what the page had and must name existing fields, so a typo fails. A select, radio or checkbox value must be one the page offers (a select option by its value or its label), as a browser can't send anything else; to send what the page doesn't offer, use a plain request with `http.form()`. Line breaks go out as CRLF, as browsers send them. When nothing matches, the error lists the page's forms and their buttons. In YAML, `submit:` uses the page the previous step requested.
 
 ### `db` — arrange and inspect the real database
 
@@ -294,7 +337,7 @@ stub("slack").calls("POST", "/hook");                         // recorded calls:
 
 Form-encoded bodies (Stripe, Twilio, OAuth token requests) are parsed into `call.form`, with bracket keys nested the way those providers read them: `metadata[order]=7&items[0][price]=p_1` is `{ metadata: { order: "7" }, items: [{ price: "p_1" }] }`. `form` conditions match a subset of it, and numbers and booleans compare with the strings sent. `multipart/form-data` bodies are read the same way, with each file as `{ filename, type, size, text }` (`text` for text, JSON, XML and CSV files), so `form: { avatar: { filename: "a.png", type: "image/png" } }` checks an upload the app passed on.
 
-Later routes win. `path` may also be a RegExp, and `method` may be `*`. Unanswered calls get a `501` and fail the scenario, with the closest route and why it didn't match (`stub.explain(call)`).
+Later routes win. `path` may also be a RegExp, and `method` may be `*`. A `*` segment matches the rest of the path, slashes included, as in Express (`/bucket/*` for `PUT /bucket/photos/2026/a.png`, `/v1/*/items`), and the part it matched is `call.params["*"]`. A path with a space or non-ASCII characters (`/検索/:word`) matches what the app sends percent-encoded. A query written into the path (`on("GET", "/search?q=tea")`, also in `calls()`, `toHaveReceived()` and YAML stubs) is a condition on those parameters, like `query: { q: "tea" }`; other parameters may come along. A list matches a repeated parameter's values in order: `query: { ids: ["1", "2"] }` for `?ids=1&ids=2`. Unanswered calls get a `501` and fail the scenario, with the closest route and why it didn't match (`stub.explain(call)`).
 
 ### OpenAPI contracts — for your app and for the services you stub
 
@@ -308,9 +351,9 @@ slicetest({
 });
 ```
 
-- **Your app's responses** must be documented (path, method, status) and match the schema.
-- **The app's requests to a stub** must match the provider's spec: required query parameters, content type and request body. Spec paths are matched with or without the server's base path (`/v1`).
-- **Your stubs' replies** must be something the real service could send. A stub that returns `200 { ok: true }` where the provider documents `202 { messageId }` makes tests pass against an API that doesn't exist; slicetest fails the scenario instead.
+- **Your app's responses** must be documented (path, method, status) and match the schema, and the headers the spec lists for a response must be sent when marked `required` and fit their schema (`X-Rate-Limit` as an integer, a `pattern`). `Content-Type`, `Accept` and `Authorization` are left to the content check.
+- **The app's requests to a stub** must match the provider's spec: required query parameters and header parameters, query values against their schema (`?limit=abc` for an integer, a value outside an `enum`), content type and request body. Spec paths are matched with or without the server's base path (`/v1`).
+- **Your stubs' replies** must be something the provider's spec says the real service sends. A stub that returns `200 { ok: true }` where the provider documents `202 { messageId }` makes tests pass against an API that doesn't exist; slicetest fails the scenario instead. A provider that differs from its own spec isn't caught.
 
 ```
 slicetest: traffic doesn't match the OpenAPI spec:
@@ -319,6 +362,8 @@ slicetest: traffic doesn't match the OpenAPI spec:
   stub mail reply (the real service wouldn't answer this way): POST /mail/send responded 200, which specs/mail.yaml doesn't document (documented: 202)
 ```
 
+These checks compare traffic with a spec; they don't prove behavior. For each one, what it catches and what it can't (a right-shaped but wrong value, request headers, a spec that has drifted from the service, …) is in [What the checks guarantee](docs/guarantees.md).
+
 At the end of the run, each stub with a spec reports which of the provider's operations the app called across all scenarios: its footprint on that API, for planning an upgrade or a switch of provider. Operations the provider marks `deprecated` are flagged, and on GitHub Actions they become a warning annotation and the list goes to the job summary:
 
 ```
@@ -465,7 +510,7 @@ Record once, with real credentials in the app's environment:
 SLICETEST_RECORD=github npx vitest     # or SLICETEST_RECORD=1 for every stub with an upstream
 ```
 
-Calls no route matches are forwarded to `upstream` (under its path prefix, headers included) and the answers are written to `recordings/github.yaml` (`recordings:` changes the path). Later runs replay them without touching the network. A request is identified by method, path, query and body (JSON key order doesn't matter); identical requests replay their recordings in the order they were made. Only `content-type`, `location`, `retry-after`, `link` and `etag` response headers are kept, and request headers are never stored, so tokens stay out of the file; bodies are stored as sent, so review the file before committing it.
+Calls no route matches are forwarded to `upstream` (under its path prefix, headers included) and the answers are written to `recordings/github.yaml` (`recordings:` changes the path). Later runs replay them without touching the network. A request is identified by method, path, query and body (JSON key order doesn't matter); identical requests replay their recordings in the order they were made. Only `content-type`, `location`, `retry-after`, `link` and `etag` response headers are kept, and request headers are never stored, so tokens stay out of the file. Credentials sent in the query or the body (`api_key`, `key` in the query, `appid`, `access_token`, `client_secret`, `password`, signatures, …) are stored as `[redacted]` and match any value on replay, so the test environment's dummy key replays what the real key recorded. Other values are stored as sent, so review the file before committing it.
 
 #### From a HAR file: `npx slicetest import`
 
@@ -478,6 +523,8 @@ npx slicetest import session.har --stub stripe --upstream https://api.stripe.com
 
 Requests under a stub's `upstream` become entries of its recordings file, in the same format and with the same filtering as recording: only the five response headers above, no request headers, the path relative to the upstream's. Preflights, aborted requests and binary responses are skipped, and the hosts it didn't import are listed. Entries already in the file aren't added twice.
 
+A replay is the real service as it was when recorded. If the provider changes, the recording keeps passing: delete the file (or entries) and record again from time to time, and review the diff, or check the provider with a separate contract test. Values other than the well-known credential names are stored as sent.
+
 Precedence is: registered route, then recording, then `autoReply`, then a 501 that says how to record the call. Replayed calls have `call.fallback === true`, and are checked against the provider's spec when the stub has one. To refresh recordings, delete the file (or the entries) and record again.
 
 ### Matchers
@@ -486,6 +533,10 @@ Registered automatically:
 
 ```ts
 expect(res).toHaveStatus(201);                                   // failure shows the response body
+expect(res).toHaveStatus("2xx");                                 // a class, or a list: [200, 204], ["2xx", 304]
+expect(res).toRespondWithin(300);                                // a response-time budget, in ms
+expect(res).toSetCookie("sid", { httpOnly: true, secure: true, sameSite: "Lax" }); // a cookie and its attributes
+expect(res.events).toContainEqual({ event: "message_stop", data: {} }); // text/event-stream responses are parsed into events
 expect(responses).toHaveStatuses({ 201: 1, 409: 9 });            // an array, e.g. from http.concurrently()
 expect(stub("slack")).toHaveReceived("POST", "/hook", { json: { text: "hi" } });
 expect(stub("slack")).toHaveReceivedTimes(1, "POST", "/hook");
@@ -631,8 +682,12 @@ In YAML, `auth` on a `request` step sends `Authorization: Bearer` with those cla
 | `zoom` | `x-zm-signature: v0=…`, `x-zm-request-timestamp` |
 | `twilio` | `X-Twilio-Signature` over the URL and the sorted form parameters; objects are sent as a form. The URL is the app's address and the path, or `url` when the app validates against a public URL it's configured with |
 | `twitch` | EventSub: `Twitch-Eventsub-Message-Signature`, `-Id`, `-Timestamp`, `-Type` (`event`, default `notification`) |
+| `typeform` | `Typeform-Signature: sha256=…` (base64) |
+| `notion` | `X-Notion-Signature: sha256=…` (the `verification_token` as `secret`) |
+| `bitbucket` | Bitbucket Cloud: `X-Hub-Signature: sha256=…` |
+| `intercom` | `X-Hub-Signature: sha1=…` (the app's `client_secret`) |
 
-The signatures are checked against the providers' documented examples where they publish one.
+The signatures are checked against the providers' documented examples where they publish one. `typeform`, `notion`, `bitbucket` and `intercom` follow the algorithm in the provider's documentation (read on 2026-10-05); none of them publishes an example to compare with, so they are recomputed in the tests, not checked against an SDK.
 
 ```ts
 const stripe = { provider: "stripe", secret: "whsec_test" } as const;   // the same secret as in app.env
@@ -674,6 +729,8 @@ scenario("voting flow", async ({ http, stub, trace }) => {
 
 Dates (`Date` values and ISO strings) become `[date]` and UUIDs `[uuid]`. Mask more with `trace({ keys: ["token"], patterns: [/^tok_/] })`, or use `mask(value, opts)` from `slicetest` on anything else. Update snapshots with `vitest -u`. In YAML, the step is `snapshot: true` (with `mask: [token]`).
 
+A snapshot records what the app does today, bugs included, and `-u` accepts whatever it shows: review a snapshot update like a change to an assertion, and don't let it be the only assertion on behavior a requirement spells out.
+
 The example apps share one snapshot file: the Node and the Python implementation must produce the same trace, byte for byte.
 
 ### Sequence diagrams of every scenario
@@ -708,12 +765,14 @@ scenario.each([{ choice: "a", status: 204 }, { choice: "x", status: 400 }])(
 );
 ```
 
-Scenarios in one file share an app and a database, so they always run one at a time; `.concurrent` is rejected.
+Scenarios in one file share an app and a database, so they always run one at a time; `.concurrent` is rejected. Test files do run in parallel, one database per Vitest worker and their own stubs and app process, and requests inside one scenario can overlap (`http.concurrently`): [what is shared and what isn't](docs/state-isolation.md#running-in-parallel).
 
 Tags select scenarios across files: `npx slicetest --tag smoke` (repeat `--tag` for any of several, `--tag '!slow'` to leave some out), or `SLICETEST_TAGS=smoke,!slow npx vitest` with your own Vitest config. Scenarios a filter leaves out are reported as skipped. In YAML, `tags: [smoke, payments]` on a scenario.
 
 ### Configuration reference
 
+`slicetest.config.yaml` has a JSON Schema for editor completion and checks: put `# yaml-language-server: $schema=https://unpkg.com/slicetest/schema/config.schema.json` on its first line (VS Code with the YAML extension, JetBrains IDEs), as `slicetest init` does. In YAML, `env` values may be numbers or booleans; they reach the process as strings.
+
 | Option | Default | |
 |---|---|---|
 | `app.command` | (required) | Shell command. May use `{{app.port}}` and the other placeholders. |
@@ -723,10 +782,13 @@ Tags select scenarios across files: `npx slicetest --tag smoke` (repeat `--tag`
 | `app.cwd` | vitest root | |
 | `app.ready` | `{ path: "/" }` | Poll a path until it answers below 500, or `{ log: "listening" \| /regex/ }`. |
 | `app.readyTimeout` | `30000` | |
+| `app.restart` | `"never"` | `"scenario"`: stop the process before each scenario's reset and start it after, so nothing in its memory survives. Also on `services`. See [state between scenarios](docs/state-isolation.md). |
+| `app.reset` | none | `{ path, method?, timeout? }`: an endpoint the app serves to drop its in-process state, called (`POST` by default) after the reset and before each scenario. `path` is a path on the app, `method` one of GET / POST / PUT / PATCH / DELETE / HEAD. Redirects aren't followed and a status of 300 or more fails the scenario. Also on `services`. |
+| `app.idle` | none | `{ path, method?, timeout? }`: an endpoint that answers 2xx once the app has no background work left, polled (`GET`) after each scenario for up to `timeout` (5000 ms, at most 600000) or the time the test's own timeout leaves. If it's still busy, the scenario fails and the process is restarted. Also on `services`. |
 | `app.scope` | `"file"` | `"worker"`: start the app (and stubs, services) once per Vitest worker and keep it for all of that worker's test files, for apps that start slowly. Sets Vitest's `isolate: false`. |
 | `db.engine` | `postgres`, or `mysql` for a `mysql://` URL | `postgres`, `mysql` (see [MySQL](#mysql)) or `sqlite` (see [SQLite](#sqlite)). |
-| `db.migrate` | none | `{ atlas: { dir } }`, `{ sql: "file-or-dir" }` or `{ command, inputs?, env? }`. The command gets `DATABASE_URL`, and both it and `env` may use the `{{db.*}}` placeholders, for tools that read other variables: `{ command: "php artisan migrate --force", env: { DB_HOST: "{{db.host}}", DB_DATABASE: "{{db.name}}" } }`, `{ command: "dotnet ef database update --connection \"{{db.adoNet}}\"" }`. |
-| `db.seed` | none | SQL file re-run after every reset. |
+| `db.migrate` | none | `{ atlas: { dir } }`, `{ sql: "file-or-dir" }` or `{ command, inputs?, env? }`. A `sql` directory is applied in name order with version numbers compared as numbers (`V2__` before `V10__`), leaving out rollbacks: `*.down.sql` (golang-migrate, sqlx, Diesel), Flyway undo files (`U2__…`) and the down section of goose (`-- +goose Down`) and dbmate (`-- migrate:down`) files. The command gets `DATABASE_URL`, and both it and `env` may use the `{{db.*}}` placeholders, for tools that read other variables: `{ command: "php artisan migrate --force", env: { DB_HOST: "{{db.host}}", DB_DATABASE: "{{db.name}}" } }`, `{ command: "dotnet ef database update --connection \"{{db.adoNet}}\"" }`. |
+| `db.seed` | none | A file, or a list of files run in that order, re-run after every reset: SQL, or YAML / JSON rows per table (`plans: [{ id: 1, name: free }]`), inserted with the engine's own quoting, so one seed serves Postgres, MySQL and SQLite. After rows with explicit ids, Postgres sequences are moved past them so the app's next insert doesn't collide (MySQL and SQLite do that themselves). |
 | `db.schemas` | `["public"]` | Schemas whose tables are reset. |
 | `db.keep` | `[]` | Extra tables (`name` or `schema.name`) never truncated. |
 | `db.ignoreChanges` | `[]` | Columns (`updated_at`, `orders.synced_at`) and tables (`sessions.*`) left out of `db.changes()`, `trace()` and the failure output. |
@@ -795,8 +857,8 @@ scenarios:
 | Step | Keys |
 |---|---|
 | `stub: <name>` | `on: METHOD /path` (`:params` allowed) or `graphql: <operation>`, `when: { query, headers, json, form, body, variables }`, one of `reply: { status, headers, body }` (`{ file }` for a fixture file, `{ data, errors }` for GraphQL) / `sequence: [...]` / `networkError: true`, plus `times`, `delay`. Replies may echo the call: `{{call.params.id}}`, `{{call.json.name}}`, `{{call.form.amount}}`, `{{call.variables.id}}`. |
-| `submit: <button>` | `form`, `fields` (`{ file: path }` for a file input), `headers`, `follow`, `expect: { status, headers, json, text }`, `capture`. Submits a form of the page the last request returned, like `http.submit()`; `submit: true` presses the form's only button. |
-| `request: METHOD /path` | `headers`, `query`, one of `json` / `form` / `multipart` / `body` / `graphql: { query, variables, operationName }`, `expect.schema` (a JSON Schema, or `../openapi.yaml#/components/schemas/Poll` relative to the file), `follow`, `expect: { status, headers, json, text }`, `capture`. `timeout: <ms>` fails it, by name, when the app doesn't answer in time. `concurrency: n` sends it `n` times at once; `expect` then applies to each response, and `expect.statuses: { 201: 1, 409: 9 }` counts them. |
+| `submit: <button>` | `form`, `fields` (`{ file: path }` for a file input), `headers`, `follow`, `expect: { status, headers, json, text }` (`status` may be a class, `2xx`, or a list, `[200, 204]`; `duration: 300` fails a response slower than 300 ms, `duration: { $lt: 300 }` takes matchers; `cookies: { sid: { httpOnly: true, sameSite: Lax }, tracking: null }` checks the cookies it sets and their attributes, `null` for one it must not set; `events: [{ event: message_start }, { data: "[DONE]" }]` checks a `text/event-stream` response's events in order, others allowed between), `capture` (also from `events.0.data.id`). Submits a form of the page the last request returned, like `http.submit()`; `submit: true` presses the form's only button. |
+| `request: METHOD /path` | `headers`, `query`, one of `json` / `form` / `multipart` / `body` / `graphql: { query, variables, operationName }`, `expect.schema` (a JSON Schema, or `../openapi.yaml#/components/schemas/Poll` relative to the file), `follow`, `expect: { status, headers, json, text }` (a header set to `null` must be absent: `x-powered-by: null`), `capture`. `timeout: <ms>` fails it, by name, when the app doesn't answer in time. `concurrency: n` sends it `n` times at once; `expect` then applies to each response, and `expect.statuses: { 201: 1, 409: 9 }` counts them (classes count the responses no listed code takes: `{ 201: 1, 4xx: 9 }`). |
 | `insert: <table>` | `rows`, `capture` (from `row` / `rows`) |
 | `request` with `auth` | `auth: true` or the claims: sends a bearer token from the `auth` issuer |
 | `request` with `webhook` | `{ provider, secret, event, stale, invalidSignature }`: signs the body like that provider's deliveries |
@@ -810,7 +872,7 @@ scenarios:
 | `changes: { <table>: { inserted, updated, deleted } }` | Each is a count or a list of subset rows (`updated` matches the row after the update). Tables that aren't listed must be unchanged. `ignore: [updated_at, sessions.*]` leaves out columns and tables. |
 | `checkpoint: true` | Later `changes` steps only see what happens after this step. |
 | `set: { name: value }` | Defines variables for later steps, e.g. `{ orderId: "{{$uuid}}", expires: "{{$now+1d}}" }`. |
-| `mail: { to, from, subject, text, html }` | `times` (exact; default at least one), `within` (ms, default 5000), `capture` from the last match (`subject`, `text`, `links.0`). Waits for mail the app sends. `{}` matches any message. |
+| `mail: { to, from, subject, text, html }` | `times` (exact; default at least one), `within` (ms, default 5000), `capture` from the last match (`subject`, `text`, `links.0`). Waits for mail the app sends. `{}` matches any message. A field takes a string or `{ $regex, $options }`. |
 | `use: <definition>` | `with: { param: value }`. Runs the steps of a `define:` entry. |
 | `snapshot: true` | The scenario's [trace](#snapshot-the-whole-scenario-trace) so far must match its stored snapshot. `mask: [keys]` hides more values. |
 
@@ -826,6 +888,8 @@ scenarios:
   capture: { file: json.url }
 ```
 
+Every wait (`within`, `log`, `mail`) ends shortly before the scenario's own timeout, which is Vitest's 5000 ms unless the scenario sets `timeout:`; the failure then names what it waited for and says the timeout cut it short. A `within: 10000` needs a scenario `timeout:` above it.
+
 - `reply: { file: replies/charge.json }` answers with a file relative to the scenario file: `.json` and `.yaml` are sent as JSON and may still use `{{call.*}}`, other files as they are (images, PDFs, CSV), with a content type from the extension unless `headers` set one. Large provider payloads stay out of the scenario.
 - `multipart:` sends `multipart/form-data`: plain values are fields, `{ file: fixtures/avatar.png }` uploads a file (relative to the scenario file, content type from its extension, or `type` / `filename`), `{ content: ..., filename: notes.json }` an inline one, and a list sends a field several times.
 - `request:` also takes a captured URL of the app, e.g. `GET {{link}}` after capturing a link from a mail.
@@ -837,13 +901,16 @@ scenarios:
   |---|---|
   | `{ $type: number }` | `string`, `number`, `integer`, `boolean`, `array`, `object`, `null` |
   | `{ $regex: "^ch_" }` | a string the pattern finds |
+  | `{ $regex: "^ab", $options: i }` | the same with flags (`i`, `m`, `s`, …), as in MongoDB |
   | `{ $contains: "ok" }` | a string with that substring, or a list with a matching item (`{ $contains: { sku: a } }`) |
-  | `{ $gte: 1, $lt: 10 }` | numbers, or strings such as ISO dates (`{ $gte: "2026-01-01" }`); also `$gt`, `$lte` |
+  | `{ $gte: 1, $lt: 10 }` | numbers (also decimals that `numeric` / `DECIMAL` columns return as strings, `"12.50"`), or strings such as ISO dates (`{ $gte: "2026-01-01" }`); also `$gt`, `$lte` |
+  | `{ $closeTo: 9.99 }` | a number within ±0.005 (`[9.99, 0.1]` for another tolerance), for floats and computed totals; decimal strings too |
   | `{ $len: 3 }` | a string or list of that length; `{ $len: { $gte: 1 } }` |
   | `{ $oneOf: [paid, pending] }` | any of the values (or matchers) |
   | `{ $not: "" }` | anything the value or matcher doesn't match |
-  | `{ $format: uuid }` | `uuid`, `email`, `date`, `date-time`, `uri`, `integer` (a string of digits) |
+  | `{ $format: uuid }` | `uuid`, `email`, `date`, `date-time`, `time`, `uri`, `hostname`, `ipv4`, `ipv6`, `ulid`, `integer` (a string of digits), `jwt` (three base64url parts with a JSON header naming its `alg`; the signature isn't checked) |
   | `{ $any: true }` | anything but `null` / missing |
+  | `{ $absent: true }` | the key isn't there at all (`null` is there): `password_hash: { $absent: true }` checks a response doesn't leak it. Expected JSON is otherwise a subset, so extra keys pass unless named this way |
 
   Several `$` keys in one mapping must all hold.
 - A file-level `setup:` list runs at the start of every scenario. `skip`, `only`, `timeout` and `tags` work per scenario.
@@ -897,6 +964,13 @@ npx slicetest                 # every *.scenario.yaml under the config's directo
 npx slicetest polls -t voting # filter by file and scenario name
 npx slicetest --watch
 npx slicetest list --tag smoke # what would run: file, line, tags, steps (--json for tools)
+npx slicetest -u              # rewrite `snapshot: true` snapshots that no longer match
+npx slicetest --reporter junit --output-file reports/slicetest.xml   # GitLab, Jenkins, CircleCI test reports
+npx slicetest --shard 2/4     # the second of four parts, for parallel CI jobs
+npx slicetest --shuffle       # scenarios in a random order (the seed is printed): a scenario that only passes after another one leans on state the reset doesn't clear
+npx slicetest --shuffle --seed 1791208846  # that order again
+npx slicetest --retry 2       # run a failing scenario up to twice more (each attempt after a fresh reset)
+npx slicetest --version       # the installed version; an unknown option names the nearest one
 ```
 
 ### Scenarios from your OpenAPI spec: `npx slicetest gen`
@@ -910,6 +984,8 @@ writes `scenarios/<resource>.gen.scenario.yaml` with one scenario per documented
 
 Operations that the spec protects with a bearer token (`http: bearer`, `oauth2` or `openIdConnect` security) get `auth:` on their requests, with the scopes the spec requires in the token's `scope` claim, and their 401 responses become runnable scenarios that send no token. With [`auth`](#auth-a-real-openid-issuer-tokens-with-any-claims) in the config, the generated scenarios run against the app's real token checks.
 
+The generated expectations come from the spec, so a mistake in the spec is copied into them; review the scenarios before committing them, as you would code (see [tests written by an AI agent](docs/guarantees.md#tests-written-by-an-ai-agent)).
+
 `--uncovered` reads the coverage the last run left in `node_modules/.cache/slicetest/`, which closes the loop: run, look at the ✗ in the coverage table, `gen --uncovered`, fill in the TODOs.
 
 ### Record a scenario by using the app: `npx slicetest record`
@@ -1038,17 +1114,37 @@ app:
 
 ## Examples
 
-`examples/` has a Node app (`node:http` + `pg`) and a Python app (`http.server` + `psycopg`) with the same API. **The same scenario files (`polls.test.ts` and `polls.scenario.yaml`) run against both**:
+`examples/` has a Node app (`node:http` + `pg`), a Python app (`http.server` + `psycopg`) and a [Spring Boot app](examples/spring-boot-api/README.md) (Java 21, Maven, JDBC) with the same API. **The same scenario files (`polls.test.ts` and `polls.scenario.yaml`) run against all three**:
 
 ```sh
 npm test            # unit tests + both example apps
 npm run test:dist   # the built package, and the CLI with examples/slicetest.config.yaml
+npm run test:spring # the same scenarios against the Spring Boot app (needs JDK 21 and Maven)
 ```
 
 ## Using a coding agent
 
 [`.claude/skills/slicetest-write-tests`](.claude/skills/slicetest-write-tests/SKILL.md) teaches Claude Code (or any agent that reads skills) how to write slicetest scenarios. Copy it into your project's `.claude/skills/`. Contributors: see [`AGENTS.md`](AGENTS.md).
 
-## Status
+## Feedback and contributing
+
+Found a bug, have an idea, or need help? [Open an issue](https://github.com/revo1290/slicetest/issues/new/choose) using the bug report, feature request, or question form. **Japanese and English are welcome.** See [CONTRIBUTING.md](CONTRIBUTING.md) for reporting and contribution guidance.
+
+不具合報告・機能提案・質問を[Issues](https://github.com/revo1290/slicetest/issues/new/choose)で受け付けています。日本語でお気軽に投稿してください。投稿方法は[貢献ガイド](CONTRIBUTING.md)をご覧ください。
+
+## Compatibility and releases
+
+Version 0.x, early. What is run, and where:
+
+| | Verified in CI on every push | Used by hand, not in CI | Not verified |
+|---|---|---|---|
+| Node.js | 24 | the maintainer's machine (also 24); 2026-10-05, the Node example app's 31 tests by hand: Node 22.23.3 (containers) and Node 20.20.2 with Vitest 5.0.2 against a database server you run (`SLICETEST_DATABASE_URL`) | the rest of the suite on 20 and 22. On Node below 22.22 the container library doesn't load (Testcontainers' requirement; checked on 20.20.2), so a database or dependency in a container needs 22.22+, and the error says so. SQLite needs 22.5+; `hosts` for Node apps needs 22.21+ / 24.5+ |
+| Vitest | 5.0.x (the version in the lockfile) | 4.x on one project (the maintainer's report); 2026-10-05, Vitest 4.1.11 in a copy of the repo: the Node example's 31 tests, the unit project's 369 tests, the CLI against the example's YAML scenarios (14) and the `record` round trip. With Vitest 4 the matcher types resolve with `skipLibCheck: true`; with `skipLibCheck: false` TypeScript reports TS2428 in `dist/matchers.d.ts`, because Vitest 4 and 5 declare `Matchers` with different type parameters | 6 and later |
+| OS | Ubuntu (the full suite); Windows (unit tests, the fixtures that crash and fail, and the built package with the CLI) | macOS | Windows with Docker-based databases: Windows CI uses a Postgres service instead of containers |
+| Database | PostgreSQL 17 (`postgres:17-alpine`, on Ubuntu) and the Postgres service of the Windows runner (its version isn't pinned); MySQL 8.4 and SQLite (Ubuntu; SQLite also in the isolation tests on Windows) | | other PostgreSQL / MySQL versions, MariaDB, other SQLite builds |
+| Container runtime | Docker (Ubuntu) | Podman (macOS) | |
+| App stacks | Node, Python and Spring Boot (Java 21, Maven) example apps (Ubuntu; the Spring one in its own job), small Node fixtures | Spring Boot, Next.js and others, on the maintainer's own projects (the maintainer's report; no automated test) | the rest of what `slicetest init` recognizes: detection is tested on fixture projects, not on running apps |
+
+The built package, not only the sources, is checked: `npm run test:dist` runs the Node example and the CLI (with the Node example's config) against `dist/` and type-checks them against the published types.
 
-Early. Postgres, MySQL and SQLite. CI runs on Linux and Windows.
+**Releases.** Before 1.0, a minor version (`0.N.0`) may change behavior or configuration, and a patch version (`0.N.P`) only fixes bugs. [CHANGELOG.md](CHANGELOG.md) lists every change. Pin the minor (`"slicetest": "~0.10.0"`) and upgrade on purpose: read the entries between your version and the new one, run `npx slicetest doctor`, and review snapshot diffs instead of running `-u`. A change that can make a passing suite fail (a check that got stricter, a key that was removed) is called out in that version's entry as it is released.

package/dist/app.js

@@ -1,7 +1,9 @@
+import { waitBudget } from "./deadline.js";
 import { spawn, spawnSync } from "node:child_process";
 import { createServer } from "node:net";
 import path from "node:path";
 import { StringDecoder } from "node:string_decoder";
+import { onProcessExit } from "./exit.js";
 const LOG_LINES = 200;
 /** How long to wait for stdout/stderr to drain after the process exits. */
 const DRAIN_MS = 100;
@@ -16,6 +18,7 @@ export class App {
     /** Lines ever received, so callers can ask for output since a point in time. */
     #lineCount = 0;
     #exit;
+    #stopped = false;
     #exited;
     #listeners = new Set();
     /** `mark()` at the start of the current scenario. */
@@ -151,6 +154,7 @@ export class App {
      * ```
      */
     async waitForLog(pattern, timeout = 5000) {
+        const { ms, note } = waitBudget(timeout);
         const re = typeof pattern === "string" ? new RegExp(escapeRegExp(pattern)) : new RegExp(pattern.source, pattern.flags.replace(/[gy]/g, ""));
         const n = Math.min(this.#lineCount - this.#scenarioMark, this.#log.length);
         const seen = n > 0 ? this.#log.slice(-n).find((l) => re.test(l)) : undefined;
@@ -166,7 +170,7 @@ export class App {
                 if (re.test(line))
                     done(() => resolve(line));
             };
-            const timer = setTimeout(() => done(() => reject(new Error(`slicetest: ${this.label} printed no line matching ${re} within ${timeout}ms. Output during this scenario:\n${this.logs(this.#scenarioMark) || "(none)"}`))), timeout);
+            const timer = setTimeout(() => done(() => reject(new Error(`slicetest: ${this.label} printed no line matching ${re} within ${ms}ms${note}. Output during this scenario:\n${this.logs(this.#scenarioMark) || "(none)"}`))), ms);
             this.#listeners.add(listener);
         });
     }
@@ -176,6 +180,10 @@ export class App {
      */
     async stop() {
         running.delete(this);
+        // Not signalling again: a process group id can be reused once the first stop has reaped it.
+        if (this.#stopped)
+            return;
+        this.#stopped = true;
         const pid = this.#child.pid;
         if (pid === undefined)
             return;
@@ -251,7 +259,7 @@ export class App {
 }
 /** Apps still running, killed synchronously if the worker exits without tearing down. */
 const running = new Set();
-process.once("exit", () => {
+onProcessExit(() => {
     for (const app of running)
         app.killNow();
 });

package/dist/auth.js

@@ -1,5 +1,6 @@
 import { createSign, generateKeyPairSync, randomUUID } from "node:crypto";
 import { createServer } from "node:http";
+import { requestUrl } from "./request-url.js";
 const b64url = (data) => Buffer.from(data).toString("base64url");
 function newKey() {
     const { privateKey, publicKey } = generateKeyPairSync("rsa", { modulusLength: 2048 });
@@ -93,7 +94,7 @@ export class Issuer {
         this.#issued = 0;
     }
     async #handle(req) {
-        const path = new URL(req.url ?? "/", this.url).pathname;
+        const path = requestUrl(req.url, this.url).pathname;
         if (req.method === "GET" && path === "/.well-known/openid-configuration") {
             return [
                 200,

package/dist/cli-run.d.ts

@@ -0,0 +1,53 @@
+export interface RunFlags {
+    update?: boolean;
+    reporter?: string[];
+    "output-file"?: string;
+    shard?: string;
+    retry?: string;
+    shuffle?: boolean;
+    seed?: string;
+}
+/** Vitest options for the run flags of `npx slicetest`; paths are taken from where the command runs, like `--diagrams`. */
+export declare function runOptions(flags: RunFlags, cwd?: string): {
+    update?: boolean | undefined;
+    retry?: number | undefined;
+    sequence?: {
+        shuffle: boolean;
+        seed?: number | undefined;
+    } | undefined;
+    reporters?: string[] | undefined;
+    outputFile?: string | undefined;
+    shard?: string | undefined;
+};
+/** parseArgs, with its errors worded for this CLI (and a guess at a misspelt option). */
+export declare function parseCliArgs(argv: string[]): {
+    values: {
+        config?: string | undefined;
+        watch?: boolean | undefined;
+        name?: string | undefined;
+        tag?: string[] | undefined;
+        help?: boolean | undefined;
+        version?: boolean | undefined;
+        force?: boolean | undefined;
+        spec?: string | undefined;
+        out?: string | undefined;
+        uncovered?: boolean | undefined;
+        port?: string | undefined;
+        diagrams?: string | undefined;
+        stub?: string | undefined;
+        upstream?: string | undefined;
+        json?: boolean | undefined;
+        update?: boolean | undefined;
+        reporter?: string[] | undefined;
+        "output-file"?: string | undefined;
+        shard?: string | undefined;
+        retry?: string | undefined;
+        shuffle?: boolean | undefined;
+        seed?: string | undefined;
+    };
+    positionals: string[];
+};
+/** This package's version: the package.json next to dist/ (and next to src/ when run from a checkout). */
+export declare function packageVersion(): Promise<string>;
+/** `startVitest` as the installed Vitest takes it: 4 wants the mode ("test") first, 5 dropped it. */
+export declare function compatStart(vitest: typeof import("vitest/node")): typeof import("vitest/node").startVitest;

package/dist/cli-run.js

@@ -0,0 +1,80 @@
+import { readFile } from "node:fs/promises";
+import path from "node:path";
+import { parseArgs } from "node:util";
+import { editDistance } from "./config.js";
+/** Vitest options for the run flags of `npx slicetest`; paths are taken from where the command runs, like `--diagrams`. */
+export function runOptions(flags, cwd = process.cwd()) {
+    if (flags.shard !== undefined) {
+        const m = /^(\d+)\/(\d+)$/.exec(flags.shard);
+        if (!m || Number(m[1]) < 1 || Number(m[1]) > Number(m[2]))
+            throw new Error(`slicetest: --shard takes <index>/<count> with 1 ≤ index ≤ count, e.g. --shard 1/3, got "${flags.shard}"`);
+    }
+    if (flags.retry !== undefined && !/^\d+$/.test(flags.retry))
+        throw new Error(`slicetest: --retry takes a whole number of retries, e.g. --retry 2, got "${flags.retry}"`);
+    if (flags.seed !== undefined && !flags.shuffle)
+        throw new Error("slicetest: --seed goes with --shuffle (--shuffle --seed 1234 repeats a shuffled order)");
+    if (flags.seed !== undefined && !/^[1-9]\d*$/.test(flags.seed))
+        throw new Error(`slicetest: --seed takes a positive whole number, got "${flags.seed}"`);
+    return {
+        ...(flags.update ? { update: true } : {}),
+        ...(flags.retry !== undefined ? { retry: Number(flags.retry) } : {}),
+        ...(flags.shuffle ? { sequence: { shuffle: true, ...(flags.seed ? { seed: Number(flags.seed) } : {}) } } : {}),
+        ...(flags.reporter?.length ? { reporters: flags.reporter } : {}),
+        ...(flags["output-file"] ? { outputFile: path.resolve(cwd, flags["output-file"]) } : {}),
+        ...(flags.shard ? { shard: flags.shard } : {}),
+    };
+}
+const OPTIONS = {
+    config: { type: "string", short: "c" },
+    watch: { type: "boolean", short: "w" },
+    name: { type: "string", short: "t" },
+    tag: { type: "string", multiple: true },
+    help: { type: "boolean", short: "h" },
+    version: { type: "boolean", short: "v" },
+    force: { type: "boolean" },
+    spec: { type: "string" },
+    out: { type: "string" },
+    uncovered: { type: "boolean" },
+    port: { type: "string" },
+    diagrams: { type: "string" },
+    stub: { type: "string" },
+    upstream: { type: "string" },
+    json: { type: "boolean" },
+    update: { type: "boolean", short: "u" },
+    reporter: { type: "string", multiple: true },
+    "output-file": { type: "string" },
+    shard: { type: "string" },
+    retry: { type: "string" },
+    shuffle: { type: "boolean" },
+    seed: { type: "string" },
+};
+/** parseArgs, with its errors worded for this CLI (and a guess at a misspelt option). */
+export function parseCliArgs(argv) {
+    try {
+        return parseArgs({ args: argv, allowPositionals: true, options: OPTIONS });
+    }
+    catch (e) {
+        const err = e;
+        const unknown = /^Unknown option '(-{1,2}[^']*)'/.exec(err.message ?? "")?.[1];
+        if (err.code === "ERR_PARSE_ARGS_UNKNOWN_OPTION" && unknown) {
+            const name = unknown.replace(/^-+/, "");
+            const near = Object.keys(OPTIONS)
+                .map((k) => ({ k, d: editDistance(k, name.toLowerCase()) }))
+                .filter(({ k, d }) => d <= Math.max(2, Math.floor(name.length / 3)) || (name.length >= 3 && k.startsWith(name.toLowerCase())))
+                .sort((a, b) => a.d - b.d)[0];
+            throw new Error(`slicetest: unknown option ${unknown}${near ? `; did you mean --${near.k}?` : ""} (see npx slicetest --help)`);
+        }
+        // A missing value (`--config` last on the line) and the like: node's own wording says what is wrong.
+        throw new Error(`slicetest: ${err.message ?? e} (see npx slicetest --help)`);
+    }
+}
+/** This package's version: the package.json next to dist/ (and next to src/ when run from a checkout). */
+export async function packageVersion() {
+    const file = new URL("../package.json", import.meta.url);
+    return JSON.parse(await readFile(file, "utf8")).version;
+}
+/** `startVitest` as the installed Vitest takes it: 4 wants the mode ("test") first, 5 dropped it. */
+export function compatStart(vitest) {
+    const { startVitest, version } = vitest;
+    return (Number.parseInt(version, 10) === 4 ? startVitest.bind(null, "test") : startVitest);
+}