slicetest 0.6.1 → 0.7.0

package/README.md

@@ -38,23 +38,25 @@ npx slicetest init   # detects your stack, writes slicetest.config.yaml and a fi
 npx slicetest        # starts Postgres, migrates, starts your app, runs scenarios/*.scenario.yaml
 ```
 
-`init` recognises Node (`npm start`), Django, FastAPI, Flask, Rails, Go and Rust apps; Atlas, Prisma, Alembic, Django, Rails, Drizzle, Knex and plain SQL migrations; and an `openapi.yaml`. If there's a `compose.yaml` / `docker-compose.yml`, its database service sets `db.image` (and `db.engine: mysql` for MySQL or MariaDB), and Redis, Valkey, Mongo, Elasticsearch, MinIO, RabbitMQ and other services with a port become [`containers`](#containers-redis-search-s3-and-other-dependencies), with a reset command where one is known and the usual variable (`REDIS_URL`, `S3_ENDPOINT`, …) passed to the app. A mail catcher there (Mailpit, MailHog, MailDev, smtp4dev, …) or a mail library in the dependencies turns on [`mail`](#mail-catch-what-the-app-sends). SQLite is picked up from Prisma's provider, Rails' `database.yml`, Django's settings or a SQLite driver, with `DATABASE_URL` in the form the framework reads (`file:…`, `sqlite3:…`). And third-party API URLs in `.env.example` (`STRIPE_API_BASE=https://api.stripe.com`) become stubs [recorded from that service](#recording-a-real-service), with the variable pointed at the stub, while local addresses, databases and your own URLs are left alone. Token issuer settings there (`OIDC_ISSUER`, `AUTH0_DOMAIN`, `JWKS_URL`, `JWT_AUDIENCE`, …) turn on [`auth`](#auth-a-real-openid-issuer-tokens-with-any-claims) and point at slicetest's issuer instead of becoming stubs. It lists every guess as a comment in the config so you know what to check.
+`init` recognises Node (`npm start`, or Bun, pnpm or Yarn from the lockfile), Deno, Phoenix, Django, FastAPI, Flask, Rails, Laravel, Symfony, plain PHP, Spring Boot, ASP.NET Core, Go and Rust apps; Atlas, Prisma, Alembic, Django, Rails, Laravel, Doctrine, EF Core, Ecto, Drizzle, Knex and plain SQL migrations; and an `openapi.yaml`. If there's a `compose.yaml` / `docker-compose.yml`, its database service sets `db.image` (and `db.engine: mysql` for MySQL or MariaDB), and Redis, Valkey, Mongo, Elasticsearch, MinIO, RabbitMQ and other services with a port become [`containers`](#containers-redis-search-s3-and-other-dependencies), with a reset command where one is known and the usual variable (`REDIS_URL`, `S3_ENDPOINT`, …) passed to the app. A mail catcher there (Mailpit, MailHog, MailDev, smtp4dev, …) or a mail library in the dependencies turns on [`mail`](#mail-catch-what-the-app-sends). SQLite is picked up from Prisma's provider, Rails' `database.yml`, Django's settings or a SQLite driver, with `DATABASE_URL` in the form the framework reads (`file:…`, `sqlite3:…`). And third-party API URLs in `.env.example` (`STRIPE_API_BASE=https://api.stripe.com`) become stubs [recorded from that service](#recording-a-real-service), with the variable pointed at the stub, while local addresses, databases and your own URLs are left alone. Token issuer settings there (`OIDC_ISSUER`, `AUTH0_DOMAIN`, `JWKS_URL`, `JWT_AUDIENCE`, …) turn on [`auth`](#auth-a-real-openid-issuer-tokens-with-any-claims) and point at slicetest's issuer instead of becoming stubs. It lists every guess as a comment in the config so you know what to check.
 
 ## What you get that's hard to find elsewhere
 
 - **One scenario, three boundaries.** Assert on the HTTP response, the rows in the real database and the calls to third-party APIs in the same test, in any language the app is written in.
 - **`db.changes()`**: a diff of every row the scenario inserted, updated or deleted. `toEqual` on it catches writes you didn't expect.
-- **Stubs that can't lie.** Give a stub the provider's OpenAPI spec, and a canned reply the real service would never send fails the test.
+- **Stubs that can't lie.** Give a stub the provider's OpenAPI spec, and a canned reply the real service would never send fails the test. The run also lists which of the provider's operations the app depends on, flagging deprecated ones.
 - **Whole-scenario snapshots.** `expect(await trace()).toMatchSnapshot()` pins the responses, the outbound calls and the database changes in one reviewable file, with dates and UUIDs masked.
-- **Record the real service once, replay forever.** Point a stub at the real API with `SLICETEST_RECORD=1`, commit the YAML it writes, and later runs are offline and deterministic.
+- **Record the real service once, replay forever.** Point a stub at the real API with `SLICETEST_RECORD=1`, or import a HAR file saved from the browser, commit the YAML it writes, and later runs are offline and deterministic.
 - **OpenAPI coverage** of your own API, per operation and status, across all scenarios, and `slicetest gen --uncovered` to scaffold scenarios for what's missing.
 - **Record instead of write.** `npx slicetest record` puts a proxy in front of the app: click through a flow, press Enter, and get a replayable YAML scenario with the stubs' answers, the responses, captured ids and the database changes.
-- **Readable in CI.** On GitHub Actions, failing YAML steps are annotated in the pull request on the line that failed, and the job summary shows the OpenAPI coverage table.
+- **Readable in CI.** On GitHub Actions, failing YAML steps are annotated in the pull request on the line that failed, and the job summary shows the OpenAPI coverage table and a sequence diagram of each failed scenario.
+- **Diagrams that can't go stale.** `--diagrams docs/flows` writes a Mermaid sequence diagram of every scenario (app, stubs, mail, database), regenerated from what really happened on each run.
 - **Races on purpose.** `http.concurrently(10, ...)` and `toHaveStatuses({ 201: 1, 409: 9 })` turn "what if two people click at once" into a test against the real database.
 - **Mail as a fourth boundary.** `mail: true` catches the app's SMTP traffic in-process, decoded, with the links pulled out, so a sign-up test can follow the confirmation link.
 - **Real token verification, any user.** `auth: true` gives the app an OpenID issuer with a JWKS, so JWT checks stay on in tests, and scenarios mint tokens with any claims, including expired or foreign-signed ones.
 - **Webhooks signed like the real sender.** Stripe, GitHub, Slack, Shopify and Standard Webhooks signatures, plus forged and replayed deliveries, so signature checks are tested instead of bypassed.
 - **Reproducible chaos.** Stubs can fail the first calls, drop connections or add latency, from a seed the failure output prints, so a resilience test that fails once fails again on demand.
+- **GraphQL on both sides.** Stubs answer by operation name and variables rather than by path, and `toHaveGraphQLData()` fails on the `errors` a GraphQL server returns with status 200.
 - **Forms as a browser sends them.** `http.submit()` presses a button on a server-rendered page, hidden fields included, so CSRF tokens and Next.js server actions work without knowing their internals.
 - **Hard-coded APIs, stubbed anyway.** `hosts: [api.github.com]` catches calls to URLs written in the code or built into a framework, over HTTPS, from Node, Python, Go, Ruby or the JVM, with no change to the app. Redirects to those hosts are followed to the stub, so OAuth logins run end to end.
 - **N+1 detection for any stack.** A wire-protocol proxy records the SQL the app runs, so query counts are asserted at the HTTP boundary, whatever the ORM or language.
@@ -142,7 +144,8 @@ slicetest prints what happened during that scenario, next to Vitest's own error:
 --- slicetest ---
 stub calls with no matching route:
   mail: POST /send
-    registered on mail: POST /other
+    closest route POST /send: json.to: expected "a@example.com", got "b@example.com"
+    registered on mail: POST /send + json conditions
 
 requests to the app:
   POST /signup → 500 (14ms)  {"error":"internal"}
@@ -158,7 +161,7 @@ TypeError: Cannot read properties of undefined (reading 'email')
 -----------------
 ```
 
-Only this scenario's app output is shown, not the whole log. The database section is a diff against the state right after the reset and seed, so you see what the app actually wrote. Requests that never got a response (for example because the app crashed) appear as `failed`.
+For a call no route answered, slicetest names the route it came closest to and the first thing that differed: the method, a path that is off by a trailing slash, letter case or a base-URL prefix (`/api/v1/charges` against `/v1/charges`), a missing header, or the JSON field and value (`json.items.0.sku: expected "a", got "b"`), the GraphQL operation or variables, or a `once()` route that was already used. Only this scenario's app output is shown, not the whole log. The database section is a diff against the state right after the reset and seed, so you see what the app actually wrote. Requests that never got a response (for example because the app crashed) appear as `failed`.
 
 ## API
 
@@ -174,6 +177,7 @@ await http.get("/polls", { query: { page: 2 }, headers: { accept: "text/html" }
 await http.post("/login", http.form({ user: "a", pass: "b" }));  // urlencoded; FormData, Blob and bytes also work
 await http.get("/old-path", { follow: true });                    // redirects are NOT followed by default
 await http.submit(await http.get("/signup"), { button: "Sign up", fields: { email: "a@b.test" } }); // a form, as a browser sends it
+await http.graphql("query Poll($id: ID!) { poll(id: $id) { title } }", { id: 1 });  // POST /graphql ({ path } for another)
 
 const admin = http.with({ headers: { authorization: `Bearer ${token}` } }); // shares cookies with http
 http.cookies.get("session");                                     // cookies persist within a scenario
@@ -280,10 +284,11 @@ stub("pay").on("GET", "/status").replySequence([{ status: 503 }, { status: 200 }
 stub("pay").on("POST", "/charge").delay(5_000).reply(200);    // exercise the app's timeouts
 stub("pay").on("POST", "/charge").networkError();             // drop the connection
 
+stub("slack").on("POST", "/hook").optional().reply(200);      // may go uncalled, even with strictStubs
 stub("slack").calls("POST", "/hook");                         // recorded calls: method, path, params, query, headers, body, json
 ```
 
-Later routes win. `path` may also be a RegExp, and `method` may be `*`. Unanswered calls get a `501` and fail the scenario.
+Later routes win. `path` may also be a RegExp, and `method` may be `*`. Unanswered calls get a `501` and fail the scenario, with the closest route and why it didn't match (`stub.explain(call)`).
 
 ### OpenAPI contracts — for your app and for the services you stub
 
@@ -308,6 +313,15 @@ slicetest: traffic doesn't match the OpenAPI spec:
   stub mail reply (the real service wouldn't answer this way): POST /mail/send responded 200, which specs/mail.yaml doesn't document (documented: 202)
 ```
 
+At the end of the run, each stub with a spec reports which of the provider's operations the app called across all scenarios: its footprint on that API, for planning an upgrade or a switch of provider. Operations the provider marks `deprecated` are flagged, and on GitHub Actions they become a warning annotation and the list goes to the job summary:
+
+```
+slicetest: the app used 3 of 587 operations of stripe (specs/stripe.yaml), 1 deprecated
+  POST /v1/charges           ⚠ deprecated
+  POST /v1/payment_intents
+  GET /v1/customers/{customer}
+```
+
 #### `autoReply`: stubs generated from the provider's spec
 
 Add `autoReply: true` to a stub with a spec, and calls that no route matches are answered with the provider's documented example, or with values built from the schema (formats such as `email` and `date-time`, enums, `minimum`, `allOf` are respected). Register routes only for what a scenario cares about; a registered route always wins.
@@ -351,6 +365,23 @@ stub("anthropic").on("POST", "/v1/messages").reply(sse([
 
 In YAML, `reply: { sse: [{ event: message_start, data: { ... } }, ...] }` instead of `body`.
 
+#### GraphQL: stubs that answer an operation
+
+GraphQL APIs (GitHub, Shopify, Linear, Contentful, your own) take every call at one path, so `on("POST", "/graphql")` can't tell them apart. `stub.graphql()` matches the operation instead, whatever the path, for POSTs with `{ query, variables, operationName }` and GETs with those as query parameters. Without `operationName`, the name in the document counts (`query Viewer { ... }`):
+
+```ts
+stub("github").graphql("Viewer").data({ viewer: { login: "octocat" } });
+stub("github").graphql("CreateIssue", { variables: { title: "Bug" } }).data((call) => ({ createIssue: { issue: { number: 1, title: call.graphql.variables.title } } }));
+stub("github").graphql("CreateIssue").once().errors(["rate limited"]);   // { errors: [{ message }] } with status 200, as servers do
+
+expect(stub("github")).toHaveReceivedGraphQL("CreateIssue", { title: "Bug" });   // variables as a subset
+expect(await http.graphql(REPORT_BUG, { title: "Bug" })).toHaveGraphQLData({ reportBug: { number: 1 } });
+```
+
+`toHaveGraphQLData()` fails on a response with `errors` and prints them, which `toHaveStatus(200)` can't, since GraphQL servers report errors with 200. Unanswered operations fail the scenario named as `GraphQL mutation CreateIssue`. `call.graphql` holds `{ operation, type, query, variables }` for every GraphQL call a stub receives.
+
+In YAML, a stub step takes `graphql: CreateIssue` instead of `on`, `when: { variables }`, and `reply: { data, errors }`; a request takes `graphql: { query, variables }` instead of `json` and fails on `errors` unless `expect.json` names them; a received step takes `graphql:` instead of `call`.
+
 #### Chaos: faults the app must survive
 
 `chaos()` makes a stub misbehave for the rest of the scenario, to test retries, timeouts and fallbacks against the app's real HTTP client:
@@ -430,6 +461,17 @@ SLICETEST_RECORD=github npx vitest     # or SLICETEST_RECORD=1 for every stub wi
 
 Calls no route matches are forwarded to `upstream` (under its path prefix, headers included) and the answers are written to `recordings/github.yaml` (`recordings:` changes the path). Later runs replay them without touching the network. A request is identified by method, path, query and body (JSON key order doesn't matter); identical requests replay their recordings in the order they were made. Only `content-type`, `location`, `retry-after`, `link` and `etag` response headers are kept, and request headers are never stored, so tokens stay out of the file; bodies are stored as sent, so review the file before committing it.
 
+#### From a HAR file: `npx slicetest import`
+
+No credentials at hand, or the call happens in a flow that's easier to click through? Save the network traffic as HAR (the browser's network panel: "Save all as HAR"; Charles, mitmproxy, Proxyman and Postman export it too) and import it:
+
+```sh
+npx slicetest import session.har                       # every stub with an upstream that the HAR has requests for
+npx slicetest import session.har --stub stripe --upstream https://api.stripe.com
+```
+
+Requests under a stub's `upstream` become entries of its recordings file, in the same format and with the same filtering as recording: only the five response headers above, no request headers, the path relative to the upstream's. Preflights, aborted requests and binary responses are skipped, and the hosts it didn't import are listed. Entries already in the file aren't added twice.
+
 Precedence is: registered route, then recording, then `autoReply`, then a 501 that says how to record the call. Replayed calls have `call.fallback === true`, and are checked against the provider's spec when the stub has one. To refresh recordings, delete the file (or the entries) and record again.
 
 ### Matchers
@@ -442,11 +484,14 @@ expect(responses).toHaveStatuses({ 201: 1, 409: 9 });            // an array, e.
 expect(stub("slack")).toHaveReceived("POST", "/hook", { json: { text: "hi" } });
 expect(stub("slack")).toHaveReceivedTimes(1, "POST", "/hook");
 expect(stub("mail")).not.toHaveReceived("POST", "/send");
+expect(stub("github")).toHaveReceivedGraphQL("CreateIssue", { title: "Bug" });
+expect(await http.graphql(QUERY)).toHaveGraphQLData({ poll: { title: "x" } }); // no errors, data as a subset
 await expect(db).toHaveRow("polls", { title: "x" });             // at least one row
 await expect(db).toHaveRow("votes", { poll_id: 1 }, 3);          // exactly three
+expect(res).toMatchSchema("openapi.yaml#/components/schemas/Poll"); // a response's JSON, or any value; inline schemas too
 ```
 
-Failure messages list the calls the stub actually received, or the first rows of the table.
+Failure messages list the calls the stub actually received, or the first rows of the table. `toMatchSchema` takes a JSON Schema object or a file with an optional pointer (relative to the working directory; JSON or YAML, OpenAPI 3.0 `nullable` understood, `$ref`s resolved within the file), and lists every mismatch by its JSON path: no OpenAPI setup is needed to check one response's shape.
 
 ### Services: workers and other processes
 
@@ -607,10 +652,30 @@ Dates (`Date` values and ISO strings) become `[date]` and UUIDs `[uuid]`. Mask m
 
 The example apps share one snapshot file: the Node and the Python implementation must produce the same trace, byte for byte.
 
+### Sequence diagrams of every scenario
+
+`await diagram()` returns the scenario so far as a [Mermaid](https://mermaid.js.org/) sequence diagram: each request to the app, the stub calls the app made while answering it (GraphQL ones by operation), their replies, the mail it sent and the tables it changed:
+
+```mermaid
+sequenceDiagram
+  participant test as scenario
+  participant app
+  participant s_slack as slack (stub)
+  participant db as database
+  test->>+app: POST /polls
+  app->>+s_slack: POST /hook
+  s_slack-->>-app: 200
+  app-->>-test: 201 id: 1
+  Note over app,db: polls +1
+```
+
+- **Living documentation.** `npx slicetest --diagrams docs/flows` (or `SLICETEST_DIAGRAMS=docs/flows` with Vitest) writes one Markdown page per scenario file, with a diagram for each scenario. Commit them, and a pull request that changes how the app talks to the outside world shows it as a diagram diff.
+- **Failures on GitHub Actions.** A failing scenario's diagram goes to the job summary, collapsed under its name, next to the error.
+
 ### Scenarios
 
 ```ts
-scenario("name", async ({ http, db, stub, app, service, container, trace }) => { ... }, timeoutMs?);
+scenario("name", async ({ http, db, stub, app, service, container, trace, diagram }) => { ... }, timeoutMs?);
 scenario.only / scenario.skip / scenario.todo
 scenario.each([{ choice: "a", status: 204 }, { choice: "x", status: 400 }])(
   "voting $choice returns $status",
@@ -627,13 +692,13 @@ Scenarios in one file share an app and a database, so they always run one at a t
 | `app.command` | (required) | Shell command. May use `{{app.port}}` and the other placeholders. |
 | `app.build` | none | Shell command run once per run before the app starts, while the database starts, e.g. `npm run build` for `next start`. It gets `app.env`'s literal values (Next.js inlines `NEXT_PUBLIC_*` at build time), not those with placeholders. Not repeated in watch mode. Services take `build` too. |
 | `db` | Postgres in a container | Database options (below), or `false` for an app without a database: nothing is started, `{{db.*}}` aren't set and `db.*` in scenarios explains that it's off. `slicetest init` writes `false` when it finds no migrations, no database service and no database library. |
-| `app.env` | `{ PORT, DATABASE_URL }` | Values may use `{{app.port}}`, `{{db.url}}`, `{{stub.<name>}}`. For apps that don't take one URL: `{{db.jdbcUrl}}` (`jdbc:postgresql://…`), `{{db.host}}`, `{{db.port}}`, `{{db.name}}`, `{{db.user}}`, `{{db.password}}`. The rest of `process.env` is inherited. |
+| `app.env` | `{ PORT, DATABASE_URL }` | Values may use `{{app.port}}`, `{{db.url}}`, `{{stub.<name>}}`. For apps that don't take one URL: `{{db.jdbcUrl}}` (`jdbc:postgresql://…`), `{{db.adoNet}}` (an ADO.NET connection string, `Host=…;Port=…;Database=…;Username=…;Password=…`, for .NET's `ConnectionStrings__Default`), `{{db.host}}`, `{{db.port}}`, `{{db.name}}`, `{{db.user}}`, `{{db.password}}`. The rest of `process.env` is inherited. |
 | `app.cwd` | vitest root | |
 | `app.ready` | `{ path: "/" }` | Poll a path until it answers below 500, or `{ log: "listening" \| /regex/ }`. |
 | `app.readyTimeout` | `30000` | |
 | `app.scope` | `"file"` | `"worker"`: start the app (and stubs, services) once per Vitest worker and keep it for all of that worker's test files, for apps that start slowly. Sets Vitest's `isolate: false`. |
 | `db.engine` | `postgres`, or `mysql` for a `mysql://` URL | `postgres`, `mysql` (see [MySQL](#mysql)) or `sqlite` (see [SQLite](#sqlite)). |
-| `db.migrate` | none | `{ atlas: { dir } }`, `{ sql: "file-or-dir" }` or `{ command, inputs? }` (gets `DATABASE_URL`). |
+| `db.migrate` | none | `{ atlas: { dir } }`, `{ sql: "file-or-dir" }` or `{ command, inputs?, env? }`. The command gets `DATABASE_URL`, and both it and `env` may use the `{{db.*}}` placeholders, for tools that read other variables: `{ command: "php artisan migrate --force", env: { DB_HOST: "{{db.host}}", DB_DATABASE: "{{db.name}}" } }`, `{ command: "dotnet ef database update --connection \"{{db.adoNet}}\"" }`. |
 | `db.seed` | none | SQL file re-run after every reset. |
 | `db.schemas` | `["public"]` | Schemas whose tables are reset. |
 | `db.keep` | `[]` | Extra tables (`name` or `schema.name`) never truncated. |
@@ -647,6 +712,7 @@ Scenarios in one file share an app and a database, so they always run one at a t
 | `auth` | `false` | `true` or `{ audience, claims }`: an OpenID Connect issuer at `{{auth.issuer}}` (JWKS at `{{auth.jwks}}`) whose tokens scenarios mint with `auth.token()`. See [Auth](#auth-a-real-openid-issuer-tokens-with-any-claims). |
 | `services` | `{}` | Other processes: `{ name: { command, env?, cwd?, ready?, readyTimeout? } }`. Without `ready` a service is not waited for. |
 | `offline` | `false` | Refuse the app's HTTP(S) calls to hosts no stub intercepts, and fail the scenario naming them. |
+| `strictStubs` | `false` | Fail a scenario that registered a stub route the app never called, so a test can't pass without reaching the code it set up for. Exempt a route with `.optional()` (YAML `optional: true`). Unused routes are listed in the failure output either way. |
 | `workers` | Vitest's default | Most Vitest workers (`maxWorkers`). Each has its own app and database. |
 | `stubs` | `[]` | Names of stubbed services, or `{ name, openapi?, autoReply?, upstream?, recordings?, hosts? }`: check calls against the provider's spec, answer from it, [replay recordings](#recording-a-real-service) of the real service, or answer for [hard-coded hosts](#hard-coded-hosts-hosts). |
 | `openapi` | none | The app's OpenAPI 3 spec, or `{ spec, minCoverage }`, or `{ fromApp: "/v3/api-docs" }` for a spec the running app serves (springdoc, FastAPI's `/openapi.json`, NestJS). Every response must match it; the run ends with a coverage report. |
@@ -700,9 +766,9 @@ scenarios:
 
 | Step | Keys |
 |---|---|
-| `stub: <name>` | `on: METHOD /path` (`:params` allowed), `when: { query, headers, json, body }`, one of `reply: { status, headers, body }` / `sequence: [...]` / `networkError: true`, plus `times`, `delay`. Replies may echo the call: `{{call.params.id}}`, `{{call.json.name}}`. |
+| `stub: <name>` | `on: METHOD /path` (`:params` allowed) or `graphql: <operation>`, `when: { query, headers, json, body, variables }`, one of `reply: { status, headers, body }` (`{ data, errors }` for GraphQL) / `sequence: [...]` / `networkError: true`, plus `times`, `delay`. Replies may echo the call: `{{call.params.id}}`, `{{call.json.name}}`, `{{call.variables.id}}`. |
 | `submit: <button>` | `form`, `fields`, `headers`, `follow`, `expect: { status, headers, json, text }`, `capture`. Submits a form of the page the last request returned, like `http.submit()`; `submit: true` presses the form's only button. |
-| `request: METHOD /path` | `headers`, `query`, one of `json` / `form` / `body`, `follow`, `expect: { status, headers, json, text }`, `capture`. `concurrency: n` sends it `n` times at once; `expect` then applies to each response, and `expect.statuses: { 201: 1, 409: 9 }` counts them. |
+| `request: METHOD /path` | `headers`, `query`, one of `json` / `form` / `body` / `graphql: { query, variables, operationName }`, `expect.schema` (a JSON Schema, or `../openapi.yaml#/components/schemas/Poll` relative to the file), `follow`, `expect: { status, headers, json, text }`, `capture`. `concurrency: n` sends it `n` times at once; `expect` then applies to each response, and `expect.statuses: { 201: 1, 409: 9 }` counts them. |
 | `insert: <table>` | `rows`, `capture` (from `row` / `rows`) |
 | `request` with `auth` | `auth: true` or the claims: sends a bearer token from the `auth` issuer |
 | `request` with `webhook` | `{ provider, secret, event, stale, invalidSignature }`: signs the body like that provider's deliveries |
@@ -710,11 +776,12 @@ scenarios:
 | `make: <table>` | `rows` (a mapping, or a list for several rows), `count`, `capture` (from `row` / `rows`) — like `db.make()` |
 | `db: <table>` | `where`, `orderBy`, `expect: { rows, count }`, `capture` |
 | `sql: <query>` | `params`, `expect: { rows, count }`, `capture` |
-| `received: <stub>` | `call: METHOD /path`, `when`, `times` (exact; default at least once) |
+| `received: <stub>` | `call: METHOD /path` or `graphql: <operation>`, `when`, `times` (exact; default at least once) |
 | `log: <regex>` | `from` (a service; default the app), `within` (ms, default 5000). Waits for a matching line printed during the scenario. |
 | `changes: { <table>: { inserted, updated, deleted } }` | Each is a count or a list of subset rows (`updated` matches the row after the update). Tables that aren't listed must be unchanged. |
 | `checkpoint: true` | Later `changes` steps only see what happens after this step. |
 | `mail: { to, from, subject, text, html }` | `times` (exact; default at least one), `within` (ms, default 5000), `capture` from the last match (`subject`, `text`, `links.0`). Waits for mail the app sends. `{}` matches any message. |
+| `use: <definition>` | `with: { param: value }`. Runs the steps of a `define:` entry. |
 | `snapshot: true` | The scenario's [trace](#snapshot-the-whole-scenario-trace) so far must match its stored snapshot. `mask: [keys]` hides more values. |
 
 `db`, `sql`, `received` and `changes` steps take `within: <ms>` to retry until they pass, for effects the app applies asynchronously.
@@ -723,8 +790,36 @@ scenarios:
 - `{{name}}` inserts a captured value or an `each` field. A string that is only `{{name}}` keeps the value's type, so `id: "{{pollId}}"` compares as a number.
 - Expected `json`, `rows` and `headers` are subsets: extra keys are fine. `{ $type: number }`, `{ $regex: "^ch_" }`, `{ $contains: "..." }` and `{ $any: true }` match loosely.
 - A file-level `setup:` list runs at the start of every scenario. `skip`, `only` and `timeout` work per scenario.
+- A file-level `define:` names step lists that `use:` steps run, like functions: `params` are given with `with:` and are `{{variables}}` inside, and what the steps capture is visible after the `use` (see below).
 - Mistakes are reported with the file and line before anything runs (`polls.scenario.yaml:12: unknown key "stauts" in expect`). A failing step reports its file, line and step number. The JSON Schema in `schema/` gives editors completion and inline errors.
 
+### Reusing steps: `define` and `use`
+
+```yaml
+define:
+  signed up:
+    params: [email]
+    steps:
+      - request: POST /signup
+        json: { email: "{{email}}", password: hunter22 }
+        expect: { status: 201 }
+        capture: { userId: json.id }
+      - request: POST /login
+        json: { email: "{{email}}", password: hunter22 }
+        capture: { token: json.token }
+
+scenarios:
+  - name: a new user has an empty cart
+    steps:
+      - use: signed up
+        with: { email: ada@example.com }
+      - request: GET /users/{{userId}}/cart
+        headers: { authorization: "Bearer {{token}}" }
+        expect: { json: { items: [] } }
+```
+
+A definition is a list of steps, or `{ params, steps }`, and may `use` other definitions. Unknown names, missing or extra `with` keys and definitions that use themselves are reported with their line before anything runs. A failing step inside one names the whole path: `cart.scenario.yaml:5 (a new user has an empty cart, step 1: use signed up → step 2: POST /login)`.
+
 ### Without any JavaScript: `npx slicetest`
 
 Put the plugin options in `slicetest.config.yaml` and run the CLI. It needs Node, but no `package.json` scripts, TypeScript or Vitest config. To add TypeScript scenarios later, `slicetest()` without options in a `vitest.config.ts` reads the same file (`slicetest("path/to/config.yaml")` for another one):

package/dist/cli.js

@@ -15,6 +15,7 @@ const HELP = `Usage: slicetest [filters...] [options]
        slicetest gen [--spec <file>] [--out <dir>] [--uncovered] [--force]
        slicetest doctor [--config <file>]
        slicetest record [--out <file>] [--port <n>]
+       slicetest import <file.har> [--stub <name>] [--upstream <url>]
 
 Runs *.scenario.yaml files against your app, as configured in slicetest.config.yaml.
 \`slicetest init\` looks at the project and writes a starting config and scenario.
@@ -24,6 +25,9 @@ app's OpenAPI spec; with --uncovered, only for those the last run didn't produce
 migrations, commands and spec files, and says what to fix.
 \`slicetest record\` starts everything and a proxy in front of the app: use the app
 through it (a browser, curl), press Enter, and get the session as a YAML scenario.
+\`slicetest import\` turns a HAR file (the browser's network panel: "Save all as
+HAR"; Charles, mitmproxy, Proxyman) into recordings for the stubs whose
+\`upstream\` it has requests for, so they replay the real service's answers.
 
 Options:
   -c, --config <file>  Config file (default: ${CONFIG_NAMES.join(" / ")} in the current directory)
@@ -33,8 +37,12 @@ Options:
       --out <dir>      gen: where to write scenarios (default: scenarios)
                        record: the scenario file (default: scenarios/recorded-<time>.scenario.yaml)
       --port <n>       record: the proxy's port (default: any free port)
+      --stub <name>    import: only this stub (with --upstream, one not in the config)
+      --upstream <url> import: the real service's base URL for --stub
       --uncovered      gen: only responses the last run didn't cover
       --force          init, gen: overwrite existing files
+      --diagrams <dir> Write a Mermaid sequence diagram of every scenario to <dir>,
+                       one Markdown page per scenario file
   -h, --help           Show this help
 
 Config (paths are relative to the config file):
@@ -47,6 +55,8 @@ Config (paths are relative to the config file):
   auth:     true | { audience, claims }   OpenID issuer at {{auth.issuer}} / {{auth.jwks}}
   openapi:  file | { spec, minCoverage }
   http:     { headers, query }
+  offline:  true    refuse calls to hosts no stub answers
+  strictStubs: true fail scenarios with stub routes the app never called
   include:  [globs]  default ["**/*.scenario.{yaml,yml}"]
 `;
 export async function main(argv = process.argv.slice(2)) {
@@ -63,6 +73,9 @@ export async function main(argv = process.argv.slice(2)) {
             out: { type: "string" },
             uncovered: { type: "boolean" },
             port: { type: "string" },
+            diagrams: { type: "string" },
+            stub: { type: "string" },
+            upstream: { type: "string" },
         },
     });
     if (values.help) {
@@ -106,6 +119,36 @@ export async function main(argv = process.argv.slice(2)) {
         process.stdout.write(`${lines.join("\n")}\n`);
         return;
     }
+    if (positionals[0] === "import") {
+        const har = positionals[1];
+        if (!har)
+            throw new Error("slicetest import: which HAR file? e.g. npx slicetest import session.har");
+        const config = configPath && existsSync(configPath) ? (parse(await readFile(configPath, "utf8")) ?? {}) : undefined;
+        const root = configPath && config ? path.dirname(configPath) : process.cwd();
+        const declared = (config?.stubs ?? []).flatMap((s) => (typeof s === "object" && s.upstream ? [s] : []));
+        let targets = declared.map((s) => ({ name: s.name, upstream: s.upstream, file: path.resolve(root, s.recordings ?? `recordings/${s.name}.yaml`) }));
+        if (values.stub) {
+            const known = targets.find((t) => t.name === values.stub);
+            const upstream = values.upstream ?? known?.upstream;
+            if (!upstream)
+                throw new Error(`slicetest import: stub "${values.stub}" has no upstream in the config; pass --upstream https://api.example.com`);
+            targets = [{ name: values.stub, upstream, file: known?.file ?? path.resolve(root, `recordings/${values.stub}.yaml`) }];
+        }
+        if (targets.length === 0)
+            throw new Error("slicetest import: no stub to import into. Give stubs an `upstream` in the config, or pass --stub <name> --upstream <url>.");
+        const { importHar } = await import("./har.js");
+        const { written, skipped, others } = await importHar(path.resolve(har), targets);
+        const lines = [
+            ...written.map((w) => `  ${w.name}: ${w.count} recording(s) → ${path.relative(process.cwd(), w.file)}`),
+            ...(written.length === 0 ? [`No requests in ${har} go to ${targets.map((t) => t.upstream).join(", ")}.`] : []),
+            ...(skipped ? [`  skipped ${skipped} preflight, aborted or binary request(s)`] : []),
+            ...(others.length ? ["", "Requests to other hosts (not imported):", ...others.slice(0, 10).map(([h, n]) => `  ${h} (${n})`), "Import one with --stub <name> --upstream <url>."] : []),
+        ];
+        process.stdout.write(`${lines.join("\n")}\n`);
+        if (written.length === 0)
+            process.exitCode = 1;
+        return;
+    }
     if (!configPath || !existsSync(configPath)) {
         process.stderr.write(`slicetest: no config found. Run "npx slicetest init" to create ${CONFIG_NAMES[0]} (see --help).\n`);
         process.exitCode = 1;
@@ -117,6 +160,9 @@ export async function main(argv = process.argv.slice(2)) {
         await record(configPath, options, { out: values.out, port: values.port ? Number(values.port) : 0 });
         return;
     }
+    // Workers inherit the environment; a relative directory is taken from where the command runs.
+    if (values.diagrams)
+        process.env.SLICETEST_DIAGRAMS = path.resolve(values.diagrams);
     const { startVitest, version } = await import("vitest/node");
     if (Number.parseInt(version, 10) < 4) {
         process.stderr.write(`slicetest: needs Vitest 4 or later, and this project has Vitest ${version}. Upgrade it (npm i -D vitest@latest), or run slicetest from a folder with its own package.json.\n`);

package/dist/config.d.ts

@@ -52,6 +52,11 @@ export interface SlicetestOptions {
      * scenario, naming the host, so a forgotten stub can't reach a real service.
      */
     offline?: boolean;
+    /**
+     * Fail a scenario that registered a stub route the app never called: the test may not be
+     * exercising what it set up. Routes marked `.optional()` (YAML `optional: true`) are exempt.
+     */
+    strictStubs?: boolean;
     /**
      * Most Vitest workers to run test files in (Vitest's `maxWorkers`). Each worker has
      * its own app and database; with `app.scope: "worker"`, fewer workers means fewer app
@@ -204,6 +209,12 @@ export type MigrateOptions = {
      * without `inputs`, the command runs on every run.
      */
     inputs?: string[];
+    /**
+     * Extra environment for the command, for tools that don't read `DATABASE_URL`
+     * (Laravel's `DB_HOST`, EF Core's connection string): `{ DB_HOST: "{{db.host}}" }`.
+     * The command itself may use the same `{{db.*}}` placeholders.
+     */
+    env?: Record<string, string>;
 };
 /** Normalized shape passed from the plugin to globalSetup and workers. Must stay JSON-serializable. */
 export interface ResolvedOptions {
@@ -216,6 +227,7 @@ export interface ResolvedOptions {
     containers: Record<string, ContainerOptions>;
     mail: boolean;
     offline: boolean;
+    strictStubs: boolean;
     workers?: number;
     auth: AuthOptions | false;
     db: Required<Pick<DbOptions, "engine" | "image" | "schemas" | "keep" | "reuse">> & Omit<DbOptions, "engine" | "image" | "schemas" | "keep" | "reuse"> & {

package/dist/config.js

@@ -9,6 +9,7 @@ export function resolveOptions(opts, root) {
         containers: opts.containers ?? {},
         mail: opts.mail ?? false,
         offline: opts.offline ?? false,
+        strictStubs: opts.strictStubs ?? false,
         workers: opts.workers,
         auth: opts.auth === true ? {} : (opts.auth ?? false),
         db: opts.db === false ? { ...resolveDb({}), none: true } : resolveDb(opts.db ?? {}),
@@ -104,6 +105,8 @@ function validate(opts) {
         fail(`workers must be a positive whole number, got ${JSON.stringify(opts.workers)}`);
     if (opts.offline !== undefined && typeof opts.offline !== "boolean")
         fail(`offline must be true or false, got ${JSON.stringify(opts.offline)}`);
+    if (opts.strictStubs !== undefined && typeof opts.strictStubs !== "boolean")
+        fail(`strictStubs must be true or false, got ${JSON.stringify(opts.strictStubs)}`);
     if (opts.mail !== undefined && typeof opts.mail !== "boolean")
         fail(`mail must be true or false, got ${JSON.stringify(opts.mail)}`);
     if (opts.auth !== undefined && typeof opts.auth !== "boolean") {
@@ -136,6 +139,13 @@ function validate(opts) {
         const keys = Object.keys(migrate).filter((k) => ["atlas", "sql", "command"].includes(k));
         if (keys.length !== 1)
             fail(`db.migrate takes exactly one of atlas / sql / command, got ${keys.join(", ") || "none"}`);
+        const menv = migrate.env;
+        if (menv !== undefined) {
+            if (!("command" in migrate))
+                fail("db.migrate.env is for a migration `command`; atlas and sql get the database URL themselves");
+            if (!menv || typeof menv !== "object" || Array.isArray(menv) || Object.values(menv).some((v) => typeof v !== "string"))
+                fail('db.migrate.env maps variable names to strings, e.g. { DB_HOST: "{{db.host}}" }');
+        }
     }
     const oas = opts.openapi;
     if (oas !== undefined) {

package/dist/connection.d.ts

@@ -0,0 +1,5 @@
+/**
+ * The parts of the database URL, for apps that don't take one URL: JDBC (Spring's
+ * `spring.datasource.url` plus username / password), or separate host / port / name settings.
+ */
+export declare function connectionVars(engine: string, url: string, sqlitePath?: string): Record<string, string>;

package/dist/connection.js

@@ -0,0 +1,31 @@
+/**
+ * The parts of the database URL, for apps that don't take one URL: JDBC (Spring's
+ * `spring.datasource.url` plus username / password), or separate host / port / name settings.
+ */
+export function connectionVars(engine, url, sqlitePath) {
+    if (engine === "sqlite")
+        return { "db.jdbcUrl": `jdbc:sqlite:${sqlitePath}`, "db.adoNet": `Data Source=${sqlitePath}` };
+    const u = new URL(url);
+    const port = u.port || (engine === "mysql" ? "3306" : "5432");
+    const name = decodeURIComponent(u.pathname.replace(/^\//, ""));
+    return {
+        "db.host": u.hostname,
+        "db.port": port,
+        "db.name": name,
+        "db.user": decodeURIComponent(u.username),
+        "db.password": decodeURIComponent(u.password),
+        "db.jdbcUrl": `jdbc:${engine === "mysql" ? "mysql" : "postgresql"}://${u.hostname}:${port}/${encodeURIComponent(name)}`,
+        // ADO.NET (Npgsql, MySqlConnector), as .NET apps read `ConnectionStrings__Default`.
+        "db.adoNet": [
+            `${engine === "mysql" ? "Server" : "Host"}=${u.hostname}`,
+            `Port=${port}`,
+            `Database=${name}`,
+            `${engine === "mysql" ? "User ID" : "Username"}=${decodeURIComponent(u.username)}`,
+            `Password=${adoValue(decodeURIComponent(u.password))}`,
+        ].join(";"),
+    };
+}
+/** ADO.NET values with `;` or quotes are quoted. */
+function adoValue(v) {
+    return /[;'"]/.test(v) ? `"${v.replace(/"/g, '""')}"` : v;
+}

package/dist/diagram.d.ts

@@ -0,0 +1,13 @@
+import type { Changes } from "./db.js";
+import type { HttpResponse } from "./http.js";
+import type { Mailbox } from "./mail.js";
+import type { Stub } from "./stub.js";
+/**
+ * A Mermaid sequence diagram of what a scenario did: each request to the app,
+ * the stub calls the app made while answering it, the mail it sent and the
+ * database tables it changed. GitHub, GitLab and most Markdown viewers render it.
+ */
+export declare function sequenceDiagram(history: readonly HttpResponse[], stubs: Iterable<Stub>, changes: Changes | undefined, mailbox?: Mailbox): string;
+export declare function diagramPage(file: string, title: string, scenario: string, diagram: string, failed: boolean): string;
+/** A collapsed section for the GitHub Actions job summary. */
+export declare function failureDiagram(where: string, scenario: string, diagram: string): string;

package/dist/diagram.js

@@ -0,0 +1,89 @@
+import { describeGraphQL } from "./graphql.js";
+import { timeline } from "./timeline.js";
+/**
+ * A Mermaid sequence diagram of what a scenario did: each request to the app,
+ * the stub calls the app made while answering it, the mail it sent and the
+ * database tables it changed. GitHub, GitLab and most Markdown viewers render it.
+ */
+export function sequenceDiagram(history, stubs, changes, mailbox) {
+    const events = [];
+    const participants = new Map();
+    const id = (name, label) => {
+        const key = `s_${name.replace(/[^A-Za-z0-9_]/g, "_")}`;
+        if (!participants.has(key))
+            participants.set(key, label);
+        return key;
+    };
+    let order = 0;
+    const add = (at, line) => events.push({ at, order: order++, line });
+    for (const res of history) {
+        const t = timeline.get(res) ?? { start: Infinity, end: Infinity };
+        const external = /^https?:/.test(res.url);
+        const target = external ? id(`ext_${new URL(res.url).host}`, new URL(res.url).host) : "app";
+        const path = external ? new URL(res.url).pathname : res.url;
+        add(t.start, `test->>+${target}: ${text(`${res.method} ${path}`)}`);
+        add(t.end ?? t.start, `${target}-->>-test: ${res.status || "failed"}${summary(res)}`);
+    }
+    for (const stub of stubs) {
+        for (const call of stub.calls()) {
+            const t = timeline.get(call) ?? { start: Infinity };
+            const s = id(stub.name, `${stub.name} (stub)`);
+            add(t.start, `app->>+${s}: ${text(describeCall(call))}`);
+            const reply = call.fault === "reset" ? "connection dropped" : call.response ? `${call.response.status}${call.fault ? " (chaos)" : ""}` : call.matched ? "no answer" : "501 no stub";
+            add(t.end ?? t.start, `${s}-->>-app: ${reply}`);
+        }
+    }
+    events.sort((a, b) => a.at - b.at || a.order - b.order);
+    const lines = ["sequenceDiagram", "  participant test as scenario", "  participant app"];
+    for (const [key, label] of participants)
+        lines.push(`  participant ${key} as ${text(label)}`);
+    if (mailbox && mailbox.messages().length)
+        lines.push("  participant mail");
+    const tables = Object.entries(changes ?? {}).filter(([, c]) => c.inserted.length || c.updated.length || c.deleted.length);
+    if (tables.length)
+        lines.push("  participant db as database");
+    for (const e of events)
+        lines.push(`  ${e.line}`);
+    for (const m of mailbox?.messages() ?? [])
+        lines.push(`  app->>mail: ${text(`${m.subject} to ${m.to.join(", ")}`)}`);
+    if (tables.length) {
+        const counts = tables.map(([table, c]) => `${table} ${[c.inserted.length && `+${c.inserted.length}`, c.updated.length && `~${c.updated.length}`, c.deleted.length && `-${c.deleted.length}`].filter(Boolean).join(" ")}`);
+        lines.push(`  Note over app,db: ${text(counts.join(", "))}`);
+    }
+    return lines.join("\n");
+}
+function describeCall(call) {
+    if (call.graphql)
+        return describeGraphQL(call.graphql);
+    return `${call.method} ${call.path}`;
+}
+/** A short hint of the response: a JSON error message or the id it created. */
+function summary(res) {
+    const j = res.json;
+    if (j && typeof j === "object" && !Array.isArray(j)) {
+        for (const key of ["error", "message", "id"]) {
+            const v = j[key];
+            if (typeof v === "string" || typeof v === "number")
+                return ` ${text(`${key}: ${v}`)}`;
+        }
+    }
+    return "";
+}
+/** Mermaid ends a message at `;` and reads `#…;` as an entity, so both are escaped; long text is cut. */
+function text(s, max = 70) {
+    const cut = s.length > max ? `${s.slice(0, max - 1)}…` : s;
+    return cut.replace(/[\r\n]+/g, " ").replace(/[#;]/g, (c) => (c === "#" ? "#35;" : "#59;"));
+}
+/** Scenario diagrams per test file, for `SLICETEST_DIAGRAMS` / `--diagrams`: one Markdown page per file, in run order. */
+const pages = new Map();
+export function diagramPage(file, title, scenario, diagram, failed) {
+    const page = pages.get(file) ?? { title, sections: new Map() };
+    pages.set(file, page);
+    page.sections.set(scenario, [`## ${scenario}${failed ? " (failed)" : ""}`, "", "```mermaid", diagram, "```", ""].join("\n"));
+    return [`# ${page.title}`, "", `Sequence diagrams of the scenarios in \`${page.title}\`, written by slicetest. Regenerated on every run.`, "", ...page.sections.values()].join("\n");
+}
+/** A collapsed section for the GitHub Actions job summary. */
+export function failureDiagram(where, scenario, diagram) {
+    const name = scenario.replace(/[<>&]/g, (c) => ({ "<": "&lt;", ">": "&gt;", "&": "&amp;" })[c]);
+    return [`<details><summary>✗ ${name} <code>${where}</code>: what happened</summary>`, "", "```mermaid", diagram, "```", "", "</details>", ""].join("\n");
+}