slicetest 0.5.0 → 0.6.1

package/README.md

@@ -55,6 +55,8 @@ npx slicetest        # starts Postgres, migrates, starts your app, runs scenario
 - **Real token verification, any user.** `auth: true` gives the app an OpenID issuer with a JWKS, so JWT checks stay on in tests, and scenarios mint tokens with any claims, including expired or foreign-signed ones.
 - **Webhooks signed like the real sender.** Stripe, GitHub, Slack, Shopify and Standard Webhooks signatures, plus forged and replayed deliveries, so signature checks are tested instead of bypassed.
 - **Reproducible chaos.** Stubs can fail the first calls, drop connections or add latency, from a seed the failure output prints, so a resilience test that fails once fails again on demand.
+- **Forms as a browser sends them.** `http.submit()` presses a button on a server-rendered page, hidden fields included, so CSRF tokens and Next.js server actions work without knowing their internals.
+- **Hard-coded APIs, stubbed anyway.** `hosts: [api.github.com]` catches calls to URLs written in the code or built into a framework, over HTTPS, from Node, Python, Go, Ruby or the JVM, with no change to the app. Redirects to those hosts are followed to the stub, so OAuth logins run end to end.
 - **N+1 detection for any stack.** A wire-protocol proxy records the SQL the app runs, so query counts are asserted at the HTTP boundary, whatever the ORM or language.
 - **Postgres, MySQL or SQLite**, with the same scenarios and the same helpers on all three, plus Redis, MinIO or any other `containers` reset between scenarios.
 - **Fast resets.** `TRUNCATE` between scenarios (about 1.5 ms) with the app still running, and a cached migrated template, so the second run skips container start-up and migrations.
@@ -171,12 +173,30 @@ res.status; res.headers; res.text; res.json; res.durationMs;
 await http.get("/polls", { query: { page: 2 }, headers: { accept: "text/html" } });
 await http.post("/login", http.form({ user: "a", pass: "b" }));  // urlencoded; FormData, Blob and bytes also work
 await http.get("/old-path", { follow: true });                    // redirects are NOT followed by default
+await http.submit(await http.get("/signup"), { button: "Sign up", fields: { email: "a@b.test" } }); // a form, as a browser sends it
 
 const admin = http.with({ headers: { authorization: `Bearer ${token}` } }); // shares cookies with http
 http.cookies.get("session");                                     // cookies persist within a scenario
 ```
 
-Requests may only go to the app under test; absolute URLs to other hosts are rejected. Defaults for every request can be set with `http: { headers }` in the plugin config.
+Requests may only go to the app under test; absolute URLs to other hosts are rejected. Defaults for every request can be set with `http: { headers }` in the plugin config. With `follow`, cookies set by each redirect are kept (a login answering `302` with `Set-Cookie`), `303` and `301`/`302` turn into a `GET` as in browsers, and a redirect to another host is returned instead of followed. Cookies follow their `Path` as in browsers.
+
+#### Forms: `http.submit()`
+
+`http.submit(page, opts)` sends a form of a page the app returned, the way a browser with JavaScript off does: every field with its value, the hidden ones included, plus the button that was pressed. So the test doesn't need to know about CSRF tokens (Django, Rails, Laravel) or the action ids and bound arguments of **Next.js server actions** — they are hidden inputs, sent along like a browser sends them.
+
+```ts
+const page = await http.get(`/q/${q.id}`);                    // a Next.js page with <button formAction={vote.bind(null, id, "a")}>
+const res = await http.submit(page, { button: "Dogs", follow: true });
+expect(res.text).toContain("Your choice");
+
+await http.submit(await http.get("/settings"), {
+  form: "profile",                                            // by id, name or position, when the page has several
+  fields: { name: "Ada", newsletter: true, tags: ["a", "b"] }, // checkboxes and radios by true / false / value
+});
+```
+
+`button` matches a submit button's text, `value`, `name` or `id`, and picks its form; its `formaction` / `formmethod` / `formenctype` apply. `fields` replace what the page had and must name existing fields, so a typo fails. When nothing matches, the error lists the page's forms and their buttons. In YAML, `submit:` uses the page the previous step requested.
 
 ### `db` — arrange and inspect the real database
 
@@ -312,6 +332,25 @@ To fail the run below a threshold, use `openapi: { spec: "openapi.yaml", minCove
 
 The example apps in `examples/` run every scenario against `examples/openapi.yaml` with `minCoverage: 100`, and their Slack calls against `examples/slack.openapi.yaml`.
 
+#### Streaming replies: `sse()`
+
+LLM APIs stream their answers as Server-Sent Events. `sse(events)` makes such a reply, each event as `[event, data]` or `{ event, data, id }`, data that isn't a string as JSON:
+
+```ts
+import { sse } from "slicetest";
+
+stub("anthropic").on("POST", "/v1/messages").reply(sse([
+  ["message_start", { type: "message_start", message: { id: "msg_1", type: "message", role: "assistant", content: [], model: "claude-sonnet-4-6", usage: { input_tokens: 5, output_tokens: 1 } } }],
+  ["content_block_start", { type: "content_block_start", index: 0, content_block: { type: "text", text: "" } }],
+  ["content_block_delta", { type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "Hello" } }],
+  ["content_block_stop", { type: "content_block_stop", index: 0 }],
+  ["message_delta", { type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } }],
+  ["message_stop", { type: "message_stop" }],
+]));
+```
+
+In YAML, `reply: { sse: [{ event: message_start, data: { ... } }, ...] }` instead of `body`.
+
 #### Chaos: faults the app must survive
 
 `chaos()` makes a stub misbehave for the rest of the scenario, to test retries, timeouts and fallbacks against the app's real HTTP client:
@@ -327,6 +366,54 @@ stub("search").chaos({ errorRate: 0.3, networkErrorRate: 0.1, latency: [50, 300]
 
 Faulted calls don't use up `once()` / `times()` routes, so a retry gets the answer you registered. 429 and 503 come with `Retry-After: 1`. Random faults are drawn from a seeded generator: a failing scenario prints `chaos on search: …; 4 of 12 calls faulted. Replay with SLICETEST_CHAOS_SEED=1840211`, and running with that variable gives the same faults. `stub.faults()` lists the calls that faulted. YAML: `- chaos: payments` with `failFirst`, `errorRate`, `statuses`, `networkErrorRate`, `latency` and `seed`.
 
+### Hard-coded hosts: `hosts`
+
+Stubs normally take over by giving the app their URL (`{{stub.github}}`) instead of the real one. When the URL is written in the code (`https://api.github.com`), or built into a framework (Spring Security's GitHub login), give the stub the hosts instead:
+
+```yaml
+stubs:
+  - name: github
+    hosts: [github.com]       # OAuth authorize and token endpoints
+  - name: github-api
+    hosts: [api.github.com]
+```
+
+The app is then started with `HTTPS_PROXY` / `HTTP_PROXY` pointing at slicetest and a certificate authority made for the run in the trust settings each runtime reads: `NODE_USE_ENV_PROXY` and `NODE_EXTRA_CA_CERTS` for Node (22.21+ / 24.5+), `SSL_CERT_FILE` / `REQUESTS_CA_BUNDLE` for Python, Ruby, Go and curl, and proxy and trust-store system properties in `JAVA_TOOL_OPTIONS` for the JVM (`HttpURLConnection`, `java.net.http.HttpClient`, Spring's `RestClient` and `RestTemplate`). Calls to those hosts reach the stub with their path and `Host` header, over HTTPS or HTTP; calls to other hosts go to the real ones and are listed in the failure output. Nothing changes in the app. Variables you set in `app.env` win over these, and `{{proxy.url}}`, `{{proxy.ca}}`, `{{proxy.bundle}}` and `{{proxy.truststore}}` are there for clients configured some other way.
+
+`*.connpass.com` covers every subdomain (one stub for `findy.connpass.com`, `mercari.connpass.com`, …); the stub sees the `Host` header, so a reply can depend on it (`{{call.headers.host}}`, or `call.headers.host` in a reply function).
+
+With `offline: true`, the app can only reach localhost and the stubs' hosts. A call anywhere else is refused and fails the scenario with the host's name, so a forgotten stub can't quietly reach a real service, and the first run tells you which hosts to add:
+
+```
+slicetest: offline: the app tried to reach api.lu.ma, which no stub answers. Add it to a stub's `hosts` (or remove `offline`).
+```
+
+Node apps also get a small preload (`NODE_OPTIONS=--require …`) that gives the proxy settings to `http.Agent`s that libraries make themselves (the Stripe SDK, many API clients), which `NODE_USE_ENV_PROXY` alone doesn't reach. A client in any language that ignores proxy settings altogether goes straight to the real host; `offline` can't stop what doesn't pass through it.
+
+The proxy settings reach every process the app command starts. When that command is a build tool (`gradle bootRun`, `mvn spring-boot:run`, `go run`), its own downloads go through the proxy too, and `offline` refuses them; the failure says so when the host is a package registry. Download dependencies in `app.build` (`./gradlew bootJar`, `mvn package`, which `slicetest init` sets up for Spring Boot), or start a built artifact.
+
+A redirect to an intercepted host is followed to its stub, as a browser would follow it to the real site. So a whole OAuth login runs in a scenario: the stub plays the provider's consent page and sends the browser back.
+
+```yaml
+setup:
+  - stub: github
+    on: GET /login/oauth/authorize
+    reply: { status: 302, headers: { location: "{{call.query.redirect_uri}}?code=c1&state={{call.query.state}}" } }
+  - stub: github
+    on: POST /login/oauth/access_token
+    reply: { body: { access_token: gho_test, token_type: bearer } }
+  - stub: github-api
+    on: GET /user
+    reply: { body: { id: 1, login: octocat } }
+scenarios:
+  - name: log in with GitHub
+    steps:
+      - request: GET /oauth2/authorization/github
+        follow: true                       # → github.com (stub) → back to the app's callback
+      - request: GET /api/me
+        expect: { json: { login: octocat } }
+```
+
 ### Recording a real service
 
 A stub can also answer from recordings of the real service, the way VCR or Polly do, except that it works for an app in any language because the stub is a server. Give it the real base URL:
@@ -464,6 +551,8 @@ scenario("the app rejects tokens it must not trust", async ({ http, auth }) => {
 
 `auth.token(claims, opts)` returns the JWT itself. Tokens get `iss`, `aud`, `sub: "user-1"`, `iat`, `nbf`, `exp` (1 hour, or `expiresIn`) and the configured `claims`, all overridable. `auth.rotate()` switches to a new signing key, to check that the app refetches the JWKS. Apps that fetch tokens themselves can use `POST {{auth.issuer}}/token` with the client-credentials grant: `sub` is the client id, and `scope` and `audience` are carried over. No dependencies: keys and signatures come from `node:crypto`.
 
+For libraries that don't take an issuer URL: `{{auth.publicKey}}` is the signing key as a PEM public key, and `auth.jwks` the JWKS document, for a stub to serve at the provider's own URL (see [Clerk](#clerk)).
+
 In YAML, `auth` on a `request` step sends `Authorization: Bearer` with those claims (`auth: true` for the defaults):
 
 ```yaml
@@ -536,15 +625,19 @@ Scenarios in one file share an app and a database, so they always run one at a t
 | Option | Default | |
 |---|---|---|
 | `app.command` | (required) | Shell command. May use `{{app.port}}` and the other placeholders. |
-| `app.env` | `{ PORT, DATABASE_URL }` | Values may use `{{app.port}}`, `{{db.url}}`, `{{stub.<name>}}`. The rest of `process.env` is inherited. |
+| `app.build` | none | Shell command run once per run before the app starts, while the database starts, e.g. `npm run build` for `next start`. It gets `app.env`'s literal values (Next.js inlines `NEXT_PUBLIC_*` at build time), not those with placeholders. Not repeated in watch mode. Services take `build` too. |
+| `db` | Postgres in a container | Database options (below), or `false` for an app without a database: nothing is started, `{{db.*}}` aren't set and `db.*` in scenarios explains that it's off. `slicetest init` writes `false` when it finds no migrations, no database service and no database library. |
+| `app.env` | `{ PORT, DATABASE_URL }` | Values may use `{{app.port}}`, `{{db.url}}`, `{{stub.<name>}}`. For apps that don't take one URL: `{{db.jdbcUrl}}` (`jdbc:postgresql://…`), `{{db.host}}`, `{{db.port}}`, `{{db.name}}`, `{{db.user}}`, `{{db.password}}`. The rest of `process.env` is inherited. |
 | `app.cwd` | vitest root | |
 | `app.ready` | `{ path: "/" }` | Poll a path until it answers below 500, or `{ log: "listening" \| /regex/ }`. |
 | `app.readyTimeout` | `30000` | |
+| `app.scope` | `"file"` | `"worker"`: start the app (and stubs, services) once per Vitest worker and keep it for all of that worker's test files, for apps that start slowly. Sets Vitest's `isolate: false`. |
 | `db.engine` | `postgres`, or `mysql` for a `mysql://` URL | `postgres`, `mysql` (see [MySQL](#mysql)) or `sqlite` (see [SQLite](#sqlite)). |
 | `db.migrate` | none | `{ atlas: { dir } }`, `{ sql: "file-or-dir" }` or `{ command, inputs? }` (gets `DATABASE_URL`). |
 | `db.seed` | none | SQL file re-run after every reset. |
 | `db.schemas` | `["public"]` | Schemas whose tables are reset. |
 | `db.keep` | `[]` | Extra tables (`name` or `schema.name`) never truncated. |
+| `db.neon` | `false` | For Neon's serverless driver over HTTP: `{{db.url}}` is a Neon-style URL and slicetest answers the driver's queries from the test database ([Neon](#neon-and-vercel-postgres)). |
 | `db.queries` | `false` | Point the app at a proxy that records its SQL (Postgres, MySQL), for [`db.queries()`](#dbqueries--the-sql-the-app-ran-from-any-language). |
 | `db.url` | `$SLICETEST_DATABASE_URL`, else a container | Use an existing Postgres server (e.g. a CI service container) instead of Testcontainers. |
 | `db.image` | `postgres:17-alpine` / `mysql:8.4` | |
@@ -553,8 +646,10 @@ Scenarios in one file share an app and a database, so they always run one at a t
 | `mail` | `false` | Start an SMTP server at `{{mail.host}}` / `{{mail.port}}` and collect the app's mail. See [Mail](#mail-catch-what-the-app-sends). |
 | `auth` | `false` | `true` or `{ audience, claims }`: an OpenID Connect issuer at `{{auth.issuer}}` (JWKS at `{{auth.jwks}}`) whose tokens scenarios mint with `auth.token()`. See [Auth](#auth-a-real-openid-issuer-tokens-with-any-claims). |
 | `services` | `{}` | Other processes: `{ name: { command, env?, cwd?, ready?, readyTimeout? } }`. Without `ready` a service is not waited for. |
-| `stubs` | `[]` | Names of stubbed services, or `{ name, openapi?, autoReply?, upstream?, recordings? }`: check calls against the provider's spec, answer from it, or [replay recordings](#recording-a-real-service) of the real service. |
-| `openapi` | none | The app's OpenAPI 3 spec, or `{ spec, minCoverage }`. Every response must match it; the run ends with a coverage report. |
+| `offline` | `false` | Refuse the app's HTTP(S) calls to hosts no stub intercepts, and fail the scenario naming them. |
+| `workers` | Vitest's default | Most Vitest workers (`maxWorkers`). Each has its own app and database. |
+| `stubs` | `[]` | Names of stubbed services, or `{ name, openapi?, autoReply?, upstream?, recordings?, hosts? }`: check calls against the provider's spec, answer from it, [replay recordings](#recording-a-real-service) of the real service, or answer for [hard-coded hosts](#hard-coded-hosts-hosts). |
+| `openapi` | none | The app's OpenAPI 3 spec, or `{ spec, minCoverage }`, or `{ fromApp: "/v3/api-docs" }` for a spec the running app serves (springdoc, FastAPI's `/openapi.json`, NestJS). Every response must match it; the run ends with a coverage report. |
 | `http` | `{}` | Default `headers` / `query` for every request. |
 
 The config is validated up front: a missing `app.command`, an ambiguous `db.migrate` or a duplicate stub name fails with a clear message instead of a timeout.
@@ -606,6 +701,7 @@ scenarios:
 | Step | Keys |
 |---|---|
 | `stub: <name>` | `on: METHOD /path` (`:params` allowed), `when: { query, headers, json, body }`, one of `reply: { status, headers, body }` / `sequence: [...]` / `networkError: true`, plus `times`, `delay`. Replies may echo the call: `{{call.params.id}}`, `{{call.json.name}}`. |
+| `submit: <button>` | `form`, `fields`, `headers`, `follow`, `expect: { status, headers, json, text }`, `capture`. Submits a form of the page the last request returned, like `http.submit()`; `submit: true` presses the form's only button. |
 | `request: METHOD /path` | `headers`, `query`, one of `json` / `form` / `body`, `follow`, `expect: { status, headers, json, text }`, `capture`. `concurrency: n` sends it `n` times at once; `expect` then applies to each response, and `expect.statuses: { 201: 1, 409: 9 }` counts them. |
 | `insert: <table>` | `rows`, `capture` (from `row` / `rows`) |
 | `request` with `auth` | `auth: true` or the claims: sends a bearer token from the `auth` issuer |
@@ -631,7 +727,7 @@ scenarios:
 
 ### Without any JavaScript: `npx slicetest`
 
-Put the plugin options in `slicetest.config.yaml` and run the CLI. It needs Node, but no `package.json` scripts, TypeScript or Vitest config:
+Put the plugin options in `slicetest.config.yaml` and run the CLI. It needs Node, but no `package.json` scripts, TypeScript or Vitest config. To add TypeScript scenarios later, `slicetest()` without options in a `vitest.config.ts` reads the same file (`slicetest("path/to/config.yaml")` for another one):
 
 ```yaml
 # slicetest.config.yaml
@@ -705,6 +801,88 @@ Checks what a run needs before it starts, instead of failing with a timeout half
 1 problem(s) to fix before running.
 ```
 
+## Neon and Vercel Postgres
+
+Apps on Neon's serverless driver (`neon()` from `@neondatabase/serverless`, `drizzle-orm/neon-http`, `@vercel/postgres`'s `sql`) send each query over HTTPS to Neon, not to a Postgres port. With `db: { neon: true }`, `{{db.url}}` is a Neon-style connection string, and slicetest answers the driver's HTTP queries (single statements and `transaction()` batches) from the test database. The driver, its type parsing and its errors (with Postgres's `code`) are the real ones; `db.queries` sees the SQL. `slicetest init` sets it when the driver is a dependency. The driver's WebSocket mode (`Pool`, `Client`) isn't covered.
+
+With Drizzle, `drizzle-kit push` and `migrate` connect through the project's driver; with only Neon's installed, they try a WebSocket a local server doesn't answer, and exit 0 anyway (slicetest stops when a migration leaves the database without tables). Generate the SQL instead (`npx drizzle-kit generate`) and let slicetest apply it: `migrate: { sql: db/migrations }`, which `slicetest init` reads from `drizzle.config`'s `out`.
+
+## Clerk
+
+Clerk verifies session tokens with keys from its Backend API, and reads users from it. Both are stubbed, and the tokens come from slicetest's issuer, so requests run as any user, with no Clerk account:
+
+```yaml
+# slicetest.config.yaml (what `slicetest init` writes when @clerk/nextjs is a dependency)
+app:
+  env:
+    NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY: pk_test_Y2xlcmsuc2xpY2V0ZXN0LnRlc3Qk   # base64 of "clerk.slicetest.test$"
+    CLERK_SECRET_KEY: sk_test_slicetest
+auth: true
+stubs:
+  - name: clerk
+    hosts: [api.clerk.com, clerk.slicetest.test]
+```
+
+```ts
+function signIn({ auth, stub }: ScenarioContext, id = "user_1", email = "ada@example.com") {
+  stub("clerk").on("GET", "/v1/jwks").reply(200, auth.jwks);   // the keys Clerk verifies tokens with
+  stub("clerk").on("GET", `/v1/users/${id}`).reply(200, {
+    object: "user", id, first_name: "Ada", last_name: null, primary_email_address_id: "e1",
+    email_addresses: [{ object: "email_address", id: "e1", email_address: email, verification: { status: "verified" }, linked_to: [] }],
+    phone_numbers: [], web3_wallets: [], external_accounts: [], created_at: 0, updated_at: 0,
+  });
+  return auth.header({ sub: id, sid: "sess_1" });
+}
+
+scenario("checkout as a signed-in user", async (ctx) => {
+  const res = await ctx.http.post("/api/checkout", undefined, { headers: signIn(ctx) });
+  expect(res).toHaveStatus(200);
+});
+```
+
+`CLERK_JWT_KEY` with `{{auth.publicKey}}` would avoid the JWKS stub, but Next.js inlines environment variables into middleware at build time, before the key exists. Without a token, `auth.protect()` in middleware answers API routes with 404.
+
+## Spring Boot and other JVM apps
+
+`slicetest init` recognizes Spring Boot (Gradle or Maven, also in a `backend/` folder) and writes:
+
+```yaml
+app:
+  cwd: backend
+  build: ./gradlew bootJar -q      # built once: workers don't compile at once, bootRun downloads nothing
+  command: ./gradlew bootRun -q
+  env:
+    SERVER_PORT: "{{app.port}}"
+    SPRING_DATASOURCE_URL: "{{db.jdbcUrl}}"       # overrides application.yml
+    SPRING_DATASOURCE_USERNAME: "{{db.user}}"
+    SPRING_DATASOURCE_PASSWORD: "{{db.password}}"
+  ready: { path: /actuator/health }
+  readyTimeout: 120000
+  scope: worker                    # one JVM per worker, kept for all of its test files
+workers: 2
+```
+
+Starting a JVM takes seconds, so by default (`scope: file`, one app per test file) the start-up dominates: 8 test files took 28 s on a Spring Boot 4 app. With `scope: worker` the app is started once per Vitest worker and kept for every file that worker runs, and `workers` caps how many there are: 18 s with the default worker count, 10 s with 2, 8 s with 1. Scenarios stay independent, since the database, stubs and cookies are reset between them either way; what's shared is the app process (and its in-memory state, such as caches).
+
+Springdoc serves the spec at `/v3/api-docs`: `openapi: { fromApp: /v3/api-docs }` checks every response against it and reports its coverage, without a spec file in the repository. Generated specs often miss error responses and nullable fields, which this makes visible.
+
+Apps that run Hibernate with `ddl-auto: validate` against schema-owning migrations (Atlas, or a migration command) work as they are: slicetest migrates, Hibernate validates the result at start-up. For a faster start, build a jar once with `build: ./gradlew bootJar -q` and run `command: java -jar build/libs/app.jar`. Calls to hard-coded hosts (GitHub, Google, …) are caught with [`hosts`](#hard-coded-hosts-hosts).
+
+## Next.js
+
+Nothing Next-specific is needed beyond two settings. `slicetest init` writes both when `package.json` has `build` and `start` scripts:
+
+```yaml
+app:
+  build: npm run build   # `next start` serves the last build; without this, scenarios can pass against old code
+  command: npm start     # next start reads $PORT
+  env: { PORT: "{{app.port}}", DATABASE_URL: "{{db.url}}" }
+```
+
+- **Server actions** are ordinary form posts when JavaScript is off. Request the page, then `submit:` the button; the action id and bound arguments travel in hidden inputs, so scenarios keep working when a build changes the ids.
+- **Route handlers and pages** are plain HTTP: `request: GET /api/...` and `text: { $contains: ... }`. React may put `<!-- -->` between adjacent text nodes, so match on a word rather than on `75%` built from two values.
+- **Rows inserted by migrations** (seed data in an Atlas or Prisma migration) are truncated before every scenario, like everything else. Create what a scenario needs with `make`, or move shared rows to `db.seed`.
+
 ## Examples
 
 `examples/` has a Node app (`node:http` + `pg`) and a Python app (`http.server` + `psycopg`) with the same API. **The same scenario files (`polls.test.ts` and `polls.scenario.yaml`) run against both**:

package/dist/app.js

@@ -66,14 +66,27 @@ export class App {
      * `service.<name>` for a service (its port is then `{{service.<name>.port}}`).
      */
     static async start(opts, root, vars, key = "app", fixedPort) {
+        // A free port can be taken by someone else before the app binds it (another worker, another
+        // program). When that is why the app died, try again on a fresh port.
+        for (let attempt = 1;; attempt++) {
+            try {
+                return await App.#start(opts, root, vars, key, fixedPort);
+            }
+            catch (e) {
+                if (fixedPort !== undefined || attempt >= PORT_ATTEMPTS || !(e instanceof PortInUse))
+                    throw e;
+            }
+        }
+    }
+    static async #start(opts, root, vars, key, fixedPort) {
         // A restarted service keeps its port, so URLs already handed to other processes stay valid.
         const port = fixedPort ?? (await freePort());
         vars = { ...vars, [`${key}.port`]: String(port) };
-        const env = opts.env ?? { PORT: `{{${key}.port}}`, DATABASE_URL: "{{db.url}}" };
+        const env = opts.env ?? { PORT: `{{${key}.port}}`, ...("db.url" in vars ? { DATABASE_URL: "{{db.url}}" } : {}) };
         const child = spawn(interpolate(opts.command, vars, `${key}.command`), {
             shell: true,
             cwd: path.resolve(root, opts.cwd ?? "."),
-            env: { ...process.env, ...mapValues(env, (v) => interpolate(v, vars, `${key}.env`)) },
+            env: { ...process.env, ...opts.baseEnv, ...mapValues(env, (v) => interpolate(v, vars, `${key}.env`)) },
             stdio: ["ignore", "pipe", "pipe"],
             // POSIX: own process group, so stop() also kills whatever the shell spawned.
             // Windows: detaching would open a console window; taskkill /T walks the tree instead.
@@ -87,6 +100,8 @@ export class App {
         }
         catch (e) {
             await app.stop();
+            if (app.#exit && ADDRESS_IN_USE.test(app.logs()))
+                throw new PortInUse(e.message);
             throw e;
         }
         return app;
@@ -266,14 +281,27 @@ function escapeRegExp(s) {
 function mapValues(obj, fn) {
     return Object.fromEntries(Object.entries(obj).map(([k, v]) => [k, fn(v)]));
 }
-function freePort() {
-    return new Promise((resolve, reject) => {
-        const srv = createServer();
-        srv.unref();
-        srv.on("error", reject);
-        srv.listen(0, "127.0.0.1", () => {
-            const { port } = srv.address();
-            srv.close(() => resolve(port));
+const PORT_ATTEMPTS = 3;
+/** How Node, Python, Go, Ruby, the JVM and others report a port that is taken. */
+const ADDRESS_IN_USE = /EADDRINUSE|address already in use|Address in use|BindException/i;
+class PortInUse extends Error {
+}
+/** Ports this process has handed out; the OS may offer one again before the app has bound it. */
+const handedOut = new Set();
+async function freePort() {
+    for (;;) {
+        const port = await new Promise((resolve, reject) => {
+            const srv = createServer();
+            srv.unref();
+            srv.on("error", reject);
+            srv.listen(0, "127.0.0.1", () => {
+                const { port } = srv.address();
+                srv.close(() => resolve(port));
+            });
         });
-    });
+        if (!handedOut.has(port)) {
+            handedOut.add(port);
+            return port;
+        }
+    }
 }

package/dist/auth.d.ts

@@ -31,6 +31,16 @@ export declare class Issuer {
     private constructor();
     static start(opts?: AuthOptions): Promise<Issuer>;
     get audience(): string;
+    /**
+     * The signing key as a PEM public key, for libraries that verify tokens with a key
+     * given in the environment instead of fetching a JWKS (Clerk's CLERK_JWT_KEY, …).
+     * The app gets it at start as {{auth.publicKey}}; tokens minted after `rotate()` use a key it doesn't know.
+     */
+    get publicKeyPem(): string;
+    /** The JWKS document itself (`{ keys: [...] }`), for a stub that serves keys at another provider's URL (Clerk's /v1/jwks, …). */
+    get jwks(): {
+        keys: Record<string, unknown>[];
+    };
     get jwksUrl(): string;
     /** A signed RS256 JWT. `claims` override the defaults (`sub: "user-1"`, `iss`, `aud`, `iat`, `exp`). */
     token(claims?: Record<string, unknown>, opts?: TokenOptions): string;

package/dist/auth.js

@@ -4,7 +4,7 @@ const b64url = (data) => Buffer.from(data).toString("base64url");
 function newKey() {
     const { privateKey, publicKey } = generateKeyPairSync("rsa", { modulusLength: 2048 });
     const kid = randomUUID();
-    return { kid, privateKey, jwk: { ...publicKey.export({ format: "jwk" }), kid, alg: "RS256", use: "sig" } };
+    return { kid, privateKey, jwk: { ...publicKey.export({ format: "jwk" }), kid, alg: "RS256", use: "sig" }, pem: publicKey.export({ type: "spki", format: "pem" }) };
 }
 /**
  * An OpenID Connect issuer for the app under test: it publishes a discovery
@@ -45,6 +45,18 @@ export class Issuer {
     get audience() {
         return this.opts.audience;
     }
+    /**
+     * The signing key as a PEM public key, for libraries that verify tokens with a key
+     * given in the environment instead of fetching a JWKS (Clerk's CLERK_JWT_KEY, …).
+     * The app gets it at start as {{auth.publicKey}}; tokens minted after `rotate()` use a key it doesn't know.
+     */
+    get publicKeyPem() {
+        return this.#key.pem;
+    }
+    /** The JWKS document itself (`{ keys: [...] }`), for a stub that serves keys at another provider's URL (Clerk's /v1/jwks, …). */
+    get jwks() {
+        return { keys: [this.#key.jwk] };
+    }
     get jwksUrl() {
         return `${this.url}/.well-known/jwks.json`;
     }

package/dist/cli.d.ts

@@ -1,5 +1,5 @@
 #!/usr/bin/env node
-import type { SlicetestOptions } from "./config.js";
+import { type SlicetestOptions } from "./config.js";
 export interface CliConfig extends SlicetestOptions {
     include?: string[];
 }

package/dist/cli.js

@@ -9,7 +9,7 @@ import { readFile } from "node:fs/promises";
 import path from "node:path";
 import { parseArgs } from "node:util";
 import { parse } from "yaml";
-const CONFIG_NAMES = ["slicetest.config.yaml", "slicetest.config.yml", "slicetest.config.json"];
+import { CONFIG_NAMES } from "./config.js";
 const HELP = `Usage: slicetest [filters...] [options]
        slicetest init [--force]
        slicetest gen [--spec <file>] [--out <dir>] [--uncovered] [--force]
@@ -117,9 +117,16 @@ export async function main(argv = process.argv.slice(2)) {
         await record(configPath, options, { out: values.out, port: values.port ? Number(values.port) : 0 });
         return;
     }
-    const { startVitest } = await import("vitest/node");
+    const { startVitest, version } = await import("vitest/node");
+    if (Number.parseInt(version, 10) < 4) {
+        process.stderr.write(`slicetest: needs Vitest 4 or later, and this project has Vitest ${version}. Upgrade it (npm i -D vitest@latest), or run slicetest from a folder with its own package.json.\n`);
+        process.exitCode = 1;
+        return;
+    }
     const { slicetest, YAML_SCENARIOS } = await import("./vitest.js");
-    const vitest = await startVitest(positionals, {
+    // Vitest 4 takes the mode ("test") first; 5 dropped it.
+    const start = (Number.parseInt(version, 10) === 4 ? startVitest.bind(null, "test") : startVitest);
+    const vitest = await start(positionals, {
         config: false,
         root: path.dirname(configPath),
         include: include ?? [YAML_SCENARIOS],

package/dist/config.d.ts

@@ -2,7 +2,8 @@ import type { AuthOptions } from "./auth.js";
 import type { RequestOptions } from "./http.js";
 export interface SlicetestOptions {
     app: AppOptions;
-    db?: DbOptions;
+    /** The database, or `false` for an app without one: no container, no resets, no `{{db.*}}`. */
+    db?: DbOptions | false;
     /**
      * Outbound HTTP services to stub. Each gets its own server, referenced as `{{stub.<name>}}` in `app.env`.
      * With `{ name, openapi }`, the app's calls to it and the stub's replies are checked against that service's spec.
@@ -18,7 +19,8 @@ export interface SlicetestOptions {
      * a lower coverage fails the run.
      */
     openapi?: string | {
-        spec: string;
+        spec?: string;
+        fromApp?: string;
         minCoverage?: number;
     };
     /** Defaults for every request made with `http`, e.g. `{ headers: { accept: "application/json" } }`. */
@@ -44,6 +46,18 @@ export interface SlicetestOptions {
      * scenarios read what arrived with `mail.messages()` / `mail.waitFor()`.
      */
     mail?: boolean;
+    /**
+     * Keep the app off the network: its HTTP(S) calls may only reach localhost and the
+     * hosts of stubs with `hosts`. A call anywhere else is refused and fails the
+     * scenario, naming the host, so a forgotten stub can't reach a real service.
+     */
+    offline?: boolean;
+    /**
+     * Most Vitest workers to run test files in (Vitest's `maxWorkers`). Each worker has
+     * its own app and database; with `app.scope: "worker"`, fewer workers means fewer app
+     * starts, which is what makes a slow-starting app fast to test.
+     */
+    workers?: number;
     /**
      * An OpenID Connect issuer for apps that verify JWTs. The app gets
      * `{{auth.issuer}}`, `{{auth.jwks}}` and `{{auth.audience}}`; scenarios mint
@@ -79,6 +93,12 @@ export interface StubOptions {
     upstream?: string;
     /** Recordings file, relative to the root. Default `recordings/<name>.yaml`. */
     recordings?: string;
+    /**
+     * Hosts the app calls directly, e.g. `["api.github.com"]`: their HTTP and HTTPS
+     * traffic is answered by this stub, for apps whose URLs can't be set from the
+     * environment. The app is started with proxy variables and a test CA it trusts.
+     */
+    hosts?: string[];
 }
 export interface ServiceOptions extends Omit<AppOptions, "ready"> {
     /** Default: no wait (for workers that don't listen). `{ path }` polls the service's own port. */
@@ -91,12 +111,20 @@ type ResolvedReady = {
     flags: string;
 };
 /** A process to start, with `ready` made JSON-serializable. The app always has `ready`; services may not. */
-export type ResolvedProcess = Omit<AppOptions, "ready"> & {
+export type ResolvedProcess = Omit<AppOptions, "ready" | "scope"> & {
     ready?: ResolvedReady;
+    /** Set by slicetest (proxy variables for intercepted hosts); `env` overrides it. */
+    baseEnv?: Record<string, string>;
 };
 export interface AppOptions {
     /** Command that starts the app, run through the shell. */
     command: string;
+    /**
+     * Command run once per run, before any worker starts the process, e.g.
+     * `npm run build` for an app started with `next start`. Runs in `cwd` while
+     * the database starts. Not repeated on re-runs in watch mode.
+     */
+    build?: string;
     cwd?: string;
     /**
      * Environment passed to the app. Values may reference
@@ -112,6 +140,13 @@ export interface AppOptions {
     };
     /** Milliseconds to wait for readiness. Default 30000. */
     readyTimeout?: number;
+    /**
+     * `"file"` (default) starts the app, stubs and services for each test file.
+     * `"worker"` starts them once per Vitest worker and keeps them for every file that
+     * worker runs, for apps that take seconds to start (a JVM, a large framework). It
+     * turns off Vitest's per-file module isolation (`isolate: false`).
+     */
+    scope?: "file" | "worker";
 }
 export interface DbOptions {
     /**
@@ -148,6 +183,12 @@ export interface DbOptions {
      * statement, whatever the app's language or driver. Off by default.
      */
     queries?: boolean;
+    /**
+     * For apps on Neon's serverless driver over HTTP (`neon()` from @neondatabase/serverless,
+     * drizzle-orm/neon-http): `{{db.url}}` becomes a Neon-style connection string, and
+     * slicetest answers the driver's HTTP queries from the test database. Postgres only.
+     */
+    neon?: boolean;
 }
 export type MigrateOptions = {
     atlas: {
@@ -167,18 +208,26 @@ export type MigrateOptions = {
 /** Normalized shape passed from the plugin to globalSetup and workers. Must stay JSON-serializable. */
 export interface ResolvedOptions {
     root: string;
-    app: Omit<AppOptions, "ready"> & {
+    app: ResolvedProcess & {
         ready: ResolvedReady;
+        scope?: "file" | "worker";
     };
     services: Record<string, ResolvedProcess>;
     containers: Record<string, ContainerOptions>;
     mail: boolean;
+    offline: boolean;
+    workers?: number;
     auth: AuthOptions | false;
-    db: Required<Pick<DbOptions, "engine" | "image" | "schemas" | "keep" | "reuse">> & Omit<DbOptions, "engine" | "image" | "schemas" | "keep" | "reuse">;
+    db: Required<Pick<DbOptions, "engine" | "image" | "schemas" | "keep" | "reuse">> & Omit<DbOptions, "engine" | "image" | "schemas" | "keep" | "reuse"> & {
+        /** `db: false`: the app has no database. */
+        none?: boolean;
+    };
     stubs: string[];
     /** Spec files, resolved against the root: the app's, and per stub name. */
+    /** `fromApp`: a path the running app serves its own spec at (springdoc, FastAPI, NestJS). */
     openapi: {
         app?: string;
+        fromApp?: string;
         minCoverage?: number;
         stubs: Record<string, string>;
         autoReply: string[];
@@ -189,7 +238,11 @@ export interface ResolvedOptions {
         upstream: string;
         record: boolean;
     }>;
+    /** Intercepted host (lower case) → the stub that answers it. */
+    intercept: Record<string, string>;
     http?: RequestOptions;
 }
+/** The config files `npx slicetest` (and `slicetest()` without options) look for, in order. */
+export declare const CONFIG_NAMES: string[];
 export declare function resolveOptions(opts: SlicetestOptions, root: string): ResolvedOptions;
 export {};