slicetest 0.1.0 → 0.2.0

package/README.md

@@ -27,10 +27,27 @@ No browser, no mocked database, no hooks inside your app. The app only has to re
 
 - **Unit tests** mock the database and the network, so broken SQL, migrations and request payloads slip through.
 - **End-to-end tests** drive a browser against a deployed stack. They are slow and hard to make deterministic.
-- **slicetest** keeps the real HTTP server, the real SQL and the real migrations, and replaces only the things you don't own: third-party APIs.
+- **slicetest** keeps the real HTTP server, the real SQL and the real migrations, and replaces only the things you don't own: third-party APIs. With OpenAPI specs, it also checks that those replacements behave like the real thing.
 
 The database is reset between scenarios with a single `TRUNCATE ... RESTART IDENTITY CASCADE` (about 1.5 ms). The app keeps its connections, so this works with any driver or ORM. Resetting by dropping and re-creating the database takes about 130 ms, and it crashed some apps when their pooled connections were cut.
 
+## Quick start
+
+```sh
+npx slicetest init   # detects your stack, writes slicetest.config.yaml and a first scenario
+npx slicetest        # starts Postgres, migrates, starts your app, runs scenarios/*.scenario.yaml
+```
+
+`init` recognises Node (`npm start`), Django, FastAPI, Flask, Rails, Go and Rust apps; Atlas, Prisma, Alembic, Django, Rails, Drizzle, Knex and plain SQL migrations; and an `openapi.yaml`. It lists every guess as a comment in the config so you know what to check.
+
+## What you get that's hard to find elsewhere
+
+- **One scenario, three boundaries.** Assert on the HTTP response, the rows in the real database and the calls to third-party APIs in the same test, in any language the app is written in.
+- **`db.changes()`**: a diff of every row the scenario inserted, updated or deleted. `toEqual` on it catches writes you didn't expect.
+- **Stubs that can't lie.** Give a stub the provider's OpenAPI spec, and a canned reply the real service would never send fails the test.
+- **OpenAPI coverage** of your own API, per operation and status, across all scenarios.
+- **Fast resets.** `TRUNCATE` between scenarios (about 1.5 ms) with the app still running, and a cached migrated template, so the second run skips container start-up and migrations.
+
 ## Install
 
 ```sh
@@ -73,11 +90,11 @@ export default defineConfig({
 
 ### How a run works
 
-1. **Once per run.** slicetest starts `postgres:17-alpine` and migrates a template database.
+1. **Once per run.** slicetest starts `postgres:17-alpine` and migrates a template database. Locally, the container is kept running and the migrated template is cached by the contents of your migrations, so the next run with unchanged migrations skips both steps (see `db.reuse`).
 2. **Once per worker.** It clones the template into the worker's own database.
-3. **Once per test file.** It starts the stub servers and your app.
-4. **Before each scenario.** It truncates every table except migration bookkeeping tables (`atlas_schema_revisions`, `_prisma_migrations`, `alembic_version`, `django_migrations`, …) and extension-owned tables such as PostGIS's `spatial_ref_sys`, re-runs the seed, and clears the stubs, cookies and request history. If the app crashed in the previous scenario, it is restarted.
-5. **After each scenario.** The scenario fails if the app crashed or called a stub route you didn't register.
+3. **Once per test file.** It starts the stub servers, your `services` and your app.
+4. **Before each scenario.** It truncates every table except migration bookkeeping tables (`atlas_schema_revisions`, `_prisma_migrations`, `alembic_version`, `django_migrations`, …) and extension-owned tables such as PostGIS's `spatial_ref_sys`, re-runs the seed, and clears the stubs, cookies and request history. If the app or a service crashed in the previous scenario, it is restarted.
+5. **After each scenario.** The scenario fails if the app or a service crashed, the app called a stub route you didn't register, or (with `openapi`) any traffic didn't match the spec.
 
 Database names are unique per run, so several projects or CI jobs can share one Postgres server via `db.url`.
 
@@ -94,16 +111,22 @@ stub calls with no matching route:
 requests to the app:
   POST /signup → 500 (14ms)  {"error":"internal"}
 
+database changes during this scenario:
+  users: 1 inserted
+    + {"id":1,"email":"a@example.com","verified":false}
+  audit_log: 1 updated
+    ~ id=7  status: "pending" → "failed"
+
 app output during this scenario:
 TypeError: Cannot read properties of undefined (reading 'email')
 -----------------
 ```
 
-Only this scenario's app output is shown, not the whole log. Requests that never got a response (for example because the app crashed) appear as `failed`.
+Only this scenario's app output is shown, not the whole log. The database section is a diff against the state right after the reset and seed, so you see what the app actually wrote. Requests that never got a response (for example because the app crashed) appear as `failed`.
 
 ## API
 
-Every scenario receives `{ http, db, stub, app }`.
+Every scenario receives `{ http, db, stub, app, service }`.
 
 ### `http` — talk to the app
 
@@ -132,7 +155,25 @@ await db.sql`SELECT * FROM users WHERE id = ${user.id}`;          // values beco
 await db.query("UPDATE users SET name = $1", ["b"]);
 ```
 
-In `where`, `null` means `IS NULL` and an array means `IN (...)`. `bigint` columns (bigserial ids, `count(*)`) come back as numbers when they fit safely.
+In `where`, `null` means `IS NULL` and an array means `IN (...)`.
+
+#### `db.changes()` — assert on everything the app wrote
+
+Instead of guessing which tables to query, ask for the diff. Rows are matched by primary key, so updates show which columns changed:
+
+```ts
+await db.insert("users", { email: "a@example.com" });
+await db.checkpoint();                                   // ignore what the test itself arranged
+
+await http.post("/users/1/verify");
+
+expect(await db.changes()).toEqual({
+  users: { inserted: [], deleted: [], updated: [expect.objectContaining({ changed: ["verified"] })] },
+  audit_log: { inserted: [expect.objectContaining({ action: "verify" })], updated: [], deleted: [] },
+});
+```
+
+`toEqual` fails if the app wrote to a table you didn't list, which catches unexpected side effects. Each entry in `updated` has `key`, `before`, `after` and `changed`. Tables without a primary key report an update as one deleted row plus one inserted row. `bigint` columns (bigserial ids, `count(*)`) come back as numbers when they fit safely.
 
 ### `stub(name)` — fake the services the app calls
 
@@ -158,6 +199,53 @@ stub("slack").calls("POST", "/hook");                         // recorded calls:
 
 Later routes win. `path` may also be a RegExp, and `method` may be `*`. Unanswered calls get a `501` and fail the scenario.
 
+### OpenAPI contracts — for your app and for the services you stub
+
+Point slicetest at OpenAPI 3.0 / 3.1 files and every scenario doubles as a contract test, with no extra assertions:
+
+```ts
+slicetest({
+  openapi: "openapi.yaml",                                       // your app's spec
+  stubs: ["slack", { name: "stripe", openapi: "specs/stripe.yaml" }], // a provider's spec
+  // ...
+});
+```
+
+- **Your app's responses** must be documented (path, method, status) and match the schema.
+- **The app's requests to a stub** must match the provider's spec: required query parameters, content type and request body. Spec paths are matched with or without the server's base path (`/v1`).
+- **Your stubs' replies** must be something the real service could send. A stub that returns `200 { ok: true }` where the provider documents `202 { messageId }` makes tests pass against an API that doesn't exist; slicetest fails the scenario instead.
+
+```
+slicetest: traffic doesn't match the OpenAPI spec:
+  app: GET /users/{id} → 200: /id must be integer
+  app → mail: POST /mail/send request: body must have required property 'subject'
+  stub mail reply (the real service wouldn't answer this way): POST /mail/send responded 200, which specs/mail.yaml doesn't document (documented: 202)
+```
+
+#### `autoReply`: stubs generated from the provider's spec
+
+Add `autoReply: true` to a stub with a spec, and calls that no route matches are answered with the provider's documented example, or with values built from the schema (formats such as `email` and `date-time`, enums, `minimum`, `allOf` are respected). Register routes only for what a scenario cares about; a registered route always wins.
+
+```ts
+stubs: [{ name: "stripe", openapi: "specs/stripe.yaml", autoReply: true }],
+```
+
+Calls answered this way have `call.fallback === true`. Paths that aren't in the spec still get a 501 and fail the scenario.
+
+At the end of the run, slicetest prints which documented responses your scenarios actually produced, merged across workers and across TypeScript and YAML scenarios:
+
+```
+slicetest: OpenAPI coverage (openapi.yaml): 8/9 documented responses (89%)
+  GET    /health            200 ✓
+  POST   /polls             201 ✓  400 ✓  502 ✓
+  GET    /polls/{id}        200 ✓  404 ✗
+  POST   /polls/{id}/votes  204 ✓  400 ✓  404 ✓
+```
+
+To fail the run below a threshold, use `openapi: { spec: "openapi.yaml", minCoverage: 100 }`. Filtered runs (`-t`, a single file) count too, so you may want `minCoverage: process.env.CI ? 100 : undefined`.
+
+The example apps in `examples/` run every scenario against `examples/openapi.yaml` with `minCoverage: 100`, and their Slack calls against `examples/slack.openapi.yaml`.
+
 ### Matchers
 
 Registered automatically:
@@ -173,6 +261,47 @@ await expect(db).toHaveRow("votes", { poll_id: 1 }, 3);          // exactly thre
 
 Failure messages list the calls the stub actually received, or the first rows of the table.
 
+### Services: workers and other processes
+
+Real apps are rarely one process. Declare the others under `services` and slicetest starts them before the app (in order), watches them like the app, and restarts one that crashed — on the same port, so URLs handed to other processes stay valid:
+
+```ts
+slicetest({
+  services: {
+    pricing: { command: "go run ./cmd/pricing", ready: { path: "/health" } }, // another HTTP service
+    worker: { command: "bundle exec sidekiq" },                                 // no port, no ready check
+  },
+  app: {
+    command: "node server.js",
+    env: { PORT: "{{app.port}}", DATABASE_URL: "{{db.url}}", PRICING_URL: "{{service.pricing}}" },
+  },
+});
+```
+
+Each service gets `PORT` = `{{service.<name>.port}}` and `DATABASE_URL` unless you pass `env`. A crash fails the scenario, and each service's output during the scenario is part of the failure output.
+
+```ts
+scenario("uploading an image queues a thumbnail job", async ({ http, db, service }) => {
+  await http.post("/images", { url: "https://example.com/cat.png" });
+
+  await service("worker").waitForLog(/thumbnail \d+ done/);   // this scenario's output only
+  await expect(db).toHaveRow("images", { thumbnail_ready: true });
+});
+```
+
+`waitForLog(pattern, timeout = 5000)` resolves with the matching line and also works on `app`. In YAML: `- log: thumbnail \d+ done` with `from: worker` and `within: <ms>`.
+
+### Asynchronous side effects
+
+If the app does work in the background (a job queue, a fire-and-forget webhook), wait for the effect with Vitest's own helpers. slicetest doesn't need its own:
+
+```ts
+await vi.waitFor(() => expect(stub("mail")).toHaveReceived("POST", "/send"));
+await expect.poll(() => db.count("jobs", { status: "done" })).toBe(1);
+```
+
+In YAML, add `within: <ms>` to a `db`, `sql`, `received` or `changes` step.
+
 ### Scenarios
 
 ```ts
@@ -195,13 +324,16 @@ Scenarios in one file share an app and a database, so they always run one at a t
 | `app.cwd` | vitest root | |
 | `app.ready` | `{ path: "/" }` | Poll a path until it answers below 500, or `{ log: "listening" \| /regex/ }`. |
 | `app.readyTimeout` | `30000` | |
-| `db.migrate` | none | `{ atlas: { dir } }`, `{ sql: "file-or-dir" }` or `{ command }` (gets `DATABASE_URL`). |
+| `db.migrate` | none | `{ atlas: { dir } }`, `{ sql: "file-or-dir" }` or `{ command, inputs? }` (gets `DATABASE_URL`). |
 | `db.seed` | none | SQL file re-run after every reset. |
 | `db.schemas` | `["public"]` | Schemas whose tables are reset. |
 | `db.keep` | `[]` | Extra tables (`name` or `schema.name`) never truncated. |
 | `db.url` | `$SLICETEST_DATABASE_URL`, else a container | Use an existing Postgres server (e.g. a CI service container) instead of Testcontainers. |
 | `db.image` | `postgres:17-alpine` | |
-| `stubs` | `[]` | Names of stubbed services. |
+| `db.reuse` | on, unless `CI` is set or `db.url` is given | Keep the container between runs and cache the migrated template. The cache key is the migration files' contents; for `{ command }`, list what it reads in `inputs: ["prisma/migrations"]`, or it migrates every run. Databases left by killed runs are dropped after a day. Remove the container (`docker rm -f` / `podman rm -f`) to start clean. |
+| `services` | `{}` | Other processes: `{ name: { command, env?, cwd?, ready?, readyTimeout? } }`. Without `ready` a service is not waited for. |
+| `stubs` | `[]` | Names of stubbed services, or `{ name, openapi }` to check calls and replies against the provider's spec. |
+| `openapi` | none | The app's OpenAPI 3 spec, or `{ spec, minCoverage }`. Every response must match it; the run ends with a coverage report. |
 | `http` | `{}` | Default `headers` / `query` for every request. |
 
 The config is validated up front: a missing `app.command`, an ambiguous `db.migrate` or a duplicate stub name fails with a clear message instead of a timeout.
@@ -237,6 +369,9 @@ scenarios:
         when:
           json: { text: "New poll: Dogs or cats?" }
 
+      - changes:                  # and nothing else was written
+          polls: { inserted: 1 }
+
   - name: voting {{choice}} returns {{status}}
     each:
       - { choice: a, status: 204 }
@@ -255,6 +390,11 @@ scenarios:
 | `db: <table>` | `where`, `orderBy`, `expect: { rows, count }`, `capture` |
 | `sql: <query>` | `params`, `expect: { rows, count }`, `capture` |
 | `received: <stub>` | `call: METHOD /path`, `when`, `times` (exact; default at least once) |
+| `log: <regex>` | `from` (a service; default the app), `within` (ms, default 5000). Waits for a matching line printed during the scenario. |
+| `changes: { <table>: { inserted, updated, deleted } }` | Each is a count or a list of subset rows (`updated` matches the row after the update). Tables that aren't listed must be unchanged. |
+| `checkpoint: true` | Later `changes` steps only see what happens after this step. |
+
+`db`, `sql`, `received` and `changes` steps take `within: <ms>` to retry until they pass, for effects the app applies asynchronously.
 
 - `{{name}}` inserts a captured value or an `each` field. A string that is only `{{name}}` keeps the value's type, so `id: "{{pollId}}"` compares as a number.
 - Expected `json`, `rows` and `headers` are subsets: extra keys are fine. `{ $type: number }`, `{ $regex: "^ch_" }`, `{ $contains: "..." }` and `{ $any: true }` match loosely.
@@ -293,4 +433,4 @@ npm run test:dist   # the built package, and the CLI with examples/slicetest.con
 
 ## Status
 
-Early prototype. Postgres only. CI runs on Linux and Windows. Planned: container reuse across runs, MySQL.
+Early. Postgres only. CI runs on Linux and Windows. Planned: MySQL.

package/dist/app.d.ts

@@ -1,24 +1,47 @@
-import type { ResolvedOptions } from "./config.js";
+import type { ResolvedProcess } from "./config.js";
+type ProcessOptions = ResolvedProcess;
 type Exit = {
     code: number | null;
     signal: NodeJS.Signals | null;
     error?: Error;
 };
-/** The app under test, running as a real child process. */
+/** The app under test (or one of its `services`), running as a real child process. */
 export declare class App {
     #private;
+    readonly port: number;
+    /** `app`, or `service <name>`, for messages. */
+    readonly label: string;
     readonly url: string;
     private constructor();
-    static start(opts: ResolvedOptions["app"], root: string, vars: Record<string, string>): Promise<App>;
+    /**
+     * `key` names the process in placeholders and messages: `app`, or
+     * `service.<name>` for a service (its port is then `{{service.<name>.port}}`).
+     */
+    static start(opts: ProcessOptions, root: string, vars: Record<string, string>, key?: string, fixedPort?: number): Promise<App>;
     get exited(): Exit | undefined;
     /**
      * The exit event is delivered asynchronously, so a crash caused by the last
      * request may not be visible yet. Give it a moment before trusting `exited`.
+     * Windows reports it later (the process runs under cmd.exe), so it waits longer.
      */
     settle(ms?: number): Promise<Exit | undefined>;
     /** The app's recent output (last 200 lines), or only what it printed after `since` (a value from `mark()`). */
     logs(since?: number): string;
     mark(): number;
+    /** What the process printed during the current scenario. */
+    scenarioLogs(): string;
+    /** Called before each scenario; `waitForLog()` only looks at output after this point. */
+    beginScenario(): number;
+    /**
+     * Wait until the process prints a line matching `pattern` during the current
+     * scenario (lines printed earlier in the scenario count). Resolves with the line.
+     *
+     * ```ts
+     * await http.post("/orders", { ... });
+     * await service("worker").waitForLog(/order \d+ shipped/);
+     * ```
+     */
+    waitForLog(pattern: string | RegExp, timeout?: number): Promise<string>;
     /**
      * POSIX: SIGTERM the whole process group, then SIGKILL whatever is left (including orphaned grandchildren).
      * Windows has no signals to ask politely with, so the tree is terminated at once.

package/dist/app.js

@@ -7,19 +7,27 @@ const LOG_LINES = 200;
 const DRAIN_MS = 100;
 const KILL_GRACE_MS = 3000;
 const WINDOWS = process.platform === "win32";
-/** The app under test, running as a real child process. */
+/** The app under test (or one of its `services`), running as a real child process. */
 export class App {
-    url;
+    port;
+    label;
     #child;
     #log = [];
     /** Lines ever received, so callers can ask for output since a point in time. */
     #lineCount = 0;
     #exit;
     #exited;
-    #onLine;
-    constructor(child, url) {
-        this.url = url;
+    #listeners = new Set();
+    /** `mark()` at the start of the current scenario. */
+    #scenarioMark = 0;
+    url;
+    constructor(child, port, 
+    /** `app`, or `service <name>`, for messages. */
+    label) {
+        this.port = port;
+        this.label = label;
         this.#child = child;
+        this.url = `http://127.0.0.1:${port}`;
         for (const stream of [child.stdout, child.stderr]) {
             // Keep partial lines (and split multi-byte characters) until the rest arrives.
             const decoder = new StringDecoder("utf8");
@@ -31,7 +39,8 @@ export class App {
                 this.#lineCount++;
                 if (this.#log.length > LOG_LINES)
                     this.#log.shift();
-                this.#onLine?.(line);
+                for (const listener of this.#listeners)
+                    listener(line);
             };
             stream.on("data", (chunk) => {
                 const lines = (partial + decoder.write(chunk)).split(/\r?\n/);
@@ -52,21 +61,26 @@ export class App {
             child.on("close", (code, signal) => done({ code, signal }));
         });
     }
-    static async start(opts, root, vars) {
-        const port = await freePort();
-        vars = { ...vars, "app.port": String(port) };
-        const env = opts.env ?? { PORT: "{{app.port}}", DATABASE_URL: "{{db.url}}" };
-        const child = spawn(interpolate(opts.command, vars, "app.command"), {
+    /**
+     * `key` names the process in placeholders and messages: `app`, or
+     * `service.<name>` for a service (its port is then `{{service.<name>.port}}`).
+     */
+    static async start(opts, root, vars, key = "app", fixedPort) {
+        // A restarted service keeps its port, so URLs already handed to other processes stay valid.
+        const port = fixedPort ?? (await freePort());
+        vars = { ...vars, [`${key}.port`]: String(port) };
+        const env = opts.env ?? { PORT: `{{${key}.port}}`, DATABASE_URL: "{{db.url}}" };
+        const child = spawn(interpolate(opts.command, vars, `${key}.command`), {
             shell: true,
             cwd: path.resolve(root, opts.cwd ?? "."),
-            env: { ...process.env, ...mapValues(env, (v) => interpolate(v, vars, "app.env")) },
+            env: { ...process.env, ...mapValues(env, (v) => interpolate(v, vars, `${key}.env`)) },
             stdio: ["ignore", "pipe", "pipe"],
             // POSIX: own process group, so stop() also kills whatever the shell spawned.
             // Windows: detaching would open a console window; taskkill /T walks the tree instead.
             detached: !WINDOWS,
             windowsHide: true,
         });
-        const app = new App(child, `http://127.0.0.1:${port}`);
+        const app = new App(child, port, key === "app" ? "app" : key.replace(/^service\./, "service "));
         running.add(app);
         try {
             await app.#waitReady(opts);
@@ -83,8 +97,9 @@ export class App {
     /**
      * The exit event is delivered asynchronously, so a crash caused by the last
      * request may not be visible yet. Give it a moment before trusting `exited`.
+     * Windows reports it later (the process runs under cmd.exe), so it waits longer.
      */
-    async settle(ms = 20) {
+    async settle(ms = WINDOWS ? 100 : 20) {
         if (this.#exit)
             return this.#exit;
         let timer;
@@ -102,6 +117,44 @@ export class App {
     mark() {
         return this.#lineCount;
     }
+    /** What the process printed during the current scenario. */
+    scenarioLogs() {
+        return this.logs(this.#scenarioMark);
+    }
+    /** Called before each scenario; `waitForLog()` only looks at output after this point. */
+    beginScenario() {
+        this.#scenarioMark = this.#lineCount;
+        return this.#scenarioMark;
+    }
+    /**
+     * Wait until the process prints a line matching `pattern` during the current
+     * scenario (lines printed earlier in the scenario count). Resolves with the line.
+     *
+     * ```ts
+     * await http.post("/orders", { ... });
+     * await service("worker").waitForLog(/order \d+ shipped/);
+     * ```
+     */
+    async waitForLog(pattern, timeout = 5000) {
+        const re = typeof pattern === "string" ? new RegExp(escapeRegExp(pattern)) : new RegExp(pattern.source, pattern.flags.replace(/[gy]/g, ""));
+        const n = Math.min(this.#lineCount - this.#scenarioMark, this.#log.length);
+        const seen = n > 0 ? this.#log.slice(-n).find((l) => re.test(l)) : undefined;
+        if (seen !== undefined)
+            return seen;
+        return new Promise((resolve, reject) => {
+            const done = (fn) => {
+                clearTimeout(timer);
+                this.#listeners.delete(listener);
+                fn();
+            };
+            const listener = (line) => {
+                if (re.test(line))
+                    done(() => resolve(line));
+            };
+            const timer = setTimeout(() => done(() => reject(new Error(`slicetest: ${this.label} printed no line matching ${re} within ${timeout}ms. Output during this scenario:\n${this.logs(this.#scenarioMark) || "(none)"}`))), timeout);
+            this.#listeners.add(listener);
+        });
+    }
     /**
      * POSIX: SIGTERM the whole process group, then SIGKILL whatever is left (including orphaned grandchildren).
      * Windows has no signals to ask politely with, so the tree is terminated at once.
@@ -139,18 +192,23 @@ export class App {
         const timeout = opts.readyTimeout ?? 30_000;
         const deadline = Date.now() + timeout;
         const ready = opts.ready;
+        // A service without `ready` (e.g. a queue worker) counts as ready once it's spawned.
+        if (!ready)
+            return;
         let logSeen = false;
+        let onLine;
         if ("log" in ready) {
             // Drop g/y so test() doesn't keep state between lines.
             const pattern = new RegExp(ready.log, ready.flags.replace(/[gy]/g, ""));
             logSeen = this.#log.some((l) => pattern.test(l));
-            this.#onLine = (line) => (logSeen ||= pattern.test(line));
+            onLine = (line) => (logSeen ||= pattern.test(line));
+            this.#listeners.add(onLine);
         }
         try {
             while (Date.now() < deadline) {
                 if (this.#exit) {
                     const why = this.#exit.error ? this.#exit.error.message : `code ${this.#exit.code}, signal ${this.#exit.signal}`;
-                    throw new Error(`slicetest: app exited before becoming ready (${why})\n${this.logs()}`);
+                    throw new Error(`slicetest: ${this.label} exited before becoming ready (${why})\n${this.logs()}`);
                 }
                 if ("log" in ready) {
                     if (logSeen)
@@ -168,10 +226,11 @@ export class App {
                 }
                 await new Promise((r) => setTimeout(r, 50));
             }
-            throw new Error(`slicetest: app did not become ready within ${timeout}ms\n${this.logs()}`);
+            throw new Error(`slicetest: ${this.label} did not become ready within ${timeout}ms\n${this.logs()}`);
         }
         finally {
-            this.#onLine = undefined;
+            if (onLine)
+                this.#listeners.delete(onLine);
         }
     }
 }
@@ -201,6 +260,9 @@ export function interpolate(template, vars, where = "app.env") {
         return vars[key];
     });
 }
+function escapeRegExp(s) {
+    return s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+}
 function mapValues(obj, fn) {
     return Object.fromEntries(Object.entries(obj).map(([k, v]) => [k, fn(v)]));
 }

package/dist/cli.js

@@ -11,21 +11,26 @@ import { parseArgs } from "node:util";
 import { parse } from "yaml";
 const CONFIG_NAMES = ["slicetest.config.yaml", "slicetest.config.yml", "slicetest.config.json"];
 const HELP = `Usage: slicetest [filters...] [options]
+       slicetest init [--force]
 
 Runs *.scenario.yaml files against your app, as configured in slicetest.config.yaml.
+\`slicetest init\` looks at the project and writes a starting config and scenario.
 
 Options:
   -c, --config <file>  Config file (default: ${CONFIG_NAMES.join(" / ")} in the current directory)
   -w, --watch          Re-run on changes
   -t, --name <pattern> Only run scenarios whose name matches
+      --force          init: overwrite existing files
   -h, --help           Show this help
 
 Config (paths are relative to the config file):
-  app:     { command, env, cwd, ready: { path } | { log }, readyTimeout }
-  db:      { migrate: { atlas: { dir } } | { sql } | { command }, seed, url, image, schemas, keep }
-  stubs:   [names]
-  http:    { headers, query }
-  include: [globs]   default ["**/*.scenario.{yaml,yml}"]
+  app:      { command, env, cwd, ready: { path } | { log }, readyTimeout }
+  db:       { migrate: { atlas: { dir } } | { sql } | { command, inputs }, seed, url, image, schemas, keep, reuse }
+  stubs:    [name | { name, openapi }]
+  services: { name: { command, env, cwd, ready } }
+  openapi:  file | { spec, minCoverage }
+  http:     { headers, query }
+  include:  [globs]  default ["**/*.scenario.{yaml,yml}"]
 `;
 export async function main(argv = process.argv.slice(2)) {
     const { values, positionals } = parseArgs({
@@ -36,17 +41,24 @@ export async function main(argv = process.argv.slice(2)) {
             watch: { type: "boolean", short: "w" },
             name: { type: "string", short: "t" },
             help: { type: "boolean", short: "h" },
+            force: { type: "boolean" },
         },
     });
     if (values.help) {
         process.stdout.write(HELP);
         return;
     }
+    if (positionals[0] === "init") {
+        const { init } = await import("./init.js");
+        const { files, notes } = await init(process.cwd(), { force: values.force });
+        process.stdout.write(`Created ${files.join(" and ")}.\n\nWhat was detected (check these in the config):\n${notes.map((n) => `  - ${n}`).join("\n")}\n\nNext: npx slicetest\n`);
+        return;
+    }
     const configPath = values.config
         ? path.resolve(values.config)
         : CONFIG_NAMES.map((n) => path.resolve(n)).find((p) => existsSync(p));
     if (!configPath || !existsSync(configPath)) {
-        process.stderr.write(`slicetest: no config found. Create ${CONFIG_NAMES[0]} (see --help).\n`);
+        process.stderr.write(`slicetest: no config found. Run "npx slicetest init" to create ${CONFIG_NAMES[0]} (see --help).\n`);
         process.exitCode = 1;
         return;
     }

package/dist/config.d.ts

@@ -2,11 +2,53 @@ import type { RequestOptions } from "./http.js";
 export interface SlicetestOptions {
     app: AppOptions;
     db?: DbOptions;
-    /** Names of outbound HTTP services to stub. Each gets its own server, referenced as `{{stub.<name>}}` in `app.env`. */
-    stubs?: string[];
+    /**
+     * Outbound HTTP services to stub. Each gets its own server, referenced as `{{stub.<name>}}` in `app.env`.
+     * With `{ name, openapi }`, the app's calls to it and the stub's replies are checked against that service's spec.
+     * With `autoReply: true` as well, calls no route matches are answered from the spec (its examples, or values
+     * built from its schemas) instead of failing, so you only register the routes a scenario cares about.
+     */
+    stubs?: (string | {
+        name: string;
+        openapi?: string;
+        autoReply?: boolean;
+    })[];
+    /**
+     * The app's own OpenAPI 3 spec (YAML or JSON, relative to the root). Every
+     * response the app gives during a scenario must be documented and match its
+     * schema, or the scenario fails. At the end of the run, slicetest prints which
+     * documented responses the scenarios produced; with `minCoverage` (percent),
+     * a lower coverage fails the run.
+     */
+    openapi?: string | {
+        spec: string;
+        minCoverage?: number;
+    };
     /** Defaults for every request made with `http`, e.g. `{ headers: { accept: "application/json" } }`. */
     http?: RequestOptions;
+    /**
+     * Other processes the app needs: a queue worker, another microservice, a
+     * mock written in another language. They start before the app, in order, and
+     * are watched like the app: a crash fails the scenario and they're restarted.
+     * Their URL is `{{service.<name>}}` and their port `{{service.<name>.port}}`,
+     * usable in `app.env` and in the env of services declared after them.
+     */
+    services?: Record<string, ServiceOptions>;
+}
+export interface ServiceOptions extends Omit<AppOptions, "ready"> {
+    /** Default: no wait (for workers that don't listen). `{ path }` polls the service's own port. */
+    ready?: AppOptions["ready"];
 }
+type ResolvedReady = {
+    path: string;
+} | {
+    log: string;
+    flags: string;
+};
+/** A process to start, with `ready` made JSON-serializable. The app always has `ready`; services may not. */
+export type ResolvedProcess = Omit<AppOptions, "ready"> & {
+    ready?: ResolvedReady;
+};
 export interface AppOptions {
     /** Command that starts the app, run through the shell. */
     command: string;
@@ -41,6 +83,13 @@ export interface DbOptions {
     schemas?: string[];
     /** Extra tables kept across resets, in addition to known migration bookkeeping tables. */
     keep?: string[];
+    /**
+     * Keep the Postgres container running between runs and cache the migrated
+     * template by the contents of the migrations, so a run with unchanged
+     * migrations skips both container start-up and migrating.
+     * Default: on, except when `CI` is set or `url` is given.
+     */
+    reuse?: boolean;
 }
 export type MigrateOptions = {
     atlas: {
@@ -50,20 +99,30 @@ export type MigrateOptions = {
     sql: string;
 } | {
     command: string;
+    /**
+     * Files or directories the command reads (e.g. `["prisma/migrations"]`).
+     * With `reuse`, the migrated template is cached until one of them changes;
+     * without `inputs`, the command runs on every run.
+     */
+    inputs?: string[];
 };
 /** Normalized shape passed from the plugin to globalSetup and workers. Must stay JSON-serializable. */
 export interface ResolvedOptions {
     root: string;
     app: Omit<AppOptions, "ready"> & {
-        ready: {
-            path: string;
-        } | {
-            log: string;
-            flags: string;
-        };
+        ready: ResolvedReady;
     };
-    db: Required<Pick<DbOptions, "image" | "schemas" | "keep">> & Omit<DbOptions, "image" | "schemas" | "keep">;
+    services: Record<string, ResolvedProcess>;
+    db: Required<Pick<DbOptions, "image" | "schemas" | "keep" | "reuse">> & Omit<DbOptions, "image" | "schemas" | "keep" | "reuse">;
     stubs: string[];
+    /** Spec files, resolved against the root: the app's, and per stub name. */
+    openapi: {
+        app?: string;
+        minCoverage?: number;
+        stubs: Record<string, string>;
+        autoReply: string[];
+    };
     http?: RequestOptions;
 }
 export declare function resolveOptions(opts: SlicetestOptions, root: string): ResolvedOptions;
+export {};