@classytic/arc-testkit 0.4.0 → 0.6.0

package/skills/arc-testkit/references/fixtures.md

@@ -0,0 +1,92 @@
+# Fixtures: boot, pool, workers, cost and plan asserts
+
+## bootModuleApp (`@classytic/arc-testkit`)
+
+`bootModuleApp(modules, options?)` boots arc (`preset: "testing"`, `auth: false`, `sensible: false`) around module(s) on
+in-memory MongoDB and returns `{ app, uri, connection, exports(name), close() }` (`close` is idempotent).
+
+`modules` is a module, an array, or a factory `({ connection, mongoUri }) => module | module[]`.
+`options`: `replset` (single-node replica set, needed for transactions), `database` (a `TestDatabaseFactory`, e.g.
+`pool.factory`), `app` (extra `createApp` options merged over the defaults), `actor` (default actor for requests without
+an `x-test-actor` header), `ports` (host ports for the modules).
+
+```ts
+const t = await bootModuleApp(({ connection }) => [createThingModule({ connection, permissions })], { replset: true });
+const engine = t.exports<ThingEngine>("thing"); // typed module export
+await t.close();
+```
+
+Types: `TestkitContext`, `ModulesInput`, `BootModuleAppOptions`, `ModuleTestApp`.
+
+## Pool (`/pool`, `/mongo`)
+
+One server per run, one isolated database per suite. `createTestDatabasePool(provider, options?)` returns
+`{ factory, withPool, leaseNamespace, globalSetup, stats }`.
+
+```ts
+// tests/support/db-pool.ts
+import { mongoServerProvider } from "@classytic/arc-testkit/mongo";
+import { createTestDatabasePool } from "@classytic/arc-testkit/pool";
+import mongoose from "mongoose";
+
+export const dbPool = createTestDatabasePool(mongoServerProvider({ mongoose, awaitIndexes: false }), {
+  envPrefix: "APP_TEST",
+  preboot: ["transactional"],
+});
+// tests/support/global-setup.ts
+export const setup = () => dbPool.globalSetup.setup();
+export const teardown = () => dbPool.globalSetup.teardown();
+// a suite
+const boot = dbPool.withPool(bootModuleApp);
+const lease = await dbPool.leaseNamespace("transactional"); // { uri, namespace, release() }, for a host that connects itself
+```
+
+| Option | Default | Meaning |
+|---|---|---|
+| `envPrefix` | `ARC_TESTKIT` | env prefix of the cross-worker handshake; use one per run kind |
+| `preboot` | `["transactional"]` | kinds `globalSetup.setup()` starts and publishes; `[]` only for a single-process run |
+| `shareTransactionalServer` | `true` | one replica set answers `standalone` and `transactional` |
+| `poolSize` | `1` | servers per kind; size with `safePoolSize(n)` (bounded by free RAM) |
+| `slotOf` | vitest worker id | which server a worker uses |
+| `stopWhenIdle` | `false` | stop servers after the last release (watch mode) |
+| `disabled` | `false` | per-suite private servers |
+
+`leaseNamespace(kind, { exact, forcePrivate, createOptions })`: `forcePrivate` gives a suite its own server (use it for
+fail points and `setParameter`); `exact` stops a `standalone` request resolving to the shared replica set.
+`mongoServerProvider({ args, awaitIndexes = true, mongoose })` builds the Mongo provider; it waits for index builds after
+the app boots (`awaitIndexBuilds(connection?)`), which is what makes unique-index assertions deterministic. Inject the
+`mongoose` instance your models register on. `publishPoolServers(servers, envPrefix)` publishes servers a custom
+`globalSetup` started.
+
+## Workers and memory (`/mongo`)
+
+- `safeWorkerCount(envVar = "ARC_TEST_WORKERS")`: explicit env value, else min(cores - 2, RAM / 6 GB), at least 2. Use in
+  `defineConfig({ test: { maxWorkers: safeWorkerCount("TEST_MAX_WORKERS") } })`.
+- `safePoolSize(requested)`: servers affordable by free RAM, at least 1.
+- `cacheSizeGB()` (env `TEST_MONGO_CACHE_GB`, floor 0.25), `mongodMemoryArgs()`, `withDbName(uri, dbName)`.
+- One heavy command at a time on a shared machine; cap workers before running a whole suite.
+
+## Wire-command counts (`/mongo-commands`)
+
+The mongoose connection must be created with `monitorCommands: true`, else `countCommands` throws.
+
+```ts
+import { assertConstantCommands, countCommands } from "@classytic/arc-testkit/mongo-commands";
+
+const { total, byCollection, byCommand, commands } = await countCommands(conn, () => placeOrder(), { collections: ["orders"], commands: ["find"], ignore: ["createIndexes"] });
+await assertConstantCommands(conn, (n) => reader.page(n), [5, 50]); // same commands at every size, at least one: a per-item read fails
+```
+
+## Plan asserts (`/mongo-explain`)
+
+```ts
+import { assertIndexServed, summarizeExplain } from "@classytic/arc-testkit/mongo-explain";
+
+const explain = await Model.find(filter).explain("executionStats");
+assertIndexServed(explain, { index: "org_state", stages: ["OR"], maxDocsExamined: 40 });
+const { stages, indexes, docsExamined } = summarizeExplain(explain);
+```
+
+`assertIndexServed` fails on COLLSCAN, on a plan scanning no index (empty or missing collection), on an index in `index`
+that is not scanned, a stage in `stages` that is absent, and on `maxDocsExamined` exceeded (needs `executionStats`).
+Rejected plans and query predicates are never read.

package/skills/arc-testkit/references/list-filters.md

@@ -0,0 +1,3 @@
+# List filters against the route
+
+`checkFilterDeclarations({ spec, declarations })`; `declarations`: `[{ route, filters: [{ queryField, op? }], search?, sorts? }]` (op default `eq`, `between` = `gte` + `lte`). A filter must be a `field[op]` query parameter of the route's GET, or in arc's `x-arc-query` (`filterFields`, `operators`, `sortFields`); `search` needs the `search` parameter. Failure: `/api/x: filter status[contains] is not declared by the route`.

package/skills/arc-testkit/references/story.md

@@ -0,0 +1,80 @@
+# Story kit (`@classytic/arc-testkit/story`)
+
+One continuous scenario through a booted app. Steps are numbered vitest tests in one file (vitest runs them in order);
+state passes through the host's own object. The kit knows no domain: roles, routes and clause ids are yours.
+
+## Steps
+
+`describeStory(title, define, options?)`; `define` receives `s` with `s.step(title, [{ clauses, timeout }], body)`.
+`body(t, ctx)`. Test names are `N. title`.
+
+| Option | Default | Meaning |
+|---|---|---|
+| `firstStep` | `1` | number of the first step |
+| `bail` | `true` | after a hard failure the remaining steps are skipped with "step N failed" |
+| `clauses` | none | clause tags added to every step |
+| `test`, `fixtures` | vitest `it` | an extended `test` and the fixture names steps use; they arrive as `ctx` |
+
+`t.expect(actual, what)` is `expect` with the message `[CLAUSE-1 | CLAUSE-2] what`. `t.soft(...)` is `expect.soft`: the
+failure is reported, the step continues. `t.known("DEFECT-7", actual, what)` is a soft assertion tagged
+`KNOWN DEFECT DEFECT-7`, for an open finding that must stay visible without hiding later asserts. `t.msg(what)` builds the
+tagged text for other matchers. `clauseMessage(clauses, what)` is the pure form. `need(bag, key)` reads a value an
+earlier step produced and throws "the story needs KEY from an earlier step" instead of a bare `undefined`.
+
+Reading a failure: the first line names the clauses and what was expected; the step number is in the test name; steps
+after the first hard failure show as skipped (look at the first red step, not the skipped ones).
+
+```ts
+describeStory("onboarding", (s) => {
+  s.step("a hire is captured", { clauses: ["HR-1"] }, async (t) => {
+    const res = await cast.get("hr").post("/api/people", payload);
+    t.expect(res.status, "the person is created").toBe(201);
+    t.known("D-3", res.body.tasks.length, "four onboarding tasks").toBe(4);
+  });
+}, { firstStep: 0 });
+```
+
+## Cast
+
+`createCast((roles, opts, name) => provisionedActor)`; the provisioner signs the user up on the app and returns what the
+story talks through (a client, a token). `await cast.build({ ann: ["admin"], bob: { roles: ["viewer"], opts: { mfa: false } } })`
+returns the actors by name; `cast.get(name)`, `cast.add(name, roles, opts?)`, `cast.names()`. `bearer(token)` returns
+`{ authorization: "Bearer ..." }`. Pair with `injectCaller` from `/conformance` for `get`/`post`.
+
+## Clock and civil dates
+
+`createStoryClock({ start, zone })` returns `{ now(), zone, set, advanceMs, advanceDays, travelTo(day, hour = 12), today(),
+onChange(listener) }`; `now()` returns a fresh `Date`, so it fits any `{ now(): Date }` clock port. `travelTo` jumps to
+`hour:00` of a civil date in the zone (DST-safe). Pure helpers: `civilDateOf(at, zone)`, `addCivilDays(day, n)`,
+`addCivilMonths(day, n)`, `daysBetween(a, b)`, `instantOn(day, zone, hour)`. Dates are `YYYY-MM-DD` in the operator zone.
+
+## Evidence ports
+
+`createStoryAssertions(ports)` where every port is optional and a missing one throws "no `mail` port was supplied" at the
+call: `drain` (outbox pass), `events` (rows), `feed(actor)` (items with `title`, `message`), `mail` (rows), `audit(filter)`
+(rows), `eventTypeKey` (default `type`).
+
+| Method | Behaviour |
+|---|---|
+| `drain(rounds = 3)` | drain the outbox repeatedly (a handler may emit more) |
+| `relayed(assertion, { timeout = 30000, interval = 50 })` | drain, then run the assertion, retrying until it holds |
+| `expectEvent(match, { count, min, since, clauses })` | events matching; default at least one |
+| `expectNoEvent(match, { since, clauses })` | none match |
+| `expectFeed(actor, match, spec)`, `expectNoFeed(actor, match, spec)` | feed items whose title or message match |
+| `expectMail(match, spec)` | mail rows; string matches `subject` |
+| `expectAudit(filter, match, spec)` | audit rows; string matches `action` |
+| `eventsOf(match, since)`, `feedOf(actor)` | the matching rows, for custom checks |
+
+`match` is a string or RegExp (tested on the type, title, subject or action field), a predicate, or an object matched as a
+deep subset (RegExp and function leaves are tested, arrays match when each expected element is found).
+`createEventLog()` returns `{ record, all, mark, clear, tap(subscribe, patterns) }`; `tap` subscribes your bus
+(`(pattern, handler) => ...`) and records every event; `mark()` returns a count to pass as `since` for "what happened
+next". Feed `events: () => log.all()` into the ports.
+
+## Idempotency
+
+`expectIdempotent({ run, snapshot, settle?, clauses?, label? })` runs the sweep, settles, snapshots, runs again, settles,
+snapshots, and fails unless the two snapshots are equal (`LABEL run twice on one day changes nothing the second time`).
+Returns `{ first, second, afterFirst, afterSecond }`. `expectSweepsIdempotent({ name: run, ... }, { snapshot, settle, clauses })`
+does it for several sweeps. Snapshot what must not grow: row counts, ids, mail rows, feed items. Do not advance the
+clock between the runs.