@jterrazz/test 16.1.1 → 17.0.0

package/docs/guides/job-specs.md

@@ -0,0 +1,180 @@
+# Job specs (`specification.job`)
+
+`specification.job()` tests background pipelines (cron jobs, queue consumers, nightly reports) by triggering them in-process against real databases and stubbed external providers. No HTTP server is involved: the subject under test is _what a job writes_, not what an endpoint returns.
+
+Use it when the behaviour you care about starts with "when the job runs...". If the behaviour starts with an HTTP request, use [api](api-specs.md).
+
+## What it specifies
+
+A job spec answers one question: **when this name is triggered, what does the pipeline do to the world it was handed?** The subject is the assembled job (its handler, its services, the network it reaches), met through the one door the scheduler would use, `.trigger(name)`. What a single pure function inside that pipeline returns is a module test's ([Module tests](module-tests.md)); what the pipeline leaves in a database is this page's.
+
+The scheduler that invokes the job in production (a cron, a queue) is out of scope: the job is the subject, and its name is how a spec reaches it.
+
+## The constructor
+
+```typescript
+// specs/job/job.specification.ts
+import { afterAll } from 'vitest';
+import { specification } from '@jterrazz/test';
+import { postgres } from '@jterrazz/test/services';
+import { nightlyReport, supportDrafts } from '../../src/jobs.js';
+
+// Hoisted, so a test can read this database after a chain that rejected
+export const analyticsDb = postgres(); // → "analytics-db", init from docker/analytics-db/
+
+export const { job, cleanup } = await specification.job({
+    services: { db: postgres(), analyticsDb },
+    jobs: ({ db, analyticsDb }) => [nightlyReport(db, analyticsDb), supportDrafts(db)],
+});
+
+afterAll(cleanup);
+```
+
+### Options
+
+| Option     | Required                     | Description                                                                                                        |
+| ---------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------ |
+| `services` | yes (if the jobs need infra) | Named record of service factories, same semantics as [api](api-specs.md), see [services](../reference/services.md) |
+| `jobs`     | yes                          | `(services) => Job[]`, or a static array. Each is a named, triggerable job (rule `server-sees-declared-services`)  |
+| `root`     | no                           | Root-resolution override, same walk-up rule as everywhere (rule `no-redundant-root`)                               |
+
+There is **no `server`** option: jobs run in-process by definition, and the record of services is the only outside world they meet. A `Job` carries the name you pass to `.trigger()`: the handles above respond to `'nightly-report'` and `'support-drafts'`.
+
+## The chain
+
+Setups: `.seed(file('./__seeds__/<name>.sql'))`, `.intercept()`, `.clock()`, and `job.with({ clock })`, a view whose every chain starts at the instant the whole file shares ([Conventions](../concepts/conventions.md#what-a-file-shares)). Terminal action: `.trigger(name)` (rule `one-terminal-action-per-chain`). No `.headers()`: there is no request.
+
+`.clock('2026-03-04T09:30:00Z')` pins the job's own `Date` for the chain and releases it when the action resolves: a nightly report that stamps a window, an expiry the job computes. The primitive and its second depth are [Time and doubles § The clock](../reference/time-and-doubles.md#the-clock).
+
+```typescript
+// specs/job/nightly-report/pipeline.spec.ts: the job, and the aspect this file asks of it
+import { file } from '@jterrazz/test';
+import { describe, expect, test } from 'vitest';
+import { job } from '../job.specification.js';
+import pipeline from './__setup__/stubs/pipeline.stubs.js';
+
+describe('pipeline', () => {
+    test('nightly report classifies, prices and drafts', async () => {
+        // Given - one pending article + the world the pipeline reaches out to
+        const result = await job
+            .seed(file('./__seeds__/one-pending-article.sql'), { database: 'db' })
+            .intercept(pipeline)
+            .trigger('nightly-report');
+
+        // Then - the pipeline produced a classified, priced report
+        await expect(result.table('reports', { database: 'analyticsDb' })).toMatch([
+            [1, 'ELECTRONICS', 109],
+        ]);
+    });
+});
+```
+
+Everything a pipeline reads from the outside world is declared: seeds set the database state, stubs pin the external providers (OpenAI, Anthropic, arbitrary HTTP; see [stubs](../reference/stubs.md)). Databases reset at the start of every chain, exactly as for API specs (rules `one-terminal-action-per-chain`, `sequence-expressed-by-seeding`).
+
+Because jobs run in-process by definition, `.intercept()` is always available. It is **strict** (rule `network-strict-after-first-stub`): once a chain declares one stub, an outgoing request no stub answers fails the spec ([Stubs § Strict by construction](../reference/stubs.md#strict-by-construction)). A chain with no stubs is not network-guarded.
+
+### Seeding and sequences for pipelines
+
+Selection is **first non-exhausted match wins**, so a multi-call scenario is a finite stub in front of an unlimited tail; `times` says how many calls the first one answers:
+
+```typescript
+test('retries then recovers from provider rate-limit', async () => {
+    // Given - the first call is rate-limited, everything after it succeeds
+    const result = await job
+        .seed(file('./__seeds__/pending-articles.sql'), { database: 'db' })
+        .intercept([
+            { request: openai.chat(), response: openai.error(429), times: 1 },
+            { request: openai.chat(), response: openai.reply({ category: 'BOOKS' }) },
+        ])
+        .trigger('nightly-report');
+
+    // Then - the retry succeeded
+    await expect(result.table('reports', { database: 'analyticsDb' })).toMatch([
+        [1, 'BOOKS', '{{number}}'],
+    ]);
+});
+```
+
+The number in `times` IS the assertion: it says the job retries exactly once. A stub with no `times` is unlimited, which is what you want for a route the pipeline may hit any number of times.
+
+## The result
+
+`.trigger()` resolves to a result whose primary subjects are the databases:
+
+| Member                      | Description                                                                 |
+| --------------------------- | --------------------------------------------------------------------------- |
+| `result.table(name, opts?)` | Table subject, compared whole: `toMatch(<rows>)` / `toBeEmpty()`, `await`ed |
+
+Whether `.seed()` and `.table()` name a database is [Services § The `database:` rule](../reference/services.md#the-database-rule).
+
+## Unique here
+
+### Error-case testing: a provider that fails on purpose
+
+Provider failure modes are first-class response builders. This is where job specs earn their keep, because these paths are nearly impossible to reproduce against live providers:
+
+```typescript
+test('handles malformed model output', async () => {
+    // Given - the model returns something that is not JSON
+    const result = await job
+        .intercept(openai.chat(), openai.malformed('not json at all'))
+        .trigger('nightly-report');
+
+    // Then - nothing half-written
+    await expect(result.table('reports', { database: 'analyticsDb' })).toBeEmpty();
+});
+
+test('times out gracefully', async () => {
+    // Given - a silent provider (delay longer than the job timeout)
+    const result = await job
+        .intercept(anthropic.messages(), anthropic.timeout())
+        .trigger('support-drafts');
+
+    // Then
+    await expect(result.table('support_drafts', { database: 'db' })).toBeEmpty();
+});
+```
+
+The three failure families:
+
+| Builder                                                                | Simulates                                              |
+| ---------------------------------------------------------------------- | ------------------------------------------------------ |
+| `openai.error(429)` / `anthropic.error(status)` / `http.error(status)` | HTTP error status from the provider                    |
+| `openai.timeout()` / `anthropic.timeout()`                             | A provider that never answers within the job's timeout |
+| `openai.malformed('...')`                                              | A 200 whose body violates the provider schema          |
+
+The full builder catalogue lives in [stubs](../reference/stubs.md).
+
+### A crash is read by its rejection
+
+A job that throws rejects `.trigger()` with its own error, and what it wrote before the throw stays in the database. Read it from the service handle, since a rejected chain hands back no result. That is why the constructor above hoists `analyticsDb` out of the record: a handle declared on its own is one a test can import and query.
+
+```typescript
+import { describe, expect, test } from 'vitest';
+
+import { analyticsDb, job } from '../job.specification.js';
+
+describe('crash', () => {
+    test('a crashed job leaves what it wrote first', async () => {
+        // Given - a job that writes one row, then throws
+        const run = job.trigger('crash-after-insert');
+
+        // Then - its error, and the row it wrote before it
+        await expect(run).rejects.toThrow('crash-after-insert: boom');
+        await expect(analyticsDb.query('events', ['type'])).resolves.toStrictEqual([['pre_crash']]);
+    });
+});
+```
+
+## Pitfalls
+
+- **Looking for a `server` option.** It exists only on `specification.api()` and `specification.website()`; jobs are in-process by definition; what a job reads from the outside is its `services` record (rule `server-sees-declared-services`).
+- **Asserting on a job's return value.** The result surface is the observable state (tables); jobs are specified by their effects.
+- **Forgetting `times` on the leading stub of a sequence.** With no `times` the first stub is unlimited, so it answers every call and the recovery stub behind it is never reached.
+- **One giant spec that triggers two jobs.** A chain has exactly one terminal action (rule `one-terminal-action-per-chain`); sequence scenarios are expressed by seeding the state the second job would have found (rule `sequence-expressed-by-seeding`).
+- **Forgetting `await` on table expectations.** `expect(result.table(...))` without `await` never runs the SQL: table matchers are IO matchers, typed `Promise<void>`, and the type-aware `typescript/no-floating-promises` refuses one dropped.
+- **Bounding the tail of a sequence.** The last stub of a sequence should stay unlimited; a `times` there turns one extra retry into an "unmatched outgoing request" failure (rule `network-strict-after-first-stub`).
+
+## Related
+
+[API specs](api-specs.md) · [Assertions](../reference/assertions.md) · [Stubs](../reference/stubs.md) · [Services](../reference/services.md)

package/docs/guides/lint-specs.md

@@ -0,0 +1,174 @@
+# Lint specs (`specification.lint`)
+
+`specification.lint()` tests the rules a project ships (an oxlint plugin, a cross-file checker, the laws a `@jterrazz/typescript` preset holds a tree to) by running the real linters over fixture trees and comparing their whole report. The subject is a rule, met the way a consumer meets it: a tree in, the linter's verdict out.
+
+Use it when the behaviour you care about starts with "when this tree is linted...". A rule's visitor alone, on one snippet, is a module test with [`ruleTester()`](#a-rule-alone-ruletester); the tree the rule is meant to judge, with its folders and its neighbours, is this page's.
+
+## What it specifies
+
+A lint spec answers one question: **when this tree is linted, where does which rule report, and what does it say?** A rule is a real subject a team tests, like a binary: it has inputs (a tree), a contract (a report), and consumers who read that report. A rule reading ONE file is proven by its report over a tree holding that file; a rule reading the whole tree (a folder's shape, a manifest, what imports what) can only be proven that way, so one specification serves both.
+
+What it does not do: judge the tree it runs in. The specification lints a fixture laid in a fresh working directory, never the repository's own sources; those are `typescript check`'s.
+
+## The constructor
+
+The examples below are a package that ships an oxlint plugin and a preset, both built to its own `dist/`: `dist/oxlint.js` is that package's built plugin, and `dist/config.js` exports its preset, here called `acme()`. Use the paths your build writes, or the published specifier of your package (`'@acme/lint/oxlint'`). A plugin is loaded from what was built, so build before the lint project runs ([§ Pitfalls](#pitfalls)).
+
+```typescript
+// specs/lint/lint.specification.ts
+import { afterAll } from 'vitest';
+
+import { recommendedRules } from '../../dist/oxlint.js';
+import { specification } from '@jterrazz/test';
+
+export const { lint, cleanup } = await specification.lint({
+    config: { categories: { correctness: 'off' }, rules: recommendedRules },
+    plugins: ['../../dist/oxlint.js'],
+});
+
+afterAll(cleanup);
+```
+
+A preset's laws, or a checker of the project's own, are linters of the same specification:
+
+```typescript
+// specs/lint/preset-lint.specification.ts
+import { afterAll } from 'vitest';
+
+import { acme } from '../../dist/config.js';
+import { specification } from '@jterrazz/test';
+
+export const { lint, cleanup } = await specification.lint({ presets: [acme()] });
+
+afterAll(cleanup);
+```
+
+### Options
+
+At least one linter: a plugin, a config, a checker or presets, `presets: []` holding the tree to the base preset alone; a specification naming none is refused.
+
+| Option     | Description                                                                                                                                                                                                                              |
+| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `plugins`  | oxlint JS plugins, each a path from the specification's folder or a package specifier. They run in the real oxlint, `oxlint` being an optional peer the project installs                                                                 |
+| `config`   | The oxlint config oxlint runs under (`rules`, `settings`, `categories`, `overrides`), less `jsPlugins`, which `plugins` states; named alone, it runs oxlint's built-in rules                                                             |
+| `presets`  | `@jterrazz/typescript` presets whose laws run on the tree (the Architecture pass over every `layers`, each declared `names` rule), resolved after the always-on base preset as `jterrazz.config.ts` would; `[]` is the base preset alone |
+| `checkers` | Cross-file checkers, each a command printing one finding a line in the compiler's convention, `<file>:<line>:<column>: error <plugin>(<rule>): <message>`; `.run()` hands it the tree last                                               |
+| `root`     | Root-resolution override (rule `no-redundant-root`); the plugins' specifiers and the checkers' commands resolve from its `node_modules`                                                                                                  |
+| `settings` | What the project adds beside its presets, written as `jterrazz.config.ts` writes it (`typescript: { names, architecture: { layers } }`, `optOuts`), so a repository's own additions are its own, never a preset it makes up              |
+
+There is nothing to start and nothing to stop: `cleanup` is handed to `afterAll` like every constructor's, and releases nothing.
+
+## The chain
+
+Setups: `.fixture(directory('./__fixtures__/<tree>/'))` (or a `file()`, copied by name) lays the tree in a fresh working directory ([CLI specs § Fixtures](cli-specs.md#the-chain)); `.rules({ ... })` states rules and their options for this chain, over the specification's `config`, a later call winning; on a specification that runs no oxlint (no plugin, no config) it is refused, since no linter would read it. Terminal action: `.run()` (rule `one-terminal-action-per-chain`).
+
+```typescript
+// specs/lint/no-module-mock/no-module-mock.spec.ts
+import { describe, expect, test } from 'vitest';
+import { directory } from '@jterrazz/test';
+
+import { lint } from '../lint.specification.js';
+
+describe('no-module-mock', () => {
+    test('rejects an allow-list entry with no reason', async () => {
+        // Given - a module on the allow-list, and no reason beside it
+        const result = await lint
+            .rules({ 'acme/no-module-mock': ['error', { modules: [{ module: 'x', reason: '' }] }] })
+            .fixture(directory('./__fixtures__/allow-listed/'))
+            .run();
+
+        // Then - the finding, where it fires, with its lesson
+        expect(result.report).toMatch(`
+            src/storage.test.ts:5:1  acme(no-module-mock)
+                found: "x" is on the \`modules\` allow-list with no reason
+                why: A test doubles a PORT, never a module.
+                fix: state why no consumer can inject it
+                read: docs/reference/linting.md#no-module-mock
+        `);
+    });
+});
+```
+
+`.run()` takes two options:
+
+| Option | Description                                                                                                                                                                                                             |
+| ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `fix`  | Apply oxlint's and the checkers' fixes, then lint the fixed tree: the report is what a run after the fix says, and `result.file()` reads what it wrote; the presets' laws have no fix, and are judged on the fixed tree |
+| `args` | Handed to every checker before the tree, as a checker takes a flag (`--member packages/web`)                                                                                                                            |
+
+## The result
+
+| Member                   | Description                                                         |
+| ------------------------ | ------------------------------------------------------------------- |
+| `result.report`          | Every linter's diagnostics, as a text subject (see below)           |
+| `result.file(path)`      | A file of the tree after the run, relative to the working directory |
+| `result.directory(path)` | A folder of the tree, compared whole to a golden tree               |
+
+### The report
+
+One diagnostic is its head line, `<file>:<line>:<column>  <plugin>(<rule>)`, then its message: a lesson a part per line, `found:`, `why:`, `fix:`, `read:`, or the message as the linter wrote it:
+
+```text
+specs/app/widget/widget.test.ts:8:36  acme(no-sleep)
+    found: an arbitrary sleep in a test
+    why: No arbitrary sleep in any test file.
+    fix: wait on what the sleep stands for
+    read: docs/reference/linting.md#no-sleep
+```
+
+The diagnostics come in the order a reader walks the tree (file, then line, then column), whichever linter printed them and whatever order its threads finished in; a tally, a summary or any other line is not one. A clean tree is the empty text: `expect(result.report).toMatch('')`. Written in place and rewritten by `TEST_UPDATE=1` ([Tokens](../reference/tokens.md#update-mode-tokens-are-preserved)), the golden proves where each rule fired, which one, and what it teaches.
+
+oxlint's graphical rendering is refused, never read as a clean run: the specification states `--format=unix` itself, and a report handed to it in frames names the format to state.
+
+## Unique here
+
+### What `.run()` runs
+
+Each linter the specification names, over the working directory:
+
+1. **oxlint**, when `plugins` or `config` is named: the real binary, with a config the specification writes outside the tree (`jsPlugins` the plugins, `rules` the specification's `config.rules` and the chain's `.rules()` over them), at `--format=unix`.
+2. **each checker**, as `<command> <args> .`, from the tree, its first word resolved from the project's `node_modules/.bin` when it is installed there.
+3. **the presets' laws**, in process, through `presetChecks` of `@jterrazz/typescript/rules`, handed the `presets` and the `settings` as one settings object: each finding at its position, one about a whole file or folder at `:1:1`. Settings the kit cannot resolve are refused like a linter that fails with no diagnostic.
+
+A preset's own `oxlint` (its plugin, its rulebook) is not run by `presets`: the presets hand the specification their laws alone. A preset's rules are proven as any plugin's are, the plugin under `plugins` and its rules under `config`, so a report holds only the rules the specification names.
+
+A linter that fails and reports no diagnostic is refused in four parts rather than read as a clean tree: a plugin that does not load, a config oxlint cannot build, a checker that cannot start.
+
+### A rule pack: a nested Vitest and a reporter fixture
+
+A package that ships lint rules for other projects proves them by running the real binary on a tree: the spec starts a nested `vitest` over a fixture project (`.fixture(directory(...))`) with a reporter fixture that prints one line per finding, and compares that output whole. A rule alone is `ruleTester()`.
+
+### A rule alone: `ruleTester()`
+
+A rule's visitor, on snippets, is a module test in the `__tests__/` of the rule's folder. `ruleTester()` is oxlint's `RuleTester` (`oxlint/plugins-dev`) bound to vitest (its suites and cases are vitest's), and it requires a position on every invalid case: each expected error is an object stating its `line` and `column`. A count, an empty list, or an error with no position, is refused before the case runs, since it passes for a report on any node.
+
+The two tools count columns differently, and each keeps its own convention: `ruleTester()` reads oxlint's RuleTester columns, **0-based** (`column: 25` below is the `setTimeout` at index 25), while a lint spec's report prints the compiler's `file:line:column`, **1-based**, so the same finding reads `:1:26` there.
+
+```typescript
+// src/rules/no-sleep.test.ts
+import { ruleTester } from '@jterrazz/test';
+
+import { noSleep } from './no-sleep.js';
+
+ruleTester().run('no-sleep', noSleep, {
+    invalid: [
+        {
+            code: 'await new Promise((r) => setTimeout(r, 100));',
+            errors: [{ column: 25, line: 1, messageId: 'slept' }],
+            filename: '/project/specs/api/users/list.spec.ts',
+        },
+    ],
+    valid: ['await waitUntil(() => ready);'],
+});
+```
+
+## Pitfalls
+
+- **Linting a stale build.** The plugins and checkers a project ships are usually loaded from its `dist/`: build before the lint project runs, or the specs judge the previous rules.
+- **A golden cut to the rule ids.** The report carries each lesson whole, so a reworded message moves the golden: that is the contract a reader meets, and `TEST_UPDATE=1` rewrites it.
+- **A literal token in a message.** A lesson that echoes `{{uuid}}` is read as the token in the golden; cover the echo with `{{string}}` ([Tokens § Known limitations](../reference/tokens.md#known-limitations)).
+- **One specification for every linter.** A plugin's fixture judged by the checker too reports what it was never built to show: give each linter a specification of its own (`checker-lint.specification.ts`, `preset-lint.specification.ts`).
+
+## Related
+
+[CLI specs](cli-specs.md) · [Assertions](../reference/assertions.md) · [Tokens](../reference/tokens.md) · [Linting](../reference/linting.md)