@lariocpt/crust-mongo 0.3.5 → 0.3.7

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lariocpt/crust-mongo",
-  "version": "0.3.5",
+  "version": "0.3.7",
   "description": "crust with the MongoDB driver bundled — the mongo variant of the crust binary launcher",
   "type": "module",
   "license": "MIT",

package/skills/crust-api-testing/SKILL.md

@@ -1,11 +1,23 @@
 ---
 name: crust-api-testing
-description: Test HTTP APIs with crust — .pipes suites (one fixture pipeline per line, request chaining via capture, inline SQL verification), .crust.ts fixtures with setup/matchers, and gen-fixtures which derives negative-case matrices and CRUD flow suites from an OpenAPI spec. Use for API integration tests, contract tests from a spec, auth/authz negative cases, or DB-verified endpoint tests.
+description: Test HTTP APIs with crust — .pipes suites (one fixture pipeline per line, request chaining via capture, inline SQL verification), .crust.ts fixtures with setup/matchers, gen-fixtures which derives negative-case matrices and CRUD flow suites from an OpenAPI spec, and the evidence-tier taxonomy (snapshot vs DB-verified vs demo). Use for API integration tests, contract tests from a spec, auth/authz negative cases, or DB-verified endpoint tests.
 ---
 
 # crust API testing
 
-Three layers, one binary. Pick the lightest that fits:
+## The shared principle: the lightest CORRECT layer
+
+Choose the lightest native layer that preserves correctness, observability,
+and safety — not merely the shortest syntax.
+
+- Use `.pipes` for straightforward requests, assertions, and supported request chaining.
+- Use `.crust.ts` when setup, custom matchers, or required safety controls cannot be expressed in pipes.
+- Use `gen-fixtures` for supported contract-derived cases, followed by review.
+- Use native load pipelines for paced traffic, measurements, and explicit success gates.
+- Do not force a case into pipes when doing so removes a required guarantee.
+- Consult repository-specific execution and credential policies before running examples.
+
+Three layers, one binary:
 
 | Layer | File | Best for |
 |---|---|---|
@@ -55,6 +67,9 @@ writes the same text report stdout shows; the file never changes the exit code.
   exits 0 — gate with `expect 200` or `stats | assert (s =>
   s.redirectRefused === 0)`; exit 0 alone is never request success in load
   mode.
+- Result shapes per execution context (`Response` vs timed `{status, ms, url}`
+  records, body draining, `.json()` availability): canonical table at
+  https://lariocpt.github.io/crust-website/docs/http-result-shapes/.
 - Exit codes: 0 all pass · 1 any fail · 2 no files/bad args.
 
 ## .crust.ts fixtures (test-fixture)
@@ -104,6 +119,36 @@ module-scope promise-cached factory. Fixture files may import ONLY relative
 modules and Bun builtins (the binary can't resolve npm at runtime) —
 `Bun.SQL`, `Bun.file`, `Bun.jwt`.
 
+## What evidence does your test produce?
+
+**Tier 1 — snapshot / contract verification.** Response vs a declared case +
+supplied snapshot; setup prepares REQUEST CONTEXT (url, headers, expected),
+not state. Never call it "seeded"/"integration" when nothing is seeded or
+verified at the data layer — it proves contract shape + declared oracle.
+
+**Tier 2 — DB-verified flow.** seed → verify the seed → call the API → query
+the DB → compare against the LIVE query result. Native shape:
+
+```crust
+{"email": "u-$RUN_ID@t.dev"} | POST $BASE/api/users -H "authorization: Bearer $TOKEN" | assert (r => r.status === 201) | (r => r.json()) | capture UID (u => u.id)
+sql "SELECT email FROM users WHERE id = '$UID'" | assert (r => r.email === "u-$RUN_ID@t.dev")
+```
+
+Carries: run-owned data (`$RUN_ID` unique per run, set by setup — never
+shared mutable state); explicit identity matching (query the id you
+captured, never an unrelated record); failure-path cleanup defined BEFORE
+writes — a later line is happy-path only; guaranteed teardown is `crust.onExit`.
+
+**Tier 3 — demo / quickstart.** Disposable local resource, ordinary CRUD —
+the built-in mock server is the native fit — isolates "how little glue does
+crust need" from app safeguards (graph resolution, provisioning denial,
+identity-sensitive stores) — separate approved fixture, never folded in.
+
+Cross-cutting: label the tier so a green run cannot imply DB-level proof it
+never produced; approval/redirect/transport safeguards stay regardless of
+tier; setup attestations (flags asserting IAM/graph/provisioning policy) are
+DECLARATIONS, not backend verification.
+
 ## gen-fixtures — spec-driven cases
 
 ```
@@ -130,6 +175,10 @@ Setup-module contract (imported lazily): `shared()` promise-cached context,
 `flowOverrides: { "<template>": { body?: {...}, skip?: true } }` for creates
 whose values a spec can't express (business date rules → `body` merge;
 required foreign keys → `skip`).
+When the contract disagrees with the 400 convention (422 backends, 404/403
+concealment), the optional `expectations` block configures correct statuses —
+a documented status always wins over a configured one;
+`docs/USAGE.md §gen-fixtures` has the schema and precedence.
 
 The `--out` dir is DELETED and recreated every run — never hand-edit
 generated files.
@@ -145,10 +194,106 @@ whose create body cannot be filled — are dropped and NAMED on stdout
 (`gen-fixtures: 2 case(s) skipped for POST /things …`, `skipping flow for
 /things …`). A silently smaller suite would look like coverage.
 
+Every generation's LAST write is `<out>/gen-manifest.json`: each case with
+its expectation's provenance (`contract:<spec pointer>` when the spec
+documents the status), each skip, each unsupported op — review it BEFORE
+running. Deterministic (same spec + version → byte-identical); it records
+generation, never execution.
+https://lariocpt.github.io/crust-website/docs/recipes/gen-fixtures/
+
+## Generated cases are candidates, not certified coverage
+
+A generator can derive request mutations from a schema. It cannot universally
+infer backend-specific status codes, error envelopes, authorization
+concealment policies, business preconditions, or cleanup behavior.
+
+For negative tests:
+
+- Begin with an otherwise valid request.
+- Mutate one intended condition.
+- Ensure authentication and resource preconditions allow the request to
+  reach the intended validation layer (a request that dies at auth never
+  proves the validation code).
+- Assert the documented status.
+- Where the contract provides structured error evidence, verify the relevant
+  field, location, or error code.
+- Avoid exact human-readable message matching unless wording is contractual.
+
+A generic error status is insufficient evidence when the request could have
+failed for an unrelated reason. And the generated **400 validation matrix is
+a generator assumption, not a universal API convention** — some contracts
+specify `422` (or another status) for validation failures; when the contract
+disagrees, say so in review and adjust expectations to the CONTRACT, never
+the other way around. Document unsupported assumptions and skipped cases
+rather than silently reducing coverage or normalizing the contract to fit
+generation.
+
+## Contract review before generation
+
+Review the contract BEFORE generating and record the review beside the
+generated output. Record fields (one per line):
+
+```
+API/service:
+Contract path:
+Contract revision:
+Backend revision, if available:
+Crust version:
+Reviewer:
+Review date:
+Generated output directory:
+Intended execution scope:
+Unresolved discrepancies:
+```
+
+Highest-risk checks (full checklist:
+https://lariocpt.github.io/crust-website/docs/contract-review/):
+
+- Expected statuses come from the reviewed contract — validation cases do
+  NOT default to 400 (a 422 contract must generate 422 expectations).
+- Authorization cases account for intentional 404 concealment; unknown-id
+  cases use syntactically valid identifiers.
+- One intended condition per negative case; unrelated fields stay valid so
+  the request reaches the intended validation layer.
+- Dedicated `--out` dir; regeneration cannot touch manual fixtures; never
+  hand-edit generated files.
+- Classify every failure (backend · contract · generator · environment ·
+  incorrect test) BEFORE changing any expectation.
+- Execution is approved separately — target, identity, resource scope,
+  writes, cleanup. This review grants no permission beyond the recorded
+  execution scope.
+
 ## Workflow for a new API
 
-1. `gen-fixtures` against the spec → run matrix + flows → fix what fails.
-2. Hand-write `.pipes` for business flows the spec can't express (SQL
-   checks, multi-resource invariants).
-3. Reach for `.crust.ts` only when a case needs real code (crypto, files,
-   custom matchers).
+1. Identify the authoritative contract and record its version or revision.
+2. Review generation assumptions against that contract: validation status
+   codes; authentication and authorization behavior; error envelopes;
+   resource identifiers; response schemas; CRUD lifecycle support.
+3. Generate into a dedicated output directory (`--out`, default `tests/gen`).
+   Generation DELETES and recreates that directory; never target manually
+   authored sources.
+4. Review generated cases, expected responses, setup requirements, and
+   skipped cases (they are NAMED on stdout) before execution.
+5. Obtain the required target, identity, resource, write, cleanup, and load
+   approvals from repository policy — the skill never grants any of them.
+6. Execute only the approved scope.
+7. Classify failures before changing expectations or implementation:
+   backend defect · contract defect · unsupported generator assumption ·
+   environment/setup failure · incorrect test.
+8. Add manually authored native fixtures or pipelines for unsupported cases.
+   Do not hand-edit generated files (the next regeneration erases the edit).
+9. Record remaining coverage gaps explicitly.
+
+> Never change an expected result solely because the backend returned
+> something different. A passing test must reflect a reviewed contract.
+
+## Version and evidence
+
+Behavior described here was verified against **crust v0.3.5** (source-read
+and run through the compiled binary where it matters: `--redirect` record
+shapes, `output.schema` $ref refusal, gen-fixtures skip reporting). New in
+**v0.3.6**: the `gen-manifest.json` review artifact, the setup-module
+`expectations` block, and the evidence-tier taxonomy. Newer
+crust versions may extend behavior; re-verify before documenting it. Keep
+approved targets, credentials, transport exceptions, resource ownership, and
+execution restrictions in repository policy, not in this skill.

package/skills/crust-load-testing/SKILL.md

@@ -27,12 +27,25 @@ load 10s 50/s, 30s 200/s | parallel 100 | GET :3000/health | stats --every 5
 load 10s 20/s | (t => ({name: "user" + t.n})) | parallel 8 | POST :3000/users | expect 2xx | stats
 ```
 
+> **Report availability:** `expect` throws at drain, so an `expect` placed
+> BEFORE `stats` that fails prevents the summary from ever being emitted
+> (the first two shapes above are illustrative of syntax, not of CI
+> reporting). When the summary/report must survive a status failure, put
+> `stats` (and `stats --out report.json`) before the gates and gate status
+> on the summary instead — see CI gates below.
+
 - `parallel` puts ANY http verb in load mode: output is `{status, ms, url}`
   timing records, bodies drained, network errors become `status: 0` records
   (they show in the histogram instead of killing the run). So a run whose
   every request was refused prints `{"status":{"0":40}}` and still exits 0
   unless a gate is present — `expect 200` over a dead port exits 1. `stats`
   reports, it does not gate.
+- **Completion ≠ success — three separate outcomes:** the run *completed*
+  (exit 0, no fatal stage error), *request correctness passed* (explicit
+  gates accepted the outcomes), *performance passed* (latency/throughput
+  thresholds held). A gate that only checks latency can be satisfied by fast
+  error responses — a 500 in 2ms is under any p95 budget. Latency-only
+  examples are NOT the definition of a passing load test; gate status too.
 - The summary carries failure tallies `timedOut` and `redirectRefused`
   (always present, 0 on a clean run): a `--timeout` budget hit or a
   `--redirect error` refusal is a `{status: 0, timedOut/redirectRefused:
@@ -43,6 +56,8 @@ load 10s 20/s | (t => ({name: "user" + t.n})) | parallel 8 | POST :3000/users |
   Little's law sizes the pool: `N ≥ rate × p99-in-seconds`.
 - Durations `ms|s|m`; rates `N/s` or `N/m`, decimals allowed.
 
+Canonical, version-pinned result-shape table: https://lariocpt.github.io/crust-website/docs/http-result-shapes/.
+
 ## Honesty guarantees (read before quoting numbers)
 
 - A gate that measured NOTHING fails: `stats` over an empty stream is tagged
@@ -67,12 +82,24 @@ load 10s 20/s | (t => ({name: "user" + t.n})) | parallel 8 | POST :3000/users |
 
 ## CI gates — thresholds are assert composition
 
+The primary shape gates **status first, then performance**, with `stats`
+(and the report) upstream of every gate so the summary survives a failing
+gate. When failure artifacts must survive request-status failures, collect
+statistics and write the report BEFORE applying aggregate gates:
+
 ```crust
-load 30s 100/s | parallel 50 | GET :3000/health | stats | assert (s => s.p95 < 200) | assert (s => s.rps > 80)
+load 30s 100/s | parallel 50 | GET :3000/health | stats --out load/last.json | assert (s => Object.keys(s.status).every(k => k.startsWith("2"))) | assert (s => s.timedOut === 0) | assert (s => s.redirectRefused === 0) | assert (s => s.p95 < 200) | assert (s => s.rps > 80)
 ```
 
 One assert per threshold: the failure names the predicate and prints the
-actual summary, and the line exits 1 (`crust -c` stops there).
+actual summary, and the line exits 1 (`crust -c` stops there). The gates
+above cover all four things a load gate must mean: every response was 2xx
+(a fast 500 is not a pass), no timeout or redirect-refusal records, the
+latency budget held, and the achieved rate held. An empty run is refused by
+`assert` itself (`empty: true`), so "measured nothing" never passes.
+`rps` is the ACHIEVED rate — if the target was not sustained, stderr already
+said so (`emitted … dropped … achieved …`); a drop report means the number
+below the rate gate is the honest one.
 
 Baseline gate — `--out` writes a versioned artifact
 (`{crustStats: 1, startedAt, urls, summary, windows?}`), an async assert
@@ -121,3 +148,14 @@ const out = await load([{ durMs: 10_000, rps: 100 }])
 
 `timedHttpItem(method, url, opts?)` is the any-verb per-item timer when you
 need POST bodies from upstream items.
+
+## Version and evidence
+
+Behavior here was verified against **crust v0.3.5** — summary schema
+(`{count, wallMs, rps, status, timedOut, redirectRefused, p50, p95, p99,
+meanMs}`, `empty: true` on a zero-request run) and the `--out`-before-gates
+write order are source-verified; throughput claims (5000+ ticks/s) are
+single-process measurements on the author's hardware — environment-dependent,
+not a promise. Pick the lightest correct layer for the job (see the
+crust-api-testing skill's shared principle), and keep approved targets and
+execution restrictions in repository policy, not in this skill.

package/skills/crust-mcp/SKILL.md

@@ -66,6 +66,31 @@ printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion
 range(1,5) | (n => n * n) | stats
 ```
 
+## Tool classes and teardown tiers
+
+The `initialize` handshake carries this taxonomy in `instructions` (0.3.6):
+
+- **Static check** — `crust_check_line`: parses, runs nothing.
+- **Execution** — `crust_run_line` and the test/gen tools: arbitrary stages,
+  real side effects. Target approval, credentials, writes and cleanup stay
+  the caller's responsibility.
+- **Network/load generation** — `crust_load_run`, `crust_verify_web_links`:
+  bounded generation, approved non-production targets only.
+- **Long-lived resources** — mock server, procs, log sessions: held by the
+  server process, die when it exits.
+
+Teardown has three tiers and no tool crosses them: **local processes** are
+killed whole (process group, grandchildren included); **mock sessions** close
+(port released — `--state` files/DBs are application data and stay on disk);
+**application data** is never touched — a container, row or queue a proc
+created remotely survives its local teardown, releasing it is your job.
+
+Transport contract: stdio JSON-RPC, one frame per line; one-shot tools spawn
+a fresh crust child per call (`cwd`/`env` per call, nothing persists). A
+refusal whose error string never appears in crust's own output (an
+"unavailable tool" style host error) never reached the server — debug the
+host's spawn command, not crust.
+
 ## Honest failure — never a false pass over MCP
 
 A nonzero child exit, a killed timeout, a failed `assert`, a mock validation
@@ -97,8 +122,11 @@ filter (l => l.proc === "api") | (l => l.line) | grep -i timeout
   the returned `url`. `proxy: "<upstream>"` turns it into a conformance audit;
   read findings with `crust_mock_server_violations`.
 - `crust_load_run` composes the documented pipeline (`load … | parallel … |
-  … | stats | assert`); an `assert` failure is `isError` — a run that
-  measured nothing is refused (`empty`), never a green gate.
+  … | stats | assert`); gates go in `assert` — one predicate, or an array of
+  them (0.3.6), one assert stage per gate. An `expect` inside `target` runs BEFORE
+  stats and suppresses the summary when it fails; a post-stats `assert`
+  failure is `isError` with the stats still included. A run that measured
+  nothing is refused (`empty`), never a green gate.
 
 ## Limits, stated
 

package/skills/crust-pipelines/SKILL.md

@@ -9,6 +9,10 @@ crust is a Bun-powered pipeline runner where every command is a typed stream
 (`Pipeline<T>`). Stages compose under `|`. Plain shell still works — a line
 whose stages are all ordinary commands is handed to `sh -c` untouched.
 
+Choose the lightest native stage that preserves correctness, observability,
+and safety — not merely the shortest syntax (full statement: the
+crust-api-testing skill's shared principle).
+
 ## How a stage is classified (first token decides)
 
 | You type | You get |
@@ -39,6 +43,33 @@ whose stages are all ordinary commands is handed to `sh -c` untouched.
 
 `:3000/path` expands to `http://localhost:3000/path` everywhere.
 
+## HTTP result shapes — same syntax, different payloads
+
+Similar HTTP syntax does NOT guarantee identical result shapes. Know which
+shape a stage receives before reaching for `.json()` or gating results
+(verified against crust v0.3.5):
+
+| Context | What the stage emits | Authoring implication |
+|---|---|---|
+| First-stage ordinary verb (`GET :3000/x`) | a `Response` | body parsing is available: `(r => r.json())` |
+| Mid-pipeline ordinary verb, non-GET (per-item, untimed) | a `Response` per item | same — bodies still parseable |
+| Timed per-item path (mid-pipeline `GET` — with or without `parallel` — or `parallel N \| <any other verb>`) | `{status, ms, url}` records; **bodies drained**; network errors are `status: 0` records; `--timeout`/`--redirect error` outcomes are `{status: 0, timedOut/redirectRefused: true}` records | `.json()` does NOT exist on these; gate status and the failure tallies explicitly |
+
+Canonical, version-pinned table: https://lariocpt.github.io/crust-website/docs/http-result-shapes/.
+
+## Completion is not success — three separate outcomes
+
+1. **Execution completed** — the pipeline ran without a fatal stage error (exit 0).
+2. **Request correctness passed** — explicit gates (`expect`, `assert`) accepted the request outcomes.
+3. **Performance passed** — explicit latency/throughput thresholds passed.
+
+`stats` reports measurements; it is not itself a success gate. In timed/load
+paths, timeouts, network failures, and redirect refusals are per-item
+RECORDS (`status: 0` + a tally field), not exits — exit 0 alone is never
+evidence that requests succeeded. Use an explicit expected-status gate
+(`expect 200`/`expect 2xx`), and where stats is in play also check
+`s.timedOut === 0` / `s.redirectRefused === 0`.
+
 ## Env expansion — parse time, opt-in per position
 
 `$VAR`/`${VAR}` expand from `process.env` in: URLs, whole `-H` header
@@ -159,3 +190,20 @@ Any stage throw → `crust: <message>` on stderr, line exits 1. `crust -c`,
 the same fail-fast loop and STOP at the first non-zero — put warmup and
 cleanup on their own lines. Blank lines and `#` comments are skipped, so a
 `#!/usr/bin/env crust` shebang works in `.crust` files.
+
+**Cleanup on a later line is happy-path cleanup, not guaranteed teardown.**
+Under fail-fast, an earlier failure means that line never runs. When
+failure-path cleanup is a requirement (containers, temp rows, mock servers),
+use the verified lifecycle hook: `crust.onExit = async (code) => {…}` in
+`init.ts` — it fires on every shutdown-path exit,
+including the fail-fast failure path of `-c`, script, and stdin runs
+(test-pinned by `tests/onexit.test.ts`), and receives the exit code. If a needed
+guarantee is still missing (e.g. `kill -9`), document the limitation rather
+than implying line placement solves it.
+
+## Version and evidence
+
+Behavior here was verified against **crust v0.3.5** — result shapes and
+failure-record semantics by source and through the compiled binary; keep
+approved targets, credentials, and execution restrictions in repository
+policy, not in this skill.

package/skills/crust-verify-web-links/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: crust-verify-web-links
-description: Crawl a site from its sitemap and verify every link, #anchor, redirect chain, OG image and meta tag against .crust.ts fixtures. `--base-url` auto-discovers the sitemap (robots.txt → /sitemap.xml → /sitemap-index.xml), `--site-map-url` takes a local sitemap so you can check build output, `--fixtures` diffs meta. Use when link or SEO health must gate a deploy, when a built site should be checked before publishing, or when OG/meta regressions need catching.
+description: Crawl a site from its sitemap and verify every link, #anchor, redirect chain, OG image and meta tag against .crust.ts fixtures. `--base-url` auto-discovers the sitemap (robots.txt → /sitemap.xml → /sitemap-index.xml), `--site-map-url` takes a local sitemap so you can check build output, `--fixtures` diffs meta. Every failure names its source page and element (structured `sources`/`meta`), and every failing run ends with an aligned fix-list. Use when link or SEO health must gate a deploy, when a built site should be checked before publishing, or when OG/meta regressions need catching.
 ---
 
 # crust verify-web-links
@@ -21,6 +21,41 @@ having left discovered links unchecked. `2` also covers `--site-map-url`
 together with `--base-url`, neither of them given, `no fixture files matched
 <glob>`, `--concurrency must be >= 1`, and `unexpected argument`.
 
+## Every failure names its source (new in 0.3.6)
+
+Failures are **one per failing target**, and each answers "which page did I
+find this on, and which element":
+
+- `sources[]` — every link that pointed at the failing target, as
+  `{fromUrl, kind, href}`. In text each prints as an indented
+  `from <page> (<kind>)` line under the failure; in `--json` it is the full
+  array. A broken URL linked from fifty pages is still **one** failure with
+  fifty sources — `totals.failures` and exit codes are unchanged.
+  `kind: "sitemap"` means the URL arrived as a sitemap `<loc>` and no page
+  linked it; the sitemap URL is named as `fromUrl` instead of the source being
+  omitted.
+- `meta` — structured `{key, expected, actual}` for `meta-mismatch`
+  (`meta-fixture-no-page` carries `key: "fixture"`).
+- **Fix-list**: every failing text run ends with a separate aligned section —
+  one line per (page, element, broken-thing) triple, nothing else (excerpt of
+  the real output; the full failing report is in USAGE and on the website):
+
+  ```text
+    fix list
+    page                           element        broken link / diff
+    http://localhost:47831/page-a  a              http://localhost:47831/broken
+    http://localhost:47831/page-a  a              http://localhost:47831/page-x#ghost
+    http://localhost:47831/page-b  img            http://localhost:47831/broken
+    http://localhost:47831/page-x  meta:og:title  got "Page X — Real Title" (expected "Page X — Wrong Title")
+  ```
+
+  A clean run prints no fix-list section on either channel. Under `--json` the
+  report owns stdout, so the same table is written to **stderr** (0.3.7) — the
+  structured `sources`/`meta` fields carry the same facts for machines.
+
+The `detail` prose is unchanged from v0.3.5: the structured fields are purely
+additive.
+
 ## Getting it pointed at the right thing
 
 ```crust
@@ -71,6 +106,9 @@ Redirect the whole invocation instead, then judge the report:
 crust -c 'verify-web-links --base-url https://example.com --json --no-progress' > report.json
 ```
 
+The redirect is the artifact; the fix-list arrives on stderr, so the CI log
+still names the broken page and element without a second command.
+
 ```crust
 read report.json | (s => JSON.parse(s)) | assert (r => r.totals.failures === 0)
 read report.json | (s => JSON.parse(s)) | assert (r => r.totals.complete)