@vibe-agent-toolkit/cli 0.2.0-rc.11 → 0.2.0-rc.12

package/docs/resources.md

@@ -14,8 +14,8 @@ including link integrity checking and anchor validation.
 **What it does:**
 1. Recursively scans for markdown files
 2. Parses each file to extract links and headings
-3. Shows statistics about discovered resources
-4. Exits 0 always (informational only)
+3. Publishes what it found as the shared report envelope
+4. Exits 0 when it scanned at least one file, 1 when it scanned none
 
 **Path Argument:**
 - `[path]` restricts the scan to that subtree of the project
@@ -29,24 +29,40 @@ including link integrity checking and anchor validation.
 **Options:**
 - `[path]` - Base directory to crawl (defaults to current directory)
 - `--debug` - Enable debug logging
-- `--verbose` - Add a `files:` list with per-file link/anchor counts and checksums
-- `--collection <id>` - Only report files in the named collection (config mode — no path argument)
+- `--verbose` - Add `data.files`: per-file link/anchor counts and checksums
+- `--collection <id>` - Only report files in the named collection (config mode — no path argument).
+  A name `resources.collections` does not declare is refused (`USAGE_INVALID`, exit 2)
 - `--format <format>` - `yaml` (default) or `json`. The same document either way
 
-**Exit Codes:**
-- `0` - Always (scan is informational)
-- `2` - System error (file access, parsing error)
-
-**Output:** YAML on stdout (JSON with `--format json`), logs on stderr
-
-`root` is stated once and is the only absolute path in the document; every
-`path` beneath it is relative to it.
-
-`lane` names which enumerator produced the population — `walk` (the default
-crawl) or `projection` (`VAT_RESOURCES_CRAWL=projection`). Two scans of one tree
+**Exit Codes:** derived from the published document, never chosen beside it.
+- `0` - Scanned at least one file
+- `1` - Scanned nothing: one non-overridable `RESOURCE_CHECK_BROKEN` finding
+  (the path or config enumerates no markdown, or `--collection` names a
+  collection no file matched) — a scan of nothing is not a population
+- `2` - The scan could not run: `[path]` names no directory (`USAGE_INVALID`), a
+  directory the OS will not list (`INPUT_UNREADABLE`), an unusable config
+  (`CONFIG_INVALID`)
+
+**Output:** the shared report envelope on stdout — YAML, or JSON with
+`--format json` (schema: `packages/cli/schemas/resources-scan.json`); logs on
+stderr.
+
+- `examined` — files scanned. A scan reports no finding of its own.
+- `durationMs` — wall time of the run.
+- `data.root` is stated once and is the only absolute path in the document;
+  every `data.files[].path` is relative to it.
+- `data.collections` — resources per configured collection (`{}` when none).
+
+`data.lane` names which enumerator produced the population — `projection` (the
+default) or `walk` (`VAT_RESOURCES_CRAWL=walk`). Two scans of one tree
 that report different populations are only interpretable if each says which lane
 produced it, so the field is derived from the load that ran rather than read back
-from the environment.
+from the environment. `data.extentSource` names the projection lane's own
+enumerator (`git` or `filesystem`), or is `null` for the walk.
+
+The lab's population facet (`packages/lab`) reads this document: the file count
+from `examined`, the population from `data.files` (so it runs the scan with
+`--verbose`).
 
 **Example:**
 ```bash
@@ -56,18 +72,22 @@ vat resources scan docs/ --verbose
 
 # Output:
 # ---
-# status: success
-# root: /home/you/my-project
-# filesScanned: 12
-# linksFound: 47
-# anchorsFound: 23
-# durationSecs: 0.234
-# files:
-#   - path: docs/README.md
-#     links: 5
-#     anchors: 3
-#     checksum: 47dd7b50af765df240fe2514f029fc697c907fc37a3267e22060f2f9f611975c
-# ---
+# status: ok
+# examined: 12
+# findings: []
+# summary: { errors: 0, warnings: 0, info: 0 }
+# gate: { strict: false }
+# durationMs: 234
+# data:
+#   root: /home/you/my-project
+#   lane: projection
+#   extentSource: git
+#   collections: {}
+#   files:
+#     - path: docs/README.md
+#       links: 5
+#       anchors: 3
+#       checksum: 47dd7b50af765df240fe2514f029fc697c907fc37a3267e22060f2f9f611975c
 ```
 
 **Requirements:**
@@ -91,8 +111,8 @@ for the loud-cwd fallback policy and the projectRoot discovery ladder.
 **What it does:**
 1. Recursively scans for markdown resources
 2. Validates all links (internal, anchors, external if configured)
-3. Reports errors in dual format (YAML + test format)
-4. Exits 0 if valid, 1 if errors found
+3. Publishes every finding flat in the shared report envelope
+4. Exits 0 with no error-severity finding, 1 with one, 2 when it could not run
 
 **Path Argument:**
 - `[path]` restricts the scan to that subtree of the project
@@ -106,24 +126,46 @@ for the loud-cwd fallback policy and the projectRoot discovery ladder.
 **Options:**
 - `[path]` - Base directory to crawl (defaults to current directory)
 - `--debug` - Enable debug logging
-- `-v, --verbose` - Show all scanned resources, including those without issues. By
-  default `issues` carries one counts-only row per file with findings
-  (`{file, errors?, warnings?, info?, codes}`); `--verbose` replaces each row with
-  its per-issue detail. Every other output field is identical in both modes.
+- `-v, --verbose` - Show all scanned resources, including those without issues:
+  adds `data.files`, one `{path, status, summary}` row per resource validated.
+  Every envelope field is identical in both modes.
+- `--collection <id>` - Scope the whole report — findings, `examined` and so the
+  exit code — to one collection's files. A finding at a path that never became a
+  resource (a file the collection matched and could not read) stays in it. A name
+  `resources.collections` does not declare is refused (`USAGE_INVALID`, exit 2)
+- `--format <format>` - `yaml` (default), `json` (the same document), or `text`
+  (one `location:line: severity: message [code]` line per finding, then a status
+  line, on stdout). Any other value is a usage error (exit 2) naming the three
+  accepted values; so is a `--validation-mode` other than `strict` or `permissive`
 - `--no-check-frontmatter-links` - Skip frontmatter URI-reference link validation across all collections (default: enabled)
 
-**Exit Codes:**
-- `0` - Validation passed
-- `1` - Validation errors found — including a run that scanned no file at all
-  (`filesScanned: 0`), which is reported as one non-overridable
-  `RESOURCE_CHECK_BROKEN` error rather than as a pass: a `--collection` that
+**Exit Codes:** derived from the published document, never chosen beside it.
+- `0` - No error-severity finding (warnings and info never fail it)
+- `1` - At least one error-severity finding — including a run that validated no
+  resource at all (`examined: 0`), which carries one non-overridable
+  `RESOURCE_CHECK_BROKEN` finding rather than passing: a declared `--collection` that
   matched nothing or a path with no markdown is not a verdict about anything.
   The reason and the remedy are also printed on stderr as a warning
-- `2` - System error
-
-**Output:**
-- YAML on stdout (structured results)
-- Test-format errors on stderr (file:line:column: message)
+- `2` - The run could not finish (`status: error`, `error.code` says which):
+  `[path]` or `--frontmatter-schema` names nothing (`USAGE_INVALID`), an input
+  the OS will not read (`INPUT_UNREADABLE`), an unusable config (`CONFIG_INVALID`)
+
+**Output:** the shared report envelope on stdout (schema:
+`packages/cli/schemas/resources-validate.json`):
+- `examined` — resources validated
+- `findings` — every finding, flat: `{code, severity, message, location, line?,
+  link?, fix?, reference?}`, `location` relative to `data.root`
+- `summary` — `{errors, warnings, info}` over `findings`
+- `data.root` — the project root, the one base every `location` and `path` is
+  relative to
+- `data.collections` — per configured collection: `resourceCount`, `hasSchema`,
+  `validationMode?`, `filesWithErrors`, and `summary` over its files' findings
+- `data.population` — where the file list came from: `store` (the projection
+  store answered), `derived` (the projection was built this run, including with
+  `VAT_PROJECTION_STORE=0`, which skips only the store), or `null` when the walk
+  answered: the walk lane was selected (`VAT_RESOURCES_CRAWL=walk`), or the crawl
+  declined a projection bound to another root and walked
+- `data.files` — under `--verbose` only
 
 **Example (success):**
 ```bash
@@ -133,40 +175,47 @@ vat resources validate docs/
 
 # Output:
 # ---
-# status: success
-# filesScanned: 12
-# linksChecked: 47
-# durationSecs: 0.456
-# ---
+# status: ok
+# examined: 12
+# findings: []
+# summary: { errors: 0, warnings: 0, info: 0 }
+# gate: { strict: false }
+# durationMs: 456
+# data:
+#   root: /home/you/my-project
+#   collections: {}
 ```
 
 **Example (errors):**
 ```bash
 vat resources validate docs/
 
-# stderr:  (file:line:column: severity: message — only `error` fails the run)
-# docs/README.md:15:25: error: Link target not found: ./missing.md
-# docs/guide.md:42:10: error: Broken anchor: #non-existent-section
-
 # stdout:
 # ---
-# status: error
-# errorsFound: 2
-# filesWithErrors: 2
-# issueCounts: { errors: 2, warnings: 0, info: 0 }
-# issueSummary: { LINK_BROKEN_FILE: 1, LINK_BROKEN_ANCHOR: 1 }
-# issues:
-#   - file: docs/README.md
-#     issues:
-#       - line: 15
-#         column: 25
-#         code: LINK_BROKEN_FILE
-#         severity: error
-#         message: Link target not found: ./missing.md
-# durationSecs: 0.456
-# ---
+# status: findings
+# examined: 12
+# findings:
+#   - code: LINK_BROKEN_FILE
+#     severity: error
+#     message: "File not found: docs/missing.md"
+#     location: docs/README.md
+#     link: ./missing.md
+#     line: 15
+#   - code: LINK_BROKEN_ANCHOR
+#     severity: error
+#     message: "Anchor not found: #non-existent-section"
+#     location: docs/guide.md
+#     line: 42
+# summary: { errors: 2, warnings: 0, info: 0 }
+# gate: { strict: false }
+# durationMs: 456
+# data:
+#   root: /home/you/my-project
+#   collections: {}
 ```
 
+Findings in a `jq` pipeline: `vat resources validate --format json | jq '.findings[] | "\(.location):\(.line) \(.code)"'`.
+
 **Requirements:**
 
 - **`projectRoot`**: optional with **loud-cwd fallback**. With an explicit
@@ -191,6 +240,35 @@ See [Roots and Config — Canonical Concepts](../../../docs/concepts/roots-and-c
 for the projectRoot ladder, the loud-cwd fallback policy, and the rationale
 behind RFC-3986-compliant leading-`/` resolution.
 
+### vat resources query &lt;sql&gt; [path]
+
+**Purpose:** Ask this tree's resource projection one read-only SQL question.
+
+**Output:** the shared report envelope on stdout — YAML, or JSON with
+`--format json` (schema: `packages/cli/schemas/resources-query.json`):
+- `examined` — resources in the population the statement ran over (the tracked
+  tree), never the rows it selected
+- `data.columns` — the result columns, in order, even when no row was selected
+- `data.rows` — the selected rows, exactly as SQLite holds them
+- `data.population` (`derived` | `store`), `data.populationSecs`,
+  `data.lensSecs`, `data.lensesEvaluated` — where the population came from and
+  what it cost; `data.boundsStatement` / `data.limits` when a bounded lens ran
+- `data.lensPopulation` (`none` | `store` | `derived`) — the same tell for the
+  lenses the statement named: `none` when it named no lensed relation, otherwise
+  whether every lens population was a store hit
+
+**Exit Codes:** derived from the published document.
+- `0` - The statement ran over a populated tree — zero rows is an answer, `ok`
+- `1` - The population was empty (`RESOURCE_CHECK_BROKEN`)
+- `2` - The statement was refused (`USAGE_INVALID`: not a query, a second
+  statement, an unbound `?`, a name the projection lacks), `[path]` names no
+  directory, or the crawl failed
+
+**Example:**
+```bash
+vat resources query 'SELECT path FROM resource_realizations LIMIT 5' --format json | jq '.data.rows[].path'
+```
+
 ## Configuration
 
 Place `vibe-agent-toolkit.config.yaml` at project root:
@@ -226,18 +304,126 @@ resources:
 
 ## Integration with vibe-validate
 
-The test-format error output (stderr) integrates seamlessly with vibe-validate:
+The exit code is the gate — 1 exactly when an error-severity finding (or the
+nothing-validated refusal) is in the document — so a vibe-validate step needs
+nothing more than the command:
 
 ```yaml
 # vibe-validate.config.yaml
 validators:
   markdown:
     run: vat resources validate docs/
-    extract:
-      - type: test-format
 ```
 
+`--format text` prints one `location:line: severity: message [code]` line per
+finding on stdout, for an extractor that reads compiler-style lines.
+
 ## More Information
 
 - GitHub: https://github.com/jdutton/vibe-agent-toolkit
 - Issues: https://github.com/jdutton/vibe-agent-toolkit/issues
+
+## Example reports
+
+Each block below is a real document from the built CLI, trimmed where noted; `packages/cli/test/integration/tagged-report-examples.integration.test.ts` validates every `vat-report=<verb>` block against that verb's registered schema.
+
+### `resources scan`
+
+A scan of a project that declares no collections. Produced by `vat resources scan .`.
+
+```yaml vat-report=resources scan
+status: ok
+examined: 2
+findings: []
+summary:
+  errors: 0
+  warnings: 0
+  info: 0
+gate:
+  strict: false
+durationMs: 184
+data:
+  root: /work/project
+  lane: projection
+  extentSource: git
+  collections: {}
+```
+
+### `resources validate`
+
+The same project validated. Produced by `vat resources validate`.
+
+```yaml vat-report=resources validate
+status: ok
+examined: 2
+findings: []
+summary:
+  errors: 0
+  warnings: 0
+  info: 0
+gate:
+  strict: false
+durationMs: 82
+data:
+  root: /work/project
+  population: derived
+  collections: {}
+```
+
+### `resources query`
+
+One read-only SQL statement; the answer is under `data.rows`. Produced by `vat resources query "SELECT count(*) AS n FROM resources"`.
+
+```yaml vat-report=resources query
+status: ok
+examined: 16
+findings: []
+summary:
+  errors: 0
+  warnings: 0
+  info: 0
+gate:
+  strict: false
+durationMs: 225
+data:
+  root: /work/project
+  columns:
+    - n
+  rows:
+    - n: 16
+  population: derived
+  populationSecs: 0.221
+  lensSecs: 3.75e-7
+  lensesEvaluated: []
+  lensPopulation: none
+```
+
+### `resources check`
+
+The built-in checks, cut to the first. Produced by `vat resources check`.
+
+```yaml vat-report=resources check
+status: ok
+examined: 16
+findings: []
+summary:
+  errors: 0
+  warnings: 0
+  info: 0
+gate:
+  strict: false
+durationMs: 86
+data:
+  root: /work/project
+  population: store
+  populationSecs: 0.0819
+  lensSecs: 3.75e-7
+  lensesEvaluated: []
+  lensPopulation: none
+  checksRun: 3
+  checks:
+    - name: claude-rule-glob-inert
+      durationSecs: 0.0000399
+      rows: 0
+      builtin: true
+```

package/docs/skill-test.md

@@ -51,7 +51,17 @@ VAT reads liberally as adopter-authored data.)
 
 `vat skill test configure <skill>` writes the **per-skill** block using a
 comment-preserving YAML upsert, and validates values before writing. Prefer it over
-hand-editing.
+hand-editing. It publishes the report envelope on stdout (schema
+`packages/cli/schemas/skill-test-configure.json`): `status: ok`, `examined: 1`, and
+`data: { configPath, skill }`, `configPath` relative to the working directory. With
+`--print` it writes nothing to disk and stdout is the updated config text alone — no
+report — so `vat skill test configure my-skill --max-turns 20 --print > new.yaml`
+yields a usable file. A refusal exits `2` with `error.code`: `USAGE_INVALID` (an invalid
+knob value, a skill the config's `skills.include` does not discover — configure a skill
+after declaring it — or no project root), `CONFIG_INVALID` (no config file at the project root, or
+the edit would leave it failing its schema), `INPUT_UNREADABLE`, or `RUN_INCOMPLETE`
+(the write failed). An unknown key already in the config is a stderr warning, never a
+refusal.
 
 ## Per-skill knobs (`skills.config.<skill>.test`)
 
@@ -80,7 +90,15 @@ A **source descriptor** is one of `{ workspace: <pkg> }`, `{ npm: <spec> }`,
 command line the same sources are written as `name=workspace:<pkg>`,
 `name=npm:<spec>`, `name=url:<u>`, `name=path:<dir>`, or `name=vendored` — the CLI
 form requires an explicit companion `name=`, the config form derives it from the
-resolved skill.
+resolved skill. An npm spec must be version-pinned (`@scope/pkg@1.2.3`, with an
+optional `/subpath` — a file or directory inside the installed package, or a subpath
+its `exports` map declares; with none, the package's own directory is staged) and
+installed under the project root. A required npm source that names nothing (a package
+that is not installed, an unpinned spec, a subpath naming nothing in the package or
+climbing out of it) and a scoped specifier given as a `path:` are refused before
+anything is staged — `USAGE_INVALID` from the command line, `CONFIG_INVALID` from the
+config. A `path:` source naming a directory that does not exist is refused when it is
+staged, as `INPUT_UNREADABLE`, like any source the OS will not read.
 
 ### Companion staging and builds
 
@@ -154,39 +172,78 @@ The suite is never copied into anything the executor can reach. Each eval's decl
 | `--refresh` | Force a full re-stage, ignoring existing staged content. |
 | `--keep` | Keep the harness directory after the run (needed to inspect `results/`). |
 | `--dry-run` | Build and stage exactly as a real run would, then stop without spawning Claude — no session, no tokens. It **does build** (when `--i-understand-this-runs-skill-code` is passed), because the question a dry run answers is "what happens if I drop this flag", and a preview built from a stale `dist/` answers it wrongly. **Without** the acknowledgement it does not build — building runs the repo's `test.build` hook, an arbitrary shell command — so it falls back to an existing `dist/` and warns it may be stale. `--no-build` skips the build either way. |
-| `--out <dir>` / `--workdir <dir>` | Override the harness output / working directory. |
-| `--allow-eval-failure` | Opt out of fail-closed: exit `0` even when an eval fails. For interactive iteration. |
+| `--out <dir>` / `--workdir <dir>` | Override the harness output / working directory. An existing `--out` must already be a directory and, on POSIX, `0700`: VAT creates a new one `0700`, and never changes the mode of one you made (Windows has no mode check). |
+| `--allow-eval-failure` | Opt out of fail-closed: each failed eval is published as a `warning` finding instead of an `error`, so the run exits `0`. For interactive iteration. |
 | `--allow-unverified-skill-source` | Skip the vendored manifest integrity check. |
 | `--debug` | Enable debug logging. |
 
-## Exit codes
+## The report
+
+`vat skill test run` publishes the report envelope on stdout (schema
+`packages/cli/schemas/skill-test-run.json`), and nothing else — the `Summary:` line that
+used to be stdout's one machine-readable channel is gone from it. The human verdict line
+stays on stderr as `Summary: <line>`, beside `Harness:`, `Results:`, `Workspaces:` and
+`Reason:`.
+
+- `examined` — the evals the run graded; on `--dry-run`, which grades nothing, the evals
+  it staged.
+- `findings` — one `SKILL_TEST_EVAL_FAILED` per eval that ran and did not pass (an output
+  expectation, or its tool verdict): `location` the suite's `evals.json` relative to the
+  project root (omitted when it lies outside the project), `field` the eval's `id`. `error` by default,
+  `warning` under `--allow-eval-failure`. Evals a fail-fast tier gate skipped are not
+  listed: they never ran, and the failure that fired the gate is.
+- `data.skill` — the reference as passed; `data.description` — the verdict line
+  (`PASS 3/3`, `FAIL 1/2 (1 tool)`, the dry-run preview); `data.evals` — `{ id, passed }`
+  per graded eval; `data.artifacts.frictionReport` — the `friction.json` written, or `null`
+  on a dry run; `data.artifacts.outputDir` — the harness root (`--out`), of which a
+  default run keeps only `results/`.
 
-The same three-way contract as every other `vat` command, so a CI consumer can
-tolerate eval failures while failing closed on a harness that could not run.
+## Exit codes
 
-- `0` - Run completed, all expectations passed (or `--allow-eval-failure` suppressed a failing verdict)
-- `1` - **Eval failure** — the run completed and produced a valid `grading.json`; expectations did not all pass
-- `2` - The harness could not run. A `Reason: <reason>` line on stderr says why:
-  - `internal` — the harness broke (executor/grader crash, stall, timeout, grader nonce failure)
-  - `preflight` — the environment or inputs need fixing (bad config, unresolvable required companion, auth guard, missing security ack, `build` hook failed)
-  - `bootstrap` — `evals.json` was absent, so VAT wrote a starter template next to the skill source; fill it in and re-run
+The same three-way contract as every other `vat` command, derived from the published
+document, so a CI consumer can tolerate eval failures while failing closed on a harness
+that could not run.
+
+- `0` - Every eval passed (or `--allow-eval-failure` published the failures as warnings)
+- `1` - **Eval failure** — the run completed and an `error` finding names each eval that did not pass
+- `2` - The harness could not run: `status: error`, and `error.code` says which refusal.
+  A `Reason: <reason>` line on stderr restates it for a CI log — `internal` exactly for
+  `INTERNAL_ERROR`, `bootstrap` for a scaffolded `evals.json`, `preflight` for the rest.
+  Each code is decided where the cause was seen, never read back from a message. A skill
+  build that threw is classified by what it threw: a coded cause keeps its own code (a
+  directory the OS will not list is `INPUT_UNREADABLE` from a build exactly as from
+  resolution), and an uncoded one is `INTERNAL_ERROR`:
+
+| `error.code` | `Reason:` | When |
+|---|---|---|
+| `BACKEND_UNAVAILABLE` | `preflight` | No `claude` binary on `PATH`, or one too old for a flag the spawn needs |
+| `USAGE_INVALID` | `preflight` | An invalid flag value, an auth guard the credentials do not meet, the missing security ack, an unsafe `--workdir`, an `--out` that exists and is not a directory, or (POSIX only — Windows has no mode check) is not `0700` (VAT never changes its mode — `chmod 700` it, or name one that does not exist yet), a held harness lock, a skill name the config does not declare (or `--no-build` with no dist), a bad `env` token, a failing `test.build` hook, a repeated staged name, a `--with` source, `--evals` or skill reference that names nothing (an npm package that is not installed, an npm spec with no version pin or a subpath naming nothing in the package, a scoped specifier given as a path) |
+| `CONFIG_INVALID` | `preflight` | The governing `vibe-agent-toolkit.config.yaml` does not parse or fails its schema (subject's or a companion's), or its `test.with` or `test.evals` names nothing (the same cases as the flags). An optional companion (`--with-optional`, `test.optional`) that names nothing is skipped with a warning instead |
+| `INPUT_UNREADABLE` | `preflight` | A declared eval input or dependency is absent, the `evals.json` is not a valid suite, the vendored copy fails its manifest, a config or directory the OS will not read, a `--with name=path:<dir>` companion that does not exist, an npm package that is installed but whose `package.json` Node cannot read, or a source holding a file or directory the OS will not read or a symlink (named) |
+| `INPUT_UNREADABLE` | `bootstrap` | `evals.json` was absent, so VAT wrote a starter template next to the skill source; fill it in and re-run |
+| `RUN_INCOMPLETE` | `preflight` | The packager refused the subject's (or a required companion's) own content — a `files:` source absent, a `SKILL.md` bundled as a resource. A `SKILL_PACKAGING_FAILED` finding at the skill's `SKILL.md` says what to change. Also, with no finding: an output the OS will not let the run write — the harness root (`--out` under a read-only directory), its lockfile, the staged skill copies and manifest, the `results/` files, a dist bundle — a full disk or a read-only directory. Known gap: a disk so full that a skill build's git snapshot of the project fails first is still `INTERNAL_ERROR` |
+| `INTERNAL_ERROR` | `internal` | The harness broke (executor/grader crash, stall, timeout, grader nonce or skew failure); the stack is on stderr |
 
 ```bash
 vat skill test run my-skill --i-understand-this-runs-skill-code
 case $? in
   0) ;;
   1) echo "evals failed (tolerated)" ;;
-  *) exit 1 ;;    # harness could not run — fail the build; read Reason: on stderr
+  *) exit 1 ;;    # harness could not run — fail the build; read error.code
 esac
 ```
 
 ## Results
 
-With `--keep`, the harness directory retains a `results/` tree that VAT is the sole
-writer of:
+The harness root keeps a `results/` tree (named on stderr as `Results:`) that VAT is
+the sole writer of — the harness writes it, not the CLI, and validates each file against
+its schema before the run reports:
 
 - `grading.json` - per-expectation verdicts and the pass/total summary
-- `friction.json` - packaging issues observed during the run (advisory)
+- `friction.json` - packaging issues observed during the run (advisory). Each item's
+  `severity` is `error|warning|info`, the vocabulary every VAT report uses. The grader is
+  still asked for `high|medium|low`; VAT maps it (`high`->`error`, `medium`->`warning`,
+  `low`->`info`) once, when it parses the grader's fragment.
 - `tool-eval.json` - tool-expectation verdicts; always written, so check
   `.evals.length` rather than file existence
 
@@ -204,11 +261,83 @@ vat skill test run my-skill --model claude-opus-5 --grader-model claude-sonnet-5
 vat skill test run router-skill --with helper=path:./skills/helper \
   --i-understand-this-runs-skill-code
 
-# Persist knobs instead of passing them every time
-vat skill test configure my-skill --auth subscription --require-auth subscription
+# Persist knobs instead of passing them every time (`requireAuth` has no
+# configure flag: set it under the skill's `test:` block by hand, or pass
+# --require-auth to each `run`)
+vat skill test configure my-skill --auth subscription
 ```
 
 ## See Also
 
 - [skills.md](./skills.md) - `vat skills` (plural): packaging, validation, install
 - [index.md](./index.md) - full CLI command index
+
+## Example reports
+
+Each block below is a real document from the built CLI, trimmed where noted; `packages/cli/test/integration/tagged-report-examples.integration.test.ts` validates every `vat-report=<verb>` block against that verb's registered schema.
+
+### `skill test configure`
+
+The test block written to the config. Produced by `vat skill test configure test-skill-2 --auth inherit`.
+
+```yaml vat-report=skill test configure
+status: ok
+examined: 1
+findings: []
+summary:
+  errors: 0
+  warnings: 0
+  info: 0
+gate:
+  strict: false
+data:
+  configPath: vibe-agent-toolkit.config.yaml
+  skill: test-skill-2
+```
+
+### `skill test run`
+
+A dry run: what a real run would do, with nothing spawned and no eval graded. Produced by `vat skill test run ./test-skill-1 --dry-run --i-understand-this-runs-skill-code --out ./out`; the absolute paths are shortened.
+
+```yaml vat-report=skill test run
+status: ok
+examined: 1
+findings: []
+summary:
+  errors: 0
+  warnings: 0
+  info: 0
+gate:
+  strict: false
+data:
+  skill: ./test-skill-1
+  description: |-
+    [dry-run] A real run would: stage the source dir as-is, then spawn claude.
+    [dry-run] Would run 1 executor→grader spawn pair at concurrency 4 — 2 claude sessions in total.
+    [dry-run] --max-budget-usd is PER SPAWN ($5), not per run: worst case ≈ $10.00 across those 2 sessions.
+    [dry-run] Executor (no --model; claude default); grader model claude-sonnet-5 (prompt via stdin).
+    [dry-run] Staged manifest: 1 entry | fingerprint: 33dee44374157bdc92af00c90fc5de39d2a3d974795a707a73c575ecf6488b4d
+    [dry-run] Provenance would be written to: /home/me/project/out/results/provenance.json
+  evals: []
+  artifacts:
+    frictionReport: null
+    outputDir: /home/me/project/out
+```
+
+Refused without the security acknowledgment: an `error` document, exit `2`. Produced by `vat skill test run test-skill-1`.
+
+```yaml vat-report=skill test run
+status: error
+examined: 0
+findings: []
+summary:
+  errors: 0
+  warnings: 0
+  info: 0
+gate:
+  strict: false
+error:
+  code: USAGE_INVALID
+  message: Security acknowledgment required. Pass --i-understand-this-runs-skill-code to proceed.
+data: null
+```