@dogfood-lab/verify 1.9.0 → 1.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -109,21 +109,23 @@ The verifier emits rejection-reason strings under stable prefixes, each mapping
109
109
 
110
110
  Discrimination happens by **class**, surfaced by `parseRejectionReason` (below). Every prefix maps to one of four classes: **submission-bad** (the submitter fixes the payload), **operational** (the verifier/tooling faulted), **ingest** (an ingest-side load fault), or **unknown** (unrecognized prefix).
111
111
 
112
+ **Retryable** (F-f8952a50, wave 10) is a SEPARATE, narrower per-prefix flag, orthogonal to `class`: may a same-`run_id` resubmission whose ONLY prior rejection carries this prefix still reach an acceptance, once corrected? `packages/ingest/persist.js`'s duplicate guard (`isDuplicate` → `isRetryableRejection`) reads this flag — never `class` or `prefix` directly — to decide whether a stale `_rejected` record blocks a same-`run_id` retry that is now headed to acceptance. Every `operational` / `ingest` / `unknown` prefix is `retryable: false` (an ops fault or an unrecognized signal is never the submitter's to retry past). Within `submission-bad`, the taxonomy splits further — **shape/addressing** prefixes ("we could not even read/place/shape your submission") are retryable; **content-verdict** prefixes ("we read your submission and rendered a verdict against its own reported content") are not, so a submitter cannot launder a genuinely-bad run into an accepted one by resubmitting different self-reported content under the same `run_id`. The table below marks each `submission-bad` prefix's retryable value explicitly.
113
+
112
114
  **Submission-bad** — `class: 'submission-bad'` (the submitter's payload failed a validator gate; fix the submission and resubmit):
113
115
 
114
- | Prefix | Source | Meaning |
115
- |---|---|---|
116
- | `schema:` | `validators/schema.js` | JSON Schema check on the submission/record envelope failed. The rest of the string carries the AJV path + message. |
117
- | `policy:` | `validators/policy.js` | Per-repo policy gate failed (forbidden tags, missing required fields, surface evidence/CI requirements, or a declarative `when`/`custom_rules` predicate matched — see the [policy DSL](https://dogfood-lab.github.io/testing-os/handbook/policy-dsl/)). |
118
- | `policy-config:` | `validators/policy.js` | **VERIFY-F1.** A REPO custom-rule predicate hit an eval-time semantic fault the schema could not catch — an unknown leading field, a numeric operator against a non-number, or a depth/width/fan-out budget. The repo authored the bad rule, so the fix is the submitter's. (A malformed GLOBAL predicate is `VALIDATOR_FAULT_POLICY:` operational instead — see below.) |
119
- | `steps[<id>]:` | `validators/steps.js` | Step-level contract check failed on a specific step id (gate accumulation, ordering, evidence shape). |
120
- | `provenance:` | `validators/provenance.js` | The run was genuinely **absent / not confirmable** — a 404 from the provider API, or the run head did not match the submitted commit/repo. The submitter's payload points at a run that does not exist or does not bind. (Operational provider faults — 429/5xx/401/403 — are NOT this class; see `provenance-fault:` below.) |
121
- | `repo:` | `index.js` cross-field guard | `submission.repo` does not match the owner/repo encoded in `source.run_url` (anti-forgery guard). Emitted as `repo:mismatch: …`. |
122
- | `submission-contains-verifier-field:` | `index.js` | The submission carried a verifier-owned field (`policy_version`, `verification`, or an object `overall_verdict`) it must not author. |
123
- | `CONTRACT_SCHEMA_TOO_NEW:` | `validators/schema-version.js` | The submission's `schema_version` declares a MAJOR **above** what this build supports (see `SUPPORTED_SCHEMA_VERSIONS` in `@dogfood-lab/schemas`). This build cannot understand a future contract **the operator must upgrade testing-os**, but the routing class stays submission-bad (the payload as-shipped cannot be accepted by THIS build). |
124
- | `CONTRACT_SCHEMA_TOO_OLD:` | `validators/schema-version.js` | The submission's `schema_version` declares a MAJOR **below** the supported floor. **The submitter must re-emit** against the current contract. A patch/minor delta inside the supported major range is NOT rejected. |
125
-
126
- **Operational** — `class: 'operational'` (the validator itself threw an internal error; investigate the verifier, do NOT bounce to the submitter):
116
+ | Prefix | Retryable | Source | Meaning |
117
+ |---|---|---|---|
118
+ | `schema:` | Yes — shape | `validators/schema.js` | JSON Schema check on the submission/record envelope failed. The rest of the string carries the AJV path + message. |
119
+ | `policy:` | **No — content verdict** | `validators/policy.js` | Per-repo policy gate failed (forbidden tags, missing required fields, surface evidence/CI requirements, or a declarative `when`/`custom_rules` predicate matched — see the [policy DSL](https://dogfood-lab.github.io/testing-os/handbook/policy-dsl/)). Judges the run's OWN reported content (tags, evidence, scenario results); consuming the run_id is the deliberate anti-gaming behavior. |
120
+ | `policy-config:` | Yes — shape | `validators/policy.js` | **VERIFY-F1.** A REPO custom-rule predicate hit an eval-time semantic fault the schema could not catch — an unknown leading field, a numeric operator against a non-number, or a depth/width/fan-out budget. The repo authored the bad RULE (config), not the run's content, so the fix belongs to the submitter and is retryable. (A malformed GLOBAL predicate is `VALIDATOR_FAULT_POLICY:` operational instead — see below.) |
121
+ | `steps[<id>]:` | Yes — shape | `validators/steps.js` | Step-level contract check failed on a specific step id (gate accumulation, ordering, evidence shape — a completeness/structure mismatch, not a verdict on whether the steps passed). |
122
+ | `provenance:` | **No — content verdict** | `validators/provenance.js` | The run was genuinely **absent / not confirmable** — a 404 from the provider API, or the run head did not match the submitted commit/repo. The provider could not confirm THIS specific run happened as claimed; resubmitting different self-reported content under the same run_id to "become confirmable" is exactly the laundering the anti-gaming doctrine blocks. (Operational provider faults — 429/5xx/401/403 — are NOT this class; see `provenance-fault:` below.) |
123
+ | `repo:` | Yes — shape | `index.js` cross-field guard | `submission.repo` does not match the owner/repo encoded in `source.run_url` (anti-forgery guard). Emitted as `repo:mismatch: …`. Pure identity/addressing — the run happened, only the repo/run_url pairing was mis-stated. |
124
+ | `submission-contains-verifier-field:` | Yes — shape | `index.js` | The submission carried a verifier-owned field (`policy_version`, `verification`, or an object `overall_verdict`) it must not author. |
125
+ | `CONTRACT_SCHEMA_TOO_OLD:` | Yes — shape | `validators/schema-version.js` | The submission's `schema_version` declares a MAJOR **below** the supported floor. **The submitter must re-emit** against the current contract. A patch/minor delta inside the supported major range is NOT rejected. (NOT symmetric with `CONTRACT_SCHEMA_TOO_NEW:` below see that row for why.) |
126
+ | `unsafe-record-path:` | Yes — shape | `packages/ingest/run.js`, `writeRecord()` catch | The record passed schema validation but `computeRecordPath()`'s traversal guard (`isUnsafeSegment`, stricter than the schema's `repo` pattern e.g. `../etc`) still refused to place it on disk. The submitter's own `repo` string is the problem; nothing is persisted (there is no safe path to write to). |
127
+
128
+ **Operational** — `class: 'operational'`, always `retryable: false` (the validator itself threw an internal error; investigate the verifier, do NOT bounce to the submitter):
127
129
 
128
130
  | Prefix | Source | Meaning |
129
131
  |---|---|---|
@@ -131,13 +133,14 @@ Discrimination happens by **class**, surfaced by `parseRejectionReason` (below).
131
133
  | `VALIDATOR_FAULT_POLICY:` | `runValidator('policy', …)` catch | Internal exception inside the policy validator — including a **GLOBAL** declarative-rule predicate fault (VERIFY-F1): a broken `policies/global-policy.yaml` is an ops incident (the studio's own config), so its predicate fault throws here rather than bouncing to the submitter. The repo-authored counterpart is `policy-config:` submission-bad. |
132
134
  | `VALIDATOR_FAULT_STEPS:` | `runValidator('steps', …)` catch | Internal exception inside the steps validator. |
133
135
  | `VALIDATOR_FAULT_CONTRACT_SCHEMA_VERSION:` | `runValidator('contract_schema_version', …)` catch | The version gate was called with an unknown contract key (a programmer error at the call site, not a submission fault). |
136
+ | `CONTRACT_SCHEMA_TOO_NEW:` | `validators/schema-version.js` | **F-be0deacd (wave 20).** The submission's `schema_version` declares a MAJOR **above** what this build supports (see `SUPPORTED_SCHEMA_VERSIONS` in `@dogfood-lab/schemas`) — THIS BUILD is behind a schema major its own submitters have already adopted. No resubmission, corrected or not, can ever satisfy a `major > maxMajor` comparison until testing-os itself ships an upgrade — **the operator must upgrade testing-os**. Page ops; do NOT bounce it back to the submitter. Unlike every other row in this table, `validators/schema-version.js` RETURNS this as an ordinary rejection string rather than throwing, so (unlike the `VALIDATOR_FAULT_*`/`provenance-fault:`/`scenario-fetch-fault:` rows) it genuinely persists to `records/_rejected/` and is reachable by `packages/ingest/persist.js`'s `isRetryableRejection()` — **F-51780da9 (wave 22):** rather than trusting the frozen `retryable: false` this prefix's own classification carries, `isRetryableRejection()` re-derives retryability for this ONE prefix against the CURRENT build's `SUPPORTED_SCHEMA_VERSIONS` ceiling, so a stale TOO_NEW rejection unblocks once the operator upgrades testing-os past the declared major. Distinct from the submission-bad `CONTRACT_SCHEMA_TOO_OLD:` above — the two prefixes are asymmetric despite the shared emitter. |
134
137
  | `submission-malformed:` | `index.js` null/non-object early-return | The submission itself was `null` or not an object — a malfunctioning **dispatcher** sent garbage, not a submitter who authored a bad-but-shaped payload. Page ops / inspect the dispatch pipeline; do NOT bounce it to a submitter. |
135
138
  | `provenance-fault:` | `index.js` provenance catch | The provenance adapter THREW an operational error confirming the run — a provider **429 rate-limit, 5xx outage, or 401/403 token** fault (`validators/provenance.js` throws these on purpose for non-404 responses). The submitter's payload is fine; the verifier could not reach a verdict. Page ops / retry; do NOT bounce it to a submitter. Distinct from the submission-bad `provenance:` (genuine absence/404). |
136
139
  | `scenario-fetch-fault:` | `packages/ingest/load-context.js` | The scenario fetcher THREW after exhausting its retry budget (**5xx/429 outage, transport reject**) or hit a **401/403 credential** fault loading a scenario definition. The submission may be perfectly good — the fetch infrastructure faulted. The ingest CLI lets this propagate (exit 2, nothing persisted); a true missing file is the ingest-class `scenario-load: … (reason: not_found)` instead. |
137
140
 
138
141
  Any future `VALIDATOR_FAULT_<NEW>:` prefix is classified `operational` by family — `parseRejectionReason` matches the `VALIDATOR_FAULT_` head, so a new validator class needs no parser edit. The `submission-malformed:` prefix is matched literally (it is not part of the `VALIDATOR_FAULT_` family).
139
142
 
140
- **Ingest** — `class: 'ingest'` (an ingest-side load fault, not a verifier gate):
143
+ **Ingest** — `class: 'ingest'`, always `retryable: false` (an ingest-side load fault, not a verifier gate):
141
144
 
142
145
  | Prefix | Source | Meaning |
143
146
  |---|---|---|
@@ -145,13 +148,13 @@ Any future `VALIDATOR_FAULT_<NEW>:` prefix is classified `operational` by family
145
148
 
146
149
  ### Operator hygiene
147
150
 
148
- Discriminate by **class**, not by hand-rolled `.startsWith()` chains. `parseRejectionReason(reason)` returns `{ class, prefix, detail }`:
151
+ Discriminate by **class**, not by hand-rolled `.startsWith()` chains. `parseRejectionReason(reason)` returns `{ class, prefix, detail, retryable }`:
149
152
 
150
153
  ```js
151
154
  import { parseRejectionReason } from '@dogfood-lab/verify';
152
155
 
153
156
  for (const r of result.rejection_reasons) {
154
- const { class: cls, prefix, detail } =
157
+ const { class: cls, prefix, detail, retryable } =
155
158
  parseRejectionReason(r);
156
159
  switch (cls) {
157
160
  case 'operational':
package/cli-lint.js CHANGED
@@ -1,5 +1,5 @@
1
1
  /**
2
- * cli-lint.js (VERIFY-F3) — the `dogfood-verify lint <file>` subcommand.
2
+ * cli-lint.js (VERIFY-F3 + F-49940082) — the `dogfood-verify lint <file>` subcommand.
3
3
  *
4
4
  * A SEPARATE parse/render path from the verify CLI (cli.js): it takes a policy YAML or
5
5
  * (with --scenario) a scenario YAML — not a submission JSON — and reports static lint
@@ -9,7 +9,8 @@
9
9
  *
10
10
  * 0 — clean, or warnings-only (footgun advisories never block).
11
11
  * 1 — one or more errors (schema-invalid, a static fault, or unparseable YAML).
12
- * 2 — operator error (file missing/unreadable, or a malformed invocation).
12
+ * 2 — operator error (file missing/unreadable, a malformed invocation, or a batch
13
+ * whose positionals resolved to zero lintable files).
13
14
  *
14
15
  * YAML that fails to parse is exit 1 (a lint FINDING about the file the author must fix —
15
16
  * surfacing "line 4: bad indentation" is the lint's job), not exit 2. A file that does not
@@ -19,11 +20,21 @@
19
20
  * - default (policy): lintPolicy(doc, { origin }) → origin global|repo|unknown
20
21
  * - --scenario: lintScenario(doc, { file }) → origin 'scenario'
21
22
  * Both return the same { ok, origin, errors, warnings, coverageNote } shape, so
22
- * renderLintText / buildLintJson are reused unchanged.
23
+ * renderLintText / buildLintJson are reused unchanged, single-file or batch.
24
+ *
25
+ * Directory-or-multi-path mode (F-49940082): one or more positionals are accepted. A
26
+ * positional that is a directory is walked recursively for YAML files (extension match,
27
+ * mirroring the discovery semantics of scripts/lint-policies.test.mjs's own policyFiles()
28
+ * walker — see walkYamlFiles below). A positional that is a file is linted directly, no
29
+ * extension filter (an explicit path is trusted as authored intent; the extension filter
30
+ * only exists to skip non-policy noise during an unattended directory walk). The exact
31
+ * single-positional-file invocation shape is untouched: it still runs through the very
32
+ * same statements it always has, so its output is byte-identical to before this mode
33
+ * existed — see the "legacy single-file path" branch in runLint.
23
34
  */
24
35
 
25
- import { readFileSync } from 'node:fs';
26
- import { resolve } from 'node:path';
36
+ import { readFileSync, readdirSync, statSync } from 'node:fs';
37
+ import { resolve, join } from 'node:path';
27
38
  import yaml from 'js-yaml';
28
39
 
29
40
  import { lintPolicy, COVERAGE_NOTE } from './validators/lint-policy.js';
@@ -40,11 +51,23 @@ class LintOperatorError extends Error {
40
51
 
41
52
  const LINT_USAGE = `dogfood-verify lint — author-time static check for a policy or scenario file
42
53
 
43
- USAGE:
54
+ Usage:
44
55
  dogfood-verify lint <policy-file> [--json]
56
+ dogfood-verify lint <path> [<path> ...] [--json]
45
57
  dogfood-verify lint --scenario <scenario-file> [--json]
58
+ dogfood-verify lint --scenario <path> [<path> ...] [--json]
59
+
60
+ Directory-or-multi-path mode:
61
+ Give more than one positional, or point at a directory, and every path is linted as
62
+ a batch. A directory positional is walked recursively for *.yaml and *.yml files; a
63
+ file positional is linted directly regardless of extension. Results print one block
64
+ per file (same format as single-file mode) followed by a batch summary — files
65
+ linted / clean / with-errors / with-warnings. The batch exit code is 1 if any file
66
+ has an error, 0 if every file is clean or warnings-only — the single-file contract,
67
+ extended across the set. A directory (or set of directories) with nothing to lint is
68
+ an operator error (exit 2), never a silent pass.
46
69
 
47
- WHAT IT CHECKS — policy mode (default, no submission needed):
70
+ What it checks — policy mode (default, no submission needed):
48
71
  - structural validity against policy.schema.json
49
72
  - every predicate's known leading field, combinator depth, and node budget
50
73
  - an ADVISORY warning on the [] footgun (a negative op over a [] path fails open)
@@ -52,7 +75,7 @@ WHAT IT CHECKS — policy mode (default, no submission needed):
52
75
  It CANNOT statically catch a type_mismatch or a fanout_budget overrun — those depend
53
76
  on submission data. Run \`dogfood-verify --file <submission> --explain\` for that.
54
77
 
55
- WHAT IT CHECKS — --scenario mode (no submission needed):
78
+ What it checks — --scenario mode (no submission needed):
56
79
  - structural validity against scenario.schema.json
57
80
  - every success_criteria.required_steps entry references a declared steps[].id
58
81
  - step ids are unique
@@ -63,27 +86,33 @@ WHAT IT CHECKS — --scenario mode (no submission needed):
63
86
  It CANNOT verify that a real submission's step_results satisfy required_steps, nor
64
87
  that the receiver can fetch the file at the attested commit — run a real ingest.
65
88
 
66
- OPTIONS:
67
- --scenario Lint the file as a scenario definition (default: policy).
68
- --json Machine-readable result for CI.
89
+ Options:
90
+ --scenario Lint the file(s) as scenario definitions (default: policy).
91
+ --json Machine-readable result for CI. A single-file invocation emits one
92
+ result object, unchanged. Multi-path/directory mode emits
93
+ { mode: "batch", ok, summary, files: [ <one result object per file> ] }.
69
94
  -h, --help Show this help.
70
95
 
71
- EXIT CODES:
72
- 0 clean or warnings-only 1 errors found 2 operator error (bad flags / IO)`;
96
+ Exit codes:
97
+ 0 clean or warnings-only 1 errors found 2 operator error (bad flags / IO / nothing to lint)`;
73
98
 
74
99
  /**
75
- * Parse the lint argv (everything AFTER the `lint` verb). Accepts exactly one positional
76
- * file path plus optional `--scenario` / `--json` / `--help`. Throws LintOperatorError
77
- * (→ exit 2) on any malformed invocation.
100
+ * Parse the lint argv (everything AFTER the `lint` verb). Accepts one or more positional
101
+ * paths (files and/or directories) plus optional `--scenario` / `--json` / `--help`.
102
+ * Throws LintOperatorError (→ exit 2) on any malformed invocation.
78
103
  *
79
104
  * `--scenario` is a boolean MODE flag (default: policy mode). It selects which linter runs
80
- * over the one positional file; it never consumes the path itself.
105
+ * over every positional; it never consumes a path itself.
106
+ *
107
+ * Whether a given invocation is "single-file" (legacy, byte-identical output) or "batch"
108
+ * (F-49940082) is NOT decided here — this just collects positionals. runLint makes that
109
+ * call once it knows whether the sole positional is a directory.
81
110
  *
82
111
  * @param {string[]} argv
83
- * @returns {{ help: boolean, file: string|null, json: boolean, scenario: boolean }}
112
+ * @returns {{ help: boolean, paths: string[], json: boolean, scenario: boolean }}
84
113
  */
85
114
  export function parseLintArgs(argv) {
86
- let file = null;
115
+ const paths = [];
87
116
  let json = false;
88
117
  let help = false;
89
118
  let scenario = false;
@@ -95,18 +124,15 @@ export function parseLintArgs(argv) {
95
124
  if (arg.startsWith('-')) {
96
125
  throw new LintOperatorError(`unknown argument: ${arg}`, 'run `dogfood-verify lint --help` for usage');
97
126
  }
98
- if (file !== null) {
99
- throw new LintOperatorError('more than one file given', 'lint one file at a time');
100
- }
101
- file = arg;
127
+ paths.push(arg);
102
128
  }
103
129
 
104
- if (help) return { help: true, file: null, json: false, scenario: false };
105
- if (file === null) {
106
- const usage = scenario ? 'dogfood-verify lint --scenario <scenario-file>' : 'dogfood-verify lint <policy-file>';
107
- throw new LintOperatorError(`no ${scenario ? 'scenario' : 'policy'} file provided`, usage);
130
+ if (help) return { help: true, paths: [], json: false, scenario: false };
131
+ if (paths.length === 0) {
132
+ const usage = scenario ? 'dogfood-verify lint --scenario <scenario-file|dir> [...]' : 'dogfood-verify lint <policy-file|dir> [...]';
133
+ throw new LintOperatorError(`no ${scenario ? 'scenario' : 'policy'} file or directory provided`, usage);
108
134
  }
109
- return { help: false, file, json, scenario };
135
+ return { help: false, paths, json, scenario };
110
136
  }
111
137
 
112
138
  /**
@@ -121,6 +147,35 @@ export function originForPath(p) {
121
147
  return 'unknown';
122
148
  }
123
149
 
150
+ /**
151
+ * Recursively collect every YAML file (extension .yaml or .yml) under `dir`, depth-first.
152
+ * This is the F-49940082 directory-walk primitive for batch mode, deliberately mirroring
153
+ * policyFiles() in scripts/lint-policies.test.mjs — the CI gate that has quietly re-walked
154
+ * `policies/` on its own since VERIFY-F3 rather than exercising this CLI (that gap is the
155
+ * whole reason F-49940082 exists). Same two rules, on purpose: no skip-list (a hidden
156
+ * directory or a stray `node_modules` is walked like any other — the intended input here is
157
+ * an author-controlled tree such as `policies/`, not an arbitrary large directory), and
158
+ * symlinks are followed (`statSync`, not `lstatSync`).
159
+ *
160
+ * One deliberate divergence: this returns results in this function's own recursion order,
161
+ * which the caller (runLint) sorts across every discovered file from every positional —
162
+ * policyFiles() returns readdirSync's raw per-directory order because its only consumer is
163
+ * a test that aggregates failures into an unordered list; a human-facing CLI report reads
164
+ * better, and diffs better across runs, sorted.
165
+ *
166
+ * @param {string} dir - an already-resolved, existing directory path
167
+ * @returns {string[]} absolute file paths, unsorted (caller sorts across the full batch)
168
+ */
169
+ export function walkYamlFiles(dir) {
170
+ const out = [];
171
+ for (const entry of readdirSync(dir)) {
172
+ const full = join(dir, entry);
173
+ if (statSync(full).isDirectory()) out.push(...walkYamlFiles(full));
174
+ else if (entry.endsWith('.yaml') || entry.endsWith('.yml')) out.push(full);
175
+ }
176
+ return out;
177
+ }
178
+
124
179
  /** Render the human (default) view of a lint result — verdict-first, ERROR before WARNING. */
125
180
  export function renderLintText(result, file) {
126
181
  const lines = [];
@@ -172,42 +227,51 @@ export function buildLintJson(result, file) {
172
227
  };
173
228
  }
174
229
 
230
+ /** Render the batch summary footer printed after every per-file block in
231
+ * directory-or-multi-path mode (F-49940082). */
232
+ export function renderBatchSummary(summary) {
233
+ const { total, clean, withErrors, withWarnings } = summary;
234
+ return [
235
+ '-'.repeat(60),
236
+ `BATCH SUMMARY: ${total} file(s) linted -- ${clean} clean, ${withWarnings} with warnings, ${withErrors} with errors`,
237
+ ].join('\n');
238
+ }
239
+
175
240
  /**
176
- * Run the lint subcommand. Returns the exit code (does not call process.exit) so it is
177
- * unit-testable with an injected stdout/stderr sink.
178
- *
179
- * @param {string[]} argv - args AFTER the `lint` verb
180
- * @param {{ stdout?: (s: string) => void, stderr?: (s: string) => void }} [io]
181
- * @returns {Promise<number>}
241
+ * Build the machine-readable (--json) result for batch mode (F-49940082). Each entry in
242
+ * `files` has the exact shape buildLintJson produces for a single file, so a consumer
243
+ * written against the single-file --json shape only needs to index into `.files[i]`
244
+ * instead of the top-level object. There is no pre-existing batch JSON consumer to match
245
+ * instead: scripts/lint-policies.test.mjs (the only other batch-shaped lint caller in this
246
+ * repo) bypasses the CLI's JSON output entirely and calls lintPolicy() directly — that
247
+ * bypass is F-49940082's own subject, not a shape for this to imitate.
182
248
  */
183
- export async function runLint(argv, io = {}) {
184
- const out = io.stdout ?? ((s) => process.stdout.write(s + '\n'));
185
- const err = io.stderr ?? ((s) => process.stderr.write(s + '\n'));
186
-
187
- let opts;
188
- try {
189
- opts = parseLintArgs(argv);
190
- } catch (e) {
191
- err(`ERROR: ${e.message}`);
192
- if (e.hint) err(` hint: ${e.hint}`);
193
- return 2;
194
- }
195
-
196
- if (opts.help) {
197
- out(LINT_USAGE);
198
- return 0;
199
- }
249
+ export function buildBatchLintJson(files, summary) {
250
+ return {
251
+ mode: 'batch',
252
+ ok: summary.withErrors === 0,
253
+ summary,
254
+ files,
255
+ };
256
+ }
200
257
 
201
- const path = resolve(opts.file);
202
- const kind = opts.scenario ? 'scenario' : 'policy';
203
- let raw;
204
- try {
205
- raw = readFileSync(path, 'utf-8');
206
- } catch (e) {
207
- err(`ERROR: could not read ${kind} file: ${path} ${e.message}`);
208
- err(' hint: check the path exists and is readable');
209
- return 2;
210
- }
258
+ /**
259
+ * Read, parse, and lint ONE file — the primitive shared by the legacy single-file path
260
+ * and batch mode (F-49940082). Deliberately does not catch a read failure: the two
261
+ * callers give it different severity. The legacy path treats "the one named file cannot
262
+ * be read" as an operator mistake (exit 2, and it aborts — there is nothing else to try).
263
+ * Batch mode treats "one file among several cannot be read" as a per-file finding folded
264
+ * into the batch (see unreadableResult) so one bad path does not hide the results for
265
+ * every other path given alongside it. A YAML parse failure, by contrast, IS handled
266
+ * here and returned as an ok:false result — that has always been a lint finding, not an
267
+ * operator error, in both modes.
268
+ *
269
+ * @param {string} path - an already-resolved file path
270
+ * @param {{ scenario: boolean }} opts
271
+ * @returns {{ ok: boolean, origin: string, errors: object[], warnings: object[], coverageNote: string }}
272
+ */
273
+ function lintOneFile(path, opts) {
274
+ const raw = readFileSync(path, 'utf-8');
211
275
 
212
276
  let doc;
213
277
  try {
@@ -217,7 +281,7 @@ export async function runLint(argv, io = {}) {
217
281
  // Mirrors the policy path exactly, only differing in the origin/label/coverageNote so the
218
282
  // scenario report reads as a scenario report.
219
283
  const where = e && e.mark ? ` at line ${e.mark.line + 1}, column ${e.mark.column + 1}` : '';
220
- const result = opts.scenario
284
+ return opts.scenario
221
285
  ? {
222
286
  ok: false,
223
287
  origin: 'scenario',
@@ -242,13 +306,164 @@ export async function runLint(argv, io = {}) {
242
306
  warnings: [],
243
307
  coverageNote: COVERAGE_NOTE,
244
308
  };
245
- out(opts.json ? JSON.stringify(buildLintJson(result, path)) : renderLintText(result, path));
246
- return 1;
247
309
  }
248
310
 
249
- const result = opts.scenario
311
+ return opts.scenario
250
312
  ? lintScenario(doc, { file: path })
251
313
  : lintPolicy(doc, { origin: originForPath(path) });
252
- out(opts.json ? JSON.stringify(buildLintJson(result, path)) : renderLintText(result, path));
253
- return result.ok ? 0 : 1;
314
+ }
315
+
316
+ /**
317
+ * Build a lint-result-shaped entry for a batch-mode path that could not be read at all
318
+ * (missing, a permission fault, or any other readFileSync failure). `detail` is the real
319
+ * underlying Error#message from that failed read — not a guessed ENOENT string — so the
320
+ * report matches what the OS actually said, the same way the legacy single-file operator
321
+ * error always has.
322
+ *
323
+ * @param {string} path
324
+ * @param {{ scenario: boolean }} opts
325
+ * @param {string} detail - e.message from the failed readFileSync
326
+ */
327
+ function unreadableResult(path, opts, detail) {
328
+ const kind = opts.scenario ? 'scenario' : 'policy';
329
+ const message = `could not read ${kind} file: ${path} — ${detail}`;
330
+ return opts.scenario
331
+ ? {
332
+ ok: false,
333
+ origin: 'scenario',
334
+ errors: [{ label: 'scenario-schema:', code: 'path_unreadable', location: '/', message }],
335
+ warnings: [],
336
+ coverageNote: SCENARIO_COVERAGE_NOTE,
337
+ }
338
+ : {
339
+ ok: false,
340
+ origin: originForPath(path),
341
+ errors: [{ label: 'policy-schema:', code: 'path_unreadable', location: '/', message }],
342
+ warnings: [],
343
+ coverageNote: COVERAGE_NOTE,
344
+ };
345
+ }
346
+
347
+ /**
348
+ * stat a path without throwing. `null` means "does not exist, or could not be stat'd for
349
+ * any other reason" — the caller never needs to know which, because either way the path
350
+ * is not a confirmed directory, so it is handled as a candidate file, and any real fault
351
+ * (missing, permission, ...) surfaces through the normal readFileSync path with its own
352
+ * accurate message instead of a guess made here.
353
+ */
354
+ function safeStat(path) {
355
+ try {
356
+ return statSync(path);
357
+ } catch {
358
+ return null;
359
+ }
360
+ }
361
+
362
+ /**
363
+ * Run the lint subcommand. Returns the exit code (does not call process.exit) so it is
364
+ * unit-testable with an injected stdout/stderr sink.
365
+ *
366
+ * Routing (F-49940082): a single positional that does not resolve to an existing
367
+ * directory takes the legacy single-file path below, unchanged in every statement and
368
+ * every error message from before directory-or-multi-path mode existed. Anything else —
369
+ * two or more positionals, or the sole positional resolving to a directory — is batch
370
+ * mode: every positional is either walked (if a directory) or linted directly (if a
371
+ * file), results are aggregated, and a batch summary is printed after the per-file
372
+ * blocks. See the module docstring for the discovery-semantics parity statement against
373
+ * scripts/lint-policies.test.mjs's policyFiles().
374
+ *
375
+ * @param {string[]} argv - args AFTER the `lint` verb
376
+ * @param {{ stdout?: (s: string) => void, stderr?: (s: string) => void }} [io]
377
+ * @returns {Promise<number>}
378
+ */
379
+ export async function runLint(argv, io = {}) {
380
+ const out = io.stdout ?? ((s) => process.stdout.write(s + '\n'));
381
+ const err = io.stderr ?? ((s) => process.stderr.write(s + '\n'));
382
+
383
+ let opts;
384
+ try {
385
+ opts = parseLintArgs(argv);
386
+ } catch (e) {
387
+ err(`ERROR: ${e.message}`);
388
+ if (e.hint) err(` hint: ${e.hint}`);
389
+ return 2;
390
+ }
391
+
392
+ if (opts.help) {
393
+ out(LINT_USAGE);
394
+ return 0;
395
+ }
396
+
397
+ const resolvedPaths = opts.paths.map((p) => resolve(p));
398
+ const soleStat = resolvedPaths.length === 1 ? safeStat(resolvedPaths[0]) : null;
399
+ const isBatch = resolvedPaths.length > 1 || (soleStat !== null && soleStat.isDirectory());
400
+
401
+ if (!isBatch) {
402
+ // ── legacy single-file path — byte-identical to pre-F-49940082 behavior. Same
403
+ // statements, same order, same error text; only factored through lintOneFile so
404
+ // batch mode below can share it instead of duplicating the read/parse/lint sequence. ──
405
+ const path = resolvedPaths[0];
406
+ const kind = opts.scenario ? 'scenario' : 'policy';
407
+ let result;
408
+ try {
409
+ result = lintOneFile(path, opts);
410
+ } catch (e) {
411
+ err(`ERROR: could not read ${kind} file: ${path} — ${e.message}`);
412
+ err(' hint: check the path exists and is readable');
413
+ return 2;
414
+ }
415
+ out(opts.json ? JSON.stringify(buildLintJson(result, path)) : renderLintText(result, path));
416
+ return result.ok ? 0 : 1;
417
+ }
418
+
419
+ // ── batch mode (F-49940082): directory-or-multi-path ──
420
+ const kind = opts.scenario ? 'scenario' : 'policy';
421
+ const discovered = new Set();
422
+ for (const p of resolvedPaths) {
423
+ const st = safeStat(p);
424
+ if (st !== null && st.isDirectory()) {
425
+ for (const f of walkYamlFiles(p)) discovered.add(f);
426
+ } else {
427
+ // Not a confirmed directory: a file, a missing path, or anything else safeStat
428
+ // could not classify. Hand it to lintOneFile as a candidate — a genuinely missing
429
+ // or unreadable path surfaces its real OS error there, not a guess made here.
430
+ discovered.add(p);
431
+ }
432
+ }
433
+ const files = [...discovered].sort((a, b) => a.localeCompare(b));
434
+
435
+ if (files.length === 0) {
436
+ // Every positional was a confirmed directory and none contributed a file to lint.
437
+ // A lint that finds nothing and exits 0 is a vacuous pass — fail loud instead of
438
+ // silently doing nothing (swarms/CLAUDE.md: "a gate that can't go red is theater").
439
+ err(`ERROR: no ${kind} files found under: ${resolvedPaths.join(', ')}`);
440
+ err(' hint: an empty directory, or one with no yaml files, has nothing to lint');
441
+ return 2;
442
+ }
443
+
444
+ const summary = { total: files.length, clean: 0, withErrors: 0, withWarnings: 0 };
445
+ const textBlocks = [];
446
+ const jsonFiles = [];
447
+
448
+ for (const path of files) {
449
+ let result;
450
+ try {
451
+ result = lintOneFile(path, opts);
452
+ } catch (e) {
453
+ result = unreadableResult(path, opts, e.message);
454
+ }
455
+
456
+ if (result.errors.length) summary.withErrors++;
457
+ else if (result.warnings.length) summary.withWarnings++;
458
+ else summary.clean++;
459
+
460
+ if (opts.json) jsonFiles.push(buildLintJson(result, path));
461
+ else textBlocks.push(renderLintText(result, path));
462
+ }
463
+
464
+ out(opts.json
465
+ ? JSON.stringify(buildBatchLintJson(jsonFiles, summary))
466
+ : `${textBlocks.join('\n\n')}\n\n${renderBatchSummary(summary)}`);
467
+
468
+ return summary.withErrors > 0 ? 1 : 0;
254
469
  }
package/cli.js CHANGED
@@ -67,6 +67,21 @@ class OperatorError extends Error {
67
67
  }
68
68
  }
69
69
 
70
+ /**
71
+ * V2-CONTRACT-004: appended to EVERY --explain rendering (accepted and
72
+ * rejected alike) so a preview verdict can never be read as covering the
73
+ * required_steps gate, which only production ingest enforces (it fetches the
74
+ * scenario definitions this preview does not have).
75
+ *
76
+ * Declared ahead of USAGE and interpolated directly into its own "Not checked
77
+ * in preview" section below (F-e0c3fa21 — that section used to restate the
78
+ * header as its own redundant lowercase lead-in, worded slightly differently
79
+ * than this constant) so --help and --explain describe the identical gap
80
+ * from one shared string and can never drift apart again.
81
+ */
82
+ const PREVIEW_GAP_NOTE =
83
+ 'Not checked in preview: required_steps (needs scenario definitions — enforced only by real ingest).';
84
+
70
85
  const USAGE = `verify — local dry-run / explain for dogfood submissions
71
86
 
72
87
  Usage:
@@ -92,8 +107,7 @@ Provenance (default: stub):
92
107
 
93
108
  -h, --help Show this help.
94
109
 
95
- Not checked in preview:
96
- not checked in preview: required_steps (needs scenario definitions).
110
+ ${PREVIEW_GAP_NOTE}
97
111
  Scenario definitions live in the source repo; only a real ingest fetches
98
112
  them and enforces success_criteria.required_steps.
99
113
 
@@ -362,15 +376,6 @@ function resolveProvenance(provenanceMode, submission) {
362
376
  * the routing decision parseRejectionReason() makes; this maps it to operator
363
377
  * language so the consumer knows WHOSE problem each reason is.
364
378
  */
365
- /**
366
- * V2-CONTRACT-004: appended to EVERY --explain rendering (accepted and
367
- * rejected alike) so a preview verdict can never be read as covering the
368
- * required_steps gate, which only production ingest enforces (it fetches the
369
- * scenario definitions this preview does not have).
370
- */
371
- const PREVIEW_GAP_NOTE =
372
- 'Not checked in preview: required_steps (needs scenario definitions — enforced only by real ingest).';
373
-
374
379
  const CLASS_LABEL = {
375
380
  'submission-bad': 'SUBMISSION — fix your payload and resubmit',
376
381
  'operational': 'OPERATIONAL — verifier/tooling fault; page ops, do not bounce to submitter',