@vibe-agent-toolkit/cli 0.2.0-rc.6 → 0.2.0-rc.8

package/docs/skills.md

@@ -39,7 +39,7 @@ skills:
 - `paths` is optional on `allow` entries and defaults to `["**/*"]` (the whole skill).
 - Expired `allow` entries still apply; a separate `ALLOW_EXPIRED` warning surfaces the stale date for re-review. Opt into strict expiry with `severity.ALLOW_EXPIRED: error`.
 - Unused `allow` entries surface as `ALLOW_UNUSED` (analogous to ESLint's unused-disable).
-- `vat audit` is advisory: it applies `severity` for display grouping only, ignores `allow`, and always exits 0.
+- `vat audit` applies `severity` (a code set to `error` gates, `warning` does not, `ignore` hides it) and ignores `allow`; its exit code follows the report's `status`.
 
 See `docs/validation-codes.md` for the full code reference with per-code descriptions and defaults.
 
@@ -47,85 +47,58 @@ See `docs/validation-codes.md` for the full code reference with per-code descrip
 
 ### vat skills validate [path]
 
-**Purpose:** Validate SKILL.md files for correctness and best practices
+**Purpose:** Validate the skills a project declares, before packaging — the gate `vat validate` and `vat verify` run as their `skills` phase.
 
 **What it does:**
-1. Discovers all SKILL.md files in the target directory
-2. Validates both resource rules (links, frontmatter) and skill-specific rules
-3. Reports errors and warnings in structured format
-4. Exits 0 if valid, 1 if errors found (warnings don't fail)
-
-**Validation Modes:**
-
-**Project mode (default):**
-- Respects `vibe-agent-toolkit.config.yaml` boundaries
-- Strict filename validation - must be exactly "SKILL.md" (case-sensitive)
-- Errors on non-standard filenames (skill.md, Skill.md, etc.)
-
-**User mode (--user flag):**
-- Scans `~/.claude/plugins` and `~/.claude/skills`
-- Permissive validation - non-standard filenames generate warnings only
-- More tolerant for user-installed content
-
-**Path mode (explicit path):**
-- Scans specified directory
-- Strict filename validation like project mode
+1. Reads `vibe-agent-toolkit.config.yaml` from `[path]` (default: current directory) and discovers every file its `skills.include` / `skills.exclude` globs match
+2. Validates each with the packaging validator: frontmatter, links and link depth, size/complexity, `files:` config, compat observations — under the merged `skills.defaults` + `skills.config.<name>` validation config (`severity` + `allow`)
+3. Publishes a YAML document to stdout and a findings report to stderr
+4. Exits 0, 1 or 2 — see below
 
 **Arguments:**
-- `[path]` - Path to validate (defaults to current directory)
+- `[path]` — a directory that holds `vibe-agent-toolkit.config.yaml`. It scopes *which config is read*, not which files are scanned; the globs in that config do the scanning. A path that does not exist, is not a directory, or holds no config is refused (exit 2) rather than silently rescoping the run to nothing. There is no `--user` mode and no "scan this directory" mode.
 
 **Options:**
-- `--user` - Validate user-installed skills in ~/.claude
-- `--debug` - Enable debug logging
+- `--skill <name>` — validate one discovered skill only (narrows what is reported on, not what counts as declared test input)
+- `-v, --verbose` — publish every validated skill, findings or not, with full detail (`allErrors`, `ignoredErrors`, `observations`, `evidence`, metadata) and one block per finding on stderr
+- `-d, --debug` — debug logging
 
-**Exit Codes:**
-- `0` - All validations passed
-- `1` - Validation errors found (warnings don't fail)
-- `2` - System error (directory not found, config invalid)
+**Exit codes:**
 
-**Validation Checks:**
+| Exit | When |
+|---|---|
+| `0` | No active `error` finding (warnings and info do not fail; allowed errors do not fail) |
+| `1` | An active `error` finding on any skill, **or** the run validated no skill: `skills.include` matched nothing (typo, renamed directory, an `exclude` that swallows every match). Refused as one non-overridable `RESOURCE_CHECK_BROKEN` naming the globs — a green over zero skills is not a verdict |
+| `2` | Could not run: `[path]` refused, config invalid, `projectRoot` not found |
 
-*Resource validation:*
-- Internal file links (relative paths)
-- Anchor links within files (#heading)
-- Cross-file anchor links (file.md#heading)
-- Frontmatter schema validation
+A config with no `skills:` block at all is "nothing to validate": no document, exit 0 — declaring no skills is a choice, and both orchestrators skip the phase on that config.
 
-*Skill-specific validation:*
-- Reserved word checks (name field)
-- XML tag detection (name/description fields)
-- Console compatibility warnings
-- Required frontmatter fields (name, description)
+**What a matched file must be.** Every file the globs match is validated as a skill and counted in `skillsValidated`. A matched file with no YAML frontmatter block — or one whose block does not parse — is refused with `SKILL_MISSING_FRONTMATTER` at `error` (non-overridable), located at the file's project-relative path. So a glob that drifts onto a `README.md`, or a `SKILL.md` that lost its fence, fails the run instead of passing under its H1 as a name. A frontmatter block without a `name` is legal (agentskills.io makes `name` optional) and is not refused on that ground.
 
-*Filename validation:*
-- Must be "SKILL.md" (uppercase)
-- Strict mode: errors on skill.md, Skill.md
-- Permissive mode: warnings only
+**Output:**
+
+Default `results[]` lists only skills WITH findings — per-skill counts and a per-code tally, dominant code first; a clean skill is omitted and a zero bucket is an absent field. `skillsValidated` is the true denominator regardless. `issueCounts` always equals the sum of the per-skill rows plus `runIssueCounts` (run-level findings: `ALLOW_UNUSED` entries and the zero-skill refusal).
 
-**Output Format:**
 ```yaml
-status: success | error
-skillsValidated: 3
+status: error            # success | warning | error — worst actionable severity in the run
+issueCounts: { errors: 1, warnings: 0, info: 0 }
+runIssueCounts: { errors: 0, warnings: 0, info: 0 }
+skillsValidated: 2
 results:
-  - path: resources/skills/SKILL.md
-    status: success
-    skill:
-      name: my-skill
-      description: Skill description
-    issues: []
-durationSecs: 0.245
+  - skillName: Beta
+    status: error
+    errors: 1
+    codes:
+      SKILL_MISSING_FRONTMATTER: 1
+runIssues: []
+durationSecs: 0.31
 ```
 
-**Examples:**
-```bash
-# Validate all project skills (default)
-vat skills validate
-
-# Validate user-installed skills
-vat skills validate --user
+stderr prints the same thing as one line per skill, with every `error` finding rendered in full beneath its row (location, message, fix) at every verbosity.
 
-# Validate specific directory
-vat skills validate packages/my-agent/resources/skills
+**Example:**
+```bash
+vat skills validate packages/my-pkg/   # read packages/my-pkg/vibe-agent-toolkit.config.yaml
 ```
 
 **Requirements:**
@@ -186,8 +159,8 @@ discovery rule that this command participates in.
 
 **Exit Codes:**
 - `0` - Build successful (or dry-run preview)
-- `1` - Invalid source or build error
-- `2` - System error (missing package.json, invalid config)
+- `1` - A skill failed its validation gate, or a shipped link check found an error
+- `2` - The build could not run (missing package.json, invalid config, an unknown flag, an internal failure)
 
 **Output Format:**
 ```yaml
@@ -269,8 +242,8 @@ for terminology.
 
 **Exit Codes:**
 - `0` - Packaging successful (or dry-run preview)
-- `1` - Invalid skill path or packaging error
-- `2` - System error
+- `1` - The skill failed its validation gate, or its ZIP exceeds the upload size limit
+- `2` - Packaging could not run (skill path not found, an unknown flag, an internal failure)
 
 **What Gets Packaged:**
 - Root SKILL.md file
@@ -391,8 +364,7 @@ any other target lands correctly but is invisible to them.
 
 **Exit Codes:**
 - `0` - Install successful (or dry-run complete)
-- `1` - Install error (validation failed, conflict without `--force`, source not found, bad target/scope)
-- `2` - System error (unexpected exception)
+- `2` - Install could not proceed: the source was not found or failed validation, the skill is already present and `--force` was not passed, a bad target/scope, an unexpected exception. None of these is a finding about a skill, so none is exit `1`; the message says which.
 
 **YAML Output Format:**
 ```yaml
@@ -525,6 +497,18 @@ skills:
     warning: Non-standard filename (should be SKILL.md)
 ```
 
+When the scan could not list a directory (a root-owned or quarantined directory under
+`~/.claude/plugins`, say), `status` is `warning` and the document names each such directory,
+root-relative, with the errno — `skillsFound` is then a floor, not the answer:
+
+```yaml
+status: warning
+...
+unreadable:
+  - path: plugins/locked
+    code: EACCES
+```
+
 **Examples:**
 ```bash
 # List project skills (default)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vibe-agent-toolkit/cli",
-  "version": "0.2.0-rc.6",
+  "version": "0.2.0-rc.8",
   "description": "Command-line interface for vibe-agent-toolkit",
   "type": "module",
   "bin": {
@@ -18,18 +18,20 @@
   "files": [
     "dist",
     "docs",
+    "schemas",
     "README.md",
     "!dist/pipeline-oracles",
     "!dist/qa-snapshot"
   ],
   "scripts": {
-    "build": "tsx src/utils/validate-help-files.ts && tsx ../dev-tools/src/tsc-clean-build.ts && tsx ../dev-tools/src/prepare-bin.ts",
+    "build": "tsx src/utils/validate-help-files.ts && tsx ../dev-tools/src/tsc-clean-build.ts && bun run generate:schemas && tsx ../dev-tools/src/prepare-bin.ts",
+    "generate:schemas": "tsx scripts/generate-json-schemas.ts",
     "test:unit": "vitest run",
     "test:watch": "vitest",
     "test:integration": "vitest run --config vitest.integration.config.ts",
     "test:system": "vitest run --config vitest.system.config.ts",
     "typecheck": "tsc --noEmit",
-    "clean": "rimraf dist *.tsbuildinfo"
+    "clean": "rimraf --glob dist \"*.tsbuildinfo\" .tsc-staging"
   },
   "keywords": [
     "cli",
@@ -41,40 +43,33 @@
   "author": "Jeff Dutton",
   "license": "MIT",
   "engines": {
-    "node": ">=22.0.0"
+    "node": ">=22.13.0"
   },
   "//dependencies-note": "DO NOT add example agent packages or adopter packages here. Adopter packages (like vat-development-agents) depend on the CLI binary at build time — declaring them as deps creates a circular build dependency. They are built in a second turbo pass via package#task overrides in turbo.json.",
   "dependencies": {
     "@anthropic-ai/sdk": "^0.71.2",
-    "@vibe-agent-toolkit/agent-config": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/agent-skills": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/claude-marketplace": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/discovery": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/gateway-mcp": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/projection-sqlite": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/resources": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/schema": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/utils": "0.2.0-rc.6",
+    "@vibe-agent-toolkit/agent-config": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/agent-skills": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/claude-marketplace": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/discovery": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/gateway-mcp": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/projection-sqlite": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/resources": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/schema": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/utils": "0.2.0-rc.8",
     "adm-zip": "^0.6.0",
     "commander": "^12.1.0",
     "picomatch": "^4.0.3",
-    "semver": "^7.7.3",
+    "semver": "^7.8.5",
     "tar": "^7.5.7",
-    "yaml": "^2.6.1",
-    "zod": "^3.24.1"
+    "yaml": "^2.9.0",
+    "zod": "^3.25.0"
   },
   "devDependencies": {
-    "@types/adm-zip": "^0.5.7",
-    "@types/node": "^22.10.5",
-    "@types/picomatch": "^4.0.2",
-    "@types/semver": "^7.7.1",
     "@types/tar": "^6.1.13",
-    "@vibe-agent-toolkit/dev-tools": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/rag": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/rag-lancedb": "0.2.0-rc.6",
-    "rimraf": "^6.0.1",
-    "typescript": "^5.7.3",
-    "vitest": "^3.2.4"
+    "@vibe-agent-toolkit/dev-tools": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/rag": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/rag-lancedb": "0.2.0-rc.8"
   },
   "publishConfig": {
     "access": "public"
@@ -83,10 +78,10 @@
     "type": "git",
     "url": "https://github.com/jdutton/vibe-agent-toolkit.git"
   },
-  "//peerDependencies-note": "The RAG backends are OPTIONAL PEERS, never optionalDependencies. `optionalDependencies` are INSTALLED BY DEFAULT by npm and pnpm alike - 'optional' there means the install may fail without failing the build, not that it is skipped - so declaring them that way shipped ~240 MB (onnxruntime-web 136 MB, a LanceDB platform binary 93 MB, a third apache-arrow, protobufjs) to every adopter, contradicting the seam in src/utils/optional-backend.ts that exists precisely to keep them out. Optional peers are NOT auto-installed, so the manifest now says what that module already says: these ship separately, and an absent one is a legible error naming the package to install. projection-sqlite is deliberately NOT in this group: it has zero third-party dependencies (only node:sqlite, which is built into Node) and a 152 KB dist, so the platform-binary argument that justifies the seam is simply false for it. It is also what `vat resources query` builds its ephemeral in-memory store from, and a query surface that works only where a backend was separately installed is the capability gate the projection design forbids.",
+  "//peerDependencies-note": "The RAG backends are OPTIONAL PEERS, never optionalDependencies. `optionalDependencies` are INSTALLED BY DEFAULT by npm and pnpm alike - 'optional' there means the install may fail without failing the build, not that it is skipped - so declaring them that way shipped ~287 MB unpacked (onnxruntime-web 137.5 MB, a LanceDB platform binary 97.8 MB, gpt-tokenizer 42.2 MB, apache-arrow 5.3 MB, protobufjs 3.0 MB; measured on darwin-arm64 against this release's pins, and the platform binary is larger on win32-x64) to every adopter, contradicting the seam in src/utils/optional-backend.ts that exists precisely to keep them out. Optional peers are NOT auto-installed, so the manifest now says what that module already says: these ship separately, and an absent one is a legible error naming the package to install. projection-sqlite is deliberately NOT in this group: it has zero third-party dependencies (only node:sqlite, which is built into Node) and a 152 KB dist, so the platform-binary argument that justifies the seam is simply false for it. It is also what `vat resources query` builds its ephemeral in-memory store from, and a query surface that works only where a backend was separately installed is the capability gate the projection design forbids.",
   "peerDependencies": {
-    "@vibe-agent-toolkit/rag": "0.2.0-rc.6",
-    "@vibe-agent-toolkit/rag-lancedb": "0.2.0-rc.6"
+    "@vibe-agent-toolkit/rag": "0.2.0-rc.8",
+    "@vibe-agent-toolkit/rag-lancedb": "0.2.0-rc.8"
   },
   "peerDependenciesMeta": {
     "@vibe-agent-toolkit/rag": {

package/schemas/ard-emit.json

@@ -0,0 +1,149 @@
+{
+  "$ref": "#/definitions/ard-emit",
+  "definitions": {
+    "ard-emit": {
+      "type": "object",
+      "properties": {
+        "status": {
+          "type": "string",
+          "enum": [
+            "ok",
+            "findings",
+            "error"
+          ]
+        },
+        "examined": {
+          "type": "integer",
+          "minimum": 0
+        },
+        "findings": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "properties": {
+              "code": {
+                "type": "string"
+              },
+              "message": {
+                "type": "string"
+              },
+              "location": {
+                "type": "string"
+              },
+              "line": {
+                "type": "integer",
+                "exclusiveMinimum": 0
+              },
+              "field": {
+                "type": "string"
+              },
+              "link": {
+                "type": "string"
+              },
+              "fix": {
+                "type": "string"
+              },
+              "reference": {
+                "type": "string"
+              },
+              "suggestion": {
+                "type": "string"
+              },
+              "severity": {
+                "type": "string",
+                "enum": [
+                  "error",
+                  "warning",
+                  "info"
+                ]
+              },
+              "column": {
+                "type": "integer",
+                "exclusiveMinimum": 0
+              }
+            },
+            "required": [
+              "code",
+              "message",
+              "severity"
+            ],
+            "additionalProperties": false
+          }
+        },
+        "summary": {
+          "type": "object",
+          "properties": {
+            "errors": {
+              "type": "integer",
+              "minimum": 0
+            },
+            "warnings": {
+              "type": "integer",
+              "minimum": 0
+            },
+            "info": {
+              "type": "integer",
+              "minimum": 0
+            }
+          },
+          "required": [
+            "errors",
+            "warnings",
+            "info"
+          ],
+          "additionalProperties": false
+        },
+        "durationMs": {
+          "type": "number",
+          "minimum": 0
+        },
+        "error": {
+          "type": "string"
+        },
+        "data": {
+          "anyOf": [
+            {
+              "type": "object",
+              "properties": {
+                "outputPath": {
+                  "type": "string"
+                },
+                "entryCount": {
+                  "type": "integer",
+                  "minimum": 0
+                },
+                "skippedCount": {
+                  "type": "integer",
+                  "minimum": 0
+                },
+                "shadowedCount": {
+                  "type": "integer",
+                  "minimum": 0
+                }
+              },
+              "required": [
+                "outputPath",
+                "entryCount",
+                "skippedCount",
+                "shadowedCount"
+              ],
+              "additionalProperties": false
+            },
+            {
+              "type": "null"
+            }
+          ]
+        }
+      },
+      "required": [
+        "status",
+        "examined",
+        "findings",
+        "summary",
+        "data"
+      ],
+      "additionalProperties": false
+    }
+  },
+  "$schema": "http://json-schema.org/draft-07/schema#"
+}

package/schemas/claude-budget.json

@@ -0,0 +1,197 @@
+{
+  "$ref": "#/definitions/claude-budget",
+  "definitions": {
+    "claude-budget": {
+      "type": "object",
+      "properties": {
+        "status": {
+          "type": "string",
+          "enum": [
+            "ok",
+            "findings",
+            "error"
+          ]
+        },
+        "examined": {
+          "type": "integer",
+          "minimum": 0
+        },
+        "findings": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "properties": {
+              "code": {
+                "type": "string"
+              },
+              "message": {
+                "type": "string"
+              },
+              "location": {
+                "type": "string"
+              },
+              "line": {
+                "type": "integer",
+                "exclusiveMinimum": 0
+              },
+              "field": {
+                "type": "string"
+              },
+              "link": {
+                "type": "string"
+              },
+              "fix": {
+                "type": "string"
+              },
+              "reference": {
+                "type": "string"
+              },
+              "suggestion": {
+                "type": "string"
+              },
+              "severity": {
+                "type": "string",
+                "enum": [
+                  "error",
+                  "warning",
+                  "info"
+                ]
+              },
+              "column": {
+                "type": "integer",
+                "exclusiveMinimum": 0
+              }
+            },
+            "required": [
+              "code",
+              "message",
+              "severity"
+            ],
+            "additionalProperties": false
+          }
+        },
+        "summary": {
+          "type": "object",
+          "properties": {
+            "errors": {
+              "type": "integer",
+              "minimum": 0
+            },
+            "warnings": {
+              "type": "integer",
+              "minimum": 0
+            },
+            "info": {
+              "type": "integer",
+              "minimum": 0
+            }
+          },
+          "required": [
+            "errors",
+            "warnings",
+            "info"
+          ],
+          "additionalProperties": false
+        },
+        "durationMs": {
+          "type": "number",
+          "minimum": 0
+        },
+        "error": {
+          "type": "string"
+        },
+        "data": {
+          "anyOf": [
+            {
+              "type": "object",
+              "properties": {
+                "root": {
+                  "type": "string"
+                },
+                "threshold": {
+                  "type": "integer",
+                  "minimum": 0
+                },
+                "scope": {
+                  "type": "array",
+                  "items": {
+                    "type": "string"
+                  }
+                },
+                "unmatchedScope": {
+                  "type": "array",
+                  "items": {
+                    "type": "string"
+                  }
+                },
+                "distinctChains": {
+                  "type": "integer",
+                  "minimum": 0
+                },
+                "skippedUnknownLocations": {
+                  "type": "integer",
+                  "minimum": 0
+                },
+                "boundsStatement": {
+                  "type": "string"
+                },
+                "limits": {
+                  "type": "array",
+                  "items": {
+                    "type": "object",
+                    "properties": {
+                      "id": {
+                        "type": "string"
+                      },
+                      "direction": {
+                        "type": "string",
+                        "enum": [
+                          "over-report",
+                          "under-report",
+                          "scope",
+                          "assumption"
+                        ]
+                      },
+                      "statement": {
+                        "type": "string"
+                      }
+                    },
+                    "required": [
+                      "id",
+                      "direction",
+                      "statement"
+                    ],
+                    "additionalProperties": false
+                  }
+                }
+              },
+              "required": [
+                "root",
+                "threshold",
+                "scope",
+                "unmatchedScope",
+                "distinctChains",
+                "skippedUnknownLocations",
+                "boundsStatement",
+                "limits"
+              ],
+              "additionalProperties": false
+            },
+            {
+              "type": "null"
+            }
+          ]
+        }
+      },
+      "required": [
+        "status",
+        "examined",
+        "findings",
+        "summary",
+        "data"
+      ],
+      "additionalProperties": false
+    }
+  },
+  "$schema": "http://json-schema.org/draft-07/schema#"
+}