@vibe-agent-toolkit/cli 0.1.41-rc.5 → 0.1.41-rc.7

package/docs/audit.md

@@ -183,9 +183,9 @@ detection strategy doc, workstream B).
 status: success
 summary:
   filesScanned: 42
-  success: 40
-  warnings: 2
-  errors: 0
+  filesPassed: 40
+  filesWithWarnings: 2
+  filesWithErrors: 0
   marketplaces: 2
   standalonePlugins: 3
   standaloneSkills: 5
@@ -313,31 +313,57 @@ Warnings indicate potential issues but don't prevent usage:
 Structured YAML report for programmatic parsing:
 
 ```yaml
+root: /abs/path/to/scan/root   # The ONE absolute path in the document
 status: success | warning | error
 summary:
   filesScanned: number
-  success: number      # Files with no issues
-  warnings: number     # Files with warnings only
-  errors: number       # Files with errors
-issues:
-  errors: number       # Total error count across all files
-  warnings: number     # Total warning count across all files
-  info: number         # Total info count across all files
+  filesPassed: number         # Files with no ACTIONABLE issues (info-only counts as passed)
+  filesWithWarnings: number   # Files whose worst actionable severity is warning
+  filesWithErrors: number     # Files carrying at least one error
+issueCounts:                  # FINDINGS, not files — same field name and meaning
+  errors: number              #   as the `issueCounts` on each entry below
+  warnings: number
+  info: number
 duration: "123ms"
 files:
-  - path: /path/to/resource
+  - path: plugins/my-plugin            # relative to `root`
     status: success | warning | error
     type: plugin | marketplace | registry | skill
-    issues: []
+    issues:
+      - location: plugins/my-plugin/.claude-plugin/plugin.json   # relative to `root`
 ```
 
+#### One stated root, and everything relative to it
+
+`root` is the invocation scan root and the only absolute path a report contains.
+Every `path` and every issue `location` beneath it is forward-slashed and
+relative to that root, so:
+
+- `join(root, path)` and `join(root, location)` name real files — a consumer
+  never has to guess, or reimplement, the base a value was written against.
+- A `location` identifies a file uniquely across the whole document, even when a
+  run spans several projects that share internal layout (two `plugins/plugin-a`
+  directories under different parents cannot collapse to one string).
+- Nothing but `root` can leak the developer's or CI runner's home directory.
+
+The scan root is an ANCHOR, not a validation-policy boundary. Per-skill
+packaging rules still come from each skill's nearest-ancestor
+`vibe-agent-toolkit.config.yaml` (configs do not compose); that discovery has no
+say in how a path is spelled.
+
+`--user` scans three sibling directories (`plugins/`, `skills/`,
+`marketplaces/`), so its `root` is their shared Claude config dir
+(`$CLAUDE_CONFIG_DIR`, else `~/.claude`). A URL audit omits `root`: the clone
+lives in a random tempdir that nothing downstream can resolve, so the provenance
+header states the base instead and paths are relative to the cloned repo.
+
 ### Standard Error (stderr)
 
-Human-readable error messages and logs:
+Human-readable error messages and logs. Paths are root-relative here too:
 
 ```
 ERROR: Audit failed: 2 file(s) with errors
-ERROR: /path/to/skill.md:
+ERROR: skills/my-skill/SKILL.md:
 ERROR:   [SKILL_MISSING_NAME] name field is required in frontmatter
 ERROR:     at: line 1
 ERROR:     fix: Add 'name: skill-name' to frontmatter

package/docs/doctor.md

@@ -21,13 +21,24 @@ providing actionable suggestions for any problems found.
 7. CLI build status (when running from VAT source tree)
 
 **Options:**
-- `--verbose` - Show all checks (including passing ones)
+- `--verbose` - Show all checks (including passing and skipped ones)
+
+**Check outcomes:**
+
+| Icon | Outcome | Meaning |
+|------|---------|---------|
+| ✅ | `pass` | The check ran and the thing is fine |
+| ❌ | `fail` | The check ran and the thing is wrong — the only outcome that affects the exit code |
+| ❓ | `undetermined` | The check could not reach an answer (registry unreachable, file unreadable). **Nothing was verified** — this is not a pass |
+| ⏭️ | `skipped` | The check does not apply here (e.g. a VAT-source-tree-only check outside the source tree) |
 
 **Exit Codes:**
-- `0` - All checks passed
+- `0` - No check failed (an undetermined check is reported in the output, not fatal)
 - `1` - One or more checks failed
 
-**Output:** Human-friendly formatted text with emojis
+**Output:** Human-friendly formatted text with emojis. The summary always prints the
+full outcome distribution and, when the concise view hides checks, how many it hid —
+so the counts can never contradict the list above them.
 
 ## Usage Examples
 
@@ -39,17 +50,34 @@ Check all diagnostic items and show only failures:
 vat doctor
 ```
 
-Output when all checks pass:
+Output when all checks pass (the concise view prints no check blocks, and says so):
 ```
 🩺 vat doctor
 
 Running diagnostic checks...
 
-📊 Results: 7/7 checks passed
+📊 Results: 7 checks — 7 passed, 0 failed, 0 undetermined, 0 skipped
+   7 not shown (nothing to report) — re-run with --verbose to see every check.
 
 ✨ All checks passed! Your vat setup looks healthy.
 ```
 
+Output when a check could not be determined (here: the npm registry was unreachable):
+```
+🩺 vat doctor
+
+Running diagnostic checks...
+
+❓ vat version
+   Unable to check for updates: not found: npm
+
+📊 Results: 7 checks — 6 passed, 0 failed, 1 undetermined, 0 skipped
+   6 not shown (nothing to report) — re-run with --verbose to see every check.
+
+❓ Nothing failed, but 1 check(s) could not be determined — that is not the same as healthy.
+```
+Exit code is still `0` — nothing failed — but the run is *not* reported as healthy.
+
 ### Verbose Mode
 
 Show all checks including passing ones:
@@ -85,7 +113,7 @@ Running diagnostic checks...
 ✅ CLI build status
    Build is up to date (v0.1.0)
 
-📊 Results: 7/7 checks passed
+📊 Results: 7 checks — 7 passed, 0 failed, 0 undetermined, 0 skipped
 
 ✨ All checks passed! Your vat setup looks healthy.
 ```
@@ -110,7 +138,8 @@ Output:
 
 Running diagnostic checks...
 
-📊 Results: 7/7 checks passed
+📊 Results: 7 checks — 7 passed, 0 failed, 0 undetermined, 0 skipped
+   7 not shown (nothing to report) — re-run with --verbose to see every check.
 
 ✨ All checks passed! Your vat setup looks healthy.
 ```
@@ -250,14 +279,20 @@ fi
 ### VAT Version Check
 
 - **Purpose:** Inform about available updates (advisory only)
-- **Always passes:** This check never fails
+- **Never fails:** an available update is a `pass` with a suggestion
+- **Undetermined:** if the npm registry cannot be reached, the outcome is
+  `undetermined` (❓), not `pass` — doctor did not verify that you are current
 - **Suggestion:** Shows upgrade command if update available
 
 ### CLI Build Status Check
 
 - **When:** Only runs when in VAT source tree
 - **Purpose:** Ensure CLI build matches source version (for VAT developers)
-- **Skipped:** When running installed VAT globally
+- **Skipped (⏭️):** When running installed VAT globally, or when no project root
+  was detected — the check does not apply
+- **Undetermined (❓):** When the version files exist but cannot be read or
+  parsed. Doctor cannot tell whether the build is stale, which is different from
+  the check not applying
 - **Fix:** Run `bun run build` in VAT source directory
 
 ## Requirements
@@ -278,4 +313,5 @@ for terminology.
 - Run `vat doctor` before reporting issues to verify environment
 - Use `--verbose` flag when debugging to see all check details
 - Doctor checks can be run from any subdirectory in your project
-- Exit code 0 means all checks passed (useful for scripts)
+- Exit code 0 means no check *failed*; read the counts line to see whether any
+  check was undetermined (useful for scripts)

package/docs/index.md

@@ -102,7 +102,7 @@ Diagnose vat setup and environment health
 
 **Exit codes:**
 
-- `0` - All checks passed
+- `0` - No check failed (a check that could not be determined is reported as ❓, not fatal)
 - `1` - One or more checks failed
 
 **Creates/modifies:** None (read-only diagnostics)

package/docs/resources.md

@@ -117,20 +117,25 @@ vat resources validate docs/
 ```bash
 vat resources validate docs/
 
-# stderr:
-# docs/README.md:15:25: Link target not found: ./missing.md
-# docs/guide.md:42:10: Broken anchor: #non-existent-section
+# stderr:  (file:line:column: severity: message — only `error` fails the run)
+# docs/README.md:15:25: error: Link target not found: ./missing.md
+# docs/guide.md:42:10: error: Broken anchor: #non-existent-section
 
 # stdout:
 # ---
 # status: failed
 # errorsFound: 2
-# errors:
+# filesWithErrors: 2
+# issueCounts: { errors: 2, warnings: 0, info: 0 }
+# issueSummary: { LINK_BROKEN_FILE: 1, LINK_BROKEN_ANCHOR: 1 }
+# issues:
 #   - file: docs/README.md
-#     line: 15
-#     column: 25
-#     type: broken-link
-#     message: Link target not found: ./missing.md
+#     issues:
+#       - line: 15
+#         column: 25
+#         code: LINK_BROKEN_FILE
+#         severity: error
+#         message: Link target not found: ./missing.md
 # duration: 456ms
 # ---
 ```
@@ -182,6 +187,17 @@ resources:
       FRONTMATTER_SCHEMA_ERROR: error
 ```
 
+**Dot-directories are scanned.** `**` traverses path segments beginning with a
+dot, so the default `**/*.md` reaches `.claude/`, `.github/` and the like — as do
+your own patterns. Visibility is decided by git and by `exclude`, never by a
+leading dot. To keep a dotted tree out of the scan, exclude it by name:
+
+```yaml
+resources:
+  exclude:
+    - ".claude/worktrees/**"
+```
+
 ## Integration with vibe-validate
 
 The test-format error output (stderr) integrates seamlessly with vibe-validate:

package/docs/skill-test.md

@@ -64,14 +64,14 @@ Every field is optional; an omitted knob falls back to its default.
 | `maxBudgetUsd` | number > 0 | none | `--max-budget-usd <n>` | Hard USD budget cap passed to the CLI. |
 | `timeout` | integer, **seconds** | scales with eval count: ~`2min + 2min/eval`, floored at 5min, capped at 1h | `--timeout <s>` | Wall-clock timeout. An explicit value always overrides the scaled default. |
 | `stall` | integer, **seconds** | none | `--stall <s>` | Stall watchdog — kill the spawn after this long with no stream output. |
-| `evals` | string (path) | bootstrapped suite | `--evals <path>` | Path to `evals.json`, **relative to the skill source**. |
+| `evals` | string (path or npm specifier) | auto-detected `evals/evals.json` (see below) | `--evals <path>` | Which `evals.json` to grade against. `test.evals` resolves **relative to the skill source**; `--evals` resolves **relative to the current directory**. Either may be absolute, and either may be an npm bare specifier. An explicit value always wins over the default convention. |
 | `auth` | `inherit` \| `subscription` \| `api-key` \| `auto` | `inherit` | `--auth <mode>` | Auth mechanism for the spawned session. |
 | `requireAuth` | `subscription` \| `api-key` | none | `--require-auth <mech>` | Fail-fast guard: preflight exits `2` if the effective mechanism isn't this. |
 | `baseline` | boolean | `false` | `--baseline` | Run the opt-in with/without A/B skill-lift comparison. |
 | `skillCreator` | source descriptor | `{ vendored: true }` | — | Source for the vendored skill-creator rubric. |
 | `with` | array of source descriptors | none | `--with name=<src>` | **Required** companion skills staged alongside the subject, invocable by it. |
 | `optional` | array of source descriptors | none | `--with-optional name=<src>` | **Optional** companions — skipped with a warning if unresolvable. |
-| `env` | map string→string | none | `--env KEY=VALUE` | Env vars injected into the **executor** spawn. Values interpolate `${fixturesDir}`, `${stagedSkillDir}`, `${harnessRoot}`, `${resultsDir}`. Protected names (PATH, auth, model, admin) cannot be overridden. |
+| `env` | map string→string | none | `--env KEY=VALUE` | Env vars injected into the **executor** spawn. Values interpolate `${fixturesDir}`, `${stagedSkillDir}`, `${harnessRoot}`, `${resultsDir}`. `${fixturesDir}` is **per-eval** — it names that eval's own staged workspace (`fixtures/` under the executor's working directory), so the eval must declare input `files`; using it on an eval without them fails the run (exit 2). Protected names (PATH, auth, model, admin) cannot be overridden. |
 | `passEnv` | array of strings | none | `--pass-env KEY` | Names of host env vars to forward to the executor spawn if present. Protected names are ignored with a warning. |
 | `build` | string (shell command) | none | — | Command run **once, before staging**, to generate build artifacts. Runs with `cwd` = config root. A non-zero exit aborts the run (exit `2`). |
 
@@ -94,6 +94,50 @@ unbuilt source only when the failure is non-destructive. Each declared skill bui
 Staging the same name twice across the subject, `--with`, and `--with-optional` is a
 duplicate-name error (exit `2`).
 
+### The default eval-suite convention
+
+**You do not need a `test:` block to have an eval suite.** If a skill has a file at:
+
+```
+<skill-source-dir>/evals/evals.json
+```
+
+VAT treats that as the skill's suite — the same path `vat skill test run` has always defaulted to. Two things follow from it, and they are deliberately the same two things an explicit `test:` block would give you:
+
+1. `vat skill test run <skill>` finds and runs the suite.
+2. **The suite directory is excluded from packaged output.** It holds the `expected_output` / `expectations` answer key, so shipping it would both publish the answers and let a skill under test read its own key.
+
+This is **auto-detection, not a requirement**. It is keyed on the suite *file* existing:
+
+| On disk | Treated as a suite? |
+|---|---|
+| `<skill>/evals/evals.json` | **Yes** — run by the harness, excluded from the bundle |
+| `<skill>/evals/` with no `evals.json` | No — ordinary content, ships normally |
+| `<skill>/docs/evals/…` (any other location named `evals`) | No — ordinary content, ships normally |
+| No `evals/` directory at all | No — nothing happens, no error, no warning |
+
+A skill with no suite is completely unaffected: it packages exactly as it would have, with no error and nothing excluded. The convention never makes evals mandatory.
+
+**Set `test.evals` explicitly** when your suite lives anywhere else — a shared directory, a per-skill subdirectory (VAT's own repo uses `evals/<skill-name>/evals.json`), or outside the skill tree. An explicit path always overrides the convention.
+
+> Both lanes honor this. Before it was shared, the harness auto-detected the suite while the packager did not, so a skill relying on the default had its answer key published even though the harness was protecting it.
+
+### Testing a skill you did not author
+
+A correctly packaged skill ships **no** eval suite — the suite is the answer key, and excluding it from the bundle is the whole point of the convention above. So for any skill you install rather than write, there is nothing in its tree to grade against, and you have to supply the suite yourself:
+
+```bash
+vat skill test run npm:@vendor/their-skill \
+  --evals ./our-audit-corpus/their-skill.json \
+  --i-understand-this-runs-skill-code
+```
+
+`--evals` resolves against the directory you run it from, so the path means what you typed. It may point anywhere — including outside the skill's tree entirely, which is the normal case here. It also accepts an npm bare specifier (`--evals @acme/skill-evals/their-skill.json`), honoring that package's `exports` map, so a shared corpus can be distributed as a package.
+
+The suite is never copied into anything the executor can reach. Each eval's declared input `files` are resolved relative to the **suite's** directory and staged into that eval's own workspace, so fixtures travel with the suite rather than with the skill.
+
+> **On sensitive fixtures.** `workspaces/` and `results/` are created `0700`, so a fixture that was never in your repo is not left readable by other local users. That is necessary but **not** sufficient: `grading.json` quotes the executor transcript verbatim as evidence, so anything the skill *reads out of* a fixture is written into `results/` as text — and `results/` survives `--keep` by design. Treat the harness output as being as sensitive as the fixtures you feed it.
+
 ## Global knobs (top-level `test:`)
 
 | Config key | Type | Default | CLI flag | Purpose |
@@ -109,7 +153,7 @@ duplicate-name error (exit `2`).
 | `--no-build` | Stage existing `dist` instead of building. Errors if absent for the subject or a **required** companion; an optional companion falls back to raw source with a warning. |
 | `--refresh` | Force a full re-stage, ignoring existing staged content. |
 | `--keep` | Keep the harness directory after the run (needed to inspect `results/`). |
-| `--dry-run` | Assemble the command without spawning Claude. Note this **skips the build**, so it does not preview `files:` injection. |
+| `--dry-run` | Build and stage exactly as a real run would, then stop without spawning Claude — no session, no tokens. It **does build** (when `--i-understand-this-runs-skill-code` is passed), because the question a dry run answers is "what happens if I drop this flag", and a preview built from a stale `dist/` answers it wrongly. **Without** the acknowledgement it does not build — building runs the repo's `test.build` hook, an arbitrary shell command — so it falls back to an existing `dist/` and warns it may be stale. `--no-build` skips the build either way. |
 | `--out <dir>` / `--workdir <dir>` | Override the harness output / working directory. |
 | `--allow-eval-failure` | Opt out of fail-closed: exit `0` even when an eval fails. For interactive iteration. |
 | `--allow-unverified-skill-source` | Skip the vendored manifest integrity check. |

package/docs/skills.md

@@ -22,9 +22,9 @@ skills:
           LINK_TO_NAVIGATION_FILE: ignore   # this skill links to READMEs on purpose
           ALLOW_EXPIRED: error              # zero-tolerance expiry
         allow:
-          PACKAGED_UNREFERENCED_FILE:
+          LINK_TO_GITIGNORED_FILE:
             - paths: ["templates/runtime.json"]
-              reason: "consumed programmatically at runtime"
+              reason: "generated at install time, deliberately untracked"
               expires: "2026-09-30"
           SKILL_LENGTH_EXCEEDS_RECOMMENDED:
             - reason: "whole-skill concern; paths defaults to ['**/*']"
@@ -315,14 +315,22 @@ for terminology.
 
 **What it does:**
 1. Resolves the source (local directory, ZIP, .tgz, or npm package)
-2. Discovers one or more skills inside the source
+2. Discovers one or more skill directories inside the source
 3. Pre-validates every skill with `validateSkill()` — zero files written if validation fails
-4. Builds an install plan, detecting conflicts before touching the filesystem
-5. Copies skill(s) to the resolved platform directory (all-or-nothing semantics)
-6. Emits a YAML summary on stdout
+4. Reads each skill's declared name, which is the name it installs under
+5. Builds an install plan, detecting conflicts before touching the filesystem
+6. Copies skill(s) to the resolved platform directory (all-or-nothing semantics)
+7. Emits a YAML summary on stdout
 
 Both `--target` and `--scope` are **required** — there are no defaults.
 
+**Installed name:** a skill installs under the `name` its `SKILL.md` frontmatter
+declares — the same identity `vat skills build` and the plugin build key on — not
+the name of the directory it came from. A ZIP or npm source has no meaningful
+directory name to use. If two skills in one source declare the same name, the
+whole install fails rather than one silently overwriting the other. Use `--name`
+to override (single-skill sources only).
+
 **Supported Sources:**
 - **Local directory:** `./path/to/skill-dir` — must contain `SKILL.md` at root or in subdirectories
 - **Local ZIP file:** `./path/to/skill.zip` — extracted to a temp directory, then installed
@@ -354,6 +362,16 @@ Both `--target` and `--scope` are **required** — there are no defaults.
 | `windsurf` | `~/.codeium/windsurf/skills/` | `.windsurf/skills/` |
 | `agents` | `~/.agents/skills/` | `.agents/skills/` |
 
+These paths were last reviewed against vendor conventions on 2026-04-08 and are
+not verified at build or test time — the tests assert the table's shape, never
+that a path is where a platform actually looks. If a vendor moves its convention,
+installs land somewhere unread and every VAT check still passes. Re-check the
+vendor's documentation before relying on a non-`claude` target.
+
+**Visibility:** VAT's own inspection commands are Claude-scoped — `vat skills
+list --user` and `vat audit --user` read `~/.claude` only. A skill installed to
+any other target lands correctly but is invisible to them.
+
 **Exit Codes:**
 - `0` - Install successful (or dry-run complete)
 - `1` - Install error (validation failed, conflict without `--force`, source not found, bad target/scope)
@@ -438,6 +456,10 @@ for terminology.
 2. Reports validation status for each skill
 3. Shows skill metadata (name, description, path)
 
+The reported `name` is the one the skill's frontmatter declares — matching what
+`vat skills install` would install it as — falling back to the directory name
+only when the `SKILL.md` declares none.
+
 **Modes:**
 
 **Project mode (default):**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vibe-agent-toolkit/cli",
-  "version": "0.1.41-rc.5",
+  "version": "0.1.41-rc.7",
   "description": "Command-line interface for vibe-agent-toolkit",
   "type": "module",
   "bin": {
@@ -41,16 +41,16 @@
   "//dependencies-note": "DO NOT add example agent packages or adopter packages here. Adopter packages (like vat-development-agents) depend on the CLI binary at build time — declaring them as deps creates a circular build dependency. They are built in a second turbo pass via package#task overrides in turbo.json.",
   "dependencies": {
     "@anthropic-ai/sdk": "^0.71.2",
-    "@vibe-agent-toolkit/agent-config": "0.1.41-rc.5",
-    "@vibe-agent-toolkit/agent-schema": "0.1.41-rc.5",
-    "@vibe-agent-toolkit/agent-skills": "0.1.41-rc.5",
-    "@vibe-agent-toolkit/claude-marketplace": "0.1.41-rc.5",
-    "@vibe-agent-toolkit/discovery": "0.1.41-rc.5",
-    "@vibe-agent-toolkit/gateway-mcp": "0.1.41-rc.5",
-    "@vibe-agent-toolkit/rag": "0.1.41-rc.5",
-    "@vibe-agent-toolkit/rag-lancedb": "0.1.41-rc.5",
-    "@vibe-agent-toolkit/resources": "0.1.41-rc.5",
-    "@vibe-agent-toolkit/utils": "0.1.41-rc.5",
+    "@vibe-agent-toolkit/agent-config": "0.1.41-rc.7",
+    "@vibe-agent-toolkit/agent-schema": "0.1.41-rc.7",
+    "@vibe-agent-toolkit/agent-skills": "0.1.41-rc.7",
+    "@vibe-agent-toolkit/claude-marketplace": "0.1.41-rc.7",
+    "@vibe-agent-toolkit/discovery": "0.1.41-rc.7",
+    "@vibe-agent-toolkit/gateway-mcp": "0.1.41-rc.7",
+    "@vibe-agent-toolkit/rag": "0.1.41-rc.7",
+    "@vibe-agent-toolkit/rag-lancedb": "0.1.41-rc.7",
+    "@vibe-agent-toolkit/resources": "0.1.41-rc.7",
+    "@vibe-agent-toolkit/utils": "0.1.41-rc.7",
     "adm-zip": "^0.6.0",
     "commander": "^12.1.0",
     "picomatch": "^4.0.3",
@@ -65,7 +65,7 @@
     "@types/picomatch": "^4.0.2",
     "@types/semver": "^7.7.1",
     "@types/tar": "^6.1.13",
-    "@vibe-agent-toolkit/dev-tools": "0.1.41-rc.5",
+    "@vibe-agent-toolkit/dev-tools": "0.1.41-rc.7",
     "rimraf": "^6.0.1",
     "typescript": "^5.7.3",
     "vitest": "^3.2.4"