@rhize/skill-forge 0.19.0 → 0.20.1

package/README.md

@@ -2,9 +2,9 @@
 
 **The supply-chain gate for agent skills.**
 
-Source version: `@rhize/skill-forge@0.19.0` (2026-09-05), 0.x beta (Pro features free until 1.0).
-Npm publication is a separate tag-driven step; `npm view @rhize/skill-forge version` reports the
-latest published version.
+Published version: `@rhize/skill-forge@0.20.0` (2026-09-14), 0.x beta (Pro features free until 1.0).
+Released through the tag/OIDC workflow; `npm view @rhize/skill-forge version` reports the
+current published version.
 
 > This project is unrelated to the [`skillforge`](https://www.npmjs.com/package/skillforge)
 > package on npm, which is a Claude Skills *evaluation* framework. `skill-forge` (this package,
@@ -12,16 +12,17 @@ latest published version.
 
 ## Why
 
-Agent skills (Claude Code skills, and the equivalent extension formats emerging in Cursor and
-Codex) are now distributed the way npm packages were a decade ago: a public registry, a one-line
-install command, and no vetting step in between. skills.sh alone lists 600k+ skills. A single
-`npx skills@latest add owner/name` drops arbitrary third-party instructions, scripts, and tool
-bindings directly into a working agent's context and file system — with the same trust model as
-copy-pasting a shell script from a stranger.
+Agent skills can include third-party instructions and executable resources. Existing marketplaces
+already provide safeguards: [skills.sh offers partner security audits and installation risk
+information](https://vercel.com/changelog/automated-security-audits-now-available-for-skills-sh).
+Skill Forge adds a local review and maintenance workflow around the skills you actually use:
+provenance, overlap review, drift checks, and recoverable project refinements.
 
-skill-forge does not replace that install command; it wraps it. Every candidate skill is routed
-through **quarantine → profile → safety scan → overlap analysis → report → an explicit
-promote/hold/reject decision**, before it is allowed anywhere near your working skill set.
+Candidates submitted through Skill Forge follow **quarantine → profile → safety scan → overlap
+analysis (when available/configured) → report → an explicit promote/hold/reject decision**.
+This gate applies to the Skill Forge path; it does not prevent direct installs through other tools
+or replace host-managed restrictions. Passing a scan does not prove that a skill is safe.
+See the [product spec](docs/PRODUCT.md) for current competitive positioning and evidence limits.
 
 ## Quickstart
 
@@ -127,7 +128,10 @@ per-feature implementation status.
   outright; a lower-severity finding produces `warn`; a clean scan is `pass`.
 - **npm-sourced MCP candidates:** fetched with `--ignore-scripts`, and every tarball member path is
   validated before extraction (a path-traversal guard).
-- **SkillSpector, when installed,** is purely additive — its absence never blocks the gate.
+- **SkillSpector, when installed,** adds its individual findings, including modern `issues`
+  reports. Its absence preserves the built-in-only gate. A failed, malformed or incomplete
+  installed scan is an explicit HIGH `skillspector-unavailable` finding, which blocks under
+  default strictness. A LOW aggregate score cannot hide a HIGH individual issue.
 
 See [docs/security-model.md](docs/security-model.md) for the full rule table and current status of
 each check.
@@ -157,8 +161,9 @@ including current caveats, in [docs/configuration.md](docs/configuration.md).
 **Is this related to the `skillforge` npm package?**
 No. [`skillforge`](https://www.npmjs.com/package/skillforge) is a Claude Skills *evaluation*
 framework — it tests whether a skill performs well. `skill-forge` (this package, hyphenated) is a
-supply-chain security gate for skill *installation* — it decides whether a skill is safe and
-non-redundant before it ever runs. Same neighborhood, different problem, name collision only.
+review gate for skill *installation* and ongoing collection maintenance. It reports detected
+risks and potential overlap; it does not certify safety or task effectiveness. The similarly named
+packages are independently maintained.
 
 **Does skill-forge replace `npx skills@latest add`?**
 No, it wraps it. `add` uses the same install mechanisms (skills.sh, git, local copy) but stages
@@ -223,3 +228,12 @@ loading in a fresh task before treating the refinement as consumed. Roll back us
 the returned backup ID with `refine rollback <id> --project --yes`.
 
 See the [governance operating guide](docs/governance-improvements.md) for scope, schemas, examples and limits.
+
+## Advisory instruction diagnostics (0.20)
+
+`audit` reports description, entrypoint and resource measurements separately, with review
+suggestions for broad triggers and resource routing. These suggestions never change safety
+findings, gate decisions or installed skills. Static reports keep actual host loading and
+truncation unknown; rough token estimates are not measured usage. Comparisons show covered
+skills and description/body deltas when both reports contain the new evidence. See
+[the audit command](docs/commands/audit.md) for metric definitions and unavailable-resource handling.

package/dist/cli.js

@@ -1271,6 +1271,20 @@ function compareAudits(current, previous) {
     }
   }
   result.comparable = true;
+  const previousSkills = new Map(previous.inventory.skills.map((skill) => [skill.path, skill]));
+  let coveredSkills = 0;
+  let descriptionCodePointDelta = 0;
+  let entrypointCodePointDelta = 0;
+  for (const skill of current.inventory.skills) {
+    const prior = previousSkills.get(skill.path);
+    const currentDiagnostics = skill.instructionDiagnostics;
+    const priorDiagnostics = prior?.instructionDiagnostics;
+    if (!prior || !currentDiagnostics || !priorDiagnostics) continue;
+    coveredSkills++;
+    descriptionCodePointDelta += (currentDiagnostics.discovery.description?.unicodeCodePoints ?? 0) - (priorDiagnostics.discovery.description?.unicodeCodePoints ?? 0);
+    entrypointCodePointDelta += currentDiagnostics.entrypoint.unicodeCodePoints - priorDiagnostics.entrypoint.unicodeCodePoints;
+  }
+  result.instructionDiagnostics = coveredSkills > 0 ? { comparable: true, hostLoad: "unknown", coveredSkills, descriptionCodePointDelta, entrypointCodePointDelta } : { comparable: false, hostLoad: "unknown", reason: "Instruction diagnostics are unavailable in one or both report inventories.", coveredSkills: 0 };
   return result;
 }
 function loadAuditComparison(current, path) {
@@ -1284,6 +1298,7 @@ function renderAuditComparison(comparison) {
   const c = comparison;
   return ["## Changes since prior audit", "", ...c.comparable ? [
     `${c.added.length} new \xB7 ${c.changed.length} changed \xB7 ${c.resolved.length} no longer active \xB7 ${c.unchanged} unchanged.`,
+    ...c.instructionDiagnostics?.comparable ? [`Instruction diagnostics: ${c.instructionDiagnostics.coveredSkills} shared skill(s); description ${signed(c.instructionDiagnostics.descriptionCodePointDelta ?? 0)} code points, entrypoint ${signed(c.instructionDiagnostics.entrypointCodePointDelta ?? 0)} code points.`] : c.instructionDiagnostics ? [`Instruction diagnostics unavailable: ${c.instructionDiagnostics.reason}`] : [],
     "No longer active can include accepted findings; it does not prove a security fix.",
     "",
     ...c.added.map((f) => `- NEW ${f.severity} [${sanitizeForMarkdown(f.rule)}] ${sanitizeForMarkdown(f.target)}`),
@@ -1291,12 +1306,16 @@ function renderAuditComparison(comparison) {
     ...c.resolved.map((f) => `- NO LONGER ACTIVE [${sanitizeForMarkdown(f.rule)}] ${sanitizeForMarkdown(f.target)}`)
   ] : [sanitizeForMarkdown(c.reason ?? "Comparison unavailable.")], ""];
 }
+function signed(value) {
+  return value >= 0 ? `+${value}` : String(value);
+}
 
 // src/audit.ts
 import {
   existsSync as existsSync10,
   lstatSync as lstatSync3,
   mkdtempSync,
+  opendirSync,
   readFileSync as readFileSync11,
   readdirSync as readdirSync5,
   realpathSync as realpathSync3,
@@ -1496,10 +1515,68 @@ function which(cmd) {
   }
   return void 0;
 }
-function normalizeSeverity(sev) {
-  const s = (sev ?? "").toUpperCase();
-  if (s === "LOW" || s === "MEDIUM" || s === "HIGH" || s === "CRITICAL") return s;
-  return "MEDIUM";
+function scannerUnavailable(targetPath) {
+  return {
+    severity: "HIGH",
+    rule: "skillspector-unavailable",
+    file: targetPath,
+    detail: "Installed SkillSpector failed or returned an unsupported/incomplete report; external safety evidence is unavailable."
+  };
+}
+function reportRecord(value) {
+  if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("Invalid report record");
+  return value;
+}
+function reportSeverity(value) {
+  if (typeof value !== "string") throw new Error("Missing severity");
+  const severity = value.toUpperCase();
+  if (severity === "LOW" || severity === "MEDIUM" || severity === "HIGH" || severity === "CRITICAL") return severity;
+  throw new Error("Unknown severity");
+}
+function parseSkillSpectorReport(value, targetPath) {
+  try {
+    const report = reportRecord(value);
+    const modern = "issues" in report;
+    const raw = modern ? report.issues : report.filtered_findings ?? report.findings;
+    if (!Array.isArray(raw)) throw new Error("Missing findings");
+    const findings = raw.map((entry) => {
+      const issue = reportRecord(entry);
+      const rule = modern ? issue.id : issue.rule_id;
+      const detail = modern ? issue.finding ?? issue.explanation : issue.message;
+      if (typeof rule !== "string" || !rule || typeof detail !== "string") throw new Error("Invalid finding");
+      const location = modern ? reportRecord(issue.location) : void 0;
+      if (location && typeof location.file !== "string") throw new Error("Invalid location");
+      const line = location?.start_line;
+      return {
+        severity: reportSeverity(issue.severity),
+        rule,
+        file: location?.file ?? targetPath,
+        detail: typeof line === "number" && Number.isInteger(line) && line > 0 ? `${line}: ${detail}` : detail
+      };
+    });
+    const assessment = modern ? reportRecord(report.risk_assessment) : void 0;
+    const summary = assessment?.severity ?? report.risk_severity;
+    if (modern || summary !== void 0) {
+      const severity = reportSeverity(summary);
+      if (severity !== "LOW") findings.push({
+        severity,
+        rule: "skillspector-summary",
+        file: targetPath,
+        detail: `SkillSpector aggregate risk: ${severity}.`
+      });
+    }
+    if (modern) {
+      const completeness = reportRecord(report.analysis_completeness);
+      if (report.execution_successful !== true || completeness.is_complete !== true || completeness.execution_successful === false) {
+        findings.push(scannerUnavailable(targetPath));
+      }
+    } else if (report.execution_successful === false) {
+      findings.push(scannerUnavailable(targetPath));
+    }
+    return findings;
+  } catch {
+    return [scannerUnavailable(targetPath)];
+  }
 }
 async function scanWithSkillSpector(targetPath) {
   const bin = which("skillspector");
@@ -1509,16 +1586,15 @@ async function scanWithSkillSpector(targetPath) {
       timeout: 6e4,
       maxBuffer: 10 * 1024 * 1024
     });
-    const parsed = JSON.parse(stdout);
-    const raw = parsed.filtered_findings ?? parsed.findings ?? [];
-    return raw.map((f) => ({
-      severity: normalizeSeverity(f.severity),
-      rule: f.rule_id ?? "skillspector",
-      file: targetPath,
-      detail: f.message ?? ""
-    }));
-  } catch {
-    return [];
+    return parseSkillSpectorReport(JSON.parse(stdout), targetPath);
+  } catch (error) {
+    const failed = scannerUnavailable(targetPath);
+    try {
+      const stdout = error.stdout;
+      return stdout ? [...parseSkillSpectorReport(JSON.parse(stdout), targetPath), failed] : [failed];
+    } catch {
+      return [failed];
+    }
   }
 }
 async function scanSafety(targetPath, strictness = DEFAULT_STRICTNESS, capturedFiles) {
@@ -2128,6 +2204,43 @@ function proFeatureStatus(feature, betaFree = BETA_FREE) {
   };
 }
 
+// src/instructionDiagnostics.ts
+function instructionTextMetrics(text) {
+  const utf8Bytes = Buffer.byteLength(text, "utf8");
+  return { utf8Bytes, unicodeCodePoints: Array.from(text).length, words: text.match(/\S+/gu)?.length ?? 0, estimatedTokens: Math.ceil(utf8Bytes / 4) };
+}
+function skillEntrypointBody(text) {
+  const opening = text.match(/^---[ \t]*\r?\n/);
+  if (!opening) return text;
+  const closing = /^---[ \t]*(?:\r?\n|$)/gm;
+  closing.lastIndex = opening[0].length;
+  const match = closing.exec(text);
+  return match ? text.slice(match.index + match[0].length) : text;
+}
+function diagnoseInstructions(name, description, entrypoint, resources) {
+  const inputs = Array.isArray(resources) ? resources : Object.entries(resources).map(([directory, files]) => ({ directory, files }));
+  const inventory = [...inputs].sort((a, b) => a.directory.localeCompare(b.directory)).map(({ directory, files, unavailableReason }) => ({
+    directory,
+    ...unavailableReason ? { status: "unavailable", reason: unavailableReason } : { status: "available", files: files ?? 0 },
+    routedFromEntrypoint: new RegExp(`(?:^|[^A-Za-z0-9_-])${escapeRegExp(directory)}/`, "m").test(entrypoint)
+  }));
+  const advice = [];
+  if (description && instructionTextMetrics(description).unicodeCodePoints > 500) advice.push({ kind: "long-description", detail: "description exceeds the advisory 500-code-point review bucket" });
+  if (description && /\b(?:always|any|every|all)\b[^.\n]{0,90}\b(?:request|task|work|question)\b/i.test(description)) advice.push({ kind: "broad-trigger-wording", detail: "description contains broad/catchall trigger wording; review selection scope" });
+  const entrypointMetrics = instructionTextMetrics(entrypoint);
+  const entrypointLines = entrypoint === "" ? 0 : entrypoint.split(/\r\n|\r|\n/).length;
+  if (entrypointLines > 500 || entrypointMetrics.words > 4e3) advice.push({ kind: "dense-entrypoint", detail: `entrypoint has ${entrypointLines} lines and ${entrypointMetrics.words} words; review routing before moving content` });
+  const unavailable = inventory.filter((resource) => resource.status === "unavailable");
+  if (unavailable.length > 0) advice.push({ kind: "resource-inventory-unavailable", detail: `resource inventory unavailable for ${unavailable.map((resource) => resource.directory).join(", ")}; no file count was inferred` });
+  const unrouted = inventory.filter((resource) => resource.status === "available" && !resource.routedFromEntrypoint);
+  if (unrouted.length > 0) advice.push({ kind: "unrouted-resources", detail: `entrypoint does not name ${unrouted.map((resource) => resource.directory).join(", ")}; confirm conditional routing or document it` });
+  if (/\b(?:always|must)\s+(?:read|load)\b[^\n]*(?:references|reference|assets|templates|scripts|commands|hooks)\//i.test(entrypoint)) advice.push({ kind: "unconditional-resource-routing", detail: "entrypoint appears to require resource loading unconditionally; review whether a narrower trigger is possible" });
+  return { hostLoad: "unknown", discovery: { ...name ? { name: instructionTextMetrics(name) } : {}, ...description ? { description: instructionTextMetrics(description) } : {} }, entrypoint: entrypointMetrics, resources: inventory, advice };
+}
+function escapeRegExp(value) {
+  return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+}
+
 // src/audit.ts
 var OVERSIZED_SKILL_EST_TOKENS = 5e3;
 function toAuditFindings(source) {
@@ -2326,9 +2439,88 @@ function profileAuditedSkill(group, findings, unreadableSkillMdPaths, pluginId)
     validFrontmatter: fm.valid,
     description: fm.description,
     sizeBytes,
-    estTokens: estimateTokens(text)
+    estTokens: estimateTokens(text),
+    instructionDiagnostics: diagnoseInstructions(fm.name, fm.description, skillEntrypointBody(text), countSkillResources(group.canonicalPath))
   };
 }
+var RESOURCE_INVENTORY_SKIP_DIRS = /* @__PURE__ */ new Set([".git", "node_modules", "dist", "build", "__pycache__", ".venv", "venv"]);
+var MAX_RESOURCE_INVENTORY_ENTRIES = 1e4;
+var MAX_RESOURCE_INVENTORY_DEPTH = 32;
+function countSkillResources(root) {
+  const resourceDirs = ["scripts", "references", "reference", "commands", "hooks", "assets", "templates", "sub-skills"];
+  const inventory = [];
+  for (const directory of resourceDirs) {
+    const resourceRoot = join13(root, directory);
+    let rootStat;
+    try {
+      rootStat = lstatSync3(resourceRoot);
+    } catch (error) {
+      if (error.code !== "ENOENT") inventory.push({ directory, unavailableReason: `could not inspect resource directory: ${error.message}` });
+      continue;
+    }
+    if (rootStat.isSymbolicLink()) {
+      inventory.push({ directory, unavailableReason: "resource directory is a symlink and was not followed" });
+      continue;
+    }
+    if (!rootStat.isDirectory()) continue;
+    const pending = [{ path: resourceRoot, depth: 0 }];
+    let files = 0;
+    let entries = 0;
+    let unavailableReason;
+    while (pending.length > 0 && !unavailableReason) {
+      const current = pending.pop();
+      let handle;
+      try {
+        handle = opendirSync(current.path);
+      } catch (error) {
+        unavailableReason = `could not read resource subtree: ${error.message}`;
+        break;
+      }
+      try {
+        let entry;
+        while ((entry = handle.readSync()) !== null) {
+          if (++entries > MAX_RESOURCE_INVENTORY_ENTRIES) {
+            unavailableReason = `resource inventory exceeded ${MAX_RESOURCE_INVENTORY_ENTRIES} entries`;
+            break;
+          }
+          const path = join13(current.path, entry.name);
+          let stat;
+          try {
+            stat = lstatSync3(path);
+          } catch (error) {
+            unavailableReason = `could not inspect resource entry: ${error.message}`;
+            break;
+          }
+          if (stat.isSymbolicLink()) {
+            unavailableReason = `resource subtree contains a symlink that was not followed: ${entry.name}`;
+            break;
+          }
+          if (stat.isDirectory()) {
+            if (RESOURCE_INVENTORY_SKIP_DIRS.has(entry.name)) {
+              unavailableReason = `resource subtree contains skipped directory: ${entry.name}`;
+              break;
+            }
+            if (current.depth >= MAX_RESOURCE_INVENTORY_DEPTH) {
+              unavailableReason = `resource inventory exceeded depth ${MAX_RESOURCE_INVENTORY_DEPTH}`;
+              break;
+            }
+            pending.push({ path, depth: current.depth + 1 });
+          } else if (stat.isFile()) files++;
+        }
+      } catch (error) {
+        unavailableReason = `could not read resource subtree: ${error.message}`;
+      } finally {
+        try {
+          handle.closeSync();
+        } catch (error) {
+          if (!unavailableReason) unavailableReason = `could not close resource subtree: ${error.message}`;
+        }
+      }
+    }
+    inventory.push(unavailableReason ? { directory, unavailableReason } : { directory, files });
+  }
+  return inventory;
+}
 function hasRecognizedMcpShape(parsed) {
   if (typeof parsed !== "object" || parsed === null) return false;
   const obj = parsed;
@@ -2898,6 +3090,17 @@ function renderSkillsSection(skills) {
     )
   ];
 }
+function renderInstructionDiagnosticsSection(skills) {
+  const lines = ["## Instruction diagnostics (advisory)", "", "- Static measurements only. Actual host loading and truncation are unknown; estimated tokens use ceil(UTF-8 bytes / 4) and are not observed usage."];
+  for (const skill of skills) {
+    const diagnostics = skill.instructionDiagnostics;
+    if (!diagnostics) continue;
+    const description = diagnostics.discovery.description;
+    lines.push(`- ${sanitizeForMarkdown(skill.name)} \u2014 description: ${description ? `${description.utf8Bytes} UTF-8 bytes, ${description.unicodeCodePoints} Unicode code points, ${description.words} words, est. ${description.estimatedTokens} tokens` : "unavailable"}; entrypoint: ${diagnostics.entrypoint.utf8Bytes} UTF-8 bytes, ${diagnostics.entrypoint.unicodeCodePoints} Unicode code points, ${diagnostics.entrypoint.words} words, est. ${diagnostics.entrypoint.estimatedTokens} tokens; host load: unknown`);
+    for (const advice of diagnostics.advice) lines.push(`  - advisory ${advice.kind}: ${sanitizeForMarkdown(advice.detail)}`);
+  }
+  return lines;
+}
 function renderMcpTargetsSection(targets) {
   const lines = ["### MCP targets", ""];
   if (targets.length === 0) {
@@ -3055,6 +3258,8 @@ function renderAuditMarkdown(report) {
     "",
     ...renderSkillsSection(report.inventory.skills),
     "",
+    ...renderInstructionDiagnosticsSection(report.inventory.skills),
+    "",
     ...renderMcpTargetsSection(report.inventory.mcpTargets),
     "",
     ...renderFindingsSection(report.hygiene),
@@ -3807,6 +4012,7 @@ function profileSkillDir(inputPath) {
       resources,
       mcpDependencies: mcpDeps,
       externalPythonDeps: externalDeps,
+      instructionDiagnostics: diagnoseInstructions(fm.name, fm.description, skillEntrypointBody(text), resources),
       artifactType: detectArtifactType(root),
       extends: fm.extends ?? []
     };