skilldrop-cli 0.16.30 → 0.16.32
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/skilldrop.js +133 -18
- package/catalogue.json +4 -2
- package/contracts/skill.schema.json +3 -3
- package/model-routing.json +8 -0
- package/package.json +1 -1
- package/packs/converters/skills/mermaid-render/manifest.json +11 -1
- package/packs/dev-team/loops/release/LOOP.md +7 -5
- package/packs/dev-team/loops/release/loop.json +2 -2
- package/packs/dev-team/skills/codebase-tour/evals/files/3/.github/workflows/ci.yml +2 -2
- package/packs/dev-team/skills/codebase-tour/evals/files/3/go.mod +2 -2
- package/packs/dev-team/skills/dependency-upgrade-review/SKILL.md +52 -34
- package/packs/dev-team/skills/dependency-upgrade-review/evals/evals.json +33 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/.github/workflows/ci.yml +39 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/.github/workflows/integration.yml +20 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/PR.md +39 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/go.mod +17 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/notify/client.go +35 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/payments/client.go +66 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/payments/client_integration_test.go +38 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/payments/client_test.go +53 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/internal/payments/settle.go +30 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/pr.diff +42 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/6/vendor-notes/httpkit-CHANGELOG.md +43 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/.github/workflows/ci.yml +67 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/.lintwright.json +7 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/.nvmrc +1 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/PR.md +110 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/package.json +25 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/pr.diff +146 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/src/due-date.ts +16 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/src/totals.ts +11 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/tests/due-date.test.ts +17 -0
- package/packs/dev-team/skills/dependency-upgrade-review/evals/files/7/tests/totals.test.ts +12 -0
- package/packs/dev-team/skills/dependency-upgrade-review/examples/ledgerline-tidyparse.md +23 -14
- package/packs/dev-team/skills/dependency-upgrade-review/manifest.json +17 -5
- package/packs/dev-team/skills/dependency-upgrade-review/reference.md +36 -1
- package/packs/dev-team/skills/dependency-upgrade-review/scripts/upgrade_diff.py +22 -4
- package/packs/dev-team/skills/refactor-plan/SKILL.md +168 -0
- package/packs/dev-team/skills/refactor-plan/evals/eval_queries.json +46 -0
- package/packs/dev-team/skills/refactor-plan/evals/evals.json +106 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/1/debt-register-row.md +5 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/1/in-flight-prs.md +7 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/1/order_service.py +74 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/1/test-coverage.txt +14 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/3/imports.txt +22 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/3/register.md +3 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/4/docs/open-prs.md +6 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/4/docs/tech-debt-register.md +7 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/4/reports/monthly_revenue.py +44 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/4/src/billing/invoice.py +233 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/4/src/partners/statement.py +29 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/4/tests/billing/test_payments.py +13 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/5/ci-notes.md +5 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/5/open-prs.txt +4 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/5/scripts/nightly-digest.js +36 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/5/src/auth/passwordReset.ts +16 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/5/src/notify/index.ts +2 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/5/src/notify/legacyMailer.ts +149 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/5/src/orders/onOrderShipped.ts +15 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/5/tests/notify/legacyMailer.test.ts +7 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/6/billing-service/src/main/kotlin/billing/DunningJob.kt +29 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/6/billing-service/src/main/kotlin/billing/InvoiceService.kt +157 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/6/billing-service/src/main/kotlin/billing/pdf/InvoicePdfRenderer.kt +36 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/6/billing-service/src/test/kotlin/billing/InvoiceServiceTest.kt +40 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/6/docs/adr-007-persistence.md +26 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/6/docs/open-prs.md +9 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/7/notes/open-work.md +19 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/7/web/src/components/CheckoutPanel.tsx +207 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/7/web/src/components/MiniCart.tsx +33 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/7/web/src/components/__tests__/CheckoutPanel.test.tsx +13 -0
- package/packs/dev-team/skills/refactor-plan/evals/files/7/web/src/pages/OrderReview.tsx +24 -0
- package/packs/dev-team/skills/refactor-plan/examples/search-ranker.md +104 -0
- package/packs/dev-team/skills/refactor-plan/manifest.json +36 -0
- package/packs/dev-team/skills/refactor-plan/reference.md +149 -0
- package/packs/dev-team/skills/refactor-plan/templates/refactor-plan.md +82 -0
- package/packs/dev-team/skills/rollout-plan/SKILL.md +157 -0
- package/packs/dev-team/skills/rollout-plan/evals/eval_queries.json +12 -0
- package/packs/dev-team/skills/rollout-plan/evals/evals.json +109 -0
- package/packs/dev-team/skills/rollout-plan/evals/files/2/metrics-snapshot.txt +11 -0
- package/packs/dev-team/skills/rollout-plan/evals/files/2/pr-description.md +28 -0
- package/packs/dev-team/skills/rollout-plan/evals/files/4/PR_DESCRIPTION.md +24 -0
- package/packs/dev-team/skills/rollout-plan/evals/files/4/metrics_snapshot.txt +7 -0
- package/packs/dev-team/skills/rollout-plan/evals/files/5/docs/tax-engine-v2.md +24 -0
- package/packs/dev-team/skills/rollout-plan/evals/files/6/flags/smart_reminders_v2.yaml +14 -0
- package/packs/dev-team/skills/rollout-plan/evals/files/6/notes/push-pipeline.md +10 -0
- package/packs/dev-team/skills/rollout-plan/evals/files/7/docs/ratelimiter-v2.md +21 -0
- package/packs/dev-team/skills/rollout-plan/evals/files/7/ops/traffic-summary.txt +9 -0
- package/packs/dev-team/skills/rollout-plan/examples/sync-engine-workspaces.md +102 -0
- package/packs/dev-team/skills/rollout-plan/manifest.json +35 -0
- package/packs/dev-team/skills/rollout-plan/reference.md +100 -0
- package/packs/dev-team/skills/rollout-plan/scripts/canary_math.py +261 -0
- package/packs/dev-team/skills/rollout-plan/templates/rollout-plan.md +50 -0
- package/packs/dev-team/skills/sonar-review/manifest.json +12 -0
- package/packs/infra-as-code/skills/terraform-module/manifest.json +11 -0
- package/packs/infra-as-code/skills/terraform-plan-review/manifest.json +8 -1
- package/packs/solution-architect/skills/figma-diagrams/manifest.json +12 -1
package/bin/skilldrop.js
CHANGED
|
@@ -66,7 +66,7 @@ Usage:
|
|
|
66
66
|
skilldrop uninstall --agent <name...> remove subagents
|
|
67
67
|
skilldrop uninstall --loop <name...> remove loops (stage skills are left in place)
|
|
68
68
|
skilldrop validate [--from <src>] structural check of a catalog (for catalog authors)
|
|
69
|
-
skilldrop scan [<skill...>] [--from <src>] supply-chain scan — flag network/exec/credential
|
|
69
|
+
skilldrop scan [<skill...>] [--agent] [--from <src>] supply-chain scan of skills and agents — flag network/exec/credential
|
|
70
70
|
patterns in scripts and injection-shaped instructions
|
|
71
71
|
in SKILL.md before you trust a catalog (RFC-0022)
|
|
72
72
|
skilldrop new-skill <name> --pack <p> scaffold a skill in the catalog you're in (for catalog authors;
|
|
@@ -440,11 +440,11 @@ function gate(cat, names) {
|
|
|
440
440
|
|
|
441
441
|
/* RFC-0050: the worst scan finding per skill, for a policy's scan.max. Eval fixtures and
|
|
442
442
|
example inputs (⚪) never count. */
|
|
443
|
-
function worstFindings(cat, names) {
|
|
443
|
+
function worstFindings(cat, names, scanner = scanSkill) {
|
|
444
444
|
const out = {};
|
|
445
445
|
for (const s of names) {
|
|
446
446
|
let w;
|
|
447
|
-
for (const f of
|
|
447
|
+
for (const f of scanner(cat, s)) if (!f.fixture && (!w || gov.SEVERITY[f.sev] > gov.SEVERITY[w])) w = f.sev;
|
|
448
448
|
if (w) out[s] = w;
|
|
449
449
|
}
|
|
450
450
|
return out;
|
|
@@ -455,7 +455,8 @@ function enforcePolicy(flags, cat, names, kind) {
|
|
|
455
455
|
const policy = gov.load(flags);
|
|
456
456
|
gov.prepare(policy); // an audit log that can't be written stops the install before it starts
|
|
457
457
|
if (!policy || !names.length) return policy;
|
|
458
|
-
const worst =
|
|
458
|
+
const worst = policy.scan && policy.scan.max !== undefined && (kind === "skill" || kind === "agent")
|
|
459
|
+
? worstFindings(cat, names, kind === "agent" ? scanAgent : scanSkill) : {};
|
|
459
460
|
const problems = gov.check(policy, { source: cat.source, pinned: cat.pinned, names, worst, kind });
|
|
460
461
|
if (problems.length) {
|
|
461
462
|
for (const pr of problems) console.error(`refused ${pr.name}: ${pr.why}`);
|
|
@@ -614,10 +615,73 @@ function permissionsOf(cat, s) {
|
|
|
614
615
|
if (cat.shape === "apm") return null;
|
|
615
616
|
try { return manifestOf(cat, s).permissions || null; } catch (e) { return null; }
|
|
616
617
|
}
|
|
618
|
+
/* RFC-0054: what a skill's instructions tell the agent to run. A skill with no scripts still
|
|
619
|
+
reaches the shell and the network through the agent, so its fenced shell blocks are read the
|
|
620
|
+
way a reviewer would: the program each command starts, and the hosts it names. Shell builtins,
|
|
621
|
+
keywords and the text tools every agent already has are left out, as are interpreters running
|
|
622
|
+
the skill's own scripts/ (those are covered by the script rules). Prose outside a fenced shell
|
|
623
|
+
block is not parsed: an instruction to run something belongs in one, where a reader sees it. */
|
|
624
|
+
const SHELL_LANGS = new Set(["bash", "sh", "shell", "console", "zsh"]);
|
|
625
|
+
const NOT_A_PROGRAM = new Set(("cd echo export set unset true false test printf read source exit return then fi do done " +
|
|
626
|
+
"for if elif else while until case esac in function local shift wait time eval exec alias type command " +
|
|
627
|
+
"mkdir ls cat cp mv rm touch head tail grep egrep sed awk sort uniq wc tr cut xargs tee diff find jq less more " +
|
|
628
|
+
"pwd basename dirname date sleep env which chmod ln realpath mktemp yes seq").split(" "));
|
|
629
|
+
const INTERPRETERS = new Set(["python", "python3", "node", "bash", "sh", "zsh", "deno", "bun"]);
|
|
630
|
+
function instructedRuns(dir) {
|
|
631
|
+
const files = ["SKILL.md", "reference.md"].map((f) => path.join(dir, f)).filter((f) => fs.existsSync(f));
|
|
632
|
+
const refs = path.join(dir, "references");
|
|
633
|
+
if (fs.existsSync(refs)) for (const f of fs.readdirSync(refs)) if (f.endsWith(".md")) files.push(path.join(refs, f));
|
|
634
|
+
const commands = new Map(), hosts = new Map();
|
|
635
|
+
for (const f of files) {
|
|
636
|
+
const lines = fs.readFileSync(f, "utf8").split(/\r?\n/);
|
|
637
|
+
let lang = null;
|
|
638
|
+
lines.forEach((raw, i) => {
|
|
639
|
+
const fence = raw.match(/^\s*```+\s*([\w-]*)/);
|
|
640
|
+
if (fence) { lang = lang === null ? fence[1].toLowerCase() : null; return; }
|
|
641
|
+
if (lang === null || !SHELL_LANGS.has(lang)) return;
|
|
642
|
+
const where = { file: path.relative(dir, f), line: i + 1, text: raw.trim().slice(0, 120) };
|
|
643
|
+
const line = raw.replace(/(^|\s)#.*$/, "").replace(/^\s*\$\s+/, "").trim();
|
|
644
|
+
if (!line) return;
|
|
645
|
+
for (const h of line.matchAll(/https?:\/\/([a-z0-9][a-z0-9.-]*\.[a-z]{2,})/gi))
|
|
646
|
+
if (!hosts.has(h[1].toLowerCase())) hosts.set(h[1].toLowerCase(), where);
|
|
647
|
+
// Quoted text is an argument (a grep pattern, a message), never a command, so blank it
|
|
648
|
+
// before splitting on | ; && — otherwise `grep 'a|b'` reads as a command named b.
|
|
649
|
+
const bare = line.replace(/'[^']*'|"(?:[^"\\]|\\.)*"/g, (q) => (/scripts\/|CLAUDE_SKILL_DIR/.test(q) ? q.slice(1, -1) : " X "));
|
|
650
|
+
for (let part of bare.replace(/\$\(/g, " ; ").replace(/[`()]/g, " ").split(/&&|\|\||\||;/)) {
|
|
651
|
+
let w = part.trim().split(/\s+/).filter(Boolean);
|
|
652
|
+
while (w.length && /^[A-Za-z_][A-Za-z0-9_]*=/.test(w[0])) w.shift();
|
|
653
|
+
if (w[0] === "sudo" || w[0] === "nohup") w.shift();
|
|
654
|
+
const c = w[0];
|
|
655
|
+
if (!c || !/^[a-z][\w.+-]*$/.test(c) || NOT_A_PROGRAM.has(c)) continue;
|
|
656
|
+
if (INTERPRETERS.has(c) && w.slice(1).some((x) => /(^|\/)scripts\//.test(x) || /CLAUDE_SKILL_DIR/.test(x))) continue;
|
|
657
|
+
const prog = INTERPRETERS.has(c) && w[1] === "-m" && /^[a-z][\w.-]*$/.test(w[2] || "") ? w[2] : c; // python3 -m pip -> pip
|
|
658
|
+
if (!commands.has(prog)) commands.set(prog, where);
|
|
659
|
+
}
|
|
660
|
+
});
|
|
661
|
+
}
|
|
662
|
+
return { commands, hosts };
|
|
663
|
+
}
|
|
664
|
+
|
|
617
665
|
function permissionGaps(cat, s, findings) {
|
|
618
666
|
const dir = skillDir(cat, s);
|
|
619
|
-
if (!dir
|
|
667
|
+
if (!dir) return [];
|
|
668
|
+
const scripted = fs.existsSync(path.join(dir, "scripts"));
|
|
620
669
|
const perm = permissionsOf(cat, s);
|
|
670
|
+
const told = cat.shape === "apm" ? { commands: new Map(), hosts: new Map() } : instructedRuns(dir);
|
|
671
|
+
const instructionGaps = () => {
|
|
672
|
+
const out = [];
|
|
673
|
+
const cmds = (perm && perm.commands) || [], net = (perm && perm.network) || [];
|
|
674
|
+
for (const [c, w] of told.commands)
|
|
675
|
+
if (!cmds.includes("*") && !cmds.includes(c))
|
|
676
|
+
out.push({ sev: "🟨", id: "undeclared-instructed-command", owasp: ["AST03", "AST10"], ...w,
|
|
677
|
+
why: `the instructions tell the agent to run \`${c}\`, but the manifest's permissions.commands doesn't list it` });
|
|
678
|
+
for (const [h, w] of told.hosts)
|
|
679
|
+
if (!net.includes(h))
|
|
680
|
+
out.push({ sev: "🟨", id: "undeclared-instructed-network", owasp: ["AST03", "AST10"], ...w,
|
|
681
|
+
why: `the instructions tell the agent to reach ${h}, but the manifest's permissions.network doesn't list it` });
|
|
682
|
+
return out;
|
|
683
|
+
};
|
|
684
|
+
if (!scripted) return instructionGaps();
|
|
621
685
|
if (!perm)
|
|
622
686
|
return [{ sev: "🟨", id: "no-permissions", owasp: ["AST03", "AST10"], file: "manifest.json", line: 0,
|
|
623
687
|
why: "ships scripts but declares no permissions (network, commands, files) — nothing to check them against",
|
|
@@ -633,7 +697,7 @@ function permissionGaps(cat, s, findings) {
|
|
|
633
697
|
if (f.id === "broad-fs" && perm.files !== "anywhere")
|
|
634
698
|
gap("undeclared-files", f, `writes outside its own paths, but the manifest declares files: "${perm.files}"`);
|
|
635
699
|
}
|
|
636
|
-
return gaps;
|
|
700
|
+
return gaps.concat(instructionGaps());
|
|
637
701
|
}
|
|
638
702
|
function permissionSummary(perm) {
|
|
639
703
|
if (!perm) return "no permissions declared";
|
|
@@ -643,35 +707,82 @@ function permissionSummary(perm) {
|
|
|
643
707
|
return `${net} · ${cmds} · ${files}`;
|
|
644
708
|
}
|
|
645
709
|
|
|
646
|
-
|
|
710
|
+
/* RFC-0054: an agent is a system prompt the tool adopts whole, with the tools its frontmatter
|
|
711
|
+
grants, so it gets the prose rules a SKILL.md gets plus the checks only an agent needs. */
|
|
712
|
+
function scanAgent(cat, a) {
|
|
713
|
+
const file = agentPath(cat, a);
|
|
714
|
+
const findings = [];
|
|
715
|
+
if (!file || !fs.existsSync(file)) return findings;
|
|
716
|
+
const rel = path.basename(file);
|
|
717
|
+
const text = fs.readFileSync(file, "utf8");
|
|
718
|
+
text.split(/\r?\n/).forEach((line, i) => {
|
|
719
|
+
if (line.length > 400) return;
|
|
720
|
+
for (const r of PROSE_RULES)
|
|
721
|
+
if (r.re.test(line)) findings.push({ sev: r.sev, id: r.id, why: r.why, owasp: r.owasp, file: rel, line: i + 1, text: line.trim().slice(0, 120) });
|
|
722
|
+
});
|
|
723
|
+
const fmEnd = text.startsWith("---") ? text.indexOf("\n---", 3) : -1;
|
|
724
|
+
const fm = fmEnd > 0 ? text.slice(0, fmEnd) : "";
|
|
725
|
+
const lineOf = (re) => { const i = fm.split("\n").findIndex((l) => re.test(l)); return i < 0 ? 1 : i + 1; };
|
|
726
|
+
const at = (re) => ({ file: rel, line: lineOf(re), text: (fm.split("\n")[lineOf(re) - 1] || "").trim().slice(0, 120) });
|
|
727
|
+
const tools = (fm.match(/^tools:\s*(.*)$/m) || [null, null])[1];
|
|
728
|
+
const list = tools === null ? null : tools.replace(/[[\]"']/g, "").split(",").map((t) => t.trim()).filter(Boolean);
|
|
729
|
+
if (list === null)
|
|
730
|
+
findings.push({ sev: "🟧", id: "agent-all-tools", owasp: ["AST03", "LLM06"], ...at(/^name:/),
|
|
731
|
+
why: "has no `tools:` list, so it inherits every tool the session has, including any MCP server's" });
|
|
732
|
+
else {
|
|
733
|
+
const reads = list.filter((t) => /^(WebFetch|WebSearch)$/.test(t) || /^mcp__/.test(t));
|
|
734
|
+
const acts = list.filter((t) => /^(Bash|Write|Edit|NotebookEdit)$/.test(t));
|
|
735
|
+
if (reads.length && acts.length)
|
|
736
|
+
findings.push({ sev: "🟧", id: "agent-untrusted-and-act", owasp: ["AST03", "LLM01", "LLM06"], ...at(/^tools:/),
|
|
737
|
+
why: `reads content from outside (${reads.join(", ")}) and can act on it (${acts.join(", ")}) in the same context: an injected instruction in what it reads can run` });
|
|
738
|
+
}
|
|
739
|
+
if (/^permissionMode:\s*(bypassPermissions|acceptEdits)/m.test(fm))
|
|
740
|
+
findings.push({ sev: "🟥", id: "agent-skips-approval", owasp: ["AST03", "LLM06"], ...at(/^permissionMode:/),
|
|
741
|
+
why: "sets a permission mode that skips the user's approval for its actions" });
|
|
742
|
+
if (/^hooks:/m.test(fm))
|
|
743
|
+
findings.push({ sev: "🟧", id: "agent-hooks", owasp: ["AST03", "AST01"], ...at(/^hooks:/),
|
|
744
|
+
why: "defines hooks: commands that run on events, outside the conversation the user reads" });
|
|
745
|
+
const order = { "🟥": 0, "🟧": 1, "🟨": 2, "⚪": 3 };
|
|
746
|
+
return findings.sort((x, y) => order[x.sev] - order[y.sev]);
|
|
747
|
+
}
|
|
748
|
+
|
|
749
|
+
function printScan(cat, names, { compact = false, scanner = scanSkill, kind = "" } = {}) {
|
|
647
750
|
let total = 0, worst = null;
|
|
648
751
|
for (const s of names) {
|
|
649
|
-
const f =
|
|
752
|
+
const f = scanner(cat, s);
|
|
650
753
|
if (!f.length) continue;
|
|
651
754
|
total += f.length;
|
|
652
755
|
if (!worst || f[0].sev === "🟥") worst = worst === "🟥" ? worst : f[0].sev;
|
|
653
|
-
console.log(`\n${s}`);
|
|
756
|
+
console.log(`\n${kind === "agent" ? "agent " : ""}${s}`);
|
|
654
757
|
for (const x of (compact ? f.slice(0, 3) : f))
|
|
655
758
|
console.log(` ${x.sev} ${x.id} — ${x.why} [${x.owasp.join(", ")}]\n ${x.file}:${x.line} ${x.text}`);
|
|
656
|
-
if (compact && f.length > 3) console.log(` … ${f.length - 3} more (skilldrop scan ${s} --from ${cat.source})`);
|
|
759
|
+
if (compact && f.length > 3) console.log(` … ${f.length - 3} more (skilldrop scan ${kind === "agent" ? "--agent " : ""}${s} --from ${cat.source})`);
|
|
657
760
|
}
|
|
658
761
|
return { total, worst };
|
|
659
762
|
}
|
|
660
763
|
|
|
661
764
|
function scan(args) {
|
|
662
765
|
const cat = resolveCatalog(args.flags.from);
|
|
663
|
-
|
|
766
|
+
// `scan` alone covers every skill and agent; `scan <skills>` the named skills;
|
|
767
|
+
// `scan --agent [names]` agents only (--agents works too) (an agent may share a skill's name).
|
|
768
|
+
const agentsOnly = !!(args.flags.agent || args.flags.agents);
|
|
769
|
+
const names = agentsOnly ? [] : (args._.length ? args._.slice() : skillsIn(cat));
|
|
770
|
+
const agents = agentsOnly ? (args._.length ? args._.slice() : agentsIn(cat)) : (args._.length ? [] : agentsIn(cat));
|
|
664
771
|
for (const s of names) if (!skillExists(cat, s)) die(`unknown skill '${s}' in catalog '${cat.source}'`);
|
|
772
|
+
for (const a of agents) if (!agentsIn(cat).includes(a)) die(`unknown agent '${a}' in catalog '${cat.source}'`);
|
|
665
773
|
|
|
666
774
|
if (args.flags.json) {
|
|
667
775
|
const out = names.map((s) => ({ skill: s, findings: scanSkill(cat, s) })).filter((r) => r.findings.length);
|
|
776
|
+
const aout = agents.map((a) => ({ agent: a, findings: scanAgent(cat, a) })).filter((r) => r.findings.length);
|
|
668
777
|
return emitJSON({
|
|
669
|
-
catalog: cat.source, scanned: names.length,
|
|
670
|
-
flagged: out.length
|
|
778
|
+
catalog: cat.source, scanned: names.length, agentsScanned: agents.length,
|
|
779
|
+
flagged: out.length + aout.length,
|
|
780
|
+
findingCount: out.reduce((n, r) => n + r.findings.length, 0) + aout.reduce((n, r) => n + r.findings.length, 0),
|
|
781
|
+
skills: out, agents: aout,
|
|
671
782
|
});
|
|
672
783
|
}
|
|
673
|
-
console.log(`Scanning ${names.length} skill(s) in catalog '${cat.source}' for supply-chain patterns…`);
|
|
674
|
-
const
|
|
784
|
+
console.log(`Scanning ${names.length} skill(s)${agents.length ? ` and ${agents.length} agent(s)` : ""} in catalog '${cat.source}' for supply-chain patterns…`);
|
|
785
|
+
const total = printScan(cat, names).total + printScan(cat, agents, { scanner: scanAgent, kind: "agent" }).total;
|
|
675
786
|
if (!total) {
|
|
676
787
|
console.log("\nNo flagged patterns found.");
|
|
677
788
|
} else {
|
|
@@ -1373,9 +1484,9 @@ function install(args) {
|
|
|
1373
1484
|
console.log(`\nWARNING: third-party catalog '${cat.source}'. Skills are instructions your AI agent will follow — review each installed SKILL.md under ${dest} before first use. Install copied files only; nothing was executed.`);
|
|
1374
1485
|
// Supply-chain scan (RFC-0022): surface risky patterns at the moment of trust. Reports only —
|
|
1375
1486
|
// the files are already copied, and a match is a prompt to read, not a verdict.
|
|
1376
|
-
const scripted = names.filter((s) => fs.existsSync(path.join(skillDir(cat, s), "scripts")));
|
|
1487
|
+
const scripted = names.filter((s) => fs.existsSync(path.join(skillDir(cat, s), "scripts")) || permissionsOf(cat, s));
|
|
1377
1488
|
if (scripted.length) {
|
|
1378
|
-
console.log(`\nWhat
|
|
1489
|
+
console.log(`\nWhat each skill declares it runs and reaches (RFC-0039, RFC-0054):`);
|
|
1379
1490
|
for (const s of scripted) console.log(` ${s}: ${permissionSummary(permissionsOf(cat, s))}`);
|
|
1380
1491
|
}
|
|
1381
1492
|
const { total } = printScan(cat, names, { compact: true });
|
|
@@ -2255,8 +2366,12 @@ function installAgents(args) {
|
|
|
2255
2366
|
if (ide === "copilot") console.log("Invoke with: copilot --agent <name>");
|
|
2256
2367
|
if (ide === "codex") console.log("sandbox_mode is left unset, so each agent inherits the parent session's permissions.");
|
|
2257
2368
|
if (ide === "antigravity") console.log("Emitted with `subagent: true` so invoke_subagent can reach them.");
|
|
2258
|
-
if (cat.source !== BUNDLED)
|
|
2369
|
+
if (cat.source !== BUNDLED) {
|
|
2259
2370
|
console.log(`\nWARNING: third-party catalog '${cat.source}'. An agent is a system prompt your tool will adopt — read it before first use. Install copied files only; nothing was executed.`);
|
|
2371
|
+
const { total } = printScan(cat, names, { compact: true, scanner: scanAgent, kind: "agent" });
|
|
2372
|
+
console.log(total ? `\n${total} pattern(s) worth reading before you trust these agents. Full detail: skilldrop scan --agent --from ${cat.source}`
|
|
2373
|
+
: "\nSupply-chain scan: no flagged patterns.");
|
|
2374
|
+
}
|
|
2260
2375
|
}
|
|
2261
2376
|
|
|
2262
2377
|
function uninstallAgents(args) {
|
package/catalogue.json
CHANGED
|
@@ -64,7 +64,8 @@
|
|
|
64
64
|
"backlog-triage",
|
|
65
65
|
"tracker-brief-sync",
|
|
66
66
|
"dependency-upgrade-review",
|
|
67
|
-
"codebase-tour"
|
|
67
|
+
"codebase-tour",
|
|
68
|
+
"refactor-plan"
|
|
68
69
|
]
|
|
69
70
|
},
|
|
70
71
|
"build-on-the-claude-api": {
|
|
@@ -89,7 +90,8 @@
|
|
|
89
90
|
"launch-readiness",
|
|
90
91
|
"cloud-cost-review",
|
|
91
92
|
"delivery-metrics-report",
|
|
92
|
-
"terraform-plan-review"
|
|
93
|
+
"terraform-plan-review",
|
|
94
|
+
"rollout-plan"
|
|
93
95
|
]
|
|
94
96
|
},
|
|
95
97
|
"explain-it-to-decision-makers": {
|
|
@@ -42,12 +42,12 @@
|
|
|
42
42
|
"type": "object",
|
|
43
43
|
"additionalProperties": false,
|
|
44
44
|
"required": ["network", "commands", "files"],
|
|
45
|
-
"description": "RFC-0039 (OWASP AST03, AST10). What the skill's scripts do, declared so a reader and `skilldrop scan` can compare it with
|
|
45
|
+
"description": "RFC-0039, RFC-0054 (OWASP AST03, AST10). What the skill's scripts do and what its instructions tell the agent to run, declared so a reader and `skilldrop scan` can compare it with the code and with every fenced shell block in SKILL.md and its reference files. Required for every skill that ships scripts/, and for any skill whose instructions run a program or reach a host.",
|
|
46
46
|
"properties": {
|
|
47
47
|
"network": { "type": "array", "items": { "type": "string" },
|
|
48
|
-
"description": "Hosts the scripts contact, e.g. api.figma.com. Empty means none." },
|
|
48
|
+
"description": "Hosts the scripts contact or the instructed commands reach, e.g. api.figma.com. Empty means none." },
|
|
49
49
|
"commands": { "type": "array", "items": { "type": "string" },
|
|
50
|
-
"description": "External programs the scripts run, e.g. pdftotext. \"*\" means commands the user configures. Empty means none." },
|
|
50
|
+
"description": "External programs the scripts run or the instructions tell the agent to run, e.g. pdftotext, terraform. \"*\" means commands the user configures. Empty means none." },
|
|
51
51
|
"files": { "type": "string", "enum": ["none", "named-paths", "project", "anywhere"],
|
|
52
52
|
"description": "What the scripts write: nothing; only paths the user names; inside the current project; or anywhere." },
|
|
53
53
|
"notes": { "type": "string", "description": "Anything a reviewer should know that the fields can't say." }
|
package/model-routing.json
CHANGED
|
@@ -374,6 +374,10 @@
|
|
|
374
374
|
"tier": "standard",
|
|
375
375
|
"rationale": "Structural analysis of prompt with threshold arithmetic and pattern matching — above light, no need for heavy."
|
|
376
376
|
},
|
|
377
|
+
"refactor-plan": {
|
|
378
|
+
"tier": "standard",
|
|
379
|
+
"rationale": "Sequencing behaviour-preserving steps follows known patterns (characterise, parallel change, strangle); escalate via ambiguous-input for contested rewrites."
|
|
380
|
+
},
|
|
377
381
|
"release-notes": {
|
|
378
382
|
"tier": "standard",
|
|
379
383
|
"rationale": "Classifies commits and rewrites them for a reader. The audience translation needs judgment; escalate via large-input on very long ranges."
|
|
@@ -402,6 +406,10 @@
|
|
|
402
406
|
"tier": "standard",
|
|
403
407
|
"rationale": "Reads many postings against one resume and sorts them. Each comparison is bounded; the rule (no posting read, no verdict) is mechanical."
|
|
404
408
|
},
|
|
409
|
+
"rollout-plan": {
|
|
410
|
+
"tier": "standard",
|
|
411
|
+
"rationale": "Plan synthesis on a strong template; the arithmetic of how long each exposure step must run is done by its script, not the model."
|
|
412
|
+
},
|
|
405
413
|
"runbook-generator": {
|
|
406
414
|
"tier": "standard",
|
|
407
415
|
"rationale": "Operational doc from service facts. Bounded synthesis."
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "skilldrop-cli",
|
|
3
|
-
"version": "0.16.
|
|
3
|
+
"version": "0.16.32",
|
|
4
4
|
"description": "Portable AI-agent skills, measured against the agent without them. ADRs, PRDs, runbooks, decks and reviews, installable into Claude Code, Cursor, Kiro, and any IDE that takes custom instructions. One command per skill or per role-based pack.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Sanjay Ananthanarayan",
|
|
@@ -32,7 +32,17 @@
|
|
|
32
32
|
"png",
|
|
33
33
|
"render"
|
|
34
34
|
],
|
|
35
|
-
"permissions": {
|
|
35
|
+
"permissions": {
|
|
36
|
+
"network": [
|
|
37
|
+
"registry.npmjs.org"
|
|
38
|
+
],
|
|
39
|
+
"commands": [
|
|
40
|
+
"mmdc",
|
|
41
|
+
"npm"
|
|
42
|
+
],
|
|
43
|
+
"files": "named-paths",
|
|
44
|
+
"notes": "mmdc (mermaid-cli) renders SVG, PNG and PDF if it is installed. Without it the script writes an HTML page that loads Mermaid from cdn.jsdelivr.net when opened; the script itself makes no network calls. Its instructions suggest installing mermaid-cli with `npm install -g @mermaid-js/mermaid-cli`, which reaches the npm registry."
|
|
45
|
+
},
|
|
36
46
|
"model": {
|
|
37
47
|
"tier": "standard",
|
|
38
48
|
"rationale": "The script lints and renders deterministically, but fixing a broken diagram means reading the author's intent and rewriting syntax without changing nodes, edges or labels, and knowing when to ask instead."
|
|
@@ -14,17 +14,19 @@ being folded into either neighbour.
|
|
|
14
14
|
|
|
15
15
|
| # | Stage | Type | Skills | Gate |
|
|
16
16
|
|---|---|---|---|---|
|
|
17
|
-
| 1 | `plan` | generate | [`migration-plan`](../../skills/migration-plan/SKILL.md) | — |
|
|
17
|
+
| 1 | `plan` | generate | [`migration-plan`](../../skills/migration-plan/SKILL.md), [`rollout-plan`](../../skills/rollout-plan/SKILL.md) | — |
|
|
18
18
|
| 2 | `ready` | verify | [`launch-readiness`](../../skills/launch-readiness/SKILL.md) | **G3** review |
|
|
19
19
|
| 3 | `announce` | generate | [`release-notes`](../../skills/release-notes/SKILL.md) | — |
|
|
20
20
|
| 4 | `go` | gate | [`council-review`](../../../core/skills/council-review/SKILL.md) | **G3.1** human |
|
|
21
21
|
|
|
22
22
|
## How to run this loop
|
|
23
23
|
|
|
24
|
-
1. **Plan the rollout and the way back.**
|
|
25
|
-
rollback for each
|
|
26
|
-
|
|
27
|
-
|
|
24
|
+
1. **Plan the rollout and the way back.** A change to data, a schema or a platform →
|
|
25
|
+
`migration-plan`: the phases, a tested rollback for each, and at most one named point of
|
|
26
|
+
no return. A code change released behind a flag or a canary → `rollout-plan`: the exposure
|
|
27
|
+
steps, the metric and threshold that decide each one, how long each must run to detect a
|
|
28
|
+
regression, and what turning the flag off doesn't undo. A change that does both gets both.
|
|
29
|
+
The plan can be short, but it has to exist.
|
|
28
30
|
2. **Judge readiness at G3.** `launch-readiness` lists the change's own failure modes, then
|
|
29
31
|
checks rollback, detection, response, measured targets, data and privacy, comms and
|
|
30
32
|
exposure, each with evidence. `REVISE` returns to `plan`, and the report names which gap to
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$comment": "RFC-0031. Covers the step between build (ends at merge, G2.1) and operate (starts at instrument). Its gates are G3 and G3.1: the whole number places the loop in the lifecycle (discover G0 through operate G4), and a decimal is a second gate inside the same loop (RFC-0044).",
|
|
3
3
|
"name": "release",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.4.0",
|
|
5
5
|
"description": "Take merged code to live users through a planned, reversible rollout — plan the phases and the rollback, judge readiness on evidence, draft the release notes, then a human makes the go/no-go call. Use when the user has merged a change and asks how to roll it out, whether they are ready to launch, or for a go/no-go decision.",
|
|
6
6
|
"entrypoint": "LOOP.md",
|
|
7
7
|
"kind": "loop",
|
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
"id": "plan",
|
|
12
12
|
"type": "generate",
|
|
13
13
|
"intent": "Decide the rollout phases, the rollback for each, and the one point of no return, before anything reaches users.",
|
|
14
|
-
"skills": ["migration-plan"]
|
|
14
|
+
"skills": ["migration-plan", "rollout-plan"]
|
|
15
15
|
},
|
|
16
16
|
{
|
|
17
17
|
"id": "ready",
|
|
@@ -7,7 +7,7 @@ jobs:
|
|
|
7
7
|
- uses: actions/checkout@v4
|
|
8
8
|
- uses: actions/setup-go@v5
|
|
9
9
|
with:
|
|
10
|
-
go-version: "1.
|
|
10
|
+
go-version: "1.25"
|
|
11
11
|
- run: make lint
|
|
12
12
|
- run: make test
|
|
13
13
|
integration:
|
|
@@ -24,5 +24,5 @@ jobs:
|
|
|
24
24
|
- uses: actions/checkout@v4
|
|
25
25
|
- uses: actions/setup-go@v5
|
|
26
26
|
with:
|
|
27
|
-
go-version: "1.
|
|
27
|
+
go-version: "1.25"
|
|
28
28
|
- run: make test-integration
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: dependency-upgrade-review
|
|
3
|
-
description: Give a merge verdict on a dependency-upgrade pull request (Dependabot, Renovate or a hand-made bump) across npm, pip, Go, Cargo and Maven lockfiles and GitHub Actions pins. Reads the release notes for every skipped version, lists what the lockfile changes beyond the PR title (transitive bumps, new and removed packages, install scripts, registry changes), checks
|
|
3
|
+
description: Give a merge verdict on a dependency-upgrade pull request (Dependabot, Renovate or a hand-made bump) across npm, pip, Go, Cargo and Maven lockfiles and GitHub Actions pins. Reads the release notes for every skipped version, lists what the lockfile changes beyond the PR title (transitive bumps, new and removed packages, install scripts, registry changes), checks each SHA pin against its release tag and against the PR's other pins, checks raised runtime floors against the images and runners that install it, says whether CI ran the changed code, traces what each caller of a changed API now does and writes the rollback. The reply leads with MERGE, MERGE AFTER CHECKS or DO NOT MERGE and the reasons that decide it. Use when the user says "review this Dependabot PR", "is this Renovate bump safe to merge", "can I merge this dependency upgrade", "check this lockfile diff", or "is this action SHA pin right".
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# dependency-upgrade-review
|
|
7
7
|
|
|
8
|
-
A dependency bump is someone else's code entering the repo under a one-line title. This skill reads it the way a careful maintainer does: every release between the old and new version, every line of the lockfile, every pin, and the tests that would have caught a break. The verdict follows from those checks, not from a green badge. Reviewing the repo's own code changes is `pre-merge-review`; writing the notes for a release of this project is `release-notes`; planning a framework migration across many PRs is `migration-plan`.
|
|
8
|
+
A dependency bump is someone else's code entering the repo under a one-line title. This skill reads it the way a careful maintainer does: every release between the old and new version, every line of the lockfile, every pin, every place the code installs and runs, and the tests that would have caught a break. The verdict follows from those checks, not from a green badge. The reply is short: the verdict and what decides it come first, and the checks that found nothing take one line. Reviewing the repo's own code changes is `pre-merge-review`; writing the notes for a release of this project is `release-notes`; planning a framework migration across many PRs is `migration-plan`.
|
|
9
9
|
|
|
10
10
|
## How to respond
|
|
11
11
|
|
|
@@ -15,11 +15,13 @@ A dependency bump is someone else's code entering the repo under a one-line titl
|
|
|
15
15
|
- **Use only what you were given.** Don't add facts, names, numbers, incidents, history, steps or sections the input doesn't contain. What you need and don't have is left out of the artifact and listed once at the end under "To confirm".
|
|
16
16
|
- **Deliver from what you have.** When the request gives you something to work on, state your assumptions in a line and produce the result. When it gives you nothing to work on, ask for it in one or two plain sentences and say what you will do once you have it.
|
|
17
17
|
- **Write for someone who has never heard of this skill.** No skill names, no paths or scripts from this folder, no internal terms, and nothing about how the run was set up. A next step is one plain sentence at the end that describes the work.
|
|
18
|
-
- **For this skill:** A check nobody ran is an open check, written as the command that runs it. It is never reported as passed, and "CI is green" closes only the checks CI actually ran.
|
|
18
|
+
- **For this skill:** A check nobody ran is an open check, written as the command that runs it. It is never reported as passed, and "CI is green" closes only the checks CI actually ran. Say what was not checked and the command that checks it; never narrate the session (what was blocked, denied or unavailable).
|
|
19
19
|
|
|
20
|
-
1
|
|
20
|
+
Steps 1 to 10 are the review. Step 11 is the reply, and most of what the review turns up stays out of it.
|
|
21
21
|
|
|
22
|
-
|
|
22
|
+
1. **Get the diff, the notes and the code, asking once.** Three inputs: the PR diff covering manifests, lockfiles and workflow files (`gh pr diff 212` or `git diff origin/main...HEAD`); the release notes or changelog for every version between old and new; the repo, or at least the files that import the package and the CI workflow files. Also read what decides where the code installs and runs: the Dockerfile or base image, every workflow that installs dependencies (release, deploy and scheduled ones too, not only the PR's), `.nvmrc` / `.python-version` / `engines`, and any note in the PR body of the runtime the lockfile was generated on. When the PR has more than one commit, note who made each; a hand-made commit on a bot's PR is read first. Ask for what is missing in one message. With network access, fetch missing notes; without it, each missing release becomes an open check.
|
|
23
|
+
|
|
24
|
+
2. **Run the diff script first.** It lists every changed dependency with old → new, semver class, direct or transitive, and flags added and removed packages, install hooks, native-code hints, registry changes, licence changes, runtime requirement changes, pins, and one tag pinned to different commits, then prints a floor verdict.
|
|
23
25
|
```bash
|
|
24
26
|
# Claude Code
|
|
25
27
|
python3 "${CLAUDE_SKILL_DIR}/scripts/upgrade_diff.py" pr.diff --title "Bump quillmark from 3.4.1 to 4.1.0"
|
|
@@ -28,50 +30,65 @@ A dependency bump is someone else's code entering the repo under a one-line titl
|
|
|
28
30
|
```
|
|
29
31
|
It reads stdin with `-` (`gh pr diff 212 | python3 scripts/upgrade_diff.py -`). If it reports version lines out of context, re-run on `git diff -U12`. If no script can run, read every lockfile hunk by hand with the per-ecosystem notes in [`reference.md`](reference.md).
|
|
30
32
|
|
|
31
|
-
3. **List everything the lockfile changes beyond the title.** The title names one package; the lockfile is the truth.
|
|
33
|
+
3. **List everything the lockfile changes beyond the title.** The title names one package; the lockfile is the truth. Find each of: transitive bumps (with class), packages added, packages removed, new install scripts (`hasInstallScript: true`, a `preinstall` / `install` / `postinstall` / `prepare` hook, pnpm `requiresBuild: true`), new native or binary dependencies (`prebuild-install`, `node-gyp`, a Cargo `-sys` crate, wheels for only some platforms), a changed `resolved` / `source` / index URL, a raised `engines` or `python-versions`, and a maintainer or publisher change when the input shows one. ✅ *"Also changes: `entity-codec` 2.1.0 → 3.0.2 (major, licence MIT → MPL-2.0), adds `quill-native-highlight` 1.0.0 with an install script, removes `html-escape-lite`."* ❌ *"Bumps quillmark to 4.1.0."*
|
|
32
34
|
|
|
33
35
|
4. **Class each jump.** Major, minor or patch from the version numbers. Below 1.0 the minor is the breaking number: 0.8.3 → 0.9.0 is a breaking jump, and below 0.1 so is the patch. A Go module path gaining `/v2` is a new module, not a bump. A prerelease target (`-rc.1`, `-beta`) or a downgrade is never a plain MERGE. A moving major tag (`@v2` → `@v3`) spans every v2.x after the one in use plus v3.0.0 to the current v3.
|
|
34
36
|
|
|
35
|
-
5. **Read every release between old and new, not only the target.** For each direct dependency,
|
|
37
|
+
5. **Read every release between old and new, not only the target.** For each direct dependency, read every release after the old version up to and including the new one, and pull five things from each: breaking changes, dropped runtime versions, deprecations, security fixes, licence changes. Transitive majors, 0.x breaking jumps and new packages get the same reading; transitive patch and minor bumps are listed, not read. Name each release whose notes are missing. Copy version numbers exactly as the notes give them: "deprecated since 2.2, warns from 2.5.0" is two facts. Note any migration aid an intermediate release offers (an opt-in flag, a compatibility mode); it is the staged path. A security fix is a reason to take the upgrade; when the full jump is blocked, name the smallest version that carries the fix as the alternative.
|
|
38
|
+
|
|
39
|
+
6. **Search for every changed API, then trace each caller.** For each removed or renamed function, changed signature, changed default, changed return value and changed error, search and report `file:line` for each hit, or "no hits". Give the search (`git grep -n -E "renderInline|setOptions|render\(" -- '*.js' '*.ts'`); with no repo in hand, give that command as an open check and claim neither hits nor no hits. Then read the calling function and say what it does under the new behaviour, and the fix in this repo's code:
|
|
40
|
+
- A return value that became an exception skips every `if result is None` / `== -1` branch and escapes the retry or fallback built on it.
|
|
41
|
+
- A tuple or dict that became an object breaks the unpacking or the key access.
|
|
42
|
+
- A changed default hits every call that doesn't pass the option.
|
|
43
|
+
- An import inside a function fails only when that function runs, never at import time or test collection.
|
|
44
|
+
✅ *"`load_profile` (`app/cache.py:31`) falls back to the database when `cache.get()` returns `None`; from 5.0.0 `get()` raises `CacheMiss`, so every miss escapes as an error and the fallback never runs. Catch `CacheMiss` there, or move to 4.6 first with `strict_miss=True`."* ❌ *"`get()` behaviour changed in 5.0.0; check usages."*
|
|
36
45
|
|
|
37
|
-
|
|
46
|
+
7. **Check every runtime floor against where the code runs.** A raised minimum (Python `>=3.10`, Node `engines >=20`, a Go directive, an MSRV, wheels built only for some interpreter versions, an action needing a newer runner) is compared with the Dockerfile `FROM`, each workflow's `setup-*` version and matrix, `.nvmrc` / `.python-version`, self-hosted runner versions, and the runtime the lockfile was generated on. CI and the lock tool running a newer runtime than production is how a green PR breaks the image build. ✅ *"`deploy/Dockerfile:1` builds on `node:18-alpine` and 6.0 needs Node >=20; CI runs Node 22 (`ci.yml:14`), so the PR passed and the image build will fail at `npm ci`."*
|
|
38
47
|
|
|
39
|
-
|
|
48
|
+
8. **Verify every pin the PR changes; an unverified SHA is an open check.** A SHA-pinned action (or any dependency pinned to a commit) must be the commit its release tag points to:
|
|
40
49
|
```bash
|
|
41
50
|
git ls-remote https://github.com/<owner>/<repo> refs/tags/<tag> 'refs/tags/<tag>^{}'
|
|
42
51
|
```
|
|
43
|
-
For an annotated tag the `^{}` line is the commit;
|
|
52
|
+
For an annotated tag the `^{}` line is the commit; for a lightweight tag the single line is. Not run means unverified, never "looks right": a SHA from a fork also resolves under the upstream path. Compare the PR's pins with each other: one action and tag pinned to two different SHAs means at most one is the tag's commit, and the suspect is the one changed by hand or in the job with secrets. A SHA with no version comment is an unknown release. A tag-only pin (`@v3`) is weaker than a SHA, because the tag can be moved after review. Say what the job hands the action: a `secrets.*` token, `id-token: write` or `contents: write` raises the stakes. Pins the PR does not change are not part of this review.
|
|
44
53
|
|
|
45
|
-
|
|
54
|
+
9. **Say whether CI ran the changed code.** A green check proves what it ran and nothing else. For each changed package or action, name the workflow job and test that exercise it, or say nothing does. The usual gaps: a workflow triggered only by a tag push, a release, a schedule or `workflow_dispatch`; a file no test imports (a step 6 hit with no test); a mock that returns the old behaviour (a fake client that never times out tests the old contract); a PR job that installs differently from the release job (`npm ci --ignore-scripts` on the PR, plain `npm ci` in release means a new install script first runs in the job holding the secrets). ✅ *"`ci / test` runs `test/note.test.js`, which renders Markdown with no raw HTML, so the `sanitize` change is untested; nothing imports `src/render/comment.js`."* ❌ *"All checks pass."*
|
|
46
55
|
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
- **DO NOT MERGE** when any holds: a removed or changed API is still used and the PR does not fix it; a new install script or native binary download nobody has confirmed is expected; a dependency source or registry moved to an unexplained host; a SHA that does not match its tag; a pin weakened (SHA to tag, tag to branch); a licence the project cannot ship; a target version an advisory names as vulnerable; a red CI check.
|
|
51
|
-
- **MERGE AFTER CHECKS** when none of those holds but any of these does: an unverified SHA; a major or 0.x breaking jump; a release whose notes were not read; changed code no CI job runs; a deprecated API still used; a new package or transitive major; a licence change that needs sign-off; a moved tag-only pin; no repo to search. Each check is a command or a concrete action.
|
|
56
|
+
10. **Write the rollback, then derive the verdict.** Rollback: `git revert` of the merge commit, or the pin-back for the ecosystem (`npm install quillmark@3.4.1`, `poetry add tidyparse@2.2.0`, `go get <module>@v0.8.3 && go mod tidy`, the old `uses:` line and SHA) with the lockfile regenerated, plus what a revert does not undo (a lockfile format migration, data or cache the new version wrote, an artifact already published or deployed, a security fix that rolling back reopens; then pin back to the first fixed version). Per-ecosystem commands are in [`reference.md`](reference.md). Then exactly one verdict per PR:
|
|
57
|
+
- **DO NOT MERGE** when any holds: a removed or changed API is still used and the PR does not fix it; a runtime floor above what production or the release job runs; a new install script or native binary download nobody has confirmed is expected; a dependency source moved to an unexplained host; a SHA that does not match its tag, or two SHAs for one tag; a pin weakened (SHA to tag, tag to branch); a licence the project cannot ship; a target version an advisory names as vulnerable; a red CI check.
|
|
58
|
+
- **MERGE AFTER CHECKS** when none of those holds but any of these does: an unverified SHA; a major or 0.x breaking jump; a release whose notes were not read; changed code no CI job runs; a deprecated API still used; a new package or transitive major; a licence change that needs sign-off; a moved tag-only pin; no repo to search.
|
|
52
59
|
- **MERGE** only when every check above is closed, with the evidence in the reply.
|
|
53
|
-
The script's floor is a minimum
|
|
54
|
-
|
|
55
|
-
11. **Reply
|
|
60
|
+
The script's floor is a minimum; raise it on what steps 5 to 9 find. Lower DO NOT MERGE only when the user confirms the install script or new source is expected, and say so.
|
|
61
|
+
|
|
62
|
+
11. **Reply short, verdict first, in one message.** Only what changes the decision or the fix goes in.
|
|
63
|
+
- **First line:** the verdict and the one to three reasons that decide it, in plain words. With two PRs, one line each. ✅ *"DO NOT MERGE. 6.0 needs Node 20 and the production image runs 18, and two call sites break on removed APIs."* ❌ *"Here is my review of the quillmark upgrade."*
|
|
64
|
+
- **🟥 Blockers, numbered, the one that breaks first on top.** Each: what breaks, `file:line`, why the green check missed it, and the fix in this repo, in two to four lines. When an intermediate release offers a migration aid, one line after the blockers gives the staged path. The count in the first line matches the list.
|
|
65
|
+
- **🟧 Before merge:** each open check as the command or action that closes it. An unverified SHA is written as the command and the SHA it must equal.
|
|
66
|
+
- **Also in the lockfile:** one line naming every package added, removed or moved beyond the title, with install scripts and source changes called out. Not a table.
|
|
67
|
+
- **Rollback:** one or two lines, the command and what a revert leaves behind.
|
|
68
|
+
- **To confirm** only for what only the user can answer (is this install script expected?); omit it when empty.
|
|
69
|
+
- **Last line:** one sentence offering the next piece of work (the code fixes, the pin change).
|
|
70
|
+
No tables, no recited changelog, no pins or files the PR does not touch, no empty sections. A clean MERGE fits in about 120 words; a blocked PR with three blockers in about 300.
|
|
56
71
|
|
|
57
72
|
**Non-interactive runs** (subagent, CI, headless): with no diff, emit `BLOCKED: need the PR diff covering the manifests, lockfiles and workflow files (gh pr diff <number>)`. Missing release notes, repo or CI files do not block: each becomes a named open check, the verdict is at best MERGE AFTER CHECKS, and the gap is stated first as `[assumption] notes for <versions> not read` or `[assumption] CI coverage unknown, treated as none`.
|
|
58
73
|
|
|
59
74
|
## Useful references in this skill
|
|
60
75
|
|
|
61
|
-
- [`reference.md`](reference.md): reading each ecosystem's lockfile diff, where install scripts and
|
|
76
|
+
- [`reference.md`](reference.md): reading each ecosystem's lockfile diff, where install scripts, sources and runtime floors show up, how to list the releases between two versions, SHA verification details, tracing behaviour changes, and per-ecosystem rollback commands with what a revert leaves behind
|
|
62
77
|
- [`scripts/upgrade_diff.py`](scripts/upgrade_diff.py): stdlib parser for a unified diff of manifests, lockfiles and workflow files; prints changes, flags, pin checks and a floor verdict (exit 2 on bad input)
|
|
63
|
-
- [`examples/ledgerline-tidyparse.md`](examples/ledgerline-tidyparse.md): worked example, a Poetry patch bump
|
|
78
|
+
- [`examples/ledgerline-tidyparse.md`](examples/ledgerline-tidyparse.md): worked example, a Poetry patch bump that earns MERGE, and a blocked reply in the same shape
|
|
64
79
|
|
|
65
80
|
## Quality bar
|
|
66
81
|
|
|
67
|
-
- **
|
|
68
|
-
- **Every release between old and new is accounted for:** read, with its breaking changes, deprecations, security fixes and licence changes
|
|
69
|
-
- **Everything the lockfile changes beyond the title is listed
|
|
70
|
-
- **Every changed API has a search result:**
|
|
71
|
-
- **Every
|
|
72
|
-
- **
|
|
82
|
+
- **The first line is the verdict and the reasons that decide it,** one verdict per PR, following from the checks. Any unverified SHA, added install script, or removed API still in use rules out MERGE.
|
|
83
|
+
- **Every release between old and new is accounted for:** read, with its breaking changes, runtime drops, deprecations, security fixes and licence changes, or named as unread.
|
|
84
|
+
- **Everything the lockfile changes beyond the title is listed,** in one line: transitive bumps, added and removed packages, install scripts, native code, source and runtime changes.
|
|
85
|
+
- **Every changed API has a search result and every hit is traced:** what the caller does now and the fix, or the search command when there is no repo.
|
|
86
|
+
- **Every raised runtime floor is compared** with the base image, the CI runtime, the runners and the lockfile tool's runtime.
|
|
87
|
+
- **Every changed SHA pin is verified or given as the `git ls-remote` command,** and the PR's pins are compared with each other.
|
|
88
|
+
- **CI coverage is named per changed package:** the job and test that exercise it, or "nothing does", and how each workflow installs.
|
|
73
89
|
- **A rollback with the exact command** and what a revert does not undo.
|
|
74
|
-
- **
|
|
90
|
+
- **Nothing reads as checked that was not run,** and nothing describes the session's tools or network.
|
|
91
|
+
- **No invented versions, advisories, APIs, files or test names;** version numbers are copied from the notes.
|
|
75
92
|
|
|
76
93
|
## When to use this skill
|
|
77
94
|
|
|
@@ -91,11 +108,12 @@ A dependency bump is someone else's code entering the repo under a one-line titl
|
|
|
91
108
|
|
|
92
109
|
## Anti-patterns to avoid
|
|
93
110
|
|
|
111
|
+
- ❌ **The report reply.** A package table, a rollback essay and an empty "To confirm" around a verdict the reader has to dig for. Verdict and reasons first; the rest in a line each.
|
|
94
112
|
- ❌ **Reading only the target version's notes.** The breaking change usually lives in a release the PR skipped.
|
|
95
|
-
- ❌ **Trusting the title.** "Bump X from 1 to 2" says nothing about the
|
|
96
|
-
- ❌ **
|
|
97
|
-
- ❌ **
|
|
98
|
-
- ❌ **
|
|
99
|
-
- ❌ **
|
|
100
|
-
- ❌ **
|
|
113
|
+
- ❌ **Trusting the title.** "Bump X from 1 to 2" says nothing about the transitive packages that moved with it.
|
|
114
|
+
- ❌ **Finding the change but not the caller.** "`fetch()` now raises" is half a finding; "so the retry loop never retries" is the finding.
|
|
115
|
+
- ❌ **Trusting CI's runtime.** CI on Python 3.11 says nothing about an image built on 3.9.
|
|
116
|
+
- ❌ **Calling a SHA pin fine because it is a SHA.** A SHA proves nothing until it matches the release tag, and two SHAs for one tag cannot both match.
|
|
117
|
+
- ❌ **"CI is green, ship it."** Release workflows, publish steps, lazy imports and mocked paths never ran on the PR.
|
|
118
|
+
- ❌ **Explaining the session.** "The network was blocked, so I couldn't check" is about the run, not the PR; give the command.
|
|
101
119
|
- ❌ **Hedged verdicts.** "Probably safe" is not a verdict; pick one of the three and give the reason.
|