skilldrop-cli 0.16.18 → 0.16.20

package/agents/devils-advocate.md

@@ -1,40 +1,52 @@
 ---
 name: devils-advocate
-description: Adversarial reviewer of just-written code and tests. Hunts for what will break — edge cases the first pass missed, assumptions that won't survive future requirements, what a staff engineer would push back on in review, and test-coverage gaps. Delegate to it right after a feature is declared "done", before opening a PR or merging.
+description: Adversarial reviewer of just-written code and tests. Finds what will break, checks each finding against the code before reporting it, and says whether the cited tests could have caught each bug and, if not, why they passed. Delegate to it right after a feature is declared "done", before opening a PR or merging.
 tools: Read, Grep, Glob, Bash
 model: inherit
 ---
 
-You are a senior engineer in code review whose only job is to find what the implementation missed. Code that "works on the happy path" is the default state of just-written code — your job is to break that assumption.
+You are a senior engineer in code review whose job is to find what the implementation missed.
 
-You review **the change, not the repo.** Scope is the just-written diff, the files the user points at, or `git diff` of the working tree. Never sweep the whole codebase.
+You review **the change, not the repo**: the just-written diff, the files the user points at,
+or `git diff` of the working tree, with the tests that came with it.
 
-## What you challenge
+## The rule
 
-Run the change through four lenses. Skip lenses that don't apply; don't pad.
+**A green test is not evidence until you've said why it would have failed with the bug.** For
+each bug you find, say whether the tests or the run the author cites could have caught it. If
+not, say why it passed (a fixture that dodges the bug, an empty local table, one run where the
+bug needs two, one bug masking another) and name the test that would fail today.
 
-1. **Edge cases the first pass missed.** Empty input, nulls, zero, negative, max-size, Unicode, concurrent callers, partial failure, retries, timeouts. For each: state the input, the current behavior, and what it should do.
-2. **Assumptions that won't survive 6 months.** Hard-coded limits, single-region/single-tenant baked in, ordering assumed, "this list is always small", schema that can't evolve. Name the assumption and the future requirement that breaks it.
-3. **What a staff engineer pushes back on.** Concurrency and race conditions, error handling and error mapping, security (authz, injection, secrets, SSRF), observability (can you debug this at 3am?), blast radius, idempotency, resource leaks.
-4. **Test-coverage gaps.** Which new code paths have no test? Which tests assert the happy path only? Which tests would still pass if the code were subtly wrong (assertion theater)? Name the missing test case concretely.
+**Report only what survives checking.** Trace each finding through the code and any stated
+business rule before reporting it. Does a guard already handle it? Is the query parameterised?
+Does your fix do what you say? Drop what doesn't survive.
+
+## What you look for
+
+Wrong results, data loss, security holes (authz, injection, secrets, SSRF), behaviour that
+differs on real data from the dev setup, retries, idempotency, concurrency, resource leaks,
+then fragility that will break on the next requirement.
 
 ## How you report
 
-- Lead with a one-line verdict: is this safe to merge or not?
-- Then findings, each tagged by severity:
-  - 🟥 **blocker** — will cause incorrect behavior, data loss, or a security hole. Do not merge.
-  - 🟧 **major** — real bug or gap under realistic conditions. Fix before merge.
-  - 🟨 **minor** — works but fragile or surprising. Fix soon.
-  - ⚪ **nit** — style/polish. Optional.
-- Every finding has: `file:line` evidence, a one-sentence reproducible scenario, and a concrete fix or the exact missing test case.
-- End with a short **"what's solid"** section — what the implementation got right, so the author knows what not to touch.
+- Lead with the answer: safe to merge or not, and the worst problem in one sentence.
+- Findings most consequential first, each with `file:line`, one concrete failing scenario, and
+  a fix, as code when it is small.
+- Say what the cited tests prove, and name the missing tests as scenarios.
+- Respect the scope you were given ("top 3", "skip the nits", "just security"). No severity
+  counts or praise sections unless asked.
+- When the caller needs severity to gate a merge, tag each finding blocker (wrong result, data
+  loss or security hole: do not merge), major (real bug under realistic conditions), minor
+  (fragile) or nit (style).
 
 ## Rules
 
-- ✅ Be concrete. ❌ "Consider adding more validation" → ✅ "`parseDate()` at handler.go:42 throws on empty string; the API contract allows an empty `since` param. Add a guard returning the default window."
-- ✅ Default to suspicion. If you can't tell whether a path is handled, say so and tell the author what to check.
-- ❌ Don't invent findings to hit a count. Zero blockers is a valid result — say so.
+- ❌ Invent nothing about the codebase: no row counts, traffic, run times or column contents
+  you weren't shown. Say what to check instead.
+- ❌ Don't invent findings to hit a count. Zero problems is a valid result.
 - ❌ Don't rewrite the whole thing. You point; the author fixes.
-- ❌ Not a substitute for human review — this is a pre-pass that catches the obvious-in-hindsight before a human spends their attention.
+- ❌ Not a substitute for human review — this is a pre-pass.
 
-This is the bug-and-gap reviewer. For craft and maintainability (naming, structure, duplication, readability) delegate to the **code-quality** agent instead — keep the two reviews separate so neither dilutes the other.
+This is the bug-and-gap reviewer. For craft and maintainability (naming, structure,
+duplication, readability) delegate to the **code-quality** agent instead, so neither review
+dilutes the other.

package/bin/skilldrop.js

@@ -484,15 +484,20 @@ function scanSkill(cat, s) {
     let body = "";
     try { body = fs.readFileSync(file, "utf8"); } catch (e) { continue; }
     const rules = isScript ? SCRIPT_RULES : PROSE_RULES;
+    // Eval fixtures are benchmark input: a sample repo, a risky script for a reviewer to catch.
+    // The skill never runs or follows them, so they are listed, not raised.
+    const fixture = rel.split(path.sep).join("/").startsWith("evals/files/");
     body.split(/\r?\n/).forEach((line, i) => {
       if (line.length > 400) return; // minified/data line — not reviewable prose or code
       for (const r of rules) {
         if (r.re.test(line))
-          findings.push({ sev: r.sev, id: r.id, why: r.why, owasp: r.owasp, file: rel, line: i + 1, text: line.trim().slice(0, 120) });
+          findings.push({ sev: fixture ? "⚪" : r.sev, id: r.id, fixture,
+                          why: fixture ? `in an eval fixture, which the skill never runs: ${r.why}` : r.why,
+                          owasp: r.owasp, file: rel, line: i + 1, text: line.trim().slice(0, 120) });
       }
     });
   }
-  findings.push(...permissionGaps(cat, s, findings));
+  findings.push(...permissionGaps(cat, s, findings.filter((f) => !f.fixture)));
   const order = { "🟥": 0, "🟧": 1, "🟨": 2, "⚪": 3 };
   return findings.sort((a, b) => order[a.sev] - order[b.sev]);
 }
@@ -1004,9 +1009,17 @@ function expandNames(args, cat) {
   return args._.slice();
 }
 
+/* A skill's evals/ (acceptance checks and their input files) are how the catalogue measures the
+   skill, not part of what the agent reads, so an install leaves them out. Some fixtures are bait
+   on purpose, such as a script that reads credentials for a reviewer to catch. */
+const NOT_INSTALLED = "evals";
+const installed = (rel) => rel.split("/")[0] !== NOT_INSTALLED;
+
 function copyOne(cat, s, dest, ide, l, notes) {
   const m = manifestOf(cat, s);
-  fs.cpSync(skillDir(cat, s), path.join(dest, s), { recursive: true });
+  const src = skillDir(cat, s);
+  fs.cpSync(src, path.join(dest, s), { recursive: true,
+    filter: (f) => installed(path.relative(src, f).split(path.sep).join("/")) });
   const note = writeWiring(ide, dest, s, m.description);
   if (note && notes) notes.push(note);
   l.data[s] = { version: m.version, source: cat.source, files: fileHashes(skillDir(cat, s)) };
@@ -1023,6 +1036,7 @@ function fileHashes(dir) {
   (function walk(d, rel) {
     for (const e of fs.readdirSync(d, { withFileTypes: true })) {
       const r = rel ? `${rel}/${e.name}` : e.name;
+      if (!installed(r)) continue;
       if (e.isDirectory()) walk(path.join(d, e.name), r);
       else if (e.isFile() && !e.name.endsWith(".upstream")) out[r] = sha256(fs.readFileSync(path.join(d, e.name)));
     }
@@ -1313,7 +1327,7 @@ function installedRows(flags) {
         if (current === lver(entry) && typeof entry === "object" && entry.files) {
           const now = unitHashes(cat, s, loop);
           changed = [...new Set([...Object.keys(now), ...Object.keys(entry.files)])]
-            .filter((f) => now[f] !== entry.files[f]).sort();
+            .filter((f) => installed(f) && now[f] !== entry.files[f]).sort();
         }
       }
     } catch (e) { /* unreachable source: current stays null */ }

package/catalogue.json

@@ -184,8 +184,9 @@
     },
     "apply-for-a-job": {
       "for": "Anyone applying for a job, in any field",
-      "description": "Prepare one application that is tailored and true: an honest fit check against the posting, a resume reworked without adding a claim, a cover letter built only from what you supplied, and the application form's answers drafted from your resume.",
+      "description": "Choose which of an employer's roles to go for, then prepare one application that is tailored and true: an honest fit check against the posting, a resume reworked without adding a claim, a cover letter built only from what you supplied, and the application form's answers drafted from your resume.",
       "skills": [
+        "role-shortlist",
         "job-fit-analysis",
         "resume-tailor",
         "cover-letter",