@olegkoval/agent-skills 1.40.0 → 1.41.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/prompts/open-source-publisher.prompt.md +3 -4
- package/.kiro/steering/open-source-publisher.md +3 -4
- package/.windsurf/rules/open-source-publisher.md +3 -4
- package/adapters/claude/olko-github-pr/skills/lekker-review/SKILL.md +15 -17
- package/adapters/claude/olko-github-pr/skills/lekker-review/references/agents/implementation.md +2 -1
- package/adapters/claude/olko-github-pr/skills/lekker-review/references/agents/prover.md +13 -10
- package/adapters/claude/olko-github-pr/skills/lekker-review/references/agents/quality.md +1 -1
- package/adapters/claude/olko-github-pr/skills/lekker-review/references/agents/test-quality.md +7 -0
- package/adapters/claude/olko-github-pr/skills/lekker-review/references/agents/triage-quality.md +1 -1
- package/adapters/claude/olko-github-pr/skills/lekker-review/references/agents/verifier.md +12 -9
- package/adapters/claude/olko-github-pr/skills/lekker-review/references/artifact-page.md +6 -1
- package/adapters/claude/olko-github-pr/skills/lekker-review/references/output-format.md +5 -8
- package/adapters/claude/olko-github-pr/skills/lekker-review/scripts/selftest.mjs +174 -0
- package/adapters/claude/olko-reflection/skills/self-critique/scripts/critique-nudge.mjs +3 -0
- package/adapters/claude/olko-release/skills/open-source-publisher/SKILL.md +4 -5
- package/adapters/cursor/olko-reflection/skills/self-critique/scripts/critique-nudge.mjs +3 -0
- package/adapters/cursor/olko-release/skills/open-source-publisher/SKILL.md +4 -5
- package/adapters/grok/olko-reflection/skills/self-critique/scripts/critique-nudge.mjs +3 -0
- package/adapters/grok/olko-release/skills/open-source-publisher/SKILL.md +4 -5
- package/package.json +1 -1
- package/plugins/olko-apple-kit/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-creative/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-garmin-kit/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-git-tools/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-github-pr/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-github-pr/skills/lekker-review/README.md +9 -9
- package/plugins/olko-github-pr/skills/lekker-review/SKILL.md +15 -17
- package/plugins/olko-github-pr/skills/lekker-review/references/agents/implementation.md +2 -1
- package/plugins/olko-github-pr/skills/lekker-review/references/agents/prover.md +13 -10
- package/plugins/olko-github-pr/skills/lekker-review/references/agents/quality.md +1 -1
- package/plugins/olko-github-pr/skills/lekker-review/references/agents/test-quality.md +7 -0
- package/plugins/olko-github-pr/skills/lekker-review/references/agents/triage-quality.md +1 -1
- package/plugins/olko-github-pr/skills/lekker-review/references/agents/verifier.md +12 -9
- package/plugins/olko-github-pr/skills/lekker-review/references/artifact-page.md +6 -1
- package/plugins/olko-github-pr/skills/lekker-review/references/output-format.md +5 -8
- package/plugins/olko-github-pr/skills/lekker-review/scripts/selftest.mjs +174 -0
- package/plugins/olko-github-pr/skills/lekker-review/workflow.js +217 -75
- package/plugins/olko-obsidian/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-product/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-reflection/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-reflection/skills/self-critique/scripts/critique-nudge.mjs +3 -0
- package/plugins/olko-release/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-release/skills/open-source-publisher/SKILL.md +4 -5
- package/plugins/olko-skill-meta/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-web-ops/.claude-plugin/plugin.json +1 -1
- package/scripts/lib/catalog.mjs +3 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: open-source-publisher
|
|
3
|
-
description: Prepare an open-source repository for polished public publishing. Use when a user asks to publish, open-source, launch, polish, package, brand, or make a GitHub project presentable with a minimal project icon, social preview image, GitHub Pages landing page, standardized README, essential shields, CI/CD quality gates, release automation checks, and optional donation setup. Prefer the external `logo-generator` skill for icon generation when available, and `nolangz/pixel2motion` for optional motion previews of
|
|
3
|
+
description: Prepare an open-source repository for polished public publishing. Use when a user asks to publish, open-source, launch, polish, package, brand, or make a GitHub project presentable with a minimal project icon, social preview image, GitHub Pages landing page, standardized README, essential shields, CI/CD quality gates, release automation checks, and optional donation setup. Prefer the external `logo-generator` skill for icon generation when available, and `nolangz/pixel2motion` for optional motion previews of supported logos or simple artwork.
|
|
4
4
|
license: MIT
|
|
5
5
|
allowed-tools: Bash, Read, Write, Edit, WebSearch, WebFetch
|
|
6
6
|
compatibility: Codex, Claude Code, Cursor, GitHub Copilot, Windsurf, Kiro, and other Agent Skills compatible tools. Requires a writable git repository; browser or image rendering tools are useful for visual validation.
|
|
@@ -44,12 +44,11 @@ Use this skill to audit whether an OSS repository is ready to publish, then help
|
|
|
44
44
|
5. For existing usable pieces, do not propose replacement by default. Ask a change-oriented question only when useful, for example:
|
|
45
45
|
- "You already have a terminal-style GitHub Pages site. Do you want to keep it or restyle it?"
|
|
46
46
|
- "You already have an icon and social card. Do you want a refresh, or should I leave them as-is?"
|
|
47
|
-
- "You already have a static social card. Do you want me to animate it with pixel2motion, or leave it static?"
|
|
48
47
|
- "You already have release automation. Do you want me to audit only, or also tighten it?"
|
|
49
48
|
6. Ask only for choices needed to fix missing or weak pieces:
|
|
50
49
|
- If GitHub Pages is missing or weak, ask for style: `oldschool linux`, `terminal`, `modern`, `brutalist`, `glassmorphism`, `y2k`, `hacker`, or custom.
|
|
51
50
|
- If donation wiring is missing, ask whether to enable it: `none`, `GitHub Sponsors`, `Ko-fi`, `Buy Me a Coffee`, `Open Collective`, `Thanks.dev`, or custom URL.
|
|
52
|
-
- If the repo has a generated
|
|
51
|
+
- If the repo has a generated logo or simple hero artwork supported by `nolangz/pixel2motion`, ask whether to animate it or leave it static.
|
|
53
52
|
- If the repo has no clear product essence, ask for a one-sentence positioning statement.
|
|
54
53
|
7. Implement only approved, missing, or weak work in this order:
|
|
55
54
|
- OSS governance and support files
|
|
@@ -213,8 +212,8 @@ Use `magick` or another renderer when `rsvg-convert` is unavailable.
|
|
|
213
212
|
|
|
214
213
|
Some repos benefit from a short animated preview. Keep this opt-in.
|
|
215
214
|
|
|
216
|
-
-
|
|
217
|
-
-
|
|
215
|
+
- After the user opts in during workflow step 6, animate only the approved generated logo or simple artwork.
|
|
216
|
+
- Use `nolangz/pixel2motion` as the optional dependency for that animation, animated HTML demos, and GIF/video previews.
|
|
218
217
|
- Do not auto-animate by default.
|
|
219
218
|
|
|
220
219
|
## README Standard
|
|
@@ -11,6 +11,9 @@ import { readFileSync } from 'node:fs';
|
|
|
11
11
|
const THRESHOLD = 1500;
|
|
12
12
|
const OFFER_MARKER = /critical-thinking review/i; // the offer line must contain this; doubles as a loop guard
|
|
13
13
|
|
|
14
|
+
/**
|
|
15
|
+
* Exits the hook cleanly when conditions for offering a critique are not met.
|
|
16
|
+
*/
|
|
14
17
|
function bail() { process.exit(0); } // any uncertainty => do nothing, never block the user
|
|
15
18
|
|
|
16
19
|
let raw = '';
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: open-source-publisher
|
|
3
|
-
description: Prepare an open-source repository for polished public publishing. Use when a user asks to publish, open-source, launch, polish, package, brand, or make a GitHub project presentable with a minimal project icon, social preview image, GitHub Pages landing page, standardized README, essential shields, CI/CD quality gates, release automation checks, and optional donation setup. Prefer the external `logo-generator` skill for icon generation when available, and `nolangz/pixel2motion` for optional motion previews of
|
|
3
|
+
description: Prepare an open-source repository for polished public publishing. Use when a user asks to publish, open-source, launch, polish, package, brand, or make a GitHub project presentable with a minimal project icon, social preview image, GitHub Pages landing page, standardized README, essential shields, CI/CD quality gates, release automation checks, and optional donation setup. Prefer the external `logo-generator` skill for icon generation when available, and `nolangz/pixel2motion` for optional motion previews of supported logos or simple artwork.
|
|
4
4
|
license: MIT
|
|
5
5
|
allowed-tools: Bash, Read, Write, Edit, WebSearch, WebFetch
|
|
6
6
|
compatibility: Codex, Claude Code, Cursor, GitHub Copilot, Windsurf, Kiro, and other Agent Skills compatible tools. Requires a writable git repository; browser or image rendering tools are useful for visual validation.
|
|
@@ -44,12 +44,11 @@ Use this skill to audit whether an OSS repository is ready to publish, then help
|
|
|
44
44
|
5. For existing usable pieces, do not propose replacement by default. Ask a change-oriented question only when useful, for example:
|
|
45
45
|
- "You already have a terminal-style GitHub Pages site. Do you want to keep it or restyle it?"
|
|
46
46
|
- "You already have an icon and social card. Do you want a refresh, or should I leave them as-is?"
|
|
47
|
-
- "You already have a static social card. Do you want me to animate it with pixel2motion, or leave it static?"
|
|
48
47
|
- "You already have release automation. Do you want me to audit only, or also tighten it?"
|
|
49
48
|
6. Ask only for choices needed to fix missing or weak pieces:
|
|
50
49
|
- If GitHub Pages is missing or weak, ask for style: `oldschool linux`, `terminal`, `modern`, `brutalist`, `glassmorphism`, `y2k`, `hacker`, or custom.
|
|
51
50
|
- If donation wiring is missing, ask whether to enable it: `none`, `GitHub Sponsors`, `Ko-fi`, `Buy Me a Coffee`, `Open Collective`, `Thanks.dev`, or custom URL.
|
|
52
|
-
- If the repo has a generated
|
|
51
|
+
- If the repo has a generated logo or simple hero artwork supported by `nolangz/pixel2motion`, ask whether to animate it or leave it static.
|
|
53
52
|
- If the repo has no clear product essence, ask for a one-sentence positioning statement.
|
|
54
53
|
7. Implement only approved, missing, or weak work in this order:
|
|
55
54
|
- OSS governance and support files
|
|
@@ -213,8 +212,8 @@ Use `magick` or another renderer when `rsvg-convert` is unavailable.
|
|
|
213
212
|
|
|
214
213
|
Some repos benefit from a short animated preview. Keep this opt-in.
|
|
215
214
|
|
|
216
|
-
-
|
|
217
|
-
-
|
|
215
|
+
- After the user opts in during workflow step 6, animate only the approved generated logo or simple artwork.
|
|
216
|
+
- Use `nolangz/pixel2motion` as the optional dependency for that animation, animated HTML demos, and GIF/video previews.
|
|
218
217
|
- Do not auto-animate by default.
|
|
219
218
|
|
|
220
219
|
## README Standard
|
package/package.json
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
FAANG-quality PR code review for GitHub. It gathers the context a human reviewer
|
|
4
4
|
would gather and checks the branch out into an isolated worktree when the depth
|
|
5
|
-
requires one. `scan` runs two triage agents
|
|
5
|
+
requires one. `scan` runs two triage agents. `medium` and
|
|
6
6
|
`deep` run five specialists and attempt proofs for at most five non-rule
|
|
7
7
|
Critical findings.
|
|
8
8
|
|
|
@@ -15,16 +15,16 @@ This skill spends most of its budget trying to disprove itself.
|
|
|
15
15
|
|
|
16
16
|

|
|
17
17
|
|
|
18
|
-
- **Adversarial verification.**
|
|
19
|
-
|
|
20
|
-
|
|
18
|
+
- **Adversarial verification.** Every Critical and Important is checked at
|
|
19
|
+
every depth because either can affect the verdict. Runtime findings use the
|
|
20
|
+
verifier's five-challenge refutation.
|
|
21
21
|
- **Proof of bug.** In `medium` and `deep`, up to five non-rule Criticals are
|
|
22
22
|
sent to prover agents that attempt to write and run a failing test. A proof
|
|
23
|
-
that comes back green is counter-evidence: the
|
|
24
|
-
|
|
25
|
-
- **Hard-rule
|
|
26
|
-
severity
|
|
27
|
-
|
|
23
|
+
that comes back green is counter-evidence: the workflow automatically
|
|
24
|
+
downgrades the finding from Critical to Important.
|
|
25
|
+
- **Hard-rule validation.** A finding tagged with a `rule` keeps Critical
|
|
26
|
+
severity only after the verifier confirms its diff anchor and rule
|
|
27
|
+
applicability. It skips only the runtime-failure challenges.
|
|
28
28
|
- **Fix mode.** `--fix` has one agent edit each file, a read-only verifier read
|
|
29
29
|
the real `git diff`, and the captured proof re-run. If the proof stays red the
|
|
30
30
|
group is reverted. Nothing is pushed without explicit confirmation.
|
|
@@ -5,7 +5,7 @@ description: >
|
|
|
5
5
|
isolated worktree, gathers context from your issue tracker, chat, docs, CI
|
|
6
6
|
checks, and (optionally) production monitoring, runs 5 parallel specialized
|
|
7
7
|
review agents (quality/implementation/simplification/conventions/test-quality),
|
|
8
|
-
verifies every finding against the diff, then outputs a single unified markdown
|
|
8
|
+
verifies every verdict-affecting finding against the diff, then outputs a single unified markdown
|
|
9
9
|
review: file + risk + bad code + why it's wrong + fix, ready to paste directly
|
|
10
10
|
into GitHub. Saves every review to ~/code-reviews/*.md. Covers business logic,
|
|
11
11
|
scalability, complexity, data integrity, security, integration contracts, error
|
|
@@ -78,14 +78,11 @@ the 5 specialist agents still run and still find real bugs. They just don't
|
|
|
78
78
|
have a codified "always Critical" rule list to check against.
|
|
79
79
|
|
|
80
80
|
If any hard rule you define carries a `rule` tag (e.g. `"TS-1"`), reviewer
|
|
81
|
-
agents attach that tag to matching findings
|
|
82
|
-
|
|
83
|
-
challenges
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
your policy declares non-negotiable. A tagged finding keeps its Critical
|
|
87
|
-
severity; the workflow returns how many were exempted as `hardRuleCount`, and
|
|
88
|
-
each carries a `verifierReasoning` saying so.
|
|
81
|
+
agents attach that tag to matching findings. The verifier checks the diff
|
|
82
|
+
anchor and rule applicability, but skips its five runtime-failure challenges.
|
|
83
|
+
Those challenges cannot evaluate a standards violation. A tagged finding keeps
|
|
84
|
+
Critical severity only when both rule-specific checks pass. The workflow
|
|
85
|
+
returns the number checked as `hardRuleCount`.
|
|
89
86
|
|
|
90
87
|
`${CLAUDE_PLUGIN_ROOT}` below refers to this skill's own installed directory:
|
|
91
88
|
resolve every `references/...` and script path relative to it.
|
|
@@ -149,7 +146,7 @@ treat as a full review and leave PREV_SHA unset. Also grep the same file for
|
|
|
149
146
|
| Context: chat/docs/framework-docs/monitoring/prior-review-memory (optional, MCP-dependent) | skip | included | included + broader recall |
|
|
150
147
|
| Worktree + static checks | skip (WORKTREE_PATH=null) unless `--fix` | included | included |
|
|
151
148
|
| Review agents | 2 triage (haiku) | 5 specialists (sonnet) | 5 specialists (sonnet) |
|
|
152
|
-
| Per-finding verification |
|
|
149
|
+
| Per-finding verification | Criticals + Importants | Criticals + Importants | Criticals + Importants |
|
|
153
150
|
| Completeness critic | skip | skip | included |
|
|
154
151
|
| Proof-of-bug (failing test per Critical) | skip | included (max 5) | included (max 5) |
|
|
155
152
|
| Living review artifact | included | included | included |
|
|
@@ -297,8 +294,9 @@ The workflow runs three phases:
|
|
|
297
294
|
description/badCode/fix of the set - a Critical is never demoted by an
|
|
298
295
|
Observation someone else filed at the same line - and records every
|
|
299
296
|
contributing dimension in `agreedBy`.
|
|
300
|
-
- **Verify:**
|
|
301
|
-
|
|
297
|
+
- **Verify:** every Critical and Important is checked at every depth because it
|
|
298
|
+
can affect the verdict. Hard-rule findings (`rule` set) use the verifier's
|
|
299
|
+
diff-anchor and rule-applicability checks instead of runtime challenges.
|
|
302
300
|
Each verifier runs the five-challenge adversarial refutation from
|
|
303
301
|
`references/agents/verifier.md` against one finding, returns
|
|
304
302
|
`{verdict, newSeverity?, reasoning}`. Verifiers run `maxConcurrent` at a time
|
|
@@ -310,13 +308,13 @@ The workflow runs three phases:
|
|
|
310
308
|
one prover agent (`references/agents/prover.md`, sonnet, max 5, run
|
|
311
309
|
`maxConcurrent` at a time) that writes a test asserting the CORRECT behaviour,
|
|
312
310
|
runs it in the worktree, and captures it failing because of the bug. The proof rides on the finding as
|
|
313
|
-
`proof: {attempted, proven, reason, testCode?, testCommand?, redOutput?}`.
|
|
311
|
+
`proof: {attempted, proven, outcome, reason, testCode?, testCommand?, redOutput?}`.
|
|
314
312
|
A proof that comes back GREEN (code behaved correctly) is counter-evidence -
|
|
315
|
-
|
|
313
|
+
the workflow automatically downgrades the finding from Critical to Important.
|
|
316
314
|
Hard-rule findings are never proved (policy violations have no failing test).
|
|
317
315
|
|
|
318
316
|
Findings have schema:
|
|
319
|
-
`{file, line, severity, title, description, badCode, fix, rule?, precedent?, agreedBy?, verifierReasoning?, proof?}`
|
|
317
|
+
`{file, line, severity, title, description, badCode, fix, rule?, precedent?, agreedBy?, verificationStatus?, verifierReasoning?, proof?}`
|
|
320
318
|
`badCode` and `fix` are schema-required: an empty string is allowed only on
|
|
321
319
|
`observation` / `idiomatic` findings.
|
|
322
320
|
|
|
@@ -325,7 +323,7 @@ provers on `sonnet`, housekeeping on `haiku`. Only the synthesis in Step 3 runs
|
|
|
325
323
|
on the session model.
|
|
326
324
|
|
|
327
325
|
Return value from the workflow:
|
|
328
|
-
`{findings, droppedCount, downgradedCount, hardRuleCount, proveAttemptCount, provenCount, agentCount, outputTokens, turnTokensTotal}`
|
|
326
|
+
`{findings, droppedCount, downgradedCount, hardRuleCount, proveAttemptCount, provenCount, acCoverage, coverageVerdict, mutationSlip, mockSmells, agentCount, outputTokens, turnTokensTotal}`
|
|
329
327
|
`outputTokens` is this workflow's own output spend; `turnTokensTotal` is the
|
|
330
328
|
whole turn's shared pool (main loop included).
|
|
331
329
|
|
|
@@ -410,7 +408,7 @@ requirements:
|
|
|
410
408
|
- When `PREV_SHA` is set: include `## 🔁 Since last review` comparing
|
|
411
409
|
`PREV_REVIEW_FILE` findings against the new head - list each as fixed or
|
|
412
410
|
still open, before any new findings.
|
|
413
|
-
- Test Quality section: populate from the
|
|
411
|
+
- Test Quality section: populate from the workflow return fields
|
|
414
412
|
(`coverageVerdict`, `mutationSlip`, `mockSmells`).
|
|
415
413
|
- Idiomatic section: populated from severity=idiomatic findings only.
|
|
416
414
|
- **💰 Review Cost block:** `outputTokens` from the workflow return is the
|
|
@@ -8,6 +8,7 @@ Axes to cover:
|
|
|
8
8
|
- Business Logic / AC coverage: for each AC in the list below, mark
|
|
9
9
|
✅ met / ⚠️ partial / ❌ missing. Scope creep is also worth flagging.
|
|
10
10
|
AC_LIST: read key "acList" from CONTEXT_FILE.
|
|
11
|
+
Return the complete status summary in the structured `acCoverage` field.
|
|
11
12
|
- Scalability: N+1 queries, missing pagination, unbounded in-memory
|
|
12
13
|
collections, missing rate-limit handling, cron jobs without overlap guard,
|
|
13
14
|
missing DB indexes for new query patterns.
|
|
@@ -23,7 +24,7 @@ Axes to cover:
|
|
|
23
24
|
Set `rule: "GQL-1"` on any Critical finding raised under this axis.
|
|
24
25
|
|
|
25
26
|
Setting rule tags the finding as a house hard rule: it keeps its Critical
|
|
26
|
-
severity and skips
|
|
27
|
+
severity only after rule-specific validation and skips runtime challenges. Only set it for a genuine GQL-1
|
|
27
28
|
violation: never to shield an ordinary finding from verification.
|
|
28
29
|
|
|
29
30
|
CI_STATUS: read key "ciStatus" from the JSON file CONTEXT_FILE.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# prover -- lekker-review agent prompt
|
|
2
2
|
# Receives: one FINDING as JSON, WORKTREE_PATH path, DIFF_FILE path, CONTEXT_FILE path
|
|
3
|
-
# Returns: PROOF_SCHEMA { attempted: boolean, proven: boolean, reason: string, testCode?: string, testCommand?: string, redOutput?: string }
|
|
3
|
+
# Returns: PROOF_SCHEMA { attempted: boolean, proven: boolean, outcome: 'proven'|'passed'|'inconclusive'|'not_attempted', reason: string, testCode?: string, testCommand?: string, redOutput?: string }
|
|
4
4
|
|
|
5
5
|
## Mission
|
|
6
6
|
|
|
@@ -29,8 +29,9 @@ explicitly.
|
|
|
29
29
|
## Step 1 -- Testability gate
|
|
30
30
|
|
|
31
31
|
Read the FINDING, the real code at `file:line` in `WORKTREE_PATH`, and the diff
|
|
32
|
-
context in `DIFF_FILE`. Return `attempted: false
|
|
33
|
-
`reason` when any of these
|
|
32
|
+
context in `DIFF_FILE`. Return `attempted: false, proven: false,
|
|
33
|
+
outcome: "not_attempted"` with an honest one-sentence `reason` when any of these
|
|
34
|
+
hold:
|
|
34
35
|
|
|
35
36
|
- The failure path requires live IO (Shopify/BC/Salesforce API, a real DB, the
|
|
36
37
|
network) and the repo has no test infra to fake it cheaply.
|
|
@@ -77,25 +78,26 @@ Exactly one run command, scoped to your file only:
|
|
|
77
78
|
Set the Bash tool's `timeout` parameter to 120000 for this call.
|
|
78
79
|
|
|
79
80
|
If the runner hangs or the environment fails (missing config, transform
|
|
80
|
-
errors), that is `attempted: true, proven: false
|
|
81
|
-
honestly -- never retry more than once for a pure
|
|
82
|
-
wrong config flag), and never loop.
|
|
81
|
+
errors), that is `attempted: true, proven: false, outcome: "inconclusive"`
|
|
82
|
+
with the reason. Report honestly -- never retry more than once for a pure
|
|
83
|
+
environment issue (e.g. a wrong config flag), and never loop.
|
|
83
84
|
|
|
84
85
|
---
|
|
85
86
|
|
|
86
87
|
## Step 4 -- Judge the outcome
|
|
87
88
|
|
|
88
89
|
- **Test FAILS, and the mismatch matches what the finding predicts** ->
|
|
89
|
-
`proven: true`. `redOutput` = the failure excerpt, trimmed to the
|
|
90
|
+
`proven: true, outcome: "proven"`. `redOutput` = the failure excerpt, trimmed to the
|
|
90
91
|
informative ~15 lines (expected vs received + the failing assertion line).
|
|
91
92
|
`testCode` = the full test file content. `testCommand` = the exact command
|
|
92
93
|
you ran.
|
|
93
|
-
- **Test PASSES** -> the finding did not reproduce. `proven: false
|
|
94
|
+
- **Test PASSES** -> the finding did not reproduce. `proven: false,
|
|
95
|
+
outcome: "passed"`, and
|
|
94
96
|
`reason` states plainly that the code behaved correctly for the tested
|
|
95
97
|
input. This is important review signal, not a failure of yours. Do NOT alter
|
|
96
98
|
the test to force a failure.
|
|
97
99
|
- **Test fails for an unrelated reason** (import error, env issue) ->
|
|
98
|
-
`proven: false`, honest `reason`.
|
|
100
|
+
`proven: false, outcome: "inconclusive"`, honest `reason`.
|
|
99
101
|
|
|
100
102
|
---
|
|
101
103
|
|
|
@@ -124,6 +126,7 @@ Return EXACTLY one JSON object matching PROOF_SCHEMA:
|
|
|
124
126
|
{
|
|
125
127
|
"attempted": true | false,
|
|
126
128
|
"proven": true | false,
|
|
129
|
+
"outcome": "proven" | "passed" | "inconclusive" | "not_attempted",
|
|
127
130
|
"reason": "<one or two sentences: why not attempted, why it proved, or why it didn't reproduce>",
|
|
128
131
|
"testCode": "<full test file content -- only when attempted>",
|
|
129
132
|
"testCommand": "<exact command run -- only when attempted>",
|
|
@@ -131,5 +134,5 @@ Return EXACTLY one JSON object matching PROOF_SCHEMA:
|
|
|
131
134
|
}
|
|
132
135
|
```
|
|
133
136
|
|
|
134
|
-
`attempted: false` implies `proven: false` and omits `testCode`/`testCommand`/
|
|
137
|
+
`attempted: false` implies `proven: false, outcome: "not_attempted"` and omits `testCode`/`testCommand`/
|
|
135
138
|
`redOutput`. Do not narrate outside the object.
|
|
@@ -56,7 +56,7 @@ Axes to cover:
|
|
|
56
56
|
or a `node_modules` path.
|
|
57
57
|
|
|
58
58
|
Setting rule tags the finding as a house hard rule: it keeps its Critical
|
|
59
|
-
severity and skips
|
|
59
|
+
severity only after rule-specific validation and skips runtime challenges. Only set it for a genuine
|
|
60
60
|
TS-1/TS-2 violation: never to shield an ordinary finding from verification.
|
|
61
61
|
|
|
62
62
|
CI_STATUS: read key "ciStatus" from the JSON file CONTEXT_FILE.
|
|
@@ -141,6 +141,13 @@ criticism without a fix is incomplete.
|
|
|
141
141
|
Report the mutation-slip analysis as a single paragraph under a
|
|
142
142
|
"**Mutation-slip risk:**" heading, not as line items.
|
|
143
143
|
|
|
144
|
+
Return the review summary through these structured fields in addition to
|
|
145
|
+
`findings`:
|
|
146
|
+
- `coverageVerdict`: one concise overall coverage verdict.
|
|
147
|
+
- `mutationSlip`: the mutation-slip risk paragraph.
|
|
148
|
+
- `mockSmells`: every mock-smell issue as `{file, line, description, fix}`.
|
|
149
|
+
Return an empty array when there are no mock smells.
|
|
150
|
+
|
|
144
151
|
EXISTING_REVIEWS: read key "existingReviews" from CONTEXT_FILE (awareness only, skip findings already raised)
|
|
145
152
|
|
|
146
153
|
PROJECT_RULES to verify: read key "projectRules" from CONTEXT_FILE.
|
|
@@ -20,7 +20,7 @@ Axes to cover (Critical/Important only):
|
|
|
20
20
|
set `rule: "GQL-1"` when Critical
|
|
21
21
|
|
|
22
22
|
Setting rule tags the finding as a house hard rule: it keeps its Critical
|
|
23
|
-
severity and skips
|
|
23
|
+
severity only after rule-specific validation and skips runtime challenges. Only set it for a genuine
|
|
24
24
|
TS-1/TS-2/GQL-1 violation: never to shield an ordinary finding from
|
|
25
25
|
verification.
|
|
26
26
|
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
# verifier -- lekker-review agent prompt
|
|
2
|
-
# Receives: one FINDING as JSON, DIFF_FILE path, CONTEXT_FILE path, WORKTREE_PATH path
|
|
2
|
+
# Receives: one FINDING as JSON, DIFF_FILE path, CONTEXT_FILE path, WORKTREE_PATH path, HOUSE_RULES_FILE path
|
|
3
3
|
# Returns: VERDICT_SCHEMA { verdict: 'confirmed'|'downgraded'|'dropped', newSeverity?, reasoning }
|
|
4
4
|
|
|
5
5
|
## Mindset
|
|
@@ -18,17 +18,19 @@ verified yourself are not confirmed findings.
|
|
|
18
18
|
|
|
19
19
|
You have received one candidate finding as JSON (field `FINDING` in your task
|
|
20
20
|
message). Run all applicable verification checks below against the finding using
|
|
21
|
-
`DIFF_FILE`, `CONTEXT_FILE`, and `
|
|
21
|
+
`DIFF_FILE`, `CONTEXT_FILE`, `WORKTREE_PATH`, and `HOUSE_RULES_FILE` (read from
|
|
22
|
+
the task message).
|
|
22
23
|
|
|
23
24
|
When in doubt, drop. A Critical must pass ALL FIVE challenges to remain Critical
|
|
24
|
-
|
|
25
|
+
unless Step 0 validates it through the house hard-rule path.
|
|
25
26
|
|
|
26
27
|
---
|
|
27
28
|
|
|
28
|
-
## Step 0 -- Hard-rule
|
|
29
|
+
## Step 0 -- Hard-rule validation (check this FIRST)
|
|
29
30
|
|
|
30
|
-
If the FINDING JSON has
|
|
31
|
-
`PR-1`, do NOT run the five adversarial
|
|
31
|
+
If the FINDING JSON has any non-empty `rule` field, including built-in tags
|
|
32
|
+
such as `TS-1`, `TS-2`, `GQL-1`, or `PR-1`, do NOT run the five adversarial
|
|
33
|
+
challenges below. They ask
|
|
32
34
|
runtime-failure questions that a standards violation can never answer, and
|
|
33
35
|
answering them honestly would drop a finding that house-rules policy declares
|
|
34
36
|
Critical on standards grounds rather than on runtime behaviour.
|
|
@@ -41,8 +43,9 @@ Instead run exactly two checks:
|
|
|
41
43
|
```bash
|
|
42
44
|
grep "^+" "$DIFF_FILE" | grep "<snippet>"
|
|
43
45
|
```
|
|
44
|
-
2. **Rule applicability**: the
|
|
45
|
-
|
|
46
|
+
2. **Rule applicability**: read the exact `HOUSE_RULES_FILE` path from the task
|
|
47
|
+
message, then confirm the code really violates the rule as written there --
|
|
48
|
+
e.g. an `as const` is not a type cast in
|
|
46
49
|
the TS-1 sense; a `nodes` query that legitimately fetches a single known
|
|
47
50
|
node with a documented comment may satisfy GQL-1; a `.js` file inside a
|
|
48
51
|
Liquid theme repo is exempt from TS-2.
|
|
@@ -253,7 +256,7 @@ the inference is the crux of the claim
|
|
|
253
256
|
## Outcome rules
|
|
254
257
|
|
|
255
258
|
The one exception to "a Critical must pass ALL FIVE challenges" is the
|
|
256
|
-
hard-rule path in Step 0: a `rule`-tagged finding is
|
|
259
|
+
hard-rule path in Step 0: a `rule`-tagged finding is validated solely on the
|
|
257
260
|
anchor + rule-applicability checks and never runs the five challenges.
|
|
258
261
|
|
|
259
262
|
A Critical finding must pass ALL FIVE challenges. Any failure downgrades:
|
|
@@ -18,7 +18,7 @@ All of the following are already in hand after Step 3 of SKILL.md:
|
|
|
18
18
|
- `findings.json` path (scratchpad) -- each finding carries: `file`, `line`,
|
|
19
19
|
`severity`, `title`, `description`, `badCode`, `fix`, `rule?`, `precedent?`,
|
|
20
20
|
`agreedBy?`, `verifierReasoning?`, `proof?` (proof = `{attempted, proven,
|
|
21
|
-
reason, testCode?, testCommand?, redOutput?}`).
|
|
21
|
+
outcome, reason, testCode?, testCommand?, redOutput?}`).
|
|
22
22
|
- The saved review file path: `~/code-reviews/YYYY-MM-DD-pr-N-repo.md`.
|
|
23
23
|
- PR metadata: `REPO_SLUG`, `PR_NUMBER`, `PR_URL`, title, author, `headRefName`
|
|
24
24
|
→ `baseRefName`, head sha, depth, verdict, `isDraft`, `mergeStateStatus`, CI
|
|
@@ -96,6 +96,11 @@ Hard requirements for the HTML page:
|
|
|
96
96
|
`<pre>`, a one-line explanation that the test asserts correct behavior, and
|
|
97
97
|
`testCode` collapsed behind its own `<details>`. This is the page's
|
|
98
98
|
centerpiece -- make it prominent (e.g. a red left border) but not garish.
|
|
99
|
+
When `proof.outcome === 'passed'`, render a distinct
|
|
100
|
+
"COUNTER-EVIDENCE -- focused proof test passed" panel with `reason`,
|
|
101
|
+
`testCommand`, and `testCode` collapsed behind its own `<details>`. Explain
|
|
102
|
+
that the workflow downgraded the finding to Important; do not present the
|
|
103
|
+
passing input as proof that every related input is safe.
|
|
99
104
|
- **Test Quality + Review Cost sections**, mirrored from the review file,
|
|
100
105
|
kept concise (verdict + gaps + cost table; no need to reproduce every
|
|
101
106
|
sentence).
|
|
@@ -217,7 +217,7 @@ if empty.)*
|
|
|
217
217
|
Depth: <⚡ scan | 🔍 medium | 🔬 deep>
|
|
218
218
|
Diff size: ~<N> lines (~<N> tokens)
|
|
219
219
|
Agents run: <N total>: <breakdown, e.g. "5 reviewers (sonnet) + 6 verifiers (sonnet) + 2 provers (sonnet) + 1 critic (sonnet)">
|
|
220
|
-
|
|
220
|
+
Rule checks: <N> hard-rule finding(s) validated with rule-specific checks (omit the line when 0)
|
|
221
221
|
Context sources: <the subset of issue-tracker / chat / docs / framework-docs / monitoring / CI / prior-review-memory actually used>
|
|
222
222
|
|
|
223
223
|
Output tokens: <N> ← ACTUAL workflow spend, from the workflow's outputTokens return value
|
|
@@ -253,10 +253,7 @@ Reviewer agents, verifiers, provers, and the critic all run on sonnet; triage an
|
|
|
253
253
|
self-contained but silently breaks callers.
|
|
254
254
|
- Blank line between findings. No double blank lines.
|
|
255
255
|
- No trailing whitespace, no HTML tags, no ANSI escapes.
|
|
256
|
-
- **Proof counter-evidence rule**: when `proof.
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
(e.g. the tested input wasn't the one that actually breaks). A finding whose
|
|
261
|
-
proof came back green cannot silently stay Critical. When `attempted` is
|
|
262
|
-
false, say nothing: untestable is not evidence either way.
|
|
256
|
+
- **Proof counter-evidence rule**: when `proof.outcome` is `passed`, the
|
|
257
|
+
workflow has already downgraded the finding from Critical to Important.
|
|
258
|
+
Explain the passing proof in the finding body. When `attempted` is false,
|
|
259
|
+
say nothing: untestable is not evidence either way.
|