@groupby/ai-dev 0.5.19 → 0.5.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +56 -0
- package/dist/index.js +885 -124
- package/package.json +2 -2
- package/skills/README.md +42 -1
- package/teams/OOF/skills/oof-review/SKILL.md +319 -0
- package/teams/OOF/skills/oof-review/output-format.md +166 -0
- package/teams/OOF/skills/oof-review/reviewer-prompt.md +99 -0
- package/teams/OOF/skills/oof-review/summarize_review_config.py +226 -0
- package/teams/OOF/skills/oof-review/technology-profiles.md +54 -0
- package/teams/rangers/third-party/mattpocock/grilling/SKILL.md +28 -0
- package/teams/rangers/third-party/mattpocock/grilling/skill-meta.yml +14 -0
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@groupby/ai-dev",
|
|
3
|
-
"version": "0.5.
|
|
3
|
+
"version": "0.5.21",
|
|
4
4
|
"description": "Interactive installer for Rezolve Ai development content",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -15,7 +15,7 @@
|
|
|
15
15
|
"scripts": {
|
|
16
16
|
"prebuild": "rm -rf skills teams toolsets && cp -r ../skills ./skills && cp -r ../teams ./teams && cp -r ../toolsets ./toolsets",
|
|
17
17
|
"build": "tsup",
|
|
18
|
-
"
|
|
18
|
+
"prepare": "npm run build",
|
|
19
19
|
"test": "vitest run",
|
|
20
20
|
"test:watch": "vitest",
|
|
21
21
|
"test:coverage": "vitest run --coverage"
|
package/skills/README.md
CHANGED
|
@@ -4,7 +4,7 @@ This folder manages all skills, archived skills, and examples/instructions on us
|
|
|
4
4
|
|
|
5
5
|
All skills in this `skills/library/` should represent _generally useful skills_ that might be used across teams.
|
|
6
6
|
|
|
7
|
-
If any particular skill goes missing, note that it may have simply been archived (see [the archive](
|
|
7
|
+
If any particular skill goes missing, note that it may have simply been archived (see [the archive](archived/)) and may no longer be recommended for use. Ideally skills are not simply deleted from this repository, since others may be expecting them to be there.
|
|
8
8
|
|
|
9
9
|
## Usage
|
|
10
10
|
|
|
@@ -58,4 +58,45 @@ license: Complete terms in LICENSE.txt
|
|
|
58
58
|
See `docs/ai/skills/frontend-design/SKILL.md` and follow closely.
|
|
59
59
|
```
|
|
60
60
|
|
|
61
|
+
## Third-party skills
|
|
62
|
+
|
|
63
|
+
Skills authored outside the company can be vendored into a team's folder under
|
|
64
|
+
`teams/<team>/third-party/<lib>/<skill>/`, each with the upstream `SKILL.md`
|
|
65
|
+
(vendored verbatim at a pinned commit) plus a `skill-meta.yml` provenance
|
|
66
|
+
sidecar. They are curated per team by evaluation status: `evaluating`,
|
|
67
|
+
`recommended`, or `deprecated`.
|
|
68
|
+
|
|
69
|
+
The lifecycle is fetch, vendor, evaluate, and promote:
|
|
70
|
+
|
|
71
|
+
1. **Fetch + Vendor** in one step with the authoring command — paste a GitHub
|
|
72
|
+
URL to the skill folder and it writes the pinned `SKILL.md` plus a generated
|
|
73
|
+
`skill-meta.yml`:
|
|
74
|
+
```bash
|
|
75
|
+
npx @groupby/ai-dev vendor <github-url> --team <team>
|
|
76
|
+
```
|
|
77
|
+
Or do it manually: fetch into a scratch directory (for example,
|
|
78
|
+
`npx skills add <owner/repo> --copy`), then copy the single skill folder into
|
|
79
|
+
`teams/<team>/third-party/<lib>/<skill>/`, recording the source, pinned
|
|
80
|
+
commit, and `status: evaluating` in `skill-meta.yml`.
|
|
81
|
+
2. **Evaluate** it in a real project with
|
|
82
|
+
`npx @groupby/ai-dev install skill <skill-name>` (naming a skill installs it
|
|
83
|
+
regardless of status).
|
|
84
|
+
3. **Promote** it to `status: recommended` once proven, or graduate it into
|
|
85
|
+
`skills/library/` as a first-party skill (dropping the sidecar).
|
|
86
|
+
|
|
87
|
+
A `recommended` third-party skill installs with its team like any other skill;
|
|
88
|
+
`evaluating` and `deprecated` skills are held back from team installs unless you
|
|
89
|
+
pass `--include-evaluating` or `--include-deprecated`. Naming a skill directly
|
|
90
|
+
with `install skill <name>` always installs it. If a name is shared by more than
|
|
91
|
+
one skill in the same tier (for example two teams that each vendored it), the CLI
|
|
92
|
+
lists the matches and asks you to scope it with `install skill <team>/<name>` or
|
|
93
|
+
`--team <team>`. See
|
|
94
|
+
[content conventions](../docs/content-conventions.md) for the `skill-meta.yml`
|
|
95
|
+
schema and the full rules.
|
|
96
|
+
|
|
97
|
+
The `skill-meta.yml` sidecar is required. A vendored skill missing it has no
|
|
98
|
+
provenance or status, so `list` shows it as `[no status] unknown source`, team
|
|
99
|
+
installs skip it, and installing it by name still works but prints a
|
|
100
|
+
`no recorded status` note. Add the sidecar to make it a fully curated skill.
|
|
101
|
+
|
|
61
102
|
Note that we MUST include the skill boilerplate in any official skills location (eg. `.github/skills/`, etc.) to enable discovery and usage. We then copy the entire skill into `docs/ai/skills/` to leave it intact, though we don't technically need the boilerplate.
|
|
@@ -0,0 +1,319 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: oof-review
|
|
3
|
+
description: >-
|
|
4
|
+
Multi-model code review council inspired by Karpathy's LLM Council. Spawns 3
|
|
5
|
+
sub-agents on different models (Claude Opus 4.8, GPT-5.3 Codex, GPT-5.5) to
|
|
6
|
+
independently review code changes, then synthesizes and votes on the best
|
|
7
|
+
comments to produce a unified, high-signal review. Use when the user says
|
|
8
|
+
/oof-review, /council-review, 'council review', 'multi-model review', 'review
|
|
9
|
+
council', or 'LLM council'.
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
<!-- Based on the snpd council-review skill. Special thanks to Haris Dervisevic for the original work. -->
|
|
13
|
+
|
|
14
|
+
# LLM Council Code Review
|
|
15
|
+
|
|
16
|
+
## Purpose
|
|
17
|
+
|
|
18
|
+
Provide a high-quality, consensus-driven code review by running **three independent
|
|
19
|
+
reviewers on different LLM models**, then synthesizing their findings into a single
|
|
20
|
+
review ranked by agreement and severity — similar to
|
|
21
|
+
[Karpathy's LLM Council](https://github.com/karpathy/llm-council).
|
|
22
|
+
|
|
23
|
+
The insight: different models catch different things. One model may spot a race
|
|
24
|
+
condition another misses; one may flag a security issue the others gloss over.
|
|
25
|
+
By requiring consensus, noise drops and signal rises.
|
|
26
|
+
|
|
27
|
+
## Models Used (The Council)
|
|
28
|
+
|
|
29
|
+
| Seat | Model ID | Strengths |
|
|
30
|
+
|------------|---------------------|-------------------------------------------------|
|
|
31
|
+
| Reviewer A | `claude-opus-4.8` | Deep reasoning, architecture, subtle logic bugs |
|
|
32
|
+
| Reviewer B | `gpt-5.3-codex` | Code-native, practical fixes, test gaps |
|
|
33
|
+
| Reviewer C | `gpt-5.5` | Broad knowledge, security, API design |
|
|
34
|
+
|
|
35
|
+
## Trigger
|
|
36
|
+
|
|
37
|
+
Activate this skill when the user says any of:
|
|
38
|
+
- `/oof-review`
|
|
39
|
+
- `/council-review`
|
|
40
|
+
- `council review my changes`
|
|
41
|
+
- `multi-model review`
|
|
42
|
+
- `LLM council review`
|
|
43
|
+
- `review council`
|
|
44
|
+
|
|
45
|
+
## Inputs
|
|
46
|
+
|
|
47
|
+
The user may provide:
|
|
48
|
+
- **No argument** → review local uncommitted changes (staged + unstaged)
|
|
49
|
+
- **`--staged`** → review only staged changes
|
|
50
|
+
- **`--branch [<target>]`** → review current branch diff vs `origin/main` (default) or a custom target, e.g. `--branch origin/develop` or `--branch origin/release/1.2`
|
|
51
|
+
- **`--commits <N>`** → review the last N commits (ignores uncommitted changes)
|
|
52
|
+
- **`--commits <sha>..<sha>`** → review a specific commit range
|
|
53
|
+
- **`--pr <number>`** → review a specific GitHub PR
|
|
54
|
+
- **A file path or glob** → review only those files
|
|
55
|
+
|
|
56
|
+
Natural language also works:
|
|
57
|
+
- "review my last 2 commits" → same as `--commits 2`
|
|
58
|
+
- "review last 3 commits before I open a PR" → same as `--commits 3`
|
|
59
|
+
|
|
60
|
+
## Workflow
|
|
61
|
+
|
|
62
|
+
### Phase 0: Repo Discovery
|
|
63
|
+
|
|
64
|
+
Before reviewing any code, discover the **current repo's own rules**. Do not
|
|
65
|
+
carry assumptions from another repo.
|
|
66
|
+
|
|
67
|
+
1. **Check write-access (advisory):**
|
|
68
|
+
|
|
69
|
+
Read `.claude/settings.local.json` in the repo root (if it exists) and check
|
|
70
|
+
whether it contains `"worktree": { "bgIsolation": "none" }`.
|
|
71
|
+
|
|
72
|
+
- **If the setting is absent or the file does not exist:** Inform the user
|
|
73
|
+
with a one-line notice, then **continue** — do not stop:
|
|
74
|
+
|
|
75
|
+
> ℹ️ `.claude/settings.local.json` does not have `bgIsolation: none`.
|
|
76
|
+
> If the skill is invoked as a background session, writing the report to
|
|
77
|
+
> `docs/reviews/` may fail. See Phase 5 for the fallback.
|
|
78
|
+
|
|
79
|
+
- **If the setting is present:** Continue normally.
|
|
80
|
+
|
|
81
|
+
2. **Confirm repository scope:**
|
|
82
|
+
- Run `git status --short` and `git branch --show-current`.
|
|
83
|
+
- Identify the repo root and project type.
|
|
84
|
+
|
|
85
|
+
3. **Discover review configuration:**
|
|
86
|
+
- Check for these files and read them if present:
|
|
87
|
+
- `.github/workflows/claude-pr-review.yml` or `.github/workflows/claude.yml`
|
|
88
|
+
- `.github/workflows/build-pr.yaml`
|
|
89
|
+
- `.github/PULL_REQUEST_TEMPLATE.md`
|
|
90
|
+
- `.github/CODEOWNERS`
|
|
91
|
+
- Run this skill's bundled `summarize_review_config.py` script for a
|
|
92
|
+
quick context summary — it is repo-agnostic and works on any repository.
|
|
93
|
+
Resolve the script from this skill's own base directory and pass the
|
|
94
|
+
current repo root as its argument:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
# The skill base directory for this repo is: docs/ai/skills/oof-review/
|
|
98
|
+
python3 "docs/ai/skills/oof-review/summarize_review_config.py" .
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
**This step is MANDATORY. Do NOT skip it.**
|
|
102
|
+
Do NOT fall back to manual reading unless `python3` returns a
|
|
103
|
+
"command not found" error (i.e. Python is genuinely absent on the machine).
|
|
104
|
+
Print the full script output verbatim under a `## Repo Config Summary`
|
|
105
|
+
heading before proceeding to Phase 1.
|
|
106
|
+
|
|
107
|
+
4. **Discover repo guidance (read if present):**
|
|
108
|
+
- `.github/copilot-instructions.md`
|
|
109
|
+
- `CLAUDE.md`, `AGENTS.md`
|
|
110
|
+
- `docs/conventions.md`, `docs/project-rule.md`, `docs/source-control.md`
|
|
111
|
+
- `README.md` (skim for architecture/setup sections)
|
|
112
|
+
|
|
113
|
+
5. **Detect technology profile:**
|
|
114
|
+
Scan build files and guidance docs for technology markers. Apply the
|
|
115
|
+
matching profile from `technology-profiles.md` (in this skill's directory). Only apply
|
|
116
|
+
rules that the **current repo actually uses**.
|
|
117
|
+
|
|
118
|
+
Key markers to scan for:
|
|
119
|
+
- Java/Gradle: `build.gradle`, `build.gradle.kts`, `settings.gradle`
|
|
120
|
+
- Maven: `pom.xml`
|
|
121
|
+
- Go: `go.mod`, `Makefile`
|
|
122
|
+
- Python: `pyproject.toml`, `requirements*.txt`
|
|
123
|
+
- Node: `package.json`
|
|
124
|
+
|
|
125
|
+
6. **Build a PROJECT_CONTEXT block** from all discovered information. This
|
|
126
|
+
block will be injected into every reviewer's prompt so all three models
|
|
127
|
+
review against the same repo-specific rules.
|
|
128
|
+
|
|
129
|
+
### Phase 0 Completion Checklist
|
|
130
|
+
|
|
131
|
+
Before moving to Phase 1, confirm each item explicitly in your response:
|
|
132
|
+
|
|
133
|
+
| Step | Status | Notes |
|
|
134
|
+
|------|--------|-------|
|
|
135
|
+
| `.claude/settings.local.json` has `bgIsolation: none` | ✅ present / ℹ️ absent (advisory) | |
|
|
136
|
+
| `git status` + current branch | ✅ / ❌ | |
|
|
137
|
+
| `summarize_review_config.py` ran | ✅ ran / ❌ skipped — reason: _______ | Output printed above |
|
|
138
|
+
| Guidance docs read (CLAUDE.md, copilot-instructions, etc.) | ✅ / ❌ | |
|
|
139
|
+
| Technology profile identified | ✅ `<profile name>` | |
|
|
140
|
+
|
|
141
|
+
A model that skips a step **must** write `❌ skipped — reason: <explicit reason>` in
|
|
142
|
+
the table. Silent omission is not acceptable.
|
|
143
|
+
|
|
144
|
+
### Phase 1: Gather the Diff
|
|
145
|
+
|
|
146
|
+
1. Determine the review scope based on user input:
|
|
147
|
+
- **Local changes (default):** If working tree has edits, use `git --no-pager diff HEAD`
|
|
148
|
+
(includes staged + unstaged). If working tree is clean but branch has
|
|
149
|
+
commits, compare against the PR base: `git --no-pager diff origin/main...HEAD`.
|
|
150
|
+
If `origin/main` is not available, inspect upstream and available remotes.
|
|
151
|
+
When the user specifies `--branch <target>`, use that target instead of `origin/main`.
|
|
152
|
+
- **Staged only:** `git --no-pager diff --cached`
|
|
153
|
+
- **Branch diff:** `git --no-pager diff <target>...HEAD` where `<target>` is the branch supplied with `--branch` (defaults to `origin/main` when not specified)
|
|
154
|
+
- **Last N commits:** `git --no-pager diff HEAD~N..HEAD` (ignores working tree entirely)
|
|
155
|
+
Example: `--commits 2` → `git --no-pager diff HEAD~2..HEAD`
|
|
156
|
+
- **Commit range:** `git --no-pager diff <sha1>..<sha2>` for explicit ranges
|
|
157
|
+
- **PR:** `gh pr diff <number>`
|
|
158
|
+
2. Also gather context:
|
|
159
|
+
- `git --no-pager diff --stat` for the file change summary
|
|
160
|
+
- The PROJECT_CONTEXT block built in Phase 0
|
|
161
|
+
3. If the diff is empty, tell the user and stop.
|
|
162
|
+
4. If the diff is very large (>5000 lines), warn the user and suggest narrowing scope.
|
|
163
|
+
5. **Classify the change** (helps reviewers focus):
|
|
164
|
+
- API/controller, service/orchestration, repository/database, search engine,
|
|
165
|
+
Mongo query/indexing, cache, Pub/Sub/messaging, auth/security, feature flags,
|
|
166
|
+
docs, tests, build/dependency, deployment, or tooling.
|
|
167
|
+
|
|
168
|
+
### Phase 2: Deploy the Council (Parallel Sub-Agents)
|
|
169
|
+
|
|
170
|
+
Launch **exactly 3 `code-review` agents in parallel** using the `task` tool, each
|
|
171
|
+
with a different `model` parameter. All three receive the **identical prompt** so
|
|
172
|
+
their reviews are directly comparable.
|
|
173
|
+
|
|
174
|
+
**CRITICAL: Launch all 3 in a single response — they run in parallel.**
|
|
175
|
+
|
|
176
|
+
Each agent receives the prompt from `reviewer-prompt.md` (in this skill's directory),
|
|
177
|
+
with the diff and project context injected.
|
|
178
|
+
|
|
179
|
+
```
|
|
180
|
+
Agent A: task(agent_type="code-review", model="claude-opus-4.8", ...)
|
|
181
|
+
Agent B: task(agent_type="code-review", model="gpt-5.3-codex", ...)
|
|
182
|
+
Agent C: task(agent_type="code-review", model="gpt-5.5", ...)
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
All three agents run in `mode="background"`. Wait for all three to complete
|
|
186
|
+
before proceeding to Phase 3.
|
|
187
|
+
|
|
188
|
+
### Phase 3: Collect & Parse Reviews
|
|
189
|
+
|
|
190
|
+
Read all three agent results. Each agent returns findings in the structured
|
|
191
|
+
format defined in `reviewer-prompt.md`. Extract:
|
|
192
|
+
- File path and line range for each comment
|
|
193
|
+
- Severity (P1/P2/P3)
|
|
194
|
+
- Category (bug, security, performance, style, test-gap, design)
|
|
195
|
+
- The finding description and suggested fix
|
|
196
|
+
|
|
197
|
+
### Phase 4: Council Vote — Synthesize & Rank
|
|
198
|
+
|
|
199
|
+
This is the core "council" step. Process the three reviews:
|
|
200
|
+
|
|
201
|
+
#### 4a. Deduplicate
|
|
202
|
+
|
|
203
|
+
Group comments that refer to the **same issue** (same file, overlapping lines,
|
|
204
|
+
same root cause). Two comments are "the same issue" if they:
|
|
205
|
+
- Point to the same file and overlapping line range, AND
|
|
206
|
+
- Describe the same underlying problem (even in different words)
|
|
207
|
+
|
|
208
|
+
#### 4b. Score by Agreement
|
|
209
|
+
|
|
210
|
+
For each unique issue, count how many of the 3 reviewers flagged it:
|
|
211
|
+
|
|
212
|
+
| Agreement | Label | Weight |
|
|
213
|
+
|-----------|--------------|--------|
|
|
214
|
+
| 3/3 | 🟢 Unanimous | High |
|
|
215
|
+
| 2/3 | 🟡 Majority | Medium |
|
|
216
|
+
| 1/3 | 🔵 Solo | Low |
|
|
217
|
+
|
|
218
|
+
#### 4c. Rank
|
|
219
|
+
|
|
220
|
+
Sort the final list by:
|
|
221
|
+
1. **Agreement** (unanimous > majority > solo)
|
|
222
|
+
2. **Severity** (P1 > P2 > P3) within each agreement tier
|
|
223
|
+
3. Within the same tier+severity, keep the most actionable/clear version of
|
|
224
|
+
the comment (pick the best phrasing from whichever model wrote it)
|
|
225
|
+
|
|
226
|
+
#### 4d. Solo Comment Filter
|
|
227
|
+
|
|
228
|
+
Solo comments (1/3) are **not discarded** but are presented separately under
|
|
229
|
+
a "Minority Opinions" section. They may contain genuine catches the other
|
|
230
|
+
models missed, or they may be noise. Let the user decide.
|
|
231
|
+
|
|
232
|
+
### Phase 5: Write and Present the Council Review
|
|
233
|
+
|
|
234
|
+
1. **Determine the output file path.**
|
|
235
|
+
Use the format: `docs/reviews/oof-review-<YYYY-MM-DD>-<branch-or-scope>.md`
|
|
236
|
+
- Replace `<YYYY-MM-DD>` with today's date.
|
|
237
|
+
- Replace `<branch-or-scope>` with the sanitized current branch name (slashes → dashes)
|
|
238
|
+
or a short descriptor when reviewing uncommitted changes (e.g. `local-changes`).
|
|
239
|
+
- Example: `docs/reviews/oof-review-2026-08-18-feature-my-branch.md`
|
|
240
|
+
|
|
241
|
+
2. **Write the review to the file** using the format in `output-format.md`
|
|
242
|
+
(in this skill's directory). Create the `docs/reviews/` directory if it does not exist.
|
|
243
|
+
|
|
244
|
+
> **IMPORTANT — do NOT enter a worktree.** Write the file directly to the
|
|
245
|
+
> user's current working tree using the Write tool. Entering a worktree
|
|
246
|
+
> creates an isolated branch and an extra checkout of the repository, which
|
|
247
|
+
> is unnecessary and confusing for a read-only report file.
|
|
248
|
+
>
|
|
249
|
+
> **Do NOT `git add`, `git commit`, or run any git command after writing.**
|
|
250
|
+
> The deliverable is the file on disk. The user decides whether and when to
|
|
251
|
+
> commit it.
|
|
252
|
+
|
|
253
|
+
**If writing fails** (e.g. the isolation guard is still active despite the
|
|
254
|
+
Phase 0 check): print the **full review content** as a fenced Markdown
|
|
255
|
+
block in chat, prefixed with:
|
|
256
|
+
|
|
257
|
+
> ⚠️ Could not write to disk — see below. Paste this into
|
|
258
|
+
> `docs/reviews/<filename>.md` manually, or add
|
|
259
|
+
> `"worktree": {"bgIsolation": "none"}` to `.claude/settings.local.json` and re-run.
|
|
260
|
+
|
|
261
|
+
3. **Confirm in chat** — after writing the file, post a short summary message:
|
|
262
|
+
```
|
|
263
|
+
✅ Council review written to `docs/reviews/council-review-<date>-<scope>.md`
|
|
264
|
+
Verdict: <PASS | PASS WITH COMMENTS | NEEDS CHANGES>
|
|
265
|
+
Top findings: <1-line summary of the highest-priority issues>
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
## Hard Rules
|
|
269
|
+
|
|
270
|
+
- **Identical prompts.** All three reviewers get exactly the same input.
|
|
271
|
+
Do not customize prompts per model — the whole point is fair comparison.
|
|
272
|
+
- **No model bias.** Do not weight one model's opinion over another during
|
|
273
|
+
voting. Agreement count is the only ranking signal.
|
|
274
|
+
- **Parallel launch.** Always launch all 3 agents in a single response.
|
|
275
|
+
Never run them sequentially.
|
|
276
|
+
- **Transparency.** Always show which models agreed on each finding.
|
|
277
|
+
- **No hallucinated code.** Do not generate suggested replacement code
|
|
278
|
+
yourself during synthesis. Use the reviewers' suggestions as-is.
|
|
279
|
+
- **Severity consistency.** If reviewers disagree on severity for the same
|
|
280
|
+
issue, use the highest severity any reviewer assigned.
|
|
281
|
+
- **Signal over noise.** The council exists to reduce noise. If a comment
|
|
282
|
+
is unclear or contradictory across reviewers, note the disagreement rather
|
|
283
|
+
than forcing consensus.
|
|
284
|
+
- **No worktree for review output.** Writing the review file is not a code
|
|
285
|
+
change. Do not call `EnterWorktree` at any point during this skill.
|
|
286
|
+
- **No auto-commit.** Do not run any git command after writing the review
|
|
287
|
+
file. The user owns the commit decision.
|
|
288
|
+
|
|
289
|
+
## Configuration
|
|
290
|
+
|
|
291
|
+
The user can customize the council by telling the agent:
|
|
292
|
+
- Different models: "use Opus 4.5 instead of Opus 4.8"
|
|
293
|
+
- Different number of reviewers: "use 5 models" (but default is 3)
|
|
294
|
+
- Focus areas: "focus on security" or "focus on performance"
|
|
295
|
+
- Strictness: "be strict" (lower the noise threshold) or "only critical" (P1 only)
|
|
296
|
+
|
|
297
|
+
## Error Handling
|
|
298
|
+
|
|
299
|
+
- If one agent fails, proceed with the remaining 2. Note the failure.
|
|
300
|
+
- If two agents fail, fall back to a single-model review and explain.
|
|
301
|
+
- If all three fail, tell the user and suggest running a simple code-review instead.
|
|
302
|
+
|
|
303
|
+
## Phase 6 (Optional): Post-Review Verification
|
|
304
|
+
|
|
305
|
+
After presenting the council review, **offer** to run verification. Do not
|
|
306
|
+
run automatically — the user may just want the review.
|
|
307
|
+
|
|
308
|
+
If the user accepts:
|
|
309
|
+
|
|
310
|
+
1. **Run targeted tests** for changed files using the repo's test command
|
|
311
|
+
(discovered in Phase 0). Prefer the narrowest test scope first.
|
|
312
|
+
2. **Run the PR build command** when feasible (from `build-pr.yaml`).
|
|
313
|
+
3. **Run `git --no-pager diff --check`** for whitespace issues.
|
|
314
|
+
4. **Check PR template compliance** — if the repo has a PR template with
|
|
315
|
+
checkboxes, note which items are affected by the change.
|
|
316
|
+
5. **Report CODEOWNERS** — if the repo has CODEOWNERS, note which owners
|
|
317
|
+
are relevant for the changed files.
|
|
318
|
+
|
|
319
|
+
Append verification results to the review output.
|
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
# Council Review Output Format
|
|
2
|
+
|
|
3
|
+
Use this format when presenting the synthesized council review to the user.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Header
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
# 🏛️ LLM Council Code Review
|
|
11
|
+
|
|
12
|
+
**Scope:** <description of what was reviewed — branch, PR #, local changes>
|
|
13
|
+
**Council:** Claude Opus 4.8 · GPT-5.3 Codex · GPT-5.5
|
|
14
|
+
**Date:** <current date>
|
|
15
|
+
**Verdict:** <PASS | PASS WITH COMMENTS | NEEDS CHANGES>
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
### Verdict Rules
|
|
19
|
+
- **PASS** — No P1 or P2 issues found by any reviewer
|
|
20
|
+
- **PASS WITH COMMENTS** — No P1 issues; some P2/P3 found
|
|
21
|
+
- **NEEDS CHANGES** — At least one P1 issue found, OR 3+ P2 issues with majority agreement
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Consensus Findings (2/3 or 3/3 agreement)
|
|
26
|
+
|
|
27
|
+
These are issues flagged by multiple models independently. High confidence.
|
|
28
|
+
|
|
29
|
+
**Severity-based format for consensus findings:**
|
|
30
|
+
|
|
31
|
+
> [!IMPORTANT]
|
|
32
|
+
> **Emoji legend — use this table as the single source of truth. Do NOT apply the P3 majority emoji (🟡) to P2 findings.**
|
|
33
|
+
>
|
|
34
|
+
> | Severity | Unanimous (3/3) | Majority (2/3) | Solo (1/3) |
|
|
35
|
+
> |----------|----------------|----------------|------------|
|
|
36
|
+
> | P1 | 🚨 | 🚨 | 🚨 |
|
|
37
|
+
> | P2 | 🔴 | 🔴 | 🔴 |
|
|
38
|
+
> | P3 | 🔴 | 🟡 | 🔵 |
|
|
39
|
+
>
|
|
40
|
+
> The only cell that uses 🟡 is **P3 + Majority (2/3)**. Every other P2 cell is 🔴.
|
|
41
|
+
|
|
42
|
+
For **P1 (Critical)** consensus findings, wrap in a `> [!CAUTION]` blockquote regardless of agreement level:
|
|
43
|
+
|
|
44
|
+
```
|
|
45
|
+
> [!CAUTION]
|
|
46
|
+
> ### 🚨 <N>. <One-line summary> — CRITICAL
|
|
47
|
+
> 🚨 Unanimous (3/3) | 🚨 Majority (2/3) — **Severity: P1**
|
|
48
|
+
> **Category:** <category>
|
|
49
|
+
> **File:** `<path>` (lines ~<range>)
|
|
50
|
+
> **Agreed by:** Opus 4.8 ✓ · Codex 5.3 ✓ · GPT-5.5 ✓
|
|
51
|
+
>
|
|
52
|
+
> <Best description from the reviewers. Pick the clearest, most actionable version.>
|
|
53
|
+
>
|
|
54
|
+
> **Suggested fix:**
|
|
55
|
+
> <Most concrete suggestion from any reviewer.>
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
For **P2 (Important)** consensus findings — **both unanimous AND majority use 🔴**:
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
### <N>. <One-line summary>
|
|
62
|
+
🔴 Unanimous (3/3) | 🔴 Majority (2/3) ← both red for P2
|
|
63
|
+
**Severity:** P2
|
|
64
|
+
**Category:** <category>
|
|
65
|
+
**File:** `<path>` (lines ~<range>)
|
|
66
|
+
**Agreed by:** Opus 4.8 ✓ · Codex 5.3 ✓ · GPT-5.5 ✓
|
|
67
|
+
|
|
68
|
+
<Best description from the reviewers. Pick the clearest, most actionable version.>
|
|
69
|
+
|
|
70
|
+
**Suggested fix:**
|
|
71
|
+
<Most concrete suggestion from any reviewer.>
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
For **P3 (Minor)** consensus findings — **majority drops to 🟡 (yellow), unlike P2 which stays 🔴)**:
|
|
75
|
+
|
|
76
|
+
```
|
|
77
|
+
### <N>. <One-line summary>
|
|
78
|
+
🔴 Unanimous (3/3) | 🟡 Majority (2/3) ← majority is yellow ONLY for P3
|
|
79
|
+
**Severity:** P3
|
|
80
|
+
**Category:** <category>
|
|
81
|
+
**File:** `<path>` (lines ~<range>)
|
|
82
|
+
**Agreed by:** Opus 4.8 ✓ · Codex 5.3 ✓ · GPT-5.5 ✓
|
|
83
|
+
|
|
84
|
+
<Best description from the reviewers. Pick the clearest, most actionable version.>
|
|
85
|
+
|
|
86
|
+
**Suggested fix:**
|
|
87
|
+
<Most concrete suggestion from any reviewer.>
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## Minority Opinions (1/3 — solo catches)
|
|
93
|
+
|
|
94
|
+
These were flagged by only one model. They may be genuine catches the others
|
|
95
|
+
missed, or false positives. Included for completeness.
|
|
96
|
+
|
|
97
|
+
**Severity-based indicators for solo catches:**
|
|
98
|
+
- P1 (Critical): Use `🚨 CRITICAL — Solo catch by <model name>` — render with a blockquote warning prefix to make it visually unmissable
|
|
99
|
+
- P2 (Important): Use `🔴 Solo — flagged by <model name> only`
|
|
100
|
+
- P3 (Minor): Use `🔵 Solo — flagged by <model name> only`
|
|
101
|
+
|
|
102
|
+
For P1 solo catches, wrap the entire finding in a `> [!CAUTION]` blockquote to trigger a rendered red alert panel:
|
|
103
|
+
|
|
104
|
+
```
|
|
105
|
+
> [!CAUTION]
|
|
106
|
+
> ### 🚨 <N>. <One-line summary> — CRITICAL SOLO CATCH
|
|
107
|
+
> 🚨 CRITICAL — Solo catch by <model name> only — **not confirmed by other reviewers but high-impact if correct**
|
|
108
|
+
> **Severity:** P1
|
|
109
|
+
> **Category:** <category>
|
|
110
|
+
> **File:** `<path>` (lines ~<range>)
|
|
111
|
+
>
|
|
112
|
+
> <Description from the flagging model.>
|
|
113
|
+
>
|
|
114
|
+
> **Suggested fix:**
|
|
115
|
+
> <Suggestion if provided.>
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
For P2/P3 solo catches, use the standard format:
|
|
119
|
+
|
|
120
|
+
```
|
|
121
|
+
### <N>. <One-line summary>
|
|
122
|
+
🔴 Solo — flagged by <model name> only ← use 🔴 for P2, 🔵 for P3
|
|
123
|
+
**Severity:** P2 | P3
|
|
124
|
+
**Category:** <category>
|
|
125
|
+
**File:** `<path>` (lines ~<range>)
|
|
126
|
+
|
|
127
|
+
<Description from the flagging model.>
|
|
128
|
+
|
|
129
|
+
**Suggested fix:**
|
|
130
|
+
<Suggestion if provided.>
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
---
|
|
134
|
+
|
|
135
|
+
## Review Statistics
|
|
136
|
+
|
|
137
|
+
```
|
|
138
|
+
| Metric | Value |
|
|
139
|
+
|---------------------------|-------|
|
|
140
|
+
| Total unique issues | <N> |
|
|
141
|
+
| Unanimous (3/3) | <N> |
|
|
142
|
+
| Majority (2/3) | <N> |
|
|
143
|
+
| Solo (1/3) | <N> |
|
|
144
|
+
| P1 (Critical) | <N> |
|
|
145
|
+
| P2 (Important) | <N> |
|
|
146
|
+
| P3 (Minor) | <N> |
|
|
147
|
+
| Files reviewed | <N> |
|
|
148
|
+
| Lines changed | +<N> / -<N> |
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## Model Agreement Matrix (optional, for large reviews)
|
|
154
|
+
|
|
155
|
+
Show which model caught what. Only include for reviews with 5+ findings.
|
|
156
|
+
|
|
157
|
+
```
|
|
158
|
+
| # | Finding | Opus 4.8 | Codex 5.3 | GPT-5.5 |
|
|
159
|
+
|---|--------------------------------|----------|-----------|---------|
|
|
160
|
+
| 1 | Race condition in UserService | ✓ | ✓ | ✓ |
|
|
161
|
+
| 2 | Missing null check in parser | ✓ | ✓ | |
|
|
162
|
+
| 3 | SQL injection in search filter | | ✓ | ✓ |
|
|
163
|
+
| 4 | Unused import (solo) | ✓ | | |
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
---
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Reviewer Prompt Template
|
|
2
|
+
|
|
3
|
+
You are one member of a 3-model code review council. Your job is to independently
|
|
4
|
+
review the code changes below and produce high-signal findings. Another process
|
|
5
|
+
will compare your review against two other models' reviews to find consensus.
|
|
6
|
+
|
|
7
|
+
## Your Review Constraints
|
|
8
|
+
|
|
9
|
+
- **Only flag things that genuinely matter.** Bugs, security issues, logic errors,
|
|
10
|
+
performance problems, missing error handling, test gaps for changed code.
|
|
11
|
+
- **Never comment on style, formatting, naming preferences, or trivial matters**
|
|
12
|
+
unless they cause a real problem (e.g., a misleading variable name that could
|
|
13
|
+
cause a bug).
|
|
14
|
+
- **Be specific.** Always include the file path, approximate line range, and a
|
|
15
|
+
concrete description of the problem.
|
|
16
|
+
- **Suggest a fix** when possible. Don't just say "this is wrong" — say what to do.
|
|
17
|
+
- **Don't be redundant.** If two issues share the same root cause, report it once.
|
|
18
|
+
- **Respect repo-specific rules.** The project context below includes this repo's
|
|
19
|
+
own conventions, technology profile, and CI expectations. Review against THOSE
|
|
20
|
+
rules, not generic best practices. Do not assume patterns from other repos.
|
|
21
|
+
|
|
22
|
+
## Project Context
|
|
23
|
+
|
|
24
|
+
{PROJECT_CONTEXT}
|
|
25
|
+
|
|
26
|
+
This context was discovered from the repo's own guidance files, CI workflows,
|
|
27
|
+
build configuration, and technology markers. If the context mentions specific
|
|
28
|
+
patterns (e.g., Micronaut DI, tenant isolation, cache key format), verify the
|
|
29
|
+
diff follows them. If the context is silent on something, don't invent rules.
|
|
30
|
+
|
|
31
|
+
## Technology Profile
|
|
32
|
+
|
|
33
|
+
{TECHNOLOGY_PROFILE}
|
|
34
|
+
|
|
35
|
+
## Change Classification
|
|
36
|
+
|
|
37
|
+
{CHANGE_CLASSIFICATION}
|
|
38
|
+
|
|
39
|
+
## Review Focus Areas (from repo's CI/review config)
|
|
40
|
+
|
|
41
|
+
Review against these standard areas, but weight them based on the change
|
|
42
|
+
classification above:
|
|
43
|
+
|
|
44
|
+
1. **Code quality:** single responsibility, clarity, maintainability, unnecessary
|
|
45
|
+
complexity/nesting, redundant abstractions, local style conventions.
|
|
46
|
+
2. **Security:** auth, authorization, tenant isolation, input validation, secrets,
|
|
47
|
+
sensitive data exposure.
|
|
48
|
+
3. **Performance:** database/query shape, cache behavior, external calls,
|
|
49
|
+
async/blocking boundaries, memory/resource lifecycle.
|
|
50
|
+
4. **Testing:** adequate coverage for changed code, edge cases, missing scenarios.
|
|
51
|
+
5. **Documentation:** README/docs/OpenAPI/API docs accuracy when behavior changes.
|
|
52
|
+
|
|
53
|
+
## Changed Files Summary
|
|
54
|
+
|
|
55
|
+
{DIFF_STAT}
|
|
56
|
+
|
|
57
|
+
## Full Diff
|
|
58
|
+
|
|
59
|
+
{DIFF}
|
|
60
|
+
|
|
61
|
+
## Output Format
|
|
62
|
+
|
|
63
|
+
Return your findings as a structured list. Each finding must follow this exact format:
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
### Finding <N>
|
|
67
|
+
- **File:** <path/to/file>
|
|
68
|
+
- **Lines:** <start>-<end> (approximate)
|
|
69
|
+
- **Severity:** P1 | P2 | P3
|
|
70
|
+
- **Category:** bug | security | performance | design | test-gap | error-handling | concurrency | data-integrity
|
|
71
|
+
- **Summary:** <one-line summary>
|
|
72
|
+
- **Details:** <1-3 sentences explaining the issue>
|
|
73
|
+
- **Suggestion:** <concrete fix or action>
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Severity Guide
|
|
77
|
+
|
|
78
|
+
- **P1 — Critical:** Likely bug, security vulnerability, data loss, crash, race condition,
|
|
79
|
+
or production incident. Must fix before merge.
|
|
80
|
+
- **P2 — Important:** Behavior regression, missing important test, incorrect error handling,
|
|
81
|
+
performance issue under realistic load. Should fix before merge.
|
|
82
|
+
- **P3 — Minor:** Low-risk test gap, minor inefficiency, documentation inaccuracy.
|
|
83
|
+
Nice to fix but not blocking.
|
|
84
|
+
|
|
85
|
+
### What NOT to Report
|
|
86
|
+
|
|
87
|
+
- Style or formatting preferences
|
|
88
|
+
- "Consider renaming X" suggestions
|
|
89
|
+
- "Add a comment explaining Y" suggestions
|
|
90
|
+
- Import ordering
|
|
91
|
+
- Trailing whitespace
|
|
92
|
+
- Suggestions that don't prevent a real problem
|
|
93
|
+
|
|
94
|
+
If you find zero genuine issues, return:
|
|
95
|
+
|
|
96
|
+
```
|
|
97
|
+
### No Issues Found
|
|
98
|
+
The changes look correct. No bugs, security issues, or significant concerns identified.
|
|
99
|
+
```
|