@excitedjs/dreamux 0.22.0-beta.156 → 0.22.0-beta.158
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@excitedjs/dreamux",
|
|
3
|
-
"version": "0.22.0-beta.
|
|
3
|
+
"version": "0.22.0-beta.158",
|
|
4
4
|
"description": "Dreamux core host for multi-dispatcher agent orchestration through provider interfaces.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -26,11 +26,11 @@
|
|
|
26
26
|
"CHANGELOG.json"
|
|
27
27
|
],
|
|
28
28
|
"dependencies": {
|
|
29
|
-
"@excitedjs/agent-runtime-claude-code": "0.5.0-beta.
|
|
30
|
-
"@excitedjs/agent-runtime-codex": "0.3.5-beta.
|
|
31
|
-
"@excitedjs/dreamux-types": "0.8.0-beta.
|
|
32
|
-
"@excitedjs/dreamux-utils": "0.4.0-beta.
|
|
33
|
-
"@excitedjs/feishu-channel": "5.0.0-beta.
|
|
29
|
+
"@excitedjs/agent-runtime-claude-code": "0.5.0-beta.158",
|
|
30
|
+
"@excitedjs/agent-runtime-codex": "0.3.5-beta.158",
|
|
31
|
+
"@excitedjs/dreamux-types": "0.8.0-beta.158",
|
|
32
|
+
"@excitedjs/dreamux-utils": "0.4.0-beta.158",
|
|
33
|
+
"@excitedjs/feishu-channel": "5.0.0-beta.158",
|
|
34
34
|
"acorn": "^8.16.0",
|
|
35
35
|
"croner": "^10.0.1",
|
|
36
36
|
"smol-toml": "^1.6.1",
|
|
@@ -186,3 +186,10 @@ export default async function run() {
|
|
|
186
186
|
Scripts cannot import modules or use host process, filesystem, network, or timer
|
|
187
187
|
APIs. `Date.now()`, `Math.random()`, and `new Date()` without arguments are also
|
|
188
188
|
unavailable so orchestration stays deterministic.
|
|
189
|
+
|
|
190
|
+
## References
|
|
191
|
+
|
|
192
|
+
| Task | Load when | Reference |
|
|
193
|
+
| --- | --- | --- |
|
|
194
|
+
| Code review run | Carrying a change review (pull/merge request, git range, or working-tree diff) as one workflow run: eligibility gate, multi-lens finders, confidence-scored verification, one synthesized report. | [Code review](references/code-review.md) |
|
|
195
|
+
| Orchestration and prompt patterns | Writing TeamMate prompts inside scripts, choosing `pipeline` versus `parallel`, and applying quality patterns: adversarial verify, judge panel, loop-until-dry, multi-modal sweep, completeness critic. | [Orchestration patterns](references/orchestration-patterns.md) |
|
|
@@ -0,0 +1,395 @@
|
|
|
1
|
+
# Code Review Workflow
|
|
2
|
+
|
|
3
|
+
`../SKILL.md` owns the run tools and the script API. This reference owns one
|
|
4
|
+
recipe: carrying a full code review as a single `workflow_run` — an eligibility
|
|
5
|
+
gate, context collection, independent multi-lens finders, per-finding
|
|
6
|
+
confidence scoring, and one synthesized report.
|
|
7
|
+
|
|
8
|
+
## Contents
|
|
9
|
+
|
|
10
|
+
- [Stage design](#stage-design) — the five stages and their failure accounting
|
|
11
|
+
- [Prompt discipline](#prompt-discipline) — output contract, rejection
|
|
12
|
+
semantics, self-contained prompts
|
|
13
|
+
- [TeamMate budget](#teammate-budget) — cost arithmetic against the run caps
|
|
14
|
+
- [Script](#script) — the complete runnable module
|
|
15
|
+
- [Invocation](#invocation) — call shape and result fields
|
|
16
|
+
|
|
17
|
+
The review target is any change the Team workspace can read: a GitHub pull
|
|
18
|
+
request, a GitLab merge request, a local branch or commit range, or an
|
|
19
|
+
uncommitted working-tree diff. Use this recipe when the operator wants one
|
|
20
|
+
terminal result for such a change. For interactive review — where the next
|
|
21
|
+
instruction depends on reading the previous finding — use `spawn` and `send`
|
|
22
|
+
instead.
|
|
23
|
+
|
|
24
|
+
## Stage design
|
|
25
|
+
|
|
26
|
+
1. **Gate** — one TeamMate checks eligibility and the run exits early when
|
|
27
|
+
review is not needed: the change is closed or a draft, is automated or
|
|
28
|
+
trivially safe, or already carries a review from this Team.
|
|
29
|
+
2. **Context** — two TeamMates in parallel: one lists the file paths (not
|
|
30
|
+
contents) of the repository's agent guidance files (`CLAUDE.md`,
|
|
31
|
+
`AGENTS.md`) at the root and in directories the change touches; one
|
|
32
|
+
summarizes the change (intent, scope, risk areas). Both results feed every
|
|
33
|
+
later prompt. When guidance discovery fails, the guidance lens is skipped
|
|
34
|
+
and recorded as missing coverage — it must not audit against a silently
|
|
35
|
+
empty list. A missing summary is tolerable: finders read the change
|
|
36
|
+
themselves, and the placeholder is visible in their prompts.
|
|
37
|
+
3. **Find** — independent finder lenses run in parallel, each blind to the
|
|
38
|
+
others: guidance-file compliance, a shallow bug scan limited to the changed
|
|
39
|
+
hunks, git history and blame context, review comments on prior change
|
|
40
|
+
requests touching the same files, guidance stated in code comments of the
|
|
41
|
+
modified files, and concrete security issues introduced by the change. The
|
|
42
|
+
barrier is justified: deduplication needs every lens's output at once —
|
|
43
|
+
and it is where coverage is accounted: lens results are index-aligned with
|
|
44
|
+
the lens list, so a failed lens is recorded as missing coverage instead of
|
|
45
|
+
being silently dropped by `.filter(Boolean)`.
|
|
46
|
+
4. **Verify** — each unique finding is scored by three TeamMates with distinct
|
|
47
|
+
focus questions (does it reproduce on the changed lines; is it pre-existing
|
|
48
|
+
rather than introduced; does the cited guidance actually say that), using a
|
|
49
|
+
fixed 0-100 rubric. Findings advance through `pipeline` independently of
|
|
50
|
+
one another: each finding's vote aggregation follows its own three votes
|
|
51
|
+
without waiting for the other findings. A finding that keeps fewer than two
|
|
52
|
+
settled votes is returned as `unverified` — verifier failure must not
|
|
53
|
+
silently launder a finding into a clean report.
|
|
54
|
+
5. **Report** — findings that keep at least two settled votes and average 80 or
|
|
55
|
+
higher survive; one TeamMate formats the final review comment, noting
|
|
56
|
+
incomplete coverage and unverified findings when there are any.
|
|
57
|
+
|
|
58
|
+
The run returns the report plus `coverage` and `unverified` records; the
|
|
59
|
+
caller decides where the report goes. When a channel reply tool is available, post it there; or
|
|
60
|
+
have a follow-up `send` to the reporter TeamMate publish it with the forge's
|
|
61
|
+
own tooling (for example `gh` for GitHub or `glab` for GitLab) once the
|
|
62
|
+
operator confirms.
|
|
63
|
+
|
|
64
|
+
## Prompt discipline
|
|
65
|
+
|
|
66
|
+
Every workflow TeamMate receives the workflow output contract: its final
|
|
67
|
+
response is the value `agent()` returns, not a human-facing message. Prompts
|
|
68
|
+
can therefore ask for data directly, without output-format boilerplate. With
|
|
69
|
+
`schema`, the runtime's native structured-output mechanism enforces the shape.
|
|
70
|
+
|
|
71
|
+
Failure semantics drive the script structure:
|
|
72
|
+
|
|
73
|
+
- An ordinary failed turn settles the `agent()` call to `null`. `parallel`
|
|
74
|
+
and `pipeline` contain per-item failures as `null` entries and keep results
|
|
75
|
+
index-aligned with their inputs — account required coverage positionally
|
|
76
|
+
first (which lens failed, which finding lost its verifiers), then drop
|
|
77
|
+
nulls with `.filter(Boolean)` only where item identity no longer matters.
|
|
78
|
+
- A schema call rejects — it does not return `null` — when the runtime cannot
|
|
79
|
+
provide structured output or reports success with an empty or invalid JSON
|
|
80
|
+
result. A directly awaited rejection fails the whole run; that is the right
|
|
81
|
+
outcome for load-bearing calls like the gate, and the reason the gate below
|
|
82
|
+
is awaited bare while per-finding work runs inside helpers.
|
|
83
|
+
|
|
84
|
+
Keep prompts self-contained: a finder must not depend on another finder's
|
|
85
|
+
output, and everything a verifier needs (the finding, the rubric, the
|
|
86
|
+
false-positive list) is embedded in its prompt.
|
|
87
|
+
|
|
88
|
+
Instruct finders and verifiers to treat these as false positives: pre-existing
|
|
89
|
+
issues; code that looks buggy but is not; pedantic nitpicks; anything a
|
|
90
|
+
linter, typechecker, or CI would catch; general quality concerns not required
|
|
91
|
+
by a guidance file; issues explicitly silenced in code; intentional behavior
|
|
92
|
+
changes related to the broader change; and real issues on lines the change did
|
|
93
|
+
not modify.
|
|
94
|
+
|
|
95
|
+
## TeamMate budget
|
|
96
|
+
|
|
97
|
+
One round costs `1 (gate) + 2 (context) + 6 (find) + 3 × findings (verify) +
|
|
98
|
+
1 (report)` TeamMates. The 1000-TeamMate lifetime cap per run leaves ample
|
|
99
|
+
headroom for a single round (hundreds of findings) and comfortably supports
|
|
100
|
+
repeat-until-dry rounds for exhaustive coverage; still count
|
|
101
|
+
`6 finders + 3 × fresh findings` per extra round before choosing round
|
|
102
|
+
counts, and match fan-out to what the operator asked for.
|
|
103
|
+
|
|
104
|
+
## Script
|
|
105
|
+
|
|
106
|
+
The script uses the top-level entry: the body follows `export const meta`
|
|
107
|
+
directly, and `await` and early `return` work at top level. Dreamux wraps
|
|
108
|
+
everything after the meta export — constants and helper functions included —
|
|
109
|
+
into the canonical entry function; hoisting inside the wrapped body keeps the
|
|
110
|
+
helpers callable exactly as written.
|
|
111
|
+
|
|
112
|
+
```js
|
|
113
|
+
export const meta = {
|
|
114
|
+
name: 'code-review',
|
|
115
|
+
description: 'Multi-lens code review with confidence-scored findings and one report',
|
|
116
|
+
whenToUse: 'One-shot review of a change request, git range, or working-tree diff.',
|
|
117
|
+
phases: [
|
|
118
|
+
{ title: 'gate', detail: 'eligibility check with early exit' },
|
|
119
|
+
{ title: 'context', detail: 'guidance discovery and change summary' },
|
|
120
|
+
{ title: 'find', detail: 'six independent finder lenses' },
|
|
121
|
+
{ title: 'verify', detail: 'three-vote confidence scoring per finding' },
|
|
122
|
+
{ title: 'report', detail: 'synthesized review comment' },
|
|
123
|
+
],
|
|
124
|
+
};
|
|
125
|
+
|
|
126
|
+
const FALSE_POSITIVES =
|
|
127
|
+
'Treat as false positives: pre-existing issues; code that looks buggy but is not; ' +
|
|
128
|
+
'pedantic nitpicks; anything a linter, typechecker, or CI would catch; general ' +
|
|
129
|
+
'quality concerns (coverage, docs) unless a guidance file (CLAUDE.md, AGENTS.md) ' +
|
|
130
|
+
'requires them; issues explicitly silenced in code; intentional behavior changes ' +
|
|
131
|
+
'related to the broader change; real issues on unmodified lines.';
|
|
132
|
+
|
|
133
|
+
const RUBRIC =
|
|
134
|
+
'0: Not confident at all. False positive that does not stand up to light scrutiny, or a pre-existing issue.\n' +
|
|
135
|
+
'25: Somewhat confident. Might be real, might be a false positive; could not verify. If stylistic, it is not explicitly called out in a relevant guidance file (CLAUDE.md, AGENTS.md).\n' +
|
|
136
|
+
'50: Moderately confident. Verified real, but may be a nitpick or rare in practice; not very important relative to the rest of the change.\n' +
|
|
137
|
+
'75: Highly confident. Double checked; very likely real and will be hit in practice; directly impacts functionality or is directly mentioned in a guidance file.\n' +
|
|
138
|
+
'100: Absolutely certain. Double checked and confirmed; will happen frequently; evidence directly confirms it.';
|
|
139
|
+
|
|
140
|
+
const FINDINGS_SCHEMA = {
|
|
141
|
+
type: 'object',
|
|
142
|
+
properties: {
|
|
143
|
+
findings: {
|
|
144
|
+
type: 'array',
|
|
145
|
+
items: {
|
|
146
|
+
type: 'object',
|
|
147
|
+
properties: {
|
|
148
|
+
file: { type: 'string' },
|
|
149
|
+
line: { type: 'integer' },
|
|
150
|
+
title: { type: 'string' },
|
|
151
|
+
detail: { type: 'string' },
|
|
152
|
+
reason: {
|
|
153
|
+
type: 'string',
|
|
154
|
+
description: 'guidance | bug | history | prior-review | comment | security',
|
|
155
|
+
},
|
|
156
|
+
},
|
|
157
|
+
required: ['file', 'title', 'detail', 'reason'],
|
|
158
|
+
additionalProperties: false,
|
|
159
|
+
},
|
|
160
|
+
},
|
|
161
|
+
},
|
|
162
|
+
required: ['findings'],
|
|
163
|
+
additionalProperties: false,
|
|
164
|
+
};
|
|
165
|
+
|
|
166
|
+
const SCORE_SCHEMA = {
|
|
167
|
+
type: 'object',
|
|
168
|
+
properties: {
|
|
169
|
+
score: { type: 'integer', minimum: 0, maximum: 100 },
|
|
170
|
+
reasoning: { type: 'string' },
|
|
171
|
+
},
|
|
172
|
+
required: ['score', 'reasoning'],
|
|
173
|
+
additionalProperties: false,
|
|
174
|
+
};
|
|
175
|
+
|
|
176
|
+
function findingKey(f) {
|
|
177
|
+
return `${f.file}:${f.line || 0}:${f.title.toLowerCase().replace(/\s+/g, ' ').trim()}`;
|
|
178
|
+
}
|
|
179
|
+
|
|
180
|
+
const target = args && args.target;
|
|
181
|
+
if (!target) {
|
|
182
|
+
return { error: 'args.target is required: a PR/MR URL or number, a git range, or a working-tree description' };
|
|
183
|
+
}
|
|
184
|
+
|
|
185
|
+
phase('gate');
|
|
186
|
+
const gate = await agent(
|
|
187
|
+
`Inspect this change: ${target}. Decide whether it should be code reviewed. ` +
|
|
188
|
+
`Ineligible if: closed; a draft; an automated or trivial change that is ` +
|
|
189
|
+
`obviously fine; or it already has a review comment from this Team.`,
|
|
190
|
+
{
|
|
191
|
+
label: 'gate',
|
|
192
|
+
phase: 'gate',
|
|
193
|
+
intent: 'Review eligibility gate',
|
|
194
|
+
schema: {
|
|
195
|
+
type: 'object',
|
|
196
|
+
properties: { eligible: { type: 'boolean' }, reason: { type: 'string' } },
|
|
197
|
+
required: ['eligible', 'reason'],
|
|
198
|
+
additionalProperties: false,
|
|
199
|
+
},
|
|
200
|
+
},
|
|
201
|
+
);
|
|
202
|
+
if (!gate || !gate.eligible) {
|
|
203
|
+
return { skipped: true, reason: gate ? gate.reason : 'gate TeamMate failed' };
|
|
204
|
+
}
|
|
205
|
+
|
|
206
|
+
phase('context');
|
|
207
|
+
const ctx = await parallel([
|
|
208
|
+
() => agent(
|
|
209
|
+
`For this change: ${target}. List the file paths (NOT contents) of the ` +
|
|
210
|
+
`repository's agent guidance files — CLAUDE.md and AGENTS.md — at the ` +
|
|
211
|
+
`repository root and in directories whose files the change modifies.`,
|
|
212
|
+
{
|
|
213
|
+
label: 'guidance',
|
|
214
|
+
phase: 'context',
|
|
215
|
+
intent: 'Guidance file discovery',
|
|
216
|
+
schema: {
|
|
217
|
+
type: 'object',
|
|
218
|
+
properties: { paths: { type: 'array', items: { type: 'string' } } },
|
|
219
|
+
required: ['paths'],
|
|
220
|
+
additionalProperties: false,
|
|
221
|
+
},
|
|
222
|
+
},
|
|
223
|
+
),
|
|
224
|
+
() => agent(
|
|
225
|
+
`View this change: ${target}. Summarize it: intent, scope, key files, risk ` +
|
|
226
|
+
`areas. Concise prose only.`,
|
|
227
|
+
{ label: 'summary', phase: 'context', intent: 'Change summarizer' },
|
|
228
|
+
),
|
|
229
|
+
]);
|
|
230
|
+
// a null discovery result is a coverage gap, distinct from a successful
|
|
231
|
+
// empty list (a repo that simply has no guidance files)
|
|
232
|
+
const guidanceFailed = !ctx[0];
|
|
233
|
+
const guidancePaths = guidanceFailed ? [] : ctx[0].paths;
|
|
234
|
+
const summary = ctx[1] || '(summary unavailable)';
|
|
235
|
+
|
|
236
|
+
const lenses = [
|
|
237
|
+
{ key: 'guidance', prompt: `Audit the change for compliance with these guidance files (read them): ${JSON.stringify(guidancePaths)}. Guidance files direct code writing, so not every instruction applies at review time.` },
|
|
238
|
+
{ key: 'shallow-bugs', prompt: 'Read only the changed hunks and scan for obvious large bugs. Do not read extra context. Skip small issues and nitpicks.' },
|
|
239
|
+
{ key: 'git-history', prompt: 'Read git blame and history of the modified code; flag bugs visible only in light of that historical context, such as reintroduced bugs or violated invariants.' },
|
|
240
|
+
{ key: 'prior-reviews', prompt: 'Find previous change requests (pull/merge requests) touching these files; check review comments on them that also apply to this change.' },
|
|
241
|
+
{ key: 'code-comments', prompt: 'Read code comments in the modified files; flag violations of guidance stated in those comments.' },
|
|
242
|
+
{ key: 'security', prompt: 'Review the changed code for concrete, exploitable security issues introduced by this change (injection, authorization gaps, secret handling). No generic hardening advice.' },
|
|
243
|
+
];
|
|
244
|
+
|
|
245
|
+
phase('find');
|
|
246
|
+
// without discovered guidance files the guidance lens has nothing real to
|
|
247
|
+
// audit — skip it and record the gap instead of auditing an empty list
|
|
248
|
+
const activeLenses = guidanceFailed
|
|
249
|
+
? lenses.filter((lens) => lens.key !== 'guidance')
|
|
250
|
+
: lenses;
|
|
251
|
+
// parallel() preserves order, so a null entry identifies the failed lens
|
|
252
|
+
const lensResults = await parallel(activeLenses.map((lens) => () =>
|
|
253
|
+
agent(
|
|
254
|
+
`Code review this change through one lens: ${target}\n${lens.prompt}\n` +
|
|
255
|
+
`${FALSE_POSITIVES}\nChange summary:\n${summary}`,
|
|
256
|
+
{
|
|
257
|
+
label: `find-${lens.key}`,
|
|
258
|
+
phase: 'find',
|
|
259
|
+
intent: `Finder lens: ${lens.key}`,
|
|
260
|
+
schema: FINDINGS_SCHEMA,
|
|
261
|
+
},
|
|
262
|
+
),
|
|
263
|
+
));
|
|
264
|
+
const failedLenses = [
|
|
265
|
+
...(guidanceFailed ? ['guidance'] : []),
|
|
266
|
+
...activeLenses.filter((lens, i) => !lensResults[i]).map((lens) => lens.key),
|
|
267
|
+
];
|
|
268
|
+
if (failedLenses.length === lenses.length) {
|
|
269
|
+
return { error: 'every finder lens failed; the change was not reviewed', failedLenses };
|
|
270
|
+
}
|
|
271
|
+
if (failedLenses.length) {
|
|
272
|
+
log(`coverage incomplete: failed lenses ${failedLenses.join(', ')}`);
|
|
273
|
+
}
|
|
274
|
+
const found = lensResults.filter(Boolean).flatMap((r) => r.findings);
|
|
275
|
+
|
|
276
|
+
const seen = new Set();
|
|
277
|
+
const unique = [];
|
|
278
|
+
for (const f of found) {
|
|
279
|
+
const key = findingKey(f);
|
|
280
|
+
if (!seen.has(key)) { seen.add(key); unique.push(f); }
|
|
281
|
+
}
|
|
282
|
+
log(`${found.length} raw findings, ${unique.length} unique`);
|
|
283
|
+
|
|
284
|
+
const checkedLenses = lenses
|
|
285
|
+
.filter((lens) => !failedLenses.includes(lens.key))
|
|
286
|
+
.map((lens) => lens.key);
|
|
287
|
+
function coverageRecord(unverifiedCount) {
|
|
288
|
+
return {
|
|
289
|
+
complete: failedLenses.length === 0 && unverifiedCount === 0,
|
|
290
|
+
failedLenses,
|
|
291
|
+
unverifiedFindings: unverifiedCount,
|
|
292
|
+
};
|
|
293
|
+
}
|
|
294
|
+
const noIssuesReport = failedLenses.length === 0
|
|
295
|
+
? 'No issues found. Checked for bugs, security, history, and guidance-file compliance.'
|
|
296
|
+
: `No issues found by the lenses that completed (${checkedLenses.join(', ')}). ` +
|
|
297
|
+
`Coverage is incomplete: ${failedLenses.join(', ')} failed.`;
|
|
298
|
+
if (!unique.length) {
|
|
299
|
+
return { issues: [], unverified: [], coverage: coverageRecord(0), report: noIssuesReport };
|
|
300
|
+
}
|
|
301
|
+
|
|
302
|
+
phase('verify');
|
|
303
|
+
const focusQuestions = [
|
|
304
|
+
'does it reproduce on the changed lines',
|
|
305
|
+
'is it pre-existing rather than introduced by this change',
|
|
306
|
+
'does the cited guidance file or code comment actually say that',
|
|
307
|
+
];
|
|
308
|
+
const judged = await pipeline(
|
|
309
|
+
unique,
|
|
310
|
+
(finding) => parallel(focusQuestions.map((focus) => () =>
|
|
311
|
+
agent(
|
|
312
|
+
`Confidence-score this code review finding for the change: ${target}\n` +
|
|
313
|
+
`Finding: ${JSON.stringify(finding)}\n` +
|
|
314
|
+
`Guidance files: ${JSON.stringify(guidancePaths)}\n` +
|
|
315
|
+
`Focus question: ${focus}.\n${FALSE_POSITIVES}\n` +
|
|
316
|
+
`Score with this rubric verbatim:\n${RUBRIC}`,
|
|
317
|
+
{ phase: 'verify', intent: 'Finding verifier', schema: SCORE_SCHEMA },
|
|
318
|
+
),
|
|
319
|
+
)),
|
|
320
|
+
(votes, finding, index) => {
|
|
321
|
+
const settled = (votes || []).filter(Boolean);
|
|
322
|
+
const average = settled.length
|
|
323
|
+
? settled.reduce((sum, vote) => sum + vote.score, 0) / settled.length
|
|
324
|
+
: 0;
|
|
325
|
+
return { ...finding, confidence: Math.round(average), votes: settled.length, index };
|
|
326
|
+
},
|
|
327
|
+
);
|
|
328
|
+
// pipeline() results are index-aligned with `unique`: a null entry or a
|
|
329
|
+
// finding with fewer than two settled votes is unverified, not clean
|
|
330
|
+
const unverified = unique.filter((f, i) => {
|
|
331
|
+
const j = judged[i];
|
|
332
|
+
return !j || j.votes < 2;
|
|
333
|
+
});
|
|
334
|
+
const confirmed = judged
|
|
335
|
+
.filter(Boolean)
|
|
336
|
+
.filter((f) => f.votes >= 2 && f.confidence >= 80);
|
|
337
|
+
const coverage = coverageRecord(unverified.length);
|
|
338
|
+
log(`${confirmed.length} findings at >=80 confidence, ${unverified.length} unverified`);
|
|
339
|
+
|
|
340
|
+
phase('report');
|
|
341
|
+
const unverifiedNote = unverified.length
|
|
342
|
+
? ` ${unverified.length} finding(s) could not be verified (verifier failures) ` +
|
|
343
|
+
`and are returned in \`unverified\` for manual triage.`
|
|
344
|
+
: '';
|
|
345
|
+
if (!confirmed.length) {
|
|
346
|
+
const reportText = unverified.length
|
|
347
|
+
? `No issues confirmed.${unverifiedNote}`
|
|
348
|
+
: noIssuesReport;
|
|
349
|
+
return { issues: [], unverified, coverage, report: reportText };
|
|
350
|
+
}
|
|
351
|
+
const report = await agent(
|
|
352
|
+
`Write the final code review comment for the change: ${target}\n` +
|
|
353
|
+
`Confirmed issues (JSON): ${JSON.stringify(confirmed)}\n` +
|
|
354
|
+
`Unverified findings — verifiers failed, no confidence claim (JSON): ${JSON.stringify(unverified)}\n` +
|
|
355
|
+
`Coverage (JSON): ${JSON.stringify(coverage)}\n` +
|
|
356
|
+
`Format: a "### Code review" heading, "Found N issues:", then a numbered list; ` +
|
|
357
|
+
`each item is a brief description with the flag reason in parentheses and a ` +
|
|
358
|
+
`citation on the next line — file#line when the finding carries a line number, ` +
|
|
359
|
+
`the file path alone otherwise; never invent a line number. If there are ` +
|
|
360
|
+
`unverified findings, add a short "Unverified" section listing them without ` +
|
|
361
|
+
`confidence claims. If coverage is incomplete, end with one line naming the ` +
|
|
362
|
+
`failed lenses. Brief, no emojis. Return markdown only.`,
|
|
363
|
+
{ label: 'report', phase: 'report', intent: 'Review report writer' },
|
|
364
|
+
);
|
|
365
|
+
return { issues: confirmed, unverified, coverage, report };
|
|
366
|
+
```
|
|
367
|
+
|
|
368
|
+
The verify stage shows the three-argument stage signature: stage one returns
|
|
369
|
+
only the vote array, and stage two recovers the finding and its position from
|
|
370
|
+
`(votes, finding, index)` instead of threading them through the return value.
|
|
371
|
+
|
|
372
|
+
## Invocation
|
|
373
|
+
|
|
374
|
+
```
|
|
375
|
+
workflow_run({
|
|
376
|
+
script: <the module above>,
|
|
377
|
+
args: { target: 'https://github.com/owner/repo/pull/123' },
|
|
378
|
+
})
|
|
379
|
+
```
|
|
380
|
+
|
|
381
|
+
`args.target` is forge-neutral — pass whatever identifies the change in the
|
|
382
|
+
Team workspace:
|
|
383
|
+
|
|
384
|
+
- a pull-request or merge-request URL or number
|
|
385
|
+
(`'https://gitlab.example.com/group/repo/-/merge_requests/45'`, `'#123'`);
|
|
386
|
+
- a local git range (`'main...feature'`);
|
|
387
|
+
- a working-tree description (`'the uncommitted diff in the workspace'`).
|
|
388
|
+
|
|
389
|
+
`max_concurrency` defaults to 16; lower it only when the workspace or runtime
|
|
390
|
+
provider needs gentler fan-out. After the terminal completion, read `issues`
|
|
391
|
+
for the confirmed findings, `unverified` for findings whose verifiers failed,
|
|
392
|
+
`coverage` for whether every finder lens ran and every finding was verified,
|
|
393
|
+
and `report` for the formatted comment; use the reporter TeamMate's concrete
|
|
394
|
+
name with `send` for follow-ups such as posting the comment through the
|
|
395
|
+
forge's tooling or re-checking one finding interactively.
|
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
# Orchestration and Prompt Patterns
|
|
2
|
+
|
|
3
|
+
`../SKILL.md` owns the run tools and the script API. This reference owns the
|
|
4
|
+
craft: how to write TeamMate prompts inside workflow scripts, when to reach
|
|
5
|
+
for `pipeline` versus `parallel`, and the reusable quality patterns that make
|
|
6
|
+
a run's output trustworthy. Snippets are fragments of a script body (either
|
|
7
|
+
entry form).
|
|
8
|
+
|
|
9
|
+
## Contents
|
|
10
|
+
|
|
11
|
+
- [Prompt discipline](#prompt-discipline) — output contract, failure
|
|
12
|
+
plumbing, persona options
|
|
13
|
+
- [pipeline versus parallel](#pipeline-versus-parallel) — barrier rules and
|
|
14
|
+
the smell test
|
|
15
|
+
- [Quality patterns](#quality-patterns) — adversarial verify, judge panel,
|
|
16
|
+
loop-until-dry, sweeps, critics
|
|
17
|
+
- [Scale to the ask](#scale-to-the-ask) — fan-out sizing and budget
|
|
18
|
+
arithmetic
|
|
19
|
+
|
|
20
|
+
## Prompt discipline
|
|
21
|
+
|
|
22
|
+
Every workflow TeamMate receives the workflow output contract: its final
|
|
23
|
+
response is the value `agent()` returns — a data channel, not a human-facing
|
|
24
|
+
message. Write prompts accordingly.
|
|
25
|
+
|
|
26
|
+
- **Data calls.** With `schema`, the runtime's native structured-output
|
|
27
|
+
mechanism enforces the shape; ask for the data directly and skip
|
|
28
|
+
output-format boilerplate.
|
|
29
|
+
- **Text calls.** For unstructured returns, still pin the shape: "Return the
|
|
30
|
+
report as markdown only" or "Concise prose only, no preamble."
|
|
31
|
+
- **Self-contained prompts.** A TeamMate sees only its own prompt, not the
|
|
32
|
+
script, not the other TeamMates' output. Embed everything it needs —
|
|
33
|
+
the item under work, rubrics, false-positive lists, prior known results —
|
|
34
|
+
directly in the prompt string. Never write "as discussed above".
|
|
35
|
+
- **Failure plumbing.** An ordinary failed turn settles `agent()` to `null`.
|
|
36
|
+
`parallel` and `pipeline` keep results index-aligned with their inputs, so
|
|
37
|
+
account required coverage positionally first — which input failed, whether
|
|
38
|
+
a load-bearing step (a discovery, a verifier set) is missing — and report
|
|
39
|
+
the gap; only then drop nulls with `.filter(Boolean)` where item identity
|
|
40
|
+
no longer matters. Blind filtering turns missing coverage into a
|
|
41
|
+
false-clean result. A schema call REJECTS (rather than returning `null`)
|
|
42
|
+
when the runtime cannot provide structured output or reports success with
|
|
43
|
+
an empty or invalid JSON result. Inside `parallel` and `pipeline` a
|
|
44
|
+
rejection is contained as that item's `null`; a directly awaited rejection
|
|
45
|
+
fails the whole run — correct for load-bearing calls (a gate, a scope
|
|
46
|
+
decision), wrap in try/catch only where partial failure is genuinely
|
|
47
|
+
acceptable.
|
|
48
|
+
- **Persona through options.** Use `intent` for the durable task description
|
|
49
|
+
and `identity` for a persona; keep the prompt itself about the work.
|
|
50
|
+
|
|
51
|
+
## pipeline versus parallel
|
|
52
|
+
|
|
53
|
+
Default to `pipeline`. Only reach for a `parallel` barrier when a stage
|
|
54
|
+
genuinely needs every prior result at once.
|
|
55
|
+
|
|
56
|
+
A barrier is justified when the next step needs cross-item context:
|
|
57
|
+
|
|
58
|
+
- deduplication or merging across the full result set before expensive
|
|
59
|
+
downstream work;
|
|
60
|
+
- an early exit decision on the total ("zero findings, skip verification");
|
|
61
|
+
- a prompt that compares items against each other.
|
|
62
|
+
|
|
63
|
+
A barrier is NOT justified by "I need to flatten or map first" (do it inside
|
|
64
|
+
a pipeline stage), "the stages are conceptually separate" (that is exactly
|
|
65
|
+
what `pipeline` models), or "it reads cleaner" (barrier latency is real: the
|
|
66
|
+
fast items idle while the slowest one finishes).
|
|
67
|
+
|
|
68
|
+
Smell test — if the script says:
|
|
69
|
+
|
|
70
|
+
```js
|
|
71
|
+
const a = await parallel(thunks);
|
|
72
|
+
const b = transform(a); // flatten, map, filter — no cross-item need
|
|
73
|
+
const c = await parallel(b.map(toThunk));
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
that middle transform does not need the barrier; rewrite as one `pipeline`
|
|
77
|
+
with the transform inside a stage. Every stage receives
|
|
78
|
+
`(previousResult, originalItem, index)`, so later stages label or join work
|
|
79
|
+
without threading context through earlier return values:
|
|
80
|
+
|
|
81
|
+
```js
|
|
82
|
+
const judged = await pipeline(
|
|
83
|
+
items,
|
|
84
|
+
(item) => agent(promptFor(item), { phase: 'inspect', schema: REPORT_SCHEMA }),
|
|
85
|
+
(report, item, index) =>
|
|
86
|
+
report && agent(rankPrompt(item, report), { phase: 'rank', label: `rank-${index}` }),
|
|
87
|
+
);
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
## Quality patterns
|
|
91
|
+
|
|
92
|
+
Compose these freely; pick by task.
|
|
93
|
+
|
|
94
|
+
- **Adversarial verify.** For each candidate finding, start N independent
|
|
95
|
+
skeptic TeamMates, each prompted to REFUTE it ("Default to refuted=true if
|
|
96
|
+
you cannot independently corroborate"). Keep the finding only when a
|
|
97
|
+
majority fails to refute. This kills plausible-but-wrong findings that a
|
|
98
|
+
single confirming pass would wave through.
|
|
99
|
+
|
|
100
|
+
```js
|
|
101
|
+
const votes = (await parallel([0, 1, 2].map((i) => () =>
|
|
102
|
+
agent(
|
|
103
|
+
`You are skeptic #${i + 1}. Try to refute: ${claim}. ` +
|
|
104
|
+
`Default to refuted=true if uncertain.`,
|
|
105
|
+
{ phase: 'verify', schema: VERDICT_SCHEMA },
|
|
106
|
+
),
|
|
107
|
+
))).filter(Boolean);
|
|
108
|
+
const survives = votes.filter((v) => !v.refuted).length >= 2;
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
- **Perspective-diverse verify.** When a finding can fail in more than one
|
|
112
|
+
way, give each verifier a distinct focus question (correctness; is it
|
|
113
|
+
pre-existing; does the cited guidance actually say that) instead of N
|
|
114
|
+
identical skeptics — diversity catches failure modes redundancy cannot.
|
|
115
|
+
- **Judge panel.** Generate N independent attempts from different angles,
|
|
116
|
+
score them with parallel judges against explicit criteria, then synthesize
|
|
117
|
+
from the winner while grafting the best ideas from the runners-up. Beats
|
|
118
|
+
one-attempt-iterated when the solution space is wide.
|
|
119
|
+
- **Loop-until-dry.** For unknown-size discovery (bugs, issues, edge cases),
|
|
120
|
+
keep starting finder rounds until K consecutive rounds return nothing new.
|
|
121
|
+
Fixed counts miss the tail. Deduplicate each round against everything
|
|
122
|
+
already SEEN — not against the accepted list — or judge-rejected findings
|
|
123
|
+
reappear every round and the loop never converges.
|
|
124
|
+
|
|
125
|
+
```js
|
|
126
|
+
const seen = new Set();
|
|
127
|
+
let dry = 0;
|
|
128
|
+
while (dry < 2) {
|
|
129
|
+
const found = (await parallel(finderThunks(seen))).filter(Boolean)
|
|
130
|
+
.flatMap((r) => r.findings)
|
|
131
|
+
.filter((f) => !seen.has(key(f)));
|
|
132
|
+
if (!found.length) { dry += 1; continue; }
|
|
133
|
+
dry = 0;
|
|
134
|
+
found.forEach((f) => seen.add(key(f)));
|
|
135
|
+
// ...verify and accept...
|
|
136
|
+
}
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Budget first: rounds multiply TeamMate count, and a run stops at 1000
|
|
140
|
+
TeamMates total.
|
|
141
|
+
- **Multi-modal sweep.** Run parallel searchers that each look a different
|
|
142
|
+
way (by file structure, by content, by naming convention, by history),
|
|
143
|
+
each blind to the others. One search angle rarely finds everything.
|
|
144
|
+
- **Completeness critic.** After synthesis, one TeamMate asks "what is
|
|
145
|
+
missing — an angle not searched, a claim unverified, a source unread?"
|
|
146
|
+
What it finds becomes the next round of work, or is reported as a known
|
|
147
|
+
gap.
|
|
148
|
+
- **No silent caps.** When the script bounds coverage (top-N, sampling, a
|
|
149
|
+
round limit), `log()` what was dropped. Silent truncation reads as
|
|
150
|
+
"covered everything" when it did not.
|
|
151
|
+
|
|
152
|
+
## Scale to the ask
|
|
153
|
+
|
|
154
|
+
Match fan-out to what the operator asked for. A quick check deserves a few
|
|
155
|
+
finders and single-vote verification; "thoroughly audit this" deserves a
|
|
156
|
+
larger finder pool, 3-vote adversarial verification, and a synthesis stage.
|
|
157
|
+
When unsure, lean toward thoroughness for research, review, and audit
|
|
158
|
+
requests, and toward brevity for quick checks. Whatever the scale, the
|
|
159
|
+
budget arithmetic comes first: count `finders + votes × findings + fixed
|
|
160
|
+
stages` per round against the 1000-TeamMate cap before choosing round
|
|
161
|
+
counts, and remember `parallel`/`pipeline` accept at most 4096 items per
|
|
162
|
+
call.
|