@excitedjs/dreamux 0.22.0-beta.160 → 0.22.0-beta.161
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@excitedjs/dreamux",
|
|
3
|
-
"version": "0.22.0-beta.
|
|
3
|
+
"version": "0.22.0-beta.161",
|
|
4
4
|
"description": "Dreamux core host for multi-dispatcher agent orchestration through provider interfaces.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -26,11 +26,11 @@
|
|
|
26
26
|
"CHANGELOG.json"
|
|
27
27
|
],
|
|
28
28
|
"dependencies": {
|
|
29
|
-
"@excitedjs/agent-runtime-claude-code": "0.5.0-beta.
|
|
30
|
-
"@excitedjs/agent-runtime-codex": "0.3.5-beta.
|
|
31
|
-
"@excitedjs/dreamux-types": "0.8.0-beta.
|
|
32
|
-
"@excitedjs/dreamux-utils": "0.4.0-beta.
|
|
33
|
-
"@excitedjs/feishu-channel": "5.0.0-beta.
|
|
29
|
+
"@excitedjs/agent-runtime-claude-code": "0.5.0-beta.161",
|
|
30
|
+
"@excitedjs/agent-runtime-codex": "0.3.5-beta.161",
|
|
31
|
+
"@excitedjs/dreamux-types": "0.8.0-beta.161",
|
|
32
|
+
"@excitedjs/dreamux-utils": "0.4.0-beta.161",
|
|
33
|
+
"@excitedjs/feishu-channel": "5.0.0-beta.161",
|
|
34
34
|
"acorn": "^8.16.0",
|
|
35
35
|
"croner": "^10.0.1",
|
|
36
36
|
"smol-toml": "^1.6.1",
|
|
@@ -161,3 +161,4 @@ unavailable so orchestration stays deterministic.
|
|
|
161
161
|
| --- | --- | --- |
|
|
162
162
|
| Code review run | Carrying a change review (pull/merge request, git range, or working-tree diff) as one workflow run: eligibility gate, multi-lens finders, confidence-scored verification, one synthesized report. | [Code review](references/code-review.md) |
|
|
163
163
|
| Orchestration and prompt patterns | Writing TeamMate prompts inside scripts, choosing `pipeline` versus `parallel`, and applying quality patterns: adversarial verify, judge panel, loop-until-dry, multi-modal sweep, completeness critic. | [Orchestration patterns](references/orchestration-patterns.md) |
|
|
164
|
+
| Ultracode orchestration techniques | Shaping a whole task as workflows: the five archetypes (understand/design/review/research/migrate), scouting before fan-out, chaining runs with the TeamLeader in the loop, worked compositions, novel harness shapes. | [Ultracode techniques](references/ultracode.md) |
|
|
@@ -1,394 +1,157 @@
|
|
|
1
1
|
# Code Review Workflow
|
|
2
2
|
|
|
3
|
-
`../SKILL.md` owns the run tools and the script API. This reference
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
## Contents
|
|
9
|
-
|
|
10
|
-
- [Stage design](#stage-design) — the five stages and their failure accounting
|
|
11
|
-
- [Prompt discipline](#prompt-discipline) — output contract, rejection
|
|
12
|
-
semantics, self-contained prompts
|
|
13
|
-
- [TeamMate budget](#teammate-budget) — cost arithmetic against the run caps
|
|
14
|
-
- [Script](#script) — the complete runnable module
|
|
15
|
-
- [Invocation](#invocation) — call shape and result fields
|
|
3
|
+
`../SKILL.md` owns the run tools and the script API. This reference teaches
|
|
4
|
+
the method for carrying a code review as one `workflow_run`: the stages, the
|
|
5
|
+
scoring rubric, and the false-positive discipline. Write the script for the
|
|
6
|
+
task at hand — the sketch at the end shows the shape, it is not a template to
|
|
7
|
+
paste.
|
|
16
8
|
|
|
17
9
|
The review target is any change the Team workspace can read: a GitHub pull
|
|
18
10
|
request, a GitLab merge request, a local branch or commit range, or an
|
|
19
|
-
uncommitted working-tree diff.
|
|
20
|
-
terminal result for such a change. For interactive review — where the next
|
|
11
|
+
uncommitted working-tree diff. For interactive review — where the next
|
|
21
12
|
instruction depends on reading the previous finding — use `spawn` and `send`
|
|
22
13
|
instead.
|
|
23
14
|
|
|
24
|
-
##
|
|
25
|
-
|
|
26
|
-
1. **Gate
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
2. **Context
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
output, and everything a verifier needs (the finding, the rubric, the
|
|
86
|
-
false-positive list) is embedded in its prompt.
|
|
15
|
+
## The flow
|
|
16
|
+
|
|
17
|
+
1. **Gate.** One TeamMate checks eligibility; exit early when review is not
|
|
18
|
+
needed: the change is closed or a draft, is automated or trivially safe,
|
|
19
|
+
or already carries a review from this Team.
|
|
20
|
+
2. **Context.** In parallel: list the file paths (not contents) of the
|
|
21
|
+
repository's agent guidance files (`CLAUDE.md`, `AGENTS.md`) at the root
|
|
22
|
+
and in directories the change touches; summarize the change (intent,
|
|
23
|
+
scope, risk areas). Feed both into every later prompt.
|
|
24
|
+
3. **Find.** Independent finder lenses in parallel, each blind to the others:
|
|
25
|
+
- guidance-file compliance (read the discovered files; not every writing
|
|
26
|
+
instruction applies at review time);
|
|
27
|
+
- a shallow bug scan limited to the changed hunks;
|
|
28
|
+
- git blame and history of the modified code;
|
|
29
|
+
- review comments on prior change requests touching the same files;
|
|
30
|
+
- guidance stated in code comments of the modified files;
|
|
31
|
+
- concrete security issues introduced by the change.
|
|
32
|
+
Every finding must state a concrete failure scenario — the specific
|
|
33
|
+
inputs or state that produce the wrong outcome; a defect that cannot be
|
|
34
|
+
stated as a scenario is not reportable. Deduplicate across lenses by
|
|
35
|
+
file, line, and title. For an exhaustive audit, repeat find/verify rounds
|
|
36
|
+
until two consecutive rounds surface nothing new, carrying already-seen
|
|
37
|
+
findings in each round's prompts.
|
|
38
|
+
4. **Verify.** Score each finding with parallel verifier TeamMates — three
|
|
39
|
+
normally, two for a quick pass — but give each verifier a *distinct focus
|
|
40
|
+
question* rather than re-asking the same one: does the defect reproduce on
|
|
41
|
+
the changed lines; is it pre-existing rather than introduced by this change;
|
|
42
|
+
does the cited guidance file or code comment actually say that. Diverse
|
|
43
|
+
focus catches failure modes that identical skeptics miss. Each verifier
|
|
44
|
+
receives the finding, the guidance paths, the false-positive list, its
|
|
45
|
+
focus question, and this rubric verbatim:
|
|
46
|
+
- 0: Not confident at all. False positive that does not stand up to light
|
|
47
|
+
scrutiny, or a pre-existing issue.
|
|
48
|
+
- 25: Somewhat confident. Might be real, might be a false positive; could
|
|
49
|
+
not verify. If stylistic, it is not explicitly called out in a relevant
|
|
50
|
+
guidance file.
|
|
51
|
+
- 50: Moderately confident. Verified real, but may be a nitpick or rare in
|
|
52
|
+
practice; not very important relative to the rest of the change.
|
|
53
|
+
- 75: Highly confident. Double checked; very likely real and will be hit
|
|
54
|
+
in practice; directly impacts functionality or is directly mentioned in
|
|
55
|
+
a guidance file.
|
|
56
|
+
- 100: Absolutely certain. Double checked and confirmed; will happen
|
|
57
|
+
frequently; evidence directly confirms it.
|
|
58
|
+
Verifiers also check that the stated failure scenario is concrete and
|
|
59
|
+
actually leads to the claimed outcome. Score toward the low end when you
|
|
60
|
+
cannot independently corroborate the finding — an uncorroborated finding is
|
|
61
|
+
a 0 or 25, not a hopeful 50. Keep a finding only when at least two verifiers
|
|
62
|
+
settled a vote and those votes average 80 or higher; fewer than two settled
|
|
63
|
+
votes is unverified coverage, not a pass. At
|
|
64
|
+
max effort, findings in the 50-79 band may be reported too — in their own
|
|
65
|
+
clearly labeled section, never as confirmed issues.
|
|
66
|
+
5. **Report.** One TeamMate writes the review comment: a short verdict, then
|
|
67
|
+
numbered issues (grouped by severity when findings carry one), each with
|
|
68
|
+
the flag reason in parentheses and a file#line citation — file path alone
|
|
69
|
+
when no line is known; never invent a line number. Brief, no emojis. When a
|
|
70
|
+
channel reply tool is available, post the returned report through it;
|
|
71
|
+
otherwise return the report for the caller, or have a follow-up `send` to
|
|
72
|
+
the reporter publish it with the forge's tooling (`gh`, `glab`) once the
|
|
73
|
+
operator confirms.
|
|
74
|
+
|
|
75
|
+
## False positives
|
|
87
76
|
|
|
88
77
|
Instruct finders and verifiers to treat these as false positives: pre-existing
|
|
89
78
|
issues; code that looks buggy but is not; pedantic nitpicks; anything a
|
|
90
79
|
linter, typechecker, or CI would catch; general quality concerns not required
|
|
91
80
|
by a guidance file; issues explicitly silenced in code; intentional behavior
|
|
92
|
-
changes related to the broader change; and real issues on lines the change
|
|
93
|
-
not modify.
|
|
81
|
+
changes related to the broader change; and real issues on lines the change
|
|
82
|
+
did not modify. When reviewing a cumulative range, add: behavior an earlier
|
|
83
|
+
change introduced and a later change in the same range deliberately replaced.
|
|
94
84
|
|
|
95
|
-
##
|
|
85
|
+
## Scaling and range reviews
|
|
96
86
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
`6 finders + 3 × fresh findings` per extra round before choosing round
|
|
102
|
-
counts, and match fan-out to what the operator asked for.
|
|
87
|
+
Match fan-out to the ask: a quick pass is one round with two votes; a normal
|
|
88
|
+
review is one round with three; "thoroughly audit this" repeats rounds until
|
|
89
|
+
dry. The confirmation bar stays the same throughout — effort widens coverage,
|
|
90
|
+
never the evidence standard.
|
|
103
91
|
|
|
104
|
-
|
|
92
|
+
For a cumulative range ("everything since X"), resolve the fuzzy description
|
|
93
|
+
into one exact `base..head` first — as the opening stage when it takes
|
|
94
|
+
repository work — and add a cross-change lens: a later change silently
|
|
95
|
+
invalidating an earlier one's assumption, dead code or docs left behind by a
|
|
96
|
+
superseding change, contradictory contracts between commits.
|
|
105
97
|
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
98
|
+
## Failure semantics worth remembering
|
|
99
|
+
|
|
100
|
+
`agent()` settles to `null` on a failed turn, and helper results stay
|
|
101
|
+
index-aligned with their inputs — note which lens or verifier failed before
|
|
102
|
+
filtering nulls, and say so in the report instead of presenting partial
|
|
103
|
+
coverage as clean. A directly awaited schema call rejects on a
|
|
104
|
+
structured-output contract breach, which is the right outcome for
|
|
105
|
+
load-bearing calls like the gate. The pattern language lives in
|
|
106
|
+
[orchestration-patterns.md](orchestration-patterns.md).
|
|
107
|
+
|
|
108
|
+
## Sketch
|
|
109
|
+
|
|
110
|
+
The shape, compressed — adapt lenses, schemas, rounds, and report format to
|
|
111
|
+
the actual task:
|
|
110
112
|
|
|
111
113
|
```js
|
|
112
114
|
export const meta = {
|
|
113
115
|
name: 'code-review',
|
|
114
|
-
description: 'Multi-lens code review with confidence-scored findings
|
|
115
|
-
whenToUse: 'One-shot review of a change request, git range, or working-tree diff.',
|
|
116
|
+
description: 'Multi-lens code review with confidence-scored findings',
|
|
116
117
|
phases: [
|
|
117
|
-
{ title: 'gate',
|
|
118
|
-
{ title: '
|
|
119
|
-
{ title: 'find', detail: 'six independent finder lenses' },
|
|
120
|
-
{ title: 'verify', detail: 'three-vote confidence scoring per finding' },
|
|
121
|
-
{ title: 'report', detail: 'synthesized review comment' },
|
|
118
|
+
{ title: 'gate' }, { title: 'context' }, { title: 'find' },
|
|
119
|
+
{ title: 'verify' }, { title: 'report' },
|
|
122
120
|
],
|
|
123
121
|
};
|
|
124
122
|
|
|
125
|
-
const FALSE_POSITIVES =
|
|
126
|
-
'Treat as false positives: pre-existing issues; code that looks buggy but is not; ' +
|
|
127
|
-
'pedantic nitpicks; anything a linter, typechecker, or CI would catch; general ' +
|
|
128
|
-
'quality concerns (coverage, docs) unless a guidance file (CLAUDE.md, AGENTS.md) ' +
|
|
129
|
-
'requires them; issues explicitly silenced in code; intentional behavior changes ' +
|
|
130
|
-
'related to the broader change; real issues on unmodified lines.';
|
|
131
|
-
|
|
132
|
-
const RUBRIC =
|
|
133
|
-
'0: Not confident at all. False positive that does not stand up to light scrutiny, or a pre-existing issue.\n' +
|
|
134
|
-
'25: Somewhat confident. Might be real, might be a false positive; could not verify. If stylistic, it is not explicitly called out in a relevant guidance file (CLAUDE.md, AGENTS.md).\n' +
|
|
135
|
-
'50: Moderately confident. Verified real, but may be a nitpick or rare in practice; not very important relative to the rest of the change.\n' +
|
|
136
|
-
'75: Highly confident. Double checked; very likely real and will be hit in practice; directly impacts functionality or is directly mentioned in a guidance file.\n' +
|
|
137
|
-
'100: Absolutely certain. Double checked and confirmed; will happen frequently; evidence directly confirms it.';
|
|
138
|
-
|
|
139
|
-
const FINDINGS_SCHEMA = {
|
|
140
|
-
type: 'object',
|
|
141
|
-
properties: {
|
|
142
|
-
findings: {
|
|
143
|
-
type: 'array',
|
|
144
|
-
items: {
|
|
145
|
-
type: 'object',
|
|
146
|
-
properties: {
|
|
147
|
-
file: { type: 'string' },
|
|
148
|
-
line: { type: 'integer' },
|
|
149
|
-
title: { type: 'string' },
|
|
150
|
-
detail: { type: 'string' },
|
|
151
|
-
reason: {
|
|
152
|
-
type: 'string',
|
|
153
|
-
description: 'guidance | bug | history | prior-review | comment | security',
|
|
154
|
-
},
|
|
155
|
-
},
|
|
156
|
-
required: ['file', 'title', 'detail', 'reason'],
|
|
157
|
-
additionalProperties: false,
|
|
158
|
-
},
|
|
159
|
-
},
|
|
160
|
-
},
|
|
161
|
-
required: ['findings'],
|
|
162
|
-
additionalProperties: false,
|
|
163
|
-
};
|
|
164
|
-
|
|
165
|
-
const SCORE_SCHEMA = {
|
|
166
|
-
type: 'object',
|
|
167
|
-
properties: {
|
|
168
|
-
score: { type: 'integer', minimum: 0, maximum: 100 },
|
|
169
|
-
reasoning: { type: 'string' },
|
|
170
|
-
},
|
|
171
|
-
required: ['score', 'reasoning'],
|
|
172
|
-
additionalProperties: false,
|
|
173
|
-
};
|
|
174
|
-
|
|
175
|
-
function findingKey(f) {
|
|
176
|
-
return `${f.file}:${f.line || 0}:${f.title.toLowerCase().replace(/\s+/g, ' ').trim()}`;
|
|
177
|
-
}
|
|
178
|
-
|
|
179
|
-
const target = args && args.target;
|
|
180
|
-
if (!target) {
|
|
181
|
-
return { error: 'args.target is required: a PR/MR URL or number, a git range, or a working-tree description' };
|
|
182
|
-
}
|
|
183
|
-
|
|
184
123
|
phase('gate');
|
|
185
|
-
const gate = await agent(
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
{
|
|
190
|
-
label: 'gate',
|
|
191
|
-
phase: 'gate',
|
|
192
|
-
intent: 'Review eligibility gate',
|
|
193
|
-
schema: {
|
|
194
|
-
type: 'object',
|
|
195
|
-
properties: { eligible: { type: 'boolean' }, reason: { type: 'string' } },
|
|
196
|
-
required: ['eligible', 'reason'],
|
|
197
|
-
additionalProperties: false,
|
|
198
|
-
},
|
|
199
|
-
},
|
|
200
|
-
);
|
|
201
|
-
if (!gate || !gate.eligible) {
|
|
202
|
-
return { skipped: true, reason: gate ? gate.reason : 'gate TeamMate failed' };
|
|
203
|
-
}
|
|
124
|
+
const gate = await agent(`Should ${args.target} be reviewed? ...`, {
|
|
125
|
+
phase: 'gate', schema: GATE_SCHEMA,
|
|
126
|
+
});
|
|
127
|
+
if (!gate?.eligible) return { skipped: true, reason: gate?.reason };
|
|
204
128
|
|
|
205
129
|
phase('context');
|
|
206
|
-
const
|
|
207
|
-
() => agent(
|
|
208
|
-
|
|
209
|
-
`repository's agent guidance files — CLAUDE.md and AGENTS.md — at the ` +
|
|
210
|
-
`repository root and in directories whose files the change modifies.`,
|
|
211
|
-
{
|
|
212
|
-
label: 'guidance',
|
|
213
|
-
phase: 'context',
|
|
214
|
-
intent: 'Guidance file discovery',
|
|
215
|
-
schema: {
|
|
216
|
-
type: 'object',
|
|
217
|
-
properties: { paths: { type: 'array', items: { type: 'string' } } },
|
|
218
|
-
required: ['paths'],
|
|
219
|
-
additionalProperties: false,
|
|
220
|
-
},
|
|
221
|
-
},
|
|
222
|
-
),
|
|
223
|
-
() => agent(
|
|
224
|
-
`View this change: ${target}. Summarize it: intent, scope, key files, risk ` +
|
|
225
|
-
`areas. Concise prose only.`,
|
|
226
|
-
{ label: 'summary', phase: 'context', intent: 'Change summarizer' },
|
|
227
|
-
),
|
|
130
|
+
const [guidance, summary] = await parallel([
|
|
131
|
+
() => agent(`List guidance files relevant to ${args.target}.`, { phase: 'context' }),
|
|
132
|
+
() => agent(`Summarize ${args.target}.`, { phase: 'context' }),
|
|
228
133
|
]);
|
|
229
|
-
// a null discovery result is a coverage gap, distinct from a successful
|
|
230
|
-
// empty list (a repo that simply has no guidance files)
|
|
231
|
-
const guidanceFailed = !ctx[0];
|
|
232
|
-
const guidancePaths = guidanceFailed ? [] : ctx[0].paths;
|
|
233
|
-
const summary = ctx[1] || '(summary unavailable)';
|
|
234
|
-
|
|
235
|
-
const lenses = [
|
|
236
|
-
{ key: 'guidance', prompt: `Audit the change for compliance with these guidance files (read them): ${JSON.stringify(guidancePaths)}. Guidance files direct code writing, so not every instruction applies at review time.` },
|
|
237
|
-
{ key: 'shallow-bugs', prompt: 'Read only the changed hunks and scan for obvious large bugs. Do not read extra context. Skip small issues and nitpicks.' },
|
|
238
|
-
{ key: 'git-history', prompt: 'Read git blame and history of the modified code; flag bugs visible only in light of that historical context, such as reintroduced bugs or violated invariants.' },
|
|
239
|
-
{ key: 'prior-reviews', prompt: 'Find previous change requests (pull/merge requests) touching these files; check review comments on them that also apply to this change.' },
|
|
240
|
-
{ key: 'code-comments', prompt: 'Read code comments in the modified files; flag violations of guidance stated in those comments.' },
|
|
241
|
-
{ key: 'security', prompt: 'Review the changed code for concrete, exploitable security issues introduced by this change (injection, authorization gaps, secret handling). No generic hardening advice.' },
|
|
242
|
-
];
|
|
243
134
|
|
|
244
135
|
phase('find');
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
// parallel() preserves order, so a null entry identifies the failed lens
|
|
251
|
-
const lensResults = await parallel(activeLenses.map((lens) => () =>
|
|
252
|
-
agent(
|
|
253
|
-
`Code review this change through one lens: ${target}\n${lens.prompt}\n` +
|
|
254
|
-
`${FALSE_POSITIVES}\nChange summary:\n${summary}`,
|
|
255
|
-
{
|
|
256
|
-
label: `find-${lens.key}`,
|
|
257
|
-
phase: 'find',
|
|
258
|
-
intent: `Finder lens: ${lens.key}`,
|
|
259
|
-
schema: FINDINGS_SCHEMA,
|
|
260
|
-
},
|
|
261
|
-
),
|
|
262
|
-
));
|
|
263
|
-
const failedLenses = [
|
|
264
|
-
...(guidanceFailed ? ['guidance'] : []),
|
|
265
|
-
...activeLenses.filter((lens, i) => !lensResults[i]).map((lens) => lens.key),
|
|
266
|
-
];
|
|
267
|
-
if (failedLenses.length === lenses.length) {
|
|
268
|
-
return { error: 'every finder lens failed; the change was not reviewed', failedLenses };
|
|
269
|
-
}
|
|
270
|
-
if (failedLenses.length) {
|
|
271
|
-
log(`coverage incomplete: failed lenses ${failedLenses.join(', ')}`);
|
|
272
|
-
}
|
|
273
|
-
const found = lensResults.filter(Boolean).flatMap((r) => r.findings);
|
|
274
|
-
|
|
275
|
-
const seen = new Set();
|
|
276
|
-
const unique = [];
|
|
277
|
-
for (const f of found) {
|
|
278
|
-
const key = findingKey(f);
|
|
279
|
-
if (!seen.has(key)) { seen.add(key); unique.push(f); }
|
|
280
|
-
}
|
|
281
|
-
log(`${found.length} raw findings, ${unique.length} unique`);
|
|
282
|
-
|
|
283
|
-
const checkedLenses = lenses
|
|
284
|
-
.filter((lens) => !failedLenses.includes(lens.key))
|
|
285
|
-
.map((lens) => lens.key);
|
|
286
|
-
function coverageRecord(unverifiedCount) {
|
|
287
|
-
return {
|
|
288
|
-
complete: failedLenses.length === 0 && unverifiedCount === 0,
|
|
289
|
-
failedLenses,
|
|
290
|
-
unverifiedFindings: unverifiedCount,
|
|
291
|
-
};
|
|
292
|
-
}
|
|
293
|
-
const noIssuesReport = failedLenses.length === 0
|
|
294
|
-
? 'No issues found. Checked for bugs, security, history, and guidance-file compliance.'
|
|
295
|
-
: `No issues found by the lenses that completed (${checkedLenses.join(', ')}). ` +
|
|
296
|
-
`Coverage is incomplete: ${failedLenses.join(', ')} failed.`;
|
|
297
|
-
if (!unique.length) {
|
|
298
|
-
return { issues: [], unverified: [], coverage: coverageRecord(0), report: noIssuesReport };
|
|
299
|
-
}
|
|
136
|
+
const findings = dedupe((await parallel(LENSES.map((lens) => () =>
|
|
137
|
+
agent(finderPrompt(lens, args.target, guidance, summary), {
|
|
138
|
+
phase: 'find', schema: FINDINGS_SCHEMA,
|
|
139
|
+
}),
|
|
140
|
+
))).filter(Boolean).flatMap((r) => r.findings));
|
|
300
141
|
|
|
301
142
|
phase('verify');
|
|
302
|
-
const
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
]
|
|
307
|
-
const judged = await pipeline(
|
|
308
|
-
unique,
|
|
309
|
-
(finding) => parallel(focusQuestions.map((focus) => () =>
|
|
310
|
-
agent(
|
|
311
|
-
`Confidence-score this code review finding for the change: ${target}\n` +
|
|
312
|
-
`Finding: ${JSON.stringify(finding)}\n` +
|
|
313
|
-
`Guidance files: ${JSON.stringify(guidancePaths)}\n` +
|
|
314
|
-
`Focus question: ${focus}.\n${FALSE_POSITIVES}\n` +
|
|
315
|
-
`Score with this rubric verbatim:\n${RUBRIC}`,
|
|
316
|
-
{ phase: 'verify', intent: 'Finding verifier', schema: SCORE_SCHEMA },
|
|
317
|
-
),
|
|
318
|
-
)),
|
|
319
|
-
(votes, finding, index) => {
|
|
320
|
-
const settled = (votes || []).filter(Boolean);
|
|
321
|
-
const average = settled.length
|
|
322
|
-
? settled.reduce((sum, vote) => sum + vote.score, 0) / settled.length
|
|
323
|
-
: 0;
|
|
324
|
-
return { ...finding, confidence: Math.round(average), votes: settled.length, index };
|
|
325
|
-
},
|
|
143
|
+
const scored = await pipeline(
|
|
144
|
+
findings,
|
|
145
|
+
(f) => parallel(FOCUS_QUESTIONS.map((focus) => () =>
|
|
146
|
+
agent(rubricPrompt(f, focus), { phase: 'verify', schema: SCORE_SCHEMA }))),
|
|
147
|
+
(votes, f) => ({ ...f, votes: (votes || []).filter(Boolean) }),
|
|
326
148
|
);
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
const unverified = unique.filter((f, i) => {
|
|
330
|
-
const j = judged[i];
|
|
331
|
-
return !j || j.votes < 2;
|
|
332
|
-
});
|
|
333
|
-
const confirmed = judged
|
|
334
|
-
.filter(Boolean)
|
|
335
|
-
.filter((f) => f.votes >= 2 && f.confidence >= 80);
|
|
336
|
-
const coverage = coverageRecord(unverified.length);
|
|
337
|
-
log(`${confirmed.length} findings at >=80 confidence, ${unverified.length} unverified`);
|
|
149
|
+
const confirmed = scored.filter(Boolean)
|
|
150
|
+
.filter((f) => f.votes.length >= 2 && average(f.votes) >= 80);
|
|
338
151
|
|
|
339
152
|
phase('report');
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
if (!confirmed.length) {
|
|
345
|
-
const reportText = unverified.length
|
|
346
|
-
? `No issues confirmed.${unverifiedNote}`
|
|
347
|
-
: noIssuesReport;
|
|
348
|
-
return { issues: [], unverified, coverage, report: reportText };
|
|
349
|
-
}
|
|
350
|
-
const report = await agent(
|
|
351
|
-
`Write the final code review comment for the change: ${target}\n` +
|
|
352
|
-
`Confirmed issues (JSON): ${JSON.stringify(confirmed)}\n` +
|
|
353
|
-
`Unverified findings — verifiers failed, no confidence claim (JSON): ${JSON.stringify(unverified)}\n` +
|
|
354
|
-
`Coverage (JSON): ${JSON.stringify(coverage)}\n` +
|
|
355
|
-
`Format: a "### Code review" heading, "Found N issues:", then a numbered list; ` +
|
|
356
|
-
`each item is a brief description with the flag reason in parentheses and a ` +
|
|
357
|
-
`citation on the next line — file#line when the finding carries a line number, ` +
|
|
358
|
-
`the file path alone otherwise; never invent a line number. If there are ` +
|
|
359
|
-
`unverified findings, add a short "Unverified" section listing them without ` +
|
|
360
|
-
`confidence claims. If coverage is incomplete, end with one line naming the ` +
|
|
361
|
-
`failed lenses. Brief, no emojis. Return markdown only.`,
|
|
362
|
-
{ label: 'report', phase: 'report', intent: 'Review report writer' },
|
|
363
|
-
);
|
|
364
|
-
return { issues: confirmed, unverified, coverage, report };
|
|
365
|
-
```
|
|
366
|
-
|
|
367
|
-
The verify stage shows the three-argument stage signature: stage one returns
|
|
368
|
-
only the vote array, and stage two recovers the finding and its position from
|
|
369
|
-
`(votes, finding, index)` instead of threading them through the return value.
|
|
370
|
-
|
|
371
|
-
## Invocation
|
|
372
|
-
|
|
373
|
-
```
|
|
374
|
-
workflow_run({
|
|
375
|
-
script: <the script above>,
|
|
376
|
-
args: { target: 'https://github.com/owner/repo/pull/123' },
|
|
377
|
-
})
|
|
153
|
+
return {
|
|
154
|
+
issues: confirmed,
|
|
155
|
+
report: await agent(reportPrompt(args.target, confirmed), { phase: 'report' }),
|
|
156
|
+
};
|
|
378
157
|
```
|
|
379
|
-
|
|
380
|
-
`args.target` is forge-neutral — pass whatever identifies the change in the
|
|
381
|
-
Team workspace:
|
|
382
|
-
|
|
383
|
-
- a pull-request or merge-request URL or number
|
|
384
|
-
(`'https://gitlab.example.com/group/repo/-/merge_requests/45'`, `'#123'`);
|
|
385
|
-
- a local git range (`'main...feature'`);
|
|
386
|
-
- a working-tree description (`'the uncommitted diff in the workspace'`).
|
|
387
|
-
|
|
388
|
-
`max_concurrency` defaults to 16; lower it only when the workspace or runtime
|
|
389
|
-
provider needs gentler fan-out. After the terminal completion, read `issues`
|
|
390
|
-
for the confirmed findings, `unverified` for findings whose verifiers failed,
|
|
391
|
-
`coverage` for whether every finder lens ran and every finding was verified,
|
|
392
|
-
and `report` for the formatted comment; use the reporter TeamMate's concrete
|
|
393
|
-
name with `send` for follow-ups such as posting the comment through the
|
|
394
|
-
forge's tooling or re-checking one finding interactively.
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
# Ultracode Orchestration Techniques
|
|
2
|
+
|
|
3
|
+
`../SKILL.md` owns the run tools and the script API;
|
|
4
|
+
[orchestration-patterns.md](orchestration-patterns.md) owns prompt discipline,
|
|
5
|
+
the `pipeline`-versus-`parallel` rules, and the individual quality patterns.
|
|
6
|
+
This reference owns the level above: how to shape a whole task as ultracode
|
|
7
|
+
workflows — choosing an archetype, scouting before orchestrating, chaining
|
|
8
|
+
runs, and composing the quality patterns into complete harnesses.
|
|
9
|
+
|
|
10
|
+
## Contents
|
|
11
|
+
|
|
12
|
+
- [Workflow archetypes](#workflow-archetypes) — five reusable single-run
|
|
13
|
+
shapes
|
|
14
|
+
- [Scout before you orchestrate](#scout-before-you-orchestrate) — discover
|
|
15
|
+
the work-list first
|
|
16
|
+
- [Chain runs, stay in the loop](#chain-runs-stay-in-the-loop) — multi-phase
|
|
17
|
+
work as a sequence of runs
|
|
18
|
+
- [Worked compositions](#worked-compositions) — barrier-when-justified and
|
|
19
|
+
the exhaustive-discovery loop
|
|
20
|
+
- [Loop-until-count](#loop-until-count) — accumulate to a target
|
|
21
|
+
- [Novel harnesses](#novel-harnesses) — beyond the named patterns
|
|
22
|
+
|
|
23
|
+
## Workflow archetypes
|
|
24
|
+
|
|
25
|
+
Most tasks that deserve a workflow fit one of five single-run shapes. Name the
|
|
26
|
+
shape first; the script structure follows from it.
|
|
27
|
+
|
|
28
|
+
- **Understand** — parallel readers over relevant subsystems, one structured
|
|
29
|
+
map out. Fan out one `agent()` per subsystem with a shared schema, then one
|
|
30
|
+
synthesis agent merges the maps. Use when the question is "how does this
|
|
31
|
+
system work" and no single context can hold it all.
|
|
32
|
+
- **Design** — a judge panel over N independent approaches. Generate attempts
|
|
33
|
+
from deliberately different angles (smallest-change-first, risk-first,
|
|
34
|
+
user-first), score them with parallel judges against explicit criteria,
|
|
35
|
+
synthesize from the winner while grafting the runners-up's best ideas. Use
|
|
36
|
+
when the solution space is wide and one-attempt-iterated would anchor early.
|
|
37
|
+
- **Review** — dimensions → find → adversarially verify. The complete recipe
|
|
38
|
+
lives in [code-review.md](code-review.md); the shape generalizes to any
|
|
39
|
+
audit: independent finder lenses, confidence-gated verification, one
|
|
40
|
+
severity-ordered report.
|
|
41
|
+
- **Research** — multi-modal sweep → deep-read → verify → synthesize: search
|
|
42
|
+
agents per angle, URL dedup at a justified barrier, per-source claim
|
|
43
|
+
extraction, adversarial claim verification, one cited report with honest
|
|
44
|
+
coverage caveats.
|
|
45
|
+
- **Migrate** — discover sites → transform each → verify each. Scout the
|
|
46
|
+
work-list first (see below), then `pipeline` items through
|
|
47
|
+
transform-then-verify stages. Keep concurrent edits on clearly independent
|
|
48
|
+
paths, or serialize the write stage — workflow TeamMates share the Team
|
|
49
|
+
workspace.
|
|
50
|
+
|
|
51
|
+
## Scout before you orchestrate
|
|
52
|
+
|
|
53
|
+
The coordination graph must be known before the orchestration step — not
|
|
54
|
+
before the task. When the work-list is unknown ("migrate every call site",
|
|
55
|
+
"review whatever changed"), scout first and fan out second:
|
|
56
|
+
|
|
57
|
+
- cheap scouting (one `git diff --stat`, one directory listing) belongs in
|
|
58
|
+
the TeamLeader's own turn before `workflow_run`, with the result passed in
|
|
59
|
+
as `args`;
|
|
60
|
+
- scouting that itself needs an agent (resolving a fuzzy range, inventorying
|
|
61
|
+
a subsystem) belongs as the script's first stage, awaited bare so an
|
|
62
|
+
unresolvable work-list fails the run before any fan-out — the resolve stage
|
|
63
|
+
in [code-review.md](code-review.md)'s range mode is this technique.
|
|
64
|
+
|
|
65
|
+
Never fan out against a guess: every downstream prompt should receive the
|
|
66
|
+
resolved work-list, not the fuzzy description.
|
|
67
|
+
|
|
68
|
+
## Chain runs, stay in the loop
|
|
69
|
+
|
|
70
|
+
A workflow is one well-scoped fan-out with a deterministic graph. Larger work
|
|
71
|
+
— understand, then design, then implement, then review — is a SEQUENCE of
|
|
72
|
+
workflows with the TeamLeader reading each terminal result before shaping the
|
|
73
|
+
next run. Resist packing multi-phase judgment into one mega-script: the
|
|
74
|
+
decisions between phases (is the map good enough? which design won? is the
|
|
75
|
+
implementation ready for review?) are exactly the parts that should stay with
|
|
76
|
+
the TeamLeader, interactive and revisable.
|
|
77
|
+
|
|
78
|
+
Three practical consequences:
|
|
79
|
+
|
|
80
|
+
- `workflow_run` returns only `{ run_id }` immediately — a durable acceptance
|
|
81
|
+
receipt, not the result. Save the id, wait for the terminal completion that
|
|
82
|
+
Dreamux pushes, and build the next run's `args` from that completion's
|
|
83
|
+
`result`; feeding the receipt into the next run is the classic mistake.
|
|
84
|
+
`workflow_status({ run_id })` is for explicit recovery, not a polling loop.
|
|
85
|
+
- return STRUCTURED results (not just prose) from each run so the next run's
|
|
86
|
+
`args` can be built from them mechanically;
|
|
87
|
+
- after a run, follow up interactively with a recorded concrete TeamMate name
|
|
88
|
+
via `send` when one result needs a clarification — reshaping and re-running
|
|
89
|
+
the whole workflow is for when the graph itself was wrong.
|
|
90
|
+
|
|
91
|
+
## Worked compositions
|
|
92
|
+
|
|
93
|
+
**A barrier when it is genuinely justified** — dedup across the full result
|
|
94
|
+
set before expensive downstream work:
|
|
95
|
+
|
|
96
|
+
```js
|
|
97
|
+
const all = await parallel(angles.map((a) => () =>
|
|
98
|
+
agent(searchPrompt(a), { phase: 'search', schema: SOURCES_SCHEMA })));
|
|
99
|
+
const sources = dedupeByUrl(all.filter(Boolean).flatMap((r) => r.sources));
|
|
100
|
+
// only now fan out the expensive per-source work
|
|
101
|
+
const reports = await pipeline(sources, fetchStage, verifyStage);
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
The barrier earns its latency because the dedup needs every angle's output at
|
|
105
|
+
once; the per-source work that follows goes back to `pipeline`.
|
|
106
|
+
|
|
107
|
+
**The exhaustive-discovery loop** — finders → dedup vs SEEN → verification →
|
|
108
|
+
repeat until dry, the composition behind every "audit everything" request:
|
|
109
|
+
|
|
110
|
+
```js
|
|
111
|
+
const seen = new Set();
|
|
112
|
+
const accepted = [];
|
|
113
|
+
let dry = 0;
|
|
114
|
+
while (dry < 2) {
|
|
115
|
+
const found = (await parallel(finderThunks(seen))).filter(Boolean)
|
|
116
|
+
.flatMap((r) => r.findings)
|
|
117
|
+
.filter((f) => !seen.has(key(f)));
|
|
118
|
+
if (!found.length) { dry += 1; continue; }
|
|
119
|
+
dry = 0;
|
|
120
|
+
found.forEach((f) => seen.add(key(f)));
|
|
121
|
+
const judged = await pipeline(found, verifyStage, aggregateStage);
|
|
122
|
+
accepted.push(...judged.filter(Boolean).filter((f) => f.accepted));
|
|
123
|
+
}
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Dedupe against SEEN, not against `accepted` — otherwise judge-rejected
|
|
127
|
+
findings reappear every round and the loop never converges. When adapting the
|
|
128
|
+
sketch, remember the standing failure-plumbing rules from
|
|
129
|
+
[orchestration-patterns.md](orchestration-patterns.md): note failed positions
|
|
130
|
+
before filtering nulls, and bound loops when the finding population may not
|
|
131
|
+
converge.
|
|
132
|
+
|
|
133
|
+
## Loop-until-count
|
|
134
|
+
|
|
135
|
+
For "give me N of X" requests, accumulate to the target instead of guessing
|
|
136
|
+
the fan-out size:
|
|
137
|
+
|
|
138
|
+
```js
|
|
139
|
+
const bugs = [];
|
|
140
|
+
while (bugs.length < 10) {
|
|
141
|
+
const result = await agent('Find bugs not in this list: ' +
|
|
142
|
+
JSON.stringify(bugs.map((b) => b.title)), { schema: BUGS_SCHEMA });
|
|
143
|
+
if (!result || !result.bugs.length) break; // a dry pass ends the hunt honestly
|
|
144
|
+
bugs.push(...result.bugs);
|
|
145
|
+
log(`${bugs.length}/10 found`);
|
|
146
|
+
}
|
|
147
|
+
if (bugs.length < 10) log(`stopped short: ${bugs.length}/10 — no more found`);
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Always pair the target with a dry-pass exit and `log()` the shortfall — a
|
|
151
|
+
loop that cannot reach N must say so, not spin against the agent cap.
|
|
152
|
+
|
|
153
|
+
## Novel harnesses
|
|
154
|
+
|
|
155
|
+
The named patterns are a vocabulary, not a ceiling. Compose new harness
|
|
156
|
+
shapes when the task calls for it:
|
|
157
|
+
|
|
158
|
+
- **tournament** — pair candidates, judge each pair, advance winners; better
|
|
159
|
+
than one big judge panel when candidates are too many to compare at once;
|
|
160
|
+
- **self-repair loop** — generate → check → feed the checker's findings back
|
|
161
|
+
to a fixer → re-check, bounded by attempts, with every iteration logged;
|
|
162
|
+
- **staged escalation** — a cheap pass filters the easy majority, and only
|
|
163
|
+
survivors reach the expensive treatment (the confidence-gated verify stage
|
|
164
|
+
in [code-review.md](code-review.md) is this shape);
|
|
165
|
+
- **completeness critic** — after synthesis, one agent asks "what is missing —
|
|
166
|
+
an angle never searched, a claim left unverified, a source unread?" What it
|
|
167
|
+
surfaces becomes the next round's work or is reported as a known gap. Pair it
|
|
168
|
+
with loop-until-dry: the loop proves discovery is exhausted, the critic proves
|
|
169
|
+
verification and coverage are.
|
|
170
|
+
|
|
171
|
+
Whatever the shape, the standing rules from
|
|
172
|
+
[orchestration-patterns.md](orchestration-patterns.md) still bind: positional
|
|
173
|
+
coverage accounting before `.filter(Boolean)`, no silent caps, self-contained
|
|
174
|
+
prompts, and budget arithmetic against the run limits before choosing sizes.
|
|
175
|
+
And because scripts run in the deterministic sandbox, `Date.now()`,
|
|
176
|
+
`Math.random()`, and argless `new Date()` throw — derive variation from an
|
|
177
|
+
item's index, and take any timestamp from `args` instead of reading the clock.
|