@excitedjs/dreamux 0.22.0-beta.160 → 0.22.0-beta.161

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@excitedjs/dreamux",
3
- "version": "0.22.0-beta.160",
3
+ "version": "0.22.0-beta.161",
4
4
  "description": "Dreamux core host for multi-dispatcher agent orchestration through provider interfaces.",
5
5
  "license": "MIT",
6
6
  "repository": {
@@ -26,11 +26,11 @@
26
26
  "CHANGELOG.json"
27
27
  ],
28
28
  "dependencies": {
29
- "@excitedjs/agent-runtime-claude-code": "0.5.0-beta.160",
30
- "@excitedjs/agent-runtime-codex": "0.3.5-beta.160",
31
- "@excitedjs/dreamux-types": "0.8.0-beta.160",
32
- "@excitedjs/dreamux-utils": "0.4.0-beta.160",
33
- "@excitedjs/feishu-channel": "5.0.0-beta.160",
29
+ "@excitedjs/agent-runtime-claude-code": "0.5.0-beta.161",
30
+ "@excitedjs/agent-runtime-codex": "0.3.5-beta.161",
31
+ "@excitedjs/dreamux-types": "0.8.0-beta.161",
32
+ "@excitedjs/dreamux-utils": "0.4.0-beta.161",
33
+ "@excitedjs/feishu-channel": "5.0.0-beta.161",
34
34
  "acorn": "^8.16.0",
35
35
  "croner": "^10.0.1",
36
36
  "smol-toml": "^1.6.1",
@@ -161,3 +161,4 @@ unavailable so orchestration stays deterministic.
161
161
  | --- | --- | --- |
162
162
  | Code review run | Carrying a change review (pull/merge request, git range, or working-tree diff) as one workflow run: eligibility gate, multi-lens finders, confidence-scored verification, one synthesized report. | [Code review](references/code-review.md) |
163
163
  | Orchestration and prompt patterns | Writing TeamMate prompts inside scripts, choosing `pipeline` versus `parallel`, and applying quality patterns: adversarial verify, judge panel, loop-until-dry, multi-modal sweep, completeness critic. | [Orchestration patterns](references/orchestration-patterns.md) |
164
+ | Ultracode orchestration techniques | Shaping a whole task as workflows: the five archetypes (understand/design/review/research/migrate), scouting before fan-out, chaining runs with the TeamLeader in the loop, worked compositions, novel harness shapes. | [Ultracode techniques](references/ultracode.md) |
@@ -1,394 +1,157 @@
1
1
  # Code Review Workflow
2
2
 
3
- `../SKILL.md` owns the run tools and the script API. This reference owns one
4
- recipe: carrying a full code review as a single `workflow_run` — an eligibility
5
- gate, context collection, independent multi-lens finders, per-finding
6
- confidence scoring, and one synthesized report.
7
-
8
- ## Contents
9
-
10
- - [Stage design](#stage-design) — the five stages and their failure accounting
11
- - [Prompt discipline](#prompt-discipline) — output contract, rejection
12
- semantics, self-contained prompts
13
- - [TeamMate budget](#teammate-budget) — cost arithmetic against the run caps
14
- - [Script](#script) — the complete runnable module
15
- - [Invocation](#invocation) — call shape and result fields
3
+ `../SKILL.md` owns the run tools and the script API. This reference teaches
4
+ the method for carrying a code review as one `workflow_run`: the stages, the
5
+ scoring rubric, and the false-positive discipline. Write the script for the
6
+ task at hand — the sketch at the end shows the shape, it is not a template to
7
+ paste.
16
8
 
17
9
  The review target is any change the Team workspace can read: a GitHub pull
18
10
  request, a GitLab merge request, a local branch or commit range, or an
19
- uncommitted working-tree diff. Use this recipe when the operator wants one
20
- terminal result for such a change. For interactive review — where the next
11
+ uncommitted working-tree diff. For interactive review — where the next
21
12
  instruction depends on reading the previous finding — use `spawn` and `send`
22
13
  instead.
23
14
 
24
- ## Stage design
25
-
26
- 1. **Gate** — one TeamMate checks eligibility and the run exits early when
27
- review is not needed: the change is closed or a draft, is automated or
28
- trivially safe, or already carries a review from this Team.
29
- 2. **Context** — two TeamMates in parallel: one lists the file paths (not
30
- contents) of the repository's agent guidance files (`CLAUDE.md`,
31
- `AGENTS.md`) at the root and in directories the change touches; one
32
- summarizes the change (intent, scope, risk areas). Both results feed every
33
- later prompt. When guidance discovery fails, the guidance lens is skipped
34
- and recorded as missing coverage — it must not audit against a silently
35
- empty list. A missing summary is tolerable: finders read the change
36
- themselves, and the placeholder is visible in their prompts.
37
- 3. **Find** — independent finder lenses run in parallel, each blind to the
38
- others: guidance-file compliance, a shallow bug scan limited to the changed
39
- hunks, git history and blame context, review comments on prior change
40
- requests touching the same files, guidance stated in code comments of the
41
- modified files, and concrete security issues introduced by the change. The
42
- barrier is justified: deduplication needs every lens's output at once —
43
- and it is where coverage is accounted: lens results are index-aligned with
44
- the lens list, so a failed lens is recorded as missing coverage instead of
45
- being silently dropped by `.filter(Boolean)`.
46
- 4. **Verify** — each unique finding is scored by three TeamMates with distinct
47
- focus questions (does it reproduce on the changed lines; is it pre-existing
48
- rather than introduced; does the cited guidance actually say that), using a
49
- fixed 0-100 rubric. Findings advance through `pipeline` independently of
50
- one another: each finding's vote aggregation follows its own three votes
51
- without waiting for the other findings. A finding that keeps fewer than two
52
- settled votes is returned as `unverified` — verifier failure must not
53
- silently launder a finding into a clean report.
54
- 5. **Report** — findings that keep at least two settled votes and average 80 or
55
- higher survive; one TeamMate formats the final review comment, noting
56
- incomplete coverage and unverified findings when there are any.
57
-
58
- The run returns the report plus `coverage` and `unverified` records; the
59
- caller decides where the report goes. When a channel reply tool is available, post it there; or
60
- have a follow-up `send` to the reporter TeamMate publish it with the forge's
61
- own tooling (for example `gh` for GitHub or `glab` for GitLab) once the
62
- operator confirms.
63
-
64
- ## Prompt discipline
65
-
66
- Every workflow TeamMate receives the workflow output contract: its final
67
- response is the value `agent()` returns, not a human-facing message. Prompts
68
- can therefore ask for data directly, without output-format boilerplate. With
69
- `schema`, the runtime's native structured-output mechanism enforces the shape.
70
-
71
- Failure semantics drive the script structure:
72
-
73
- - An ordinary failed turn settles the `agent()` call to `null`. `parallel`
74
- and `pipeline` contain per-item failures as `null` entries and keep results
75
- index-aligned with their inputs — account required coverage positionally
76
- first (which lens failed, which finding lost its verifiers), then drop
77
- nulls with `.filter(Boolean)` only where item identity no longer matters.
78
- - A schema call rejects — it does not return `null` — when the runtime cannot
79
- provide structured output or reports success with an empty or invalid JSON
80
- result. A directly awaited rejection fails the whole run; that is the right
81
- outcome for load-bearing calls like the gate, and the reason the gate below
82
- is awaited bare while per-finding work runs inside helpers.
83
-
84
- Keep prompts self-contained: a finder must not depend on another finder's
85
- output, and everything a verifier needs (the finding, the rubric, the
86
- false-positive list) is embedded in its prompt.
15
+ ## The flow
16
+
17
+ 1. **Gate.** One TeamMate checks eligibility; exit early when review is not
18
+ needed: the change is closed or a draft, is automated or trivially safe,
19
+ or already carries a review from this Team.
20
+ 2. **Context.** In parallel: list the file paths (not contents) of the
21
+ repository's agent guidance files (`CLAUDE.md`, `AGENTS.md`) at the root
22
+ and in directories the change touches; summarize the change (intent,
23
+ scope, risk areas). Feed both into every later prompt.
24
+ 3. **Find.** Independent finder lenses in parallel, each blind to the others:
25
+ - guidance-file compliance (read the discovered files; not every writing
26
+ instruction applies at review time);
27
+ - a shallow bug scan limited to the changed hunks;
28
+ - git blame and history of the modified code;
29
+ - review comments on prior change requests touching the same files;
30
+ - guidance stated in code comments of the modified files;
31
+ - concrete security issues introduced by the change.
32
+ Every finding must state a concrete failure scenario — the specific
33
+ inputs or state that produce the wrong outcome; a defect that cannot be
34
+ stated as a scenario is not reportable. Deduplicate across lenses by
35
+ file, line, and title. For an exhaustive audit, repeat find/verify rounds
36
+ until two consecutive rounds surface nothing new, carrying already-seen
37
+ findings in each round's prompts.
38
+ 4. **Verify.** Score each finding with parallel verifier TeamMates — three
39
+ normally, two for a quick pass — but give each verifier a *distinct focus
40
+ question* rather than re-asking the same one: does the defect reproduce on
41
+ the changed lines; is it pre-existing rather than introduced by this change;
42
+ does the cited guidance file or code comment actually say that. Diverse
43
+ focus catches failure modes that identical skeptics miss. Each verifier
44
+ receives the finding, the guidance paths, the false-positive list, its
45
+ focus question, and this rubric verbatim:
46
+ - 0: Not confident at all. False positive that does not stand up to light
47
+ scrutiny, or a pre-existing issue.
48
+ - 25: Somewhat confident. Might be real, might be a false positive; could
49
+ not verify. If stylistic, it is not explicitly called out in a relevant
50
+ guidance file.
51
+ - 50: Moderately confident. Verified real, but may be a nitpick or rare in
52
+ practice; not very important relative to the rest of the change.
53
+ - 75: Highly confident. Double checked; very likely real and will be hit
54
+ in practice; directly impacts functionality or is directly mentioned in
55
+ a guidance file.
56
+ - 100: Absolutely certain. Double checked and confirmed; will happen
57
+ frequently; evidence directly confirms it.
58
+ Verifiers also check that the stated failure scenario is concrete and
59
+ actually leads to the claimed outcome. Score toward the low end when you
60
+ cannot independently corroborate the finding — an uncorroborated finding is
61
+ a 0 or 25, not a hopeful 50. Keep a finding only when at least two verifiers
62
+ settled a vote and those votes average 80 or higher; fewer than two settled
63
+ votes is unverified coverage, not a pass. At
64
+ max effort, findings in the 50-79 band may be reported too — in their own
65
+ clearly labeled section, never as confirmed issues.
66
+ 5. **Report.** One TeamMate writes the review comment: a short verdict, then
67
+ numbered issues (grouped by severity when findings carry one), each with
68
+ the flag reason in parentheses and a file#line citation — file path alone
69
+ when no line is known; never invent a line number. Brief, no emojis. When a
70
+ channel reply tool is available, post the returned report through it;
71
+ otherwise return the report for the caller, or have a follow-up `send` to
72
+ the reporter publish it with the forge's tooling (`gh`, `glab`) once the
73
+ operator confirms.
74
+
75
+ ## False positives
87
76
 
88
77
  Instruct finders and verifiers to treat these as false positives: pre-existing
89
78
  issues; code that looks buggy but is not; pedantic nitpicks; anything a
90
79
  linter, typechecker, or CI would catch; general quality concerns not required
91
80
  by a guidance file; issues explicitly silenced in code; intentional behavior
92
- changes related to the broader change; and real issues on lines the change did
93
- not modify.
81
+ changes related to the broader change; and real issues on lines the change
82
+ did not modify. When reviewing a cumulative range, add: behavior an earlier
83
+ change introduced and a later change in the same range deliberately replaced.
94
84
 
95
- ## TeamMate budget
85
+ ## Scaling and range reviews
96
86
 
97
- One round costs `1 (gate) + 2 (context) + 6 (find) + 3 × findings (verify) +
98
- 1 (report)` TeamMates. The 1000-TeamMate lifetime cap per run leaves ample
99
- headroom for a single round (hundreds of findings) and comfortably supports
100
- repeat-until-dry rounds for exhaustive coverage; still count
101
- `6 finders + 3 × fresh findings` per extra round before choosing round
102
- counts, and match fan-out to what the operator asked for.
87
+ Match fan-out to the ask: a quick pass is one round with two votes; a normal
88
+ review is one round with three; "thoroughly audit this" repeats rounds until
89
+ dry. The confirmation bar stays the same throughout — effort widens coverage,
90
+ never the evidence standard.
103
91
 
104
- ## Script
92
+ For a cumulative range ("everything since X"), resolve the fuzzy description
93
+ into one exact `base..head` first — as the opening stage when it takes
94
+ repository work — and add a cross-change lens: a later change silently
95
+ invalidating an earlier one's assumption, dead code or docs left behind by a
96
+ superseding change, contradictory contracts between commits.
105
97
 
106
- The script uses the top-level entry: the body follows `export const meta`
107
- directly, and `await` and early `return` work at top level. Constants and helper
108
- functions remain in the same private async execution scope, so normal function
109
- hoisting applies exactly as written.
98
+ ## Failure semantics worth remembering
99
+
100
+ `agent()` settles to `null` on a failed turn, and helper results stay
101
+ index-aligned with their inputs — note which lens or verifier failed before
102
+ filtering nulls, and say so in the report instead of presenting partial
103
+ coverage as clean. A directly awaited schema call rejects on a
104
+ structured-output contract breach, which is the right outcome for
105
+ load-bearing calls like the gate. The pattern language lives in
106
+ [orchestration-patterns.md](orchestration-patterns.md).
107
+
108
+ ## Sketch
109
+
110
+ The shape, compressed — adapt lenses, schemas, rounds, and report format to
111
+ the actual task:
110
112
 
111
113
  ```js
112
114
  export const meta = {
113
115
  name: 'code-review',
114
- description: 'Multi-lens code review with confidence-scored findings and one report',
115
- whenToUse: 'One-shot review of a change request, git range, or working-tree diff.',
116
+ description: 'Multi-lens code review with confidence-scored findings',
116
117
  phases: [
117
- { title: 'gate', detail: 'eligibility check with early exit' },
118
- { title: 'context', detail: 'guidance discovery and change summary' },
119
- { title: 'find', detail: 'six independent finder lenses' },
120
- { title: 'verify', detail: 'three-vote confidence scoring per finding' },
121
- { title: 'report', detail: 'synthesized review comment' },
118
+ { title: 'gate' }, { title: 'context' }, { title: 'find' },
119
+ { title: 'verify' }, { title: 'report' },
122
120
  ],
123
121
  };
124
122
 
125
- const FALSE_POSITIVES =
126
- 'Treat as false positives: pre-existing issues; code that looks buggy but is not; ' +
127
- 'pedantic nitpicks; anything a linter, typechecker, or CI would catch; general ' +
128
- 'quality concerns (coverage, docs) unless a guidance file (CLAUDE.md, AGENTS.md) ' +
129
- 'requires them; issues explicitly silenced in code; intentional behavior changes ' +
130
- 'related to the broader change; real issues on unmodified lines.';
131
-
132
- const RUBRIC =
133
- '0: Not confident at all. False positive that does not stand up to light scrutiny, or a pre-existing issue.\n' +
134
- '25: Somewhat confident. Might be real, might be a false positive; could not verify. If stylistic, it is not explicitly called out in a relevant guidance file (CLAUDE.md, AGENTS.md).\n' +
135
- '50: Moderately confident. Verified real, but may be a nitpick or rare in practice; not very important relative to the rest of the change.\n' +
136
- '75: Highly confident. Double checked; very likely real and will be hit in practice; directly impacts functionality or is directly mentioned in a guidance file.\n' +
137
- '100: Absolutely certain. Double checked and confirmed; will happen frequently; evidence directly confirms it.';
138
-
139
- const FINDINGS_SCHEMA = {
140
- type: 'object',
141
- properties: {
142
- findings: {
143
- type: 'array',
144
- items: {
145
- type: 'object',
146
- properties: {
147
- file: { type: 'string' },
148
- line: { type: 'integer' },
149
- title: { type: 'string' },
150
- detail: { type: 'string' },
151
- reason: {
152
- type: 'string',
153
- description: 'guidance | bug | history | prior-review | comment | security',
154
- },
155
- },
156
- required: ['file', 'title', 'detail', 'reason'],
157
- additionalProperties: false,
158
- },
159
- },
160
- },
161
- required: ['findings'],
162
- additionalProperties: false,
163
- };
164
-
165
- const SCORE_SCHEMA = {
166
- type: 'object',
167
- properties: {
168
- score: { type: 'integer', minimum: 0, maximum: 100 },
169
- reasoning: { type: 'string' },
170
- },
171
- required: ['score', 'reasoning'],
172
- additionalProperties: false,
173
- };
174
-
175
- function findingKey(f) {
176
- return `${f.file}:${f.line || 0}:${f.title.toLowerCase().replace(/\s+/g, ' ').trim()}`;
177
- }
178
-
179
- const target = args && args.target;
180
- if (!target) {
181
- return { error: 'args.target is required: a PR/MR URL or number, a git range, or a working-tree description' };
182
- }
183
-
184
123
  phase('gate');
185
- const gate = await agent(
186
- `Inspect this change: ${target}. Decide whether it should be code reviewed. ` +
187
- `Ineligible if: closed; a draft; an automated or trivial change that is ` +
188
- `obviously fine; or it already has a review comment from this Team.`,
189
- {
190
- label: 'gate',
191
- phase: 'gate',
192
- intent: 'Review eligibility gate',
193
- schema: {
194
- type: 'object',
195
- properties: { eligible: { type: 'boolean' }, reason: { type: 'string' } },
196
- required: ['eligible', 'reason'],
197
- additionalProperties: false,
198
- },
199
- },
200
- );
201
- if (!gate || !gate.eligible) {
202
- return { skipped: true, reason: gate ? gate.reason : 'gate TeamMate failed' };
203
- }
124
+ const gate = await agent(`Should ${args.target} be reviewed? ...`, {
125
+ phase: 'gate', schema: GATE_SCHEMA,
126
+ });
127
+ if (!gate?.eligible) return { skipped: true, reason: gate?.reason };
204
128
 
205
129
  phase('context');
206
- const ctx = await parallel([
207
- () => agent(
208
- `For this change: ${target}. List the file paths (NOT contents) of the ` +
209
- `repository's agent guidance files — CLAUDE.md and AGENTS.md — at the ` +
210
- `repository root and in directories whose files the change modifies.`,
211
- {
212
- label: 'guidance',
213
- phase: 'context',
214
- intent: 'Guidance file discovery',
215
- schema: {
216
- type: 'object',
217
- properties: { paths: { type: 'array', items: { type: 'string' } } },
218
- required: ['paths'],
219
- additionalProperties: false,
220
- },
221
- },
222
- ),
223
- () => agent(
224
- `View this change: ${target}. Summarize it: intent, scope, key files, risk ` +
225
- `areas. Concise prose only.`,
226
- { label: 'summary', phase: 'context', intent: 'Change summarizer' },
227
- ),
130
+ const [guidance, summary] = await parallel([
131
+ () => agent(`List guidance files relevant to ${args.target}.`, { phase: 'context' }),
132
+ () => agent(`Summarize ${args.target}.`, { phase: 'context' }),
228
133
  ]);
229
- // a null discovery result is a coverage gap, distinct from a successful
230
- // empty list (a repo that simply has no guidance files)
231
- const guidanceFailed = !ctx[0];
232
- const guidancePaths = guidanceFailed ? [] : ctx[0].paths;
233
- const summary = ctx[1] || '(summary unavailable)';
234
-
235
- const lenses = [
236
- { key: 'guidance', prompt: `Audit the change for compliance with these guidance files (read them): ${JSON.stringify(guidancePaths)}. Guidance files direct code writing, so not every instruction applies at review time.` },
237
- { key: 'shallow-bugs', prompt: 'Read only the changed hunks and scan for obvious large bugs. Do not read extra context. Skip small issues and nitpicks.' },
238
- { key: 'git-history', prompt: 'Read git blame and history of the modified code; flag bugs visible only in light of that historical context, such as reintroduced bugs or violated invariants.' },
239
- { key: 'prior-reviews', prompt: 'Find previous change requests (pull/merge requests) touching these files; check review comments on them that also apply to this change.' },
240
- { key: 'code-comments', prompt: 'Read code comments in the modified files; flag violations of guidance stated in those comments.' },
241
- { key: 'security', prompt: 'Review the changed code for concrete, exploitable security issues introduced by this change (injection, authorization gaps, secret handling). No generic hardening advice.' },
242
- ];
243
134
 
244
135
  phase('find');
245
- // without discovered guidance files the guidance lens has nothing real to
246
- // audit — skip it and record the gap instead of auditing an empty list
247
- const activeLenses = guidanceFailed
248
- ? lenses.filter((lens) => lens.key !== 'guidance')
249
- : lenses;
250
- // parallel() preserves order, so a null entry identifies the failed lens
251
- const lensResults = await parallel(activeLenses.map((lens) => () =>
252
- agent(
253
- `Code review this change through one lens: ${target}\n${lens.prompt}\n` +
254
- `${FALSE_POSITIVES}\nChange summary:\n${summary}`,
255
- {
256
- label: `find-${lens.key}`,
257
- phase: 'find',
258
- intent: `Finder lens: ${lens.key}`,
259
- schema: FINDINGS_SCHEMA,
260
- },
261
- ),
262
- ));
263
- const failedLenses = [
264
- ...(guidanceFailed ? ['guidance'] : []),
265
- ...activeLenses.filter((lens, i) => !lensResults[i]).map((lens) => lens.key),
266
- ];
267
- if (failedLenses.length === lenses.length) {
268
- return { error: 'every finder lens failed; the change was not reviewed', failedLenses };
269
- }
270
- if (failedLenses.length) {
271
- log(`coverage incomplete: failed lenses ${failedLenses.join(', ')}`);
272
- }
273
- const found = lensResults.filter(Boolean).flatMap((r) => r.findings);
274
-
275
- const seen = new Set();
276
- const unique = [];
277
- for (const f of found) {
278
- const key = findingKey(f);
279
- if (!seen.has(key)) { seen.add(key); unique.push(f); }
280
- }
281
- log(`${found.length} raw findings, ${unique.length} unique`);
282
-
283
- const checkedLenses = lenses
284
- .filter((lens) => !failedLenses.includes(lens.key))
285
- .map((lens) => lens.key);
286
- function coverageRecord(unverifiedCount) {
287
- return {
288
- complete: failedLenses.length === 0 && unverifiedCount === 0,
289
- failedLenses,
290
- unverifiedFindings: unverifiedCount,
291
- };
292
- }
293
- const noIssuesReport = failedLenses.length === 0
294
- ? 'No issues found. Checked for bugs, security, history, and guidance-file compliance.'
295
- : `No issues found by the lenses that completed (${checkedLenses.join(', ')}). ` +
296
- `Coverage is incomplete: ${failedLenses.join(', ')} failed.`;
297
- if (!unique.length) {
298
- return { issues: [], unverified: [], coverage: coverageRecord(0), report: noIssuesReport };
299
- }
136
+ const findings = dedupe((await parallel(LENSES.map((lens) => () =>
137
+ agent(finderPrompt(lens, args.target, guidance, summary), {
138
+ phase: 'find', schema: FINDINGS_SCHEMA,
139
+ }),
140
+ ))).filter(Boolean).flatMap((r) => r.findings));
300
141
 
301
142
  phase('verify');
302
- const focusQuestions = [
303
- 'does it reproduce on the changed lines',
304
- 'is it pre-existing rather than introduced by this change',
305
- 'does the cited guidance file or code comment actually say that',
306
- ];
307
- const judged = await pipeline(
308
- unique,
309
- (finding) => parallel(focusQuestions.map((focus) => () =>
310
- agent(
311
- `Confidence-score this code review finding for the change: ${target}\n` +
312
- `Finding: ${JSON.stringify(finding)}\n` +
313
- `Guidance files: ${JSON.stringify(guidancePaths)}\n` +
314
- `Focus question: ${focus}.\n${FALSE_POSITIVES}\n` +
315
- `Score with this rubric verbatim:\n${RUBRIC}`,
316
- { phase: 'verify', intent: 'Finding verifier', schema: SCORE_SCHEMA },
317
- ),
318
- )),
319
- (votes, finding, index) => {
320
- const settled = (votes || []).filter(Boolean);
321
- const average = settled.length
322
- ? settled.reduce((sum, vote) => sum + vote.score, 0) / settled.length
323
- : 0;
324
- return { ...finding, confidence: Math.round(average), votes: settled.length, index };
325
- },
143
+ const scored = await pipeline(
144
+ findings,
145
+ (f) => parallel(FOCUS_QUESTIONS.map((focus) => () =>
146
+ agent(rubricPrompt(f, focus), { phase: 'verify', schema: SCORE_SCHEMA }))),
147
+ (votes, f) => ({ ...f, votes: (votes || []).filter(Boolean) }),
326
148
  );
327
- // pipeline() results are index-aligned with `unique`: a null entry or a
328
- // finding with fewer than two settled votes is unverified, not clean
329
- const unverified = unique.filter((f, i) => {
330
- const j = judged[i];
331
- return !j || j.votes < 2;
332
- });
333
- const confirmed = judged
334
- .filter(Boolean)
335
- .filter((f) => f.votes >= 2 && f.confidence >= 80);
336
- const coverage = coverageRecord(unverified.length);
337
- log(`${confirmed.length} findings at >=80 confidence, ${unverified.length} unverified`);
149
+ const confirmed = scored.filter(Boolean)
150
+ .filter((f) => f.votes.length >= 2 && average(f.votes) >= 80);
338
151
 
339
152
  phase('report');
340
- const unverifiedNote = unverified.length
341
- ? ` ${unverified.length} finding(s) could not be verified (verifier failures) ` +
342
- `and are returned in \`unverified\` for manual triage.`
343
- : '';
344
- if (!confirmed.length) {
345
- const reportText = unverified.length
346
- ? `No issues confirmed.${unverifiedNote}`
347
- : noIssuesReport;
348
- return { issues: [], unverified, coverage, report: reportText };
349
- }
350
- const report = await agent(
351
- `Write the final code review comment for the change: ${target}\n` +
352
- `Confirmed issues (JSON): ${JSON.stringify(confirmed)}\n` +
353
- `Unverified findings — verifiers failed, no confidence claim (JSON): ${JSON.stringify(unverified)}\n` +
354
- `Coverage (JSON): ${JSON.stringify(coverage)}\n` +
355
- `Format: a "### Code review" heading, "Found N issues:", then a numbered list; ` +
356
- `each item is a brief description with the flag reason in parentheses and a ` +
357
- `citation on the next line — file#line when the finding carries a line number, ` +
358
- `the file path alone otherwise; never invent a line number. If there are ` +
359
- `unverified findings, add a short "Unverified" section listing them without ` +
360
- `confidence claims. If coverage is incomplete, end with one line naming the ` +
361
- `failed lenses. Brief, no emojis. Return markdown only.`,
362
- { label: 'report', phase: 'report', intent: 'Review report writer' },
363
- );
364
- return { issues: confirmed, unverified, coverage, report };
365
- ```
366
-
367
- The verify stage shows the three-argument stage signature: stage one returns
368
- only the vote array, and stage two recovers the finding and its position from
369
- `(votes, finding, index)` instead of threading them through the return value.
370
-
371
- ## Invocation
372
-
373
- ```
374
- workflow_run({
375
- script: <the script above>,
376
- args: { target: 'https://github.com/owner/repo/pull/123' },
377
- })
153
+ return {
154
+ issues: confirmed,
155
+ report: await agent(reportPrompt(args.target, confirmed), { phase: 'report' }),
156
+ };
378
157
  ```
379
-
380
- `args.target` is forge-neutral — pass whatever identifies the change in the
381
- Team workspace:
382
-
383
- - a pull-request or merge-request URL or number
384
- (`'https://gitlab.example.com/group/repo/-/merge_requests/45'`, `'#123'`);
385
- - a local git range (`'main...feature'`);
386
- - a working-tree description (`'the uncommitted diff in the workspace'`).
387
-
388
- `max_concurrency` defaults to 16; lower it only when the workspace or runtime
389
- provider needs gentler fan-out. After the terminal completion, read `issues`
390
- for the confirmed findings, `unverified` for findings whose verifiers failed,
391
- `coverage` for whether every finder lens ran and every finding was verified,
392
- and `report` for the formatted comment; use the reporter TeamMate's concrete
393
- name with `send` for follow-ups such as posting the comment through the
394
- forge's tooling or re-checking one finding interactively.
@@ -0,0 +1,177 @@
1
+ # Ultracode Orchestration Techniques
2
+
3
+ `../SKILL.md` owns the run tools and the script API;
4
+ [orchestration-patterns.md](orchestration-patterns.md) owns prompt discipline,
5
+ the `pipeline`-versus-`parallel` rules, and the individual quality patterns.
6
+ This reference owns the level above: how to shape a whole task as ultracode
7
+ workflows — choosing an archetype, scouting before orchestrating, chaining
8
+ runs, and composing the quality patterns into complete harnesses.
9
+
10
+ ## Contents
11
+
12
+ - [Workflow archetypes](#workflow-archetypes) — five reusable single-run
13
+ shapes
14
+ - [Scout before you orchestrate](#scout-before-you-orchestrate) — discover
15
+ the work-list first
16
+ - [Chain runs, stay in the loop](#chain-runs-stay-in-the-loop) — multi-phase
17
+ work as a sequence of runs
18
+ - [Worked compositions](#worked-compositions) — barrier-when-justified and
19
+ the exhaustive-discovery loop
20
+ - [Loop-until-count](#loop-until-count) — accumulate to a target
21
+ - [Novel harnesses](#novel-harnesses) — beyond the named patterns
22
+
23
+ ## Workflow archetypes
24
+
25
+ Most tasks that deserve a workflow fit one of five single-run shapes. Name the
26
+ shape first; the script structure follows from it.
27
+
28
+ - **Understand** — parallel readers over relevant subsystems, one structured
29
+ map out. Fan out one `agent()` per subsystem with a shared schema, then one
30
+ synthesis agent merges the maps. Use when the question is "how does this
31
+ system work" and no single context can hold it all.
32
+ - **Design** — a judge panel over N independent approaches. Generate attempts
33
+ from deliberately different angles (smallest-change-first, risk-first,
34
+ user-first), score them with parallel judges against explicit criteria,
35
+ synthesize from the winner while grafting the runners-up's best ideas. Use
36
+ when the solution space is wide and one-attempt-iterated would anchor early.
37
+ - **Review** — dimensions → find → adversarially verify. The complete recipe
38
+ lives in [code-review.md](code-review.md); the shape generalizes to any
39
+ audit: independent finder lenses, confidence-gated verification, one
40
+ severity-ordered report.
41
+ - **Research** — multi-modal sweep → deep-read → verify → synthesize: search
42
+ agents per angle, URL dedup at a justified barrier, per-source claim
43
+ extraction, adversarial claim verification, one cited report with honest
44
+ coverage caveats.
45
+ - **Migrate** — discover sites → transform each → verify each. Scout the
46
+ work-list first (see below), then `pipeline` items through
47
+ transform-then-verify stages. Keep concurrent edits on clearly independent
48
+ paths, or serialize the write stage — workflow TeamMates share the Team
49
+ workspace.
50
+
51
+ ## Scout before you orchestrate
52
+
53
+ The coordination graph must be known before the orchestration step — not
54
+ before the task. When the work-list is unknown ("migrate every call site",
55
+ "review whatever changed"), scout first and fan out second:
56
+
57
+ - cheap scouting (one `git diff --stat`, one directory listing) belongs in
58
+ the TeamLeader's own turn before `workflow_run`, with the result passed in
59
+ as `args`;
60
+ - scouting that itself needs an agent (resolving a fuzzy range, inventorying
61
+ a subsystem) belongs as the script's first stage, awaited bare so an
62
+ unresolvable work-list fails the run before any fan-out — the resolve stage
63
+ in [code-review.md](code-review.md)'s range mode is this technique.
64
+
65
+ Never fan out against a guess: every downstream prompt should receive the
66
+ resolved work-list, not the fuzzy description.
67
+
68
+ ## Chain runs, stay in the loop
69
+
70
+ A workflow is one well-scoped fan-out with a deterministic graph. Larger work
71
+ — understand, then design, then implement, then review — is a SEQUENCE of
72
+ workflows with the TeamLeader reading each terminal result before shaping the
73
+ next run. Resist packing multi-phase judgment into one mega-script: the
74
+ decisions between phases (is the map good enough? which design won? is the
75
+ implementation ready for review?) are exactly the parts that should stay with
76
+ the TeamLeader, interactive and revisable.
77
+
78
+ Three practical consequences:
79
+
80
+ - `workflow_run` returns only `{ run_id }` immediately — a durable acceptance
81
+ receipt, not the result. Save the id, wait for the terminal completion that
82
+ Dreamux pushes, and build the next run's `args` from that completion's
83
+ `result`; feeding the receipt into the next run is the classic mistake.
84
+ `workflow_status({ run_id })` is for explicit recovery, not a polling loop.
85
+ - return STRUCTURED results (not just prose) from each run so the next run's
86
+ `args` can be built from them mechanically;
87
+ - after a run, follow up interactively with a recorded concrete TeamMate name
88
+ via `send` when one result needs a clarification — reshaping and re-running
89
+ the whole workflow is for when the graph itself was wrong.
90
+
91
+ ## Worked compositions
92
+
93
+ **A barrier when it is genuinely justified** — dedup across the full result
94
+ set before expensive downstream work:
95
+
96
+ ```js
97
+ const all = await parallel(angles.map((a) => () =>
98
+ agent(searchPrompt(a), { phase: 'search', schema: SOURCES_SCHEMA })));
99
+ const sources = dedupeByUrl(all.filter(Boolean).flatMap((r) => r.sources));
100
+ // only now fan out the expensive per-source work
101
+ const reports = await pipeline(sources, fetchStage, verifyStage);
102
+ ```
103
+
104
+ The barrier earns its latency because the dedup needs every angle's output at
105
+ once; the per-source work that follows goes back to `pipeline`.
106
+
107
+ **The exhaustive-discovery loop** — finders → dedup vs SEEN → verification →
108
+ repeat until dry, the composition behind every "audit everything" request:
109
+
110
+ ```js
111
+ const seen = new Set();
112
+ const accepted = [];
113
+ let dry = 0;
114
+ while (dry < 2) {
115
+ const found = (await parallel(finderThunks(seen))).filter(Boolean)
116
+ .flatMap((r) => r.findings)
117
+ .filter((f) => !seen.has(key(f)));
118
+ if (!found.length) { dry += 1; continue; }
119
+ dry = 0;
120
+ found.forEach((f) => seen.add(key(f)));
121
+ const judged = await pipeline(found, verifyStage, aggregateStage);
122
+ accepted.push(...judged.filter(Boolean).filter((f) => f.accepted));
123
+ }
124
+ ```
125
+
126
+ Dedupe against SEEN, not against `accepted` — otherwise judge-rejected
127
+ findings reappear every round and the loop never converges. When adapting the
128
+ sketch, remember the standing failure-plumbing rules from
129
+ [orchestration-patterns.md](orchestration-patterns.md): note failed positions
130
+ before filtering nulls, and bound loops when the finding population may not
131
+ converge.
132
+
133
+ ## Loop-until-count
134
+
135
+ For "give me N of X" requests, accumulate to the target instead of guessing
136
+ the fan-out size:
137
+
138
+ ```js
139
+ const bugs = [];
140
+ while (bugs.length < 10) {
141
+ const result = await agent('Find bugs not in this list: ' +
142
+ JSON.stringify(bugs.map((b) => b.title)), { schema: BUGS_SCHEMA });
143
+ if (!result || !result.bugs.length) break; // a dry pass ends the hunt honestly
144
+ bugs.push(...result.bugs);
145
+ log(`${bugs.length}/10 found`);
146
+ }
147
+ if (bugs.length < 10) log(`stopped short: ${bugs.length}/10 — no more found`);
148
+ ```
149
+
150
+ Always pair the target with a dry-pass exit and `log()` the shortfall — a
151
+ loop that cannot reach N must say so, not spin against the agent cap.
152
+
153
+ ## Novel harnesses
154
+
155
+ The named patterns are a vocabulary, not a ceiling. Compose new harness
156
+ shapes when the task calls for it:
157
+
158
+ - **tournament** — pair candidates, judge each pair, advance winners; better
159
+ than one big judge panel when candidates are too many to compare at once;
160
+ - **self-repair loop** — generate → check → feed the checker's findings back
161
+ to a fixer → re-check, bounded by attempts, with every iteration logged;
162
+ - **staged escalation** — a cheap pass filters the easy majority, and only
163
+ survivors reach the expensive treatment (the confidence-gated verify stage
164
+ in [code-review.md](code-review.md) is this shape);
165
+ - **completeness critic** — after synthesis, one agent asks "what is missing —
166
+ an angle never searched, a claim left unverified, a source unread?" What it
167
+ surfaces becomes the next round's work or is reported as a known gap. Pair it
168
+ with loop-until-dry: the loop proves discovery is exhausted, the critic proves
169
+ verification and coverage are.
170
+
171
+ Whatever the shape, the standing rules from
172
+ [orchestration-patterns.md](orchestration-patterns.md) still bind: positional
173
+ coverage accounting before `.filter(Boolean)`, no silent caps, self-contained
174
+ prompts, and budget arithmetic against the run limits before choosing sizes.
175
+ And because scripts run in the deterministic sandbox, `Date.now()`,
176
+ `Math.random()`, and argless `new Date()` throw — derive variation from an
177
+ item's index, and take any timestamp from `args` instead of reading the clock.