amsd-pipeline-shared 0.5.37 → 0.5.39
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/gateway-files.txt +1 -0
- package/orchestrations/prompts/templates/agent-proposal.json +4 -3
- package/orchestrations/prompts/templates/roster-review.json +7 -6
- package/orchestrations/prompts/templates/set-fidelity-check.json +18 -0
- package/orchestrations/scripts/lib/prelaunch.js +68 -6
- package/orchestrations/scripts/lib/project-prompt-builder.js +11 -6
- package/orchestrations/scripts/spec-mode-runner.js +35 -4
- package/package.json +1 -1
package/docs/gateway-files.txt
CHANGED
|
@@ -103,6 +103,7 @@ prompts/templates/roster-specialisation-plan.json
|
|
|
103
103
|
prompts/templates/roster-specialisation.json
|
|
104
104
|
prompts/templates/roster-tool-grant.json
|
|
105
105
|
prompts/templates/runtime-boundary-review.json
|
|
106
|
+
prompts/templates/set-fidelity-check.json
|
|
106
107
|
prompts/templates/shared-file-ownership.json
|
|
107
108
|
prompts/templates/skill-assessment-postphase.json
|
|
108
109
|
prompts/templates/skill-assessment-prephase.json
|
|
@@ -23,9 +23,10 @@
|
|
|
23
23
|
"was perverse — a WEAK estate survey gave a weak investigator signal and the mint guessed",
|
|
24
24
|
"'implementer'; a CORRECT, investigator-rich survey made it label everything 'investigator'.",
|
|
25
25
|
"Improving the survey made the roster worse.",
|
|
26
|
-
"2026-08-17: the name rule said 'ending in \"-engineer\" or \"-specialist\"'. The invocation registry has twenty name-shape rules and `specialist` is in none of them, so every name of that shape threw at resolveSeam and killed the run at mint — a coin flip on the model's wording. The rule is now __NAME_RULE__, derived from the registry's own seamPatterns, so the mint can only be offered a shape that routes. The examples above lost their suffix for the same reason: they were a fourth copy of the same fact."
|
|
26
|
+
"2026-08-17: the name rule said 'ending in \"-engineer\" or \"-specialist\"'. The invocation registry has twenty name-shape rules and `specialist` is in none of them, so every name of that shape threw at resolveSeam and killed the run at mint — a coin flip on the model's wording. The rule is now __NAME_RULE__, derived from the registry's own seamPatterns, so the mint can only be offered a shape that routes. The examples above lost their suffix for the same reason: they were a fourth copy of the same fact.",
|
|
27
|
+
"v2 (operator, 2026-10-24: the roster minted for AMSD-2974 named every agent after the ticket's subject (<codeline>-toast-detective, toast-accessibility-engineer) and its briefs described that one ticket; the roster and its prompts are cached per codeline and reused for every later story, so the next ticket on these codelines would inherit them). A role is the codeline's, for every story it will ever be given."
|
|
27
28
|
],
|
|
28
|
-
"body": "You are a senior engineering manager designing an AI agent team for a software project.\n\nThe following __FIXED_ROLE_COUNT__ fixed roles are ALWAYS present and must NOT be reproduced:\n__FIXED_ROLE_LIST__\n\nGiven the project manifest, tech stack, and clarification answers, propose 2-6 project-specific engineering agent roles that will implement the user stories. Each role should map to a distinct domain of the project — one per domain, e.g. a react-frontend role, a postgres-data role, an auth-security role. The exact name shape is fixed below.\n\nEvery role is one of TWO KINDS, and you must state which:\n\n- \"implementer\" — WRITES the change. This is the role that edits source and makes the work\n happen. An implementer spans the project, so its codeline is \"*\".\n- \"investigator\" — READS one codeline and characterises what is there for the implementer. It\n never writes. An investigator names exactly ONE codeline, spelled as listed in scope.\n\nYOU MUST PROPOSE AT LEAST ONE IMPLEMENTER. A team of investigators can describe the work and\ncannot do it: every story is assigned to an implementer, so a roster without one leaves every\nstory with nobody to write it and the run cannot proceed. Investigators are optional and\nadditive — propose them where a codeline genuinely needs its own reader.\n\nThe survey above recommends both: which codelines need investigating, and which need work\nwritten. Those are recommendations about the team, not instructions — you decide the roster.\n\nFor each role provide:\n__NAME_RULE__\n- kind: exactly \"implementer\" or \"investigator\"\n- codeline: for an investigator, the ONE codeline it reads; for an implementer, \"*\"\n- systemPrompt: A detailed system prompt (150-300 words) describing the agent's expertise, the project's conventions, key files/directories they own, coding patterns they follow, and tools they use. Be specific to this project.\n- rationale: One sentence explaining why THIS project needs THIS role. It is read by a human\n reviewing the team and seeds the agent's knowledge base, so it must say something.\n\nRespond with ONLY valid JSON (no markdown fences):\n{\n \"proposedAgents\": [\n { \"name\": \"<string>\", \"kind\": \"<implementer|investigator>\", \"codeline\": \"<name or *>\", \"systemPrompt\": \"<string>\", \"rationale\": \"<string>\" }\n ]\n}",
|
|
29
|
+
"body": "You are a senior engineering manager designing an AI agent team for a software project.\n\nThe following __FIXED_ROLE_COUNT__ fixed roles are ALWAYS present and must NOT be reproduced:\n__FIXED_ROLE_LIST__\n\nGiven the project manifest, tech stack, and clarification answers, propose 2-6 project-specific engineering agent roles that will implement the user stories. Each role should map to a distinct domain of the project — one per domain, e.g. a react-frontend role, a postgres-data role, an auth-security role. The exact name shape is fixed below.\n\nEvery role is one of TWO KINDS, and you must state which:\n\n- \"implementer\" — WRITES the change. This is the role that edits source and makes the work\n happen. An implementer spans the project, so its codeline is \"*\".\n- \"investigator\" — READS one codeline and characterises what is there for the implementer. It\n never writes. An investigator names exactly ONE codeline, spelled as listed in scope.\n\nYOU MUST PROPOSE AT LEAST ONE IMPLEMENTER. A team of investigators can describe the work and\ncannot do it: every story is assigned to an implementer, so a roster without one leaves every\nstory with nobody to write it and the run cannot proceed. Investigators are optional and\nadditive — propose them where a codeline genuinely needs its own reader.\n\nThe survey above recommends both: which codelines need investigating, and which need work\nwritten. Those are recommendations about the team, not instructions — you decide the roster.\n\nFor each role provide:\n__NAME_RULE__\n- kind: exactly \"implementer\" or \"investigator\"\n- codeline: for an investigator, the ONE codeline it reads; for an implementer, \"*\"\n- systemPrompt: A detailed system prompt (150-300 words) describing the agent's expertise, the project's conventions, key files/directories they own, coding patterns they follow, and tools they use. Be specific to this project.\n- rationale: One sentence explaining why THIS project needs THIS role. It is read by a human\n reviewing the team and seeds the agent's knowledge base, so it must say something.\n\nRespond with ONLY valid JSON (no markdown fences):\n{\n \"proposedAgents\": [\n { \"name\": \"<string>\", \"kind\": \"<implementer|investigator>\", \"codeline\": \"<name or *>\", \"systemPrompt\": \"<string>\", \"rationale\": \"<string>\" }\n ]\n}\n\nA ROLE OUTLIVES THE TICKET. The roster is kept with its codeline and reused for every story that codeline is given later, so each\nrole is minted for the codeline's work, not for the work in scope now: its name says what kind of engineer or investigator it is and\nwhich codeline it serves, never the subject, component or ticket of the current story; its system prompt describes the codeline — its\nstack, conventions, surfaces and how to verify there — and the kind of work the role does, and names the current ticket's subject,\ncomponents or files only as examples marked as such. A role that only makes sense for the story in scope is a defect.\n",
|
|
29
30
|
"description": "Phase B of scaffolding: propose this project's own engineering agent roles. THE MINT PROMPT.",
|
|
30
31
|
"id": "agent-proposal",
|
|
31
32
|
"placeholders": [
|
|
@@ -33,7 +34,7 @@
|
|
|
33
34
|
"__FIXED_ROLE_LIST__",
|
|
34
35
|
"__NAME_RULE__"
|
|
35
36
|
],
|
|
36
|
-
"version":
|
|
37
|
+
"version": 2,
|
|
37
38
|
"seams": [
|
|
38
39
|
"agent-mint"
|
|
39
40
|
]
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
"",
|
|
5
5
|
"The roster reviewer is the only adversary the roster has: it is what stops a defective",
|
|
6
6
|
"brief reaching an implementer. Extracted verbatim into a builder first, then pinned byte",
|
|
7
|
-
"for byte on BOTH branches
|
|
7
|
+
"for byte on BOTH branches — the documentation section is entirely different prose when",
|
|
8
8
|
"nothing was fetched, and pinning only the populated case would let that branch rot.",
|
|
9
9
|
"",
|
|
10
10
|
"The documentation section is one value, assembled by the caller. A template that branches",
|
|
@@ -20,9 +20,10 @@
|
|
|
20
20
|
"positive and was removed. The reviewer is the right place: it is handed what the registry",
|
|
21
21
|
"already declares - the artefacts this run requires, the seam that produces each, and which",
|
|
22
22
|
"roster member resolves to which seam - and asked whether the team covers them. No role or kind",
|
|
23
|
-
"is named in this prompt; the coverage block is derived."
|
|
23
|
+
"is named in this prompt; the coverage block is derived.",
|
|
24
|
+
"v2 (operator, 2026-10-24: the roster minted for AMSD-2974 named every agent after the ticket's subject (<codeline>-toast-detective, toast-accessibility-engineer) and its briefs described that one ticket; the roster and its prompts are cached per codeline and reused for every later story, so the next ticket on these codelines would inherit them). A role is the codeline's, for every story it will ever be given."
|
|
24
25
|
],
|
|
25
|
-
"body": "THE ROSTER JUST MINTED FOR THIS PROJECT
|
|
26
|
+
"body": "THE ROSTER JUST MINTED FOR THIS PROJECT — review it, both as individual briefs and AS A TEAM.\n\nWHAT THIS RUN NEEDS THE TEAM TO PRODUCE, and who in this roster can produce it:\n__COVERAGE_BLOCK__\n\nA roster can be made entirely of well-formed agents and still be unable to do the work. Where a\nline above says NOBODY, no one in this roster can produce something the run requires — that is a\ndefect of the ROSTER, and it is invisible to reading briefs one at a time, because an absent role\nhas no brief to find fault with. Report it as a blocking finding naming what cannot be produced\nand what kind of role would produce it.\n\nNow review every brief below:\n\n__BRIEF_BLOCK__\n\nTHE CODELINES THESE BRIEFS DESCRIBE, and what each one declares:\n__CODELINE_BLOCK__\n\nTHE WORK THE PROJECT WAS ASKED TO DO:\n__TICKET_BLOCK__\n\n__DOC_SECTION__\n\n__TOOL_LINE__\n\nTHE PIPELINE STAGES THAT RUN ALONGSIDE THIS ROSTER:\n\n__PIPELINE_STAGES__\n\nA ROSTER IS NOT THE WHOLE PIPELINE. A brief may correctly defer work to a stage above rather than\nto a role beside it, and a brief that tells its implementer to leave such work alone is describing\nthe pipeline accurately, not inventing cover. Check any such claim against the list above before\ncalling it false: the absence of a matching ROLE is not evidence when the brief named a STAGE.\n\nWhere the list above does not contain the stage a brief names, that IS a finding, and a blocking\none — the implementer has been told to rely on something that will never run.\n\nReturn every defect you can evidence, and nothing you cannot.\n\nWhere a finding turns on whether a named dependency or path is present in a codeline, fill in\nthe verification field as well as writing the prose. The pipeline re-runs that exact check\nagainst the repository and DISCARDS any finding the check refutes — so a careless reading\ncosts nothing, and a correct one is confirmed independently of how convincingly you argued it.\n\nA ROLE OUTLIVES THE TICKET. This roster is kept with its codelines and reused for every story they are given later. A role minted for\nthe work in scope now rather than for the codeline — named after the current story's subject, component or ticket, or briefed so that\nit would mislead an implementer on a different story of the same codeline — is a blocking finding: name the role, quote the words,\nand say what the codeline-level role would be. A brief that names the current ticket's subject or files only as marked examples of the\ncodeline's conventions is not this finding.\n",
|
|
26
27
|
"description": "Review every brief in a freshly minted roster and report evidenced defects.",
|
|
27
28
|
"id": "roster-review",
|
|
28
29
|
"placeholders": [
|
|
@@ -34,13 +35,13 @@
|
|
|
34
35
|
"__TICKET_BLOCK__",
|
|
35
36
|
"__TOOL_LINE__"
|
|
36
37
|
],
|
|
37
|
-
"version":
|
|
38
|
+
"version": 2,
|
|
38
39
|
"seams": [
|
|
39
40
|
"roster-review"
|
|
40
41
|
],
|
|
41
|
-
"$whyPipelineStages": "Added 2026-08-27 after run 20260827T100559Z died here. Both implementer briefs said a dedicated pipeline stage writes test files
|
|
42
|
+
"$whyPipelineStages": "Added 2026-08-27 after run 20260827T100559Z died here. Both implementer briefs said a dedicated pipeline stage writes test files — TRUE: tc-writer and repro-test-writer are declared seams — and the reviewer, whose only inputs were the roster, the codelines, the tickets and the coverage, concluded 'no such role exists in this roster' and blocked twice. It was asked to falsify a claim about the PIPELINE while being handed only the ROSTER. The gate was working; its evidence was incomplete. The list is derived from the seam registry at render time, so it names no stage here and cannot go stale when a seam is added or removed.",
|
|
42
43
|
"mayBeEmpty": [
|
|
43
44
|
"__DOC_SECTION__"
|
|
44
45
|
],
|
|
45
|
-
"$whyMayBeEmpty": "Declared 2026-08-27 by sweeping the CLASS rather than the instance, after three runs died on it one placeholder at a time. Every call site for these writes an explicit empty branch
|
|
46
|
+
"$whyMayBeEmpty": "Declared 2026-08-27 by sweeping the CLASS rather than the instance, after three runs died on it one placeholder at a time. Every call site for these writes an explicit empty branch — the story is not being split, attempt one carries no forced-retry note, no documentation was fetched, the persona prefix is optional — so absence is a real state, which is the renderer's own test. Placeholders supplied by a lookup that may FAIL (`x || ''`) are deliberately NOT here: for those, empty means broken, and declaring them would hide the blank-brief defect the renderer exists to surface. Unanimity is required — one deliberate branch among several defensive lookups is not a licence."
|
|
46
47
|
}
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
{
|
|
2
|
+
"id": "set-fidelity-check",
|
|
3
|
+
"version": 1,
|
|
4
|
+
"seams": [],
|
|
5
|
+
"layer": "engine",
|
|
6
|
+
"description": "A judge reads a codeline's minted roster and reusable prompts against the ticket they were minted for and reports what is geared to that ticket rather than to the codeline.",
|
|
7
|
+
"placeholders": [
|
|
8
|
+
"__CODELINE__",
|
|
9
|
+
"__TICKET_KEY__",
|
|
10
|
+
"__TICKET_TITLE__",
|
|
11
|
+
"__ROSTER__",
|
|
12
|
+
"__PROMPTS__"
|
|
13
|
+
],
|
|
14
|
+
"$why": [
|
|
15
|
+
"v1 (operator, 2026-10-24: \"We need some post pause 1 checkpoints to ensure no contamination in codeline level prompts and profiles\"): run 13's roster named every agent after the ticket's subject and ~half its reusable prompts described that one ticket; the sets are cached per codeline and reused for every later story. Judged by an agent, not a word list; run by the pre-launch of the leg after pause 1 and reported at pause 1."
|
|
16
|
+
],
|
|
17
|
+
"body": "You judge what a pipeline minted for ONE codeline and will REUSE for every later story on it: its agent roster (names and briefs) and\nits reusable agent prompts. They were minted while this ticket was in scope:\n\nTicket __TICKET_KEY__: __TICKET_TITLE__\nCodeline: __CODELINE__\n\nTHE ROSTER (each agent: name, kind, brief):\n__ROSTER__\n\nTHE PROMPTS (each: id, then its body):\n__PROMPTS__\n\nReport every place where the roster or a prompt is geared to THIS ticket rather than to the codeline: an agent named after the\nticket's subject, component or key; a brief or prompt that takes this ticket's subject, components, files or requirements as its\nsubject or as standing instruction, so that an implementer on a DIFFERENT story of this codeline would be misled. A mention of the\nticket's subject or files only as a marked example of the codeline's conventions is not geared. Quote the words. Report nothing you\ncannot quote. Answer in the structured form you are given; `geared` is empty when nothing is geared."
|
|
18
|
+
}
|
|
@@ -95,8 +95,11 @@ function ticketSnapshots(runsRoots, ticket) {
|
|
|
95
95
|
return snaps;
|
|
96
96
|
}
|
|
97
97
|
const snapshotCopy = (snap, f) => (f.kind === 'prompt' ? [path.join(snap.dir, 'prompts', f.name)] : [path.join(snap.dir, f.name), path.join(snap.dir, 'reviewed', f.name)]).find((p) => fs.existsSync(p)) || null;
|
|
98
|
-
function seed({ install, project, ticket, runsRoots }) {
|
|
98
|
+
function seed({ install, project, ticket, runsRoots, freshMint = false }) {
|
|
99
99
|
const files = seededFiles(install, project);
|
|
100
|
+
// A FRESH MINT HAS NOTHING TO SEED (2026-10-24: the seeded sets were removed as ticket-geared; the Agents phase mints this run's own
|
|
101
|
+
// roster and prompts). Nothing seeded is then the expected state, not a defect; anything seeded is still judged as before.
|
|
102
|
+
if (!files.length && freshMint) return res('seed', true, 'nothing seeded — a fresh mint: the Agents phase mints this run\'s roster and prompts', { files: 0, freshMint: true });
|
|
100
103
|
if (!files.length) return res('seed', false, 'nothing is seeded — no prompts and no minted files on disk');
|
|
101
104
|
const prefix = String(ticket).replace(/-\d+$/, '').replace(/[^A-Za-z0-9]/g, '');
|
|
102
105
|
const keyRe = new RegExp(`\\b${prefix}-\\d+\\b`, 'g');
|
|
@@ -167,7 +170,8 @@ function cache({ install, project, projectConfigDir, codeline, variant = process
|
|
|
167
170
|
* perfect"): each codeline folder's recorded agents and prompts, and its completion built from the current inputs
|
|
168
171
|
* (project-prompt-builder.js codelineSetComplete). A lane whose codeline has none does not run, so the launch is not started.
|
|
169
172
|
*/
|
|
170
|
-
function codelineSets({ install, project, projectConfigDir, codelines = [], env = {} }) {
|
|
173
|
+
function codelineSets({ install, project, projectConfigDir, codelines = [], env = {}, freshMint = false }) {
|
|
174
|
+
if (freshMint) return res('codeline-sets', true, `a fresh mint: ${codelines.length} codeline(s) get their sets from this run's Agents phase: ${codelines.join(', ')}`, { rows: [], freshMint: true });
|
|
171
175
|
const O = orchOf(install);
|
|
172
176
|
const b = require(path.join(scriptsOf(install), 'lib', 'project-prompt-builder.js'));
|
|
173
177
|
const prev = process.env.EPAM_BROWNFIELD; process.env.EPAM_BROWNFIELD = env.EPAM_BROWNFIELD === '1' ? '1' : '';
|
|
@@ -323,17 +327,75 @@ function checklist({ env, results }) {
|
|
|
323
327
|
return res('checklist', !red.length, `${items.filter((i) => i.state === 'GREEN').length} green, ${red.length} red, ${items.filter((i) => i.state === 'NOT-YET').length} not yet${red.length ? ` — red: ${red.map((i) => i.n).join(', ')}` : ''}`, { items });
|
|
324
328
|
}
|
|
325
329
|
|
|
330
|
+
/**
|
|
331
|
+
* NOTHING MINTED FOR A CODELINE IS GEARED TO THE TICKET (operator, 2026-10-24: "post pause 1 checkpoints to ensure no contamination in
|
|
332
|
+
* codeline level prompts and profiles"). After pause 1 the Agents phase has minted each codeline's roster and prompt set, which are kept
|
|
333
|
+
* and reused for every later story on that codeline. A judge agent (template set-fidelity-check, the given `judge` call) reads each
|
|
334
|
+
* codeline's roster and prompts against the ticket and quotes what is geared to it; anything quoted is RED, naming the codeline, the
|
|
335
|
+
* file and the words. Not run when the launch gives no judge (a £0 leg) — then NOT-YET, never GREEN by silence.
|
|
336
|
+
*/
|
|
337
|
+
const SET_FIDELITY_SCHEMA = { type: 'object', required: ['geared'], properties: { geared: { type: 'array', items: { type: 'object',
|
|
338
|
+
required: ['where', 'quote', 'why'], properties: { where: { type: 'string' }, quote: { type: 'string' }, why: { type: 'string' } } } } } };
|
|
339
|
+
async function setFidelity({ install, projectConfigDir, ticket, codelines = [], judge }) {
|
|
340
|
+
if (typeof judge !== 'function') return res('set-fidelity', false, 'not checked: no judge given — what was minted for each codeline is unexamined');
|
|
341
|
+
const b = require(path.join(scriptsOf(install), 'lib', 'project-prompt-builder.js'));
|
|
342
|
+
const { renderEngineTemplate } = require(path.join(scriptsOf(install), 'lib', 'engine-prompt.js'));
|
|
343
|
+
const rows = []; let examined = 0;
|
|
344
|
+
for (const codeline of codelines) {
|
|
345
|
+
const rp = b.recordedPromptsDir(projectConfigDir, codeline); const mi = b.mintedSnapshotDir(projectConfigDir, codeline);
|
|
346
|
+
const prompts = fs.existsSync(rp) ? fs.readdirSync(rp).filter((n) => n.endsWith('.json')).sort().map((n) => { const d = readJson(path.join(rp, n), {}); return `=== ${n} ===\n${d.body || Object.values(d.bodies || {}).join('\n')}`; }) : [];
|
|
347
|
+
let roster = '(none recorded)';
|
|
348
|
+
try {
|
|
349
|
+
const ag = readJson(path.join(mi, 'roster.json'), {}).agents || {}; const pr = (readJson(path.join(mi, 'agent-profiles.json'), {}).profiles) || {};
|
|
350
|
+
const names = Object.keys(ag);
|
|
351
|
+
if (names.length) roster = names.map((n) => `- ${n} [${(ag[n] || {}).kind || ''}]\n${String(pr[n] || (ag[n] || {}).persona || '(no brief)')}`).join('\n\n');
|
|
352
|
+
} catch { /* judged as recorded */ }
|
|
353
|
+
if (!prompts.length && roster === '(none recorded)') { rows.push({ codeline, examined: false, geared: [] }); continue; }
|
|
354
|
+
const prompt = renderEngineTemplate('set-fidelity-check', { __CODELINE__: codeline, __TICKET_KEY__: ticket.key || '', __TICKET_TITLE__: ticket.title || '',
|
|
355
|
+
__ROSTER__: roster, __PROMPTS__: prompts.join('\n\n') || '(none recorded)' });
|
|
356
|
+
const out = await judge({ prompt, schema: SET_FIDELITY_SCHEMA, codeline });
|
|
357
|
+
examined += 1;
|
|
358
|
+
rows.push({ codeline, examined: true, geared: (out && Array.isArray(out.geared)) ? out.geared : null });
|
|
359
|
+
}
|
|
360
|
+
const unjudged = rows.filter((r) => r.examined && r.geared === null).map((r) => r.codeline);
|
|
361
|
+
const geared = rows.filter((r) => r.examined && Array.isArray(r.geared) && r.geared.length);
|
|
362
|
+
const ok = examined > 0 && !unjudged.length && !geared.length;
|
|
363
|
+
const why = [geared.length && geared.map((r) => `${r.codeline}: ${r.geared.length} geared — ${r.geared.slice(0, 3).map((g) => `${g.where}: "${String(g.quote).slice(0, 80)}"`).join('; ')}`).join('; '),
|
|
364
|
+
unjudged.length && `no verdict for ${unjudged.join(', ')}`, !examined && 'nothing minted to examine'].filter(Boolean).join('; ');
|
|
365
|
+
return res('set-fidelity', ok, ok ? `${examined} codeline set(s) judged against ${ticket.key}: nothing geared to the ticket` : why, { rows });
|
|
366
|
+
}
|
|
367
|
+
|
|
368
|
+
/** The set-fidelity judge a LIVE launch gives setFidelity: the prompt-review seam's own model, ladder and tools (seam-invocation), through the
|
|
369
|
+
* pipeline's own agent call (spec-mode-runner.js runClaude), answering the check's schema. One definition — the launcher and the live test call it. */
|
|
370
|
+
function setFidelityJudge({ install, env = process.env, logDir }) {
|
|
371
|
+
const S = scriptsOf(install);
|
|
372
|
+
const spec = require(path.join(S, 'spec-mode-runner.js'));
|
|
373
|
+
const { seamInvocationEnv } = require(path.join(S, 'lib', 'seam-invocation.js'));
|
|
374
|
+
const promptExec = spec.resolvePromptExec(env.AI_RUNNER_CMD || path.join(S, 'ai-run.sh'));
|
|
375
|
+
return async ({ prompt, schema, codeline }) => {
|
|
376
|
+
const seamEnv = seamInvocationEnv('prompt-review', path.join(orchOf(install), 'agents'));
|
|
377
|
+
if (seamEnv.EPAM_ALLOWED_TOOLS) seamEnv.AI_GATE_ALLOW_TOOLS = '1';
|
|
378
|
+
// the call runs AS the prompt-review seam (llm-handler resolves the seam by agent name); the ledger names the check
|
|
379
|
+
seamEnv.EPAM_AGENT_NAME = 'prompt-review';
|
|
380
|
+
seamEnv.EPAM_RESPONSE_SCHEMA = spec.schemaEnv({ name: 'set_fidelity', description: 'what is geared to the ticket', parameters: schema });
|
|
381
|
+
const text = await spec.runClaude(promptExec, prompt, path.join(logDir || os.tmpdir(), `set-fidelity-${codeline}.log`), seamEnv, { costAgent: 'set-fidelity' });
|
|
382
|
+
const m = String(text || '').match(/\{[\s\S]*\}/);
|
|
383
|
+
try { return m ? JSON.parse(m[0]) : null; } catch { return null; }
|
|
384
|
+
};
|
|
385
|
+
}
|
|
386
|
+
|
|
326
387
|
/** The whole pre-launch, in order; every check runs (a report lists every red at once), the verdict is all-green. */
|
|
327
|
-
function prelaunch(o) {
|
|
388
|
+
async function prelaunch(o) {
|
|
328
389
|
const results = [];
|
|
329
390
|
if (o.preflight !== false) results.push(preflight(o));
|
|
330
391
|
results.push(langfuse({ install: o.install, env: o.env, runId: o.runId, outDir: o.outDir }));
|
|
331
|
-
if (o.ticket) results.push(seed({ install: o.install, project: o.project, ticket: o.ticket, runsRoots: o.runsRoots || [] }));
|
|
392
|
+
if (o.ticket) results.push(seed({ install: o.install, project: o.project, ticket: o.ticket, runsRoots: o.runsRoots || [], freshMint: !!o.freshMint }));
|
|
332
393
|
// the variant (brownfield) is the run's, from the env the launch hands over — not this process's own environment
|
|
333
394
|
const pcd = launchProject(o);
|
|
334
395
|
if (o.codeline) results.push(cache({ install: o.install, project: o.project, projectConfigDir: pcd, codeline: o.codeline, variant: (o.env || {}).EPAM_BROWNFIELD === '1' ? 'brownfield' : '', maxRebuild: o.maxRebuild || 0 }));
|
|
335
|
-
if (Array.isArray(o.codelines)) results.push(codelineSets({ install: o.install, project: o.project, projectConfigDir: pcd, codelines: o.codelines, env: o.env || {} }));
|
|
396
|
+
if (Array.isArray(o.codelines)) results.push(codelineSets({ install: o.install, project: o.project, projectConfigDir: pcd, codelines: o.codelines, env: o.env || {}, freshMint: !!o.freshMint }));
|
|
336
397
|
if (o.copy) results.push(permissions({ install: o.install, project: o.project, runDir: o.runDir, copy: o.copy }));
|
|
398
|
+
if (o.fidelity) results.push(await setFidelity({ install: o.install, projectConfigDir: pcd, ticket: o.fidelity.ticket, codelines: o.fidelity.codelines || o.codelines || [], judge: o.fidelity.judge }));
|
|
337
399
|
results.push(cost({ install: o.install, set: o.set || o.env.EPAM_PROVIDER_SET || 'claude', tmpDir: o.outDir }));
|
|
338
400
|
results.push(alone({ install: o.install }));
|
|
339
401
|
results.push(memory({ needMb: Number(o.needMb || 4096) }));
|
|
@@ -346,7 +408,7 @@ function prelaunch(o) {
|
|
|
346
408
|
return report;
|
|
347
409
|
}
|
|
348
410
|
|
|
349
|
-
module.exports = { prelaunch, prelaunchCli, preflight, langfuse, langfuseEnv, seed, cache, codelineSets, launchProject, permissions, cost, ladderModels, alone, memory, noTimeout, checklist, ancestors };
|
|
411
|
+
module.exports = { prelaunch, prelaunchCli, preflight, langfuse, langfuseEnv, seed, cache, codelineSets, launchProject, setFidelity, setFidelityJudge, permissions, cost, ladderModels, alone, memory, noTimeout, checklist, ancestors };
|
|
350
412
|
|
|
351
413
|
/**
|
|
352
414
|
* THE COMMAND, for every pipeline:
|
|
@@ -924,18 +924,23 @@ async function buildProjectPrompts({
|
|
|
924
924
|
// above exists so a corrected generator or reviewer reaches a resumed run; a copy whose template, generator and
|
|
925
925
|
// reviewer are the ones that approved it has nothing new to be judged by. A copy that predates the record is
|
|
926
926
|
// kept as built — re-judging it is the full review this replaced.
|
|
927
|
-
|
|
927
|
+
// A COPY NO REVIEWER IS RECORDED TO HAVE JUDGED IS JUDGED NOW (operator, 2026-10-24: "That is not permitted"). The exception
|
|
928
|
+
// below this one — "a copy that predates the record is kept as built" — let run 13's 128 oldest copies through every refresh
|
|
929
|
+
// unexamined, and they carried the ticket they were minted for into reusable prompts. Only a copy the CURRENT judges approved
|
|
930
|
+
// is kept without a call; one they approve now is recorded as theirs, so it is not asked again.
|
|
931
|
+
if (_kept.judgedWith && _kept.judgedWith === _judges()) {
|
|
928
932
|
built.push(id);
|
|
929
|
-
log(`[prompt-builder] kept ${id} (installed for this run) —
|
|
930
|
-
? 'its template, generator and reviewer are the ones that approved it' : 'no record of what judged it; kept as built'}`);
|
|
933
|
+
log(`[prompt-builder] kept ${id} (installed for this run) — its template, generator and reviewer are the ones that approved it`);
|
|
931
934
|
continue;
|
|
932
935
|
}
|
|
933
936
|
const _v = await reviewPrompt({ id, template: readJson(_tplPath(id)), generated: _kept });
|
|
934
937
|
if (!_v || _v.ok !== false) {
|
|
935
938
|
built.push(id);
|
|
936
|
-
|
|
937
|
-
|
|
938
|
-
|
|
939
|
+
const _judgedNow = !(_v && _v.reviewed === false);
|
|
940
|
+
if (_judgedNow) { _kept.judgedWith = _judges(); fs.writeFileSync(path.join(outDir, `${id}.json`), JSON.stringify(_kept, null, 2) + '\n'); }
|
|
941
|
+
log(`[prompt-builder] kept ${id} (installed for this run) — ${_judgedNow
|
|
942
|
+
? `judged ${_kept.judgedWith && !_judgedNow ? 'again ' : ''}by the current reviewer and approved`
|
|
943
|
+
: `unreviewed: the reviewer gave no verdict (${_v.reason || 'no reason'})`}`);
|
|
939
944
|
continue;
|
|
940
945
|
}
|
|
941
946
|
_keptRejected = true;
|
|
@@ -4470,13 +4470,29 @@ const TOOL_ROSTER_REVIEW = {
|
|
|
4470
4470
|
'checked with your tools. Do not answer in prose.',
|
|
4471
4471
|
parameters: {
|
|
4472
4472
|
type: 'object',
|
|
4473
|
-
required: ['verdict', 'findings'],
|
|
4473
|
+
required: ['verdict', 'findings', 'roles'],
|
|
4474
4474
|
properties: {
|
|
4475
4475
|
verdict: {
|
|
4476
4476
|
type: 'string',
|
|
4477
4477
|
enum: ['sound', 'defects_found', 'nothing_to_review'],
|
|
4478
4478
|
description: 'sound = every checkable claim held. defects_found = at least one did not.',
|
|
4479
4479
|
},
|
|
4480
|
+
// A ROLE OUTLIVES THE TICKET (operator, 2026-10-24): the roster is kept with its codelines and reused for every later story, so
|
|
4481
|
+
// every minted role is answered for, one by one — the rule in prose alone was answered "sound" on a roster whose every agent was
|
|
4482
|
+
// named after the ticket. A role that does not outlive the ticket becomes a blocking finding (rolesOutliveTicketFindings).
|
|
4483
|
+
roles: {
|
|
4484
|
+
type: 'array',
|
|
4485
|
+
description: 'One entry for EVERY minted role reviewed.',
|
|
4486
|
+
items: {
|
|
4487
|
+
type: 'object',
|
|
4488
|
+
required: ['name', 'outlivesTicket', 'why'],
|
|
4489
|
+
properties: {
|
|
4490
|
+
name: { type: 'string', description: 'The minted role, named exactly as in the roster.' },
|
|
4491
|
+
outlivesTicket: { type: 'boolean', description: 'true only when this role — its name and its brief — would serve a story on this codeline about something OTHER than the current ticket: named for the kind of engineer/investigator and the codeline, briefed on the codeline, not on this ticket\'s subject, components or files (except as marked examples).' },
|
|
4492
|
+
why: { type: 'string', description: 'Quote the words that gear the role to the ticket, or say what makes it the codeline\'s.' },
|
|
4493
|
+
},
|
|
4494
|
+
},
|
|
4495
|
+
},
|
|
4480
4496
|
findings: {
|
|
4481
4497
|
type: 'array',
|
|
4482
4498
|
items: {
|
|
@@ -4885,6 +4901,19 @@ function _readJsonOrNull(f) {
|
|
|
4885
4901
|
* Extracted and exported because it could not be tested otherwise: it sat inside the async function
|
|
4886
4902
|
* that also does the batching and makes the model calls, so exercising it meant paying for a run.
|
|
4887
4903
|
*/
|
|
4904
|
+
/** A ROLE THAT DOES NOT OUTLIVE THE TICKET IS A BLOCKING FINDING (operator, 2026-10-24): the review answers for every minted role
|
|
4905
|
+
* (TOOL_ROSTER_REVIEW.roles); one it marks outlivesTicket:false — named or briefed for the ticket in scope rather than the codeline —
|
|
4906
|
+
* is turned into the finding the mint's correction loop acts on. The reviewer's own words are the claim. */
|
|
4907
|
+
function rolesOutliveTicketFindings(part) {
|
|
4908
|
+
const roles = part && Array.isArray(part.roles) ? part.roles : [];
|
|
4909
|
+
return roles.filter((r) => r && r.name && r.outlivesTicket === false).map((r) => ({
|
|
4910
|
+
agent: String(r.name), severity: 'blocking', claim: String(r.why || '(the reviewer gave no words)'),
|
|
4911
|
+
checked: 'whether this role, as named and briefed, would serve a story on its codeline about something other than the current ticket',
|
|
4912
|
+
found: 'minted for the ticket in scope, not for the codeline it is kept with',
|
|
4913
|
+
remedy: 'a role named for its kind of work and its codeline, briefed on the codeline — its stack, conventions and how to verify there — naming the current ticket only as a marked example',
|
|
4914
|
+
}));
|
|
4915
|
+
}
|
|
4916
|
+
|
|
4888
4917
|
function aggregateRosterReview(results, legalVerdicts) {
|
|
4889
4918
|
const legal = Array.isArray(legalVerdicts) ? legalVerdicts : [];
|
|
4890
4919
|
const rows = Array.isArray(results) ? results : [];
|
|
@@ -4892,7 +4921,9 @@ function aggregateRosterReview(results, legalVerdicts) {
|
|
|
4892
4921
|
const unexamined = rows.filter((r) => !r || !r.part || typeof r.part !== 'object'
|
|
4893
4922
|
|| r.part.verdict === 'nothing_to_review' || !r.part.verdict || illegal(r.part.verdict));
|
|
4894
4923
|
const examined = rows.filter((r) => !unexamined.includes(r));
|
|
4895
|
-
|
|
4924
|
+
// a role an examined batch says does not outlive the ticket is a finding of that batch, whatever verdict it gave
|
|
4925
|
+
const roleFindings = examined.flatMap((r) => rolesOutliveTicketFindings(r.part));
|
|
4926
|
+
const findings = rows.flatMap((r) => (r && r.part && Array.isArray(r.part.findings) ? r.part.findings : [])).concat(roleFindings);
|
|
4896
4927
|
|
|
4897
4928
|
if (!examined.length) {
|
|
4898
4929
|
return {
|
|
@@ -4903,7 +4934,7 @@ function aggregateRosterReview(results, legalVerdicts) {
|
|
|
4903
4934
|
};
|
|
4904
4935
|
}
|
|
4905
4936
|
|
|
4906
|
-
const verdict = examined.some((r) => r.part.verdict === 'defects_found') ? 'defects_found' : 'sound';
|
|
4937
|
+
const verdict = (examined.some((r) => r.part.verdict === 'defects_found') || roleFindings.length) ? 'defects_found' : 'sound';
|
|
4907
4938
|
return {
|
|
4908
4939
|
verdict,
|
|
4909
4940
|
findings,
|
|
@@ -11107,7 +11138,7 @@ module.exports = {
|
|
|
11107
11138
|
storyAcsBlock,
|
|
11108
11139
|
specAgentContract,
|
|
11109
11140
|
rosterReviewVerdict,
|
|
11110
|
-
aggregateRosterReview,
|
|
11141
|
+
aggregateRosterReview, rolesOutliveTicketFindings,
|
|
11111
11142
|
reviewOutcomeKeepsChange,
|
|
11112
11143
|
// Exported so the brief lookup can be asserted without a run.
|
|
11113
11144
|
profilesWithProjectBriefs,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "amsd-pipeline-shared",
|
|
3
|
-
"version": "0.5.
|
|
3
|
+
"version": "0.5.39",
|
|
4
4
|
"description": "The AMSD pipeline's shared components, one source for the bash pipeline and its Claude workflows: the model-call gateway (llm-handler.sh — retry, ladder, self-heal, cost) and the libraries it runs.",
|
|
5
5
|
"license": "UNLICENSED",
|
|
6
6
|
"bin": {
|