@sun-asterisk/sungen 3.2.1-beta.1 → 3.2.2-beta.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli/commands/delivery.d.ts.map +1 -1
- package/dist/cli/commands/delivery.js +31 -0
- package/dist/cli/commands/delivery.js.map +1 -1
- package/dist/cli/commands/depth-lint.d.ts +3 -0
- package/dist/cli/commands/depth-lint.d.ts.map +1 -0
- package/dist/cli/commands/depth-lint.js +88 -0
- package/dist/cli/commands/depth-lint.js.map +1 -0
- package/dist/cli/commands/gate.d.ts +3 -0
- package/dist/cli/commands/gate.d.ts.map +1 -0
- package/dist/cli/commands/gate.js +83 -0
- package/dist/cli/commands/gate.js.map +1 -0
- package/dist/cli/commands/journey.d.ts +3 -0
- package/dist/cli/commands/journey.d.ts.map +1 -0
- package/dist/cli/commands/journey.js +96 -0
- package/dist/cli/commands/journey.js.map +1 -0
- package/dist/cli/index.js +6 -0
- package/dist/cli/index.js.map +1 -1
- package/dist/exporters/feature-parser.d.ts +25 -0
- package/dist/exporters/feature-parser.d.ts.map +1 -1
- package/dist/exporters/feature-parser.js +59 -0
- package/dist/exporters/feature-parser.js.map +1 -1
- package/dist/exporters/types.d.ts +38 -0
- package/dist/exporters/types.d.ts.map +1 -1
- package/dist/exporters/xlsx-exporter.d.ts +31 -2
- package/dist/exporters/xlsx-exporter.d.ts.map +1 -1
- package/dist/exporters/xlsx-exporter.js +144 -1
- package/dist/exporters/xlsx-exporter.js.map +1 -1
- package/dist/harness/depth-lint.d.ts +25 -0
- package/dist/harness/depth-lint.d.ts.map +1 -0
- package/dist/harness/depth-lint.js +118 -0
- package/dist/harness/depth-lint.js.map +1 -0
- package/dist/harness/journey.d.ts +68 -0
- package/dist/harness/journey.d.ts.map +1 -0
- package/dist/harness/journey.js +328 -0
- package/dist/harness/journey.js.map +1 -0
- package/dist/harness/parse.d.ts.map +1 -1
- package/dist/harness/parse.js +4 -1
- package/dist/harness/parse.js.map +1 -1
- package/dist/orchestrator/ai-rules-updater.d.ts.map +1 -1
- package/dist/orchestrator/ai-rules-updater.js +1 -0
- package/dist/orchestrator/ai-rules-updater.js.map +1 -1
- package/dist/orchestrator/templates/ai-instructions/claude-agent-generator.md +44 -0
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +24 -1
- package/dist/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +3 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-delivery.md +27 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +22 -3
- package/dist/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +2 -0
- package/dist/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +3 -0
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +4 -1
- package/dist/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +3 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +27 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +22 -3
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +2 -0
- package/dist/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +3 -0
- package/dist/orchestrator/templates/specs-api.d.ts +7 -0
- package/dist/orchestrator/templates/specs-api.d.ts.map +1 -1
- package/dist/orchestrator/templates/specs-api.js +13 -2
- package/dist/orchestrator/templates/specs-api.js.map +1 -1
- package/dist/orchestrator/templates/specs-api.ts +13 -2
- package/package.json +3 -3
- package/src/cli/commands/delivery.ts +32 -2
- package/src/cli/commands/depth-lint.ts +51 -0
- package/src/cli/commands/gate.ts +44 -0
- package/src/cli/commands/journey.ts +59 -0
- package/src/cli/index.ts +6 -0
- package/src/exporters/feature-parser.ts +57 -0
- package/src/exporters/types.ts +38 -0
- package/src/exporters/xlsx-exporter.ts +176 -2
- package/src/harness/depth-lint.ts +122 -0
- package/src/harness/journey.ts +333 -0
- package/src/harness/parse.ts +4 -1
- package/src/orchestrator/ai-rules-updater.ts +1 -0
- package/src/orchestrator/templates/ai-instructions/claude-agent-generator.md +44 -0
- package/src/orchestrator/templates/ai-instructions/claude-cmd-create-test.md +24 -1
- package/src/orchestrator/templates/ai-instructions/claude-cmd-run-test.md +3 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-delivery.md +27 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-error-mapping.md +22 -3
- package/src/orchestrator/templates/ai-instructions/claude-skill-gherkin-syntax.md +2 -0
- package/src/orchestrator/templates/ai-instructions/claude-skill-tc-generation.md +3 -0
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-create-test.md +4 -1
- package/src/orchestrator/templates/ai-instructions/copilot-cmd-run-test.md +3 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-delivery.md +27 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-error-mapping.md +22 -3
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-gherkin-syntax.md +2 -0
- package/src/orchestrator/templates/ai-instructions/github-skill-sungen-tc-generation.md +3 -0
- package/src/orchestrator/templates/specs-api.ts +13 -2
|
@@ -0,0 +1,333 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Journey board (epic #381, story S1) — the durable, read-only "you are here" view.
|
|
3
|
+
*
|
|
4
|
+
* Reconstructs the QA's position from artifacts ALREADY on disk (no recomputation, no context
|
|
5
|
+
* needed): the audit report (`.sungen/reports/<slug>-audit.json`) supplies the obligation status
|
|
6
|
+
* via its calibration axes + gate + findings; the ledger (`.sungen/ledger/<slug>.jsonl`) supplies
|
|
7
|
+
* the phase history ("you are here"). The output answers the three QA questions — what's next /
|
|
8
|
+
* what to review / what's doubtful — and persists `.sungen/journey/<slug>.{json,board.md}`.
|
|
9
|
+
*
|
|
10
|
+
* S1 = the read-only synthesis. S2 (this file) adds the **writable lifecycle**: persisted
|
|
11
|
+
* waivers (reason-required, anti-amnesia), reconcile (auto-close satisfied; re-surface a waiver
|
|
12
|
+
* when its evidence changed), via `runJourney` + `waive`. Gate-bound predicates + inter-phase
|
|
13
|
+
* gates are S3. Pure-deterministic, no AI.
|
|
14
|
+
*/
|
|
15
|
+
import * as fs from 'fs';
|
|
16
|
+
import * as path from 'path';
|
|
17
|
+
import * as crypto from 'crypto';
|
|
18
|
+
import { reportSlug } from './unit-paths';
|
|
19
|
+
|
|
20
|
+
export type ObStatus = 'satisfied' | 'needs-work' | 'pending' | 'waived';
|
|
21
|
+
|
|
22
|
+
export interface Obligation {
|
|
23
|
+
id: string;
|
|
24
|
+
title: string;
|
|
25
|
+
status: ObStatus;
|
|
26
|
+
detail: string;
|
|
27
|
+
waivedReason?: string; // S2 — set when the QA explicitly waived this obligation
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
/** S5 — a parsed anchor so an IDE can jump straight to what needs review. */
|
|
31
|
+
export interface Anchor { vpId?: string; file?: string; line?: number; }
|
|
32
|
+
export interface ReviewItem { text: string; anchor: Anchor; }
|
|
33
|
+
|
|
34
|
+
export interface JourneyReport {
|
|
35
|
+
unit: string;
|
|
36
|
+
generatedFrom: string[]; // which artifacts were available
|
|
37
|
+
youAreHere: string; // last ledger phase, or 'not started'
|
|
38
|
+
phasesDone: string[];
|
|
39
|
+
gateStatus: string | null;
|
|
40
|
+
score: number | null;
|
|
41
|
+
obligations: Obligation[];
|
|
42
|
+
needsYou: ReviewItem[]; // S5 — the review queue: human-judgment findings, each with an anchor
|
|
43
|
+
nextSuggested: string;
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
function readJSON(p: string): any {
|
|
47
|
+
try { return fs.existsSync(p) ? JSON.parse(fs.readFileSync(p, 'utf-8')) : null; } catch { return null; }
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
function readLedgerPhases(p: string): string[] {
|
|
51
|
+
if (!fs.existsSync(p)) return [];
|
|
52
|
+
const out: string[] = [];
|
|
53
|
+
for (const line of fs.readFileSync(p, 'utf-8').split('\n')) {
|
|
54
|
+
if (!line.trim()) continue;
|
|
55
|
+
try { const d = JSON.parse(line); if (d.step) out.push(String(d.step)); } catch { /* skip */ }
|
|
56
|
+
}
|
|
57
|
+
return out;
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
/** Findings that ask for HUMAN judgment (the review-queue seed), vs pure machine-repair targets. */
|
|
61
|
+
function isHumanFinding(f: string): boolean {
|
|
62
|
+
return /@manual|MANUAL-|DEPTH-DEFERRED|UNSOURCEABLE|CAPABILITY-SUGGESTION|judgment|oracle|review/i.test(f);
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
/** S5 — pull an IDE-jumpable anchor out of a finding (VP id, scenario, or `file.ext:line`). */
|
|
66
|
+
function extractAnchor(text: string): Anchor {
|
|
67
|
+
const vp = text.match(/\bVP-[A-Z0-9-]+\b/);
|
|
68
|
+
const fl = text.match(/([\w./-]+\.(?:md|feature|ts|yaml)):(\d+)/);
|
|
69
|
+
return { vpId: vp ? vp[0] : undefined, file: fl ? fl[1] : undefined, line: fl ? Number(fl[2]) : undefined };
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
const SAT = 0.8; // axis at/above this = satisfied (below = needs-work)
|
|
73
|
+
|
|
74
|
+
function computeFresh(projectRoot: string, unit: string): JourneyReport {
|
|
75
|
+
const slug = reportSlug(unit);
|
|
76
|
+
const audit = readJSON(path.join(projectRoot, '.sungen', 'reports', `${slug}-audit.json`));
|
|
77
|
+
const phases = readLedgerPhases(path.join(projectRoot, '.sungen', 'ledger', `${slug}.jsonl`));
|
|
78
|
+
|
|
79
|
+
const from: string[] = [];
|
|
80
|
+
if (audit) from.push('audit');
|
|
81
|
+
if (phases.length) from.push('ledger');
|
|
82
|
+
|
|
83
|
+
const youAreHere = phases.length ? phases[phases.length - 1] : 'not started';
|
|
84
|
+
const ran = (p: string) => phases.some((s) => s === p || s.startsWith(p));
|
|
85
|
+
|
|
86
|
+
const obligations: Obligation[] = [];
|
|
87
|
+
const needsYou: ReviewItem[] = [];
|
|
88
|
+
|
|
89
|
+
if (!audit) {
|
|
90
|
+
// Nothing measured yet — the journey hasn't really begun.
|
|
91
|
+
obligations.push({ id: 'OB-create', title: 'Generate test cases', status: 'pending', detail: 'No audit report yet — run /sungen:create-test.' });
|
|
92
|
+
return {
|
|
93
|
+
unit, generatedFrom: from, youAreHere, phasesDone: phases, gateStatus: null, score: null,
|
|
94
|
+
obligations, needsYou, nextSuggested: 'Run `/sungen:create-test ' + unit + '` to begin.',
|
|
95
|
+
};
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
const ax = (audit.calibration && audit.calibration.axes) || {};
|
|
99
|
+
const depthThreshold = (audit.depth && audit.depth.threshold) || SAT;
|
|
100
|
+
const ob = (id: string, title: string, val: number | undefined, thr: number, detail: string): Obligation => ({
|
|
101
|
+
id, title,
|
|
102
|
+
status: val === undefined ? 'pending' : val >= thr ? 'satisfied' : 'needs-work',
|
|
103
|
+
detail: val === undefined ? detail : `${Math.round(val * 100)}% (need ${Math.round(thr * 100)}%) — ${detail}`,
|
|
104
|
+
});
|
|
105
|
+
|
|
106
|
+
obligations.push(ob('OB-spec', 'Spec readiness', ax.specFR, 1, 'FR/sections sufficient to design from'));
|
|
107
|
+
obligations.push(ob('OB-coverage', 'Viewpoint coverage', ax.coverage, SAT, 'critical themes covered'));
|
|
108
|
+
obligations.push(ob('OB-depth', 'Assertion depth', ax.businessDepth, depthThreshold, 'business-critical scenarios assert data'));
|
|
109
|
+
obligations.push(ob('OB-trace', 'Traceability', ax.taxonomy ?? ax.traceability, SAT, 'scenarios trace to viewpoints'));
|
|
110
|
+
|
|
111
|
+
// Automation obligation: automatable @manual still pending a driver = needs-work.
|
|
112
|
+
const autoPending = audit.automatableManual && audit.automatableManual.automatable > 0;
|
|
113
|
+
obligations.push({
|
|
114
|
+
id: 'OB-automation', title: 'Automation coverage',
|
|
115
|
+
status: autoPending ? 'needs-work' : 'satisfied',
|
|
116
|
+
detail: autoPending ? `${audit.automatableManual.automatable} @manual scenario(s) a driver could automate` : 'no capability-pending automatable cases',
|
|
117
|
+
});
|
|
118
|
+
|
|
119
|
+
// Human sign-off: always pending until S5 explicit signoff; carries the review-queue count.
|
|
120
|
+
const manualCount = (audit.manualOracle && audit.manualOracle.total) || 0;
|
|
121
|
+
obligations.push({
|
|
122
|
+
id: 'OB-signoff', title: 'Human sign-off', status: 'pending',
|
|
123
|
+
detail: `${needsYou.length} item(s) queued for your review (single sign-off at the end)`,
|
|
124
|
+
});
|
|
125
|
+
|
|
126
|
+
// Needs-you: human-judgment findings (the review-queue seed), with their anchors (titles) intact.
|
|
127
|
+
for (const f of (audit.findings || [])) if (isHumanFinding(f)) needsYou.push({ text: f, anchor: extractAnchor(f) });
|
|
128
|
+
// Re-stamp the signoff detail now that needsYou is filled.
|
|
129
|
+
const signoff = obligations.find((o) => o.id === 'OB-signoff');
|
|
130
|
+
if (signoff) signoff.detail = `${needsYou.length} item(s) queued for your review · ${manualCount} @manual scenario(s)`;
|
|
131
|
+
|
|
132
|
+
// Next suggested = first non-satisfied obligation; if all satisfied but run-test not done → run.
|
|
133
|
+
const firstGap = obligations.find((o) => o.status !== 'satisfied' && o.id !== 'OB-signoff');
|
|
134
|
+
let nextSuggested: string;
|
|
135
|
+
if (firstGap) nextSuggested = `Repair "${firstGap.title}" (${firstGap.detail}).`;
|
|
136
|
+
else if (!ran('run')) nextSuggested = `Quality satisfied — run \`/sungen:run-test ${unit}\`.`;
|
|
137
|
+
else nextSuggested = `All obligations satisfied — review the ${needsYou.length} queued item(s), then sign off & deliver.`;
|
|
138
|
+
|
|
139
|
+
return {
|
|
140
|
+
unit, generatedFrom: from, youAreHere, phasesDone: phases,
|
|
141
|
+
gateStatus: audit.gateStatus ?? null, score: (audit.score && audit.score.overall) ?? null,
|
|
142
|
+
obligations, needsYou, nextSuggested,
|
|
143
|
+
};
|
|
144
|
+
}
|
|
145
|
+
|
|
146
|
+
// ---------------- S2: writable lifecycle — persisted waivers + reconcile ----------------
|
|
147
|
+
|
|
148
|
+
interface Waiver { reason: string; at: string; auditHashAtWaive: string; }
|
|
149
|
+
interface Signoff { by?: string; at: string; auditHashAtSignoff: string; } // S5
|
|
150
|
+
interface JourneyState { unit: string; auditHash: string; waivers: Record<string, Waiver>; signoff?: Signoff; }
|
|
151
|
+
|
|
152
|
+
function statePath(projectRoot: string, slug: string): string {
|
|
153
|
+
return path.join(projectRoot, '.sungen', 'journey', `${slug}.state.json`);
|
|
154
|
+
}
|
|
155
|
+
/** Evidence cursor: the audit report's content hash. A waiver is invalidated when this changes. */
|
|
156
|
+
function auditHashOf(projectRoot: string, slug: string): string {
|
|
157
|
+
const p = path.join(projectRoot, '.sungen', 'reports', `${slug}-audit.json`);
|
|
158
|
+
return fs.existsSync(p) ? crypto.createHash('sha256').update(fs.readFileSync(p)).digest('hex') : '';
|
|
159
|
+
}
|
|
160
|
+
function loadState(p: string): JourneyState | null { return readJSON(p); }
|
|
161
|
+
function saveState(p: string, s: JourneyState): void {
|
|
162
|
+
fs.mkdirSync(path.dirname(p), { recursive: true });
|
|
163
|
+
fs.writeFileSync(p, JSON.stringify(s, null, 2), 'utf-8');
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
/** Recompute nextSuggested AFTER waivers are applied (a waived obligation is not a gap). */
|
|
167
|
+
function computeNext(r: JourneyReport, unit: string): string {
|
|
168
|
+
const gap = r.obligations.find((o) => o.status !== 'satisfied' && o.status !== 'waived' && o.id !== 'OB-signoff');
|
|
169
|
+
if (gap) return `Repair "${gap.title}" (${gap.detail}).`;
|
|
170
|
+
if (!r.phasesDone.some((s) => s === 'run' || s.startsWith('run'))) return `Quality satisfied — run \`/sungen:run-test ${unit}\`.`;
|
|
171
|
+
return `All obligations satisfied/waived — review the ${r.needsYou.length} queued item(s), then sign off & deliver.`;
|
|
172
|
+
}
|
|
173
|
+
|
|
174
|
+
/**
|
|
175
|
+
* The public entry: compute fresh, then RECONCILE with the persisted state —
|
|
176
|
+
* - auto-close is automatic (fresh recompute reflects the current artifacts);
|
|
177
|
+
* - an active waiver (evidence unchanged) sets status='waived' (carries the reason);
|
|
178
|
+
* - a STALE waiver (audit changed since it was waived) is re-surfaced for re-decision (anti-amnesia).
|
|
179
|
+
* Then persist the current evidence cursor.
|
|
180
|
+
*/
|
|
181
|
+
export function runJourney(projectRoot: string, unit: string): JourneyReport {
|
|
182
|
+
const slug = reportSlug(unit);
|
|
183
|
+
const report = computeFresh(projectRoot, unit);
|
|
184
|
+
const sp = statePath(projectRoot, slug);
|
|
185
|
+
const state = loadState(sp) || { unit, auditHash: '', waivers: {} };
|
|
186
|
+
const curHash = auditHashOf(projectRoot, slug);
|
|
187
|
+
|
|
188
|
+
for (const ob of report.obligations) {
|
|
189
|
+
const w = state.waivers[ob.id];
|
|
190
|
+
if (!w) continue;
|
|
191
|
+
if (w.auditHashAtWaive === curHash) {
|
|
192
|
+
ob.status = 'waived';
|
|
193
|
+
ob.waivedReason = w.reason;
|
|
194
|
+
ob.detail = `waived — ${w.reason}`;
|
|
195
|
+
} else {
|
|
196
|
+
report.needsYou.unshift({ text: `⚠️ Waiver on "${ob.title}" is STALE (evidence changed since ${w.at}) — re-decide. Was: ${w.reason}`, anchor: { vpId: ob.id } });
|
|
197
|
+
}
|
|
198
|
+
}
|
|
199
|
+
// S5 — sign-off reconcile: a recorded sign-off satisfies OB-signoff while its evidence holds;
|
|
200
|
+
// if the audit changed since, the sign-off is STALE → re-surface for re-review (anti-amnesia).
|
|
201
|
+
const signoffOb = report.obligations.find((o) => o.id === 'OB-signoff');
|
|
202
|
+
if (signoffOb && state.signoff) {
|
|
203
|
+
if (state.signoff.auditHashAtSignoff === curHash) {
|
|
204
|
+
signoffOb.status = 'satisfied';
|
|
205
|
+
signoffOb.detail = `signed off${state.signoff.by ? ' by ' + state.signoff.by : ''} at ${state.signoff.at}`;
|
|
206
|
+
} else {
|
|
207
|
+
report.needsYou.unshift({ text: `⚠️ Sign-off is STALE (suite changed since ${state.signoff.at}) — re-review and sign off again.`, anchor: { vpId: 'OB-signoff' } });
|
|
208
|
+
}
|
|
209
|
+
}
|
|
210
|
+
report.nextSuggested = computeNext(report, unit);
|
|
211
|
+
saveState(sp, { unit, auditHash: curHash, waivers: state.waivers, signoff: state.signoff });
|
|
212
|
+
return report;
|
|
213
|
+
}
|
|
214
|
+
|
|
215
|
+
/**
|
|
216
|
+
* Waive an obligation — REQUIRES a reason (anti-amnesia: a waiver leaves a recorded "why").
|
|
217
|
+
* Records the current evidence cursor so reconcile can invalidate it if the audit changes.
|
|
218
|
+
*/
|
|
219
|
+
export function waive(projectRoot: string, unit: string, obId: string, reason: string): JourneyReport {
|
|
220
|
+
if (!reason || !reason.trim()) {
|
|
221
|
+
throw new Error('A reason is required to waive (anti-amnesia: a waiver must record WHY). Use --reason "...".');
|
|
222
|
+
}
|
|
223
|
+
const slug = reportSlug(unit);
|
|
224
|
+
const fresh = computeFresh(projectRoot, unit);
|
|
225
|
+
const valid = fresh.obligations.map((o) => o.id);
|
|
226
|
+
if (!valid.includes(obId)) {
|
|
227
|
+
throw new Error(`Unknown obligation "${obId}". Valid: ${valid.join(', ')}`);
|
|
228
|
+
}
|
|
229
|
+
const sp = statePath(projectRoot, slug);
|
|
230
|
+
const state = loadState(sp) || { unit, auditHash: '', waivers: {} };
|
|
231
|
+
state.waivers[obId] = { reason: reason.trim(), at: new Date().toISOString(), auditHashAtWaive: auditHashOf(projectRoot, slug) };
|
|
232
|
+
saveState(sp, state);
|
|
233
|
+
return runJourney(projectRoot, unit);
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
/**
|
|
237
|
+
* Sign off — the single human gate (S5). Allowed ONLY when every other deliver-required
|
|
238
|
+
* obligation is satisfied or waived (you cannot sign off an unclean suite). Records who/when +
|
|
239
|
+
* the evidence cursor; reconcile invalidates it (stale → re-review) if the suite later changes.
|
|
240
|
+
*/
|
|
241
|
+
export function signoff(projectRoot: string, unit: string, by?: string): JourneyReport {
|
|
242
|
+
const before = runJourney(projectRoot, unit);
|
|
243
|
+
const blockers = before.obligations.filter((o) => o.id !== 'OB-signoff' && o.status !== 'satisfied' && o.status !== 'waived');
|
|
244
|
+
if (blockers.length) {
|
|
245
|
+
throw new Error(`Cannot sign off — still open: ${blockers.map((b) => b.id).join(', ')}. Fix, or waive each with a reason, first.`);
|
|
246
|
+
}
|
|
247
|
+
const slug = reportSlug(unit);
|
|
248
|
+
const sp = statePath(projectRoot, slug);
|
|
249
|
+
const state = loadState(sp) || { unit, auditHash: '', waivers: {} };
|
|
250
|
+
state.signoff = { by, at: new Date().toISOString(), auditHashAtSignoff: auditHashOf(projectRoot, slug) };
|
|
251
|
+
saveState(sp, state);
|
|
252
|
+
return runJourney(projectRoot, unit);
|
|
253
|
+
}
|
|
254
|
+
|
|
255
|
+
const ICON: Record<ObStatus, string> = { satisfied: '✅', 'needs-work': '⚠️ ', pending: '⏳', waived: '🚫' };
|
|
256
|
+
|
|
257
|
+
export function renderJourneyBoard(r: JourneyReport): string {
|
|
258
|
+
const L: string[] = [];
|
|
259
|
+
L.push(`# Journey — ${r.unit}`);
|
|
260
|
+
L.push('');
|
|
261
|
+
L.push(`📍 **You are here:** ${r.youAreHere}` + (r.phasesDone.length ? ` (phases: ${r.phasesDone.join(' → ')})` : ''));
|
|
262
|
+
if (r.gateStatus) L.push(` gate: **${r.gateStatus}** · score: ${r.score ?? '—'}/10`);
|
|
263
|
+
L.push('');
|
|
264
|
+
L.push('## What must still be true');
|
|
265
|
+
for (const o of r.obligations) L.push(`- ${ICON[o.status]} **${o.title}** — ${o.detail}`);
|
|
266
|
+
L.push('');
|
|
267
|
+
L.push(`## 🔎 Review queue — needs your judgment (${r.needsYou.length})`);
|
|
268
|
+
if (r.needsYou.length) {
|
|
269
|
+
for (const it of r.needsYou.slice(0, 20)) {
|
|
270
|
+
const a = it.anchor;
|
|
271
|
+
const tag = [a.vpId, a.file && `${a.file}${a.line ? ':' + a.line : ''}`].filter(Boolean).join(' · ');
|
|
272
|
+
L.push(`- ${it.text}${tag ? ` 〔${tag}〕` : ''}`);
|
|
273
|
+
}
|
|
274
|
+
} else L.push('- (nothing queued)');
|
|
275
|
+
L.push('');
|
|
276
|
+
L.push(`## → Next`);
|
|
277
|
+
L.push(`${r.nextSuggested}`);
|
|
278
|
+
L.push('');
|
|
279
|
+
return L.join('\n');
|
|
280
|
+
}
|
|
281
|
+
|
|
282
|
+
// ---------------- S3: inter-phase gate — obligations as HALT predicates (#398) ----------------
|
|
283
|
+
//
|
|
284
|
+
// A phase boundary is a deterministic gate: before the next phase may run, this phase's required
|
|
285
|
+
// obligations must each be `satisfied` OR `waived` (S2 — an explicit, reasoned human acceptance).
|
|
286
|
+
// A required obligation still `needs-work`/`pending` is a BLOCKER → HALT (no silent bad output
|
|
287
|
+
// crosses the boundary, §9). Reuses runJourney → obligations already reflect waivers/reconcile.
|
|
288
|
+
|
|
289
|
+
export type GatePhase = 'create' | 'run' | 'deliver';
|
|
290
|
+
|
|
291
|
+
export interface GateVerdict {
|
|
292
|
+
unit: string;
|
|
293
|
+
phase: GatePhase;
|
|
294
|
+
status: 'pass' | 'halt';
|
|
295
|
+
required: string[];
|
|
296
|
+
blockers: { id: string; title: string; detail: string }[];
|
|
297
|
+
waivedCredit: { id: string; title: string }[]; // required obligations accepted via an explicit waiver
|
|
298
|
+
}
|
|
299
|
+
|
|
300
|
+
const PHASE_REQUIRED: Record<GatePhase, string[]> = {
|
|
301
|
+
// post-create (design quality): spec + coverage + depth + traceability must hold.
|
|
302
|
+
create: ['OB-spec', 'OB-coverage', 'OB-depth', 'OB-trace'],
|
|
303
|
+
// post-run: the design gates + automation coverage.
|
|
304
|
+
run: ['OB-spec', 'OB-coverage', 'OB-depth', 'OB-trace', 'OB-automation'],
|
|
305
|
+
// pre-delivery: everything automated PLUS the single human sign-off (S5).
|
|
306
|
+
deliver: ['OB-spec', 'OB-coverage', 'OB-depth', 'OB-trace', 'OB-automation', 'OB-signoff'],
|
|
307
|
+
};
|
|
308
|
+
|
|
309
|
+
export function runGate(projectRoot: string, unit: string, phase: GatePhase): GateVerdict {
|
|
310
|
+
const r = runJourney(projectRoot, unit);
|
|
311
|
+
const required = PHASE_REQUIRED[phase];
|
|
312
|
+
const reqObs = r.obligations.filter((o) => required.includes(o.id));
|
|
313
|
+
const blockers = reqObs
|
|
314
|
+
.filter((o) => o.status !== 'satisfied' && o.status !== 'waived')
|
|
315
|
+
.map((o) => ({ id: o.id, title: o.title, detail: o.detail }));
|
|
316
|
+
const waivedCredit = reqObs.filter((o) => o.status === 'waived').map((o) => ({ id: o.id, title: o.title }));
|
|
317
|
+
return { unit, phase, status: blockers.length ? 'halt' : 'pass', required, blockers, waivedCredit };
|
|
318
|
+
}
|
|
319
|
+
|
|
320
|
+
export function renderGate(v: GateVerdict): string {
|
|
321
|
+
const L: string[] = [];
|
|
322
|
+
L.push('');
|
|
323
|
+
L.push(`━━━ Gate: ${v.unit} @ phase "${v.phase}" → ${v.status === 'pass' ? '✅ PASS' : '⛔ HALT'} ━━━`);
|
|
324
|
+
if (v.blockers.length) {
|
|
325
|
+
L.push(' Blocking obligations (must be satisfied or explicitly waived):');
|
|
326
|
+
for (const b of v.blockers) L.push(` • ${b.id} ${b.title} — ${b.detail}`);
|
|
327
|
+
L.push(' → Self-correct (repair / run-test), or `sungen journey --screen ' + v.unit + ' --waive <OB> --reason "..."` if accepted.');
|
|
328
|
+
} else {
|
|
329
|
+
L.push(' All required obligations satisfied' + (v.waivedCredit.length ? ` (${v.waivedCredit.length} accepted via waiver)` : '') + '.');
|
|
330
|
+
}
|
|
331
|
+
L.push('');
|
|
332
|
+
return L.join('\n');
|
|
333
|
+
}
|
package/src/harness/parse.ts
CHANGED
|
@@ -106,7 +106,10 @@ function classifyScenario(sc: ParsedScenario): ScenarioInfo {
|
|
|
106
106
|
const deferredToFlow = tags.includes('@deferred:flow');
|
|
107
107
|
const ownedByFlow = (tags.find((t: string) => /^@owned-by:/i.test(t)) || '').slice('@owned-by:'.length) || undefined;
|
|
108
108
|
// @deferred:flow is owned by a flow → not automated on this screen, so it accounts like @manual (H6).
|
|
109
|
-
|
|
109
|
+
// Recognize both bare `@manual` and the reason-coded `@manual:Mx` convention (what the generator emits);
|
|
110
|
+
// must match capability-plan.ts's detection, or `@manual:Mx` scenarios stay in the businessDepth
|
|
111
|
+
// denominator and silently suppress the ratio (#386).
|
|
112
|
+
const manual = tags.some((t) => /^@manual\b/i.test(t)) || deferredToFlow;
|
|
110
113
|
const casesTag = tags.find((t) => t.startsWith('@cases:'));
|
|
111
114
|
const casesDataset = casesTag ? casesTag.slice('@cases:'.length).trim() : undefined;
|
|
112
115
|
// Named-query references: @query:<name>[(overrides)] tags + inline `query [name]` step refs.
|
|
@@ -74,6 +74,7 @@ export const AI_RULES_FILE_MAPPING: [string, string][] = [
|
|
|
74
74
|
['claude-agent-reviewer.md', '.claude/agents/sungen-reviewer.md'],
|
|
75
75
|
['claude-agent-discovery.md', '.claude/agents/sungen-discovery.md'],
|
|
76
76
|
['claude-agent-challenge.md', '.claude/agents/sungen-challenge.md'],
|
|
77
|
+
['claude-agent-generator.md', '.claude/agents/sungen-generator.md'],
|
|
77
78
|
|
|
78
79
|
// Skills — GitHub Copilot
|
|
79
80
|
['github-skill-sungen-gherkin-syntax.md', '.github/skills/sungen-gherkin-syntax/SKILL.md'],
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sungen-generator
|
|
3
|
+
description: Generates Gherkin scenarios for ONE shard (a viewpoint theme or a spec section) in an isolated context and writes a self-contained fragment — so create-test can fan out many generators in parallel and the orchestrator stays lean. Each shard owns a disjoint VP-prefix namespace, so fragments merge without renumbering. Invoked by create-test/design during parallel generation.
|
|
4
|
+
tools: Read, Grep, Glob, Bash, Write, Edit, Skill
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
You are a **single-shard test-case generator**. You run in an **isolated context** and produce the scenarios for **exactly one shard** — never the whole screen. The orchestrator runs several of you in parallel, then merges the fragments. Keeping each fragment small is also what keeps every generator under the output-token cap.
|
|
8
|
+
|
|
9
|
+
## What a shard is
|
|
10
|
+
A shard is one **coverage unit**, sized for real parallelism (not the 5 coarse viewpoint-router groups — a screen loads only 1–2 of those). It is **one of**:
|
|
11
|
+
- a **viewpoint theme** — a `VP-` prefix from the viewpoint overview (e.g. `VP-SEC`, `VP-ERROR-EMPTY-STATE`, `VP-CAROUSEL`), or
|
|
12
|
+
- a **spec section** — one `spec.md` section per the `sungen-tc-generation` Mapping Contract (Table 1).
|
|
13
|
+
|
|
14
|
+
Your shard owns its `VP-` prefix, so your ids never collide with sibling shards.
|
|
15
|
+
|
|
16
|
+
## Inputs (passed by the orchestrator)
|
|
17
|
+
- **Your shard**: the theme/section name + its viewpoint items (the slice).
|
|
18
|
+
- **The `sungen-discovery` report** (Step 3): condensed facts — use it instead of re-reading every source.
|
|
19
|
+
- **Relevant context**: only the `spec.md` section(s) your shard maps to, and **which** `sungen-viewpoint` group file holds your shard's patterns (load only that one).
|
|
20
|
+
- **Unit context**: screen vs flow, the unit name, the chosen tier (1 / 2 / 3 / full), and your fragment paths.
|
|
21
|
+
|
|
22
|
+
## Generate (your shard ONLY)
|
|
23
|
+
1. Load **only** the skills you need: `sungen-tc-generation` (output format + mapping), `sungen-gherkin-syntax` (step patterns), and the **one** `sungen-viewpoint` group file your shard belongs to. Do not load the others.
|
|
24
|
+
2. Produce the scenarios for your shard's viewpoint items at the requested tier, following the skill's mapping contract. Keep every `VP-` id under **your shard's prefix** so it stays in a disjoint namespace.
|
|
25
|
+
3. **Flows**: use `[Screen:Element]` namespace refs, namespace test-data by phase, add the `@flow` tag per the skill.
|
|
26
|
+
4. Tag `@manual:Mx` (with a reason) only for true judgment / missing-capability items, per the skill.
|
|
27
|
+
|
|
28
|
+
## Write your fragment (do NOT write the final feature)
|
|
29
|
+
Write two self-contained fragment files (the orchestrator merges them):
|
|
30
|
+
- `.sungen/fragments/<unit>/<shard>.feature` — a **headerless** block: just your `@tag`-decorated `Scenario:` / `Scenario Outline:` blocks, no `Feature:` line (the orchestrator owns the single Feature header).
|
|
31
|
+
- `.sungen/fragments/<unit>/<shard>.test-data.yaml` — only the `{{variables}}` your scenarios introduce.
|
|
32
|
+
|
|
33
|
+
Distinct paths per shard ⇒ no write conflict with sibling generators.
|
|
34
|
+
|
|
35
|
+
## Return (compact — your only message back)
|
|
36
|
+
```
|
|
37
|
+
SHARD: <theme-or-section>
|
|
38
|
+
SCENARIOS: <n> (VP ids: <VP-...-001..NNN>)
|
|
39
|
+
TEST-DATA KEYS: <keys you added>
|
|
40
|
+
SPEC SECTIONS COVERED: <list>
|
|
41
|
+
ASSUMPTIONS / DEFERRED: <items you marked @manual or could not source>
|
|
42
|
+
FRAGMENT: .sungen/fragments/<unit>/<shard>.feature
|
|
43
|
+
```
|
|
44
|
+
Keep it tight. Do not audit, do not merge, do not touch other shards' fragments or the final `.feature`.
|
|
@@ -71,18 +71,41 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
71
71
|
Summarize what you found in requirements and present to the user.
|
|
72
72
|
|
|
73
73
|
4. Follow the `sungen-tc-generation` skill for section identification, viewpoint generation, and output format. **Viewpoint loading discipline:** `sungen-viewpoint` is a **router** — from the page-type (form / list / detail / auth / dashboard …) read **only the matching group file(s)** (e.g. a login screen → group-e-identity; a product list → group-c-data-explore), never all five groups. This keeps the generation context lean. **For flows**, use the "Flow Test Generation" section in the skill. When requirements exist, use the "Requirements-Driven Generation" strategy. **For Tier 1**, apply the **Lightweight Guard** — verify required fields, validation rules, business rules, security checks, and key state transitions all have TCs after generation. **For Tier 2+**, **MUST** apply the full **Mapping Contract** — walk every `spec.md` section top-to-bottom and produce the indicated TCs per Table 1; handle `test-viewpoint.md` per Table 2. Do not silently skip sections.
|
|
74
|
-
5. Generate
|
|
74
|
+
5. Generate `.feature` + `test-data.yaml` following `sungen-gherkin-syntax` and `sungen-tc-generation`. **Partition the work into shards and generate them in parallel** when there are ≥2.
|
|
75
|
+
|
|
76
|
+
**5a. Decide the shards.** A shard is one **coverage unit** sized for parallelism — NOT the 5 coarse viewpoint-router groups (a screen loads only 1–2 of those). Use **either**:
|
|
77
|
+
- one **viewpoint theme** per shard — a `VP-` prefix from the viewpoint overview (`VP-SEC`, `VP-ERROR-EMPTY-STATE`, `VP-CAROUSEL`, …) — preferred when the viewpoint overview is rich (test-2/home had 47 items across many themes); **or**
|
|
78
|
+
- one **`spec.md` section** per shard (the Mapping Contract walk, Table 1) — preferred when generating from spec.
|
|
79
|
+
Each shard owns a disjoint `VP-` prefix ⇒ ids never collide. One shard → skip to 5c (no fan-out gain).
|
|
80
|
+
|
|
81
|
+
**5b. Parallel fan-out (Claude Code).** Spawn one **`sungen-generator`** sub-agent **per shard** (Task tool, `subagent_type: sungen-generator`) — issue all the Task calls **in a single message** so they run concurrently. Pass each: its shard (theme/section) + viewpoint slice, the **`sungen-discovery` report** (Step 3), only the `spec.md` section(s) it maps to, which one `sungen-viewpoint` group file holds its patterns, the unit (screen/flow) + name + tier, and its fragment paths `.sungen/fragments/<name>/<shard>.{feature,test-data.yaml}`. Each writes a **headerless** fragment + a test-data fragment and returns a compact summary. Small fragments also keep every generator under the output-token cap (the reason the single-pass path writes incrementally).
|
|
82
|
+
|
|
83
|
+
**5c. Merge (orchestrator — barrier; only after all generators return).**
|
|
84
|
+
- Write the final `qa/<screens|flows>/<name>/features/<name>.feature`: one `Feature:` header (+ `@flow` for flows), then concatenate the fragments in **stable order** — spec-section order top-to-bottom (or theme order from the viewpoint overview) — so output is coherent and reproducible across runs.
|
|
85
|
+
- **Dedup** cross-shard scenarios with near-identical titles (a generic "navigation works" from two shards): keep the earlier shard's, drop the duplicate, note it. No id renumber needed — prefixes are disjoint by construction.
|
|
86
|
+
- **Union** the test-data fragments into `test-data.yaml`; dedup keys, and **flag** any key two shards define with different values.
|
|
87
|
+
- Delete `.sungen/fragments/<name>/` once merged.
|
|
88
|
+
- Guarantees a **coherent** suite (no dup, valid ids, passes `audit`), not a byte-identical one — generation is AI-authored; the determinism asset lives downstream in the Gherkin→`.spec.ts` compiler.
|
|
89
|
+
|
|
90
|
+
**5d. Sequential fallback.** Use the single-context incremental path (Step 2: tier-by-tier `Write`/`Edit` batches) when: only **one** shard applies, **Copilot / no sub-agents**, or a constrained setup. Same output, just no speedup. **For flows**: `[Screen:Element]` namespace refs, test-data namespaced by phase, `@flow` tag.
|
|
91
|
+
|
|
92
|
+
5.4. **Depth self-check (deterministic — run BEFORE the audit).** Run `sungen depth-lint --screen <name>` (Bash). It reuses the audit's businessDepth classifier and splits every shallow business-critical scenario into two actionable buckets — act on them now so the audit/repair loop doesn't burn rounds on depth:
|
|
93
|
+
- **DEEPEN IN PLACE** — add a real value assertion to each (`User see all [X] contain {{v}}`, `User remember [X] as {{v}}` + `… with {{v}}`). The printed `template` is a **hint** keyed off the theme — apply judgment to the scenario's actual claim; do NOT paste a value assertion that doesn't fit (e.g. a carousel-visibility scenario should assert the product SET, not a price). If a flagged scenario is genuinely visibility/behavior (not data-correctness), that's an over-count — leave it and note it, never fake an assertion.
|
|
94
|
+
- **CROSS-SCREEN** — route to a flow (`/sungen:add-flow`) or tag `@manual:Mx` + reason; do NOT fake an on-screen data assertion. This removes it from the screen's depth denominator honestly.
|
|
95
|
+
Re-run `sungen depth-lint` until `deepen` is empty (or only honest over-counts remain), THEN proceed to the gate. This lifts first-pass `businessDepth` mechanically instead of via 2–3 organic repair rounds.
|
|
75
96
|
|
|
76
97
|
5.5. **Quality gate & repair (harness — always run, do NOT skip).** Follow the `sungen-harness-audit` skill:
|
|
77
98
|
- Run `sungen audit --screen <name>` (Bash) and read `gateStatus` + `findings` (deterministic, structural).
|
|
78
99
|
- **Independent semantic review.** **Claude Code:** spawn the **`sungen-reviewer`** sub-agent (Task tool, `subagent_type: sungen-reviewer`) — it judges what the gate can't (does each scenario's steps PROVE its title/viewpoint, observable Thens, business-critical assertion depth) and returns `VERDICT` + `ISSUES` with concrete fixes. **Merge its NEEDS-REPAIR issues with the audit findings.** (Copilot / no sub-agents: run the same review inline using the `sungen-reviewer` criteria.)
|
|
79
100
|
- Repair **both** the audit findings and the reviewer issues (budget 3 rounds), then re-audit:
|
|
101
|
+
- **Repair runs single-agent by default** (it edits the one `.feature` — concurrent edits to the same file conflict, and BALANCE/dedup needs whole-suite context). **Exception:** a finding that is purely **additive new coverage** (GATE missing-theme → generate scenarios for an uncovered theme) is just more shards — fan it out as `sungen-generator` sub-agent(s) (new disjoint `VP-` prefix) and merge, exactly like Step 5b. Findings that **edit existing** scenarios (DEPTH/BALANCE/TRACE) stay serial.
|
|
80
102
|
- If the gate FAILs or there are findings, **repair** (budget 3 rounds), then re-audit:
|
|
81
103
|
- **GATE** missing critical theme → generate scenarios for it. If it is **cross-screen** (cart-correctness, product-detail-consistency, filter-result-correctness): **automate it in the flow** (`/sungen:add-flow` if none exists) with observable data assertions (`... with {{value}}`, `see all ... contain {{v}}`) — a single home→target journey runs as one Playwright test. Do **not** write a full `@manual` duplicate of it on the screen (that is a non-running dead copy — `sungen audit` flags it `MANUAL-AUTOMATABLE`), and do **not** fake a shallow single-screen pass. Reserve `@manual` for true judgment / missing-capability, tagged `@manual:Mx`.
|
|
82
104
|
- **DEPTH** → replace `see [X] page/section` on business-critical scenarios with data assertions.
|
|
83
105
|
- **BALANCE** → stop expanding secondary viewpoints; add business-core scenarios first.
|
|
84
106
|
- **TRACE** → align `VP-` ids with the viewpoint-overview.
|
|
85
107
|
- Stop when the gate PASSes and findings clear, **or** the budget is exhausted → report residual gaps honestly (never fake a pass).
|
|
108
|
+
- **Phase gate (boundary — do NOT skip).** Run `sungen gate --screen <name> --phase create` (Bash, exit 2 = HALT). It is the deterministic create-boundary: every required obligation (spec · coverage · depth · trace) must be **satisfied or explicitly waived**. On **HALT**, you have not cleared the phase — keep repairing the blocking obligation(s) within budget; if a blocker is a genuinely-accepted gap (e.g. cross-screen depth owned by a flow), record it with `sungen journey --screen <name> --waive <OB> --reason "..."` (reason mandatory). **Do not converge (step 6) past a HALT** without a fix or a reasoned waiver — no bad output crosses the boundary.
|
|
86
109
|
|
|
87
110
|
5.6. **Record (reuse + observability).** Build the manifest and report usage:
|
|
88
111
|
- `sungen manifest --screen <name>` — fingerprints for next-run change detection. On a **re-run**, start the whole command by `sungen manifest --screen <name> --diff` and only regenerate scenarios whose spec section changed (keep/regenerate/retire).
|
|
@@ -9,6 +9,8 @@ allowed-tools: Read, Grep, Bash, Glob, Edit, Write, AskUserQuestion, mcp__playwr
|
|
|
9
9
|
|
|
10
10
|
You are a **Senior Developer**. Use `sungen-selector-fix`, `sungen-selector-keys`, and `sungen-error-mapping` skills.
|
|
11
11
|
|
|
12
|
+
> ⛔ **Source of truth — the live page is NOT the oracle; `.feature`/`test-data`/`spec.md` are.** Auto-fix is for **selector-resolution** failures (wrong locator → fix `selectors.yaml`). An **assertion-value** failure where the app contradicts the spec is a **CANDIDATE BUG → report it, let it FAIL** — never loosen the rule, weaken the assertion, edit the expected value/`.feature`, or hand-edit the generated `.spec.ts` to make it pass. See `sungen-error-mapping` § "Source of truth". (A `password > 8` test that fails on 6 chars is a bug to report, not a `>= 6` edit.)
|
|
13
|
+
|
|
12
14
|
## Parameters
|
|
13
15
|
|
|
14
16
|
Parse from `$ARGUMENTS`:
|
|
@@ -102,6 +104,7 @@ If the unit is **api-first**, skip every selector/capture phase (an API test has
|
|
|
102
104
|
9. **Integrity check & trace (always run after the final run).**
|
|
103
105
|
- `sungen script-check --screen <name>` — verify the generated spec is a **1:1** of the Gherkin (every non-@manual scenario ↔ one `test()`, no drift). If it reports **DRIFT** (spec hand-edited or stale), re-run `sungen generate --screen <name>` so the spec matches the feature, then re-run — **never hand-edit the generated spec** (auto-fix must edit `selectors.yaml`, not the `.spec.ts`).
|
|
104
106
|
- `sungen ledger record --screen <name> --step run --ms <elapsed>` (record this run), then `sungen trace --screen <name>` — show the process map + bottlenecks + **HUMAN-LOOP FOCUS** (the @manual scenarios the QA must verify) to the user.
|
|
107
|
+
- **Phase gate (boundary — do NOT skip).** `sungen gate --screen <name> --phase run` (exit 2 = HALT): the run-boundary obligations (incl. automation) must be **satisfied or explicitly waived**. On **HALT**, classify per `sungen-error-mapping` § Source of truth (#387): a **selector-resolution** failure → fix `selectors.yaml` + re-run; an **assertion-vs-spec** failure → **report it as a candidate bug / leave it FAIL** (never weaken the assertion or edit the expected to pass); a genuinely-accepted gap → `sungen journey --screen <name> --waive <OB> --reason "..."`. Do **not** declare the run "done" past a HALT without a fix, a reported bug, or a reasoned waiver.
|
|
105
108
|
10. **Capability-pending offer (consent-gated).** If `sungen audit --screen <name>` reports `AUTOMATION-READY-PENDING` (or the run shows `@requires:<cap>` tests skipped "requires …"), these are **automation-ready** scenarios waiting on an opt-in driver. Use `AskUserQuestion` to offer: *"N scenario(s) are automation-ready — enable `<cap>` to run them? (`sungen capability add <cap>`)"*. **Only on the user's yes** run `sungen capability add <cap>` then re-run those specs; on no, leave them skipped (they are NOT failures and NOT manual). **Never auto-install.**
|
|
106
109
|
|
|
107
110
|
## Playwright command guidelines
|
|
@@ -88,6 +88,33 @@ Multi-locale (no `SUNGEN_ENV`): one **`<LOCALE> Auto`** sheet per locale + a sin
|
|
|
88
88
|
|
|
89
89
|
---
|
|
90
90
|
|
|
91
|
+
## API delivery — extra worksheet
|
|
92
|
+
|
|
93
|
+
For **api-kind units** (`qa/api/<area>/`), the `.xlsx` gains a third worksheet **`API detail`** (appended after Auto/Manual). The main BM-2-901-13 Testcases layout is unchanged. The CSV is unchanged (16-column, no extra sheet).
|
|
94
|
+
|
|
95
|
+
### Required sources (API detail sheet only)
|
|
96
|
+
|
|
97
|
+
| Source | Path | Created by |
|
|
98
|
+
|--------|------|------------|
|
|
99
|
+
| Endpoint catalog | `qa/api/<area>/api/apis.yaml` | `sungen add --api` or `sungen api import` |
|
|
100
|
+
| Scenario annotations | `qa/api/<area>/features/<feature>.feature` | `create-test` |
|
|
101
|
+
|
|
102
|
+
### API detail column mapping
|
|
103
|
+
|
|
104
|
+
| Column | Source |
|
|
105
|
+
|--------|--------|
|
|
106
|
+
| Endpoint | `path` from `apis.yaml` catalog entry |
|
|
107
|
+
| Method | `method` from catalog entry (uppercased) |
|
|
108
|
+
| Auth / Datasource | catalog `datasource` + any `@auth:<role>` tag from scenarios calling this endpoint |
|
|
109
|
+
| Request shape | catalog `body` + `params` fields composed as `body: {…}; params: [a, b]` |
|
|
110
|
+
| Expected-status matrix | `@cases:<dataset>` label for data-driven scenarios; catalog `expect.status` as fallback |
|
|
111
|
+
| Flow steps | Ordered `@api:<name>` call chain from multi-call scenarios (e.g. `register → count_users`) |
|
|
112
|
+
| Concurrency invariant | `@concurrent:<N>` + `@query:<oracle>` from concurrent scenarios (e.g. `ok_count=2; @query user_count`) |
|
|
113
|
+
|
|
114
|
+
**Sources are catalog + annotations only** — Field Metadata (FM) is not required for this sheet.
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
91
118
|
## Excluded from CSV
|
|
92
119
|
|
|
93
120
|
- `@steps:<name>` **base** scenarios — these are setup-only, inlined into `@extend:...` scenarios at compile time
|
|
@@ -21,6 +21,23 @@ Then choose the fix from the patterns below.
|
|
|
21
21
|
|
|
22
22
|
---
|
|
23
23
|
|
|
24
|
+
## ⛔ Source of truth — classify EVERY failure before you "fix" it
|
|
25
|
+
|
|
26
|
+
`.feature` + `test-data.yaml` + `spec.md` are the **oracle**. The **live page is NOT** — it may be the thing that's broken. A failing test is not automatically a test to "make pass". Classify first:
|
|
27
|
+
|
|
28
|
+
- **Selector-resolution failure** (element not found / wrong locator / strict-mode / wrong element type) → the test looked in the wrong place. **Fix the locator in `selectors.yaml`** (re-snapshot, copy the exact accessible name). Legit auto-fix.
|
|
29
|
+
- **Assertion-value failure** (element FOUND, but observed value ≠ expected) → STOP and ask: *is the TEST wrong, or is the APP wrong?*
|
|
30
|
+
- Expected value/rule is wrong **relative to `spec.md`** (typo, stale test-data) → fix `test-data.yaml`/`.feature` so it matches the **spec** — never the live page.
|
|
31
|
+
- App behaviour contradicts `spec.md` (spec says X, app shows Y) → **CANDIDATE BUG**. **Report it** (let the test FAIL / surface to the QA in the run summary). **NEVER** change the expected value, loosen the rule, weaken the assertion (`toHaveText`→`toContainText` to dodge a mismatch), edit `.feature`, or edit the generated `.spec.ts` to make it pass.
|
|
32
|
+
|
|
33
|
+
> **Cardinal sin (do NOT do this):** a `password > 8 chars` rule fails on a 6-char input → "fix" it to `>= 6` so the test passes. The logic is now meaningless. A failing assertion is a **finding**, not a chore.
|
|
34
|
+
|
|
35
|
+
**Auto-fix loop scope:** the run-test auto-fix loop engages ONLY on **selector-resolution** failures. On an assertion-value failure where the app contradicts the spec → **HALT and report**, do not loop it into passing.
|
|
36
|
+
|
|
37
|
+
**Never hand-edit the generated `.spec.ts`** (e.g. inserting `page.evaluate`/`fetch` to bypass a broken control). `sungen script-check` regenerates the spec from `.feature` and flags any edit as DRIFT — regenerate, don't patch.
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
24
41
|
## Fix Priority (try in order)
|
|
25
42
|
|
|
26
43
|
1. **Auth issue** — page redirected to login? Fix auth first, everything else is noise
|
|
@@ -43,11 +60,13 @@ Then choose the fix from the patterns below.
|
|
|
43
60
|
| not a select | Custom dropdown, not native `<select>` | Set `variant: 'custom'` |
|
|
44
61
|
| Frame not found | iframe selector wrong or doesn't exist | Fix `frame` value, verify iframe in snapshot |
|
|
45
62
|
|
|
46
|
-
### Assertion errors →
|
|
63
|
+
### Assertion errors → apply the Source-of-truth gate above FIRST
|
|
47
64
|
|
|
48
|
-
|
|
65
|
+
> The "Fix" column below applies **only when the expected value was wrong relative to `spec.md`** (a test defect). If the app's value contradicts the spec, the row is a **candidate bug → report it, do not edit the expected to match live**. Never weaken `toHaveText`→`toContainText` just to pass.
|
|
66
|
+
|
|
67
|
+
| Error | Diagnosis | Fix (only if the TEST was wrong per spec) |
|
|
49
68
|
|---|---|---|
|
|
50
|
-
| toHaveText mismatch | Expected text differs from actual |
|
|
69
|
+
| toHaveText mismatch | Expected text differs from actual | If the test's expected was wrong per spec → fix value in test-data. If element is input type → change Gherkin type to `field`/`textarea` (triggers `toHaveValue`). If the app value contradicts spec → **report as bug**. |
|
|
51
70
|
| toHaveValue mismatch | Expected value differs from actual | Fix value in test-data |
|
|
52
71
|
| toContainText mismatch | Partial text not found | Fix expected partial text in test-data |
|
|
53
72
|
| toBeVisible timeout | Element exists but hidden, or name wrong | Check: is element conditionally visible? Wrong name? Inside dialog? |
|
|
@@ -214,6 +214,8 @@ Options: `nth` `exact` `scope` `match` `variant` `frame` `contenteditable` `colu
|
|
|
214
214
|
| `@cases:dataset` | Data-driven: run the scenario once per row of the `dataset` LIST in test-data → one `test()` per row |
|
|
215
215
|
| `@query:name` | Database: run the named query from `database/queries.yaml` (precondition) and bind its rows to `{{name}}`; assert with `expect {{name.count}} …` + path access. Override params `@query:name(p={{v}})`. Repeatable. (Optional Data Driver — see Database verification above) |
|
|
216
216
|
| `@api:name` | API: run the named request from `api/apis.yaml` (precondition) and bind the response to `{{name}}`; assert with `expect {{name.status}} …` + path access (`{{name.body.<path>}}`). Override params `@api:name(p={{v}})`. Repeatable. (Optional API Driver) |
|
|
217
|
+
| `@concurrent:N` | API idempotency: fire the bound `@api` request N times in parallel, then bind aggregates on the `@api` name — `{{name.ok_count}}` (2xx count) and `{{name.status_counts}}` (status→count map). Assert the exactly-once invariant (`expect {{name.ok_count}} is 1`); pair with `@query` as a DB oracle. Tag order = run order: `@api` (mutate) before `@query` (verify). (Optional API Driver) |
|
|
218
|
+
| `@hybrid` | One unit, two capabilities: a signed-in browser session (UI) authorizes the `@api` call — the API request reuses the UI `storageState`. (Optional API + UI Drivers) |
|
|
217
219
|
|
|
218
220
|
### Data-driven scenarios (`@cases`)
|
|
219
221
|
|
|
@@ -9,6 +9,8 @@ user-invocable: false
|
|
|
9
9
|
- **Write incrementally — never emit the whole suite in one response.** Build the `.feature` in batches via successive `Write`/`Edit` (≈10–15 scenarios per call). For **Full coverage**, write tier-by-tier: `Write` Tier 1 → `Edit` append Tier 2 → `Edit` append Tier 3.
|
|
10
10
|
→ One huge `Write` can exceed the model's output-token cap → `API Error: Claude's response exceeded the N output token maximum`. Single-pass full coverage only fits when `CLAUDE_CODE_MAX_OUTPUT_TOKENS ≥ 64000`; otherwise batch. Batching also lets the audit/reviewer run per batch — higher quality.
|
|
11
11
|
|
|
12
|
+
- **Sharded (parallel) generation — keep each shard self-contained.** When `create-test` fans out one `sungen-generator` sub-agent per shard (a viewpoint theme like `VP-SEC`, or a `spec.md` section — see create-test Steps 5a–5c), you are generating **only your shard**: emit your scenarios under **your own `VP-` prefix** (disjoint namespace, so ids never collide), as a **headerless fragment** (no `Feature:` line — the orchestrator owns the single header). Do not reference or renumber other shards. The orchestrator concatenates fragments in stable order (spec-section / theme order), dedups by title, and unions test-data. Small fragments also stay under the output-token cap by construction.
|
|
13
|
+
|
|
12
14
|
- `spec_figma.md` exists → read file only, **NEVER** call `mcp__figma__*`
|
|
13
15
|
→ PAT auth flow already done by `sungen-capture` (mode figma-pat); re-calling fails or duplicates work.
|
|
14
16
|
|
|
@@ -273,6 +275,7 @@ Security: [S1 – admin only]
|
|
|
273
275
|
|
|
274
276
|
**Depth is a GATE dimension (harness-roadmap P1) — self-raise, never silently go shallow:**
|
|
275
277
|
- For every data-correctness theme the catalog marks `depth.requires: data-assertion`, emit its `depth.template` shape by **default** — don't wait for the repair loop. `sungen audit` measures `businessDepth` (ratio of these scenarios that assert data) against an intent threshold (functional ≥ 0.70); below it the **gate FAILs**.
|
|
278
|
+
- **Verify depth deterministically before the gate:** run `sungen depth-lint --screen <name>`. It classifies every shallow business-critical scenario into **deepen-in-place** (add the theme's value assertion — the printed `template` is a hint, fit it to the actual claim) vs **cross-screen** (route to a flow / `@manual:Mx`). Clear the `deepen` list first — this is the mechanical way to hit `businessDepth` on the first pass instead of churning repair rounds. Never fake a value assertion onto a visibility/behavior scenario the lint over-counts; leave it and note the over-count.
|
|
276
279
|
- `depth.cross_screen: true` (cart / detail / filter / brand correctness) → write the deep capture/compare shape as an **automated flow scenario** (in the flow — do NOT leave a full-step `@manual` duplicate on the screen). `@manual` is **only** for genuine judgment (M6 visual/UX · M8 not-worth · M9 human) or a missing capability (M1–M5/M7), and it **must** carry a reason code (`@manual:Mx`, or a reason comment the planner can infer). A `@manual` scenario that still has full automatable steps (a data assertion, no visual/mock/a11y judgment) is now flagged by `sungen audit` as `MANUAL-AUTOMATABLE`, and business-critical scenarios you defer to `@manual` are reported as `DEPTH-DEFERRED` (they do NOT silently inflate `businessDepth`). Deferring automatable work to `@manual` lowers quality — automate it in the flow instead.
|
|
277
280
|
- **Pick the right `@manual:Mx` code — it decides which driver can later automate the case** (`sungen audit` flags a code↔reason mismatch). Tag the code that matches the **oracle the reason describes**:
|
|
278
281
|
|
|
@@ -64,8 +64,11 @@ If the unit is **api-first** (`qa/api/<name>/` or `qa/api/flows/<name>/`), the d
|
|
|
64
64
|
Summarize what you found in requirements and present to the user.
|
|
65
65
|
|
|
66
66
|
4. Follow the `sungen-tc-generation` skill for section identification, viewpoint generation, and output format. **For flows**, use the "Flow Test Generation" section in the skill. When requirements exist, use the "Requirements-Driven Generation" strategy. **For Tier 1**, apply the **Lightweight Guard** — verify required fields, validation rules, business rules, security checks, and key state transitions all have TCs after generation. **For Tier 2+**, **MUST** apply the full **Mapping Contract** — walk every `spec.md` section top-to-bottom and produce the indicated TCs per Table 1; handle `test-viewpoint.md` per Table 2. Do not silently skip sections. Present sections as a numbered list and let user pick.
|
|
67
|
-
5. Generate or update `.feature` + `test-data.yaml` following `sungen-gherkin-syntax` and `sungen-tc-generation` skills. **For flows**: use `[Screen:Element]` namespace format, namespace test-data by phase, add `@flow` tag.
|
|
67
|
+
5. Generate or update `.feature` + `test-data.yaml` following `sungen-gherkin-syntax` and `sungen-tc-generation` skills. Generate **group-by-group** (one viewpoint group at a time, tier-by-tier `Write`/`Edit` batches) to stay under the output-token cap. **For flows**: use `[Screen:Element]` namespace format, namespace test-data by phase, add `@flow` tag.
|
|
68
|
+
> **No parallel fan-out here.** Copilot has no sub-agents, so generation is sequential (the Claude Code variant fans out one `sungen-generator` per viewpoint group and merges). Same output, no speedup.
|
|
69
|
+
5.4. **Depth self-check (deterministic — BEFORE the audit).** Run `sungen depth-lint --screen ${input:name}`. It splits every shallow business-critical scenario into **DEEPEN IN PLACE** (add a real value assertion — the printed `template` is a theme-keyed hint, apply judgment to the actual claim; never fake one onto a visibility/behavior scenario) and **CROSS-SCREEN** (route to a flow / tag `@manual:Mx` + reason — removes it from the depth denominator honestly). Act on both, re-run until `deepen` is empty (or only honest over-counts remain), THEN gate. Lifts first-pass `businessDepth` mechanically instead of via 2–3 repair rounds.
|
|
68
70
|
5.5. **Quality gate & repair (harness — always run).** Per `sungen-harness-audit`: run `sungen audit --screen ${input:name}` (structural), THEN do an **independent semantic review inline** using the `sungen-reviewer` criteria (does each scenario's steps PROVE its title/viewpoint? observable Thens? business-critical assertion depth?). Merge both sets of issues; if gate FAILs / findings exist, repair (budget 3) and re-audit — GATE missing theme → generate it (cross-screen → **automate it in the flow** via `/sungen:add-flow`, NOT a full `@manual` screen duplicate — `sungen audit` flags an automatable `@manual` as `MANUAL-AUTOMATABLE`; reserve `@manual:Mx` for true judgment/missing-capability); DEPTH → add data assertions; BALANCE → add business-core first; TRACE → align VP ids. Never fake a pass.
|
|
71
|
+
5.5b. **Phase gate (boundary — do NOT skip).** Run `sungen gate --screen ${input:name} --phase create` (exit 2 = HALT): every required obligation (spec · coverage · depth · trace) must be **satisfied or explicitly waived**. On **HALT**, keep repairing within budget; a genuinely-accepted gap → `sungen journey --screen ${input:name} --waive <OB> --reason "..."` (reason mandatory). Do **not** converge (step 6) past a HALT without a fix or a reasoned waiver.
|
|
69
72
|
5.6. **Record.** `sungen manifest --screen ${input:name}`. Ledger **each phase** (not just repair) — pick one `runId` at the start and pass it so `trace`/`ledger report` show THIS run, not a mix: `sungen ledger record --screen ${input:name} --run <runId> --step <discovery|viewpoint|gherkin|audit|repair:N> --ms <elapsed>`. On re-run, start with `sungen manifest --screen ${input:name} --diff` and only regenerate changed sections.
|
|
70
73
|
6. **Converge — show the trace.** Run `sungen trace --screen ${input:name}` and present: process map (phases + repair rounds), bottlenecks, **HUMAN-LOOP FOCUS** (@manual to verify), audit score + gate + residual gaps. Then offer next steps based on which tier was just generated:
|
|
71
74
|
|
|
@@ -9,6 +9,8 @@ tools: [read, execute, edit, vscode/askQuestions, playwright/*]
|
|
|
9
9
|
|
|
10
10
|
You are a **Senior Developer**. Use `sungen-selector-fix`, `sungen-selector-keys`, and `sungen-error-mapping` skills.
|
|
11
11
|
|
|
12
|
+
> ⛔ **Source of truth — the live page is NOT the oracle; `.feature`/`test-data`/`spec.md` are.** Auto-fix is for **selector-resolution** failures (wrong locator → fix `selectors.yaml`). An **assertion-value** failure where the app contradicts the spec is a **CANDIDATE BUG → report it, let it FAIL** — never loosen the rule, weaken the assertion, edit the expected value/`.feature`, or hand-edit the generated `.spec.ts` to make it pass. See `sungen-error-mapping` § "Source of truth". (A `password > 8` test that fails on 6 chars is a bug to report, not a `>= 6` edit.)
|
|
13
|
+
|
|
12
14
|
## Parameters
|
|
13
15
|
|
|
14
16
|
Parse from `$ARGUMENTS`:
|
|
@@ -93,6 +95,7 @@ If the unit is **api-first**, skip every selector/capture phase (an API test has
|
|
|
93
95
|
7. **Phase 3 — Full Run**: Run all tests. Fix only **new** failures (elements unique to `@normal`/`@low`). Max 1 attempt. Don't loop on low-priority failures.
|
|
94
96
|
8. **Phase 4 — Regression**: One final full run. Report results. No more fix loops.
|
|
95
97
|
9. **Integrity & trace (always run after the final run).** `sungen script-check --screen <name>` — verify the spec is a **1:1** of the Gherkin; if **DRIFT**, re-run `sungen generate --screen <name>` (never hand-edit the `.spec.ts` — auto-fix edits `selectors.yaml`). Then `sungen ledger record --screen <name> --step run --ms <elapsed>` and `sungen trace --screen <name>` to show the process map + bottlenecks + **HUMAN-LOOP FOCUS**.
|
|
98
|
+
9b. **Phase gate (boundary — do NOT skip).** `sungen gate --screen <name> --phase run` (exit 2 = HALT): run-boundary obligations (incl. automation) must be **satisfied or explicitly waived**. On HALT, classify per `sungen-error-mapping` § Source of truth (#387): selector-resolution failure → fix `selectors.yaml` + re-run; assertion-vs-spec failure → **report as a candidate bug / leave it FAIL** (never weaken to pass); accepted gap → `sungen journey --screen <name> --waive <OB> --reason "..."`. Don't declare "done" past a HALT without a fix, a reported bug, or a reasoned waiver.
|
|
96
99
|
10. **Capability-pending offer (consent-gated).** If `sungen audit` reports `AUTOMATION-READY-PENDING` (or `@requires:<cap>` tests are skipped "requires …"), offer: *"N scenario(s) are automation-ready — enable `<cap>` to run them? (`sungen capability add <cap>`)"*. Only on the user's yes, run `sungen capability add <cap>` + re-run; on no, leave skipped (not failures, not manual). **Never auto-install.**
|
|
97
100
|
|
|
98
101
|
## Playwright command guidelines
|