done-done-axi 0.6.20 → 0.6.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/dist/agents.js +38 -3
- package/dist/cli.js +77 -12
- package/dist/config.js +24 -0
- package/dist/design.js +36 -0
- package/docs/reference.md +32 -1
- package/package.json +1 -1
- package/skills/done-done/SKILL.md +9 -0
package/README.md
CHANGED
|
@@ -38,7 +38,7 @@ The last line is your first result: your agent drives the real app, a fresh judg
|
|
|
38
38
|
You need **[Node 22.18+](https://nodejs.org) and git**, plus your own signed-in coding agent or API key for the judge.
|
|
39
39
|
|
|
40
40
|
1. **Install.** The first line above puts `done-done` and `done-done-axi` on your PATH. No `sudo`? Add `--prefix ~/.local` and put `~/.local/bin` on your `PATH`.
|
|
41
|
-
2. **Set up your repo.** `init` asks which coding agent drives your app and which judge checks it, and never picks either for you. It writes a minimal `.done-done.yaml` and a verify skill that says how to launch and drive your app. Outside a terminal, pass `--agent <agent> --model <model> --judge <agent> --judge-model <model>`.
|
|
41
|
+
2. **Set up your repo.** `init` asks which coding agent drives your app and which judge checks it, and never picks either for you. It writes a minimal `.done-done.yaml` and a verify skill that says how to launch and drive your app. For an AI app (a model SDK, prompt file or prompt-building code in the repo) the skill also proposes 3 to 6 basic behaviour journeys; you accept or drop each, and the judge you chose grades the ones you keep. Outside a terminal, pass `--agent <agent> --model <model> --judge <agent> --judge-model <model>`.
|
|
42
42
|
3. **Get your first result.** Say what works now, in plain words. Your agent drives the real product as a user would and saves screenshots, a recording or output; a fresh judge checks whether each capture shows your claim. A pass names the capture and writes a receipt for this commit; anything else stops at a gate with the judge's reason, which `done-done respond` answers.
|
|
43
43
|
|
|
44
44
|
Commit `.done-done.yaml` and the verify skill so later runs and your teammates use the same recipe. Captures go to your judge's provider, on your plan or key.
|
|
@@ -48,12 +48,12 @@ Commit `.done-done.yaml` and the verify skill so later runs and your teammates u
|
|
|
48
48
|
Installing done-done from another installer (a fleet bootstrap, a Juggernaut-style one-command install) takes three calls and no prompts. Pin the version:
|
|
49
49
|
|
|
50
50
|
```sh
|
|
51
|
-
npm install -g done-done-axi@0.6.
|
|
51
|
+
npm install -g done-done-axi@0.6.21
|
|
52
52
|
done-done init --no-skill --agent <agent> --model <model> --judge <route> --judge-model <model>
|
|
53
53
|
done-done doctor --setup --json
|
|
54
54
|
```
|
|
55
55
|
|
|
56
|
-
- `done-done --version` prints the installed version (here `0.6.
|
|
56
|
+
- `done-done --version` prints the installed version (here `0.6.21`); check it before anything else.
|
|
57
57
|
- `init --no-skill` writes `.done-done.yaml` and the global `~/.done-done/config.yaml`, and starts no agent: it skips the verify skill, so it spends nothing and asks nothing. `<route>` is `answerer:<agent>` or a bare agent name. Exit `0` when the agent is set and the config is valid, `2` when it is not.
|
|
58
58
|
- `doctor --setup --json` is install health: exit `0` when git, the config, the evidence root, the agent and the judge are ok, `2` when one of them is not. The verify skill and the journey rows are warnings, so a fresh install passes.
|
|
59
59
|
- Exit codes: `0` pass, `2` usage or config error ([all codes](docs/reference.md#exit-codes)).
|
package/dist/agents.js
CHANGED
|
@@ -69,9 +69,11 @@ function stderrHead(stderr) {
|
|
|
69
69
|
const PERMISSION_REJECT = /permission requested:\s*(\S+)\s*\(([^)]*)\);\s*auto-rejecting/;
|
|
70
70
|
class ProviderError extends Error {
|
|
71
71
|
detail;
|
|
72
|
-
|
|
72
|
+
verdict;
|
|
73
|
+
constructor(text, prefix = '', suffix = '', verdict) {
|
|
73
74
|
super(`${prefix}${errorLine(text)}${suffix}`);
|
|
74
75
|
this.detail = `${prefix}${text.slice(0, 500)}${suffix}`;
|
|
76
|
+
this.verdict = verdict;
|
|
75
77
|
}
|
|
76
78
|
}
|
|
77
79
|
// The answer object in an agent's free text: the first top-level '{'..'}' span that parses to an object with every
|
|
@@ -151,6 +153,23 @@ const piProviderExtension = (model) => {
|
|
|
151
153
|
const provider = model?.split('/')[0] ?? '';
|
|
152
154
|
return Object.hasOwn(PI_PROVIDER_EXTENSIONS, provider) ? ['-e', PI_PROVIDER_EXTENSIONS[provider]] : [];
|
|
153
155
|
};
|
|
156
|
+
// What claude's own result says when it holds no answer: the fields that explain why, on one line (and in the call record).
|
|
157
|
+
function claudeVerdictFields(r) {
|
|
158
|
+
const tail = typeof r.result === 'string' ? errorLine(r.result.slice(-300)) : '';
|
|
159
|
+
return {
|
|
160
|
+
subtype: r.subtype, stop_reason: r.stop_reason, terminal_reason: r.terminal_reason, num_turns: r.num_turns,
|
|
161
|
+
permission_denials: Array.isArray(r.permission_denials) ? r.permission_denials.length : 0,
|
|
162
|
+
...(r.safety_stops !== undefined && { safety_stops: r.safety_stops }),
|
|
163
|
+
...(r.api_error_status !== undefined && { api_error_status: r.api_error_status }),
|
|
164
|
+
...(tail && { result_tail: tail }),
|
|
165
|
+
};
|
|
166
|
+
}
|
|
167
|
+
function claudeVerdict(r) {
|
|
168
|
+
const f = claudeVerdictFields(r);
|
|
169
|
+
const { result_tail: tail, ...rest } = f;
|
|
170
|
+
const facts = Object.entries(rest).map(([k, v]) => `${k}=${typeof v === 'string' ? v : JSON.stringify(v)}`).join(', ');
|
|
171
|
+
return `claude returned no structured answer (${facts})${tail ? `; result ends: ${tail}` : ''}`;
|
|
172
|
+
}
|
|
154
173
|
const ADAPTERS = {
|
|
155
174
|
// claude -p: --json-schema puts the validated object in structured_output; --tools '' turns every tool off.
|
|
156
175
|
// An edit step accepts edits only in cwd and --add-dir folders, and may run commands (the drive launches the app).
|
|
@@ -171,7 +190,20 @@ const ADAPTERS = {
|
|
|
171
190
|
throw new ProviderError('claude printed no result');
|
|
172
191
|
if (r.is_error)
|
|
173
192
|
throw new ProviderError(String(r.result || r.subtype));
|
|
174
|
-
|
|
193
|
+
const plain = (v) => !!v && typeof v === 'object' && !Array.isArray(v);
|
|
194
|
+
if (plain(r.structured_output))
|
|
195
|
+
return r.structured_output;
|
|
196
|
+
// An exit-0 result can carry no structured_output after a long tool-driven step; its result text is the model's final answer.
|
|
197
|
+
const required = c.schema.required ?? [];
|
|
198
|
+
if (typeof r.result === 'string') {
|
|
199
|
+
try {
|
|
200
|
+
const found = extractJsonObject(r.result, required);
|
|
201
|
+
if (plain(found) && required.every((k) => k in found))
|
|
202
|
+
return found;
|
|
203
|
+
}
|
|
204
|
+
catch { }
|
|
205
|
+
}
|
|
206
|
+
throw new ProviderError(claudeVerdict(r), '', '', claudeVerdictFields(r));
|
|
175
207
|
},
|
|
176
208
|
},
|
|
177
209
|
// opencode run reads no stdin: the prompt and the schema go in an attached file,
|
|
@@ -706,6 +738,7 @@ export async function askAgent(c) {
|
|
|
706
738
|
let attempts = 0;
|
|
707
739
|
let out;
|
|
708
740
|
let failure;
|
|
741
|
+
let verdict;
|
|
709
742
|
try {
|
|
710
743
|
dir = mkdtempSync(join(tmpdir(), 'done-done-agent-'));
|
|
711
744
|
cwd = c.tools === 'none' ? dir : c.cwd;
|
|
@@ -816,6 +849,8 @@ export async function askAgent(c) {
|
|
|
816
849
|
const note = (text) => (!provider && said && !text.includes(said.split(' | ')[0]) ? `${text}; ${call.agent} stderr: ${said}` : text);
|
|
817
850
|
out = { ok: false, error: note(e.message), ...(!provider && said ? { stderr: said } : {}) };
|
|
818
851
|
failure = note(e instanceof ProviderError ? e.detail : e.message);
|
|
852
|
+
if (e instanceof ProviderError)
|
|
853
|
+
verdict = e.verdict;
|
|
819
854
|
}
|
|
820
855
|
finally {
|
|
821
856
|
if (dir)
|
|
@@ -824,7 +859,7 @@ export async function askAgent(c) {
|
|
|
824
859
|
// The record holds what ran and what came back, never the prompt text or an image.
|
|
825
860
|
const { prompt, images, pool, ...spec } = c;
|
|
826
861
|
try {
|
|
827
|
-
const record = { ...spec, agent: call.agent, model: call.model, ...(picked && (picked.route ? { route: picked.id } : { account: picked.id })), ...(images && { images: images.map((i) => i.name) }), cwd, prompt_bytes: Buffer.byteLength(prompt), argv: argv('<tmp>'), ms: Date.now() - started, exit: res?.status ?? null, attempts, ...out, error: failure, stderr: String(res?.stderr ?? '').slice(-2000) };
|
|
862
|
+
const record = { ...spec, agent: call.agent, model: call.model, ...(picked && (picked.route ? { route: picked.id } : { account: picked.id })), ...(images && { images: images.map((i) => i.name) }), cwd, prompt_bytes: Buffer.byteLength(prompt), argv: argv('<tmp>'), ms: Date.now() - started, exit: res?.status ?? null, attempts, ...out, error: failure, ...(verdict && { verdict }), stderr: String(res?.stderr ?? '').slice(-2000) };
|
|
828
863
|
mkdirSync(dirname(c.record), { recursive: true });
|
|
829
864
|
writeFileSync(c.record, `${JSON.stringify(record, null, 2)}\n`, { mode: 0o600 });
|
|
830
865
|
out = { ...out, record: c.record };
|
package/dist/cli.js
CHANGED
|
@@ -15,7 +15,7 @@ import { backend, judge, judgeable, judgeEnv, onPath, quote, screen } from './ju
|
|
|
15
15
|
import { startInstance, stopInstance } from './instance.js';
|
|
16
16
|
import { attemptProblem, changedSince, commitFix, dirtyTracked, FIX_SCHEMA, inWorktree, maintainSteps, journeyProposal, named, pathspec, readerProblem, repoState, triage } from './maintain.js';
|
|
17
17
|
import { cellEnv, cellName, cellSuffix, hasMatrix, matrixManifest, splitMatrix } from './matrix.js';
|
|
18
|
-
import { design, interactions, KINDS } from './design.js';
|
|
18
|
+
import { design, interactions, KINDS, redesign } from './design.js';
|
|
19
19
|
import { checkMedia } from './media.js';
|
|
20
20
|
import { domIssues, domRecord, isDomRecord, lookIssues, SQUASHED_TEXT } from './dom.js';
|
|
21
21
|
import { BASELINE_SCHEMA, baselinePrompt, criteriaOf, detectAi, EVALS_SCHEMA, evalsProblem, evalsPrompt, evalsYaml, judgePrompt, judgeSchema, PROMPTS_FILE, promptsJsonl, README_FILE, readmeMd, readPrompts, reportMd, RUBRIC_FILE, rubricMd, scoresOf, totals } from './evals.js';
|
|
@@ -50,6 +50,7 @@ const JOURNEYS_TEXT = `journeys (.done-done.yaml): the command runs with DONE_DO
|
|
|
50
50
|
matrix (.done-done.yaml): with theme, width or form axes the journey runs once per required cell with DONE_DONE_THEME, DONE_DONE_WIDTH, DONE_DONE_FORM and DONE_DONE_CELL (the capture name suffix) set; name each capture with its suffix, so every cell is captured or skipped with a reason`;
|
|
51
51
|
// A claim about how the product looks or what it outputs needs a comparison with the approved result.
|
|
52
52
|
const DESIGN_CLAIM = /\b(design(ed|s)?|redesign(ed)?|mock(up)?s?|prototype|approved)\b/i;
|
|
53
|
+
const REDESIGN_CLAIM = /\bredesign(ed|s|ing)?\b/i;
|
|
53
54
|
const CHECKS_TEXT = `Checks block (--from): the newest "## Checks" heading, then these four keys, each at the start of a line:
|
|
54
55
|
outcome: <what a user can now do>
|
|
55
56
|
repro: <command that failed before and passes now, or n/a; a fix PR needs one>
|
|
@@ -57,6 +58,8 @@ const CHECKS_TEXT = `Checks block (--from): the newest "## Checks" heading, then
|
|
|
57
58
|
evidence: <the evidence file paths, or "not proven: <cause>">
|
|
58
59
|
approved_design (.done-done.yaml): a non-empty manifest of {screen, size: <width>x<height>, theme, mock: <repo PNG/JPEG>, capture: <unique filename>};
|
|
59
60
|
every claim needs every listed live screen, paired with its mock and a passing design verdict; missing or unsure matches fail by screen/size/theme
|
|
61
|
+
approved_redesign (.done-done.yaml): a non-empty manifest of {screen, size: <width>x<height>, theme, capture: <unique filename>} for a claim that says redesign:
|
|
62
|
+
each capture is one live image of the unchanged source and the redesign side by side, judged blind; a redesign claim with no manifest fails as not compared to approved result
|
|
60
63
|
approved_requirements (.done-done.yaml): approved assets, color tokens and interactions as {kind: asset|color|interaction, name, capture, mock?}; skip one only with {kind, name, exclusion: <reason>}
|
|
61
64
|
(reference images under design/, mocks/, approved/ or prototype/ must be listed or excluded; a design claim with no approved reference or base_ref baseline fails as not compared to approved result)
|
|
62
65
|
interaction_checklist (.done-done.yaml, opt-in): map the six ids in skills/done-done/interaction-checklist.json to {captures: [<live PNG/JPEG filename>, ...]}
|
|
@@ -960,7 +963,10 @@ async function runCheck(argv, run) {
|
|
|
960
963
|
}
|
|
961
964
|
// The approved inventory is the contract, not the subset the claim happened to cite. Enforce it on every
|
|
962
965
|
// claim when configured, so rewording a design-done/release outcome cannot bypass missing screens.
|
|
963
|
-
|
|
966
|
+
// A redesign claim differs from its source on purpose, so it is proven by the blind pair judge, not by layout match.
|
|
967
|
+
const redesigning = !!claim && REDESIGN_CLAIM.test(claim.outcome);
|
|
968
|
+
const redesigns = redesigning ? cfg.approved_redesign.map(entry => ({ entry, live: captures.filter(c => screen(c) && basename(cite(c.name)) === entry.capture) })) : [];
|
|
969
|
+
const approved = (redesigning ? [] : cfg.approved_design).map(entry => ({ entry, live: captures.filter(c => screen(c) && basename(cite(c.name)) === entry.capture) }));
|
|
964
970
|
const coverageFaults = [];
|
|
965
971
|
for (const { entry: e, live } of approved) {
|
|
966
972
|
const label = `${e.screen} / ${e.size} / ${e.theme}`;
|
|
@@ -969,6 +975,14 @@ async function runCheck(argv, run) {
|
|
|
969
975
|
else if (`${live[0].media.w}x${live[0].media.h}` !== e.size)
|
|
970
976
|
coverageFaults.push(`${label}: live capture is ${live[0].media.w}x${live[0].media.h}, not the approved size`);
|
|
971
977
|
}
|
|
978
|
+
const redesignFaults = [];
|
|
979
|
+
for (const { entry: e, live } of redesigns) {
|
|
980
|
+
const label = `${e.screen} / ${e.size} / ${e.theme}`;
|
|
981
|
+
if (live.length !== 1)
|
|
982
|
+
redesignFaults.push(`${label}: ${live.length ? 'ambiguous live captures' : 'no live capture'} (${e.capture})`);
|
|
983
|
+
else if (`${live[0].media.w}x${live[0].media.h}` !== e.size)
|
|
984
|
+
redesignFaults.push(`${label}: live capture is ${live[0].media.w}x${live[0].media.h}, not the approved size`);
|
|
985
|
+
}
|
|
972
986
|
// Approved assets, color tokens and interactions: each needs its own live capture, or an explicit exclusion with a reason.
|
|
973
987
|
const requirements = cfg.approved_requirements.filter(r => r.capture).map(r => ({ r, live: captures.filter(c => screen(c) && basename(cite(c.name)) === r.capture) }));
|
|
974
988
|
for (const { r, live } of requirements)
|
|
@@ -981,10 +995,15 @@ async function runCheck(argv, run) {
|
|
|
981
995
|
coverageFaults.push(`approved reference material is not in the approved_design manifest: ${unlisted.join(', ')}; list each as an approved_design screen, an approved_requirements mock, or {kind: reference, name: <path>, exclusion: <reason>}`);
|
|
982
996
|
// A design or output claim must be compared to the approved result (a manifest mock, or the base_ref baseline), not only rendered.
|
|
983
997
|
const compared = approved.length > 0 || requirements.some(({ r }) => r.mock) || heads.some(([c]) => c.before);
|
|
984
|
-
if (
|
|
998
|
+
if (redesigning && !redesigns.length)
|
|
999
|
+
redesignFaults.push('not compared to approved result: the claim is a redesign but nothing is compared to its source; add an approved_redesign manifest of live side-by-side source/redesign captures, because render, fit and dark-mode checks alone do not prove it');
|
|
1000
|
+
else if (claim && !redesigning && !compared && DESIGN_CLAIM.test(claim.outcome))
|
|
985
1001
|
coverageFaults.push('not compared to approved result: the claim is about design or output but nothing is compared to an approved reference or a same-model baseline; add an approved_design manifest (or an approved_requirements row with a mock) or set base_ref, because render, fit and dark-mode checks alone do not prove it');
|
|
986
1002
|
for (const why of coverageFaults)
|
|
987
1003
|
findings.push({ level: 'fail', why: `approved design: ${why}` });
|
|
1004
|
+
for (const why of redesignFaults)
|
|
1005
|
+
findings.push({ level: 'fail', why: `approved redesign: ${why}` });
|
|
1006
|
+
coverageFaults.push(...redesignFaults);
|
|
988
1007
|
// A head capture of a base_ref run with no base capture of its name is reported, then judged alone.
|
|
989
1008
|
for (const [c, inRoot, rec] of heads)
|
|
990
1009
|
if (!(c.before = befores.get(inRoot ?? '')))
|
|
@@ -1105,6 +1124,19 @@ async function runCheck(argv, run) {
|
|
|
1105
1124
|
if (judgeBlock)
|
|
1106
1125
|
judgeBlock.approved_design = matches;
|
|
1107
1126
|
}
|
|
1127
|
+
let redesigned = [];
|
|
1128
|
+
if (redesigns.length) {
|
|
1129
|
+
const verdicts = await redesign(cfg, choice, redesigns.map(r => r.live[0]), agentRecords);
|
|
1130
|
+
redesigned = verdicts.map((v, i) => {
|
|
1131
|
+
const { screen, size, theme } = redesigns[i].entry;
|
|
1132
|
+
const reason = v.failed.length ? v.failed.map(d => `${d.kind}: ${d.where}`).join('; ') : v.reason;
|
|
1133
|
+
if (v.verdict !== 'pass')
|
|
1134
|
+
findings.push({ level: v.verdict === 'fail' ? 'fail' : 'inconclusive', why: `approved redesign: ${screen} / ${size} / ${theme}: ${v.verdict === 'unsure' ? 'redesign not proven: ' : ''}${reason}`, file: v.path });
|
|
1135
|
+
return { screen, size, theme, path: receiptPath(repo, cfg, cite(v.path)), sha256: redesigns[i].live[0].sha256, verdict: v.verdict, ...(reason && { reason }) };
|
|
1136
|
+
});
|
|
1137
|
+
if (judgeBlock)
|
|
1138
|
+
judgeBlock.approved_redesign = redesigned;
|
|
1139
|
+
}
|
|
1108
1140
|
const withMock = requirements.filter(({ r }) => r.mock);
|
|
1109
1141
|
if (withMock.length) {
|
|
1110
1142
|
const pairs = withMock.map(({ r, live }) => pairUp(`${r.kind} ${r.name}`, r.mock, live[0]));
|
|
@@ -1130,6 +1162,8 @@ async function runCheck(argv, run) {
|
|
|
1130
1162
|
rep.result.evals = evals;
|
|
1131
1163
|
if (matches.length)
|
|
1132
1164
|
rep.result.design = matches;
|
|
1165
|
+
if (redesigned.length)
|
|
1166
|
+
rep.result.redesign = redesigned;
|
|
1133
1167
|
const excluded = cfg.approved_requirements.filter(r => r.exclusion).map(({ kind, name, exclusion }) => ({ kind, name, exclusion }));
|
|
1134
1168
|
if (excluded.length)
|
|
1135
1169
|
rep.result.approved_exclusions = excluded;
|
|
@@ -1294,6 +1328,8 @@ function receiptFault(rec, short) {
|
|
|
1294
1328
|
return `receipt for ${short} has a malformed judge block`;
|
|
1295
1329
|
if (j.approved_design !== undefined && (!Array.isArray(j.approved_design) || !j.approved_design.length || !j.approved_design.every(v => v && v.verdict === 'pass' && [v.screen, v.size, v.theme].every(s => typeof s === 'string' && s.trim()) && rec.files.some(f => f.path === v.path && f.sha256 === v.sha256) && rec.files.some(f => f.path === v.mock && f.sha256 === v.mock_sha256))))
|
|
1296
1330
|
return `receipt for ${short} has a malformed approved-design verdict`;
|
|
1331
|
+
if (j.approved_redesign !== undefined && (!Array.isArray(j.approved_redesign) || !j.approved_redesign.length || !j.approved_redesign.every(v => v && v.verdict === 'pass' && [v.screen, v.size, v.theme].every(s => typeof s === 'string' && s.trim()) && rec.files.some(f => f.path === v.path && f.sha256 === v.sha256))))
|
|
1332
|
+
return `receipt for ${short} has a malformed approved-redesign verdict`;
|
|
1297
1333
|
if (j.interaction_checklist !== undefined) {
|
|
1298
1334
|
const rows = j.interaction_checklist;
|
|
1299
1335
|
if (!Array.isArray(rows) || rows.length !== Object.keys(INTERACTIONS).length || new Set(rows.map(r => r?.id)).size !== rows.length || !rows.every(r => r && Object.hasOwn(INTERACTIONS, r.id) && typeof r.reason === 'string' && r.reason.trim() && Array.isArray(r.captures) && (r.status === 'not_applicable' ? !r.captures.length : r.status === 'pass' && r.captures.length && r.captures.every(c => c && rec.files.some(f => f.path === c.path && f.sha256 === c.sha256)))))
|
|
@@ -1463,6 +1499,8 @@ function checkHelp(code, findings, files, { cfg, repo, claim, source, citationBa
|
|
|
1463
1499
|
}
|
|
1464
1500
|
if (findings.some((f) => f.why.includes('step record')))
|
|
1465
1501
|
help.push(`Have the journey write ${STEPS_FILE} into $DONE_DONE_EVIDENCE_ROOT, shaped ${STEPS_SHAPE}, naming the steps and the result the claim is about; then re-run it (${captureHint(cfg)}) and cite the new files`);
|
|
1502
|
+
if (findings.some((f) => f.why.includes('add an approved_redesign manifest')))
|
|
1503
|
+
help.push('Add approved_redesign to .done-done.yaml: one {screen, size, theme, capture} row per screen, each capture a live image of the unchanged source and the redesign side by side; commit it, capture the pairs in a run, then cite them');
|
|
1466
1504
|
// Proof refused as code-only needs a journey that drives the product, not a re-capture of the same run.
|
|
1467
1505
|
if (findings.some((f) => f.why.endsWith(RULE)))
|
|
1468
1506
|
help.push(`Capture the real product: ${captureHint(cfg).startsWith('done-done init') ? captureHint(cfg) : `${captureHint(cfg)}, or ${RECORD_REAL}`}, then cite its files`);
|
|
@@ -3073,6 +3111,13 @@ const SCAFFOLD_SCHEMA = {
|
|
|
3073
3111
|
required: ['surface', 'first_release', 'launch', 'ready', 'harness', 'observe', 'isolate', 'features'],
|
|
3074
3112
|
additionalProperties: false,
|
|
3075
3113
|
};
|
|
3114
|
+
// An AI-output app (evals.ts detectAi) is also asked for a few basic behaviour journeys; any other app gets the schema above unchanged.
|
|
3115
|
+
const BEHAVIOUR = { min: 3, max: 6 };
|
|
3116
|
+
const scaffoldSchema = (ai) => !ai ? SCAFFOLD_SCHEMA : {
|
|
3117
|
+
...SCAFFOLD_SCHEMA,
|
|
3118
|
+
properties: { ...SCAFFOLD_SCHEMA.properties, behaviour: { type: 'array', minItems: BEHAVIOUR.min, maxItems: BEHAVIOUR.max, items: { type: 'object', properties: { title: STR, user_does: STR, user_sees: STR, source: STR }, required: ['title', 'user_does', 'user_sees', 'source'], additionalProperties: false } } },
|
|
3119
|
+
required: [...SCAFFOLD_SCHEMA.required, 'behaviour'],
|
|
3120
|
+
};
|
|
3076
3121
|
const NO_EDIT = 'The drive changes no repo file: note findings in the run\'s report; after the drive, run done-done maintain to see what drifted, then fix the skill';
|
|
3077
3122
|
const MAX_FEATURES = 5;
|
|
3078
3123
|
const slug = (t) => t.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-+|-+$/g, '').slice(0, 40) || 'feature';
|
|
@@ -3131,7 +3176,7 @@ function repoContext(repo, skill) {
|
|
|
3131
3176
|
const out = left.length ? `\n\n[left out: ${left.length} files cannot be read or pass the ${CONTEXT_CHARS}-character cap: ${left.slice(0, 20).join(', ')}${left.length > 20 ? `, +${left.length - 20} more` : ''}]` : '';
|
|
3132
3177
|
return { files: new Set(files), read, text: `${list}\n\n${parts.join('\n\n')}${out}` };
|
|
3133
3178
|
}
|
|
3134
|
-
function scaffoldPrompt(cfg, context, toolless, ships) {
|
|
3179
|
+
function scaffoldPrompt(cfg, context, toolless, ships, ai = '') {
|
|
3135
3180
|
return `You write the verify skill for the app ${cfg.app}, which \`done-done init\` saves at ${cfg.skill}: how an agent launches this repo's app safely, drives it the way a user does, and captures proof that it works. Do not edit, create or delete any file, and do not run the app or any command.
|
|
3136
3181
|
${toolless ? "You have no tools: the repo's tracked file list and the text of its key files are below, and they are all you can read." : 'The repository is your working directory: read the files below first, then any other file you need.'} Use only what the repo shows: never invent a command, path, flag, route or label. Never open a secret-looking file: a .env or .env.* file, a path whose name contains secret, credential, token or password, or a *.pem, *.key or *.p12 file. Never write a secret value into steps or why; name the environment variable instead.
|
|
3137
3182
|
|
|
@@ -3146,13 +3191,13 @@ A proof drives the running product the way a user does: a test suite, typecheck,
|
|
|
3146
3191
|
- first_release: true only when the repo explicitly says no earlier release yet; false when an earlier version shipped or you cannot tell. This records upgrade proof as pending, never as passed. Do not invent a previous version or an upgrade drive when none exists.
|
|
3147
3192
|
- core: true on exactly one feature, listed first: the core user journey, the one thing a user cannot live without, driven end to end on the real target the way a user does it (for a remote terminal app: pair, open a pane, type, see the output), never a part of it or a setting. Every other feature has core false.
|
|
3148
3193
|
${ships.length ? `- upgrade: this app ships an installable or upgradable artifact (${ships.slice(0, 3).join('; ')}), so users get each release by upgrading over the version they have. Set upgrade true on one feature whose drive installs the previous shipped version, upgrades over it in place the way a user does (install the new APK over the old one, update the host or CLI package in place), then drives the core journey; a fresh install alone is no release proof. It may be the core feature itself when its drive does all of that. Exception: when first_release is true, upgrade may be false on every feature; the skill will record the upgrade journey as pending until there is a published version to upgrade from.` : '- upgrade: false on every feature: no tracked file shows the app ships an installable or upgradable artifact.'}
|
|
3149
|
-
Each source is one path exactly as the tracked file list spells it, with no other words (scripts/verify/drive.sh, never a list or a note); launch.source may be empty only when safe is false.
|
|
3194
|
+
${ai ? `- behaviour: this app's output is a model's (${ai}), so also propose ${BEHAVIOUR.min} to ${BEHAVIOUR.max} basic behaviour journeys: few and plain, not a test suite. Each is one thing a user does end to end on the real product that shows its model-driven behaviour is right (a tool the app calls, a decision it takes, an answer it shows), {title: a short name, user_does: what the user does through the repo's real handles, user_sees: what they see when the behaviour is right, source: the repo file that shows it}. Use generic examples, never private names or data.\n` : ''}Each source is one path exactly as the tracked file list spells it, with no other words (scripts/verify/drive.sh, never a list or a note); launch.source may be empty only when safe is false.
|
|
3150
3195
|
|
|
3151
3196
|
${context}`;
|
|
3152
3197
|
}
|
|
3153
3198
|
// Why an answer cannot be rendered into the skill, or '' when it can. A citation must be a tracked file the agent was
|
|
3154
3199
|
// shown: shown is every tracked file for claude, which reads the repo itself, and the text plain code gave a tool-less agent.
|
|
3155
|
-
function scaffoldProblem(a, tracked, shown, ships) {
|
|
3200
|
+
function scaffoldProblem(a, tracked, shown, ships, ai = '') {
|
|
3156
3201
|
const s = a;
|
|
3157
3202
|
const text = (v) => typeof v === 'string' && clause(v) !== '';
|
|
3158
3203
|
const citation = (p) => {
|
|
@@ -3191,9 +3236,28 @@ function scaffoldProblem(a, tracked, shown, ships) {
|
|
|
3191
3236
|
return `names ${cores} core journeys among its first ${MAX_FEATURES} features, and the skill needs exactly one: the one thing a user cannot live without, end to end on the real target`;
|
|
3192
3237
|
if (ships.length && !kept.some((f) => f.upgrade) && s.first_release !== true)
|
|
3193
3238
|
return `names no upgrade journey, and the app ships an installable artifact (${ships.slice(0, 3).join('; ')}): a fresh install alone is no release proof`;
|
|
3239
|
+
if (ai) {
|
|
3240
|
+
const b = s.behaviour;
|
|
3241
|
+
if (!Array.isArray(b) || b.length < BEHAVIOUR.min || b.length > BEHAVIOUR.max)
|
|
3242
|
+
return `has ${Array.isArray(b) ? b.length : 0} behaviour journeys, and an AI-output app needs ${BEHAVIOUR.min} to ${BEHAVIOUR.max}`;
|
|
3243
|
+
for (const j of b) {
|
|
3244
|
+
if (!j || typeof j !== 'object' || !text(j.title) || !text(j.user_does) || !text(j.user_sees) || !text(j.source))
|
|
3245
|
+
return 'has a behaviour journey without a title, user_does, user_sees or source';
|
|
3246
|
+
const why = citation(j.source);
|
|
3247
|
+
if (why)
|
|
3248
|
+
return `the behaviour journey "${j.title}" ${why}`;
|
|
3249
|
+
if (codeOnly(j.user_does))
|
|
3250
|
+
return `the behaviour journey "${j.title}" user_does ${j.user_does.trim()} is code-only: ${RULE}`;
|
|
3251
|
+
}
|
|
3252
|
+
}
|
|
3194
3253
|
return '';
|
|
3195
3254
|
}
|
|
3196
|
-
|
|
3255
|
+
// One proposed journey as the line the user keeps or deletes, with the command that accepts it (a one-line description in plain words).
|
|
3256
|
+
const proposedJourney = (j) => {
|
|
3257
|
+
const desc = `${clause(j.title)}: ${clause(j.user_does)}; expect ${clause(j.user_sees).replace(/^[A-Z](?![A-Z])/, (c) => c.toLowerCase())}`.replace(/\s+/g, ' ');
|
|
3258
|
+
return `- ${clause(j.title)}: ${clause(j.user_does)}. Expect: ${clause(j.user_sees)}. (\`${cited(j.source)}\`)\n Accept: \`done-done journey add '${desc.replace(/'/g, "'\\''")}'\``;
|
|
3259
|
+
};
|
|
3260
|
+
function verifySkillMd(app, agent, s, isolated, features, omitted, ships, ai = '') {
|
|
3197
3261
|
const name = `done-done-verify-${app}`;
|
|
3198
3262
|
const source = cited(s.launch.source);
|
|
3199
3263
|
const launch = isolated
|
|
@@ -3232,7 +3296,7 @@ ${found.join('\n')}
|
|
|
3232
3296
|
## Core journey
|
|
3233
3297
|
|
|
3234
3298
|
${link(core)} is the core user journey: the one thing a user cannot live without. A proof that the app works drives it end to end on the real target; a proof of another feature alone does not show the app works.
|
|
3235
|
-
${ships.length ? `\n## Release proof\n\nThis app ships an installable artifact (${ships.slice(0, 3).join('; ')}), and users get a release by upgrading over the version they have. ${upgrade ? `${link(upgrade)} is the release proof: install the previous shipped version, upgrade over it in place, then drive the core journey. A fresh install alone is no release proof.` : 'Upgrade journey pending: no earlier release yet. Before the next release, install the first published version, upgrade over it in place, then drive the core journey. This is not upgrade proof; doctor still reports the missing upgrade journey until one is named.'}\n` : ''}
|
|
3299
|
+
${ai && s.behaviour ? `\n## Behaviour journeys (proposed)\n\nThis app's output is a model's (${ai}), so \`done-done init\` proposes a few basic journeys that show its behaviour. They are proposals, not part of the feature map: the user accepts or drops each one, and nothing here runs until accepted. Accept one by running its command (the configured agent writes its capture script and runs it once); drop one by deleting its line. A kept journey is captured from the real product and graded by the judge the user chose; never pick a judge.\n\n${s.behaviour.map(proposedJourney).join('\n')}\n` : ''}${ships.length ? `\n## Release proof\n\nThis app ships an installable artifact (${ships.slice(0, 3).join('; ')}), and users get a release by upgrading over the version they have. ${upgrade ? `${link(upgrade)} is the release proof: install the previous shipped version, upgrade over it in place, then drive the core journey. A fresh install alone is no release proof.` : 'Upgrade journey pending: no earlier release yet. Before the next release, install the first published version, upgrade over it in place, then drive the core journey. This is not upgrade proof; doctor still reports the missing upgrade journey until one is named.'}\n` : ''}
|
|
3236
3300
|
## Evidence
|
|
3237
3301
|
|
|
3238
3302
|
What to capture: ${clause(s.observe)}. Exercise the real user path, not internal setters; done-done's judge checks every capture in the run folder against the outcome, so save only captures of the resulting state there and keep any of earlier steps outside it; verify side effects alongside what is visible. Evidence goes to the run folder \`done-done run\` prints.
|
|
@@ -3454,11 +3518,12 @@ async function scaffoldVerify(repo, cfg) {
|
|
|
3454
3518
|
return { status: "not written: the repo has no tracked files to cite; commit the repo's files, then re-run init", failed: true };
|
|
3455
3519
|
const record = join(home(), 'agents', `init-${new Date().toISOString().replace(/[:.]/g, '-')}-${agent.agent.replace(/[^\w-]/g, '-')}.json`);
|
|
3456
3520
|
const ships = shipped(repo);
|
|
3457
|
-
const
|
|
3521
|
+
const ai = detectAi([...context.files], (f) => repoText(join(repo, f)));
|
|
3522
|
+
const a = await askAgent({ ...agent, prompt: scaffoldPrompt(cfg, context.text, toolless, ships, ai), schema: scaffoldSchema(!!ai), cwd: repo, tools: toolless ? 'none' : 'read', record });
|
|
3458
3523
|
const where = a.record ? `; record ${a.record}` : a.recordError ? `; the call record could not be written: ${a.recordError}` : '';
|
|
3459
3524
|
if (!a.ok)
|
|
3460
3525
|
return { status: `not written: ${agent.agent} did not answer: ${a.error}; done-done doctor --probe checks the agent${where}`, failed: true };
|
|
3461
|
-
const problem = scaffoldProblem(a.answer, context.files, toolless ? context.read : context.files, ships);
|
|
3526
|
+
const problem = scaffoldProblem(a.answer, context.files, toolless ? context.read : context.files, ships, ai);
|
|
3462
3527
|
if (problem)
|
|
3463
3528
|
return { status: `not written: ${agent.agent}'s answer is unusable: ${problem}${where}`, failed: true };
|
|
3464
3529
|
const s = a.answer;
|
|
@@ -3475,7 +3540,7 @@ async function scaffoldVerify(repo, cfg) {
|
|
|
3475
3540
|
const files = new Map([[join('features', 'README.md'), verifyFeaturesMd(cfg.app, features, omitted)]]);
|
|
3476
3541
|
for (const f of features)
|
|
3477
3542
|
files.set(join('features', `${f.slug}.md`), verifyFeatureMd(cfg.app, s, f));
|
|
3478
|
-
files.set('SKILL.md', verifySkillMd(cfg.app, agent.agent, s, isolated, features, omitted, ships));
|
|
3543
|
+
files.set('SKILL.md', verifySkillMd(cfg.app, agent.agent, s, isolated, features, omitted, ships, ai));
|
|
3479
3544
|
const targets = [...files].map(([rel, body]) => ({ rel, p: join(dir, rel), body }));
|
|
3480
3545
|
link = linked(targets.map(({ p }) => p));
|
|
3481
3546
|
if (link !== undefined)
|
|
@@ -3531,7 +3596,7 @@ async function scaffoldVerify(repo, cfg) {
|
|
|
3531
3596
|
rmSync(tmp, { recursive: true, force: true });
|
|
3532
3597
|
}
|
|
3533
3598
|
catch { }
|
|
3534
|
-
return { status: `written by ${agent.agent}${isolated ? '' : '; its launch is not isolated, so read Launch before a drive'}` };
|
|
3599
|
+
return { status: `written by ${agent.agent}${isolated ? '' : '; its launch is not isolated, so read Launch before a drive'}${ai ? `; proposed ${s.behaviour.length} behaviour journeys (AI-output app: ${ai}), each accepted or dropped in its Behaviour journeys section` : ''}` };
|
|
3535
3600
|
}
|
|
3536
3601
|
// The session-start hook `init --hook <agent>` adds; the agent shows its stdout to the session.
|
|
3537
3602
|
// Without done-done on PATH it prints nothing and exits 0, so it never breaks a session.
|
package/dist/config.js
CHANGED
|
@@ -72,6 +72,7 @@ const KEYS = {
|
|
|
72
72
|
look: { scope: 'repo', type: 'look' },
|
|
73
73
|
evals: { scope: 'repo', type: 'evals' },
|
|
74
74
|
approved_design: { scope: 'repo', type: 'approved_design' },
|
|
75
|
+
approved_redesign: { scope: 'repo', type: 'approved_redesign' },
|
|
75
76
|
approved_requirements: { scope: 'repo', type: 'approved_requirements' },
|
|
76
77
|
interaction_checklist: { scope: 'repo', type: 'interaction_checklist' },
|
|
77
78
|
base_ref: { scope: 'repo', type: 'string' },
|
|
@@ -288,6 +289,28 @@ function apply(target, text, file, scope) {
|
|
|
288
289
|
target.from[key] = scope;
|
|
289
290
|
continue;
|
|
290
291
|
}
|
|
292
|
+
if (key === 'approved_redesign') {
|
|
293
|
+
const fail = () => { throw new UsageError(`${file}: approved_redesign must be a non-empty list of unique {screen, size: <width>x<height> in pixels, theme, capture: <unique PNG/JPEG filename>}`); };
|
|
294
|
+
const relativeImage = (p) => typeof p === 'string' && !isAbsolute(p) && !p.split(/[\\/]/).includes('..') && /\.(png|jpe?g)$/i.test(p);
|
|
295
|
+
if (!Array.isArray(value) || !value.length)
|
|
296
|
+
fail();
|
|
297
|
+
const rows = value;
|
|
298
|
+
const ids = new Set(), names = new Set();
|
|
299
|
+
for (const r of rows) {
|
|
300
|
+
if (!isMap(r) || Object.keys(r).length !== 4 || !['screen', 'size', 'theme', 'capture'].every(k => typeof r[k] === 'string' && r[k].trim()))
|
|
301
|
+
fail();
|
|
302
|
+
if (!/^[1-9]\d*x[1-9]\d*$/.test(r.size) || !relativeImage(r.capture) || /[\\/]/.test(r.capture))
|
|
303
|
+
fail();
|
|
304
|
+
const id = JSON.stringify([r.screen, r.size, r.theme]);
|
|
305
|
+
if (ids.has(id) || names.has(r.capture))
|
|
306
|
+
fail();
|
|
307
|
+
ids.add(id);
|
|
308
|
+
names.add(r.capture);
|
|
309
|
+
}
|
|
310
|
+
target.approved_redesign = rows;
|
|
311
|
+
target.from[key] = scope;
|
|
312
|
+
continue;
|
|
313
|
+
}
|
|
291
314
|
if (key === 'dom') {
|
|
292
315
|
const d = parseDom(value);
|
|
293
316
|
if (typeof d === 'string')
|
|
@@ -376,6 +399,7 @@ function defaultConfig(repo) {
|
|
|
376
399
|
evals: null,
|
|
377
400
|
approved_requirements: [],
|
|
378
401
|
approved_design: [],
|
|
402
|
+
approved_redesign: [],
|
|
379
403
|
interaction_checklist: null,
|
|
380
404
|
base_ref: '',
|
|
381
405
|
evidence: { local_root: join(home(), 'evidence'), root: '', allow_dirty: false, ignore_paths: [] },
|
package/dist/design.js
CHANGED
|
@@ -224,3 +224,39 @@ export async function design(cfg, choice, screens, records) {
|
|
|
224
224
|
}
|
|
225
225
|
return out;
|
|
226
226
|
}
|
|
227
|
+
const PAIR = 'The image "pair" shows two versions of one screen side by side, in an unknown order: one is the unchanged source and one is a redesign. Do not infer which half is which from position, or from any caption or label in the image. Describe what you see in each half first, then answer.';
|
|
228
|
+
export const PAIR_KINDS = {
|
|
229
|
+
'two-versions': { yes: 'The image shows two visually different versions of the same screen side by side.', no: 'The image does not show two versions of one screen.' },
|
|
230
|
+
complete: { yes: 'Both versions are complete, usable screens: no overlapping, clipped, squashed or unreadable UI and no blank band.', no: 'At least one version is broken: UI overlaps, is clipped, squashed or unreadable, or a blank band shows.' },
|
|
231
|
+
preserved: { yes: 'Both versions keep the same essential content, navigation entries and function, so a user can still do the same things.', no: 'Content, navigation or function is lost in one version.' },
|
|
232
|
+
};
|
|
233
|
+
const pairQuestions = (floor) => Object.fromEntries(Object.entries(PAIR_KINDS).map(([kind, k]) => [kind, { kind: 'choice', instructions: `Write the rationale first: describe each half of the image. ${PAIR} Question ${kind}.`, options: { yes: k.yes, no: k.no }, floor, personReason: true, images: ['pair'] }]));
|
|
234
|
+
export async function redesign(cfg, choice, screens, records) {
|
|
235
|
+
const dir = join(home(), 'runs', cfg.app, 'redesign');
|
|
236
|
+
const stamp = new Date().toISOString().replace(/[:.]/g, '-');
|
|
237
|
+
const out = [];
|
|
238
|
+
let be;
|
|
239
|
+
for (const [n, s] of screens.entries()) {
|
|
240
|
+
const shot = image(s);
|
|
241
|
+
if (typeof shot === 'string')
|
|
242
|
+
throw new UsageError(`redesign: ${shot}, so it cannot be sent to the judge`, 'Capture the side-by-side pair again as a PNG or JPEG, then re-run the check');
|
|
243
|
+
const q = pairQuestions(cfg.judge.floor);
|
|
244
|
+
const key = createHash('sha256').update(JSON.stringify([VERSION, s.sha256, q, choice.route, choice.model])).digest('hex');
|
|
245
|
+
const hit = readCache(dir, key);
|
|
246
|
+
const cached = hit?.route === choice.route && hit?.model === choice.model && Array.isArray(hit.failed) && hit.failed.every((d) => !!d && typeof d === 'object' && Object.hasOwn(PAIR_KINDS, d.kind) && typeof d.where === 'string' && !!d.where) && Array.isArray(hit.unsure) && hit.unsure.every((k) => typeof k === 'string' && Object.hasOwn(PAIR_KINDS, k));
|
|
247
|
+
let judge = hit;
|
|
248
|
+
if (!cached) {
|
|
249
|
+
const call = `${records}/redesign-${stamp}-${n + 1}`;
|
|
250
|
+
be ??= await backend(cfg, choice, (request) => `${call}-${request}.agent.json`, q, ASK_ATTEMPTS);
|
|
251
|
+
const answers = await askJudge(cfg, be, choice, { screen: 'a side-by-side pair of an unchanged source screen and its redesign' }, q, { pair: shot });
|
|
252
|
+
judge = {
|
|
253
|
+
failed: Object.entries(answers).filter(([, a]) => !a.abstained && a.answer === 'no').map(([kind, a]) => ({ kind, where: (a.personReason || a.rationale || 'not named').slice(0, 200) })),
|
|
254
|
+
unsure: Object.entries(answers).filter(([, a]) => a.abstained).map(([kind]) => kind),
|
|
255
|
+
};
|
|
256
|
+
writeCache(dir, key, { ...judge, route: choice.route, model: choice.model });
|
|
257
|
+
}
|
|
258
|
+
const reason = judge.unsure.length ? `not confident enough about ${judge.unsure.join(', ')}` : undefined;
|
|
259
|
+
out.push({ path: s.name, verdict: judge.failed.length ? 'fail' : reason ? 'unsure' : 'pass', failed: judge.failed, ...(reason && { reason }), ...(cached && { cached }) });
|
|
260
|
+
}
|
|
261
|
+
return out;
|
|
262
|
+
}
|
package/docs/reference.md
CHANGED
|
@@ -18,6 +18,8 @@ done-done init
|
|
|
18
18
|
|
|
19
19
|
`init` also has the agent the user set write the verify skill at `skill` (default `.agents/skills/done-done-verify-<app>`) from the repo's own files: plain code gives it the tracked file list and the text of the files that show how the repo proves itself (manifests, `.done-done.yaml` journeys, verify, e2e, smoke and demo scripts, `bin/` files, Dockerfiles, then source files (including Vue, Svelte, HTML, CSS and common compiled-language sources); JSON, YAML and Markdown are excluded except for manifests; at most 150,000 characters, and the prompt names what it left out; a `.env` or `.env.*` file, a file or folder whose path contains `secret`, `credential`, `token` or `password`, and `*.pem`, `*.key`, `*.p12` files never have their text sent by `init`, the agent is told not to open them, and no launch or feature can cite them; the tracked file list still names them), and one read-only call answers with the launch, readiness check, harness, evidence and up to five features (claude reads the repo with its read tools; every other agent gets only that text, and gemini cannot run it tool-less). Launch prefers the repo's own safe proof path, a stub or demo engine with its own state, and when the repo shows none, `SKILL.md` opens Launch with `Not isolated:` and what the launch would touch, and `verify_skill` says so. `Safe:` is written only when the model marks the launch safe and its source is a tracked file that is not a manifest such as `package.json` or `README.md`; any other launch is `Not isolated:`, and its steps are not parsed. Each feature names the repo file it comes from. The prompt says a test suite, typecheck, lint or build is never a launch, a drive or the proof, asks for exactly one feature marked `core` (the core user journey) and, when a tracked file shows the app ships an installable artifact, one marked `upgrade` (see [Core journey and release path](#core-journey-and-release-path)), and plain code refuses an answer with no core feature, more than one, or no upgrade feature for such an app unless the agent explicitly reports `first_release: true` (the repo says no earlier release yet; false when unknown). That first-release answer writes the skill with `Upgrade journey pending` in `Release proof`, so `maintain` and `affected` can use its core feature map without inventing an earlier version; it does not mark an upgrade journey or satisfy doctor's release check; `SKILL.md` gains a `Core journey` section (and a `Release proof` section for such an app) and `features/README.md` marks those entries `(core journey)` and `(upgrade journey)`. The prompt also says and plain code refuses an answer whose launch steps or a feature's drive is code-only ([Real-product evidence](#real-product-evidence)), as it refuses an answer that cites a file the repo does not track, a secret-looking file, or, for a tool-less agent, a tracked file whose text its prompt left out. It writes `SKILL.md`, plus `features/README.md` and one file per feature with numbered drive and capture steps; `features/README.md` says how many more were found and not written. With no agent set, no skill is written and `init` asks for the agent (exit 2); an agent that fails or answers with an uncited or missing field writes no skill, and `verify_skill` names the cause and the call record (exit 1). On a repo with nothing tracked yet (a fresh `git init`), no skill is written and no agent is asked — no answer could cite a tracked file — and `verify_skill` names the remedy: commit the repo's files, then re-run (exit 1). The write is all-or-nothing: when any target already exists — `SKILL.md`, `features/README.md` or a `features/<name>.md`, as a file or a directory — nothing is written and `verify_skill` says `skipped: <paths> already exist; delete the skill folder to regenerate`, so a re-run after a lost `SKILL.md` can never pair new answers with the old feature map; a finished skill is left byte-identical and a re-run asks the agent nothing. A symlink on the path skips the scaffold and reports why in `verify_skill`, with exit 0. Otherwise the files are written into a temporary directory inside the skill's parent and moved into place, so a failed write leaves no partial skill: the temporary directory and any moved file are removed, `init` exits 1, and `verify_skill` names the error and says to re-run `init`. The config is kept either way. A `skill` path with a `..` segment is refused (exit 2) before anything is written. A repo whose older `init` wrote `.agents/skills/verify-<app>` (and sets no `skill`) keeps working: every command reads that folder, and the next `init` moves it once to `.agents/skills/done-done-verify-<app>` (also updating the `name:` line in its `SKILL.md`) and reports `skill_renamed: <old> -> <new>`; when the prefixed folder already exists, the old one is left alone. Commit `.done-done.yaml` and the verify skill before the first capture: a capture from a tree with uncommitted changes fails `check`, and `init` says so in its help.
|
|
20
20
|
|
|
21
|
+
For an AI-output app, `init` also proposes a small behaviour eval inside that skill. The same detection as [Evals](#evals) (`detectAi`: a model provider SDK, a committed prompt file or prompt-building code in tracked files) decides; the guess is named in `verify_skill`. A detected app's scaffold call also asks for 3 to 6 basic behaviour journeys (not a suite): each is one thing a user does end to end on the real product that shows its model-driven behaviour is right (a tool the app calls, a decision it takes, an answer it shows), with what the user does, what they should see and the tracked file that shows it. Plain code refuses fewer than 3 or more than 6, a journey citing an untracked or secret-looking file, and a code-only step, as it does for features. `SKILL.md` gains a `Behaviour journeys (proposed)` section listing each as a line with its own `done-done journey add '<description>'` command. A proposal is not in the feature map or `.done-done.yaml`: the user accepts one by running its command (the configured agent writes the capture script and runs it once, as for any `journey add`) and drops one by deleting its line, and nothing runs until accepted. An accepted journey is captured from the real product and graded by the judge the user set (`judge.route`, `judge.model`), through the existing `verify`, `check` and `receipt` path; no judge is chosen for them. Any other repo's scaffold prompt, schema and skill are unchanged. `--no-skill` makes no agent call, so it proposes none.
|
|
22
|
+
|
|
21
23
|
```sh
|
|
22
24
|
done-done init --agent opencode --model opencode-go/muse-spark-1.3-contributor
|
|
23
25
|
```
|
|
@@ -601,6 +603,34 @@ approved_requirements:
|
|
|
601
603
|
|
|
602
604
|
**Behavior that tightens.** (1) Committed PNG/JPEG files under a top-level `design*/`, `mock*/`, `approved*/` or `prototype*/` folder that no `approved_design` `mock`, requirement `mock` or `kind: reference` row names fail every claim as "approved reference material is not in the approved_design manifest". (2) A claim whose outcome mentions design, mock, prototype or approved, with no `approved_design` manifest, no requirement `mock` and no `base_ref` baseline capture, fails with `not compared to approved result`; render, fit and dark-mode checks alone do not prove it. Manifests that list every reference and compare against it keep working unchanged.
|
|
603
605
|
|
|
606
|
+
#### Redesign claims
|
|
607
|
+
|
|
608
|
+
A redesign differs from its source on purpose, so it can never match an approved mock. A claim whose outcome says *redesign*, *redesigned*, *redesigns* or *redesigning* is a redesign claim: it is proven by a blind side-by-side judge against the unchanged source, together with the usual checks (run provenance, freshness, step record, relevance judge). Claims that implement an approved mock, such as "the board matches the approved design", keep [layout match](#approved-design-coverage) exactly as before, and a redesign claim does not use layout match.
|
|
609
|
+
|
|
610
|
+
List one row per screen × size × theme in `.done-done.yaml`:
|
|
611
|
+
|
|
612
|
+
```yaml
|
|
613
|
+
approved_redesign:
|
|
614
|
+
- { screen: board, size: 1280x800, theme: light, capture: board.light.pair.png }
|
|
615
|
+
- { screen: board, size: 1280x800, theme: dark, capture: board.dark.pair.png }
|
|
616
|
+
```
|
|
617
|
+
|
|
618
|
+
`capture` is the unique PNG/JPEG **filename** the redesign journey writes into its run folder: one image of the unchanged source and the redesign of that screen side by side, in either order. `size` is that image's width × height in pixels. Tuples and filenames must be unique, and a filename cannot contain a slash or `..`. The key only gates redesign claims.
|
|
619
|
+
|
|
620
|
+
Before any judge call, each row needs exactly one cited, run-bound image with that filename and pixel size. A missing, ambiguous or wrong-sized capture fails the claim (exit 1), naming the screen / size / theme. A redesign claim with no `approved_redesign` rows fails with `not compared to approved result` and a help line to add the manifest.
|
|
621
|
+
|
|
622
|
+
Each pair is then sent alone to the judge you set, as one tools-off request per screen. The judge is told the image shows two versions of one screen in an unknown order, one the unchanged source and one a redesign, not to infer either half from position or from a caption in the image, and to describe what it sees first. It answers three questions:
|
|
623
|
+
|
|
624
|
+
| Kind | `yes` means | `no` means |
|
|
625
|
+
|---|---|---|
|
|
626
|
+
| `two-versions` | the image shows two visually different versions of the same screen side by side | it is not two versions of one screen |
|
|
627
|
+
| `complete` | both versions are complete, usable screens: no overlapping, clipped, squashed or unreadable UI and no blank band | at least one version is broken |
|
|
628
|
+
| `preserved` | both versions keep the same essential content, navigation entries and function | content, navigation or function is lost |
|
|
629
|
+
|
|
630
|
+
All three `yes` is a pass. Any `no` fails the claim (exit 1), naming the kind and the judge's reason. An answer below `judge.floor` is unsure and makes the claim inconclusive (exit 3), naming the kinds; it is never a pass. Verdicts are cached per exact input (the image, the questions, the route, the model and the done-done version). No pixel measure runs on a pair, since the seam between the halves is not a defect.
|
|
631
|
+
|
|
632
|
+
The result's `redesign` list carries every row's screen, size, theme, capture path and hash, verdict and reason. Receipts keep it under `judge.approved_redesign`, bound to the cited capture; `receipt --verify` and `gate` check that every recorded verdict is a pass for a capture the receipt cites, without a model call, and refuse a malformed one. This proves the redesign is whole and keeps what the source had; it does not prove the redesign looks better.
|
|
633
|
+
|
|
604
634
|
#### Interaction checklist (opt-in)
|
|
605
635
|
|
|
606
636
|
Visual comparisons do not prove interaction polish. The package ships six requirements in `skills/done-done/interaction-checklist.json`: `detail-navigation` (next/previous **and swipe**), `instant-state` (immediate feedback on every send/answer), `destructive-undo` (undo restores content), `scroll-restoration` (back keeps list position), `list-search` (search/filter on long lists), and `empty-next-step` (empty states explain what to do next).
|
|
@@ -865,7 +895,8 @@ An `agent_config` entry merges with the defaults field by field, so `claude: { m
|
|
|
865
895
|
| `look` | none; the look step's approved design reference (`tokens`, `max_control`), see [Look check](#look-check) |
|
|
866
896
|
| `interaction_checklist` | none; opt-in six-item interaction evidence map; omitted/unproven items flagged, absent surfaces need reasons; see [Interaction checklist](#interaction-checklist-opt-in) |
|
|
867
897
|
| `approved_requirements` | none; approved assets, colors and interactions with the live capture each needs, or an explicit `exclusion`, see [Assets, colors, interactions](#assets-colors-interactions-and-unlisted-references) |
|
|
868
|
-
| `approved_design` | none; committed screen × pixel size × theme manifest of mocks and required live filenames; gates
|
|
898
|
+
| `approved_design` | none; committed screen × pixel size × theme manifest of mocks and required live filenames; gates every claim except a redesign claim, see [Approved-design coverage](#approved-design-coverage) |
|
|
899
|
+
| `approved_redesign` | none; screen × pixel size × theme manifest of live side-by-side source/redesign captures; gates only redesign claims, see [Redesign claims](#redesign-claims) |
|
|
869
900
|
| `inputs` | `[]`; repo-relative paths or globs the drive must leave alone, see [Input guard](#input-guard) |
|
|
870
901
|
| `base_ref` | none; a ref `verify` also drives, pairing its captures with the head's, see [Base-ref before](#base-ref-before) |
|
|
871
902
|
| `evidence.root` | `<local_root>/<app>`, outside the repo. A relative path is under the repo; an absolute path is used as-is |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "done-done-axi",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.21",
|
|
4
4
|
"license": "Apache-2.0",
|
|
5
5
|
"description": "Prove your app really works: done-done drives your app the way a user does, a fresh judge checks every capture, and the proof is tied to a commit.",
|
|
6
6
|
"type": "module",
|
|
@@ -77,6 +77,14 @@ with a surface-specific reason. `check`, `receipt` and `verify` flag omitted or
|
|
|
77
77
|
can pass. Static controls do not prove motion or timing. Capture the interaction, never substitute a mock. Without the key, approved-design behavior is unchanged;
|
|
78
78
|
`done-done check --help` and `docs/reference.md` explain wiring.
|
|
79
79
|
|
|
80
|
+
A claim that says an app, screen or flow is redesigned has no approved mock to match, so layout match cannot pass it. List
|
|
81
|
+
`approved_redesign` rows `{screen, size: <width>x<height>, theme, capture: <filename>}` in `.done-done.yaml`, each capture one live PNG/JPEG of
|
|
82
|
+
the unchanged source and the redesign side by side (either order), written by the redesign journey into its run. `check`, `receipt` and
|
|
83
|
+
`verify` then send each pair alone to the judge blind, asking whether it shows two versions, whether both are complete and whether
|
|
84
|
+
both keep the same content, navigation and function; a `no` fails naming the kind, below-floor is inconclusive, and a missing or wrong-size
|
|
85
|
+
capture fails before any judge call. A redesign claim with no `approved_redesign` fails as not compared to approved result. A claim that implements
|
|
86
|
+
an approved mock keeps layout match. Receipts keep the pass under `judge.approved_redesign`; `gate` re-checks it with no model call.
|
|
87
|
+
|
|
80
88
|
To gate a worker's claim cheaply, `done-done lint <PR URL | report file> <evidence folder>` reads the newest `## Checks` block
|
|
81
89
|
with no judge or model and prints `PASS` or `FAIL: <reason>` (exit 1 on a fail): missing lines, evidence that is absent,
|
|
82
90
|
empty, stale, duplicated or already used by another PR, a n/a repro on a fix, or a PR of 1,000+ added lines all fail.
|
|
@@ -89,6 +97,7 @@ receipt it wrote (`<evidence root>/.done-done/receipts/<head>.json`) into `.done
|
|
|
89
97
|
|
|
90
98
|
`init` also has that agent write the repo's `.agents/skills/done-done-verify-<app>/SKILL.md` (never overwriting one): read it before
|
|
91
99
|
you drive the app, and fix it when it is wrong.
|
|
100
|
+
For an AI-output app that skill has a `Behaviour journeys (proposed)` section: show the user each journey and let them accept it (run its `journey add` command) or drop it (delete its lines); never keep one unasked, and the judge that grades it is the one the user chose.
|
|
92
101
|
|
|
93
102
|
done-done starts a coding agent the user chose. When it says no agent is set, ask the user which agent
|
|
94
103
|
(and model) to use and save their answer with the command it prints; never choose one for them.
|