done-done-axi 0.4.6 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -6
- package/dist/cli.js +59 -10
- package/dist/proof.js +99 -0
- package/docs/index.html +1 -0
- package/docs/reference.md +21 -8
- package/integrations/firstmate/brief-include.md +5 -4
- package/package.json +1 -1
- package/skills/done-done/SKILL.md +5 -3
package/README.md
CHANGED
|
@@ -120,29 +120,29 @@ cd your-app
|
|
|
120
120
|
done-done init
|
|
121
121
|
```
|
|
122
122
|
|
|
123
|
-
`init` writes a minimal `.done-done.yaml` (only if you don't already have one) that names your app, and a commented global `~/.done-done/config.yaml` (only if there is none). done-done never picks the coding agent it starts: `init` asks you in a terminal, or takes `--agent <agent> --model <model>`. It never picks the judge either, the agent of its own that `check`, `receipt`, `verify` and `design` always start to look at each capture: `init` asks in a terminal, or takes `--judge <agent> --judge-model <model>`. Any agent and model you choose runs, your coding agent's own too (the judge is a fresh session of its own). The config keys are in the [reference](docs/reference.md#config). A plain `init` never overwrites an existing config. When your coding agent is set, `init` also has it write a verify skill, `.agents/skills/done-done-verify-<app>/SKILL.md`, from the repo's own files (never overwriting one; with no agent set, no skill is written; an older `verify-<app>` skill is renamed to this once). Check its launch never touches real accounts or your own state. Evidence lives outside the repo, in `~/.done-done/evidence/<app>`, so it outlives the checkout that captured it. For an `answerer:pi` judge with no explicit model, done-done uses `openai-codex/gpt-6-luna`; `judge.model` or `agent_config.pi.model` overrides it. Then describe how to capture proof — swap the command for whatever drives your real app and saves screenshots, recordings or
|
|
123
|
+
`init` writes a minimal `.done-done.yaml` (only if you don't already have one) that names your app, and a commented global `~/.done-done/config.yaml` (only if there is none). done-done never picks the coding agent it starts: `init` asks you in a terminal, or takes `--agent <agent> --model <model>`. It never picks the judge either, the agent of its own that `check`, `receipt`, `verify` and `design` always start to look at each capture: `init` asks in a terminal, or takes `--judge <agent> --judge-model <model>`. Any agent and model you choose runs, your coding agent's own too (the judge is a fresh session of its own). The config keys are in the [reference](docs/reference.md#config). A plain `init` never overwrites an existing config. When your coding agent is set, `init` also has it write a verify skill, `.agents/skills/done-done-verify-<app>/SKILL.md`, from the repo's own files (never overwriting one; with no agent set, no skill is written; an older `verify-<app>` skill is renamed to this once). Check its launch never touches real accounts or your own state. Evidence lives outside the repo, in `~/.done-done/evidence/<app>`, so it outlives the checkout that captured it. For an `answerer:pi` judge with no explicit model, done-done uses `openai-codex/gpt-6-luna`; `judge.model` or `agent_config.pi.model` overrides it. Then describe how to capture proof — swap the command for whatever drives your real app the way a user does and saves its screenshots, recordings or output (here a CLI's own output):
|
|
124
124
|
|
|
125
125
|
```yaml
|
|
126
126
|
journeys:
|
|
127
127
|
smoke:
|
|
128
|
-
command: npm
|
|
128
|
+
command: npm start > $DONE_DONE_EVIDENCE_ROOT/${RUN_ID}.log
|
|
129
129
|
```
|
|
130
130
|
|
|
131
|
-
The command runs from the repo root with `DONE_DONE_EVIDENCE_ROOT` (a new folder for this run in the evidence root) and `DONE_DONE_RUN_ID` set, and `${RUN_ID}` in it is replaced. A run that writes nothing into its folder fails.
|
|
131
|
+
The command runs from the repo root with `DONE_DONE_EVIDENCE_ROOT` (a new folder for this run in the evidence root) and `DONE_DONE_RUN_ID` set, and `${RUN_ID}` in it is replaced. A run that writes nothing into its folder fails. A journey that only runs code checks (`npm test`, `yarn run check`, `tsc`, `pytest`, a linter or a build) is code-only: its log says the code checks pass, not that the product works, so `init --journey` refuses it and `check`, `receipt` and `gate` never count it as proof ([the rule](docs/reference.md#real-product-evidence)).
|
|
132
132
|
|
|
133
133
|
Optional: `done-done init --hook <claude|codex|opencode|gemini|pi>` adds a session hook or plugin to the repo that shows each session the bare `done-done` state. Each agent's file and behavior are in the [reference](docs/reference.md#set-up-a-repo).
|
|
134
134
|
|
|
135
135
|
Record the recipe in the repo so anyone can re-run the proof, then commit it (and `.claude/settings.json`, `.codex/hooks.json`, `.gemini/settings.json`, `.opencode/plugins/done-done.js` or `.pi/extensions/done-done.js` if you added a hook):
|
|
136
136
|
|
|
137
137
|
```sh
|
|
138
|
-
done-done init --journey smoke -- 'npm
|
|
138
|
+
done-done init --journey smoke -- 'npm start > $DONE_DONE_EVIDENCE_ROOT/${RUN_ID}.log'
|
|
139
139
|
git add .done-done.yaml
|
|
140
140
|
git commit -m "Add done-done"
|
|
141
141
|
```
|
|
142
142
|
|
|
143
143
|
A `run -- <command>` whose command matches no recorded journey still runs, but its result hints that the recipe is not in the repo. For a proof you drive by hand or by your agent instead of a journey, `done-done run --open` prints a run id and a folder to capture into, and `done-done run --close <run-id>` seals it ([how](docs/reference.md#run-a-journey)).
|
|
144
144
|
|
|
145
|
-
At any point, `done-done doctor` checks the setup and changes nothing except the `--probe` call record. Each item reports `ok`, `warning` or `missing` with the exact fix command; an agent you have not set yet is `missing`, with the command that saves your choice. `done-done doctor --probe` also asks the agent one question to prove it answers; on a new machine that probe fails until the agent is signed in (for pi, run `pi` and then `/login`). The quick start above never calls the coding agent, so `run` and `check` work before it is signed in; `check` does call the judge, so set one first. The items are listed in the [reference](docs/reference.md#check-the-setup).
|
|
145
|
+
At any point, `done-done doctor` checks the setup and changes nothing except the `--probe` call record. Each item reports `ok`, `warning` or `missing` with the exact fix command; an agent you have not set yet is `missing`, with the command that saves your choice, and so is a config whose journeys only run code checks. `done-done doctor --probe` also asks the agent one question to prove it answers; on a new machine that probe fails until the agent is signed in (for pi, run `pi` and then `/login`). The quick start above never calls the coding agent, so `run` and `check` work before it is signed in; `check` does call the judge, so set one first. The items are listed in the [reference](docs/reference.md#check-the-setup).
|
|
146
146
|
|
|
147
147
|
`done-done maintain` keeps the verify skill true: it checks the skill's links and feature index, checks that every `file:line` it cites exists, and has your coding agent read the source for each feature, read-only, to report drift. It ends `clean`, `drift` with the findings to fix, or `blocked`. It never runs without that reader: an agent that is not set asks, and one that cannot start or cannot read without tools stops it before the run (exit 2). With `auto_fix: { maintain: <n> }` (as in no-mistakes; 0, the default, reports only), your judge first sorts each drift finding (a skill out of date, or a product that may be wrong, which stops and asks you), then your agent fixes the skill folder only, in a done-done worktree, and maintain runs again, up to n times. A clean end is `changed`: one commit on a `done-done/maintain-<run-id>` branch, your branch untouched ([reference](docs/reference.md#keep-the-verify-skill-true)).
|
|
148
148
|
|
|
@@ -172,6 +172,7 @@ Run `done-done` with no arguments to see the current config and the next step.
|
|
|
172
172
|
|
|
173
173
|
## What it checks
|
|
174
174
|
|
|
175
|
+
- 🧪 **The real product, not code checks**: Proof must come from driving the app, CLI or site the way a user does. A config whose journeys only run tests, typecheck, lint or a build, or proof whose captures all come from such runs, is refused by `check`, `receipt` and `gate`, and `doctor` flags it ([rule](docs/reference.md#real-product-evidence)).
|
|
175
176
|
- 🔗 **Tied to a commit**: Every run records the commit, and whether there were uncommitted changes.
|
|
176
177
|
- 🖼️ **Real media**: Empty, cut-off, blank or tiny screenshots fail. Videos must hold real frames (needs `ffprobe`).
|
|
177
178
|
- ⏱️ **Fresh**: Proof from an older commit fails. Commits that only touch docs can be set to keep it fresh.
|
|
@@ -182,7 +183,7 @@ Run `done-done` with no arguments to see the current config and the next step.
|
|
|
182
183
|
- 🎨 **Looks right**: `done-done design <screens...> [--mock <screen>=<mock>]` has the same judge look at each captured screen, alone or next to its approved mock, and fails it on stretched or squashed images, clipped or squashed text, overlap, elements off-screen, blank or black strips and broken layout, naming each defect and where it is ([how](docs/reference.md#design-check)).
|
|
183
184
|
- 📐 **Fits the screen**: `done-done look <url>` opens a web page in headless Chrome at a 390x844 phone viewport, captures it, and fails the journey that calls it on squashed text, clipped text, elements cut off by an edge or a page wider than the screen, naming each element, with no model ([how](docs/reference.md#look-check)).
|
|
184
185
|
- 🧮 **Honest, cheap claim gate**: `done-done lint <PR URL | report file> <evidence folder>` is a no-model gate on a worker's `## Checks` block, on whenever you call it. It prints `PASS` or `FAIL: <reason>` and exits 1 on a fail: the four lines present, proof naming a command or path, a real repro on a fix PR, evidence files that exist, are non-empty, are newer than the evidence folder and differ from each other, at least one never used by another PR (hashes kept in `~/.done-done/lint.hashes`, every run logged to `lint.log` there), under 1,000 added non-generated lines, and a `WARN` when a PR changes `src/` with no `journeys/` file ([how](docs/reference.md#lint-check)). `evidence: not proven: <cause>` passes.
|
|
185
|
-
- 🚦 **CI gate, no model**: `done-done gate [<head>]` passes when a receipt committed under `.done-done-receipts/` covers the head, with no judge, config, key or evidence files. CI never runs `check`, `run` or `verify` and never gets a model key; proof runs on your desktop ([how](docs/reference.md#ci-gate)).
|
|
186
|
+
- 🚦 **CI gate, no model**: `done-done gate [<head>]` passes when a receipt committed under `.done-done-receipts/` covers the head and its captures come from real-product runs, with no judge, config, key or evidence files. CI never runs `check`, `run` or `verify` and never gets a model key; proof runs on your desktop ([how](docs/reference.md#ci-gate)).
|
|
186
187
|
- 🧩 **Page is clean** (with a `dom` block): each captured screen also needs its page-check record. A page that overflows the viewport, logs a console error, has a failed request or shows squashed, clipped or cut-off UI fails, unless an allow list names it ([how](docs/reference.md#dom-assertions)).
|
|
187
188
|
|
|
188
189
|
Every answer is an exit code: **0** pass, **1** fail, **2** usage or config error (including no agent or judge set, a check that fails before its judge, a capture the judge cannot be sent, or a judge that did not answer: never a result), **3** can't tell either way, **4** a `verify`, `maintain` or `rerun` run is still running, or a `verify` run waits at its gate.
|
package/dist/cli.js
CHANGED
|
@@ -21,6 +21,7 @@ import { domIssues, domRecord, isDomRecord, lookIssues, SQUASHED_TEXT } from './
|
|
|
21
21
|
import { lint } from './lint.js';
|
|
22
22
|
import { look } from './look.js';
|
|
23
23
|
import { scanSecrets } from './secrets.js';
|
|
24
|
+
import { codeOnly, codeOnlyJourneys, configFault, RULE, runsFault } from './proof.js';
|
|
24
25
|
const PASS = 0, FAIL = 1, USAGE = 2, INCONCLUSIVE = 3, WAITING = 4;
|
|
25
26
|
// The shape of a journey, so a new user or agent can write one without opening the reference.
|
|
26
27
|
// done-done sets DONE_DONE_EVIDENCE_ROOT (the run's own folder) and replaces ${RUN_ID}, so the example works for any evidence root.
|
|
@@ -28,6 +29,8 @@ const journeyExample = 'journeys: { smoke: { command: "<drive the real app> > $D
|
|
|
28
29
|
const JOURNEYS_TEXT = `journeys (.done-done.yaml): the command runs with DONE_DONE_EVIDENCE_ROOT (this run's own new folder in the evidence
|
|
29
30
|
root) and DONE_DONE_RUN_ID set, and \${RUN_ID} in it is replaced; it must write its files into that folder, e.g.
|
|
30
31
|
${journeyExample}
|
|
32
|
+
real product: a journey whose command only runs tests, typecheck, lint or a build (npm test, yarn run check, tsc, pytest) is code-only;
|
|
33
|
+
init --journey refuses it, and check, receipt and gate refuse proof that comes only from code-only runs or a config of only code-only journeys
|
|
31
34
|
spends (journeys.<name>.spends): what a run spends, e.g. "one live generation"; verify parks the journey until the user approves that run (respond --action approve)
|
|
32
35
|
timeout (journeys.<name>.timeout): how long a verify run may drive the journey, e.g. 90s or 30m (default 30m); past it the journey is killed and the run is inconclusive
|
|
33
36
|
matrix (.done-done.yaml): with theme, width or form axes the journey runs once per required cell with DONE_DONE_THEME, DONE_DONE_WIDTH, DONE_DONE_FORM and DONE_DONE_CELL (the capture name suffix) set; name each capture with its suffix, so every cell is captured or skipped with a reason`;
|
|
@@ -307,12 +310,17 @@ function cap(items) {
|
|
|
307
310
|
return FULL || items.length <= COMPACT_LIST ? { items, omitted: 0 } : { items: items.slice(0, COMPACT_LIST), omitted: items.length - COMPACT_LIST };
|
|
308
311
|
}
|
|
309
312
|
// A hint that names the exact command that moves the work forward, naming the journey when there is one.
|
|
313
|
+
const RECORD_REAL = "done-done init --journey <name> -- '<command that drives the real app and saves screenshots, a recording or its transcript into $DONE_DONE_EVIDENCE_ROOT>'";
|
|
314
|
+
// A code-only journey is never named: its run is no proof.
|
|
310
315
|
function captureHint(cfg) {
|
|
311
|
-
const
|
|
316
|
+
const code = codeOnlyJourneys(cfg.journeys);
|
|
317
|
+
const names = Object.keys(cfg.journeys).filter((n) => !code.includes(n));
|
|
312
318
|
if (names.length === 1)
|
|
313
319
|
return `done-done run ${names[0]}`;
|
|
314
320
|
if (names.length > 1)
|
|
315
321
|
return `done-done run <journey> (${names.join(', ')})`;
|
|
322
|
+
if (code.length)
|
|
323
|
+
return `${RECORD_REAL} (${code.join(', ')} only run${code.length > 1 ? '' : 's'} code checks), commit, then done-done run <name>`;
|
|
316
324
|
return 'done-done run -- <capture command>';
|
|
317
325
|
}
|
|
318
326
|
// The words after --, as the one shell command string a run executes. One word is already a whole command,
|
|
@@ -446,6 +454,9 @@ function closeRun(repo, cfg, rec, log) {
|
|
|
446
454
|
return { code: reason ? FAIL : PASS, result, reason };
|
|
447
455
|
}
|
|
448
456
|
async function cmdRun(argv) {
|
|
457
|
+
const literal = argv.find((a) => /\$\{?DONE_DONE_EVIDENCE_ROOT\b/.test(a));
|
|
458
|
+
if (literal)
|
|
459
|
+
throw new UsageError(`the argument ${JSON.stringify(literal)} holds the literal text $DONE_DONE_EVIDENCE_ROOT: the shell did not expand it (single quotes?), so it would name a folder with that name. Use the env var inside the script, or double quotes`);
|
|
449
460
|
const dash = argv.indexOf('--');
|
|
450
461
|
const { values, positionals } = parseArgs({
|
|
451
462
|
args: dash < 0 ? argv : argv.slice(0, dash),
|
|
@@ -674,6 +685,8 @@ async function runCheck(argv, run) {
|
|
|
674
685
|
const hashes = [];
|
|
675
686
|
const cited = [];
|
|
676
687
|
const captures = [];
|
|
688
|
+
// The run behind each cited capture, for the real-product rule and the receipt.
|
|
689
|
+
const runs = new Map();
|
|
677
690
|
// base_ref: a capture in <run id>/before/ of a run that drove its base ref pairs with <run id>/<the same name>.
|
|
678
691
|
const pairOf = (inRoot) => {
|
|
679
692
|
const [id, dir, ...rest] = inRoot?.split(sep) ?? [];
|
|
@@ -741,12 +754,15 @@ async function runCheck(argv, run) {
|
|
|
741
754
|
findings.push({ level: 'inconclusive', why: `${name}: not in any done-done run record, so it is not tied to a commit` });
|
|
742
755
|
continue;
|
|
743
756
|
}
|
|
744
|
-
//
|
|
745
|
-
|
|
757
|
+
// Identical bytes (a deterministic capture) are claimed by every run that wrote them; a run at the current commit is enough.
|
|
758
|
+
// With none, records from before runs had their own folders can both claim one file; never guess which run wrote it.
|
|
759
|
+
const atNow = claims.filter((r) => r.commit === now.commit);
|
|
760
|
+
if (!atNow.length && claims.length > 1) {
|
|
746
761
|
findings.push({ level: 'inconclusive', why: `${name}: claimed by more than one run (${claims.map((r) => r.run_id).join(', ')}), so it is not tied to one commit` });
|
|
747
762
|
continue;
|
|
748
763
|
}
|
|
749
|
-
const rec = claims
|
|
764
|
+
const rec = (atNow.length ? atNow : claims).reduce((a, b) => (b.started_at > a.started_at ? b : a));
|
|
765
|
+
runs.set(rec.run_id, { run_id: rec.run_id, journey: rec.journey, command: rec.command });
|
|
750
766
|
// Only a capture tied to a run record can reach the judge, so a stray cited file is never sent.
|
|
751
767
|
const c = { name, bytes, sha256: hash, media: v };
|
|
752
768
|
if (v.status === 'ok' && !record && pair)
|
|
@@ -772,6 +788,13 @@ async function runCheck(argv, run) {
|
|
|
772
788
|
if (hasMatrix(cfg.matrix))
|
|
773
789
|
for (const c of matrixManifest(cfg.matrix, files.filter((f) => (!cfg.dom || !isDomRecord(f)) && !pairOf(under(cfg.evidence.root, cite(f))))).cells.filter((c) => !c.files.length))
|
|
774
790
|
findings.push({ level: 'fail', why: `cell ${c.cell} has no capture: name one file *${c.suffix}* (journeys name captures with DONE_DONE_CELL)` });
|
|
791
|
+
// Real product: a config of only code-only journeys, or captures that all come from code-only runs, are no proof.
|
|
792
|
+
const codeConfig = configFault(cfg.journeys);
|
|
793
|
+
if (codeConfig)
|
|
794
|
+
findings.push({ level: 'fail', why: codeConfig });
|
|
795
|
+
const codeRuns = runs.size ? runsFault([...runs.values()]) : null;
|
|
796
|
+
if (codeRuns)
|
|
797
|
+
findings.push({ level: 'fail', why: codeRuns });
|
|
775
798
|
// Reuse: fail only when every cited file was already used for a different claim id. The ledger lives in the
|
|
776
799
|
// evidence root next to the run records it guards: one per app (repo), shared by every checkout of it, never in git.
|
|
777
800
|
const ledger = join(cfg.evidence.root, '.done-done', 'ledger');
|
|
@@ -832,7 +855,7 @@ async function runCheck(argv, run) {
|
|
|
832
855
|
mkdirSync(dirname(ledger), { recursive: true });
|
|
833
856
|
appendFileSync(ledger, hashes.filter((h) => !used.get(h)?.has(claimId)).map((h) => `${h} ${claimId}\n`).join(''));
|
|
834
857
|
}
|
|
835
|
-
return { code: rep.code, result: rep.result, judge: judged, judgeBlock, claim: claim?.outcome, skill: claim?.skill, files, cited, head: now, findings };
|
|
858
|
+
return { code: rep.code, result: rep.result, judge: judged, judgeBlock, claim: claim?.outcome, skill: claim?.skill, files, cited, runs: [...runs.values()], head: now, findings };
|
|
836
859
|
}
|
|
837
860
|
async function cmdCheck(argv) {
|
|
838
861
|
const r = await runCheck(argv);
|
|
@@ -857,7 +880,8 @@ function writeReceipt(cfg, rec) {
|
|
|
857
880
|
renameSync(`${path}.tmp`, path);
|
|
858
881
|
return path;
|
|
859
882
|
}
|
|
860
|
-
// Why a parsed receipt is not a well-formed passing one (a judge block that names its judge and passes only captures the receipt cites
|
|
883
|
+
// Why a parsed receipt is not a well-formed passing one (a judge block that names its judge and passes only captures the receipt cites,
|
|
884
|
+
// and capture runs that are not all code-only), or null.
|
|
861
885
|
function receiptFault(rec, short) {
|
|
862
886
|
if (!rec || typeof rec !== 'object' || rec.schema !== 2 || rec.result !== 'pass' || !Array.isArray(rec.files) || !rec.files.length)
|
|
863
887
|
return `receipt for ${short} is malformed`;
|
|
@@ -866,7 +890,10 @@ function receiptFault(rec, short) {
|
|
|
866
890
|
const j = rec.judge;
|
|
867
891
|
const badJudge = !j || typeof j !== 'object' || typeof j.route !== 'string' || typeof j.model !== 'string' || !['agent', 'model', 'session-only'].includes(j.independence) || !Array.isArray(j.verdicts) || !j.verdicts.length
|
|
868
892
|
|| !j.verdicts.every((v) => v && v.verdict === 'pass' && rec.files.some((f) => f.path === v.path && f.sha256 === v.sha256));
|
|
869
|
-
|
|
893
|
+
if (badJudge)
|
|
894
|
+
return `receipt for ${short} has a malformed judge block`;
|
|
895
|
+
const code = runsFault(rec.runs);
|
|
896
|
+
return code ? `receipt for ${short} is no real-product proof: ${code}` : null;
|
|
870
897
|
}
|
|
871
898
|
// Verify a recorded receipt for `headArg`, which may be a full sha or a ref (HEAD, a branch or a tag).
|
|
872
899
|
// Read-only, so a missing or changed record is a failure, not a usage error.
|
|
@@ -955,7 +982,7 @@ function recordReceipt(repo, cfg, r) {
|
|
|
955
982
|
// A missing citation is not a verdict: the judge passed, but the receipt cannot bind it, so it stops like a judge error.
|
|
956
983
|
if (!r.cited.length)
|
|
957
984
|
throw new UsageError('receipt: the check cited no evidence files, so no receipt was written; a receipt must bind at least one cited evidence file', 'Cite real proof: done-done receipt --outcome "<claim>" <files...>');
|
|
958
|
-
const rec = { schema: 2, repo: cfg.app, head: r.head.commit, tree: r.head.tree, claim: r.claim ?? '', ...(r.skill && { skill: r.skill }), result: 'pass', at: new Date().toISOString(), files: r.cited, judge: r.judgeBlock };
|
|
985
|
+
const rec = { schema: 2, repo: cfg.app, head: r.head.commit, tree: r.head.tree, claim: r.claim ?? '', ...(r.skill && { skill: r.skill }), result: 'pass', at: new Date().toISOString(), files: r.cited, runs: r.runs, judge: r.judgeBlock };
|
|
959
986
|
const path = writeReceipt(cfg, rec);
|
|
960
987
|
const result = { command: 'receipt', result: 'pass', head: r.head.commit, tree: r.head.tree, claim: rec.claim, ...(rec.skill && { skill: rec.skill }), files: rec.files.length, receipt: shown(repo, path) };
|
|
961
988
|
return { code: PASS, result: { ...result, ...(r.judge && { judge: r.judge }), help: [`Verify this receipt (merge gate): done-done receipt --verify ${r.head.commit}`] } };
|
|
@@ -1003,7 +1030,10 @@ function checkHelp(code, findings, files, { cfg, repo, claim, source, citationBa
|
|
|
1003
1030
|
? `${absent} named file${absent > 1 ? 's are' : ' is'} not in this checkout. If the evidence is in another folder, check against it: ${form}`
|
|
1004
1031
|
: `${absent} named file${absent > 1 ? 's are' : ' is'} not under --citation-base ${arg(citationBase)} either: check the paths in the Checks block`);
|
|
1005
1032
|
}
|
|
1006
|
-
|
|
1033
|
+
// Proof refused as code-only needs a journey that drives the product, not a re-capture of the same run.
|
|
1034
|
+
if (findings.some((f) => f.why.endsWith(RULE)))
|
|
1035
|
+
help.push(`Capture the real product: ${captureHint(cfg).startsWith('done-done init') ? captureHint(cfg) : `${captureHint(cfg)}, or ${RECORD_REAL}`}, then cite its files`);
|
|
1036
|
+
else if (findings.some((f) => f.level === 'fail' && f.missing !== 'checkout'))
|
|
1007
1037
|
help.push(`Commit your change, re-capture with ${capture}, then check the new files`);
|
|
1008
1038
|
return help;
|
|
1009
1039
|
}
|
|
@@ -2536,6 +2566,7 @@ function scaffoldPrompt(cfg, context, toolless) {
|
|
|
2536
2566
|
return `You write the verify skill for the app ${cfg.app}, which \`done-done init\` saves at ${cfg.skill}: how an agent launches this repo's app safely, drives it the way a user does, and captures proof that it works. Do not edit, create or delete any file, and do not run the app or any command.
|
|
2537
2567
|
${toolless ? "You have no tools: the repo's tracked file list and the text of its key files are below, and they are all you can read." : 'The repository is your working directory: read the files below first, then any other file you need.'} Use only what the repo shows: never invent a command, path, flag, route or label. Never open a secret-looking file: a .env or .env.* file, a path whose name contains secret, credential, token or password, or a *.pem, *.key or *.p12 file. Never write a secret value into steps or why; name the environment variable instead.
|
|
2538
2568
|
|
|
2569
|
+
A proof drives the running product the way a user does: a test suite, typecheck, lint or build (npm test, yarn run check, tsc, pytest, go test) only checks the code, so it is never a launch, a drive or the proof, even when the repo's own checks use it.
|
|
2539
2570
|
- launch: how to start the app for a proof. Prefer the repo's own proof path: a verify, e2e, smoke or demo script, or a journey in .done-done.yaml, that runs the app with a stub, fake or demo engine and its own state (a temporary HOME or data folder), the way the repo proves itself; copy its exact command and environment flags. {steps: the exact command on one line, with every environment flag, whose first command runs source (or passes it to an interpreter such as bash, node or npx tsx), source: the repo file they come from (for a .done-done.yaml journey, the script or file that journey runs, never .done-done.yaml itself), safe: true only when these steps use a stub or demo mode or isolated state, so no real account, paid service or the owner's own state is touched, why: what makes it safe, or when safe is false what it would touch}. When the repo has no such path, give the closest launch with safe false; never present a command that uses real accounts or the owner's own state (for example a start script that runs the real engine) as safe.
|
|
2540
2571
|
- surface: what a user drives: web UI, CLI, API/service, desktop app or other.
|
|
2541
2572
|
- ready: the check that shows the launched app is ready to drive (a URL that answers, a line it prints, an exit code).
|
|
@@ -2568,6 +2599,8 @@ function scaffoldProblem(a, tracked, shown) {
|
|
|
2568
2599
|
const launchCite = cited(s.launch.source) ? citation(s.launch.source) : '';
|
|
2569
2600
|
if (launchCite)
|
|
2570
2601
|
return `the launch ${launchCite}`;
|
|
2602
|
+
if (codeOnly(s.launch.steps))
|
|
2603
|
+
return `the launch ${s.launch.steps.trim()} is code-only: ${RULE}`;
|
|
2571
2604
|
if (!Array.isArray(s.features) || !s.features.length)
|
|
2572
2605
|
return 'has no features';
|
|
2573
2606
|
for (const f of s.features) {
|
|
@@ -2576,6 +2609,8 @@ function scaffoldProblem(a, tracked, shown) {
|
|
|
2576
2609
|
const why = citation(f.source);
|
|
2577
2610
|
if (why)
|
|
2578
2611
|
return `the feature "${f.title}" ${why}`;
|
|
2612
|
+
if (codeOnly(f.drive))
|
|
2613
|
+
return `the feature "${f.title}" drive ${f.drive.trim()} is code-only: ${RULE}`;
|
|
2579
2614
|
}
|
|
2580
2615
|
return '';
|
|
2581
2616
|
}
|
|
@@ -3104,6 +3139,8 @@ async function cmdInit(argv) {
|
|
|
3104
3139
|
throw new UsageError(`init --journey ${journey} needs a command after --`);
|
|
3105
3140
|
if (journey !== null && !/^[A-Za-z0-9][A-Za-z0-9_-]*$/.test(journey))
|
|
3106
3141
|
throw new UsageError(`invalid journey name ${journey}; use letters, digits, - or _`);
|
|
3142
|
+
if (journey !== null && codeOnly(commandOf(parts)))
|
|
3143
|
+
throw new UsageError(`init --journey ${journey}: ${commandOf(parts)} is code-only: ${RULE}, so check and gate never count its run as proof; nothing was written`, `Record a command that drives the real app, CLI or site the way a user does and saves screenshots, a recording or its transcript into $DONE_DONE_EVIDENCE_ROOT: done-done init --journey ${journey} -- '<that command>'`);
|
|
3107
3144
|
// Only the judge named: that is a machine setting, so it changes the global config alone, in any directory.
|
|
3108
3145
|
if (values.judge !== undefined && values.agent === undefined && values.hook === undefined && journey === null) {
|
|
3109
3146
|
const cfg = loadConfig(process.cwd());
|
|
@@ -3472,7 +3509,19 @@ async function cmdDoctor(argv) {
|
|
|
3472
3509
|
}
|
|
3473
3510
|
if (agent)
|
|
3474
3511
|
checks.push({ check: 'agent_settings', status: 'ok', detail: agentSettings(agentCfg, agent.agent), fix: '' });
|
|
3475
|
-
|
|
3512
|
+
// Real product: a journey that only runs code checks is no proof, so a config of only those can never pass check or gate.
|
|
3513
|
+
if (!cfg || !repo) {
|
|
3514
|
+
checks.push(unknownRow('real_product'));
|
|
3515
|
+
}
|
|
3516
|
+
else {
|
|
3517
|
+
const names = Object.keys(cfg.journeys);
|
|
3518
|
+
const code = codeOnlyJourneys(cfg.journeys);
|
|
3519
|
+
const fix = `Record a journey that drives the real app and saves screenshots, a recording or its transcript into $DONE_DONE_EVIDENCE_ROOT: done-done init --journey <name> -- '<command>'; then delete ${code.join(', ')} from .done-done.yaml`;
|
|
3520
|
+
checks.push(configFault(cfg.journeys) ? { check: 'real_product', status: 'missing', detail: configFault(cfg.journeys), fix }
|
|
3521
|
+
: code.length ? { check: 'real_product', status: 'warning', detail: `${code.join(', ')} ${code.length > 1 ? 'are' : 'is'} code-only (${RULE}): proof only from ${code.length > 1 ? 'them' : 'it'} is refused; ${names.filter((n) => !code.includes(n)).join(', ')} drive${names.length - code.length > 1 ? '' : 's'} the product`, fix: `Cite captures from the journeys that drive the product; delete ${code.join(', ')} from .done-done.yaml if it is not needed` }
|
|
3522
|
+
: { check: 'real_product', status: 'ok', detail: names.length ? `${names.join(', ')} drive${names.length > 1 ? '' : 's'} the product` : 'no journeys: verify has the agent drive the verify skill\'s features', fix: '' });
|
|
3523
|
+
}
|
|
3524
|
+
const required = ['git', 'config', 'evidence_root', 'real_product', 'agent', 'judge', ...(values.probe ? ['probe'] : [])];
|
|
3476
3525
|
const missingRequired = checks.filter((c) => c.status === 'missing' && required.includes(c.check));
|
|
3477
3526
|
const result = { command: 'doctor', result: missingRequired.length ? 'fail' : 'pass', checks };
|
|
3478
3527
|
// Several missing items can share one fix (set up the config once); name each fix only once.
|
package/dist/proof.js
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
// The one rule for real-product evidence, shared by init, doctor, check, receipt and gate. It is model-free and reads
|
|
2
|
+
// the command text only: a command is code-only when every step in it is a test, typecheck, lint or build runner, or
|
|
3
|
+
// shell plumbing around one, and at least one step is such a runner. Its output says the code checks pass, never that
|
|
4
|
+
// the product works. Any other step (a capture script, done-done look, the app's own binary, curl, adb) drives the product.
|
|
5
|
+
// Steps that neither drive the product nor check the code: they move files and text around a runner.
|
|
6
|
+
const PLUMBING = new Set(['cd', 'echo', 'printf', 'cat', 'tee', 'mkdir', 'cp', 'mv', 'rm', 'touch', 'ln', 'true', 'false', ':', 'export', 'set', 'unset', 'ls', 'date', 'sleep', 'test', '[', 'wc', 'grep', 'sed', 'awk', 'head', 'tail', 'sort', 'uniq', 'tr', 'cut', 'exit', 'source', '.', 'pwd', 'wait']);
|
|
7
|
+
// Prefixes that run the next word: stripped (with their own flags) before the step is read.
|
|
8
|
+
const WRAPPERS = new Set(['env', 'time', 'nice', 'nohup', 'exec', 'command', 'cross-env', 'npx', 'bunx', 'pnpx', 'xvfb-run', 'sh', 'bash', 'zsh', 'timeout']);
|
|
9
|
+
const RUN_PREFIX = { pnpm: ['exec', 'dlx'], yarn: ['exec', 'dlx'], npm: ['exec'], uv: ['run'], poetry: ['run'], pipenv: ['run'], bundle: ['exec'], python: ['-m'], python3: ['-m'] };
|
|
10
|
+
// Tools that are a code check whatever their arguments.
|
|
11
|
+
const RUNNERS = new Set(['tsc', 'vue-tsc', 'eslint', 'prettier', 'biome', 'oxlint', 'stylelint', 'jest', 'vitest', 'mocha', 'ava', 'tap', 'jasmine', 'karma', 'pytest', 'py.test', 'tox', 'nox', 'mypy', 'pyright', 'ruff', 'flake8', 'pylint', 'black', 'isort', 'unittest', 'rspec', 'rubocop', 'phpunit', 'phpstan', 'golangci-lint', 'gofmt', 'shellcheck', 'hadolint', 'ktlint', 'detekt', 'swiftlint']);
|
|
12
|
+
// Gradle tasks that test, lint or build (connectedAndroidTest, testDebugUnitTest, lintRelease, assembleDebug), with or without a :module: prefix.
|
|
13
|
+
const GRADLE = /^(.*:)?((test|check|lint|build|assemble)\w*|\w*Test)$/;
|
|
14
|
+
// Tools that are a code check with one of these subcommands (playwright test is, playwright screenshot is not).
|
|
15
|
+
const SUBCOMMANDS = {
|
|
16
|
+
node: /^--test$/, deno: /^(test|check|lint|fmt)$/, go: /^(test|vet|build)$/, cargo: /^(test|check|clippy|fmt|build|nextest)$/,
|
|
17
|
+
make: /^(test|tests|check|lint|typecheck|build)$/, just: /^(test|tests|check|lint|typecheck|build)$/,
|
|
18
|
+
gradle: GRADLE, './gradlew': GRADLE, gradlew: GRADLE, mvn: /^(test|verify|package)$/,
|
|
19
|
+
dotnet: /^(test|build)$/, swift: /^(test|build)$/, flutter: /^(test|analyze)$/, dart: /^(test|analyze)$/, xcodebuild: /^test$/, mix: /^test$/,
|
|
20
|
+
playwright: /^test$/, cypress: /^run$/,
|
|
21
|
+
};
|
|
22
|
+
// A package script by name: test, check, lint, typecheck, build... and their variants (test:unit, check-types).
|
|
23
|
+
const SCRIPT = /^(t|tst|tests?|checks?|lint|lints|typecheck|type-check|types|tsc|format|fmt|prettier|eslint|build|coverage|validate|unit|spec|vet)([:._-].*)?$/i;
|
|
24
|
+
const INSTALL = /^(i|install|ci|add)$/;
|
|
25
|
+
const PACKAGE_MANAGERS = new Set(['npm', 'yarn', 'pnpm', 'bun']);
|
|
26
|
+
function stepKind(step) {
|
|
27
|
+
let words = step.replace(/["'`(){}]/g, ' ').split(/\s+/).filter(Boolean);
|
|
28
|
+
for (;;) {
|
|
29
|
+
while (words.length && /^\w+=/.test(words[0]))
|
|
30
|
+
words = words.slice(1);
|
|
31
|
+
if (!words.length)
|
|
32
|
+
return 'plumbing';
|
|
33
|
+
const [w, next] = words;
|
|
34
|
+
if (WRAPPERS.has(w)) {
|
|
35
|
+
words = words.slice(1);
|
|
36
|
+
// A wrapper's own flags (npx -y, sh -c, timeout 60s) are not the command it runs.
|
|
37
|
+
while (words.length > 1 && (words[0].startsWith('-') || (w === 'timeout' && /^\d/.test(words[0]))))
|
|
38
|
+
words = words.slice(1);
|
|
39
|
+
continue;
|
|
40
|
+
}
|
|
41
|
+
if (RUN_PREFIX[w]?.includes(next)) {
|
|
42
|
+
words = words.slice(2);
|
|
43
|
+
continue;
|
|
44
|
+
}
|
|
45
|
+
break;
|
|
46
|
+
}
|
|
47
|
+
const [w, ...rest] = words;
|
|
48
|
+
const base = w.split('/').pop();
|
|
49
|
+
if (PLUMBING.has(base))
|
|
50
|
+
return 'plumbing';
|
|
51
|
+
if (RUNNERS.has(base))
|
|
52
|
+
return 'runner';
|
|
53
|
+
if (PACKAGE_MANAGERS.has(base)) {
|
|
54
|
+
const args = rest.filter((a) => !a.startsWith('-'));
|
|
55
|
+
if (!args.length || INSTALL.test(args[0]))
|
|
56
|
+
return 'plumbing';
|
|
57
|
+
const script = ['run', 'run-script'].includes(args[0]) ? args[1] : args[0];
|
|
58
|
+
return script && SCRIPT.test(script) ? 'runner' : 'product';
|
|
59
|
+
}
|
|
60
|
+
const sub = SUBCOMMANDS[w] ?? SUBCOMMANDS[base];
|
|
61
|
+
// node takes --test among its flags before the script; every other tool names its subcommand first.
|
|
62
|
+
const first = rest.findIndex((a) => !a.startsWith('-'));
|
|
63
|
+
const said = base === 'node' ? rest.slice(0, first < 0 ? rest.length : first) : first < 0 ? [] : [rest[first]];
|
|
64
|
+
if (sub)
|
|
65
|
+
return said.some((a) => sub.test(a)) ? 'runner' : 'product';
|
|
66
|
+
return 'product';
|
|
67
|
+
}
|
|
68
|
+
// True when the shell command only runs code checks (tests, typecheck, lint, build) and plumbing around them.
|
|
69
|
+
export function codeOnly(command) {
|
|
70
|
+
if (!command)
|
|
71
|
+
return false;
|
|
72
|
+
const steps = command
|
|
73
|
+
.replace(/&>>?\s*[^\s;&|]+|\d*>>?\s*&?\s*[^\s;&|]+|<\s*[^\s;&|]+/g, ' ')
|
|
74
|
+
.split(/&&|\|\||[;|\n&]/)
|
|
75
|
+
.map((s) => s.trim())
|
|
76
|
+
.filter(Boolean);
|
|
77
|
+
const kinds = steps.map(stepKind);
|
|
78
|
+
return kinds.includes('runner') && kinds.every((k) => k !== 'product');
|
|
79
|
+
}
|
|
80
|
+
export const RULE = 'a test, typecheck, lint or build run proves the code checks pass, not that the product works';
|
|
81
|
+
// The configured journeys whose command is code-only.
|
|
82
|
+
export const codeOnlyJourneys = (journeys) => Object.keys(journeys).filter((n) => codeOnly(journeys[n].command));
|
|
83
|
+
// Why the config's journeys capture no real-product evidence (every journey is code-only), or null. No journeys is fine:
|
|
84
|
+
// verify then has the agent drive the verify skill's features.
|
|
85
|
+
export function configFault(journeys) {
|
|
86
|
+
const names = Object.keys(journeys);
|
|
87
|
+
const code = codeOnlyJourneys(journeys);
|
|
88
|
+
return names.length && code.length === names.length ? `every journey in .done-done.yaml is code-only (${code.join(', ')}): ${RULE}` : null;
|
|
89
|
+
}
|
|
90
|
+
// Why these capture runs are no real-product evidence (none recorded, or every one code-only), or null.
|
|
91
|
+
export function runsFault(runs) {
|
|
92
|
+
if (!Array.isArray(runs) || !runs.length)
|
|
93
|
+
return 'it records no capture runs, so nothing shows its proof came from the real product';
|
|
94
|
+
if (!runs.every((r) => r && typeof r === 'object' && typeof r.run_id === 'string'))
|
|
95
|
+
return 'its capture runs are malformed';
|
|
96
|
+
if (!runs.every((r) => codeOnly(r.command)))
|
|
97
|
+
return null;
|
|
98
|
+
return `every cited capture comes from a code-only run (${[...new Set(runs.map((r) => r.journey ?? r.command))].join(', ')}): ${RULE}`;
|
|
99
|
+
}
|
package/docs/index.html
CHANGED
|
@@ -106,6 +106,7 @@
|
|
|
106
106
|
<section>
|
|
107
107
|
<h2>What it checks</h2>
|
|
108
108
|
<ul class="plain">
|
|
109
|
+
<li><b>The real product, not code checks.</b> Proof comes from driving the app the way a user does; a journey that only runs tests, typecheck or lint is refused.</li>
|
|
109
110
|
<li><b>Tied to a commit.</b> Every run records the commit, and whether there were uncommitted changes.</li>
|
|
110
111
|
<li><b>Real media.</b> Empty, cut-off, blank or tiny screenshots fail. Videos must hold real frames.</li>
|
|
111
112
|
<li><b>Fresh, and not reused.</b> Proof from an older commit fails, and so does proof reused for a different claim.</li>
|
package/docs/reference.md
CHANGED
|
@@ -16,7 +16,7 @@ done-done init
|
|
|
16
16
|
|
|
17
17
|
`init` writes a minimal `.done-done.yaml` only when one does not exist: just `app`, the repo directory name, so every checkout of the repo finds the same evidence (see [Where proof lives](#where-proof-lives)). It sets no `evidence.root`, so evidence goes outside the repo. When an existing config puts `evidence.root` inside the repo (an older `root: .proof`), `init` adds it to `.gitignore` when git does not already ignore it. A plain `init` never rewrites the config: re-running is a no-op and a customized `.done-done.yaml` is left byte-for-byte; `.gitignore` gains a line only when the ignore entry is missing. A malformed `.done-done.yaml` makes `init` exit 2 with the file named and untouched. Its result says whether the config was `written` or left `unchanged`, the evidence root, what it added to `.gitignore`, the verify skill outcome (`verify_skill`), and the next step.
|
|
18
18
|
|
|
19
|
-
`init` also has the agent the user set write the verify skill at `skill` (default `.agents/skills/done-done-verify-<app>`) from the repo's own files: plain code gives it the tracked file list and the text of the files that show how the repo proves itself (manifests, `.done-done.yaml` journeys, verify, e2e, smoke and demo scripts, `bin/` files, Dockerfiles, then source files (including Vue, Svelte, HTML, CSS and common compiled-language sources); JSON, YAML and Markdown are excluded except for manifests; at most 150,000 characters, and the prompt names what it left out; a `.env` or `.env.*` file, a file or folder whose path contains `secret`, `credential`, `token` or `password`, and `*.pem`, `*.key`, `*.p12` files never have their text sent by `init`, the agent is told not to open them, and no launch or feature can cite them; the tracked file list still names them), and one read-only call answers with the launch, readiness check, harness, evidence and up to five features (claude reads the repo with its read tools; every other agent gets only that text, and codex and gemini cannot run it tool-less). Launch prefers the repo's own safe proof path, a stub or demo engine with its own state, and when the repo shows none, `SKILL.md` opens Launch with `Not isolated:` and what the launch would touch, and `verify_skill` says so. `Safe:` is written only when the model marks the launch safe and its source is a tracked file that is not a manifest such as `package.json` or `README.md`; any other launch is `Not isolated:`, and its steps are not parsed. Each feature names the repo file it comes from, and plain code refuses an answer that cites a file the repo does not track, a secret-looking file, or, for a tool-less agent, a tracked file whose text its prompt left out. It writes `SKILL.md`, plus `features/README.md` and one file per feature with numbered drive and capture steps; `features/README.md` says how many more were found and not written. With no agent set, no skill is written and `init` asks for the agent (exit 2); an agent that fails or answers with an uncited or missing field writes no skill, and `verify_skill` names the cause and the call record (exit 1). On a repo with nothing tracked yet (a fresh `git init`), no skill is written and no agent is asked — no answer could cite a tracked file — and `verify_skill` names the remedy: commit the repo's files, then re-run (exit 1). The write is all-or-nothing: when any target already exists — `SKILL.md`, `features/README.md` or a `features/<name>.md`, as a file or a directory — nothing is written and `verify_skill` says `skipped: <paths> already exist; delete the skill folder to regenerate`, so a re-run after a lost `SKILL.md` can never pair new answers with the old feature map; a finished skill is left byte-identical and a re-run asks the agent nothing. A symlink on the path skips the scaffold and reports why in `verify_skill`, with exit 0. Otherwise the files are written into a temporary directory inside the skill's parent and moved into place, so a failed write leaves no partial skill: the temporary directory and any moved file are removed, `init` exits 1, and `verify_skill` names the error and says to re-run `init`. The config is kept either way. A `skill` path with a `..` segment is refused (exit 2) before anything is written. A repo whose older `init` wrote `.agents/skills/verify-<app>` (and sets no `skill`) keeps working: every command reads that folder, and the next `init` moves it once to `.agents/skills/done-done-verify-<app>` (also updating the `name:` line in its `SKILL.md`) and reports `skill_renamed: <old> -> <new>`; when the prefixed folder already exists, the old one is left alone. Commit `.done-done.yaml` and the verify skill before the first capture: a capture from a tree with uncommitted changes fails `check`, and `init` says so in its help.
|
|
19
|
+
`init` also has the agent the user set write the verify skill at `skill` (default `.agents/skills/done-done-verify-<app>`) from the repo's own files: plain code gives it the tracked file list and the text of the files that show how the repo proves itself (manifests, `.done-done.yaml` journeys, verify, e2e, smoke and demo scripts, `bin/` files, Dockerfiles, then source files (including Vue, Svelte, HTML, CSS and common compiled-language sources); JSON, YAML and Markdown are excluded except for manifests; at most 150,000 characters, and the prompt names what it left out; a `.env` or `.env.*` file, a file or folder whose path contains `secret`, `credential`, `token` or `password`, and `*.pem`, `*.key`, `*.p12` files never have their text sent by `init`, the agent is told not to open them, and no launch or feature can cite them; the tracked file list still names them), and one read-only call answers with the launch, readiness check, harness, evidence and up to five features (claude reads the repo with its read tools; every other agent gets only that text, and codex and gemini cannot run it tool-less). Launch prefers the repo's own safe proof path, a stub or demo engine with its own state, and when the repo shows none, `SKILL.md` opens Launch with `Not isolated:` and what the launch would touch, and `verify_skill` says so. `Safe:` is written only when the model marks the launch safe and its source is a tracked file that is not a manifest such as `package.json` or `README.md`; any other launch is `Not isolated:`, and its steps are not parsed. Each feature names the repo file it comes from. The prompt says a test suite, typecheck, lint or build is never a launch, a drive or the proof, and plain code refuses an answer whose launch steps or a feature's drive is code-only ([Real-product evidence](#real-product-evidence)), as it refuses an answer that cites a file the repo does not track, a secret-looking file, or, for a tool-less agent, a tracked file whose text its prompt left out. It writes `SKILL.md`, plus `features/README.md` and one file per feature with numbered drive and capture steps; `features/README.md` says how many more were found and not written. With no agent set, no skill is written and `init` asks for the agent (exit 2); an agent that fails or answers with an uncited or missing field writes no skill, and `verify_skill` names the cause and the call record (exit 1). On a repo with nothing tracked yet (a fresh `git init`), no skill is written and no agent is asked — no answer could cite a tracked file — and `verify_skill` names the remedy: commit the repo's files, then re-run (exit 1). The write is all-or-nothing: when any target already exists — `SKILL.md`, `features/README.md` or a `features/<name>.md`, as a file or a directory — nothing is written and `verify_skill` says `skipped: <paths> already exist; delete the skill folder to regenerate`, so a re-run after a lost `SKILL.md` can never pair new answers with the old feature map; a finished skill is left byte-identical and a re-run asks the agent nothing. A symlink on the path skips the scaffold and reports why in `verify_skill`, with exit 0. Otherwise the files are written into a temporary directory inside the skill's parent and moved into place, so a failed write leaves no partial skill: the temporary directory and any moved file are removed, `init` exits 1, and `verify_skill` names the error and says to re-run `init`. The config is kept either way. A `skill` path with a `..` segment is refused (exit 2) before anything is written. A repo whose older `init` wrote `.agents/skills/verify-<app>` (and sets no `skill`) keeps working: every command reads that folder, and the next `init` moves it once to `.agents/skills/done-done-verify-<app>` (also updating the `name:` line in its `SKILL.md`) and reports `skill_renamed: <old> -> <new>`; when the prefixed folder already exists, the old one is left alone. Commit `.done-done.yaml` and the verify skill before the first capture: a capture from a tree with uncommitted changes fails `check`, and `init` says so in its help.
|
|
20
20
|
|
|
21
21
|
```sh
|
|
22
22
|
done-done init --agent opencode --model opencode-go/muse-spark-1.3-contributor
|
|
@@ -30,7 +30,7 @@ Record a journey's capture recipe in the repo so any checkout can re-run the pro
|
|
|
30
30
|
done-done init --journey smoke -- 'echo added: milk > $DONE_DONE_EVIDENCE_ROOT/${RUN_ID}.log'
|
|
31
31
|
```
|
|
32
32
|
|
|
33
|
-
`init --journey <name> -- <command>` writes `journeys.<name>.command` into `.done-done.yaml` (creating the file with the pinned `app` when missing) and reports `written` or `updated` with the journey name. `${RUN_ID}` in the command becomes the run id, exactly as in `run`. The recipe is committed, so an absolute path inside the repo is refused (exit 2, nothing written) with the relative form to use; name the path relative to the repo, or use a placeholder such as `${B:?}` that the caller sets. Commit the config: proof only counts from a clean tree.
|
|
33
|
+
`init --journey <name> -- <command>` writes `journeys.<name>.command` into `.done-done.yaml` (creating the file with the pinned `app` when missing) and reports `written` or `updated` with the journey name. `${RUN_ID}` in the command becomes the run id, exactly as in `run`. The recipe is committed, so an absolute path inside the repo is refused (exit 2, nothing written) with the relative form to use; name the path relative to the repo, or use a placeholder such as `${B:?}` that the caller sets. A code-only command (one that only runs tests, typecheck, lint or a build, see [Real-product evidence](#real-product-evidence)) is refused the same way, before anything is written, with the shape of a command to record instead. Commit the config: proof only counts from a clean tree.
|
|
34
34
|
|
|
35
35
|
```sh
|
|
36
36
|
done-done init --hook claude
|
|
@@ -70,7 +70,7 @@ done-done init --hook pi
|
|
|
70
70
|
done-done doctor
|
|
71
71
|
```
|
|
72
72
|
|
|
73
|
-
`doctor` answers "is this repo worth driving" and changes nothing except the `--probe` call record (the agent itself may keep its own session files). It checks
|
|
73
|
+
`doctor` answers "is this repo worth driving" and changes nothing except the `--probe` call record (the agent itself may keep its own session files). It checks ten items, each reported `ok`, `warning` or `missing` with the exact fix command (plus a `tool_gate` `warning` row when the agent is pi or codex; see [Agents](#agents)): `git` (inside a repository), `config` (a valid `.done-done.yaml`), `evidence_root` (writable), `verify_skill` (`.agents/skills/done-done-verify-<app>/SKILL.md`), `agent` (the coding agent the user set and its model; `missing` when none is set, with the command that saves the user's choice, or when it is not installed or done-done cannot start it yet; see [Agents](#agents)), `agent_settings` (each effective agent setting, `agent`, its model, effort, how many extra flags are set (never their values), binary and `agent_timeout`, and where it came from: the global config, `.done-done.yaml` or `default`), `judge` (`missing` when no judge is set, with the command that saves the user's choice; else the judge kit is present, and the route's own setup: for `claude-code` an executable `claude` binary (`agent_path_override.claude`, else PATH), with your own sign-in checked only when a capture is judged; for `answerer:<agent>` the agent on PATH; for `openai:key` the `OPENAI_API_KEY` variable and a `judge.model`; for `minimax:key` the `MINIMAX_API_KEY` variable; doctor does not call the model; it is checked even with no agent set), `ffprobe` (video evidence only) and `judge_choice` (the route and model the judge runs and the author it judges, see [Judge relevance](#judge-relevance): `ok`, naming how far apart they are (`agent`, `model` or `session-only`), since any judge the user sets runs; `missing`, with no fix of its own, until the agent is set; `missing` when no judge is set or the judge config is incomplete, such as a key route with no `judge.model`) and `real_product` (whether the journeys drive the product, see [Real-product evidence](#real-product-evidence): `ok` naming them, or with no journeys, since `verify` then drives the verify skill; a `warning` naming code-only journeys beside real ones; `missing` when every journey is code-only, with the command that records a real one). `git`, `config`, `evidence_root`, `real_product`, `agent` and `judge` are required, since done-done gives no result without its agent and judge, and refuses all proof from a code-only config: `doctor` exits 0 when they are all ok and 2 otherwise. A missing `verify_skill` or `ffprobe`, or a `warning`, never fails it. In a fresh repo with nothing set up, `doctor` names each missing piece, starting with `done-done init`.
|
|
74
74
|
|
|
75
75
|
```sh
|
|
76
76
|
done-done doctor --probe
|
|
@@ -247,6 +247,17 @@ base_ref: origin/main
|
|
|
247
247
|
|
|
248
248
|
done-done checks the base ref out in a scratch git worktree of its own under the run folder (never your checkout) and removes it after. The same journey or agent drive runs there with the same `DONE_DONE_RUN_ID`, its own instance (when an `instance` block is set) and every matrix cell's env, and `DONE_DONE_EVIDENCE_ROOT` is the run folder's `before/`, so a journey must write its captures there, not to a path it builds itself. An agent drive is told it is at the base ref, where the outcome is not expected to hold. The run record keeps `base` (the ref and its commit). Each head capture pairs with the base capture of the same name, matrix cell suffix included (`home.dark.png` with `before/home.dark.png`): the judge gets both in one request (two images, or two texts for a text capture) and passes it only when the after capture shows the outcome as a change from before; a base capture is never judged against the outcome alone, and it may match its head capture byte for byte (the judge then sees no change). A head capture with no base capture of its name is reported (inconclusive, exit 3, with why the base drive failed when it did) and judged alone. Without `base_ref` (the default) nothing changes: each capture is judged alone.
|
|
249
249
|
|
|
250
|
+
### Real-product evidence
|
|
251
|
+
|
|
252
|
+
done-done proves the real product works, not that its code checks pass. One model-free rule, on a command's text alone, decides which runs count; `init`, `doctor`, `check`, `receipt` and `gate` all use it.
|
|
253
|
+
|
|
254
|
+
A command is **code-only** when every step in it (split on `&&`, `||`, `;`, `|` and newlines, with redirections dropped and wrappers such as `npx`, `env`, `VAR=value`, `sh -c`, `pnpm exec`, `uv run`, `bundle exec` and `python -m` read through) is a test, typecheck, lint or build runner or shell plumbing around one (`cd`, `echo`, `tee`, `mkdir`, `cp`, `cat`, `npm ci`...), and at least one step is a runner. The runners are a package script named `test`, `check`, `lint`, `typecheck`, `types`, `tsc`, `build`, `format`, `coverage`, `validate` and their variants (`npm test`, `yarn run check`, `pnpm test:unit`, `bun test`), and tools such as `tsc`, `eslint`, `prettier`, `biome`, `jest`, `vitest`, `mocha`, `node --test`, `playwright test`, `cypress run`, `pytest`, `tox`, `mypy`, `ruff`, `go test`, `cargo test`, `make test`, `gradle test`, `mvn test`, `dotnet test`, `flutter test`, `deno test`, `rspec` and `phpunit`. Any other step (a capture script, `done-done look`, the app's own binary, `npm start`, `curl`, `adb`, `playwright screenshot`) drives the product, so a command with one is not code-only. A run opened by hand (`run --open`) or driven by the agent (`verify` with no journey) has no command and is never code-only.
|
|
255
|
+
|
|
256
|
+
- `init --journey` refuses a code-only command (exit 2, nothing written), and `init`'s verify skill refuses a code-only launch or feature drive.
|
|
257
|
+
- `check` and `receipt` stop before the judge (exit 2) when every journey in `.done-done.yaml` is code-only, naming them, or when every cited capture comes from a code-only run, naming its journey or command. A test log cited beside real captures is fine: they carry the proof.
|
|
258
|
+
- A receipt records the run behind each cited capture (`runs`), so `receipt --verify` and `gate` apply the same rule with no config: a receipt whose runs are all code-only, or one with no `runs` (written before this rule), fails.
|
|
259
|
+
- `doctor`'s `real_product` row is `missing` (and doctor exits 2) when every journey is code-only, and a `warning` naming the code-only journeys beside real ones.
|
|
260
|
+
|
|
250
261
|
### Check a claim
|
|
251
262
|
|
|
252
263
|
```sh
|
|
@@ -286,7 +297,9 @@ evidence: <paths of the evidence files, or "not proven: <cause>">
|
|
|
286
297
|
|
|
287
298
|
A plain check is never a verdict on its own. A cited file that fails one of these checks, or a Checks block that breaks a form rule, stops the check before its judge is asked: it exits 2 with every cause in `error` and the next command in `help`, and gives no result. A check with no cited capture the judge reads (only video or other files, or none from a done-done run) stops the same way, before the judge is asked.
|
|
288
299
|
|
|
289
|
-
Files that are in no run record give an inconclusive result when another cited capture is one the judge reads: nothing ties them to a commit. So does a file that more than one run record claims (possible only with records from before each run had its own folder). A file whose path is named in a run record but whose bytes no longer match fails as `changed after run <run-id>` (exit 2, before the judge), not inconclusive: the path ties it to that run, so editing it after the run is tampering, not untied evidence.
|
|
300
|
+
Files that are in no run record give an inconclusive result when another cited capture is one the judge reads: nothing ties them to a commit. So does a file that more than one run record claims when none of those runs is at the current commit (possible only with records from before each run had its own folder). Byte-identical captures from several runs are fine when one of those runs is at the current commit: that run ties the file to the commit. A file whose path is named in a run record but whose bytes no longer match fails as `changed after run <run-id>` (exit 2, before the judge), not inconclusive: the path ties it to that run, so editing it after the run is tampering, not untied evidence.
|
|
301
|
+
|
|
302
|
+
A config whose journeys are all code-only, or cited captures that all come from code-only runs, stop the check the same way, before the judge, naming the journeys ([Real-product evidence](#real-product-evidence)).
|
|
290
303
|
|
|
291
304
|
`evidence: not proven: <cause>` stops the check before its judge is asked, always: exit 2, no result, no receipt. `checks.not_proven` is gone: a config that still sets it is a config error (exit 2) that says to delete the line.
|
|
292
305
|
|
|
@@ -380,9 +393,9 @@ done-done receipt --from pr-body.md # or --outcome "<claim>" <files...>, o
|
|
|
380
393
|
done-done receipt --verify <head> # read-only: check a recorded receipt
|
|
381
394
|
```
|
|
382
395
|
|
|
383
|
-
`receipt` takes the same claim inputs as `check` and runs that check. It writes `<root>/.done-done/receipts/<head>.json` only when the check passes on the current HEAD, the tracked tree is clean, and the check cited at least one evidence file; a dirty tracked tree, a failed or inconclusive check, a pass that cited no evidence, or a moved HEAD writes nothing and exits non-zero. The record holds the repo, the head and tree sha, the claim outcome,
|
|
396
|
+
`receipt` takes the same claim inputs as `check` and runs that check. It writes `<root>/.done-done/receipts/<head>.json` only when the check passes on the current HEAD, the tracked tree is clean, and the check cited at least one evidence file; a dirty tracked tree, a failed or inconclusive check, a pass that cited no evidence, or a moved HEAD writes nothing and exits non-zero. The record holds the repo, the head and tree sha, the claim outcome, every cited file with its sha256, the run behind each cited capture (`runs`: run id, journey and command, the command `null` for a run opened by hand or driven by the agent) and a timestamp. Every receipt is schema 2 and holds a `judge` block: the route, model, independence and author, and each judged file's path, sha256 and verdict. A cited file is stored relative to the repo, or as `<evidence root>/<path>` when it is under an evidence root outside the repo, so the receipt still finds it from another checkout or a restored evidence root.
|
|
384
397
|
|
|
385
|
-
`receipt --verify <head>` changes nothing. The argument may be a full commit sha or a ref (`HEAD`, a branch or a tag), resolved to the commit with `git rev-parse --verify <arg>^{commit}` before the receipt lookup; an unresolvable argument fails with the same missing-receipt shape. It succeeds only when the receipt exists, is PASS, the current HEAD is still that head, and every cited file still hashes to the recorded sha256. A receipt is written only after a judge pass, so it reads schema 2 only: any other receipt (one recorded without the judge, or by an older build that let a run past its gate) fails as malformed, and a schema 2 receipt needs a well-formed `judge` block whose every verdict is `pass` for a cited file with the same sha256, which the result shows. A receipt from `--from` (or a `verify` run that drove the skill) also records the verify skill that ran as `skill` (for example `done-done-verify-<app>`), which `--verify` and so the merge gate show; an older receipt without it still verifies. A missing or malformed receipt and a hash mismatch are failures (exit 1), not usage errors. When the mismatched path is named in a run record, the failure names it as `changed after run <run-id>`.
|
|
398
|
+
`receipt --verify <head>` changes nothing. The argument may be a full commit sha or a ref (`HEAD`, a branch or a tag), resolved to the commit with `git rev-parse --verify <arg>^{commit}` before the receipt lookup; an unresolvable argument fails with the same missing-receipt shape. It succeeds only when the receipt exists, is PASS, the current HEAD is still that head, and every cited file still hashes to the recorded sha256. A receipt is written only after a judge pass, so it reads schema 2 only: any other receipt (one recorded without the judge, or by an older build that let a run past its gate) fails as malformed, and a schema 2 receipt needs a well-formed `judge` block whose every verdict is `pass` for a cited file with the same sha256, which the result shows, and `runs` that are not all code-only ([Real-product evidence](#real-product-evidence)); a receipt with no `runs`, written before that rule, fails: re-run `receipt` for the head. A receipt from `--from` (or a `verify` run that drove the skill) also records the verify skill that ran as `skill` (for example `done-done-verify-<app>`), which `--verify` and so the merge gate show; an older receipt without it still verifies. A missing or malformed receipt and a hash mismatch are failures (exit 1), not usage errors. When the mismatched path is named in a run record, the failure names it as `changed after run <run-id>`.
|
|
386
399
|
|
|
387
400
|
### Verify a change in one run
|
|
388
401
|
|
|
@@ -486,7 +499,7 @@ Running `done-done` with no subcommand prints the repo's config state (`config:
|
|
|
486
499
|
| 3 | inconclusive (could not be proven either way) |
|
|
487
500
|
| 4 | the run is still running, or a `verify` run waits at its gate (`verify`, `maintain --no-wait`, `rerun`, `status`, `respond`) |
|
|
488
501
|
|
|
489
|
-
`doctor` exits 2 when a required item (`git`, `config`, `evidence_root`, `agent`, `judge`, and `probe` with `--probe`) is missing.
|
|
502
|
+
`doctor` exits 2 when a required item (`git`, `config`, `evidence_root`, `real_product`, `agent`, `judge`, and `probe` with `--probe`) is missing.
|
|
490
503
|
|
|
491
504
|
### Merge gate on the desktop
|
|
492
505
|
|
|
@@ -515,7 +528,7 @@ npx -y done-done-axi gate "$PR_HEAD_SHA"
|
|
|
515
528
|
|
|
516
529
|
The receipt is committed in a follow-up commit, because a receipt cannot sit inside the commit it proves (its hash would change). `gate [<head>] [--dir <folder>] [--pushed <time>]` prints `PASS` or `FAIL: <reason>` and exits 0, 1, or 2 on a usage error. It passes when a receipt in `.done-done-receipts/` (or `--dir`):
|
|
517
530
|
|
|
518
|
-
- is a well-formed pass from a judged run (the same shape `receipt --verify` requires) and is filed as `<its head>.json`;
|
|
531
|
+
- is a well-formed pass from a judged run (the same shape `receipt --verify` requires, so its `runs` are not all code-only, see [Real-product evidence](#real-product-evidence)) and is filed as `<its head>.json`;
|
|
519
532
|
- has a head that is `<head>` (default `HEAD`) or an ancestor of it, with every file changed since that head inside the receipts folder, so the proof still covers the code;
|
|
520
533
|
- was made no later than the push (`--pushed`, an ISO time; default the commit time of `<head>`), compared to the second.
|
|
521
534
|
|
|
@@ -5,10 +5,11 @@ Before you report done, load the done-done skill and follow it. If the skill is
|
|
|
5
5
|
Run the CLI as `done-done` if it is on PATH; otherwise run `npx -y done-done-axi`
|
|
6
6
|
wherever this text or the skill says `done-done`. If done-done cannot run, report that instead of writing proof by hand.
|
|
7
7
|
|
|
8
|
-
1. If the repo has no `.done-done.yaml`, run `done-done init`. Then record a journey that drives the real app
|
|
9
|
-
writes its screenshots, recordings or
|
|
10
|
-
folder): `done-done init --journey <name> -- <command>`, so the recipe lives in
|
|
11
|
-
|
|
8
|
+
1. If the repo has no `.done-done.yaml`, run `done-done init`. Then record a journey that drives the real app, CLI or
|
|
9
|
+
site the way a user does and writes its screenshots, recordings or transcript into `$DONE_DONE_EVIDENCE_ROOT`
|
|
10
|
+
(done-done sets it to the run's own folder): `done-done init --journey <name> -- <command>`, so the recipe lives in
|
|
11
|
+
the repo and anyone can re-run the proof. Never record a test, typecheck, lint or build run (`npm test`,
|
|
12
|
+
`yarn run check`, `tsc`): it is code-only, `init` refuses it, and `check`, `receipt` and the gate refuse its proof. Commit `.done-done.yaml` and the journey script. If `init` says no agent is set, ask the user which agent
|
|
12
13
|
and model to use and run the command it prints; never choose one yourself.
|
|
13
14
|
2. Commit your change, then run `done-done run <journey>` from the repo root. Proof only counts from a clean,
|
|
14
15
|
committed tree.
|
package/package.json
CHANGED
|
@@ -31,8 +31,10 @@ First run `command -v done-done`. If it prints a path, run `done-done` as shown
|
|
|
31
31
|
Results are TOON: read `result` and follow the `help` lines through
|
|
32
32
|
`init`, `doctor`, `run`, `verify`, `check` and `receipt`. `design` judges captured screens for visual defects,
|
|
33
33
|
`maintain` keeps the repo's verify skill true, and `signoff` writes a run's gate or evidence as a sign-off message
|
|
34
|
-
(it prints the post command and never posts). Proof comes from driving the real product
|
|
35
|
-
|
|
34
|
+
(it prints the post command and never posts). Proof comes from driving the real product (the app, CLI or site the way a user does,
|
|
35
|
+
captured as screenshots, a recording or its transcript), never from a test, typecheck, lint or build run: `init --journey` refuses a journey
|
|
36
|
+
that only wraps one (`npm test`, `yarn run check`, `tsc`, `pytest`), and `check`, `receipt` and `gate` refuse proof from it. Proof counts only
|
|
37
|
+
from a committed tree: commit the done-done setup (config, capture script) before you capture. `fail` or
|
|
36
38
|
`inconclusive` is not done: fix the cause and re-run, or tell the user exactly what is still unproven.
|
|
37
39
|
A run waiting at its judge gate (`awaiting_approval`) did not pass, and nothing moves it past its judge: show the user
|
|
38
40
|
its findings and captures, and end it with `done-done respond --action fix` only when they say so.
|
|
@@ -57,7 +59,7 @@ run the journey, commit it, then open the PR, in that order. Fix the cause it na
|
|
|
57
59
|
|
|
58
60
|
A CI job gets no model and no key, so it never runs `check`, `run` or `verify`. Proof runs on the desktop, then commit the
|
|
59
61
|
receipt it wrote (`<evidence root>/.done-done/receipts/<head>.json`) into `.done-done-receipts/` in a follow-up commit. CI runs only
|
|
60
|
-
`done-done gate <head>` (full history): `PASS` or `FAIL: <reason>` for a missing, late, other-head
|
|
62
|
+
`done-done gate <head>` (full history): `PASS` or `FAIL: <reason>` for a missing, late, other-head, moved-on or code-only receipt.
|
|
61
63
|
|
|
62
64
|
`init` also has that agent write the repo's `.agents/skills/done-done-verify-<app>/SKILL.md` (never overwriting one): read it before
|
|
63
65
|
you drive the app, and fix it when it is wrong.
|