retake-demos 0.12.1 → 0.13.2

package/AGENTS.md

@@ -226,6 +226,23 @@ Most real demos start behind a sign-in. The agent never sees a password:
   and cuts the few seconds of video that show each one, so a long demo does not
   have to be re-watched to learn whether one thing changed. Exit 3 while
   anything flagged is still wrong. `retake unflag` stops watching one.
+- **Name things so a person can find them.** The window groups demos by
+  whatever it can work out — which, with no product note, means folders called
+  `localhost:3200`. The first heading of a product note names that folder.
+  Writing one is part of setting a demo up, not an afterthought.
+- `retake calibrate demos/<name>.yaml` — how much `sweep` is worth on this app.
+  Records the demo once per known visual defect, seeded for real, plus a
+  control where nothing is wrong, then reports recall and false positives.
+  Expensive: nine takes, about ten minutes, needs the app running. The report
+  it leaves is cheap to read, and **you should read it before telling anybody
+  how much a sweep finding is worth.** Its stated limits travel with it, and
+  all four belong in anything you say about the numbers: the seeds were written
+  by somebody who has read the checklist, so recall is an upper bound; injected
+  faults are cleaner than real ones; the run names the checklist items nothing
+  seeds, and says nothing whatever about those; and a run whose control did not
+  record cleanly measured nothing at all and exits 3 rather than printing a
+  reassuring zero. From MCP you can only READ the last measurement — making one
+  is a terminal command, because it records for ten minutes.
 - `retake intent` — what the product IS, in the builder's words: what it does,
   who uses it, and what about it is deliberate. Every visual check reads it.
   Without it they judge the app as a generic web page — measured on a real

package/dist/cli.js

@@ -11,6 +11,7 @@
  *   retake render outputs/x                       re-render from the existing take (--preset to switch)
  *   retake check outputs/x                        pass/fail on resolution, fps, duration, files
  *   retake verify outputs/x                       did it LOOK right — each scene's `expect`, judged
+ *   retake calibrate demos/x.yaml               how much to trust `sweep` — seed defects, count what it finds
  *   retake intent                                 what this product IS — context every visual check needs
  *   retake sweep outputs/x                        look at every frame as a whole — what nobody thought to ask about
  *   retake sense outputs/x                        does the run ADD UP — what went in vs what came out
@@ -44,6 +45,7 @@ import { notes } from "./ext/notes.js";
 import { sense } from "./ext/sense.js";
 import { sweep } from "./ext/sweep.js";
 import { TEMPLATE, readIntent, writeIntent } from "./ext/intent.js";
+import { planCalibration, report, usable } from "./ext/calibrate.js";
 import { checkFlags, flag, unflag } from "./ext/flags.js";
 import { describeOrphans, findOrphans, healOrphans } from "./ext/heal.js";
 import { SHAPES, describePlan, describeTrials, planDestroy, refuseToRun, tryCandidates } from "./ext/destroy.js";
@@ -382,6 +384,71 @@ program
     if (trials.some((t) => t.verdict === "broke"))
         process.exitCode = 3;
 });
+program
+    .command("calibrate")
+    .description("how much to trust `sweep` — seeds a known defect into a working demo, records each one, and counts what it finds")
+    .argument("<manifest>", "a demo whose app is running")
+    .option("-o, --out <dir>", "output root", "outputs")
+    .option("--only <names>", "just these defects, comma-separated")
+    .option("--plan", "write the variants and stop", false)
+    .action(async (file, opts) => {
+    const loaded = loadManifest(file);
+    const outRoot = path.resolve(opts.out);
+    const plan = planCalibration(loaded.manifest, outRoot, { only: opts.only?.split(",").map((x) => x.trim()).filter(Boolean) });
+    say(`${plan.variants.length} variants in ${path.relative(process.cwd(), plan.root)}/ — one control, ${plan.variants.length - 1} seeded`);
+    if (opts.plan)
+        return;
+    const results = [];
+    /** Sweep once, and give back what it said. */
+    const look = async (m, dir, fresh) => (await sweep(m, dir, () => { }, { demosDir: path.resolve("demos"), fresh })).frames.flatMap((f) => f.issues.map((i) => i.kind));
+    let stability = null;
+    for (const v of plan.variants) {
+        const dir = path.join(plan.root, v.name);
+        say(`\n${v.name}${v.defect ? ` — ${v.defect.looks}` : ""}`);
+        try {
+            const take = await record(v.manifest, { outDir: dir, manifestDir: loaded.dir, headed: false, log: () => { } });
+            await render(v.manifest, take, dir, { log: () => { }, noMaster: true });
+            // A recording that fell over, recorded nothing, or produced no frames
+            // measures nothing — whichever end of it broke. Asking sweep about it
+            // and reporting the empty answer is how a run against an app that was
+            // not even running came back reading "none on the control".
+            const fit = usable(dir, take);
+            if (!fit.ok) {
+                results.push({ variant: v.name, defect: v.defect, kinds: [], found: false, seeded: false, why: fit.why, other: 0 });
+                say(`  → not measured: ${fit.why}`);
+                continue;
+            }
+            const kinds = await look(v.manifest, dir, false);
+            const found = !v.defect ? false : kinds.some((k) => k.toUpperCase().startsWith(v.defect.expect.split(" ")[0]));
+            results.push({ variant: v.name, defect: v.defect, kinds, found, seeded: true, other: v.defect ? kinds.filter((k) => !k.toUpperCase().startsWith(v.defect.expect.split(" ")[0])).length : 0 });
+            say(`  → ${kinds.join(", ") || "nothing"}`);
+            // The control, asked the same question about the same frames a second
+            // time. Judges are not deterministic, and a harness that never checks
+            // this cannot tell a real finding from a coin landing heads.
+            if (!v.defect) {
+                say("  asking the control again, to see whether the answer holds");
+                stability = { first: kinds, again: await look(v.manifest, dir, true) };
+                say(`  → ${stability.again.join(", ") || "nothing"}`);
+            }
+        }
+        catch (e) {
+            const msg = String(e.message).split("\n")[0].slice(0, 90);
+            results.push({ variant: v.name, defect: v.defect, kinds: [], found: false, seeded: false, why: `the run threw — ${msg}`, other: 0, error: msg });
+            say(`  → could not run: ${msg}`);
+        }
+    }
+    for (const l of report(results, stability))
+        say(l);
+    fs.writeFileSync(path.join(plan.root, "report.json"), JSON.stringify({ at: new Date().toISOString(), demo: loaded.manifest.name, results, stability }, null, 2) + "\n");
+    // The false-positive rate is the number people act on. If the control did
+    // not record, this run produced no number at all, and exiting 0 says the
+    // opposite.
+    const control = results.find((r) => !r.defect);
+    if (!control?.seeded) {
+        process.exitCode = 3;
+        say("\nnothing was measured — fix the run before quoting any of this.");
+    }
+});
 program
     .command("intent")
     .description("what this product IS — the context every visual check is missing without it")

package/dist/dryrun.js

@@ -260,9 +260,23 @@ export async function dryRun(m, manifestDir, log, opts = {}) {
                         }
                         break;
                     }
+                    // `to: top` and `to: bottom` are the PAGE's ends, not selectors — the
+                    // recorder reads scrollY and scrollHeight for them. Dry was looking for
+                    // an element called "bottom", timing out, and failing a step that
+                    // records perfectly. The starter demo shipped with `retake init` hits
+                    // it, so a new person's first dry run failed on a demo that works.
                     case "scroll":
-                        if (step.to)
+                        // And actually scroll. Skipping it left the page at the top while
+                        // every later step ran, so a dry run rehearsed a different page from
+                        // the one the recording would see.
+                        if (step.to === "top")
+                            await page.evaluate(() => window.scrollTo(0, 0));
+                        else if (step.to === "bottom")
+                            await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight));
+                        else if (step.to) {
                             await page.locator(step.to).first().boundingBox({ timeout: short });
+                            await page.locator(step.to).first().scrollIntoViewIfNeeded({ timeout: short });
+                        }
                         break;
                     case "upload":
                         await page.locator(step.selector).first().waitFor({ timeout: short });

package/dist/ext/calibrate.js

@@ -0,0 +1,341 @@
+/**
+ * How much should anybody trust what `sweep` says?
+ *
+ * Every other check in Retake is tested: the seam, the guard, the parity, the
+ * refusals. The one thing nothing measures is whether the model-based checks
+ * are RIGHT — and that is the part the product is now sold on. Today `sweep`
+ * found three real bugs, produced at least one clear false positive, and
+ * missed a genuine overlap three passes running. Three anecdotes are not a
+ * number.
+ *
+ * Retake can measure this with what it already has. `evaluate` injects
+ * anything into the page before the camera rolls, so a defect can be seeded
+ * deterministically: one variant of a working demo per known fault, recorded
+ * for real, plus a clean control. Then:
+ *
+ *   recall          how many seeded defects were found
+ *   false positives what was reported on the control, where nothing is wrong
+ *   stability       whether the same frame gives the same answer twice
+ *
+ * WHERE THIS WORKS, AND WHERE IT DOES NOT.
+ *
+ * On a page that is served and then left alone — a document, a landing page,
+ * Retake's own guide — injection is reliable and the numbers mean something:
+ * eight of eight seeded faults found, nothing reported on the control.
+ *
+ * On a live single-page app it is not. Against Capture, seeds provably present
+ * in the DOM after hydration were provably absent from every recorded frame,
+ * four different ways of injecting them, for reasons not isolated. A harness
+ * built on injection therefore measures the injection as much as the check,
+ * and a low recall on such an app is not evidence about the check at all.
+ *
+ * So: use this to calibrate against static pages, and take the control result
+ * — findings where nothing is wrong — from anywhere, because that half needs
+ * no injection and is the half that matters most for trust. To calibrate a
+ * real app, label real recordings and have a person adjudicate them. That is
+ * slower, and it is the only thing that answers the question honestly.
+ *
+ * Two honest limits, stated rather than buried. The person writing the seeds
+ * is not blind to the checklist, so recall here is an upper bound — a defect
+ * nobody thought to seed is still not measured. And CSS-injected faults are
+ * cleaner than the ones real apps produce; a laboratory number, not a field
+ * one. Both are worth having, neither is the whole answer.
+ */
+import fs from "node:fs";
+import path from "node:path";
+import YAML from "yaml";
+import { createHash } from "node:crypto";
+import { LOOK_FOR } from "./sweep.js";
+/**
+ * Deliberately mechanical. Every one of these is visible to a person in one
+ * glance, which is the bar: if a human cannot see it in the frame, a miss is
+ * not evidence of anything.
+ *
+ * And every one targets ORDINARY TEXT — p, li, headings — rather than a
+ * specific element. Three seeds failed here before that rule existed: they
+ * changed parts of the page the demo's frames never showed, so the run
+ * reported a miss when nothing had been asked of the check. A calibration
+ * harness that can silently measure nothing is worse than none, because its
+ * number looks the same either way.
+ */
+export const DEFECTS = [
+    {
+        name: "clipped-text",
+        proof: "[...document.querySelectorAll('p,li,h1,h2,h3')].some(e => e.scrollHeight > e.clientHeight + 4)",
+        expect: "CLIPPED",
+        looks: "headings cut off mid-letter by a box too short for them",
+        css: "h1,h2,h3,li,p{max-height:14px!important;overflow:hidden!important;display:block!important}",
+    },
+    {
+        name: "overlapping-text",
+        proof: "[...document.querySelectorAll('p,li')].some(e => parseFloat(getComputedStyle(e).marginBottom) < -10)",
+        expect: "OVERLAP",
+        looks: "paragraphs sitting on top of each other",
+        css: "p,li{margin-bottom:-26px!important;position:relative!important;z-index:2!important}",
+    },
+    {
+        name: "unreadable-contrast",
+        // Was `.replace(/\\s/g,'')`, which after string escaping built a regex for
+        // a literal backslash followed by "s" — so the spaces in "rgb(233, 235,
+        // 228)" were never stripped, the proof was never true, and this seed
+        // reported "did not land" on every app it was ever run against. It was
+        // counted as one of eight for two releases. No regex now.
+        proof: "getComputedStyle(document.body).color.split(' ').join('') === 'rgb(233,235,228)'",
+        expect: "CONTRAST",
+        looks: "body text almost the same colour as the page behind it",
+        css: "body,p,li,span{color:#e9ebe4!important}",
+    },
+    {
+        name: "runs-off-the-edge",
+        proof: 'document.documentElement.scrollWidth > document.documentElement.clientWidth + 40',
+        expect: "CUT OFF",
+        looks: "content pushed off the right-hand side of the frame",
+        css: "main,body>*{margin-left:62%!important;white-space:nowrap!important}",
+    },
+    {
+        name: "doubled-heading",
+        proof: "!!document.querySelector('[data-retake-dup]')",
+        expect: "DOUBLED",
+        looks: "every heading rendered twice, one under the other",
+        // The first version targeted h2 and used attr(data-x), which is empty —
+        // so nothing appeared and the run looked like a MISS. A seed that fails
+        // silently is indistinguishable from a check that failed, which is why
+        // `landed` below exists.
+        // Twice wrong before this: CSS pseudo-content on headings, which produced
+        // nothing the first time and nothing IN FRAME the second. A seed that is
+        // not plainly visible in the frames being judged measures nothing, and the
+        // temptation to keep tuning it until the number improves is exactly how a
+        // calibration harness starts lying. So it clones the real node instead.
+        css: "",
+        js: "for (const h of [...document.querySelectorAll('h1,h2,h3,p,li')]) { if (h.textContent && h.textContent.trim().length > 8) { const c = h.cloneNode(true); c.setAttribute('data-retake-dup',''); h.after(c); } }",
+    },
+    {
+        name: "stuck-spinner",
+        proof: "!!document.getElementById('retake-seed-spinner')",
+        expect: "UNFINISHED",
+        looks: "a loading spinner sitting on the page that never resolves",
+        css: "",
+        js: "const d=document.createElement('div');d.id='retake-seed-spinner';d.textContent='Loading…';d.style.cssText='position:fixed;top:14px;left:50%;transform:translateX(-50%);background:#fff;border:1px solid #ccc;border-radius:999px;padding:6px 16px;font:14px sans-serif;z-index:2147483647';document.body.appendChild(d);",
+    },
+    {
+        name: "broken-image",
+        proof: "!!document.getElementById('retake-seed-img')",
+        expect: "BROKEN MEDIA",
+        looks: "an image that failed to load, showing its alt text instead",
+        css: "",
+        js: "const i=document.createElement('img');i.id='retake-seed-img';i.src='/definitely-not-here-'+'x'.repeat(4)+'.png';i.alt='product photograph';i.style.cssText='position:fixed;top:90px;left:24px;width:150px;height:100px;border:1px solid #ccc;background:#fff;z-index:2147483647';document.body.appendChild(i);",
+    },
+    {
+        name: "debug-badge",
+        proof: "!!document.getElementById('retake-seed-badge')",
+        expect: "NOT FOR THE CAMERA",
+        looks: "a development badge left in the corner of the screen",
+        css: "",
+        js: "const b=document.createElement('div');b.id='retake-seed-badge';b.textContent='DEV BUILD · staging';b.style.cssText='position:fixed;bottom:12px;left:12px;background:#111;color:#0f0;font:12px monospace;padding:5px 10px;border-radius:5px;z-index:2147483647';document.body.appendChild(b);",
+    },
+];
+/** Where calibration writes. Never near a real take. */
+export const lab = (outRoot, demo) => path.join(outRoot, ".calibrate", demo);
+/**
+ * A control and one variant per defect, as ordinary manifests.
+ *
+ * The control matters more than any single variant: without a run where
+ * nothing is wrong, a low false-positive rate cannot be claimed at all.
+ */
+export function planCalibration(m, outRoot, opts = {}) {
+    const root = lab(outRoot, m.name);
+    fs.mkdirSync(root, { recursive: true });
+    // `--only typo` used to match nothing and run a control alone, then print a
+    // recall line reading "0 of 0" — which is not a failure, it is a measurement
+    // of nothing wearing the clothes of one.
+    const unknown = (opts.only ?? []).filter((n) => !DEFECTS.some((d) => d.name === n));
+    if (unknown.length)
+        throw new Error(`no such defect: ${unknown.join(", ")} — try ${DEFECTS.map((d) => d.name).join(", ")}`);
+    const want = opts.only?.length ? DEFECTS.filter((d) => opts.only.includes(d.name)) : DEFECTS;
+    const variants = [];
+    const write = (name, defect) => {
+        const clone = JSON.parse(JSON.stringify(m));
+        clone.name = `${m.name}-${name}`.slice(0, 60);
+        if (defect) {
+            // Before anything is recorded, so every frame carries the fault.
+            clone.setup = [
+                ...(clone.setup ?? []),
+                // A stylesheet in <head> survives re-render; the elements below may not.
+                ...(defect.css ? [{ action: "evaluate", script: `{const s=document.createElement('style');s.id='retake-seed-css';s.textContent=${JSON.stringify(defect.css)};document.head.appendChild(s);setInterval(()=>{if(!document.getElementById('retake-seed-css'))document.head.appendChild(s);},250);}` }] : []),
+                // Re-applied on a timer, because a React app hydrates and re-renders
+                // after the seed runs and takes anything appended to the body with it.
+                // Capture did exactly that: the element passed its own proof at seed
+                // time and was gone from every recorded frame, which looked identical
+                // to the check missing it.
+                ...(defect.js ? [{ action: "evaluate", script: `{ const seed = () => { try { ${defect.js} } catch {} }; seed(); setInterval(() => { if (!(${defect.proof})) seed(); }, 250); }` }] : []),
+                // The seed proves itself, in the page, before anything is recorded.
+                // A seed that did not render must fail the take rather than produce
+                // frames that look exactly like a check missing something.
+                { action: "wait", ms: 1200 },
+                // Checked after the app has had time to hydrate and re-render, not at
+                // the instant of injection — that was the difference between a seed
+                // that exists and a seed that is in the picture.
+                { action: "evaluate", script: `{ if (!(${defect.proof})) throw new Error("the ${defect.name} seed did not take on this app"); }` },
+                { action: "wait", ms: 300 },
+            ];
+            // And again at the END. The setup proof says the seed was on the page
+            // before the camera rolled; it says nothing about whether it was still
+            // there for the frames that were actually judged. A single-page app that
+            // navigates or re-renders can drop it halfway through, and every frame
+            // after that point asks the check to find something that is not there.
+            // Failing the step is right: a run that lost its seed is not a miss, it
+            // is not a measurement.
+            clone.steps = [
+                ...clone.steps,
+                { action: "evaluate", script: `{ if (!(${defect.proof})) throw new Error("the ${defect.name} seed was gone by the end of the run"); }` },
+            ];
+        }
+        const file = path.join(root, `${name}.yaml`);
+        fs.writeFileSync(file, [
+            defect ? `# ${defect.name} — ${defect.looks}.` : "# control — nothing is wrong with this one.",
+            `# Generated by \`retake calibrate ${m.name}\`. Recording it seeds the fault for real.`,
+            "",
+            YAML.stringify(clone),
+        ].join("\n"));
+        return { name, defect, file, manifest: clone };
+    };
+    variants.push(write("control", null));
+    for (const d of want)
+        variants.push(write(d.name, d));
+    return { root, variants };
+}
+/**
+ * Did the seed actually take?
+ *
+ * A variant whose frames are byte-identical to the control's is a variant
+ * where nothing was injected — the CSS missed, the selector was wrong, the
+ * page changed. Counting that as a MISS blames the check for the harness's
+ * mistake, and it happened on the very first run: a doubled-heading seed that
+ * targeted the wrong element reported 7 of 8 when the honest answer was 7 of
+ * 7 and one seed that never landed.
+ */
+/**
+ * @deprecated Seeds prove themselves in the page now — see `Defect.proof`.
+ *
+ * Kept only as a record of why: this compares a variant's frames to the
+ * control's, which cannot work on an app whose content changes between runs.
+ * Capture's board carries timestamps and counts, so every variant differed
+ * from its control whether or not anything had been injected, and this
+ * reported eight seeds as landed when at least one plainly had not.
+ */
+export function landed(controlDir, variantDir) {
+    // NOTE what this does and does not prove: that the frames differ from the
+    // control's, so SOMETHING was injected. It cannot prove the fault is
+    // plainly visible in them. A seed that changes the page off-screen passes
+    // here and still measures nothing — which happened, twice, before the
+    // doubled-heading seed was rewritten to clone a real node.
+    const shots = (d) => {
+        try {
+            return fs.readdirSync(path.join(d, "stills")).sort()
+                .map((f) => createHash("sha1").update(fs.readFileSync(path.join(d, "stills", f))).digest("hex"));
+        }
+        catch {
+            return [];
+        }
+    };
+    const a = shots(controlDir), b = shots(variantDir);
+    if (!a.length || !b.length)
+        return true; // cannot tell; do not accuse the harness
+    return a.join() !== b.join();
+}
+/**
+ * Is this recording fit to measure anything at all?
+ *
+ * It was not asked. A control recorded against an app that was not running
+ * came back with an empty timeline, `ok: false` and no frames — and the report
+ * said "FALSE POSITIVES none on the control", which is the single number this
+ * harness exists to produce. Nothing was wrong with the control because
+ * nothing had happened in it. A blank measurement that reads as a clean one is
+ * worse than no measurement, and it exited 0.
+ */
+export function usable(dir, take) {
+    if (!take)
+        return { ok: false, why: "nothing was recorded" };
+    if (!take.timeline?.length)
+        return { ok: false, why: "the run recorded no steps at all" };
+    if (take.ok === false)
+        return { ok: false, why: "the run did not finish cleanly" };
+    let stills = 0;
+    try {
+        stills = fs.readdirSync(path.join(dir, "stills")).filter((f) => /\.(png|jpe?g)$/i.test(f)).length;
+    }
+    catch { /* none */ }
+    if (!stills)
+        return { ok: false, why: "the run produced no frames to look at" };
+    return { ok: true, why: "" };
+}
+/** The `sweep` items nothing here seeds, named rather than left to arithmetic. */
+export function uncovered() {
+    const seeded = new Set(DEFECTS.map((d) => d.expect.toUpperCase()));
+    return LOOK_FOR.map((l) => l.split(" — ")[0].trim())
+        .filter((kind) => !seeded.has(kind.toUpperCase()));
+}
+/** Recall, false positives, and the sentence that says what they mean. */
+export function report(rs, stability = null) {
+    const all = rs.filter((r) => r.defect);
+    const seeded = all.filter((r) => r.seeded);
+    const unseeded = all.filter((r) => !r.seeded);
+    const control = rs.find((r) => !r.defect);
+    const found = seeded.filter((r) => r.found).length;
+    const out = [];
+    out.push("");
+    out.push(`RECALL   ${found} of ${seeded.length} defects that actually appeared were found`);
+    for (const r of seeded) {
+        out.push(`  ${r.found ? "\u2713" : "\u2717"} ${r.variant.padEnd(20)} wanted ${String(r.defect.expect).padEnd(19)} got ${r.kinds.join(", ") || (r.error ? `\u2014 ${r.error}` : "nothing")}`);
+    }
+    if (unseeded.length) {
+        out.push("");
+        out.push(`NOT MEASURED   ${unseeded.length} of ${all.length} seeded runs asked the check nothing:`);
+        for (const r of unseeded)
+            out.push(`  \u2014 ${r.variant.padEnd(20)} ${r.why ?? r.error ?? "the injection did not land"}. Fix the harness, not the checklist.`);
+    }
+    out.push("");
+    // The control is the half that needs no injection, so it is the half that
+    // travels — and the only one that can speak to false positives. If it did
+    // not record, nothing here may be claimed, in either direction.
+    if (!control) {
+        out.push("FALSE POSITIVES   not measured \u2014 no control was run, so no rate can be claimed");
+    }
+    else if (!control.seeded) {
+        out.push(`FALSE POSITIVES   NOT MEASURED \u2014 on the control, ${control.why ?? control.error ?? "the run did not record"}. No rate can be claimed from this run, and neither can the recall above: every variant recorded under the same conditions.`);
+    }
+    else {
+        out.push(control.kinds.length
+            ? `FALSE POSITIVES   ${control.kinds.length} finding(s) on the control, where nothing is wrong: ${control.kinds.join(", ")}`
+            : "FALSE POSITIVES   none on the control");
+    }
+    const noise = seeded.reduce((n, r) => n + r.other, 0);
+    if (noise)
+        out.push(`                  and ${noise} extra finding(s) across the variants, beyond the fault seeded in each`);
+    out.push("");
+    // Promised in this file's own header for two releases and never produced.
+    // A number nobody measures is a claim, and this harness exists because
+    // claims about the checks were not good enough.
+    if (!stability) {
+        out.push("STABILITY   not measured this run \u2014 the control was asked once, so nothing here says whether the same frames give the same answer twice.");
+    }
+    else {
+        const a = [...stability.first].sort().join(", ") || "nothing";
+        const b = [...stability.again].sort().join(", ") || "nothing";
+        out.push(a === b
+            ? `STABILITY   the control gave the same answer twice: ${a}`
+            : `STABILITY   the control gave DIFFERENT answers to the same frames \u2014 first ${a}, then ${b}. Treat single sweep results on this app as one opinion, not a reading.`);
+    }
+    out.push("");
+    const gaps = uncovered();
+    if (gaps.length) {
+        // The count is of what this run attempted, not of the catalogue — `--only`
+        // makes those different numbers, and printing the catalogue's would
+        // overstate a narrow run.
+        const tried = all.length || DEFECTS.length;
+        out.push(`NOT COVERED   ${tried} seed${tried === 1 ? "" : "s"} against a ${LOOK_FOR.length}-item checklist. Nothing here measures ${gaps.join(" or ")} at all, so the recall above says nothing about ${gaps.length === 1 ? "it" : "them"}.`);
+        out.push("");
+    }
+    out.push("What this is and is not: the seeds were written by someone who has read the checklist, so recall here is an upper bound \u2014 a fault nobody thought to seed is still unmeasured. Injected faults are also cleaner than the ones real apps produce. Treat it as a floor for trust, not a score.");
+    return out;
+}

package/dist/ext/flags.js

@@ -113,6 +113,10 @@ function stillFor(outDir, take, label) {
  * have to be re-watched.
  */
 export async function checkFlags(manifestFile, _m, outDir, opts = {}) {
+    // Both derived from the manifest's own path, so `fixed` reads the note that
+    // belongs to this demo's app.
+    const demosDir = path.dirname(manifestFile);
+    const demoName = path.basename(manifestFile).replace(/\.(ya?ml|json)$/i, "");
     const lines = [];
     const say = (l) => { lines.push(l); opts.log?.(l); };
     const flags = readFlags(manifestFile);
@@ -139,8 +143,11 @@ export async function checkFlags(manifestFile, _m, outDir, opts = {}) {
             why = `no still for scene "${f.scene}" in the newest take — was that scene recorded?`;
         else if (!provider)
             why = noJudge ?? "nothing available to look at the frame";
+        // The same product context `verify` gets. Without it `fixed` judged the
+        // same frames by a generic standard while `verify` judged them knowing
+        // what the app is — two answers to one question, by design accident.
         else
-            ({ ok, why } = await judgeWith(provider, still, f.expect));
+            ({ ok, why } = await judgeWith(provider, still, f.expect, demosDir, demoName));
         let clip;
         let clipError;
         if (opts.clips !== false) {

package/dist/ext/intent.js

@@ -20,6 +20,30 @@
  */
 import fs from "node:fs";
 import path from "node:path";
+/**
+ * A demo name that cannot leave the folder it is meant to be in.
+ *
+ * `intentPath` and `writeIntent` put this straight into `path.join`, so
+ * `--demo ../../escaped` wrote its file outside `demos/` entirely. Measured,
+ * not theorised, and in a released package. Kebab-case is the rule every other
+ * name in Retake already follows; the containment check afterwards is the belt
+ * for anything that rule ever stops catching.
+ */
+export function safeDemo(demo) {
+    if (!/^[a-z0-9][a-z0-9-]*$/.test(demo)) {
+        throw new Error(`"${demo}" is not a demo name — lowercase letters, numbers and hyphens only`);
+    }
+    return demo;
+}
+/** Belt for the braces: whatever we built, it has to be inside `dir`. */
+function within(dir, file) {
+    const root = path.resolve(dir);
+    const full = path.resolve(file);
+    if (full !== root && !full.startsWith(root + path.sep)) {
+        throw new Error(`refusing to write outside ${dir}`);
+    }
+    return full;
+}
 /**
  * Where a demo's product note lives.
  *
@@ -30,13 +54,15 @@ import path from "node:path";
  */
 export function intentPath(demosDir, demo) {
     if (demo) {
-        const own = path.join(demosDir, `${demo}.product.md`);
+        const own = within(demosDir, path.join(demosDir, `${safeDemo(demo)}.product.md`));
         if (fs.existsSync(own))
             return own;
     }
-    return path.join(demosDir, "product.md");
+    return within(demosDir, path.join(demosDir, "product.md"));
 }
 export function readIntent(demosDir, demo) {
+    // A bad name is not an error when reading — it is simply no note. Only
+    // writing refuses, because writing is where the damage would be.
     try {
         const t = fs.readFileSync(intentPath(demosDir, demo), "utf8").trim();
         return filled(t) ? t : null;
@@ -57,15 +83,37 @@ export function filled(text) {
     return words.length > 0;
 }
 export function writeIntent(demosDir, text, demo) {
+    const f = within(demosDir, path.join(demosDir, demo ? `${safeDemo(demo)}.product.md` : "product.md"));
     fs.mkdirSync(demosDir, { recursive: true });
-    const f = demo ? path.join(demosDir, `${demo}.product.md`) : path.join(demosDir, "product.md");
     fs.writeFileSync(f, text.trim() + "\n");
     return f;
 }
+/**
+ * The product's name, from the first heading of its note.
+ *
+ * The window groups demos by URL, so a workspace of real products showed
+ * folders called "localhost:3200" and "localhost:4990" — accurate and
+ * meaningless. Naming the product once, for the checks, names its folder too.
+ * Returns null for the starter's own placeholder, so an unfilled note never
+ * becomes a folder called "What this product is".
+ */
+export function productName(demosDir, demo) {
+    const t = readIntent(demosDir, demo);
+    if (!t)
+        return null;
+    const h = /^#\s+(.+?)\s*$/m.exec(t);
+    const name = h?.[1]?.trim();
+    if (!name || /^what this product is$/i.test(name))
+        return null;
+    return name.slice(0, 32);
+}
 /** The starter, so nobody has to guess what belongs in it. */
-export const TEMPLATE = `# What this product is
+export const TEMPLATE = `# Name this product
 
-<!-- One or two sentences. What it does, and for whom. -->
+<!-- Replace the heading above with what this product is CALLED — that name
+     becomes its folder in the Retake window, which otherwise says something
+     like "localhost:3200". Then, below: one or two sentences on what it does
+     and for whom. -->
 
 ## Who uses it
 

package/dist/ext/notes.js

@@ -56,10 +56,19 @@ function readRun(dir) {
         return null; // an empty folder is not a run
     return { name, dir, take, used, priors, when, checks: readChecks(dir) };
 }
-/** Every scene label in a used manifest that carries no `expect:`. */
-function scenesWithoutExpect(used) {
+/**
+ * How many scenes carry an `expect:`, and how many do not.
+ *
+ * The note below used to fire on any demo with two bare scenes and say
+ * "nothing has ever checked how it looks". It said that about a demo carrying
+ * seventeen expectations that `verify` had just answered thirty-nine questions
+ * from. A watcher that says something plainly untrue about the workspace it is
+ * watching is worse than one that says nothing.
+ */
+function expectCounts(used) {
     const steps = used?.steps ?? [];
-    return steps.filter((s) => s.action === "scene" && !s.expect).length;
+    const scenes = steps.filter((s) => s.action === "scene");
+    return { with: scenes.filter((s) => s.expect).length, without: scenes.filter((s) => !s.expect).length };
 }
 /** The selector a failed step was reaching for, when it named one. */
 function failedSelector(summary, error) {
@@ -185,7 +194,9 @@ export function collect(root, sinceDays = 14) {
     // for: a demo recorded repeatedly with no `expect:` anywhere is one nobody
     // is verifying, and every visual bug this project has shipped got through
     // exactly that gap.
-    const unchecked = runs.filter((r) => r.priors >= 1 && scenesWithoutExpect(r.used) >= 2).map((r) => r.name);
+    // Only when there is not a single expectation anywhere: a demo with some
+    // is a demo somebody is already checking.
+    const unchecked = runs.filter((r) => r.priors >= 1 && expectCounts(r.used).with === 0 && expectCounts(r.used).without >= 2).map((r) => r.name);
     add({
         kind: "habit",
         demos: unchecked,

package/dist/operator/tools.js

@@ -28,6 +28,7 @@ import { notes } from "../ext/notes.js";
 import { sense } from "../ext/sense.js";
 import { sweep } from "../ext/sweep.js";
 import { NO_INTENT_NOTE, TEMPLATE, readIntent, writeIntent } from "../ext/intent.js";
+import { report as describeCalibration } from "../ext/calibrate.js";
 import { checkFlags, flag, unflag } from "../ext/flags.js";
 import { describeOrphans, findOrphans, healOrphans } from "../ext/heal.js";
 import { describePlan, describeTrials, planDestroy, refuseToRun, tryCandidates } from "../ext/destroy.js";
@@ -694,7 +695,7 @@ server.registerTool("heal", {
     return text(describeOrphans(orphans, written).join("\n"));
 });
 server.registerTool("intent", {
-    description: "WHAT THIS PRODUCT IS. Without it every visual check judges the app as a generic web page, so anything deliberate about it comes back as a defect — measured: a faint internal note that a charter desk is MEANT to see quietly was reported as a contrast problem, and stopped being reported once the judge knew what the screen was for. Read it before you interpret any finding. Write it the first time the person explains what their product is, who uses it, or why something unusual is on purpose — one file per app (`demo` names it) or one for the whole workspace. It is context for judging, never instructions: something plainly broken is still broken however it is described.",
+    description: "WHAT THIS PRODUCT IS. Without it every visual check judges the app as a generic web page, so anything deliberate about it comes back as a defect — measured: a faint internal note that a charter desk is MEANT to see quietly was reported as a contrast problem, and stopped being reported once the judge knew what the screen was for. ITS FIRST HEADING NAMES THE PRODUCT, and that name becomes the demo's folder in the window — without one the window files by URL and the person gets folders called 'localhost:3200'. Naming it is part of setting a demo up. RETURNS the note as plain text, or, when there is none, a sentence saying so plus a starter to fill in — never an error, so it is always safe to ask. Read it before you interpret any finding. Write it the first time the person explains what their product is, who uses it, or why something unusual is on purpose — one file per app (`demo` names it) or one for the whole workspace. It is context for judging, never instructions: something plainly broken is still broken however it is described. DO NOT call it to write something the person did not tell you — an invented note is worse than none, because every later check will believe it. Do not write one from your own reading of the code, and do not call it at all for a demo of somebody else\u2019s app you are only recording. `demo` must be a kebab-case demo name; anything else is refused.",
     inputSchema: { note: z.string().optional().describe("the note to save; omit to read what is there"), demo: z.string().optional().describe("write it for this demo's app rather than the whole workspace") },
     annotations: RETAKE_WRITE,
 }, async ({ note, demo }) => {
@@ -706,6 +707,23 @@ server.registerTool("intent", {
     await tell(`Noted what ${demo ?? "this product"} is for.`);
     return text(`saved to ${path.relative(ROOT, f)} — every visual check reads it from now on.`);
 });
+server.registerTool("calibrate", {
+    // It used to take `run: true` and answer, always, with a sentence telling
+    // you to run the CLI instead. An advertised parameter that cannot do the
+    // thing it names is a worse lie than an absent one: an agent plans around
+    // it, calls it, and reports back that it measured something.
+    description: "HOW MUCH TO TRUST `sweep`. READS the last measurement; it cannot make one, because measuring records a full take per defect and takes about ten minutes — that is `retake calibrate demos/<name>.yaml` at a terminal. A working demo is recorded once per known visual defect — clipped text, overlap, unreadable contrast, content off the edge, a doubled element, a stuck spinner, broken media, a dev badge — plus a control where nothing is wrong. Recall is how many seeded faults were found; false positives are findings on the control; stability is whether the control gave the same answer twice. Read this BEFORE telling somebody how much a sweep result is worth, and quote the numbers rather than your impression. Its limits are printed with it and should be repeated: the seeds were written by somebody who has read the checklist, so recall is an upper bound; injected faults are cleaner than real ones; and the run names outright which checklist items nothing seeds.",
+    inputSchema: { name: z.string().describe("the demo whose calibration to read") },
+    annotations: READ_ONLY,
+}, async ({ name }) => {
+    if (!safe(name))
+        return text(`no demo "${name}"`);
+    const file = path.join(OUT, ".calibrate", name, "report.json");
+    if (!fs.existsSync(file))
+        return text(`${name} has never been calibrated, so there is no measured answer to how much its sweep results are worth. \`retake calibrate demos/${name}.yaml\` produces one — at a terminal, not from here.`);
+    const r = JSON.parse(fs.readFileSync(file, "utf8"));
+    return text([...describeCalibration(r.results, r.stability ?? null), "", `measured ${r.at}`].join("\n"));
+});
 server.registerTool("done", { description: "Call when the demo is recorded and acceptable (or when you are stopping). One sentence for the person.", inputSchema: { summary: z.string(), demo: z.string().optional() }, annotations: RETAKE_WRITE }, async ({ summary, demo }) => {
     await tell(`Done: ${summary}`);
     if (UI && !SESSION)

package/dist/ui/chat.html

@@ -11,18 +11,27 @@
   /* The review surface. Retake is driven from an agent (or the workshop);
      this page is where a person watches, judges, and collects the output.
      It accepts no prose on purpose. */
-  :root { --paper:#edefe8; --card:#f8f9f5; --ink:#191d19; --muted:#767e74; --rule:#dbdfd3; --rule2:#c9cec0; --accent:#1a1c1e; --amber:#e8b04b; --ok:#2f5d3f; --bad:#a23b2c; }
+  :root { --paper:#edefe8; --card:#f8f9f5; --ink:#191d19; --muted:#656c63; --rule:#dbdfd3; --rule2:#c9cec0; --accent:#1a1c1e; --amber:#e8b04b; --ok:#2f5d3f; --bad:#a23b2c; }
   * { box-sizing:border-box; }
   html,body { margin:0; height:100%; background:var(--paper); color:var(--ink); font:15px/1.6 -apple-system, BlinkMacSystemFont, "SF Pro Text", "Helvetica Neue", Inter, Arial, sans-serif; -webkit-font-smoothing:antialiased; }
   button, input { font:inherit; color:inherit; }
   .app { min-height:100%; display:flex; flex-direction:column; }
   .top { display:flex; align-items:center; gap:10px; padding:0 24px; height:56px; border-bottom:1px solid var(--rule); background:var(--paper); }
+  /* A phone has room for the pills or for the padding, not both. */
+  @media (max-width: 700px) {
+    .top { padding:0 12px; gap:6px; overflow-x:auto; scrollbar-width:none; }
+    .top::-webkit-scrollbar { display:none; }
+    .chip { padding:0 11px; font-size:12.5px; }
+  }
   .mark { width:22px; height:22px; border-radius:7px; background:var(--accent); position:relative; flex:none; }
   .mark i { position:absolute; left:5.5px; height:2.5px; border-radius:2px; background:#eceee9; }
   .mark i:nth-child(1){top:4px;width:4px;opacity:.55} .mark i:nth-child(2){top:8px;width:7px;opacity:.8} .mark i:nth-child(3){top:12px;width:11px;background:var(--amber)} .mark i:nth-child(4){top:16px;width:7px;opacity:.8}
   .top h1 { font-size:15px; font-weight:600; margin:0; }
   .controls { position:relative; display:flex; align-items:center; gap:8px; margin-left:8px; }
-  .chip { border:1px solid var(--rule2); background:#fff; border-radius:999px; height:32px; padding:0 14px; display:inline-flex; align-items:center; font-size:13px; font-weight:500; color:var(--ink); cursor:pointer; transition:border-color .12s; }
+  /* `white-space: nowrap` because these are pills: without it "How it works"
+     and "What's new" wrapped INSIDE their own borders on a phone, which reads
+     as a broken control rather than a narrow screen. */
+  .chip { border:1px solid var(--rule2); background:#fff; border-radius:999px; height:32px; padding:0 14px; display:inline-flex; align-items:center; white-space:nowrap; flex:none; font-size:13px; font-weight:500; color:var(--ink); cursor:pointer; transition:border-color .12s; }
   .chip:hover { border-color:var(--ink); }
   .pop { position:absolute; left:0; top:38px; z-index:40; background:#fff; border:1px solid var(--rule2); border-radius:14px; box-shadow:0 12px 34px rgba(24,28,24,.13); padding:8px; width:290px; display:none; }
   .pop.show { display:block; }
@@ -64,6 +73,13 @@
   .under { display:flex; align-items:baseline; gap:8px; margin-top:16px; flex-wrap:wrap; }
   .vtitle { font-size:15px; font-weight:600; color:var(--ink); }
   .vmeta { font-size:13px; color:var(--muted); }
+  /* The recording's own state, in its own colour. A finished take was quiet
+     grey metadata while a failed OPTIONAL check was a red mark — so a demo
+     that recorded perfectly read as broken. */
+  .vstate { font-size:12px; font-weight:500; border-radius:999px; padding:2px 9px; white-space:nowrap; }
+  .vstate.done { color:var(--ok); background:rgba(47,93,63,.10); }
+  .vstate.part { color:var(--amber); background:rgba(232,176,75,.16); }
+  .vstate.bad  { color:var(--bad); background:rgba(162,59,44,.10); }
   .under a, .under button { color:var(--ink); text-decoration:none; border:1px solid var(--rule2); background:#fff; border-radius:999px; padding:0 14px; cursor:pointer; font-size:13px; font-weight:500; display:inline-flex; align-items:center; transition:border-color .12s; }
   .under a:hover, .under button:hover { border-color:var(--ink); }
   /* A native select with its arrow removed leaves lopsided padding; make it
@@ -72,16 +88,36 @@
   .under a, .under button { height:32px; box-sizing:border-box; align-self:center; }
   .under select:hover { border-color:var(--ink); }
   .frames { margin-top:16px; border:1px solid var(--rule); background:var(--card); border-radius:16px; padding:14px 16px; }
+  /* On a phone the heading, its hint and "Use current frame" fought for one
+     line and the button wrapped to three. Stack them instead. */
+  @media (max-width: 700px) {
+    .frames .fhead { flex-wrap:wrap; row-gap:8px; }
+    .frames .fhead .sp { display:none; }
+    .frames .fhead .chip { order:3; width:100%; justify-content:center; }
+    .frames .fhead .hint { order:2; flex-basis:100%; }
+  }
   /* What you flagged, and whether it is still wrong. Shown only when there is
      something flagged — an empty one would be clutter with a heading on it. */
   /* One line: what has actually been looked at, and whether the answer is
      about the recording that is there now. Never a panel — the question is
      "did anyone check this", and that is one line's worth of answer. */
-  .checked { margin-top:10px; font-size:12.5px; color:var(--muted); display:flex; gap:14px; flex-wrap:wrap; align-items:baseline; }
+  /* Its own area, named. Loose grey dashes under the video read as parts of
+     the recording that were missing; they are optional services you can run. */
+  .review { margin-top:14px; border:1px solid var(--rule); border-radius:12px; padding:10px 14px 12px; background:var(--card); }
+  .rhead { font-size:11.5px; letter-spacing:.04em; text-transform:uppercase; color:var(--muted); display:flex; gap:10px; align-items:baseline; flex-wrap:wrap; }
+  /* No opacity. It took a colour already at the AA line and pushed it under.
+     Case and letter-spacing separate this from the label beside it already. */
+  .rhint { text-transform:none; letter-spacing:0; font-size:12px; }
+  .checked { margin-top:8px; font-size:12.5px; color:var(--muted); display:flex; gap:14px; flex-wrap:wrap; align-items:baseline; }
+  /* One per row on a phone: four of these side by side break into fragments
+     that read as unrelated words. */
+  @media (max-width: 700px) {
+    .checked { display:grid; grid-template-columns:1fr; gap:6px; }
+    .checked .c { justify-content:flex-start; }
+  }
   .checked .c { display:inline-flex; gap:5px; align-items:baseline; }
   .checked .c b { font-weight:500; color:var(--ink); }
   .checked .c.none b { color:var(--muted); }
-  .checked .c.old { opacity:.6; }
   .checked .c.bad b { color:var(--bad); }
   .checked .why { font-size:12px; }
   .flagged { margin-top:16px; border:1px solid var(--rule); background:var(--card); border-radius:16px; padding:14px 16px; }
@@ -238,6 +274,7 @@
       <video id="vid" controls playsinline></video>
       <div class="under">
         <span class="vtitle" id="vidName"></span>
+        <span class="vstate" id="vidState"></span>
         <span class="vmeta" id="vidMeta"></span>
         <span class="sp"></span>
         <select id="vidSpeed" title="Playback speed — preview now, and Download gets this speed">
@@ -256,7 +293,10 @@
         </div>
         <div id="flagRows"></div>
       </div>
-      <div class="checked" id="checked"></div>
+      <div class="review" id="review" style="display:none">
+        <div class="rhead">Optional review<span class="rhint">checks you can run against this recording — it is complete without them</span></div>
+        <div class="checked" id="checked"></div>
+      </div>
       <div class="frames">
         <div class="fhead">
           <span class="lab">Poster & frames</span>
@@ -335,20 +375,31 @@ $('newsBtn').onclick = async () => {
   const fmt = (d) => { const x = new Date(d + 'T00:00:00'); return isNaN(x) ? '' : x.toLocaleDateString(undefined, { month: 'long', day: 'numeric', year: 'numeric' }); };
   const veil = document.createElement('div');
   veil.className = 'veil';
-  veil.innerHTML = '<div class="modal wide"><h3>What\'s new</h3>' +
-    '<div class="scroll">' + rs.map((r, i) =>
+  // The newest release, and a way to the rest. Every release at once made the
+  // one thing that changed today compete with everything that ever changed.
+  let showAll = false;
+  const body = () => (showAll ? rs : rs.slice(0, 1)).map((r, i) =>
       '<div class="rel' + (i ? '' : ' latest') + '">' +
         '<div class="relhead"><b>' + esc(r.headline) + '</b><span class="relmeta">' + esc(r.version) + (r.date ? ' · ' + fmt(r.date) : '') + '</span></div>' +
         '<ul>' + (r.notes || []).map((n) => '<li>' + esc(n) + '</li>').join('') + '</ul>' +
-      '</div>').join('') + '</div>' +
-    '<div class="row" style="margin-top:18px"><button class="btn" id="newsClose">Close</button></div></div>';
+      '</div>').join('');
+  const render = () => {
+    veil.innerHTML = '<div class="modal wide"><h3>What\'s new</h3><div class="scroll">' + body() + '</div>' +
+      '<div class="row" style="margin-top:18px">' +
+      (showAll || rs.length < 2 ? '' : '<button class="btn" id="newsAll">All ' + rs.length + ' releases</button>') +
+      '<span class="sp"></span><button class="btn" id="newsClose">Close</button></div></div>';
+    const all = veil.querySelector('#newsAll');
+    if (all) all.onclick = () => { showAll = true; render(); };
+    veil.querySelector('#newsClose').onclick = () => veil.remove();
+  };
+  render();
   document.body.appendChild(veil);
   localStorage.setItem(SEEN_KEY, rs[0].version);
   $('newsBtn').querySelector('.ndot')?.remove();
-  const shut = () => veil.remove();
-  veil.querySelector('#newsClose').onclick = shut;
-  veil.onclick = (e) => { if (e.target === veil) shut(); };
-  document.addEventListener('keydown', function esc2(e) { if (e.key === 'Escape') { shut(); document.removeEventListener('keydown', esc2); } });
+  veil.onclick = (e) => { if (e.target === veil) veil.remove(); };
+  // `shut()` does not exist here — Escape threw a ReferenceError and left the
+  // modal sitting there. Copied from a dialog that had one.
+  document.addEventListener('keydown', function esc2(e) { if (e.key === 'Escape') { veil.remove(); document.removeEventListener('keydown', esc2); } });
 };
 
 document.getElementById('gremlinBtn').onclick = async () => {
@@ -430,6 +481,15 @@ async function pin(name) {
   const t = d.take;
   const failed = (t.timeline || []).find((e) => e.ok === false);
   $('vidName').textContent = name;
+  // What the RECORDING did — separate from anything optional that looked at it.
+  const stateEl = $('vidState');
+  // Failure first, always. `brisk` was tested before `ok`, so a brisk take
+  // whose steps fell over displayed the neutral "Brisk take" and nothing said
+  // it had failed — on the one path people use while iterating fast.
+  if (t.ok === false) { stateEl.className = 'vstate bad'; stateEl.textContent = t.brisk ? 'Brisk take — steps failed' : 'Steps failed'; stateEl.title = 'Some steps did not pass — see the proof log.'; }
+  else if (t.brisk) { stateEl.className = 'vstate part'; stateEl.textContent = 'Brisk take'; stateEl.title = 'All the steps, none of the pacing. For iterating, not for keeping.'; }
+  else if (t.partial) { stateEl.className = 'vstate part'; stateEl.textContent = 'Partial'; stateEl.title = t.partial; }
+  else { stateEl.className = 'vstate done'; stateEl.textContent = 'Recording complete'; stateEl.title = 'The take is sound: every step passed and the artifacts check out.'; }
   $('vidMeta').textContent = `${(t.duration - t.trimBefore).toFixed(0)}s · ${t.ok ? 'all steps passed' : failed ? `stopped at step ${failed.index}: ${failed.summary}` : 'some steps failed'}${d.facts ? ` · ${d.facts.width}×${d.facts.height}` : ''}`;
   $('vidMeta').title = failed?.screen ? 'On screen at that moment: ' + failed.screen : '';
   $('vidDl').href = `/out/${name}/demo.mp4${q}`; $('vidDl').setAttribute('download', name + '.mp4');
@@ -484,7 +544,8 @@ function drawChecked(checks, take) {
   const now = take && take.finishedAt;
   // Labelled, because four bare dashes read as four things missing. They are
   // optional checks against a finished recording, not parts of it.
-  box.innerHTML = `<span class="c none" title="These are optional checks you can run against this take. The recording is complete without them.">checked:</span>` + CHECKS.map(([key, label, title]) => {
+  $('review').style.display = '';
+  box.innerHTML = CHECKS.map(([key, label, title]) => {
     const r = checks && checks[key];
     if (!r) return `<span class="c none" title="${esc(title)} — not run on this take. Optional; the recording is complete without it.">${label} <b>—</b></span>`;
     const stale = !now || r.takeFinishedAt !== now;

package/dist/ui/guide.html

@@ -6,7 +6,7 @@
 <title>How Retake works</title>
 <link rel="icon" href="/favicon.svg" type="image/svg+xml" />
 <style>
-  :root { --paper:#edefe8; --card:#f8f9f5; --ink:#191d19; --muted:#767e74; --rule:#dbdfd3; --amber:#e8b04b; --accent:#1a1c1e; }
+  :root { --paper:#edefe8; --card:#f8f9f5; --ink:#191d19; --muted:#656c63; --rule:#dbdfd3; --amber:#e8b04b; --accent:#1a1c1e; }
   * { box-sizing:border-box; }
   body { margin:0; background:var(--paper); color:var(--ink); font:16px/1.7 -apple-system, BlinkMacSystemFont, "SF Pro Text", "Helvetica Neue", Inter, Arial, sans-serif; -webkit-font-smoothing:antialiased; }
   .top { display:flex; align-items:center; gap:10px; padding:14px 22px; --rule2:#c9cec0; }
@@ -16,6 +16,8 @@
   .top a { display:flex; align-items:center; gap:10px; text-decoration:none; color:inherit; }
   .top h1 { font-size:15px; font-weight:600; margin:0; }
   main { max-width:640px; margin:0 auto; padding:28px 22px 80px; }
+  /* The flow needs three columns of room; the prose below stays narrow. */
+  @media (min-width: 900px) { main { max-width:960px; } main > section, main > .lead, main > h2, main > .loop { max-width:640px; } }
   h2 { font-size:22px; letter-spacing:-.01em; margin:0 0 6px; }
   .lead { color:var(--muted); margin:0 0 34px; }
   section { margin-bottom:34px; }
@@ -28,6 +30,24 @@
   .quiet { color:var(--muted); font-size:14px; }
   code { background:var(--card); border:1px solid var(--rule); border-radius:6px; padding:1px 7px; font:14px ui-monospace, Menlo, monospace; }
   hr { border:0; border-top:1px solid var(--rule); margin:34px 0; }
+  /* The first three steps are one loop, not three articles. Prose can say
+     that; only a shape shows it. */
+  .flow { display:grid; grid-template-columns:1fr auto 1fr auto 1fr; gap:14px; align-items:start; margin:10px 0 18px; }
+  .beat { background:var(--card); border:1px solid var(--rule); border-radius:14px; padding:16px; }
+  .beat .n { display:inline-flex; align-items:center; justify-content:center; width:22px; height:22px; border-radius:999px; background:var(--accent); color:#eceee9; font-size:12px; font-weight:600; }
+  .beat h3 { margin:10px 0 8px; font-size:15px; }
+  .beat p { font-size:13.5px; line-height:1.6; color:var(--muted); margin:10px 0 0; }
+  .beat .say { font-size:13.5px; margin:0; padding:10px 13px; }
+  .thing { display:flex; flex-direction:column; gap:5px; background:var(--paper); border:1px solid var(--rule); border-radius:10px; padding:11px 13px; font-size:14px; }
+  .thing span { color:var(--muted); font-size:13px; }
+  .arrow { color:#c9cec0; font-size:20px; padding-top:56px; }
+  .loop { display:flex; gap:10px; align-items:baseline; font-size:14px; color:var(--muted); background:var(--card); border:1px solid var(--rule); border-radius:12px; padding:12px 16px; }
+  .loop span { color:var(--amber); font-size:16px; }
+  /* Stacked on a phone, with the arrows turned to point down the page. */
+  @media (max-width: 760px) {
+    .flow { grid-template-columns:1fr; gap:10px; }
+    .arrow { padding:0; text-align:center; transform:rotate(90deg); }
+  }
 </style>
 </head>
 <body>
@@ -40,33 +60,59 @@
   <h2>How Retake works</h2>
   <p class="lead">Describe a demo of your app in a sentence. Retake drives a real browser through it and hands you the video. When the app changes, you re-run — nobody re-records anything by hand.</p>
 
-  <section>
-    <h3>1 · Ask for a demo</h3>
-    <p>Talk to the agent you already use — Claude Code or Codex. It has Retake's tools.</p>
-    <div class="say">Record a demo of my app at localhost:3000 showing the sign-up flow.</div>
-    <div class="say">What demos are worth recording for this project? Record the best three.</div>
-    <p class="quiet">The agent reads your app, writes the steps, checks every one of them against the live page, and only then records. You can watch it work on this window's front page.</p>
-  </section>
+  <div class="flow">
+    <div class="beat">
+      <span class="n">1</span>
+      <h3>You say it</h3>
+      <div class="say">Record a demo of my app at localhost:3000 showing the sign-up flow.</div>
+      <p>To the agent you already use. It reads your app, writes the steps, checks every one against the live page, and only then records.</p>
+    </div>
+    <div class="arrow" aria-hidden="true">&rarr;</div>
+    <div class="beat">
+      <span class="n">2</span>
+      <h3>You watch it</h3>
+      <div class="thing"><b>demo.mp4</b><span>a still for every scene</span><span>a written record of what happened</span></div>
+      <p>It lands in this window, newest on top. <b>Download</b> for the file, <b>Show in folder</b> for the rest.</p>
+    </div>
+    <div class="arrow" aria-hidden="true">&rarr;</div>
+    <div class="beat">
+      <span class="n">3</span>
+      <h3>You change your mind</h3>
+      <div class="say">No zooms. Hide the cursor. Square for a post.</div>
+      <p>In plain words, and most of it comes back in seconds &mdash; the recording is already made. Say it once and Retake keeps it for the project.</p>
+    </div>
+  </div>
+  <p class="loop"><span>&#8634;</span> And when your app changes, you run the same file again. That is the whole idea: the demo outlives the video.</p>
 
-  <section>
-    <h3>2 · Judge the take</h3>
-    <p>The video lands here — newest on top, one still per scene under it. <b>Download</b> for the file, <b>Show in folder</b> for everything else — the stills, the proof log, the manifest.</p>
-  </section>
+  <hr>
 
   <section>
-    <h3>3 · Direct it like a videographer</h3>
-    <p>Say what you want changed, in plain words:</p>
+    <h3>When you want to know it is right</h3>
+    <p>A recording is finished when it records. Everything below is optional, run
+    afterwards on a take you already have, and none of it changes the video.</p>
+    <p>Ask your agent in plain words. It has all of these.</p>
     <ul>
-      <li>“No zooms — keep the camera still.”</li>
-      <li>“Hide the cursor.” · “Shorter captions.” · “No captions.”</li>
-      <li>“Hold the ending longer.” · “Cut the first two seconds.”</li>
-      <li>“Make it square for a post.” · “Vertical for a reel.”</li>
+      <li><b>Does it look right?</b> Write what should be visible on a scene —
+        <i>“the board shows two items”</i> — and every later recording answers it.</li>
+      <li><b>What did nobody think to check?</b> Every frame gets read for the ten
+        ways a screen goes wrong: text cut off, things overlapping, writing you
+        cannot read, a spinner still spinning, a dev badge left in shot.</li>
+      <li><b>Does it add up?</b> What the demo typed against what came back. It found
+        a quote showing one price for two flights that way.</li>
+      <li><b>What happens off the happy path?</b> The button pressed twice, the page
+        reloaded mid-flight, the app with nothing in it yet.</li>
+      <li><b>Did that get fixed?</b> Say something is wrong once and every later take
+        answers it, with the six seconds of video that show it.</li>
     </ul>
-    <p class="quiet">Say it once — Retake remembers it for the project. Most changes re-render in seconds without re-recording.</p>
+    <p class="quiet">Tell it what your product is for and these get much better —
+    otherwise they judge your app as a generic web page and anything you did on
+    purpose can come back as a mistake. Say it once; it is kept.</p>
+    <p class="quiet">They report, they do not block. On Retake's own help page the
+    frame check found eight of eight faults deliberately planted in it and
+    complained about nothing on a clean recording, and that number is a floor
+    rather than a promise.</p>
   </section>
 
-  <hr>
-
   <section>
     <h3>Where things live</h3>
     <p>Each demo is a small YAML file in <code>demos/</code> — that file <i>is</i> the demo. Videos land in <code>outputs/&lt;name&gt;/</code>: <code>demo.mp4</code>, <code>stills/</code>, and <code>proof-log.md</code> saying exactly what happened, step by step.</p>

package/dist/ui/server.js

@@ -40,6 +40,7 @@ import { digest } from "../digest.js";
 import { readFlags } from "../ext/flags.js";
 import { findOrphans } from "../ext/heal.js";
 import { readChecks } from "../ext/checked.js";
+import { productName } from "../ext/intent.js";
 import { running } from "../progress.js";
 import { MAX_BODY, checkRequest, clearToken, newToken, readBodyLimited, writeToken } from "./guard.js";
 import { startOffer, startApp, listeningPorts } from "../appserver.js";
@@ -201,14 +202,31 @@ function listDemos(project) {
         return [];
     const wanted = project?.trim() ? projectKey(project) : null;
     const assignments = demoProjects();
+    // The precedence for what a demo's folder is called, in one place, because
+    // it was three places and the release note described a fourth:
+    //
+    //   1. a folder you assigned by hand
+    //   2. a folder assigned by hand to another demo of the same app
+    //   3. the name in this demo's own product note
+    //   4. the name in a product note written for another demo of the same app
+    //   5. the app's address, which is where this started and helps nobody
+    //
+    // Four is new. Writing `retake intent --demo checkout "# Acme"` named the
+    // checkout folder and left the other six Acme demos in a fold called
+    // "localhost:3200", which is the exact complaint the naming was added to
+    // answer.
     const byUrl = new Map();
+    const namedByUrl = new Map();
     for (const f of fs.readdirSync(DEMOS).filter((x) => /\.ya?ml$/.test(x))) {
         const mm = cachedManifest(path.join(DEMOS, f));
-        if (mm) {
-            const asg = assignments[mm.name];
-            if (asg && mm.url && !byUrl.has(mm.url))
-                byUrl.set(mm.url, asg);
-        }
+        if (!mm)
+            continue;
+        const asg = assignments[mm.name];
+        if (asg && mm.url && !byUrl.has(mm.url))
+            byUrl.set(mm.url, asg);
+        const named = productName(DEMOS, mm.name);
+        if (named && mm.url && !namedByUrl.has(mm.url))
+            namedByUrl.set(mm.url, named);
     }
     const listed = fs
         .readdirSync(DEMOS)
@@ -239,7 +257,10 @@ function listDemos(project) {
         // that points at the same app — otherwise an agent's new draft for
         // Capture lands in a second "localhost:3100" fold next to "capture".
         const assigned = assignments[name] ?? (url ? byUrl.get(url) : undefined);
-        const group = assigned ? path.basename(assigned) : shortGroup(url);
+        // A product that has said what it is called is filed under that name.
+        // Grouping by URL gave folders like "localhost:3200" — true, and no help
+        // to anybody looking for their charter demos.
+        const group = assigned ? path.basename(assigned) : (productName(DEMOS, name) ?? (url ? namedByUrl.get(url) : undefined) ?? shortGroup(url));
         // Heal a folder whose run was killed mid-recording: the previous take is
         // in its stash and nothing has put it back yet. The window must never
         // show someone an empty demo when their recording is safe on disk.
@@ -276,7 +297,7 @@ function orphanedRecordings(known) {
         // rows that look like failed demos and are not demos at all.
         .filter((o) => o.rendered && !o.fragment && !known.includes(o.name))
         .map((o) => ({
-        name: o.name, file: "", title: o.title, url: o.url, group: shortGroup(o.url),
+        name: o.name, file: "", title: o.title, url: o.url, group: productName(DEMOS, o.name) ?? shortGroup(o.url),
         valid: true, settings: {},
         lastTake: o.finishedAt ? { finishedAt: o.finishedAt, ok: true, partial: null, duration: o.seconds ?? 0 } : null,
         needsRecord: true,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "retake-demos",
-  "version": "0.12.1",
+  "version": "0.13.2",
   "description": "Demo-as-code for coding agents: a YAML manifest in, a product walkthrough video out. Rerun the demo instead of re-recording it.",
   "keywords": [
     "demo",

package/site/releases.json

@@ -1,6 +1,83 @@
 {
   "_readme": "What's new, written for the person using Retake — not for whoever wrote it. Both the landing page and the Retake window read THIS file; there is no second copy. Rules for adding one: say what changed for them, name no files, no flags they will not type, no words like refactor or regression. If a line only makes sense to someone who has read the source, it does not belong here. Newest first. Add an entry on every release that changes what people see.",
   "releases": [
+    {
+      "version": "0.13.2",
+      "date": "2026-08-28",
+      "headline": "The thing that measures the checks was not measuring",
+      "notes": [
+        "Calibration is the number that says how much a visual finding is worth. Run against an app that was not even running, it used to print \"no false positives on the control\" and finish happily — because nothing had happened, so nothing was wrong. It now refuses: a run that fell over, recorded no steps, or produced no frames measures nothing, says which, and does not pretend otherwise.",
+        "One of its eight seeded faults could never have worked. A typo in the unreadable-contrast seed meant its own self-check could never be true, so that fault reported \"did not land\" on every app it was ever tried on. Fixed, and every seed is now tested in a real browser so a silently dead one cannot happen again.",
+        "Two more honest things it now says out loud: that two of the ten things it looks for have no seeded test at all, so the score says nothing about them; and whether asking the same question about the same pictures twice gives the same answer. That second one was promised in the code for two releases and never actually measured.",
+        "The checks run in a browser, and the browser was never installed on the build server — so every automated run since those tests were written failed before it started, and the build and packaging checks after them never ran. Installed now.",
+        "Fixed: pressing Escape on the What's New window did nothing. Fixed: a quick take whose steps failed showed a neutral \"Brisk take\" and never said it had failed. Fixed: the small print in the review area was too faint to meet the readability standard for text that size. Fixed: naming one demo's product now names the folder for every demo of the same app, not just that one. Fixed: a dry run now really scrolls when the demo says to scroll.",
+        "The agent tool for calibration used to advertise a setting that would run a fresh measurement, and always answered by telling you to go and run it yourself. That setting is gone.",
+        "And the landing page said \"nothing uploaded\", which was true of recording and not of the optional visual checks — those send the frames they look at to whichever model you point them at. It now says so."
+      ]
+    },
+    {
+      "version": "0.13.1",
+      "date": "2026-08-28",
+      "headline": "The help page explains the checks, and the watcher stops saying something untrue",
+      "notes": [
+        "The How it works page inside Retake never mentioned any of the checks. It now explains them in a paragraph each, and says plainly that a recording is finished when it records — everything else is optional and runs afterwards on a take you already have.",
+        "Fixed: the workspace summary told you a demo had never been checked when that demo carried seventeen written expectations, all of which had just been answered. It only says that now when there is genuinely not one."
+      ]
+    },
+    {
+      "version": "0.13.0",
+      "date": "2026-08-28",
+      "headline": "You can now measure how much the picture checks are worth",
+      "notes": [
+        "New: Retake can test its own eyesight. It takes a demo that works, records it once for each way a screen can go wrong — text cut off, things overlapping, writing you can't read, content off the edge, something drawn twice, a stuck spinner, a broken image, a developer badge left in shot — and once more with nothing wrong at all. Then it counts what the check found.",
+        "The first measurement, on Retake's own help page: all eight seeded faults found, and nothing at all reported on the clean recording.",
+        "It says what that number is and is not. The faults were written by somebody who knew what the check looks for, so it is a floor for trust rather than a score, and a fault nobody thought to seed is still unmeasured.",
+        "It also refuses to fool itself: if a seeded fault never actually appeared on screen, the run says so instead of blaming the check. That happened three times while this was being built, and each time it looked exactly like a miss."
+      ]
+    },
+    {
+      "version": "0.12.5",
+      "date": "2026-08-28",
+      "headline": "How Retake works, shown rather than written",
+      "notes": [
+        "The How it works page opened with three numbered articles you had to read. It now shows the loop: you say it, you watch it, you change your mind — and when your app changes you run the same file again. Same words, arranged so the shape is visible before you read a sentence.",
+        "On the website, the three products Retake was built inside now show a frame from the actual recording instead of only describing one. The third says plainly that its walkthrough belongs to the client and is not mine to publish, which is the honest version of a case study."
+      ]
+    },
+    {
+      "version": "0.12.4",
+      "date": "2026-08-28",
+      "headline": "Your demos get filed under your product's name",
+      "notes": [
+        "The window used to file demos under whatever it could work out from the address — so real products ended up in folders called localhost:3200 and localhost:4990. True, and no help to anyone looking for their charter demos.",
+        "Name your product in its note and that name becomes its folder. One thing to write, two things fixed: the checks stop judging your app as a generic web page, and your library reads like your work instead of a list of ports.",
+        "What's new now opens on the newest release, with a way to the rest — instead of every release you have ever had, all at once.",
+        "On the website: the five bugs a recording found now sit directly under the first video instead of most of the way down the page, and two sections that explained the same thing twice are one."
+      ]
+    },
+    {
+      "version": "0.12.3",
+      "date": "2026-08-28",
+      "headline": "A finished recording now looks finished",
+      "notes": [
+        "Fixed: a completed recording was quiet grey text while a failed optional check was a bold red mark — so a demo that recorded perfectly read as broken. Every recording now says what it is, next to its name: complete, partial, brisk, or steps failed.",
+        "The optional checks have moved into their own area, headed and explained. They are things you can run against a finished recording, and the window now looks like that instead of like four missing parts.",
+        "On a phone: each check gets its own line instead of breaking into fragments, the buttons along the top stop wrapping inside themselves, and Use current frame is a proper full-width button rather than three wrapped lines.",
+        "On the website: the button that says it copies the setup prompt now copies it. It used to scroll you five thousand pixels down the page to a different button that did the copying.",
+        "And two sentences on that page have been corrected. They claimed watching a recording checks everything at once. Retake's own evidence says otherwise — one pass can miss something a second pass finds — so the page now says what is true: a recording puts the whole screen in front of you, the optional checks surface things nobody wrote down, and you decide what matters."
+      ]
+    },
+    {
+      "version": "0.12.2",
+      "date": "2026-08-28",
+      "headline": "A security fix, and the starter demo now passes its own check",
+      "notes": [
+        "Fixed, and please update: naming a demo when writing your product note could write the file outside your demos folder. Any name is now refused unless it is an ordinary demo name.",
+        "Fixed: `retake dry` failed on `scroll to: bottom` — it looked for something on the page called \"bottom\" instead of understanding it as the end of the page. The starter demo that ships with a new workspace uses it, so a new person's very first check failed on a demo that records perfectly.",
+        "The check that tells you whether something you flagged is fixed now reads your product note too, like the others already did.",
+        "Reading your product note is now part of the written order of work, not a footnote — it comes before anything looks at a frame."
+      ]
+    },
     {
       "version": "0.12.1",
       "date": "2026-08-28",

package/skill/SKILL.md

@@ -49,20 +49,24 @@ say so in one line and stop.
    selectors, logins under `auth.setup` with `${ENV}` placeholders and
    `secret: true`, `reducedMotion: true` if things animate forever, scrolls
    before anything below the fold.
-5. **`dry`** — ALWAYS before `run`. Seconds, no camera, strict; failures
+5. **`intent`** — read it. One call, and it is what stops the checks judging
+   this app as a generic web page. If there is nothing there and the person
+   has told you what their product is, write it; if they have not, say so
+   when you report, and never invent one.
+6. **`dry`** — ALWAYS before `run`. Seconds, no camera, strict; failures
    include what was on screen. Fix and repeat until it passes. Never record
    a manifest whose dry run fails.
-6. **While iterating, record at `--preset draft`** (CLI) — same layout as the
+7. **While iterating, record at `--preset draft`** (CLI) — same layout as the
    final at a quarter of the pixels, so a take costs ~real time instead of
    real time plus an encoder. Switch to a post preset only for the take you
    intend to keep.
-7. **`run` with `preview=true`**, then read the `receipts` AND `look` at
+8. **`run` with `preview=true`**, then read the `receipts` AND `look` at
    the take. Receipts say what happened; `look` shows how it looks — one
    image per scene. Judge it like a viewer would: is the thing the caption
    names actually on screen? Anything in shot that shouldn't be (dev
    badges, leftover data, cut-off text)? Wrong story or failed step →
    `edit`, run again. More than four rounds → ask the person.
-8. **Before the expensive take, if it is expensive, show them the draft.**
+9. **Before the expensive take, if it is expensive, show them the draft.**
    `run` prints an estimate before capture. If it said this is a long take
    (over ~2.5 minutes), do not go straight to full quality: `ask` them,
    naming the demo and pointing at the draft in the Retake window, and say
@@ -79,7 +83,7 @@ say so in one line and stop.
    - **Nobody there to ask?** Never block and never guess. Leave the draft,
      `done` with what you have and what it would cost, and let them start
      the expensive one when they are back.
-9. When they say go: `run` with `preview=false` once (full quality), then
+10. When they say go: `run` with `preview=false` once (full quality), then
    `done` with one sentence. Outputs: `demo.mp4`, `stills/` (one PNG per
    scene), `proof-log.md`.
 
@@ -130,6 +134,18 @@ Three things about reading what it says:
 - **A clean sweep is a floor, not a ceiling.** A closed checklist cannot find
   what is not on it. It catches the ten things that go wrong in every app; it
   does not certify the frame.
+- **Say what it is worth, if anybody has measured.** `calibrate` seeds known
+  defects into a real recording of the app and counts what `sweep` finds. Read
+  the report before telling somebody how much a finding is worth, and quote its
+  limits rather than only its number: the seeds were written by somebody who
+  had read the checklist, so recall is an upper bound; two of the ten checklist
+  items have no seed at all, so the number says nothing about them; and a run
+  whose control did not record cleanly measured nothing. Earlier versions of
+  this file quoted "seven or eight of eight, no false positives" — one of those
+  eight seeds could never have landed, because its proof was broken, so the
+  denominator was wrong and the claim about repeated runs had never been
+  measured. Quote the report you have, not a remembered figure. Never run it
+  unasked: nine takes, about ten minutes, and only from a terminal.
 - **One pass is not reliable, and this is measured.** On a frame with a real
   overlap on it, three consecutive sweeps missed it and a fourth found it. The
   judge is not deterministic. So a frame that has already been looked at is
@@ -163,6 +179,12 @@ left in the navigation of a customer-facing recording.
 Same number of findings. Better findings.
 
 - `intent` with no text **reads** it. Do this before you report anything.
+- **The first heading names the product, and that name becomes its folder in
+  the window.** Without one the window files demos by URL — real workspaces
+  end up with folders called `localhost:3200` and `localhost:4990`, which is
+  accurate and no help to anybody looking for their charter demos. Naming the
+  product once, for the checks, names its folder too. That part is your job:
+  the person should not have to tidy up after you.
 - `intent` with text **writes** it — the first time the person explains what
   their product does, who uses it, or why something unusual is on purpose.
   Use `demo` to write it for one app; a workspace with three products cannot
@@ -209,6 +231,7 @@ These are separate questions. Running the wrong one and reporting it as
 | What happens just off the happy path? | `destroy` | minutes | flags, you judge |
 | Did the things they flagged get fixed? | `fixed` | ~13s each | yes — exit 3 |
 | What is this product FOR? | `intent` | instant | context, not a check |
+| How much is a `sweep` finding worth here? | `calibrate` | ~10 min, nine takes | a measurement, not a check |
 | What keeps going wrong across every demo? | `notes` | instant | no |
 
 `dry` really does click and fill — at full speed, with short timeouts, no