driftproof 0.10.0 → 0.10.2

package/README.md

@@ -25,13 +25,13 @@ differ in what moves underneath the skill — or, in the value report, in which
 axes are measured; or, in the instrument re-measurement, in the instrument itself:
 
 - **[Report #001](https://driftproofhq.com/reports/001/)** — *release drift*: ten
-  public agent skills across a current-vs-previous Sonnet release; 9 of 10 moved
-  beyond noise. ([markdown](reports/report-001.md))
+  public agent skills across a current-vs-previous Sonnet release; 9 of 10 showed
+  a separation detected under the rule. ([markdown](reports/report-001.md))
 - **[Report #002](https://driftproofhq.com/reports/002/)** — *substrate
   durability*: the same suites across two vendors' CLIs (Claude vs Codex);
   3 durable, 4 substrate-dependent, 2 regressed, 1 no effect.
 - **[Report #003](https://driftproofhq.com/reports/003/)** — *release drift*:
-  `claude-opus-4-8` → `claude-opus-5`; 4 improved, 2 regressed, 4 within noise.
+  `claude-opus-4-8` → `claude-opus-5`; 4 improved, 2 regressed, 4 with no separation detected.
 - **[Report #004](https://driftproofhq.com/reports/004/)** — *capability gap*:
   `claude-opus-5` (flagship) vs `claude-fable-5` (frontier tier);
   3 durable, 5 tier-dependent, 0 regressions, 2 no effect — encoded expertise
@@ -68,8 +68,9 @@ axes are measured; or, in the instrument re-measurement, in the instrument itsel
 - **[Report #008](https://driftproofhq.com/reports/008/)** — *release drift*:
   two of Report #007's cells re-measured on `claude-fable-5-1` against
   `claude-fable-5`, with the skill `content_hash` and `suite_hash` asserted
-  identical before the first call. **Both cells came back within noise**: the
-  14 cases read 0 improved, 0 regressed, 14 within noise, 0 not measured. The
+  identical before the first call. **Neither cell showed a separation detected
+  under the rule**: the 14 cases read 0 improved, 0 regressed, 14 with no separation
+  detected, 0 not measured, which is not evidence that nothing changed. The
   first release pair in this project where both sides are generation-sampled
   receipts, which is what makes the delta attributable to the model rather than
   to the instrument. One case sits inside the verdict on the effect floor alone
@@ -97,8 +98,10 @@ instead of a stale one.
 The hard part isn't running an eval once — it's making the number **credible
 enough to act on**. An LLM judge is noisy, so a naive score can swing run to run
 by more than the drift you're trying to detect. Driftproof's answer is to **sample
-the judge and report confidence bands**, and to **only claim a regression when the
-bands don't overlap**. A tool that cries wolf is worse than no tool.
+and report bands**, each a descriptive spread (the mean plus or minus one sample
+standard deviation), and to **only claim a regression when the bands don't
+overlap**: a separation detected under that rule, not proof. A tool that cries wolf
+is worse than no tool.
 
 **A verdict without a price is half an answer.** The same receipts price the
 marginal cost of a skill firing, and Report #005 found the dominant cost driver is
@@ -139,17 +142,24 @@ effect floor, `NO_EFFECT` when it doesn't, `REGRESSED` when the skill hurts. Eac
 case is judged several times, so it carries a **band** (`mean ± stddev`) instead of
 one fragile number, and a case is only ever called regressed/improved when its two
 bands don't overlap. The **effect floor** (0.05, one judge quantization step) is the
-minimum real move required before a change counts as more than noise — band
-separation *plus* a floor-sized delta, never either alone.
-
-**What the band does not cover.** A verdict rests on **one generation draw per
-arm**: the band is the spread of the *judge* re-scoring that single response, not
-the spread of the model writing a different one. Report #006 measured the second
-directly and found it larger — draw-to-draw spread up to **sd 0.186** on the 0–1
-scale, against judge-level noise several times smaller. So treat a surprising
-single-run verdict as **provisional and worth re-running** before you act on it.
-Generation sampling lands in the next receipt spec; until it does, this is a
-limit of the instrument, stated rather than implied.
+minimum move required before a separation is called a change: band separation
+*plus* a floor-sized delta, never either alone.
+
+**What the band covers.** Since receipt spec v0.5 a run **samples the generation**
+as well as the judge: each arm is drawn at least 3 and at most 10 times
+(`GENERATION_SAMPLES_MIN` and `GENERATION_SAMPLES_MAX` in `config.js`, applied by
+`lib/sampling.js`), and each draw is judged several times. A run stops at 3 draws
+when the across-draw spread is no wider than the 0.05 effect floor; otherwise it
+keeps drawing until two successive estimates of that spread agree to within 0.01, or
+the ceiling is reached, and each case records its `stopping_reason`. So a band on a
+generation-sampled receipt is the spread of the model writing different responses,
+not only of the judge re-scoring one. Report #006 measured that spread directly and
+found it larger than judge-level noise, draw-to-draw **sd 0.186** at the most, which
+is why v0.5 samples it. A receipt from before v0.5 carries a **legacy** band, one
+generation per arm re-scored by the judge, and a comparison report labels each band
+`(legacy)` or `(generation)`. Either way a band describes the draws a run made, at
+the sample size it used, so treat a surprising verdict as **worth re-running**
+before you act on it.
 
 ### Install
 
@@ -179,6 +189,13 @@ optional and pulled in only for `CLAUDE_PROVIDER=api`. The CLI resolves its spec
 schema, and model registry from inside the package, so it runs the same from a
 global/npx install as it does in a checkout.
 
+**Platforms.** Driftproof is tested on Linux and macOS. In an outside retest of
+0.10.1 on macOS the shipped gate passed 592 of 611, with 19 failed and 1 not
+applicable, and all nineteen failures are in the publishing helper, which needs
+GNU `realpath -m`. Windows is untested: from Node's source the plugin would not
+find `npx` there, but that failure has never been observed. A CI matrix across
+Linux, macOS and Windows is planned.
+
 ### Other commands
 
 ```bash
@@ -287,7 +304,7 @@ A receipt is the unit of evidence — one JSON document conforming to
     "model_release_date": "2025-10-01",
     "provider": "anthropic",
     "surface": "claude-cli",
-    "runner_version": "0.10.0",
+    "runner_version": "0.10.2",
     "date_utc": "2026-07-27T…Z",
     "registry": "registered",
     "transcripts": "hashes-only",
@@ -329,7 +346,7 @@ Key ideas:
 
 - **`content_hash` / `suite_hash`** are computed over a canonical JSON form, so the
   same skill and suite hash identically on any machine — receipts are comparable.
-- **Per-case `samples` / `mean` / `stddev`** give each case a confidence band. An
+- **Per-case `samples` / `mean` / `stddev`** give each case a band, a descriptive spread of one sample standard deviation. An
   `outcome` of **`borderline`** means the pass threshold sits *inside* the band —
   the run can't confidently call it pass or fail.
 - **`delta_uncertainty`** is the combined band on the with-skill-vs-baseline lift.
@@ -349,7 +366,8 @@ Key ideas:
 `driftproof diff A.json B.json` compares the with-skill bands per case across two
 receipts. The rule that keeps it honest: a **regression** (or improvement) is
 claimed **only when the two bands do not overlap**. Overlapping bands are reported
-as **within noise** and never counted as a regression.
+as **no separation detected** at the sample size used: never counted as a
+regression, and never evidence that nothing changed.
 
 ## Verification in CI (GitHub Action + badge)
 
@@ -365,7 +383,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: driftproofhq/driftproof@v0.10.0
+      - uses: driftproofhq/driftproof@v0.10.2
         with:
           skill-dir: skills/my-skill
           models: claude-haiku-4-5

package/config.js

@@ -9,7 +9,7 @@ const PROJECT_NAME = 'driftproof';
 // Bumped whenever the runner's behaviour or receipt-generation semantics change
 // in a way that could affect results. Recorded into every receipt as
 // run.runner_version so a receipt is reproducible against a known engine.
-const RUNNER_VERSION = '0.10.0';
+const RUNNER_VERSION = '0.10.2';
 
 // The eval format we CONSUME (we deliberately do not invent our own).
 const SUITE_FORMAT = 'agentskills.io/evals';
@@ -99,8 +99,8 @@ const DEFAULT_JUDGE_SAMPLES = 5;
 // to a coarse ~0.05–0.1 grid, so a confident grade often has stddev 0 (a "point
 // band"). Two point bands that differ by a single quantum (e.g. 0.60 vs 0.64)
 // are technically non-overlapping yet represent no meaningful behaviour change.
-// The floor turns such statistically-separated-but-trivial moves into
-// "within noise (below effect floor)". 0.05 = one judge quantum. Documented in
+// The floor keeps such separated but trivial moves from being called a verdict
+// ("below effect floor" in the per-case table). 0.05 = one judge quantum. Documented in
 // spec/RECEIPT.md § "Drift verdict rule" and the report methodology.
 const EFFECT_FLOOR = 0.05;
 
@@ -131,7 +131,7 @@ const REPORT_002_GPT_MODEL = 'gpt-5.6-sol';
 const REPORT_002_JUDGE_MODEL = 'claude-haiku-4-5';
 
 // Report #003 is a RELEASE DRIFT report (Report #001's type): one provider, two
-// model versions, verdicts REGRESSED/IMPROVED/MIXED/WITHIN NOISE per skill under
+// model versions, a per-skill verdict label (regressed, improved, mixed, or none separated) under
 // the per-case band-separation rule + effect floor. New vs its family predecessor.
 const REPORT_003_NEW_MODEL = 'claude-opus-5';
 const REPORT_003_OLD_MODEL = 'claude-opus-4-8';

package/lib/diff.js

@@ -1,7 +1,7 @@
 // SPDX-License-Identifier: Apache-2.0
 'use strict';
 
-const { bandVerdict, round } = require('./stats');
+const { bandVerdict, round, WITHIN_NOISE } = require('./stats');
 const { EFFECT_FLOOR } = require('../config');
 const { revisionHeadline } = require('./revision');
 const { baselineReproduces, REFUSAL_REASONS, bandOf } = require('./reuse');
@@ -9,8 +9,9 @@ const { baselineReproduces, REFUSAL_REASONS, bandOf } = require('./reuse');
 // Practical-significance gate applied ON TOP of band separation. bandVerdict()
 // stays a pure geometry test (kept that way so its unit checks are unambiguous);
 // this wrapper additionally requires |delta| >= EFFECT_FLOOR before a verdict is
-// claimed. A separated-but-trivial move (below the judge's quantization floor)
-// becomes "within noise (below effect floor)".
+// claimed. A separated but trivial move (below the judge's quantization floor)
+// gets the verdict value WITHIN_NOISE_FLOOR, which the table labels "below effect
+// floor".
 const WITHIN_NOISE_FLOOR = 'within noise (below effect floor)';
 function verdictWithFloor(before, after, delta) {
   const raw = bandVerdict(before.mean, before.stddev, after.mean, after.stddev);
@@ -19,17 +20,18 @@ function verdictWithFloor(before, after, delta) {
   }
   return raw;
 }
-function isWithinNoise(v) { return v === 'within noise' || v === WITHIN_NOISE_FLOOR; }
+function isWithinNoise(v) { return v === WITHIN_NOISE || v === WITHIN_NOISE_FLOOR; }
 
 // Build a drift report (markdown) between two receipts.
 //
-// CREDIBILITY CORE — the anti-false-positive rule: a per-case regression (or
-// improvement) is claimed ONLY when (1) the two confidence bands do NOT overlap
-// AND (2) the mean moved by at least EFFECT_FLOOR (practical-significance floor,
-// see config.js). When the bands overlap OR the move is below the floor, the
-// change is reported as "within noise" and is NOT counted as a regression. A tool
-// that cries wolf is worse than useless, so band separation PLUS a real-sized
-// delta — not either alone — is what triggers a verdict.
+// CREDIBILITY CORE, the anti-false-positive rule: a per-case regression (or
+// improvement) is claimed ONLY when (1) the two bands, each the mean plus or
+// minus one sample standard deviation, do NOT overlap AND (2) the mean moved by at
+// least EFFECT_FLOOR (practical-significance floor, see config.js). When the
+// bands overlap OR the move is below the floor, no separation is detected at the
+// sample size used, and the case is NOT counted as a regression. A tool that cries
+// wolf is worse than useless, so band separation PLUS a real-sized delta, not
+// either alone, is what triggers a verdict.
 
 // Map case-id → { mean, stddev } for a receipt's with_skill cases. Falls back to
 // score/0 for v0.1 receipts that have no per-case band.
@@ -124,14 +126,21 @@ function judgeDiffers(a, b, ids, aB, bB) {
 // It deliberately does NOT run a separate band test on the aggregate mean: the
 // aggregate band is suite dispersion, and a separate test there would either cry
 // wolf (if too tight) or mask real per-case drift (if too wide).
+//
+// THE WORDING (spec 031 A-031-20). A band is a descriptive spread, the mean plus or
+// minus one sample standard deviation. A separation is stated as detected under
+// the rule, never as proof that the skill moved; no separation is stated as none
+// detected at this sample size, never as evidence that nothing changed. The
+// leading word is the label; the verdict values it summarises are unchanged.
 function headlineVerdict(perCase) {
   const reg = perCase.filter((r) => r.verdict === 'regression').length;
   const imp = perCase.filter((r) => r.verdict === 'improvement').length;
   const s = (n) => (n === 1 ? '' : 's');
-  if (reg && imp) return `MIXED — ${reg} case regression${s(reg)} and ${imp} improvement${s(imp)} on non-overlapping bands.`;
-  if (reg) return `DRIFT — ${reg} case${s(reg)} regressed (bands do not overlap); the skill is measurably weaker on ${reg} case${s(reg)}.`;
-  if (imp) return `IMPROVED — ${imp} case${s(imp)} improved (bands do not overlap); none regressed.`;
-  return 'WITHIN NOISE — no case moved beyond its confidence band; the skill holds up.';
+  const rule = 'bands do not overlap and the move clears the effect floor';
+  if (reg && imp) return `MIXED: separation detected under the rule on ${reg} case${s(reg)} downward and ${imp} case${s(imp)} upward (${rule}).`;
+  if (reg) return `DRIFT: separation detected under the rule on ${reg} case${s(reg)}, downward (${rule}); none upward.`;
+  if (imp) return `IMPROVED: separation detected under the rule on ${imp} case${s(imp)}, upward (${rule}); none downward.`;
+  return 'NO SEPARATION DETECTED: no case separated under the rule at this sample size (band = mean \u00b1 1 sd); this is not evidence that nothing changed.';
 }
 
 // A revision pair is a pair in which the SKILL TEXT is the only thing that
@@ -199,7 +208,7 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B', mode = 'release' }
     return { id, before, after, delta, verdict };
   });
   // Sort worst-first: regressions, then by delta.
-  const order = { regression: 0, 'within noise': 1, [WITHIN_NOISE_FLOOR]: 1, improvement: 2, 'n/a': 3, 'not measured': 3, refused: 3 };
+  const order = { regression: 0, [WITHIN_NOISE]: 1, [WITHIN_NOISE_FLOOR]: 1, improvement: 2, 'n/a': 3, 'not measured': 3, refused: 3 };
   perCase.sort((x, y) => (order[x.verdict] - order[y.verdict]) || ((x.delta || 0) - (y.delta || 0)));
 
   const aAgg = aggWithBand(a);
@@ -310,10 +319,33 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B', mode = 'release' }
   // carries a source. A legend explains markers on the page; if `bandStr` emits
   // none, the legend is describing something the reader cannot see. Asked of
   // `bandStr` itself rather than recomputed, so the two cannot disagree.
-  const anyLabelRendered = [...Object.values(aB), ...Object.values(bB)]
-    .some((x) => x && /\([a-z]+\)\s*$/.test(bandStr(x)));
-  if (anyLabelRendered) {
-    warnings.push('band provenance: `(generation)` is an ACROSS-DRAW spread — receipt spec v0.5, n generation draws per arm. `(legacy)` is a JUDGE-SAMPLE spread over a single generation, which is what v0.4 and earlier recorded. They are different statistics. The comparison is the only one the older receipt admits, and it is not like for like.');
+  //
+  // THE LEGEND DESCRIBES ONLY THE SOURCES ON THE PAGE, AND THE PAIR IS CALLED
+  // NOT LIKE FOR LIKE ONLY WHEN IT IS NOT (spec 031 AC-6). This used to print one
+  // sentence whenever any label rendered, naming both sources and saying the
+  // comparison was the only one an older receipt admits and was not like for
+  // like. Every v0.4 and v0.5 band carries a label, so two v0.5 receipts with
+  // `generation` bands on both sides were told they were not like for like, and
+  // shown a `(legacy)` source neither carried. The sources are read per receipt,
+  // from the markers each side renders; the difference is stated only when the
+  // two sides' sets differ, naming which side carries which.
+  const renderedSources = (bands) => [...new Set(Object.values(bands)
+    .map((x) => (x && (/\(([a-z]+)\)\s*$/.exec(bandStr(x)) || [])[1]) || null).filter(Boolean))].sort();
+  const sourcesA = renderedSources(aB);
+  const sourcesB = renderedSources(bB);
+  const sourcesOnPage = [...new Set([...sourcesA, ...sourcesB])].sort();
+  const sourcesDiffer = sourcesA.join() !== sourcesB.join();
+  if (sourcesOnPage.length) {
+    const LEGEND = {
+      generation: '`(generation)` is an ACROSS-DRAW spread: the standard deviation of the case\'s score across n generation draws per arm.',
+      legacy: '`(legacy)` is a JUDGE-SAMPLE spread over a single generation: the standard deviation across the judge\'s samples of that one text.',
+    };
+    const legend = sourcesOnPage.map((s) => LEGEND[s] || `\`(${s})\` is a band source this differ does not describe.`);
+    if (sourcesDiffer) {
+      const side = (label, s) => `${label} carries ${s.length ? s.map((x) => `\`(${x})\``).join(' and ') : 'no labelled'} bands`;
+      legend.push(`The two receipts' bands come from different sources: ${side(labelA, sourcesA)}, ${side(labelB, sourcesB)}. They are different statistics, so a per-case comparison across them is not like for like.`);
+    }
+    warnings.push(`band provenance: ${legend.join(' ')}`);
   }
   if (warnings.length) {
     L.push('> **⚠ Caveats**');
@@ -350,7 +382,14 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B', mode = 'release' }
   } else {
     L.push(`**${revision ? revisionHeadline(perCase) : headlineVerdict(perCase)}**`);
     L.push('');
-    L.push(`with_skill mean moved ${fmt(headlineDelta)} (${bandStr(aAgg)} → ${bandStr(bAgg)}; band = suite dispersion). Per-case band-overlap verdicts: ${regressions.length} regression(s), ${perCase.filter((r) => r.verdict === 'improvement').length} improvement(s), ${nWithin} within noise${nFloor ? ` (${nFloor} of them band-separated but below the ${EFFECT_FLOOR} effect floor)` : ''}.`);
+    // ONE LABEL PER CASE, THE TABLE'S (F-5 of
+    // specs/031-artefact-claims/evidence/approval-20260915T032341Z.md). A case whose
+    // bands do not overlap but whose move is below the floor is labelled below
+    // effect floor in the per-case table. This line counted it among the cases with
+    // no separation detected and then called the same cases band-separated; it now
+    // counts the two apart, as the table labels them. Neither is a separation under
+    // the rule, which is what the headline above says.
+    L.push(`with_skill mean moved ${fmt(headlineDelta)} (${bandStr(aAgg)} → ${bandStr(bAgg)}; band = suite dispersion). Per-case band-overlap verdicts: ${regressions.length} regression(s), ${perCase.filter((r) => r.verdict === 'improvement').length} improvement(s), ${nWithin - nFloor} with no separation detected${nFloor ? `, ${nFloor} below the ${EFFECT_FLOOR} effect floor` : ''}.`);
     L.push('');
   }
 
@@ -359,7 +398,7 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B', mode = 'release' }
   L.push(`| case | ${labelA} (mean ± sd) | ${labelB} (mean ± sd) | Δ | verdict |`);
   L.push(`|---|---|---|---|---|`);
   for (const r of perCase) {
-    const flag = r.verdict === 'regression' ? '🔻 regression' : r.verdict === 'improvement' ? '🔼 improvement' : r.verdict === WITHIN_NOISE_FLOOR ? 'within noise (below floor)' : r.verdict === 'within noise' ? 'within noise' : r.verdict === 'not measured' ? 'not measured' : 'n/a';
+    const flag = r.verdict === 'regression' ? '🔻 regression' : r.verdict === 'improvement' ? '🔼 improvement' : r.verdict === WITHIN_NOISE_FLOOR ? 'below effect floor' : r.verdict === WITHIN_NOISE ? 'no separation detected' : r.verdict === 'not measured' ? 'not measured' : 'n/a';
     L.push(`| \`${r.id}\` | ${r.before ? bandStr(r.before) : 'n/a'} | ${r.after ? bandStr(r.after) : 'n/a'} | ${fmt(r.delta)} | ${flag} |`);
   }
   L.push('');

package/lib/judge.js

@@ -78,8 +78,15 @@ function promptTemplateHash() {
 const TRUNCATION_STOP_REASONS = new Set(['max_tokens', 'length']);
 function isTruncated(stopReason) { return TRUNCATION_STOP_REASONS.has(String(stopReason || '')); }
 
+// Only a JSON number is a score (spec 032 AC-1). Number() turned null, false,
+// "" and [] into 0 and true into 1, and each of those became a measured sample:
+// a judge that said nothing scored the worst possible grade, and two receipts
+// over the same generations read as a regression. Every non-number becomes NaN
+// here and is refused by the isFinite line below, which stays the one line every
+// non-number reaches (spec 026's clamp mutation is planted on it).
 function scoreOf(parsed) {
-  const x = Number(parsed && parsed.score);
+  const raw = parsed ? parsed.score : undefined;
+  const x = typeof raw === 'number' ? raw : NaN;
   if (!Number.isFinite(x)) return null;
   if (x < 0 || x > 1) return null;
   return x;
@@ -128,10 +135,11 @@ async function gradeOnce({ task, response, rubric, model, timeoutMs, temperature
   }
   const score = scoreOf(parsed);
   if (score === null) {
-    const x = parsed && parsed.score;
-    const reason = x === undefined || x === null ? 'judge output carries no numeric score'
-      : !Number.isFinite(Number(x)) ? `judge output carries no numeric score (score ${JSON.stringify(x)})`
-        : `judge score ${x} outside [0, 1]`;
+    const x = parsed ? parsed.score : undefined;
+    const reason = x === undefined ? 'judge output carries no numeric score'
+      : typeof x !== 'number' ? `judge output carries no numeric score (score ${JSON.stringify(x)})`
+        : !Number.isFinite(x) ? `judge output carries no numeric score (score ${String(x)})`
+          : `judge score ${x} outside [0, 1]`;
     return { ...base, unmeasured: true, reason };
   }
   return { ...base, score, reason: String(parsed.reason || '').slice(0, 300) };
@@ -143,7 +151,7 @@ async function gradeOnce({ task, response, rubric, model, timeoutMs, temperature
 // samples: a partial sample set must never become a band, and a draw one of
 // whose judge samples carried no score is unmeasured as a whole (spec 026
 // AC-3). The remaining samples are not taken.
-// `mean` ± `stddev` is the per-case confidence band (raw spread of the N scores)
+// `mean` ± `stddev` is the per-case band, a descriptive spread (one sample sd of the N scores)
 // used by the borderline-outcome rule and per-case drift band-overlap logic.
 // NO DEFAULT TIMEOUT HERE (spec 017 AC-2). This defaulted to 120000, which
 // outranked the per-surface policy exactly as lib/run.js's literal did — so the

package/lib/receipt.js

@@ -114,7 +114,7 @@ function comparisonOf(aggWith, aggBase) {
     baseline_score: aggBase.mean_score,
     delta: noCases ? null : round(aggWith.mean_score - aggBase.mean_score),
     // Combined uncertainty of the delta: quadrature sum of the two aggregate
-    // bands. Diff uses this for the headline "within noise" vs real-move rule.
+    // bands. A figure beside the delta; the per-case band rule decides the headline.
     delta_uncertainty: noCases ? null : combineUncertainty(aggWith.stddev, aggBase.stddev),
   };
   if (noCases) cmp.delta_uncertainty_unavailable = 'no_cases';

package/lib/revision.js

@@ -21,29 +21,38 @@ const { EFFECT_FLOOR } = require('../config');
 
 // ── the cell headline ────────────────────────────────────────────────────────
 // A summary of the per-case band-overlap verdicts, worded about the REVISION.
-// The release-drift headline says "the skill is measurably weaker", which is a
-// sentence about a skill under a moving model. Here the model is the control.
+// The release-drift headline is a sentence about a skill under a moving model.
+// Here the model is the control. Worded under spec 031 A-031-20: a separation is
+// detected under the rule, never proof; none detected is not evidence of sameness.
 function revisionHeadline(perCase) {
   const reg = perCase.filter((r) => r.verdict === 'regression').length;
   const imp = perCase.filter((r) => r.verdict === 'improvement').length;
   const s = (n) => (n === 1 ? '' : 's');
   if (reg && imp) {
-    return `MIXED — the revision improved ${imp} case${s(imp)} and regressed ${reg} on non-overlapping bands.`;
+    return `MIXED: separation detected under the rule on ${imp} case${s(imp)} upward and ${reg} downward under the current upstream text (bands do not overlap).`;
   }
   if (reg) {
-    return `REVISION REGRESSED — ${reg} case${s(reg)} scored lower under the current upstream text (bands do not overlap).`;
+    return `REVISION REGRESSED: separation detected under the rule on ${reg} case${s(reg)}, lower under the current upstream text (bands do not overlap).`;
   }
   if (imp) {
-    return `REVISION IMPROVED — ${imp} case${s(imp)} scored higher under the current upstream text (bands do not overlap); none regressed.`;
+    return `REVISION IMPROVED: separation detected under the rule on ${imp} case${s(imp)}, higher under the current upstream text (bands do not overlap); none lower.`;
   }
-  return 'WITHIN NOISE — the revision moved no case beyond its confidence band; the pinned text and the current text measure the same.';
+  return 'NO SEPARATION DETECTED: no case separated under the rule at this sample size (band = mean \u00b1 1 sd); this is not evidence that the pinned text and the current text behave alike.';
 }
 
-// Classification word for a cell, from its headline. Kept separate so a caller
-// can branch on the class without parsing prose.
+// Classification value for a cell. Kept separate so a caller can branch on the
+// class without parsing prose. It used to be the headline's first word, cut at
+// its dash; the headline's wording is now a display matter (spec 031 A-031-20),
+// so the values are stated here and are unchanged: scripts/prepare-report-006.js
+// and spec 009's gate read them. CLASS_NOT_SEPARATED is the value, not a label.
+const CLASS_NOT_SEPARATED = 'WITHIN NOISE';
 function revisionClass(perCase) {
-  const h = revisionHeadline(perCase);
-  return h.split(' —')[0];
+  const reg = perCase.filter((r) => r.verdict === 'regression').length;
+  const imp = perCase.filter((r) => r.verdict === 'improvement').length;
+  if (reg && imp) return 'MIXED';
+  if (reg) return 'REVISION REGRESSED';
+  if (imp) return 'REVISION IMPROVED';
+  return CLASS_NOT_SEPARATED;
 }
 
 // ── the fairness sentence ────────────────────────────────────────────────────
@@ -54,7 +63,7 @@ function revisionClass(perCase) {
 // one-directional courtesy — the symmetry is a property of the code, and the
 // gate asserts it.
 //
-// A cell within noise gets NO sentence. #005's figure stands unamended, because
+// A cell with no separation detected gets NO sentence. #005's figure stands unamended, because
 // nothing was measured that would amend it, and manufacturing a hedge for a null
 // result is how a report launders noise into a finding.
 function fairnessSentence({ slug, classification, report005Delta, measuredDelta }) {

package/lib/stats.js

@@ -1,8 +1,9 @@
 // SPDX-License-Identifier: Apache-2.0
 'use strict';
 
-// Small statistics helpers for sampled judging and confidence bands. Kept
-// dependency-free and deterministic.
+// Small statistics helpers for sampled scores and their bands. A band is a
+// descriptive spread, the mean plus or minus one sample standard deviation; it
+// carries no coverage probability. Kept dependency-free and deterministic.
 
 function round(n, dp = 6) { const f = Math.pow(10, dp); return Math.round(n * f) / f; }
 
@@ -60,16 +61,24 @@ function aggregateBands(cases) {
   return { mean: mean(means), stddev: means.length < 2 ? null : stddev(means) };
 }
 
-// Do two confidence bands (mean ± half-width) fail to overlap, and in which
-// direction? Returns 'regression' (b below a), 'improvement' (b above a), or
-// 'within noise' (bands touch/overlap). This is the anti-false-positive rule:
-// a change is only claimed when the bands are fully separated.
+// THE NOT-SEPARATED VERDICT VALUE, which is a value and not a label (spec 031
+// A-031-20). Callers compare against it, and every display label is mapped from
+// it where it is rendered: lib/diff.js prints "no separation detected". The value
+// itself is unchanged pending the verdict-rule spec.
+const WITHIN_NOISE = 'within noise';
+
+// Do two bands (mean ± half-width, the half-width one sample standard deviation)
+// fail to overlap, and in which direction? Returns 'regression' (b below a),
+// 'improvement' (b above a), or WITHIN_NOISE (the bands touch or overlap). This
+// is the anti-false-positive rule: a separation is detected only when the bands
+// are fully apart, and overlap is the absence of a detected separation at the
+// sample size used, not evidence that nothing changed.
 //   regression   : meanB + hwB < meanA - hwA
 //   improvement  : meanB - hwB > meanA + hwA
 function bandVerdict(meanA, hwA, meanB, hwB) {
   if (meanB + hwB < meanA - hwA) return 'regression';
   if (meanB - hwB > meanA + hwA) return 'improvement';
-  return 'within noise';
+  return WITHIN_NOISE;
 }
 
 // The variance ratio: how much larger the GENERATION-level spread is than the
@@ -87,4 +96,4 @@ function varianceRatio(generationSd, judgeSdMean) {
   return round(g / j);
 }
 
-module.exports = { mean, stddev, stderr, combineUncertainty, aggregateBands, bandVerdict, round, varianceRatio };
+module.exports = { mean, stddev, stderr, combineUncertainty, aggregateBands, bandVerdict, round, varianceRatio, WITHIN_NOISE };

package/lib/value.js

@@ -25,8 +25,8 @@ const { EFFECT_FLOOR } = require('../config');
 //
 // RATIO FRAMINGS ARE FLOOR-GATED. A ratio like "dollars per 0.01 lift" is
 // only meaningful when the benefit it prices is a real move. When the lift did not clear
-// the effect floor (or its bands overlap), the ratio renders "n/a (within noise)"
-// — never a number, however tempting the arithmetic. Dividing noise by a cost
+// the effect floor (or its bands overlap), the ratio renders "n/a (no separation
+// detected)", never a number, however tempting the arithmetic. Dividing noise by a cost
 // produces a precise-looking figure with no evidence under it.
 //
 // PRICING IS FROZEN AT RUN TIME. Registry prices change (vendors cut prices; we
@@ -47,14 +47,14 @@ const COST_BASIS = {
 const LATENCY_DISCLOSURE =
   'observed on subscription CLI surface, indicative';
 
-const NOISE_CELL = 'n/a (within noise)';
+const NOISE_CELL = 'n/a (no separation detected)';
 
 // A floor-clearing NEGATIVE lift: the skill measurably hurt, so there is no
 // benefit to put a price on.
 const REGRESSED_CELL = 'n/a (skill regressed)';
 
 // A cell with at least one separated, floor-clearing DRIVER whose AGGREGATE lift
-// does not clear the floor. It is not "within noise" — the QA re-derivation found
+// does not clear the floor. It is not the no-separation cell: the QA re-derivation found
 // six such cells rendering the noise string while the Verdict basis on the same
 // page named their drivers (QA V-4, 2026-08-19). The aggregate stays the
 // denominator (spec 002 AC-4, DECISIONS #9), so no price is quoted, but the cell
@@ -461,7 +461,7 @@ function costPerLiftPoint({ lift, separated, incrementalCostPer1kCalls }) {
     // Two different absences, two different strings. `separated` means at least
     // one case cleared the floor with non-overlapping bands; when that holds and
     // only the aggregate falls short, the evidence exists and is listed under
-    // Verdict basis — saying "within noise" there contradicts the same page.
+    // Verdict basis, and the no-separation cell there would contradict the same page.
     return separated ? DRIVER_ONLY_CELL : NOISE_CELL;
   }
   const cost = Number(incrementalCostPer1kCalls);

package/lib/verdict.js

@@ -15,7 +15,7 @@ const { EFFECT_FLOOR } = require('../config');
 // judge's quantization grid is reported as "no effect", never as "passing".
 //
 //   delta >=  EFFECT_FLOOR → PASSED    (the skill measurably helps)
-//   |delta| <  EFFECT_FLOOR → NO_EFFECT (within the judge's noise floor)
+//   |delta| <  EFFECT_FLOOR → NO_EFFECT (below the effect floor, one judge quantum)
 //   delta <= -EFFECT_FLOOR → REGRESSED (the skill measurably hurts on this model)
 
 const VERDICTS = {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "driftproof",
-  "version": "0.10.0",
+  "version": "0.10.2",
   "description": "A dated proof that this skill, this hash, this model, still helps: run a skill's eval suite with and without the skill across model versions, emit hash-verified dated receipts, and diff receipts into drift reports.",
   "license": "Apache-2.0",
   "keywords": [

package/spec/RECEIPT.md

@@ -3,7 +3,8 @@
 
 A **receipt** is a **hash-verified**, dated record of running one agent skill's eval suite
 **with** and **without** the skill on one model version, with the judge **sampled**
-so every score carries a confidence band. Receipts are the unit of evidence
+so every score carries a band: a descriptive spread, the mean plus or minus one
+sample standard deviation, with no coverage probability. Receipts are the unit of evidence
 Driftproof produces; diffing two receipts across model releases yields a **drift
 report**.
 
@@ -429,10 +430,13 @@ A verdict requires **two** conditions — band separation **and** a minimum effe
 1. **Band separation** (geometry):
    - **regression** — `meanB + stddevB < meanA − stddevA` (B's band is entirely below A's)
    - **improvement** — `meanB − stddevB > meanA + stddevA`
-   - **within noise** — otherwise (the bands touch or overlap)
+   - otherwise no separation is detected (the bands touch or overlap), and a report
+     labels the case *no separation detected*; the verdict value the differ records
+     is unchanged pending the verdict-rule spec
 2. **Minimum effect floor** — `|meanB − meanA| ≥ EFFECT_FLOOR` (default **0.05**, see
    `config.js`). A regression/improvement from step 1 whose mean moved by *less*
-   than the floor is downgraded to **`within noise (below effect floor)`**.
+   than the floor keeps a verdict value of its own, labelled *below effect floor*: a
+   separation was detected, and it is too small to call.
 
 **Why the floor.** The LLM judge quantizes scores to a coarse ~0.05–0.1 grid, so a
 confident grade frequently has `stddev 0` — a zero-width "point band." Two point
@@ -442,8 +446,10 @@ but-practically-trivial move from being reported as drift. `0.05` ≈ one judge
 quantization step; the comparison is inclusive (a move of exactly 0.05 counts).
 
 A regression is claimed **only** when both conditions hold. The **headline verdict
-summarizes the per-case verdicts** (e.g. "DRIFT — 2 cases regressed", or "WITHIN
-NOISE" when none moved beyond its band + floor); it does not run a separate, tighter
+summarizes the per-case verdicts** (e.g. "DRIFT: separation detected under the rule
+on 2 cases, downward", or "NO SEPARATION DETECTED" when no case separated under the
+rule at the sample size used, which is not evidence that nothing changed); it does
+not run a separate, tighter
 test on the aggregate mean. This is the anti-false-positive core: band separation
 **plus a real-sized delta**, not either alone, is what triggers a verdict.