driftproof 0.5.0 → 0.6.0

package/README.md

@@ -9,15 +9,17 @@
 Driftproof is an open **receipt spec** plus a **runner** that measures whether an
 agent skill actually helps — by running the skill's eval suite **with** and
 **without** the skill on a named model version, judging each case several times to
-get a confidence band, and emitting a signed, dated **receipt**. Diff two receipts
+get a confidence band, and emitting a **hash-verified**, dated **receipt**. Diff two receipts
 across model releases and you get a **drift report**.
 
 Driftproof **consumes** the [`agentskills.io/evals`](https://agentskills.io) eval
 format; it does not invent its own.
 
-📊 **Four published reports** (each re-derived from committed receipts, nothing
-hand-entered), spanning three report types that share one band-based, floor-gated
-verdict rule and differ only in what moves underneath the skill:
+📊 **Six published reports** (each re-derived from committed receipts, nothing
+hand-entered), spanning five published report types, the fifth being revision
+drift. All six share one band-based, floor-gated verdict rule and differ in what
+moves underneath the skill — or, in the value report, in which axes are
+measured:
 
 - **[Report #001](https://driftproofhq.com/reports/001/)** — *release drift*: ten
   public agent skills across a current-vs-previous Sonnet release; 9 of 10 moved
@@ -31,10 +33,29 @@ verdict rule and differ only in what moves underneath the skill:
   `claude-opus-5` (flagship) vs `claude-fable-5` (frontier tier);
   3 durable, 5 tier-dependent, 0 regressions, 2 no effect — encoded expertise
   survives the frontier tier.
+- **[Report #005](https://driftproofhq.com/reports/005/)** — *value*: what a skill
+  *costs* to run, on three axes (accuracy, cost, latency); the same ten suites on
+  three substrates (`claude-sonnet-5`, `claude-fable-5`, `gpt-5.6-sol`) —
+  14 of 30 cells cleared the floor on aggregate: 10 carry a price and 4 report a
+  saving instead, having improved quality while reducing cost.
+  *Three of those cells carry an amendment (v1.1, applied when Report #006
+  published): their lifts rest on single-draw baselines since shown
+  unstable. Cause-agnostic, no corrected figures offered, and the cost-driver and
+  substrate-disagreement findings below are unaffected.*
+- **[Report #006](https://driftproofhq.com/reports/006/)** — *revision drift*: the pinned skill revision against the one
+  upstream ships today, on a held substrate. **The reuse premise was tested and
+  refused: 3 of 3 cells returned no verdict**, each blocked by its own baseline
+  control. A 120-call probe found generation-level sampling noise 3.2× and 7.5×
+  larger than the judge-level noise this instrument actually samples — enough to
+  account for every gap the controls saw without any other cause being
+  established — and the receipt spec gains generation sampling as a result. **No
+  cause is asserted**; the control proves non-reproduction and cannot say why.
+  *The tally a refusal carries: 3 cells, 0 measured, 3 refused.*
 
 ✍️ The launch essay, **[Three model releases later: what actually happens to agent
-skills](https://driftproofhq.com/writing/three-releases/)**, reads the first three
-reports together.
+skills](https://driftproofhq.com/writing/three-releases/)**, reads all six reports
+together: what moves underneath a skill, and what the skill costs to run. Revised
+2026-08-29; every figure in it is gate-checked against the report page it cites.
 
 ## Why
 
@@ -55,6 +76,15 @@ by more than the drift you're trying to detect. Driftproof's answer is to **samp
 the judge and report confidence bands**, and to **only claim a regression when the
 bands don't overlap**. A tool that cries wolf is worse than no tool.
 
+**A verdict without a price is half an answer.** The same receipts price the
+marginal cost of a skill firing, and Report #005 found the dominant cost driver is
+not the skill's own text but the input it causes the model to pull in: across those
+30 cells the input delta tracks cost at `r = +0.92` while the skill's own length
+tracks it at only `r = +0.33`, and one 738-token skill drew 34× its own size in
+extra input. Identical token deltas also price very differently across substrates —
+the same skill at near-identical deltas costs 3.3× more on `claude-fable-5` than on
+`claude-sonnet-5`, which is exactly their input-rate ratio in the frozen snapshot.
+
 ## Quickstart — receipt for your own skill in ~10 minutes
 
 You need Node ≥ 22 and an `ANTHROPIC_API_KEY`.
@@ -86,6 +116,15 @@ bands don't overlap. The **effect floor** (0.05, one judge quantization step) is
 minimum real move required before a change counts as more than noise — band
 separation *plus* a floor-sized delta, never either alone.
 
+**What the band does not cover.** A verdict rests on **one generation draw per
+arm**: the band is the spread of the *judge* re-scoring that single response, not
+the spread of the model writing a different one. Report #006 measured the second
+directly and found it larger — draw-to-draw spread up to **sd 0.186** on the 0–1
+scale, against judge-level noise several times smaller. So treat a surprising
+single-run verdict as **provisional and worth re-running** before you act on it.
+Generation sampling lands in the next receipt spec; until it does, this is a
+limit of the instrument, stated rather than implied.
+
 ### Install
 
 ```bash
@@ -171,7 +210,7 @@ A receipt is the unit of evidence — one JSON document conforming to
     "model_release_date": "2025-10-01",
     "provider": "anthropic",
     "surface": "claude-cli",
-    "runner_version": "0.5.0",
+    "runner_version": "0.6.0",
     "date_utc": "2026-07-27T…Z",
     "registry": "registered",
     "transcripts": "hashes-only",
@@ -194,6 +233,12 @@ A receipt is the unit of evidence — one JSON document conforming to
   },
   "comparison": { "with_skill_score": 0.81, "baseline_score": 0.42,
                   "delta": 0.39, "delta_uncertainty": 0.036 },
+  // v0.4 economics, all derived and never composited into one score: "run.pricing_snapshot"
+  // freezes the rates; each case carries "usage" and a separate "judge_usage"; "economics"
+  // holds basis, surface, with_skill/baseline (call_count, mean_input_tokens,
+  // mean_output_tokens, mean_cost_usd_per_call, median_wall_ms + p25/p75/IQR),
+  // skill_incremental_cost_usd_per_call, skill_incremental_cost_usd_per_1k_calls,
+  // output_tokens_delta, median_wall_ms_delta, judge_excluded (const true), judge_overhead.
   "verification_level": "TESTED",
   "receipt_hash": "…sha256 of the canonical receipt with this field removed…"
 }
@@ -209,6 +254,12 @@ Key ideas:
 - **`delta_uncertainty`** is the combined band on the with-skill-vs-baseline lift.
 - **`verification_level`** uses the community lattice: `UNVERIFIED` / `DECLARED` /
   `TESTED` (Driftproof emits `TESTED`). `FORMAL` is reserved.
+- **`economics`** is *derived*, never a second measurement: the token delta is the
+  durable fact, and the dollars are exactly those tokens at the rates frozen into
+  `run.pricing_snapshot`, so a receipt keeps its meaning after a vendor reprices.
+  Judge cost is recorded apart as `judge_usage` and excluded from every skill-value
+  figure (`judge_excluded` is `const true`) — measuring the skill is our cost, not
+  the skill's.
 - **`receipt_hash`** is a self-hash for tamper-evidence (integrity, not yet a key
   signature — see the spec's open questions).
 
@@ -233,7 +284,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: driftproofhq/driftproof@v0.5.0
+      - uses: driftproofhq/driftproof@v0.6.0
         with:
           skill-dir: skills/my-skill
           models: claude-haiku-4-5
@@ -272,11 +323,14 @@ site, so it reflects a real dated run, not a hand-set color.
 
 ## Reports
 
-Four reports are published, spanning three report types (see the roll at the top
-of this README, and [REPORT-STYLE.md](REPORT-STYLE.md) for the shared chrome
-every report inherits). Each report and every verdict in it are **re-derived from
-the receipts** committed under [`receipts/`](receipts/)
-(`receipts/report-001/` … `receipts/report-004/`) — nothing is hand-entered.
+Six reports are published, spanning five report types. A report page lives at a
+draft path — `docs/reports/NNN-draft/` — until the publish sequence renames it, and
+`scripts/build-public.sh` excludes every `*-draft/` path from the published tree
+(see the roll at the top of this README, and
+[REPORT-STYLE.md](REPORT-STYLE.md) for the shared chrome every report inherits).
+Each report and every verdict in it are **re-derived from the receipts** committed
+under [`receipts/`](receipts/)
+(`receipts/report-001/` … `receipts/report-006/`) — nothing is hand-entered.
 
 Driftproof does **not** commit third-party skill content. Each `SKILL.md` is
 fetched at run time from a pinned commit and verified by sha256 against
@@ -290,9 +344,10 @@ node scripts/run-report-001.js --concurrency 5   # run both models × with/basel
 node scripts/build-report-001.js                 # re-derive the report from the receipts
 ```
 
-Reports #002–#004 have their own runners
-(`scripts/prepare-report-00N.js`) following the same
-fetch → run → re-derive shape.
+Reports #002–#005 have their own runners
+(`scripts/prepare-report-00N.js` — Report #005's is
+[`scripts/prepare-report-005.js`](scripts/prepare-report-005.js)) following the
+same fetch → run → re-derive shape.
 
 **Model-release triggers are live**: `scripts/release-watch.js` (keyless — it
 reads the public models registry) notices a new model release, re-runs the

package/bin/driftproof

@@ -8,7 +8,7 @@ const { PROJECT_NAME, RUNNER_VERSION, DEFAULT_JUDGE_SAMPLES, DEV_MAX_USD } = req
 const { loadSkill } = require('../lib/skill');
 const { runSkillOnModel, summarizeReceipt, projectCalls } = require('../lib/run');
 const { validateReceipt, verifyReceiptHash } = require('../lib/receipt');
-const { buildDriftReport } = require('../lib/diff');
+const { buildDriftReport, revisionPairProblem } = require('../lib/diff');
 const { surfaceForModel, isSubscriptionSurface, resolveModel } = require('../lib/provider');
 const { estimateRunCostUSD, BudgetTracker } = require('../lib/cost');
 const { registryStatus } = require('../lib/models');
@@ -76,7 +76,7 @@ USAGE
   ${PROJECT_NAME} run <skill-dir> [--models a,b] [--samples N] [--max-cases N] [--max-calls N]
                         [--judge-model M] [--concurrency N] [--max-usd N]
                         [--keep-transcripts] [--out DIR]
-  ${PROJECT_NAME} diff <receiptA.json> <receiptB.json> [--out FILE]
+  ${PROJECT_NAME} diff <receiptA.json> <receiptB.json> [--out FILE] [--mode release|revision]
   ${PROJECT_NAME} validate <receipt.json>
   ${PROJECT_NAME} badge <receipt.json> [--out FILE] [--github-output]
   ${PROJECT_NAME} import <results.json> --from agent-skills-eval|skillgrade [--out DIR]
@@ -250,9 +250,32 @@ function cmdDiff(positional, flags) {
     if (!verifyReceiptHash(r)) console.error(`  ⚠ ${path.basename(p)}: receipt_hash does not verify (tampered or hand-edited)`);
   }
 
-  const labelA = dateStamp(a.run.date_utc);
-  const labelB = dateStamp(b.run.date_utc);
-  const { markdown } = buildDriftReport(a, b, { labelA, labelB });
+  // --mode revision inverts the axis: the skill text is the variable under test
+  // and the substrate is the control. The fields release drift merely warns about
+  // are preconditions here, so a pair that is not a revision pair is REFUSED
+  // (exit 6) rather than rendered with a caveat nobody reads. A differing model
+  // would be release drift wearing a revision label — the one confound this mode
+  // exists to exclude — and an EQUAL content_hash has no revision to measure.
+  const mode = flags.mode || 'release';
+  if (mode !== 'release' && mode !== 'revision') {
+    console.error(`unknown --mode "${mode}" — supported: release, revision`);
+    process.exit(2);
+  }
+  if (mode === 'revision') {
+    const problem = revisionPairProblem(a, b);
+    if (problem) {
+      const why = problem === 'skill.content_hash'
+        ? 'the two receipts carry the SAME skill.content_hash — there is no revision between them to measure'
+        : `${problem} differs between the two receipts — revision drift requires the substrate to be held fixed, and a differing ${problem} would confound the revision with release drift`;
+      console.error(`  ✗ REFUSED (--mode revision): ${why}.`);
+      console.error(`    Compare these two with the default release mode, or supply a pair that differs only in skill.content_hash.`);
+      process.exit(6);
+    }
+  }
+
+  const labelA = mode === 'revision' ? `pinned (${dateStamp(a.run.date_utc)})` : dateStamp(a.run.date_utc);
+  const labelB = mode === 'revision' ? `current (${dateStamp(b.run.date_utc)})` : dateStamp(b.run.date_utc);
+  const { markdown } = buildDriftReport(a, b, { labelA, labelB, mode });
 
   if (flags.out) {
     fs.writeFileSync(path.resolve(flags.out), markdown);

package/config.js

@@ -9,7 +9,7 @@ const PROJECT_NAME = 'driftproof';
 // Bumped whenever the runner's behaviour or receipt-generation semantics change
 // in a way that could affect results. Recorded into every receipt as
 // run.runner_version so a receipt is reproducible against a known engine.
-const RUNNER_VERSION = '0.5.0';
+const RUNNER_VERSION = '0.6.0';
 
 // The eval format we CONSUME (we deliberately do not invent our own).
 const SUITE_FORMAT = 'agentskills.io/evals';

package/lib/diff.js

@@ -3,6 +3,7 @@
 
 const { bandVerdict, round } = require('./stats');
 const { EFFECT_FLOOR } = require('../config');
+const { revisionHeadline } = require('./revision');
 
 // Practical-significance gate applied ON TOP of band separation. bandVerdict()
 // stays a pure geometry test (kept that way so its unit checks are unambiguous);
@@ -68,7 +69,22 @@ function headlineVerdict(perCase) {
   return 'WITHIN NOISE — no case moved beyond its confidence band; the skill holds up.';
 }
 
-function buildDriftReport(a, b, { labelA = 'A', labelB = 'B' } = {}) {
+// A revision pair is a pair in which the SKILL TEXT is the only thing that
+// moved. `diff` was built for release drift, where the model varies and the text
+// is fixed; this inverts it, so the fields that release drift merely warns about
+// become the preconditions of the comparison. Returns the offending field name,
+// or null when the pair is a valid revision pair.
+function revisionPairProblem(a, b) {
+  if ((a.run.model_id || '') !== (b.run.model_id || '')) return 'run.model_id';
+  if ((a.run.provider || 'anthropic') !== (b.run.provider || 'anthropic')) return 'run.provider';
+  if ((a.run.surface || '') !== (b.run.surface || '')) return 'run.surface';
+  if ((a.suite.suite_hash || '') !== (b.suite.suite_hash || '')) return 'suite.suite_hash';
+  if ((a.skill.content_hash || '') === (b.skill.content_hash || '')) return 'skill.content_hash';
+  return null;
+}
+
+function buildDriftReport(a, b, { labelA = 'A', labelB = 'B', mode = 'release' } = {}) {
+  const revision = mode === 'revision';
   const aB = withSkillBands(a);
   const bB = withSkillBands(b);
   const ids = [...new Set([...Object.keys(aB), ...Object.keys(bB)])];
@@ -99,6 +115,32 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B' } = {}) {
   const regressions = perCase.filter((r) => r.verdict === 'regression');
 
   const L = [];
+  if (revision) {
+    // The axis leads. In a release-drift report the model is what moved and it
+    // belongs at the top; here the model is the control and the skill's own text
+    // is the finding, so content_hash is the first row a reader meets.
+    L.push(`# Revision drift report`);
+    L.push('');
+    L.push(`**Skill:** ${a.skill.name} \`${a.skill.version}\``);
+    L.push('');
+    L.push(`**Report type:** revision drift — the skill's own text is the variable under test; the substrate is held fixed.`);
+    L.push('');
+    L.push(`Left column is the **pinned** revision; right column is the **current** upstream revision.`);
+    L.push('');
+    L.push(`| | ${labelA} | ${labelB} |`);
+    L.push(`|---|---|---|`);
+    L.push(`| skill content_hash | \`${short(a.skill.content_hash)}\` | \`${short(b.skill.content_hash)}\` |`);
+    L.push(`| model (held) | \`${a.run.model_id}\` | \`${b.run.model_id}\` |`);
+    L.push(`| provider (held) | ${a.run.provider || 'anthropic'} | ${b.run.provider || 'anthropic'} |`);
+    L.push(`| surface (held) | ${a.run.surface} | ${b.run.surface} |`);
+    L.push(`| suite_hash (held) | \`${short(a.suite.suite_hash)}\` | \`${short(b.suite.suite_hash)}\` |`);
+    L.push(`| run date (UTC) | ${a.run.date_utc} | ${b.run.date_utc} |`);
+    L.push(`| judge samples/case | ${(a.run.judge || {}).samples || 1} | ${(b.run.judge || {}).samples || 1} |`);
+    L.push(`| with_skill (mean ± band) | ${bandStr(aAgg)} | ${bandStr(bAgg)} |`);
+    L.push(`| baseline score | ${pct(a.comparison.baseline_score)} | ${pct(b.comparison.baseline_score)} |`);
+    L.push(`| skill lift (Δ) | ${fmt(a.comparison.delta)} | ${fmt(b.comparison.delta)} |`);
+    L.push('');
+  } else {
   L.push(`# Drift report`);
   L.push('');
   L.push(`**Skill:** ${a.skill.name} \`${a.skill.version}\``);
@@ -115,10 +157,19 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B' } = {}) {
   L.push(`| baseline score | ${pct(a.comparison.baseline_score)} | ${pct(b.comparison.baseline_score)} |`);
   L.push(`| skill lift (Δ) | ${fmt(a.comparison.delta)} | ${fmt(b.comparison.delta)} |`);
   L.push('');
+  }
 
   const warnings = [];
-  if (a.skill.content_hash !== b.skill.content_hash) warnings.push('skill content_hash differs — the skill itself changed between receipts, so drift mixes skill edits with model drift.');
-  if (a.suite.suite_hash !== b.suite.suite_hash) warnings.push('suite_hash differs — the eval suite changed; per-case comparison may be misleading.');
+  // In revision mode the changed skill text is the INDEPENDENT VARIABLE, not
+  // contamination, and the held-constant substrate is what makes the comparison
+  // valid. The release-drift caveat below says the opposite of both, so it is
+  // replaced rather than suppressed: a reader is told what is held, and why a
+  // separated band is attributable to the revision.
+  if (revision) {
+    warnings.push(`model \`${a.run.model_id}\`, provider ${a.run.provider || 'anthropic'}, surface ${a.run.surface} and suite_hash \`${short(a.suite.suite_hash)}\` are held constant across both receipts — the skill text is the only variable under test, so a band-separated move above the ${EFFECT_FLOOR} floor is attributable to the revision.`);
+  }
+  if (!revision && a.skill.content_hash !== b.skill.content_hash) warnings.push('skill content_hash differs — the skill itself changed between receipts, so drift mixes skill edits with model drift.');
+  if (!revision && a.suite.suite_hash !== b.suite.suite_hash) warnings.push('suite_hash differs — the eval suite changed; per-case comparison may be misleading.');
   if (a.skill.name !== b.skill.name) warnings.push(`different skills (${a.skill.name} vs ${b.skill.name}) — comparison is not meaningful.`);
   if ((a.run.judge || {}).samples <= 1 || (b.run.judge || {}).samples <= 1) warnings.push('one or both receipts are single-sample (no bands) — non-overlap can only be trusted when both sides are sampled.');
   if (!measured) warnings.push(`verdicts NOT computed — ${belowTested.map(([l, r]) => `${l} is ${levelOf(r)}${r.run && r.run.source ? ` (${r.run.source})` : ''}`).join('; ')}. Drift verdicts require TESTED receipts on both sides; declared numbers are shown as context only (see /interop.html).`);
@@ -126,8 +177,8 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B' } = {}) {
   // providers is a skill-DURABILITY comparison across substrates, not model drift
   // over time; across surfaces, sampling control differs. Both are flagged so a
   // reader never mistakes one for the other (see docs/neutrality.html).
-  if ((a.run.provider || 'anthropic') !== (b.run.provider || 'anthropic')) warnings.push(`different providers (${a.run.provider || 'anthropic'} vs ${b.run.provider || 'anthropic'}) — this is a cross-substrate durability comparison, not model drift over time; read the delta, not absolute scores (see the neutrality policy).`);
-  if (a.run.surface !== b.run.surface) warnings.push(`different surfaces (${a.run.surface} vs ${b.run.surface}) — sampling control differs between surfaces; compare with care.`);
+  if (!revision && (a.run.provider || 'anthropic') !== (b.run.provider || 'anthropic')) warnings.push(`different providers (${a.run.provider || 'anthropic'} vs ${b.run.provider || 'anthropic'}) — this is a cross-substrate durability comparison, not model drift over time; read the delta, not absolute scores (see the neutrality policy).`);
+  if (!revision && a.run.surface !== b.run.surface) warnings.push(`different surfaces (${a.run.surface} vs ${b.run.surface}) — sampling control differs between surfaces; compare with care.`);
   if (warnings.length) {
     L.push('> **⚠ Caveats**');
     for (const w of warnings) L.push(`> - ${w}`);
@@ -142,7 +193,7 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B' } = {}) {
     L.push(`**NOT MEASURED — ${belowTested.length} receipt(s) below TESTED. Drift verdicts require TESTED receipts on both sides; the declared numbers above are context, not band-verified evidence.**`);
     L.push('');
   } else {
-    L.push(`**${headlineVerdict(perCase)}**`);
+    L.push(`**${revision ? revisionHeadline(perCase) : headlineVerdict(perCase)}**`);
     L.push('');
     L.push(`with_skill mean moved ${fmt(headlineDelta)} (${bandStr(aAgg)} → ${bandStr(bAgg)}; band = suite dispersion). Per-case band-overlap verdicts: ${regressions.length} regression(s), ${perCase.filter((r) => r.verdict === 'improvement').length} improvement(s), ${nWithin} within noise${nFloor ? ` (${nFloor} of them band-separated but below the ${EFFECT_FLOOR} effect floor)` : ''}.`);
     L.push('');
@@ -170,4 +221,4 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B' } = {}) {
   return { markdown: L.join('\n'), perCase, headlineDelta, regressions };
 }
 
-module.exports = { buildDriftReport, withSkillBands };
+module.exports = { buildDriftReport, withSkillBands, revisionPairProblem };

package/lib/hygiene.js

@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: Apache-2.0
+//
+// The single definition of what counts as a leak.
+//
+// Two callers scan for the same classes of secret: `tests/gate.js`, which walks
+// the working tree at gate time, and `scripts/merge-check.js`, which walks the
+// candidate tree read out of the commit before permitting a merge. They must
+// agree. Two copies of a deny-list drift, and the copy that quietly stops
+// matching still reads as protection — the same argument the acknowledgment
+// deny-list bite test already makes, applied to the scanner itself.
+//
+// Backlog B-8 records four approval records that reached `dev` carrying a host
+// path. The recorded cause — "the scan walks tracked files" — is false, and this
+// module's existence does not depend on it: `listFiles()` in the repo gate has
+// always enumerated untracked files too, and a planted path in an untracked
+// evidence record does take the repo gate red. The real window is VANTAGE. An
+// approval session measures in a disposable copy made before it writes its
+// record into the shared checkout, so the record is simply not in the tree that
+// was scanned. Scanning the working tree harder cannot close that. Scanning the
+// commit that is about to merge does, and that is what merge-check now uses this
+// for.
+//
+// Patterns are written so a regex literal cannot match its own text: a
+// metacharacter or a character class follows each fixed prefix.
+
+const dec = (b64) => JSON.parse(Buffer.from(b64, 'base64').toString('utf8'));
+
+// Build-box host + private IP, base64 so the plaintext never lands in a file.
+const HOSTIP = dec('WyJpcC0xNzItMzEtNDAtMTU5LmFwLXNvdXRoZWFzdC0xLmNvbXB1dGUuaW50ZXJuYWwiLCIxNzIuMzEuNDAuMTU5Il0=');
+
+const EMAIL_ALLOW = new Set(['example.com', 'example.org', 'driftproofhq.com']);
+
+const PATTERNS = [
+  { name: 'home-path', re: /\/home\/[a-z0-9_-]+\/|\/Users\/[A-Za-z0-9_-]+\// },
+  { name: 'ec2-internal-host', re: /ip-\d+-\d+-\d+-\d+\.[a-z0-9.-]*compute\.(internal|amazonaws\.com)/i },
+  { name: 'private-ip', re: /\b(10\.\d{1,3}\.\d{1,3}\.\d{1,3}|172\.(1[6-9]|2\d|3[01])\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3})\b/ },
+  { name: 'anthropic-key', re: /sk-ant-[A-Za-z0-9_-]{8,}/ },
+  { name: 'github-token', re: /gh[pousr]_[A-Za-z0-9]{20,}/ },
+  { name: 'aws-key', re: /\bAKIA[0-9A-Z]{16}\b/ },
+  { name: 'private-key-block', re: /-----BEGIN [A-Z ]*PRIVATE KEY-----/ },
+  { name: 'env-secret-assignment', re: /\b(ANTHROPIC_API_KEY|AWS_SECRET_ACCESS_KEY|OPENAI_API_KEY)\s*=\s*\S+/ },
+  // Codex subscription auth material (~/.codex/auth.json contents) must NEVER
+  // land in a committed file: the id_token/access_token are JWTs, and an OpenAI
+  // secret key is sk-proj-/sk-svcacct-/sk-admin-. We ban the CONTENTS (tokens),
+  // not the documented path string (`~/.codex/auth.json` is referenced in help
+  // text and docs by design).
+  { name: 'jwt-token', re: /\beyJ[A-Za-z0-9_=-]{10,}\.eyJ[A-Za-z0-9_=-]{10,}\.[A-Za-z0-9_=-]{6,}/ },
+  { name: 'openai-secret-key', re: /\bsk-(proj|svcacct|admin)-[A-Za-z0-9_-]{20,}/ },
+];
+
+// A path is a hit on its own name, with no content read: a committed .env file
+// is a leak whatever it happens to contain.
+function scanPath(rel) {
+  return /(^|\/)\.env(\.|$)/.test(rel) ? [{ file: rel, kind: 'env-file' }] : [];
+}
+
+function scanContent(rel, content) {
+  const hits = [];
+  for (const p of PATTERNS) {
+    const m = content.match(p.re);
+    if (m) hits.push({ file: rel, kind: p.name, sample: m[0].slice(0, 24) });
+  }
+  for (const hv of HOSTIP) if (content.includes(hv)) hits.push({ file: rel, kind: 'build-host-or-ip' });
+  const emails = content.match(/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b/g) || [];
+  for (const e of emails) {
+    const dom = e.split('@')[1].toLowerCase();
+    if (!EMAIL_ALLOW.has(dom) && !dom.endsWith('.example')) hits.push({ file: rel, kind: 'email', sample: e });
+  }
+  return hits;
+}
+
+// scanFiles(files, read, readLink) — `read(rel)` returns the file's text, or
+// throws/returns null for anything unreadable (binary, deleted, a symlink to
+// nowhere), which is skipped exactly as the gate has always skipped it.
+//
+// `readLink(rel)` is optional and returns a symbolic link's TARGET PATH as a
+// string, or null for an entry that is not a link. Where it is supplied, that
+// target is scanned as the entry's content through `scanContent`, so every
+// pattern above applies to it with no second deny-list to drift (spec 011,
+// AC-2). A link's target is a path, and a path is exactly the class of leak the
+// `home-path` pattern exists to catch: P-1 shipped `node_modules ->
+// /home/<user>/... ` into a public tree past a scan that read the link as an
+// unreadable file and skipped it.
+//
+// The argument is optional so a caller supplying nothing behaves exactly as
+// before. That compatibility is deliberate, and it is also the standing risk: a
+// caller added later is blind by default. The repo gate asserts that every call
+// site supplies a reader, with `scripts/merge-check.js` the one exception — its
+// reader is `git show <ref>:<path>`, and git stores a link's blob as its target
+// string, so the target already arrives as content there.
+function scanFiles(files, read, readLink) {
+  const hits = [];
+  for (const rel of files) {
+    hits.push(...scanPath(rel));
+    // The link branch is a single condition on purpose: it is the mutation seam
+    // the spec-011 gate neutralises to prove the scanner goes blind without it.
+    if (typeof readLink === 'function') {
+      let target;
+      try { target = readLink(rel); } catch (_e) { target = null; }
+      if (typeof target === 'string') {
+        hits.push(...scanContent(rel, target));
+        continue;
+      }
+    }
+    let c;
+    try { c = read(rel); } catch (_e) { continue; }
+    if (typeof c !== 'string') continue;
+    hits.push(...scanContent(rel, c));
+  }
+  return hits;
+}
+
+module.exports = { PATTERNS, EMAIL_ALLOW, HOSTIP, scanPath, scanContent, scanFiles };

package/lib/revision.js

@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: Apache-2.0
+'use strict';
+
+const { bandVerdict, round } = require('./stats');
+const { EFFECT_FLOOR } = require('../config');
+
+// Revision drift — the fifth report type (spec 009, Report #006).
+//
+// Every other report type holds the skill text fixed and moves something
+// underneath it: the model release (#001, #003), the vendor surface (#002), the
+// capability tier (#004), the axes and the price (#005). This one inverts the
+// design. The substrate is held still — same model, same provider, same surface,
+// same suite, same fixed judge, same sampling — and the SKILL'S OWN TEXT moves,
+// from the revision a report pinned to the revision upstream ships today.
+//
+// This module holds the report type's LANGUAGE as code rather than as
+// hand-written page copy. That is deliberate: the fairness rule below is the one
+// a measurement project is most tempted to apply in one direction only, and a
+// rule that lives in prose cannot be gated before the run that would tempt it.
+
+// ── the cell headline ────────────────────────────────────────────────────────
+// A summary of the per-case band-overlap verdicts, worded about the REVISION.
+// The release-drift headline says "the skill is measurably weaker", which is a
+// sentence about a skill under a moving model. Here the model is the control.
+function revisionHeadline(perCase) {
+  const reg = perCase.filter((r) => r.verdict === 'regression').length;
+  const imp = perCase.filter((r) => r.verdict === 'improvement').length;
+  const s = (n) => (n === 1 ? '' : 's');
+  if (reg && imp) {
+    return `MIXED — the revision improved ${imp} case${s(imp)} and regressed ${reg} on non-overlapping bands.`;
+  }
+  if (reg) {
+    return `REVISION REGRESSED — ${reg} case${s(reg)} scored lower under the current upstream text (bands do not overlap).`;
+  }
+  if (imp) {
+    return `REVISION IMPROVED — ${imp} case${s(imp)} scored higher under the current upstream text (bands do not overlap); none regressed.`;
+  }
+  return 'WITHIN NOISE — the revision moved no case beyond its confidence band; the pinned text and the current text measure the same.';
+}
+
+// Classification word for a cell, from its headline. Kept separate so a caller
+// can branch on the class without parsing prose.
+function revisionClass(perCase) {
+  const h = revisionHeadline(perCase);
+  return h.split(' —')[0];
+}
+
+// ── the fairness sentence ────────────────────────────────────────────────────
+// Spec 009 § Fairness, clauses 1 and 4. Where a revision IMPROVED a skill, the
+// published #005 figure understates the pack a reader can install today; where it
+// REGRESSED one, #005 overstates it. Both sentences are generated by the same
+// function, from the same template, so the disclosure cannot quietly become a
+// one-directional courtesy — the symmetry is a property of the code, and the
+// gate asserts it.
+//
+// A cell within noise gets NO sentence. #005's figure stands unamended, because
+// nothing was measured that would amend it, and manufacturing a hedge for a null
+// result is how a report launders noise into a finding.
+function fairnessSentence({ slug, classification, report005Delta, measuredDelta }) {
+  const cls = String(classification || '');
+  if (cls !== 'REVISION IMPROVED' && cls !== 'REVISION REGRESSED') return null;
+  const improved = cls === 'REVISION IMPROVED';
+  const direction = improved ? 'understates' : 'overstates';
+  const d = (n) => (n == null ? 'n/a' : (n >= 0 ? '+' : '') + Number(n).toFixed(3));
+  return `Report #005 measured ${slug} at ${d(report005Delta)} on the text it had pinned. `
+    + `Report #006 measures the current upstream revision at ${d(measuredDelta)} on the same substrate and the same suite. `
+    + `#005's published figure therefore ${direction} the pack upstream ships today for this skill, `
+    + `and is amended by this report rather than corrected in place.`;
+}
+
+// ── per-cell scoping disclosures ─────────────────────────────────────────────
+// Spec 009 AC-10. One cell in Report #006 measures something other than what its
+// upstream author changed it to do, and the reader looking at that row is the
+// reader who needs to be told.
+const SCOPING_NOTES = {
+  'git-workflow-and-versioning':
+    'This revision changes the frontmatter `description:` line. In a skill runtime a description is a '
+    + 'routing trigger: it decides whether the skill loads, and never reaches the model as guidance. '
+    + 'Driftproof makes no routing decision — it always injects the skill, and passes the whole file, '
+    + 'frontmatter included, as the system prompt. This cell therefore measures the revision as added '
+    + 'context and cannot measure it as a trigger.',
+};
+function scopingNote(slug) {
+  return SCOPING_NOTES[slug] || null;
+}
+
+// ── the baseline-reproduction control ────────────────────────────────────────
+// Spec 009 AC-6, and the thing that makes the free pinned arm honest.
+//
+// Report #006 reuses #005's receipts as the pinned-text arm. That is valid only
+// if the substrate has not moved, and `run.model_release_date` is null on every
+// #005 receipt, so id equality is the only version evidence a receipt carries. A
+// provider that re-points a concrete id at a new snapshot is invisible to it.
+//
+// It does not have to be. Every fresh run emits a BASELINE arm: the same cases,
+// the same substrate, and no skill text at all. The revision cannot touch it by
+// construction, so comparing the fresh baseline against the reused receipt's
+// baseline re-measures exactly the thing id equality could not prove — at no
+// extra cost, because that arm is already paid for.
+//
+// A cell whose baselines do not reproduce is NOT MEASURED. The reuse is a tested
+// prediction, not an assumption the report asks the reader to grant.
+function baselineBands(receipt) {
+  const out = {};
+  for (const c of receipt.results.cases) {
+    if (c.mode !== 'baseline') continue;
+    out[c.id] = { mean: c.mean != null ? c.mean : c.score, stddev: c.stddev || 0 };
+  }
+  return out;
+}
+
+function baselineControl(reused, fresh) {
+  const A = baselineBands(reused);
+  const B = baselineBands(fresh);
+  const ids = [...new Set([...Object.keys(A), ...Object.keys(B)])];
+
+  const perCase = ids.map((id) => {
+    const before = A[id] || null;
+    const after = B[id] || null;
+    if (!before || !after) return { id, before, after, delta: null, moved: false, missing: true };
+    const delta = round(after.mean - before.mean);
+    // The same rule the study uses everywhere else: band separation AND the
+    // effect floor. A baseline that wobbles inside its band has not moved.
+    const raw = bandVerdict(before.mean, before.stddev, after.mean, after.stddev);
+    const separated = raw === 'regression' || raw === 'improvement';
+    return { id, before, after, delta, moved: separated && Math.abs(delta) >= EFFECT_FLOOR, missing: false };
+  });
+
+  const movedCases = perCase.filter((r) => r.moved);
+  const missing = perCase.filter((r) => r.missing);
+  const reproduced = movedCases.length === 0 && missing.length === 0;
+  const aggDelta = round(
+    (fresh.comparison && fresh.comparison.baseline_score != null ? fresh.comparison.baseline_score : 0)
+    - (reused.comparison && reused.comparison.baseline_score != null ? reused.comparison.baseline_score : 0),
+  );
+
+  return {
+    reproduced,
+    blocked: !reproduced,
+    verdict: reproduced ? 'MEASURED' : 'NOT MEASURED',
+    moved_cases: movedCases.map((r) => r.id),
+    missing_cases: missing.map((r) => r.id),
+    aggregate_baseline_delta: aggDelta,
+    floor: EFFECT_FLOOR,
+    // THE CONTROL PROVES NON-REPRODUCTION. IT CANNOT SAY WHY. These strings used
+    // to read 'the substrate moved' and 'the substrate held still' — a cause,
+    // asserted by a comparison that measures two baseline arms and nothing else.
+    // A 120-call stability probe then found generation-level sampling noise large
+    // enough to account for every gap this control saw, with no substrate movement
+    // required, and the report page retracted the claim while three committed
+    // control records still carried it (approval finding F-009-N). Reason strings
+    // only: no score, sample, hash or verdict changed with this edit.
+    reason: reproduced
+      ? 'the fresh baseline reproduces the reused receipt\'s baseline within the band and the floor, so the pinned-arm reuse stands for this cell'
+      : `the fresh baseline does not reproduce the reused receipt's baseline (${movedCases.length} case(s) moved beyond the band and the ${EFFECT_FLOOR} floor${missing.length ? `, ${missing.length} case(s) absent on one side` : ''}) — the reused pinned arm is not comparable to the fresh arm, so revision drift cannot be separated from whatever else changed in this cell; the control establishes non-reproduction and does not identify a cause`,
+    perCase,
+  };
+}
+
+module.exports = {
+  revisionHeadline, revisionClass, fairnessSentence, scopingNote, baselineControl,
+};

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "driftproof",
-  "version": "0.5.0",
-  "description": "Continuous verification of agent skills: run a skill's eval suite with and without the skill across model versions, emit signed dated receipts, and diff receipts into drift reports.",
+  "version": "0.6.0",
+  "description": "Continuous verification of agent skills: run a skill's eval suite with and without the skill across model versions, emit hash-verified dated receipts, and diff receipts into drift reports.",
   "license": "Apache-2.0",
   "keywords": [
     "agent-skills",

package/spec/RECEIPT.md

@@ -1,7 +1,7 @@
 <!-- SPDX-License-Identifier: Apache-2.0 -->
 # Driftproof receipt — spec v0.4
 
-A **receipt** is a signed, dated record of running one agent skill's eval suite
+A **receipt** is a **hash-verified**, dated record of running one agent skill's eval suite
 **with** and **without** the skill on one model version, with the judge **sampled**
 so every score carries a confidence band. Receipts are the unit of evidence
 Driftproof produces; diffing two receipts across model releases yields a **drift
@@ -164,7 +164,7 @@ The v0.3.1 schema gained an **additive interop revision** so receipts can be
 
 ## Fields
 
-### `schema_version` (string, required) — `"0.3.1"`.
+### `schema_version` (string, required) — `"0.4"`.
 
 ### `skill` (object, required)
 | field | type | notes |

package/spec/receipt.schema.json

@@ -2,7 +2,7 @@
   "$schema": "https://json-schema.org/draft/2020-12/schema",
   "$id": "https://driftproofhq.com/spec/receipt.schema.json",
   "title": "driftproof receipt",
-  "description": "A signed, dated record of running one agent skill's eval suite with and without the skill on one model version, with sampled judge scores and confidence bands. Receipt spec v0.4 — additive over v0.3.1: adds per-case, per-arm generation `usage` (input/output/cached tokens + measured wall_ms) captured from the surfaces that report it, a separate per-case `judge_usage` (measurement overhead, EXCLUDED from every skill-value figure by construction — `economics.judge_excluded` is const true), a run-level `run.pricing_snapshot` freezing the registry prices the derived dollar figures were computed from (so a receipt keeps its meaning when prices later change), and a derived `economics` block (per-arm mean cost/call, skill incremental cost per call and per 1k calls, output-length delta, median wall_ms with IQR). The three value axes — accuracy lift, cost, latency — are recorded separately and NEVER combined into a composite score. All v0.3.1 semantics are unchanged and every prior receipt still validates against its own frozen schema (v0.1, v0.2, v0.3, v0.3.1). The TESTED tightening (see allOf) is unchanged: the interop relaxations remain available only below TESTED.",
+  "description": "A hash-verified, dated record of running one agent skill's eval suite with and without the skill on one model version, with sampled judge scores and confidence bands. Receipt spec v0.4 — additive over v0.3.1: adds per-case, per-arm generation `usage` (input/output/cached tokens + measured wall_ms) captured from the surfaces that report it, a separate per-case `judge_usage` (measurement overhead, EXCLUDED from every skill-value figure by construction — `economics.judge_excluded` is const true), a run-level `run.pricing_snapshot` freezing the registry prices the derived dollar figures were computed from (so a receipt keeps its meaning when prices later change), and a derived `economics` block (per-arm mean cost/call, skill incremental cost per call and per 1k calls, output-length delta, median wall_ms with IQR). The three value axes — accuracy lift, cost, latency — are recorded separately and NEVER combined into a composite score. All v0.3.1 semantics are unchanged and every prior receipt still validates against its own frozen schema (v0.1, v0.2, v0.3, v0.3.1). The TESTED tightening (see allOf) is unchanged: the interop relaxations remain available only below TESTED.",
   "type": "object",
   "additionalProperties": false,
   "required": [