driftproof 0.3.0 → 0.5.0

package/README.md

@@ -15,9 +15,26 @@ across model releases and you get a **drift report**.
 Driftproof **consumes** the [`agentskills.io/evals`](https://agentskills.io) eval
 format; it does not invent its own.
 
-📊 **[Report #001](docs/reports/001/index.html)** ([markdown](reports/report-001.md))
-— ten public agent skills, run across a current-vs-previous Sonnet release with
-sampled, band-based verdicts.
+📊 **Four published reports** (each re-derived from committed receipts, nothing
+hand-entered), spanning three report types that share one band-based, floor-gated
+verdict rule and differ only in what moves underneath the skill:
+
+- **[Report #001](https://driftproofhq.com/reports/001/)** — *release drift*: ten
+  public agent skills across a current-vs-previous Sonnet release; 9 of 10 moved
+  beyond noise. ([markdown](reports/report-001.md))
+- **[Report #002](https://driftproofhq.com/reports/002/)** — *substrate
+  durability*: the same suites across two vendors' CLIs (Claude vs Codex);
+  3 durable, 4 substrate-dependent, 2 regressed, 1 no effect.
+- **[Report #003](https://driftproofhq.com/reports/003/)** — *release drift*:
+  `claude-opus-4-8` → `claude-opus-5`; 4 improved, 2 regressed, 4 within noise.
+- **[Report #004](https://driftproofhq.com/reports/004/)** — *capability gap*:
+  `claude-opus-5` (flagship) vs `claude-fable-5` (frontier tier);
+  3 durable, 5 tier-dependent, 0 regressions, 2 no effect — encoded expertise
+  survives the frontier tier.
+
+✍️ The launch essay, **[Three model releases later: what actually happens to agent
+skills](https://driftproofhq.com/writing/three-releases/)**, reads the first three
+reports together.
 
 ## Why
 
@@ -92,8 +109,21 @@ npx driftproof validate receipt.json
 
 # Emit a shields.io badge from a receipt (see "Badge", below)
 npx driftproof badge receipt.json --out badges/my-skill.json
+
+# Interop: convert another tool's results into a DECLARED receipt (honest
+# epistemics — no fabricated hashes, excluded from drift verdicts), and emit
+# the minimal stable summary other tools can consume. See docs/interop.md.
+npx driftproof import results.json --from agent-skills-eval   # or: skillgrade
+npx driftproof export receipt.json --to summary-json
 ```
 
+Receipts are an **open format** — the JSON Schema is served at its canonical id
+([driftproofhq.com/spec/receipt.schema.json](https://driftproofhq.com/spec/receipt.schema.json)),
+and any harness is encouraged to emit them. The interop contract (DECLARED vs
+TESTED, import mappings, `summary-json`) is documented in
+[`docs/interop.md`](docs/interop.md) and on
+[the interop page](https://driftproofhq.com/interop.html).
+
 A skill directory is expected to look like:
 
 ```
@@ -132,16 +162,19 @@ A receipt is the unit of evidence — one JSON document conforming to
 
 ```jsonc
 {
-  "schema_version": "0.2",
+  "schema_version": "0.4",
   "skill":  { "name": "commit-message-conventions", "version": "0.2.0",
               "content_hash": "…sha256 over SKILL.md + bundled files…" },
   "suite":  { "format": "agentskills.io/evals", "suite_hash": "…", "case_count": 10 },
   "run": {
     "model_id": "claude-haiku-4-5-20251001",
     "model_release_date": "2025-10-01",
+    "provider": "anthropic",
     "surface": "claude-cli",
-    "runner_version": "0.2.0",
+    "runner_version": "0.5.0",
     "date_utc": "2026-07-27T…Z",
+    "registry": "registered",
+    "transcripts": "hashes-only",
     "judge": { "samples": 5, "temperature": null, "sampling": "surface-controlled",
                "surface": "claude-cli" }
   },
@@ -200,7 +233,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: driftproofhq/driftproof@v0.3.0
+      - uses: driftproofhq/driftproof@v0.5.0
         with:
           skill-dir: skills/my-skill
           models: claude-haiku-4-5
@@ -232,38 +265,49 @@ git add badges/my-skill.json && git commit -m "chore: driftproof badge"
 ![driftproof](https://img.shields.io/endpoint?url=https://<your-site>/badges/my-skill.json)
 ```
 
-The [badge at the top of this README](badges/) is the living demo: it is generated
+The [badge at the top of this README](docs/badges/commit-message-conventions.json)
+is the living demo: it is generated
 from the `commit-message-conventions` example's own receipt and served from the
 site, so it reflects a real dated run, not a hand-set color.
 
 ## Reports
 
-**[Report #001](reports/report-001.md)** diffs ten public, text-representable
-agent skills across a Sonnet model pair. The report and every verdict in it are
-**re-derived from the receipts** in [`receipts/report-001/`](receipts/report-001/) —
-nothing is hand-entered.
+Four reports are published, spanning three report types (see the roll at the top
+of this README, and [REPORT-STYLE.md](REPORT-STYLE.md) for the shared chrome
+every report inherits). Each report and every verdict in it are **re-derived from
+the receipts** committed under [`receipts/`](receipts/)
+(`receipts/report-001/` … `receipts/report-004/`) — nothing is hand-entered.
 
 Driftproof does **not** commit third-party skill content. Each `SKILL.md` is
 fetched at run time from a pinned commit and verified by sha256 against
 [`suites/manifest.json`](suites/manifest.json); we commit only the manifest, our
-authored eval suites ([`suites/`](suites/)), the receipts, and the report.
-Reproduce it in three commands:
+authored eval suites ([`suites/`](suites/)), the receipts, and the reports.
+Reproduce Report #001 in three commands:
 
 ```bash
 node scripts/fetch-skills.js                     # fetch pinned SKILL.md files (sha256-verified) → untracked workdir
-node scripts/run-report-001.js --concurrency 5   # run both models × with/baseline, emit v0.2 receipts
+node scripts/run-report-001.js --concurrency 5   # run both models × with/baseline, emit receipts
 node scripts/build-report-001.js                 # re-derive the report from the receipts
 ```
 
+Reports #002–#004 have their own runners
+(`scripts/prepare-report-00N.js`) following the same
+fetch → run → re-derive shape.
+
+**Model-release triggers are live**: `scripts/release-watch.js` (keyless — it
+reads the public models registry) notices a new model release, re-runs the
+receipts, and queues a draft report in `reports/pending-publish.md` for human
+review — drift is caught at the release, not months later.
+
 ## Roadmap
 
-- **Monthly drift reports** for a curated set of public skills, published under
-  `reports/` and at driftproofhq.com.
-- **Model-release triggers** — re-run receipts automatically when a new model
-  version ships, so drift is caught at the release, not months later.
 - **A sandboxed-execution harness** for tool-execution skills (document
   renderers, diagram/asset generators) that Report #001 scopes out.
-- **Signed receipts** — key signatures / attestation over the canonical form.
+- **Signed receipts** — key signatures / attestation over the canonical form
+  (today's `receipt_hash` is tamper-evidence, not authenticity).
+- **More import mappings** — the interop wall (DECLARED vs TESTED) is built;
+  adding a converter for another harness's results format is a small,
+  well-marked PR (see [docs/interop.md](docs/interop.md)).
 
 ## Contributing
 

package/bin/driftproof

@@ -9,7 +9,7 @@ const { loadSkill } = require('../lib/skill');
 const { runSkillOnModel, summarizeReceipt, projectCalls } = require('../lib/run');
 const { validateReceipt, verifyReceiptHash } = require('../lib/receipt');
 const { buildDriftReport } = require('../lib/diff');
-const { surfaceLabel, providerName, resolveModel } = require('../lib/provider');
+const { surfaceForModel, isSubscriptionSurface, resolveModel } = require('../lib/provider');
 const { estimateRunCostUSD, BudgetTracker } = require('../lib/cost');
 const { registryStatus } = require('../lib/models');
 const { verdictFromReceipt, badgeEndpoint, githubOutputLines } = require('../lib/verdict');
@@ -79,6 +79,8 @@ USAGE
   ${PROJECT_NAME} diff <receiptA.json> <receiptB.json> [--out FILE]
   ${PROJECT_NAME} validate <receipt.json>
   ${PROJECT_NAME} badge <receipt.json> [--out FILE] [--github-output]
+  ${PROJECT_NAME} import <results.json> --from agent-skills-eval|skillgrade [--out DIR]
+  ${PROJECT_NAME} export <receipt.json> [--to summary-json] [--report-url URL] [--out FILE]
 
 ENV
   CLAUDE_PROVIDER   api | cli   (default: cli — spawns \`claude -p\`, strips ANTHROPIC_API_KEY)
@@ -104,7 +106,13 @@ NOTES
     model still runs but is marked registry:"unregistered" and costed at the
     conservative default price.
   - --concurrency runs that many (case,mode) tasks at once (default 1). Higher
-    values cut wall-clock on the cli surface (cold-start dominated).`);
+    values cut wall-clock on the cli surface (cold-start dominated).
+  - import converts another tool's results into a valid receipt with HONEST
+    epistemics: verification_level DECLARED (never TESTED), surface "external",
+    source "imported/<tool>", and NO fabricated hashes. Imported receipts are
+    excluded from drift verdicts (diff reports NOT MEASURED). See docs/interop.md.
+  - export --to summary-json emits the minimal stable interchange summary
+    (driftproof/summary v1) other tools can consume without a receipt parser.`);
 }
 
 function slug(s) { return String(s).toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, ''); }
@@ -137,7 +145,10 @@ async function cmdRun(positional, flags) {
   // On `claude-cli` the metered spend is $0 (subscription); the figure is the
   // hypothetical "if run on the metered API" cost — printed, never blocks.
   const cost = estimateRunCostUSD({ caseCount: nCases, samples, models: models.map((m) => require('../lib/provider').resolveModel(m)), judgeModel: judgeModel || 'haiku' });
-  const surface = surfaceLabel();
+  // Surface is per-model now (a run may mix an Anthropic and an OpenAI target).
+  const surfaces = [...new Set(models.map((m) => surfaceForModel(m)))];
+  const surface = surfaces.join(', ');
+  const subSurfaces = surfaces.filter(isSubscriptionSurface);
 
   const regStatuses = models.map((m) => `${m}:${registryStatus(m)}`);
   console.log(`\n${PROJECT_NAME} run — skill "${skill.name}" v${skill.version}`);
@@ -146,7 +157,7 @@ async function cmdRun(positional, flags) {
   console.log(`  registry: ${regStatuses.join('  ')}${keepTranscripts ? '   transcripts: retained-local' : ''}`);
   console.log(`  projected calls: ${perModelCalls}/model × ${models.length} model(s) = ${totalProjected}   per-model cap: ${maxCalls}`);
   console.log(`  projected cost: ~$${cost.totalUSD.toFixed(2)} (rough upper bound; budget $${maxUsd.toFixed(2)}, hard-stop $${(maxUsd * 1.25).toFixed(2)})`);
-  if (surface !== 'api') console.log(`  actual metered spend on ${surface}: $0.00 (subscription; the $ figure is the estimated-equivalent API cost, counted against the cap identically)`);
+  if (subSurfaces.length) console.log(`  actual metered spend on ${subSurfaces.join(', ')}: $0.00 (subscription; the $ figure is the estimated-equivalent API cost, counted against the cap identically)`);
 
   // Call-count guard: refuse before spending anything if a single model run
   // would exceed the per-run call cap.
@@ -307,6 +318,55 @@ function cmdBadge(positional, flags) {
   }
 }
 
+// Convert another tool's results file into a valid DECLARED receipt (interop,
+// Phase 7). The converted receipt is validated + self-hash-verified before it
+// is written; a failed conversion writes nothing.
+function cmdImport(positional, flags) {
+  const p = positional[0];
+  const from = flags.from;
+  const { IMPORT_TOOLS, importResults } = require('../lib/importers');
+  if (!p || typeof from !== 'string') {
+    console.error(`usage: driftproof import <results.json> --from ${IMPORT_TOOLS.join('|')} [--out DIR]`);
+    process.exit(2);
+  }
+  const data = JSON.parse(fs.readFileSync(p, 'utf8'));
+  const receipt = importResults(data, { from, importedAt: new Date().toISOString() });
+  const { valid, errors } = validateReceipt(receipt);
+  if (!valid || !verifyReceiptHash(receipt)) {
+    console.error('✗ converted receipt failed validation — nothing written:', JSON.stringify(errors, null, 2));
+    process.exit(1);
+  }
+  const outDir = flags.out ? path.resolve(flags.out) : RECEIPTS_DIR;
+  fs.mkdirSync(outDir, { recursive: true });
+  const base = `${slug(receipt.skill.name)}-${slug(receipt.run.model_id)}-imported-${slug(from)}-${dateStamp(receipt.run.date_utc)}`;
+  const jsonPath = path.join(outDir, `${base}.json`);
+  fs.writeFileSync(jsonPath, JSON.stringify(receipt, null, 2));
+  console.log(`imported → ${path.relative(process.cwd(), jsonPath)}`);
+  console.log(`  verification_level: DECLARED (source ${receipt.run.source}) — the source tool's declaration, converted faithfully;`);
+  console.log(`  no generation hashes (never fabricated); excluded from drift verdicts unless re-run TESTED. See docs/interop.md.`);
+}
+
+// Emit the minimal stable interchange summary for one receipt (interop, Phase 7).
+function cmdExport(positional, flags) {
+  const p = positional[0];
+  const to = flags.to || 'summary-json';
+  if (!p) { console.error('usage: driftproof export <receipt.json> [--to summary-json] [--report-url URL] [--out FILE]'); process.exit(2); }
+  if (to !== 'summary-json') { console.error(`unknown export target "${to}" — supported: summary-json`); process.exit(2); }
+  const { toSummaryJson } = require('../lib/export');
+  const receipt = JSON.parse(fs.readFileSync(p, 'utf8'));
+  if (!verifyReceiptHash(receipt)) console.error(`  ⚠ ${path.basename(p)}: receipt_hash does not verify (tampered or hand-edited)`);
+  const summary = toSummaryJson(receipt, { reportUrl: typeof flags['report-url'] === 'string' ? flags['report-url'] : null });
+  const json = JSON.stringify(summary, null, 2);
+  if (flags.out) {
+    const out = path.resolve(flags.out);
+    fs.mkdirSync(path.dirname(out), { recursive: true });
+    fs.writeFileSync(out, json + '\n');
+    console.log(`summary written to ${flags.out}  (${summary.verdict}, delta ${summary.delta === null ? 'n/a' : summary.delta})`);
+  } else {
+    console.log(json);
+  }
+}
+
 async function main() {
   const [, , cmd, ...rest] = process.argv;
   const { positional, flags } = parseArgs(rest);
@@ -316,6 +376,8 @@ async function main() {
     case 'diff': return cmdDiff(positional, flags);
     case 'validate': return cmdValidate(positional);
     case 'badge': return cmdBadge(positional, flags);
+    case 'import': return cmdImport(positional, flags);
+    case 'export': return cmdExport(positional, flags);
     case 'version': case '--version': case '-v': console.log(RUNNER_VERSION); return;
     case 'help': case '--help': case '-h': case undefined: return usage();
     default: console.error(`unknown command: ${cmd}\n`); usage(); process.exit(2);

package/config/models.json

@@ -1,10 +1,14 @@
 {
-  "_comment": "Driftproof model registry. The runner resolves --models ids against this list; an unknown id still runs but its receipt is marked registry:\"unregistered\" and its cost is estimated with the conservative default price in lib/models.js. Prices are STANDARD first-party USD per 1,000,000 tokens (input/output) from platform.claude.com (July 2026); Sonnet 5's introductory rate is deliberately NOT used so projections stay an upper bound. `released` is best-effort (spec RECEIPT.md open question #4): dates derived from a dated model id or Anthropic launch posts, else null. `tier` drives nothing but reporting; `judge_eligible` encodes the fixed-judge policy (see docs/judge-policy.html) — only the cheap Haiku judge is eligible. The release trigger (scripts/release-watch.js) appends auto-discovered ids here with auto_added:true.",
-  "registry_version": "1.0",
+  "_comment": "Driftproof model registry. The runner resolves --models ids against this list; an unknown id still runs but its receipt is marked registry:\"unregistered\" and its cost is estimated with the conservative default price in lib/models.js. Prices are STANDARD first-party USD per 1,000,000 tokens (input/output). Anthropic prices are from platform.claude.com (July 2026); Sonnet 5's introductory rate is deliberately NOT used so projections stay an upper bound. OpenAI prices are the public standard per-MTok rates as of July 2026 (see reports/phase-6-providers.md for the fetched source): GPT-5.6 Sol $5/$30, Terra $2.50/$15, Luna $1/$6; GPT-5.5 $5/$30; GPT-5.4 $2.50/$15. gpt-5.4-mini is NOT in the published table (which lists a Nano tier at $0.20/$1.25), so its price here is a deliberately conservative UPPER-bound estimate flagged price_estimate:true. `released` is best-effort (spec RECEIPT.md open question #4). `tier` drives reporting; `judge_eligible` encodes the fixed-judge policy (docs/judge-policy.html) — only the cheap Haiku judge is eligible, and NO OpenAI model is judge-eligible (the judge stays claude-haiku across providers so a cross-substrate report varies only the target model). `provider` is the two-axis provider; per-provider surface/base_url config is under `providers`. The release trigger (scripts/release-watch.js) appends auto-discovered ids here with auto_added:true.",
+  "registry_version": "1.1",
   "provider": "anthropic",
+  "providers": {
+    "anthropic": { "surfaces": ["api", "cli"] },
+    "openai": { "base_url": "https://api.openai.com/v1", "api_key_env": "OPENAI_API_KEY", "surfaces": ["api", "cli"] }
+  },
   "default_price_note": "Unregistered ids are costed at the most expensive known tier (Fable, $10/$50) — a budget guard must never under-estimate. See lib/models.js DEFAULT_PRICE.",
   "models": [
-    { "id": "claude-fable-5",              "family": "fable",  "provider": "anthropic", "released": null,         "input_price": 10.0, "output_price": 50.0, "tier": "frontier", "judge_eligible": false },
+    { "id": "claude-fable-5",              "family": "fable",  "provider": "anthropic", "released": "2026-06-09", "input_price": 10.0, "output_price": 50.0, "tier": "frontier", "judge_eligible": false },
     { "id": "claude-opus-5",               "family": "opus",   "provider": "anthropic", "released": null,         "input_price": 5.0,  "output_price": 25.0, "tier": "frontier", "judge_eligible": false },
     { "id": "claude-opus-4-8",             "family": "opus",   "provider": "anthropic", "released": null,         "input_price": 5.0,  "output_price": 25.0, "tier": "frontier", "judge_eligible": false },
     { "id": "claude-opus-4-7",             "family": "opus",   "provider": "anthropic", "released": null,         "input_price": 5.0,  "output_price": 25.0, "tier": "frontier", "judge_eligible": false },
@@ -14,6 +18,12 @@
     { "id": "claude-sonnet-4-6",           "family": "sonnet", "provider": "anthropic", "released": "2026-02-17", "input_price": 3.0,  "output_price": 15.0, "tier": "standard", "judge_eligible": false },
     { "id": "claude-sonnet-4-5-20250929",  "family": "sonnet", "provider": "anthropic", "released": "2025-09-29", "input_price": 3.0,  "output_price": 15.0, "tier": "standard", "judge_eligible": false },
     { "id": "claude-haiku-4-5",            "family": "haiku",  "provider": "anthropic", "released": "2025-10-01", "input_price": 1.0,  "output_price": 5.0,  "tier": "cheap",    "judge_eligible": true },
-    { "id": "claude-haiku-4-5-20251001",   "family": "haiku",  "provider": "anthropic", "released": "2025-10-01", "input_price": 1.0,  "output_price": 5.0,  "tier": "cheap",    "judge_eligible": true }
+    { "id": "claude-haiku-4-5-20251001",   "family": "haiku",  "provider": "anthropic", "released": "2025-10-01", "input_price": 1.0,  "output_price": 5.0,  "tier": "cheap",    "judge_eligible": true },
+    { "id": "gpt-5.6-sol",                 "family": "gpt-5.6", "provider": "openai",   "released": "2026-07-09", "input_price": 5.0,  "output_price": 30.0, "tier": "frontier", "judge_eligible": false },
+    { "id": "gpt-5.6-terra",               "family": "gpt-5.6", "provider": "openai",   "released": "2026-07-09", "input_price": 2.5,  "output_price": 15.0, "tier": "standard", "judge_eligible": false },
+    { "id": "gpt-5.6-luna",                "family": "gpt-5.6", "provider": "openai",   "released": "2026-07-09", "input_price": 1.0,  "output_price": 6.0,  "tier": "cheap",    "judge_eligible": false },
+    { "id": "gpt-5.5",                     "family": "gpt-5.5", "provider": "openai",   "released": null,         "input_price": 5.0,  "output_price": 30.0, "tier": "frontier", "judge_eligible": false },
+    { "id": "gpt-5.4",                     "family": "gpt-5.4", "provider": "openai",   "released": null,         "input_price": 2.5,  "output_price": 15.0, "tier": "standard", "judge_eligible": false },
+    { "id": "gpt-5.4-mini",                "family": "gpt-5.4", "provider": "openai",   "released": null,         "input_price": 0.4,  "output_price": 2.0,  "tier": "cheap",    "judge_eligible": false, "price_estimate": true, "price_note": "conservative upper-bound estimate; the public table lists a GPT-5.4 Nano tier ($0.20/$1.25) but not a Mini rate as of July 2026" }
   ]
 }

package/config.js

@@ -9,16 +9,27 @@ const PROJECT_NAME = 'driftproof';
 // Bumped whenever the runner's behaviour or receipt-generation semantics change
 // in a way that could affect results. Recorded into every receipt as
 // run.runner_version so a receipt is reproducible against a known engine.
-const RUNNER_VERSION = '0.3.0';
+const RUNNER_VERSION = '0.5.0';
 
 // The eval format we CONSUME (we deliberately do not invent our own).
 const SUITE_FORMAT = 'agentskills.io/evals';
 
 // Receipt schema version this runner emits. Loader/validator accept older
 // versions too (see lib/receipt.js), but new receipts are stamped current.
-// v0.3 adds transcript-auditability: per-case generation_hash + judge_sample_
-// hashes[], plus run.registry and run.transcripts. Additive; v0.1/v0.2 still load.
-const RECEIPT_SCHEMA_VERSION = '0.3';
+// v0.3   adds transcript-auditability: per-case generation_hash + judge_sample_
+//        hashes[], plus run.registry and run.transcripts.
+// v0.3.1 (additive over v0.3) adds run.provider (two-axis provider), the OpenAI
+//        surface enums (openai-api/openai-cli), optional run.surface_overhead_note,
+//        optional per-case checks[] (deterministic post-checks), and optional
+//        skill.tokens (value-per-token axis). v0.1/v0.2/v0.3 receipts still load.
+// v0.4   (additive over v0.3.1) adds the ECONOMICS axis: per-case, per-arm
+//        generation `usage` (input/output/cached tokens + measured wall_ms), a
+//        separate per-case `judge_usage` (measurement overhead, excluded from
+//        every skill-value figure), run.pricing_snapshot (registry prices frozen
+//        at run time so derived dollars stay reproducible), and the derived
+//        `economics` block. v0.3.1 is frozen as receipt.v0.3.1.schema.json;
+//        v0.1/v0.2/v0.3/v0.3.1 receipts all still load.
+const RECEIPT_SCHEMA_VERSION = '0.4';
 
 // Hard USD budget defaults per entry point (Week 4). --max-usd overrides any of
 // these. The projection is refused before any call if it exceeds the cap, on
@@ -45,4 +56,43 @@ const DEFAULT_JUDGE_SAMPLES = 5;
 // spec/RECEIPT.md § "Drift verdict rule" and the report methodology.
 const EFFECT_FLOOR = 0.05;
 
-module.exports = { PROJECT_NAME, RUNNER_VERSION, SUITE_FORMAT, RECEIPT_SCHEMA_VERSION, DEFAULT_JUDGE_SAMPLES, EFFECT_FLOOR, DEV_MAX_USD, REPORT_MAX_USD, TRIGGER_MAX_USD };
+// Report #002 is the first CROSS-PROVIDER report: the same suites, the same fixed
+// Haiku judge, run on two substrates — a Claude flagship and a GPT flagship. The
+// GPT flagship is a config constant (not hard-coded across scripts) so a future
+// edition swaps one line. Default: the current GPT flagship, gpt-5.6-sol.
+const REPORT_002_CLAUDE_MODEL = 'claude-sonnet-5';
+const REPORT_002_GPT_MODEL = 'gpt-5.6-sol';
+const REPORT_002_JUDGE_MODEL = 'claude-haiku-4-5';
+
+// Report #003 is a RELEASE DRIFT report (Report #001's type): one provider, two
+// model versions, verdicts REGRESSED/IMPROVED/MIXED/WITHIN NOISE per skill under
+// the per-case band-separation rule + effect floor. New vs its family predecessor.
+const REPORT_003_NEW_MODEL = 'claude-opus-5';
+const REPORT_003_OLD_MODEL = 'claude-opus-4-8';
+const REPORT_003_JUDGE_MODEL = 'claude-haiku-4-5';
+
+// Report #004 is a CAPABILITY-GAP report (Report #002's verdict style, ONE
+// provider, TWO tiers): fable-5 has NO family predecessor, so this is a
+// cross-family study, NOT release drift. Per skill: the with/without delta on
+// EACH model with bands; tier comparison is context. Headline question: does
+// encoded expertise still lift output on the frontier tier?
+const REPORT_004_BASE_MODEL = 'claude-opus-5';     // flagship tier
+const REPORT_004_FRONTIER_MODEL = 'claude-fable-5'; // frontier tier (full id — no alias)
+const REPORT_004_JUDGE_MODEL = 'claude-haiku-4-5';
+
+// Report #005 is a VALUE report — the fourth report type. It asks what a skill
+// COSTS to run alongside whether it helps, over three substrates, and shows the
+// three axes (accuracy lift / Δcost / Δlatency) side by side and never combined.
+// Same suites, same fixed judge; the substrate list spans two providers so the
+// economics are read across surfaces, not within one vendor's pricing.
+const REPORT_005_MODELS = ['claude-sonnet-5', 'claude-fable-5', 'gpt-5.6-sol'];
+const REPORT_005_JUDGE_MODEL = 'claude-haiku-4-5';
+
+module.exports = {
+  PROJECT_NAME, RUNNER_VERSION, SUITE_FORMAT, RECEIPT_SCHEMA_VERSION, DEFAULT_JUDGE_SAMPLES,
+  EFFECT_FLOOR, DEV_MAX_USD, REPORT_MAX_USD, TRIGGER_MAX_USD,
+  REPORT_002_CLAUDE_MODEL, REPORT_002_GPT_MODEL, REPORT_002_JUDGE_MODEL,
+  REPORT_003_NEW_MODEL, REPORT_003_OLD_MODEL, REPORT_003_JUDGE_MODEL,
+  REPORT_004_BASE_MODEL, REPORT_004_FRONTIER_MODEL, REPORT_004_JUDGE_MODEL,
+  REPORT_005_MODELS, REPORT_005_JUDGE_MODEL,
+};

package/lib/checks.js

@@ -0,0 +1,50 @@
+// SPDX-License-Identifier: Apache-2.0
+'use strict';
+
+// Deterministic post-checks.
+//
+// A per-case eval suite may declare optional `checks[]`: structural / regex
+// assertions on the model OUTPUT that either hold or don't — no LLM judgment.
+// They run ALONGSIDE the sampled judge and are reported as a SEPARATE column.
+//
+// IMPORTANT (scope): post-checks are SUPPLEMENTARY EVIDENCE ONLY. They are NOT
+// folded into the case `outcome` or the band-overlap drift verdict — a check is a
+// cheap, unambiguous signal ("did the output contain a Conventional-Commits
+// subject?") that corroborates or contradicts the judge, not a second grader. A
+// suite author adds them where they are natural and in-text-groundable; they are
+// never required.
+//
+// Supported kinds (kept small and unambiguous):
+//   regex        — the pattern (with optional `flags`) matches the output
+//   contains     — the output includes the literal `value` substring
+//   not_contains — the output does NOT include the literal `value` substring
+//   min_length   — the trimmed output is at least `value` characters long
+
+function runOneCheck(check, output) {
+  const text = String(output || '');
+  switch (check && check.kind) {
+    case 'regex': {
+      let re;
+      try { re = new RegExp(check.pattern, check.flags || ''); } catch (_e) { return false; }
+      return re.test(text);
+    }
+    case 'contains': return text.includes(String(check.value));
+    case 'not_contains': return !text.includes(String(check.value));
+    case 'min_length': return text.trim().length >= Number(check.value || 0);
+    default: return false;
+  }
+}
+
+// Run every declared check against one output. Returns a compact result array
+// [{ name, kind, pass }] suitable for the receipt (v0.3.1 optional per-case
+// `checks`). Empty array when the case declares no checks.
+function runChecks(output, checks) {
+  if (!Array.isArray(checks) || !checks.length) return [];
+  return checks.map((c) => ({
+    name: String((c && c.name) || (c && c.kind) || 'check'),
+    kind: c && c.kind,
+    pass: !!runOneCheck(c, output),
+  }));
+}
+
+module.exports = { runChecks, runOneCheck };

package/lib/diff.js

@@ -73,15 +73,24 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B' } = {}) {
   const bB = withSkillBands(b);
   const ids = [...new Set([...Object.keys(aB), ...Object.keys(bB)])];
 
+  // Interop (Phase 7): drift verdicts require TESTED receipts on BOTH sides.
+  // An imported (DECLARED) receipt is a faithful record of another tool's
+  // declaration — its numbers are shown, but band-verified regression/
+  // improvement verdicts are never computed from evidence we did not run.
+  const levelOf = (r) => r.verification_level || 'TESTED';
+  const belowTested = [[labelA, a], [labelB, b]].filter(([, r]) => levelOf(r) !== 'TESTED');
+  const measured = belowTested.length === 0;
+
   const perCase = ids.map((id) => {
     const before = aB[id] || null;
     const after = bB[id] || null;
     const delta = (before && after) ? round(after.mean - before.mean) : null;
-    const verdict = (before && after) ? verdictWithFloor(before, after, delta) : 'n/a';
+    const verdict = !measured ? 'not measured'
+      : (before && after) ? verdictWithFloor(before, after, delta) : 'n/a';
     return { id, before, after, delta, verdict };
   });
   // Sort worst-first: regressions, then by delta.
-  const order = { regression: 0, 'within noise': 1, [WITHIN_NOISE_FLOOR]: 1, improvement: 2, 'n/a': 3 };
+  const order = { regression: 0, 'within noise': 1, [WITHIN_NOISE_FLOOR]: 1, improvement: 2, 'n/a': 3, 'not measured': 3 };
   perCase.sort((x, y) => (order[x.verdict] - order[y.verdict]) || ((x.delta || 0) - (y.delta || 0)));
 
   const aAgg = aggWithBand(a);
@@ -112,6 +121,13 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B' } = {}) {
   if (a.suite.suite_hash !== b.suite.suite_hash) warnings.push('suite_hash differs — the eval suite changed; per-case comparison may be misleading.');
   if (a.skill.name !== b.skill.name) warnings.push(`different skills (${a.skill.name} vs ${b.skill.name}) — comparison is not meaningful.`);
   if ((a.run.judge || {}).samples <= 1 || (b.run.judge || {}).samples <= 1) warnings.push('one or both receipts are single-sample (no bands) — non-overlap can only be trusted when both sides are sampled.');
+  if (!measured) warnings.push(`verdicts NOT computed — ${belowTested.map(([l, r]) => `${l} is ${levelOf(r)}${r.run && r.run.source ? ` (${r.run.source})` : ''}`).join('; ')}. Drift verdicts require TESTED receipts on both sides; declared numbers are shown as context only (see /interop.html).`);
+  // Cross-provider / cross-surface disclosure (Phase 6). A comparison across
+  // providers is a skill-DURABILITY comparison across substrates, not model drift
+  // over time; across surfaces, sampling control differs. Both are flagged so a
+  // reader never mistakes one for the other (see docs/neutrality.html).
+  if ((a.run.provider || 'anthropic') !== (b.run.provider || 'anthropic')) warnings.push(`different providers (${a.run.provider || 'anthropic'} vs ${b.run.provider || 'anthropic'}) — this is a cross-substrate durability comparison, not model drift over time; read the delta, not absolute scores (see the neutrality policy).`);
+  if (a.run.surface !== b.run.surface) warnings.push(`different surfaces (${a.run.surface} vs ${b.run.surface}) — sampling control differs between surfaces; compare with care.`);
   if (warnings.length) {
     L.push('> **⚠ Caveats**');
     for (const w of warnings) L.push(`> - ${w}`);
@@ -122,17 +138,22 @@ function buildDriftReport(a, b, { labelA = 'A', labelB = 'B' } = {}) {
   const nFloor = perCase.filter((r) => r.verdict === WITHIN_NOISE_FLOOR).length;
   L.push(`## Headline`);
   L.push('');
-  L.push(`**${headlineVerdict(perCase)}**`);
-  L.push('');
-  L.push(`with_skill mean moved ${fmt(headlineDelta)} (${bandStr(aAgg)} → ${bandStr(bAgg)}; band = suite dispersion). Per-case band-overlap verdicts: ${regressions.length} regression(s), ${perCase.filter((r) => r.verdict === 'improvement').length} improvement(s), ${nWithin} within noise${nFloor ? ` (${nFloor} of them band-separated but below the ${EFFECT_FLOOR} effect floor)` : ''}.`);
-  L.push('');
+  if (!measured) {
+    L.push(`**NOT MEASURED — ${belowTested.length} receipt(s) below TESTED. Drift verdicts require TESTED receipts on both sides; the declared numbers above are context, not band-verified evidence.**`);
+    L.push('');
+  } else {
+    L.push(`**${headlineVerdict(perCase)}**`);
+    L.push('');
+    L.push(`with_skill mean moved ${fmt(headlineDelta)} (${bandStr(aAgg)} → ${bandStr(bAgg)}; band = suite dispersion). Per-case band-overlap verdicts: ${regressions.length} regression(s), ${perCase.filter((r) => r.verdict === 'improvement').length} improvement(s), ${nWithin} within noise${nFloor ? ` (${nFloor} of them band-separated but below the ${EFFECT_FLOOR} effect floor)` : ''}.`);
+    L.push('');
+  }
 
   L.push(`## Per-case with_skill (band overlap → verdict)`);
   L.push('');
   L.push(`| case | ${labelA} (mean ± sd) | ${labelB} (mean ± sd) | Δ | verdict |`);
   L.push(`|---|---|---|---|---|`);
   for (const r of perCase) {
-    const flag = r.verdict === 'regression' ? '🔻 regression' : r.verdict === 'improvement' ? '🔼 improvement' : r.verdict === WITHIN_NOISE_FLOOR ? 'within noise (below floor)' : r.verdict === 'within noise' ? 'within noise' : 'n/a';
+    const flag = r.verdict === 'regression' ? '🔻 regression' : r.verdict === 'improvement' ? '🔼 improvement' : r.verdict === WITHIN_NOISE_FLOOR ? 'within noise (below floor)' : r.verdict === 'within noise' ? 'within noise' : r.verdict === 'not measured' ? 'not measured' : 'n/a';
     L.push(`| \`${r.id}\` | ${r.before ? bandStr(r.before) : 'n/a'} | ${r.after ? bandStr(r.after) : 'n/a'} | ${fmt(r.delta)} | ${flag} |`);
   }
   L.push('');

package/lib/export.js

@@ -0,0 +1,52 @@
+// SPDX-License-Identifier: Apache-2.0
+'use strict';
+
+// Receipt interop — export (Phase 7). The lightweight interchange: a minimal,
+// STABLE, flat JSON summary of one receipt (skill, model, bands, delta,
+// verdict) that other tools can consume without a receipt parser. The
+// "driftproof/summary" v1 keys are frozen; additions bump format_version.
+// Documented in docs/interop.md; snapshot-tested in the gate.
+
+const { verdictFromReceipt } = require('./verdict');
+
+const SUMMARY_FORMAT = 'driftproof/summary';
+const SUMMARY_FORMAT_VERSION = '1';
+
+// Build the summary object for one receipt. Deterministic for a given receipt:
+// fixed key order, no export-time timestamps. `reportUrl` is caller-supplied
+// (receipts do not know where their report lives), else null.
+function toSummaryJson(receipt, { reportUrl = null } = {}) {
+  const agg = receipt.results.aggregates;
+  const cmp = receipt.comparison || {};
+  const hasBaseline = agg.baseline && agg.baseline.case_count > 0;
+  const v = verdictFromReceipt(receipt);
+  return {
+    format: SUMMARY_FORMAT,
+    format_version: SUMMARY_FORMAT_VERSION,
+    skill: { name: receipt.skill.name, version: receipt.skill.version },
+    model: {
+      id: receipt.run.model_id,
+      provider: receipt.run.provider || 'anthropic',
+      surface: receipt.run.surface,
+    },
+    run_date_utc: receipt.run.date_utc,
+    scores: {
+      with_skill: { mean: agg.with_skill.mean_score, stddev: agg.with_skill.stddev },
+      baseline: hasBaseline ? { mean: agg.baseline.mean_score, stddev: agg.baseline.stddev } : null,
+    },
+    delta: typeof cmp.delta === 'number' ? cmp.delta : null,
+    delta_uncertainty: typeof cmp.delta_uncertainty === 'number' ? cmp.delta_uncertainty : null,
+    verdict: v.verdict,
+    verification_level: receipt.verification_level,
+    source: receipt.run.source || 'driftproof',
+    judge: {
+      model_id: (receipt.results.cases.find((c) => c.judge) || { judge: { model_id: 'unknown' } }).judge.model_id,
+      samples: (receipt.run.judge || {}).samples || 1,
+    },
+    receipt_hash: receipt.receipt_hash,
+    report_url: reportUrl,
+    spec: 'https://driftproofhq.com/spec/receipt.schema.json',
+  };
+}
+
+module.exports = { toSummaryJson, SUMMARY_FORMAT, SUMMARY_FORMAT_VERSION };