driftproof 0.7.2 → 0.8.0

package/README.md

@@ -1,25 +1,28 @@
 <!-- SPDX-License-Identifier: Apache-2.0 -->
 # Driftproof
 
-**A dated proof that this skill, this hash, this model, still helps.**
+**Your skill passed. On which model? On what date?**
+
+A SKILL.md teaches an AI coding agent how you like things done. When a new model
+ships, the same file can stop helping, or start hurting. Driftproof re-runs the
+skill's tests on the new model and hands you a dated, hash-verified receipt
+saying whether it still helps.
 
 [![driftproof](https://img.shields.io/endpoint?url=https://driftproofhq.com/badges/commit-message-conventions.json)](https://driftproofhq.com)
 &nbsp;— live badge for the bundled `commit-message-conventions` example, generated from its own receipt.
 
-Driftproof is an open **receipt spec** plus a **runner** that measures whether an
-agent skill actually helps — by running the skill's eval suite **with** and
-**without** the skill on a named model version, judging each case several times to
-get a confidence band, and emitting a **hash-verified**, dated **receipt**. Diff two receipts
-across model releases and you get a **drift report**.
+**[Quickstart](https://driftproofhq.com/#quickstart)** ·
+**[Latest report](https://driftproofhq.com/reports/008/)** ·
+[driftproofhq.com](https://driftproofhq.com)
 
 Driftproof **consumes** the [`agentskills.io/evals`](https://agentskills.io) eval
 format; it does not invent its own.
 
-📊 **Seven published reports** (each re-derived from committed receipts, nothing
-hand-entered), spanning six published report types, the newest being instrument
-re-measurement. All seven share one band-based, floor-gated verdict rule and
+📊 **Eight published reports** (each re-derived from committed receipts, nothing
+hand-entered), spanning six published report types, the newest being release
+drift. All eight share one band-based, floor-gated verdict rule and
 differ in what moves underneath the skill — or, in the value report, in which
-axes are measured; or, in the newest, in the instrument itself:
+axes are measured; or, in the instrument re-measurement, in the instrument itself:
 
 - **[Report #001](https://driftproofhq.com/reports/001/)** — *release drift*: ten
   public agent skills across a current-vs-previous Sonnet release; 9 of 10 moved
@@ -62,9 +65,18 @@ axes are measured; or, in the newest, in the instrument itself:
   2026-07-27, and the truncated run it caused measured *less* variance than the
   clean re-run, which is the direction that flatters an instrument. Both runs are
   published, the broken one as evidence. Amends #005 to v1.2 and #006 to v1.1.
+- **[Report #008](https://driftproofhq.com/reports/008/)** — *release drift*:
+  two of Report #007's cells re-measured on `claude-fable-5-1` against
+  `claude-fable-5`, with the skill `content_hash` and `suite_hash` asserted
+  identical before the first call. **Both cells came back within noise**: the
+  14 cases read 0 improved, 0 regressed, 14 within noise, 0 not measured. The
+  first release pair in this project where both sides are generation-sampled
+  receipts, which is what makes the delta attributable to the model rather than
+  to the instrument. One case sits inside the verdict on the effect floor alone
+  and the report names it.
 
 ✍️ The launch essay, **[Three model releases later: what actually happens to agent
-skills](https://driftproofhq.com/writing/three-releases/)**, reads all seven reports
+skills](https://driftproofhq.com/writing/three-releases/)**, reads all eight reports
 together: what moves underneath a skill, what the skill costs to run, and what a
 corrected instrument did to three published results. Revised 2026-09-01; every
 figure in it is gate-checked against the report page it cites.
@@ -229,7 +241,7 @@ A receipt is the unit of evidence — one JSON document conforming to
     "model_release_date": "2025-10-01",
     "provider": "anthropic",
     "surface": "claude-cli",
-    "runner_version": "0.7.2",
+    "runner_version": "0.8.0",
     "date_utc": "2026-07-27T…Z",
     "registry": "registered",
     "transcripts": "hashes-only",
@@ -303,7 +315,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: driftproofhq/driftproof@v0.7.2
+      - uses: driftproofhq/driftproof@v0.8.0
         with:
           skill-dir: skills/my-skill
           models: claude-haiku-4-5
@@ -343,7 +355,7 @@ site, so it reflects a real dated run, not a hand-set color.
 
 ## Reports
 
-Seven reports are published, spanning six report types. A report page lives at a
+Eight reports are published, spanning six report types. A report page lives at a
 draft path — `docs/reports/NNN-draft/` — until the publish sequence renames it, and
 `scripts/build-public.sh` excludes every `*-draft/` path from the published tree
 (see the roll at the top of this README, and

package/config/models.json

@@ -1,5 +1,5 @@
 {
-  "_comment": "Driftproof model registry. The runner resolves --models ids against this list; an unknown id still runs but its receipt is marked registry:\"unregistered\" and its cost is estimated with the conservative default price in lib/models.js. Prices are STANDARD first-party USD per 1,000,000 tokens (input/output). Anthropic prices are from platform.claude.com (July 2026); Sonnet 5's introductory rate is deliberately NOT used so projections stay an upper bound. OpenAI prices are the public standard per-MTok rates as of July 2026 (see reports/phase-6-providers.md for the fetched source): GPT-5.6 Sol $5/$30, Terra $2.50/$15, Luna $1/$6; GPT-5.5 $5/$30; GPT-5.4 $2.50/$15. gpt-5.4-mini is NOT in the published table (which lists a Nano tier at $0.20/$1.25), so its price here is a deliberately conservative UPPER-bound estimate flagged price_estimate:true. `released` is best-effort (spec RECEIPT.md open question #4). `tier` drives reporting; `judge_eligible` encodes the fixed-judge policy (docs/judge-policy.html) — only the cheap Haiku judge is eligible, and NO OpenAI model is judge-eligible (the judge stays claude-haiku across providers so a cross-substrate report varies only the target model). `provider` is the two-axis provider; per-provider surface/base_url config is under `providers`. The release trigger (scripts/release-watch.js) appends auto-discovered ids here with auto_added:true.",
+  "_comment": "Driftproof model registry. The runner resolves --models ids against this list; an unknown id still runs but its receipt is marked registry:\"unregistered\" and its cost is estimated with the conservative default price in lib/models.js. Prices are STANDARD first-party USD per 1,000,000 tokens (input/output). Anthropic prices are from platform.claude.com (July 2026); Sonnet 5's introductory rate is deliberately NOT used so projections stay an upper bound. OpenAI prices are the public standard per-MTok rates as of July 2026 (see reports/phase-6-providers.md for the fetched source): GPT-5.6 Sol $5/$30, Terra $2.50/$15, Luna $1/$6; GPT-5.5 $5/$30; GPT-5.4 $2.50/$15. gpt-5.4-mini is NOT in the published table (which lists a Nano tier at $0.20/$1.25), so its price here is a deliberately conservative UPPER-bound estimate flagged price_estimate:true. `released` is best-effort (spec RECEIPT.md open question #4). `tier` drives reporting; `judge_eligible` encodes the fixed-judge policy (docs/judge-policy.html) — only the cheap Haiku judge is eligible, and NO OpenAI model is judge-eligible (the judge stays claude-haiku across providers so a cross-substrate report varies only the target model). `provider` is the two-axis provider; per-provider surface/base_url config is under `providers`. The release trigger (scripts/release-watch.js) appends auto-discovered ids here with auto_added:true. Three optional per-row annotations were added by spec-019b (2026-09-01) and are read by NO code path in lib/: `cache_read_price` is the documented cache-read rate per MTok, recorded for provenance and NOT consumed -- lib/cost.js prices input and output only, and the 019b gate asserts that it stays that way, so wiring caching into the projection turns that assertion red rather than leaving this note quietly false; `lifecycle: legacy` records that the docs moved an id to the Legacy list, and never means removed (claude-fable-5 is referenced by receipts/report-007 and receipts/report-007-rerun); `surface_verified` records that a live call reached the id on that surface on `surface_verified_at`, which is an OBSERVATION, not a routing setting -- the surface is chosen at call time by lib/provider.js from CLAUDE_PROVIDER. claude-fable-5-1 prices are sourced to specs/019b-fable-5-1-registration/evidence/docs-pricing-snapshot-2026-09-01.json.",
   "registry_version": "1.1",
   "provider": "anthropic",
   "providers": {
@@ -8,7 +8,8 @@
   },
   "default_price_note": "Unregistered ids are costed at the most expensive known tier (Fable, $10/$50) — a budget guard must never under-estimate. See lib/models.js DEFAULT_PRICE.",
   "models": [
-    { "id": "claude-fable-5",              "family": "fable",  "provider": "anthropic", "released": "2026-06-09", "input_price": 10.0, "output_price": 50.0, "tier": "frontier", "judge_eligible": false },
+    { "id": "claude-fable-5-1",            "family": "fable",  "provider": "anthropic", "released": "2026-09-01", "input_price": 10.0, "output_price": 50.0, "cache_read_price": 0.25, "tier": "frontier", "judge_eligible": false, "surface_verified": "claude-cli", "surface_verified_at": "2026-09-01" },
+    { "id": "claude-fable-5",              "family": "fable",  "provider": "anthropic", "released": "2026-06-09", "input_price": 10.0, "output_price": 50.0, "tier": "frontier", "judge_eligible": false, "lifecycle": "legacy" },
     { "id": "claude-opus-5",               "family": "opus",   "provider": "anthropic", "released": null,         "input_price": 5.0,  "output_price": 25.0, "tier": "frontier", "judge_eligible": false },
     { "id": "claude-opus-4-8",             "family": "opus",   "provider": "anthropic", "released": null,         "input_price": 5.0,  "output_price": 25.0, "tier": "frontier", "judge_eligible": false },
     { "id": "claude-opus-4-7",             "family": "opus",   "provider": "anthropic", "released": null,         "input_price": 5.0,  "output_price": 25.0, "tier": "frontier", "judge_eligible": false },

package/config.js

@@ -9,7 +9,7 @@ const PROJECT_NAME = 'driftproof';
 // Bumped whenever the runner's behaviour or receipt-generation semantics change
 // in a way that could affect results. Recorded into every receipt as
 // run.runner_version so a receipt is reproducible against a known engine.
-const RUNNER_VERSION = '0.7.2';
+const RUNNER_VERSION = '0.8.0';
 
 // The eval format we CONSUME (we deliberately do not invent our own).
 const SUITE_FORMAT = 'agentskills.io/evals';

package/package.json

@@ -1,14 +1,16 @@
 {
   "name": "driftproof",
-  "version": "0.7.2",
+  "version": "0.8.0",
   "description": "A dated proof that this skill, this hash, this model, still helps: run a skill's eval suite with and without the skill across model versions, emit hash-verified dated receipts, and diff receipts into drift reports.",
   "license": "Apache-2.0",
   "keywords": [
     "agent-skills",
-    "evals",
-    "llm",
+    "skill-md",
+    "skill-eval",
+    "llm-evaluation",
+    "claude-code",
     "drift",
-    "verification"
+    "receipts"
   ],
   "homepage": "https://driftproofhq.com",
   "repository": {
@@ -41,5 +43,8 @@
   },
   "engines": {
     "node": ">=22"
+  },
+  "devDependencies": {
+    "html-validate": "^11.11.0"
   }
 }