driftproof 0.11.2 → 0.12.0

package/README.md

@@ -13,16 +13,16 @@ there were too few draws to tell, the receipt says so.
  — live badge for the bundled `commit-message-conventions` example, generated from its own receipt.
 
 **[Quickstart](https://driftproofhq.com/#quickstart)** ·
-**[Latest report](https://driftproofhq.com/reports/010/)** ·
+**[Latest report](https://driftproofhq.com/reports/011/)** ·
 [driftproofhq.com](https://driftproofhq.com)
 
 Driftproof **consumes** the [`agentskills.io/evals`](https://agentskills.io) eval
 format; it does not invent its own.
 
-📊 **Ten published reports** (each re-derived from committed files, nothing
+📊 **Eleven published reports** (each re-derived from committed files, nothing
 hand-entered: Driftproof's receipts, or for Report #010 the upstream harness's own
 output), spanning seven published report types, the newest being instrument
-comparison. The nine that measure with Driftproof read its own arms by one
+comparison. The ten that measure with Driftproof read its own arms by one
 band-based, floor-gated verdict rule and differ in what moves underneath the
 skill — or, in the value report, in which axes are measured; or, in the instrument re-measurement, in the
 instrument itself; or, in the instrument comparison, in which instrument measures:
@@ -92,11 +92,21 @@ instrument itself; or, in the instrument comparison, in which instrument measure
   and 2 failed, both failures on one expectation the skill's ADR instructions do
   not state. No Driftproof measurement was taken and no Driftproof judge ran, and
   the report does not isolate what caused the variation.
+- **[Report #011](https://driftproofhq.com/reports/011/)** — *release drift*:
+  Claude Opus 5.5 on its release day, measured on Report #009's three skills and
+  cases, one case each, beside a fresh Claude Opus 5 arm on the same Claude Code
+  version. Read by the runner's own comparison of the with-skill arms, the three
+  skills read 1 with no separation detected and 2 with not enough draws to conclude
+  at the effect floor, which is not evidence that nothing changed. A second table
+  sets Report #009's own Claude Opus 5 receipts beside this run's; between them the
+  Claude Code version, its host build and the date all changed. Every draw was
+  judged with `claude-opus-5`, which departs from the judge policy, so its figures
+  are not comparable with Reports #001 to #008.
 
 ✍️ The launch essay, **[Three model releases later: what actually happens to agent
-skills](https://driftproofhq.com/writing/three-releases/)**, reads all ten reports
+skills](https://driftproofhq.com/writing/three-releases/)**, reads all eleven reports
 together: what moves underneath a skill, what the skill costs to run, and what a
-corrected instrument did to three published results. Revised 2026-09-22; every
+corrected instrument did to three published results. Revised 2026-09-23; every
 figure in it is gate-checked against the report page it cites.
 
 ## Why
@@ -229,6 +239,7 @@ npx driftproof badge receipt.json --out badges/my-skill.json
 # epistemics — no fabricated hashes, excluded from drift verdicts), and emit
 # the minimal stable summary other tools can consume. See docs/interop.md.
 npx driftproof import results.json --from agent-skills-eval   # or: skillgrade
+npx driftproof import evals/results/<timestamp>/ --from claude-plugin-eval   # or: skill-creator
 npx driftproof export receipt.json --to summary-json
 ```
 
@@ -245,7 +256,9 @@ A skill directory is expected to look like:
 my-skill/
   SKILL.md              # the skill instructions (required)
   evals/evals.json      # agentskills.io/evals suite (required)
-  .driftproofrc         # optional per-project run defaults (models, samples, max_usd)
+  .driftproofrc         # optional run defaults (models, max_usd); samples, max_cases
+                        # and judge_model are read only from the working directory's rc
+                        # (the GitHub Action reads no working-directory rc at all)
   ...                   # any bundled files (contribute to content_hash)
 ```
 
@@ -312,7 +325,7 @@ A receipt is the unit of evidence — one JSON document conforming to
 
 ```jsonc
 {
-  "schema_version": "0.7",
+  "schema_version": "0.9",
   "skill":  { "name": "commit-message-conventions", "version": "0.2.0",
               "content_hash": "…sha256 over SKILL.md + bundled files…" },
   "suite":  { "format": "agentskills.io/evals", "suite_hash": "…", "case_count": 10 },
@@ -321,7 +334,7 @@ A receipt is the unit of evidence — one JSON document conforming to
     "model_release_date": "2025-10-01",
     "provider": "anthropic",
     "surface": "claude-cli",
-    "runner_version": "0.11.2",
+    "runner_version": "0.12.0",
     "date_utc": "2026-07-27T…Z",
     "registry": "registered",
     "transcripts": "hashes-only",
@@ -400,7 +413,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: driftproofhq/driftproof@v0.11.2
+      - uses: driftproofhq/driftproof@v0.12.0
         with:
           skill-dir: skills/my-skill
           models: claude-haiku-4-5
@@ -416,6 +429,13 @@ model, uploads the whole receipt directory as a build artifact, and renders the
 run on three surfaces: the **badge**, a **job summary** carrying one row per
 requested model, and the **check title** of the enforcement step.
 
+**The checkout's `.driftproofrc` is not read.** The action starts the run in an
+empty working directory, so no key of a `.driftproofrc` at the repository root
+is read, and a skill directory's `.driftproofrc` may not set `max_cases`,
+`samples` or `judge_model`. Both files are pull-request content, and a pull
+request may not narrow the run that measures it. The run takes its models and
+caps from the inputs above.
+
 **The decision is taken over every receipt the run produced.** Each requested
 model gets one decision state, and the run's `verdict` is the **worst** of them —
 worst first, in this order:
@@ -448,9 +468,15 @@ the skill hurt — but they **never render as success** on any of the three
 surfaces: not in the badge, not in the summary row, and not in the check title,
 which carries a `::warning` naming the state and the models it came from.
 
+A receipt set that does not say one thing is `REFUSED` as well. Two receipts for
+one requested model, or a receipt with two rows for one case and arm, fail the job
+naming the files or the case, because a decision that depends on which one was read
+is not a pass. Each invocation of the Action writes to its own directory and uploads
+its own artifact, so a job may run it more than once, one skill per step.
+
 Step outputs: `verdict`, `delta` (of the model the worst decision came from),
-`worst_state`, `regressed_models`, `missing_models` and `receipts_dir`. Each is
-described in [`action.yml`](action.yml).
+`worst_state`, `regressed_models`, `missing_models`, `receipts_dir` and
+`artifact_name`. Each is described in [`action.yml`](action.yml).
 
 For a free CI dry-run with **zero model calls**, set `DRIFTPROOF_STUB=1` in the job
 env — the runner returns canned receipts so the wiring can be tested without spend
@@ -473,6 +499,47 @@ surface. The CLI has carried its own input contract at its own door since
 `npx driftproof` refuses a malformed cap or model id before it projects a run.
 Neither statement covers the other. Said of the Action, of CI, or of the runner, "hostile input is refused" is only ever true of the one surface it was measured on.
 
+### Scheduled stale check
+
+`driftproof stale` says whether each receipt's conclusion still stands under the model, harness,
+skill, suite and judge that would run today. The staleness check runs it on a schedule in your
+repository. While any receipt needs a rerun or a regrade, one issue labelled `driftproof-stale` lists
+each one, what moved, and the command to run next. Later runs update that issue, and the first run
+that finds everything current closes it. It makes no model call and needs no API key.
+
+Copy [`examples/workflows/driftproof-stale.yml`](examples/workflows/driftproof-stale.yml) into
+`.github/workflows/`. It runs weekly and on demand, with these permissions and no others:
+
+```yaml
+permissions:
+  contents: read
+  issues: write
+# ...
+      - uses: driftproofhq/driftproof/stale@v0.12.0
+        with:
+          receipts: receipts/**/*.json
+          skill: skills/my-skill
+```
+
+| Input | Default | What it does |
+|---|---|---|
+| `receipts` | `receipts/**/*.json` | Glob of receipt files, one per line for several. |
+| `skill` | none | The skill directory as it is today. Without it, the skill axis reads unknown. |
+| `suite` | the skill's `evals/evals.json` | The eval suite file. |
+| `model`, `judge` | `.driftproofrc` | The model and the judge that would run today. |
+| `harness-version` | `latest` | A Claude Code version, `latest` (read from npm), or `none` (not checked). |
+| `strict` | `false` | Count an unknown axis or an advisory as stale. |
+| `fail-on-stale` | `false` | Fail the job while anything is stale. |
+| `open-issue` | `true` | Keep the issue. |
+| `issue-label` | `driftproof-stale` | Give each check its own label to keep separate issues. |
+| `github-token` | the workflow's token | Used for the issue only. |
+
+The job fails on an error, such as a receipt that does not validate, and on stale only when
+`fail-on-stale` is `'true'`. A patch release of the harness alone (2.1.280 to 2.1.281) is an
+advisory: the job summary reports it and nothing fails. An axis that cannot be known is reported as
+unknown, never as current. Outputs: `result` (`current`, `advisory`, `stale` or `error`),
+`stale-count`, `issue-number` and `report-dir`.
+
 ### Badge
 
 `driftproof badge <receipt>` emits a [shields.io endpoint](https://shields.io/badges/endpoint-badge)
@@ -495,7 +562,7 @@ site, so it reflects a real dated run, not a hand-set color.
 
 ## Reports
 
-Ten reports are published, spanning seven report types. A report page lives at a
+Eleven reports are published, spanning seven report types. A report page lives at a
 draft path — `docs/reports/NNN-draft/` — until the publish sequence renames it, and
 `scripts/build-public.sh` excludes every `*-draft/` path from the published tree
 (see the roll at the top of this README, and