webmcp-gauge 0.1.0 → 0.1.1

package/README.md

@@ -2,7 +2,7 @@
 
 Measures whether an AI agent actually calls the tools your web page exposes through WebMCP.
 
-**Status: measuring, and the metric discriminates.** The harness runs end to end and has produced real numbers against a live page. It launches its own browser per session — a cold profile seeded with the WebMCP flag, in headless Chrome — captures the settled tool manifest, asks a judge model which tool to call, classifies the choice, executes it, and classifies the result. The utterance set is frozen at `1.3.0` (140 utterances plus 20 negative controls), and sixteen runs totalling **4,500 trials** are published under [`reports/`](reports/) — the best isolated reference run being 960 trials across three separate processes, browsers and cold profiles, and a second client joining them: the same page, set and judge through Edge `153.0.4234.13` reads **99.3% (139/140)** with every tool inside or above its Chrome interval.
+**Status: measuring, and the metric discriminates.** The harness runs end to end and has produced real numbers against a live page. It launches its own browser per session — a cold profile seeded with the WebMCP flag, in headless Chrome — captures the settled tool manifest, asks a judge model which tool to call, classifies the choice, executes it, and classifies the result. The utterance set is frozen at `1.3.0` (140 utterances plus 20 negative controls), and twenty-three runs totalling **5,220 trials** are published under [`reports/`](reports/) — the best isolated reference run being 960 trials across three separate processes, browsers and cold profiles, a second client joining them: the same page, set and judge through Edge `153.0.4234.13` reads **99.3% (139/140)** with every tool inside or above its Chrome interval, and a third client at a second judge family: Brave `154.1.96.59` reads **100% (140/140)** under `claude-opus-4-8`, identical to its paired Chrome arm.
 
 The question that mattered most has an answer. Every early number came from a page chosen for being well described, so the same frozen utterance set was fired at a deliberately mis-described twin of that page: one implementation, one dataset, several manifests, **1,320 trials at three isolated sessions per arm**. Clean manifest **99.3%**; degraded manifest **83.1%** overall, with `sum_by_category` at **60.0%** [47.4, 71.4] and `top_expenses` at **26.7%** [17.1, 39.0] against a clean 95.0% and 100%. Between-session σ is at most **0.094** where the effects are 0.35 and larger, so the gap is not noise.
 
@@ -10,11 +10,11 @@ Four ablations, each the clean manifest plus exactly one defect, then answered *
 
 Re-running one arm with its sessions **9 to 17 hours apart instead of minutes** then asked whether any of this survives a clock. The reproducibility figures do: between-session σ came back 0.085 worst-case against 0.062 back-to-back. Point estimates did not sit still — both mid-range tools declined monotonically across the 26-hour span, 8 to 12 points with overlapping intervals — so a single arm measured at one time is sound, while comparing a page against itself across days inherits a drift question. [`spacing-2026-09-01.md`](reports/spacing-2026-09-01.md).
 
-What exists: `lint` (static manifest rules, no judge or key), `trial` (one utterance, one outcome), `run` (S isolated sessions × R repeats, Wilson intervals, control false-positive rate, stamped JSON and Markdown reports, plus a badge that refuses to report a rate it cannot stand behind), JSONL checkpointing with `--resume`, a CI gate with split exit codes where a crash is *"could not measure"* rather than a failing page, a [GitHub Action](action.yml) wrapping both modes, cohort-capture tooling that records what a builder shipped and what an agent can actually call as two separate numbers, private per-project scorecards, a served fixture page, a browser-side tool view so a client that drops a tool can be told from a page that never registered one, and a pre-push visibility gate with its own tested rules. **229 tests.** The findings live in the published report, [`reports/public-report-draft.md`](reports/public-report-draft.md): the invocation-rate result on two builds, the compatibility matrix, and the complete WebMCP Challenge census — **80.8% of reachable submissions register WebMCP tools, and 43.9% of pages carry tools an agent can call that no page surface lists**.
+What exists: `lint` (static manifest rules, no judge or key), `trial` (one utterance, one outcome), `run` (S isolated sessions × R repeats, Wilson intervals, control false-positive rate, stamped JSON and Markdown reports, plus a badge that refuses to report a rate it cannot stand behind), JSONL checkpointing with `--resume`, a CI gate with split exit codes where a crash is *"could not measure"* rather than a failing page, a [GitHub Action](action.yml) wrapping both modes, cohort-capture tooling that records what a builder shipped and what an agent can actually call as two separate numbers, private per-project scorecards, a served fixture page, a browser-side tool view so a client that drops a tool can be told from a page that never registered one, and a pre-push visibility gate with its own tested rules. **229 tests.** The findings live in the published report, [`reports/public-report-draft.md`](reports/public-report-draft.md): the invocation-rate result on three builds, the compatibility matrix, and the complete WebMCP Challenge census — **80.8% of reachable submissions register WebMCP tools, and 43.9% of pages carry tools an agent can call that no page surface lists**.
 
 What does not: **a measurement inside a real assistant.** The *browser* in the ChatGPT desktop app is CDP-drivable and its WebMCP surface is documented here — but that assistant reads pages through a Chrome extension bridge into the user's ordinary Chrome, not through the app's own browser, so no agent-side invocation has been observed. Every number below is a judge-model stand-in for a real client, and whether the stand-in predicts it is still an open assumption.
 
-Findings have gone back to the spec: two comments on [`webmcp#227`](https://github.com/webmachinelearning/webmcp/issues/227#issuecomment-5499568493) establish what Chrome 152 actually does across a frame boundary — a same-origin embed's tools join the host's manifest, a cross-origin embed is blocked by a Permissions Policy feature named `tools`, and once that is delegated the browser offers an agent a tool **no page can list**. The full compatibility record, including the cross-site case where even a host-attached browser-side view loses the embed entirely, is [`reports/compatibility-matrix.md`](reports/compatibility-matrix.md): 22 measured behaviours across four builds, each cell dated and traced to a re-runnable probe, with a standing Corrections section for the one finding that turned out to be this project's own bug.
+Findings have gone back to the spec: two comments on [`webmcp#227`](https://github.com/webmachinelearning/webmcp/issues/227#issuecomment-5499568493) establish what Chrome 152 actually does across a frame boundary — a same-origin embed's tools join the host's manifest, a cross-origin embed is blocked by a Permissions Policy feature named `tools`, and once that is delegated the browser offers an agent a tool **no page can list**. The full compatibility record, including the cross-site case where even a host-attached browser-side view loses the embed entirely, is [`reports/compatibility-matrix.md`](reports/compatibility-matrix.md): 22 measured behaviours across five builds, each cell dated and traced to a re-runnable probe, with a standing Corrections section for the one finding that turned out to be this project's own bug.
 
 ## The problem
 
@@ -32,7 +32,7 @@ Every trial lands in exactly one bucket — `not_supported`, `not_registered`, `
 
 - **A static linter** — no browser needed to reason, no model, no API key. Fourteen rules across four families: invalid or colliding tool names, missing, thin, duplicate or near-duplicate descriptions, **a pair of close-named tools whose descriptions neither of them distinguishes** (adopted 2026-09-05 on measurement, after five arms showed that pattern costs 45 points while no similarity threshold catches it), over-parameterised and under-documented schemas, and tool counts approaching a per-page budget that is still unmeasured (the reported 296-tool figure does **not** reproduce on Chrome 152, where 507 registered tools were all accepted and surfaced). Thresholds are calibrated so the reference page — the one measured at 100% over 960 trials — lints clean, because a default that flags a manifest known to work is a broken default. *Built.*
 - **The harness** — drives real browsers over the Chrome DevTools Protocol, fires the utterance set at the page's registered tools, classifies every outcome, and emits a JSON report plus a CI gate. Every trial reads the tool set **twice** — the page's own `getTools()` and the browser's `WebMCP.toolsAdded` stream — because "the page never registered it" and "the client dropped it" are indistinguishable from inside the page. *Built.*
-- **A public dataset** — the cross-client compatibility record and invocation-rate corpus, regenerated as browsers change, published with the code that produced every number. *Thirteen runs so far, in `reports/` — including the four-build compatibility matrix and a second client for the rate.*
+- **A public dataset** — the cross-client compatibility record and invocation-rate corpus, regenerated as browsers change, published with the code that produced every number. *Sixteen runs and the complete WebMCP Challenge census so far, in `reports/` — including the four-build compatibility matrix, a second client for the rate, and the census with 80.8% adoption across 2,183 captured pages.*
 
 ## How a number is reported
 
@@ -45,17 +45,54 @@ Conflating those two was a real defect in this project's first two sweeps: a too
 
 ## Usage
 
+**Quick start, zero install, no API key:**
+
+```bash
+npx webmcp-gauge lint --url https://your-page.example
+```
+
+**Requirements:**
+
+| Mode | Node | Browser | Judge key |
+|---|---|---|---|
+| `lint --manifest <file>` | ≥ 22 (24 tested) | not needed | **not needed** |
+| `lint --url <page>` | ≥ 22 (24 tested) | yes — launches its own Chrome | **not needed** |
+| `trial` / `run` | ≥ 22 (24 tested) | yes | **yes** — any OpenAI-compatible endpoint |
+
+Install once with `npm i -g webmcp-gauge`, or from a clone use `npm run gauge …` / `node bin/webmcp-gauge.mjs …` — every form runs the same CLI:
+
 ```
 webmcp-gauge lint --url https://example.com
+webmcp-gauge lint --manifest page-tools.json
 webmcp-gauge trial --utterance sum_by_category-05 --judge <model> --base-url <endpoint>
 webmcp-gauge run --sessions 3 --repeats 2 --out artifacts/run --judge <model> --base-url <endpoint>
 webmcp-gauge run --sessions 1 --fail-under 0.9 --judge <model> --base-url <endpoint>
 webmcp-gauge run --serve fixtures/broken --url "twin.html?variant=degraded" --subject "twin" ...
 ```
 
-`lint` needs no judge and no key — it reads the page's manifest and applies static rules. Everything else calls a judge model, which must not be the model that wrote the utterances: the frozen set records which one did, and the CLI refuses to run if they match. Credentials come from the environment; see [`.env.example`](.env.example).
+Every flag is documented in `--help`. A `run` writes four files into `--out`: `report.md` for reading, `report.json` (the machine record with per-tool outcome counts and coverage), `badge.json` + `badge.svg`, and a JSONL checkpoint that `--resume` continues. The cohort capture and harvest tools live in [`probes/`](probes/) in this repository, not in the npm package — they are one-day workflows, documented in their own READMEs.
 
-Install once from npm and every command runs as `webmcp-gauge …`; from a clone, `npm run gauge …` or `node bin/webmcp-gauge.mjs …` runs the same thing.
+**The API key, only where it is actually needed.** `lint` reads the manifest and applies static rules — no model anywhere in the path, so it runs on every push with no secrets. `trial` and `run` measure whether an agent-model *chooses* the tool, so they need a judge: any OpenAI-compatible endpoint (hosted or local — Ollama and LM Studio both speak the protocol), costing one model call per trial. An **anthropic-messages endpoint works too** — pass `--judge-shape anthropic` (or `WEBMCP_GAUGE_JUDGE_SHAPE=anthropic`); the shape is the endpoint's and the CLI will not guess it from the hostname. The CLI refuses to run if the judge is the model that authored the frozen utterance set; the set records which one that was.
+
+Credentials come from the environment:
+
+| Variable | Purpose |
+|---|---|
+| `WEBMCP_GAUGE_JUDGE_API_KEY` | judge key (run/trial only) |
+| `WEBMCP_GAUGE_JUDGE_BASE_URL` | judge endpoint (OpenAI-compatible by default) |
+| `WEBMCP_GAUGE_JUDGE_MODEL` | judge model id, stamped into every report |
+| `WEBMCP_GAUGE_JUDGE_SHAPE` | `anthropic` for an anthropic-messages endpoint (default: `openai`) |
+| `WEBMCP_GAUGE_CHROME` | browser override (default: find Chrome; any Chromium works) |
+| `CDP_PORT` | attach to a browser you started instead of launching one |
+| `WEBMCP_GAUGE_CHROME_LOG` | `1` passes Chrome's own stderr through, for launch debugging |
+
+**Nothing loads `.env` automatically** — the CLI reads the environment. Export the variables, or run through the env file directly:
+
+```bash
+node --env-file=.env bin/webmcp-gauge.mjs run --sessions 1 --judge "$WEBMCP_GAUGE_JUDGE_MODEL" ...
+```
+
+See [`.env.example`](.env.example) for the annotated list.
 
 `--serve <dir>` publishes a directory on 127.0.0.1 and resolves `--url` against it, which is how the deliberately mis-described fixture page in [`fixtures/broken/`](fixtures/broken/) gets measured with the same frozen utterance set as the reference page.
 
@@ -69,6 +106,8 @@ A gate is only useful if `1` means one thing, so the three cases are separated:
 | `1` | Every planned trial was measured, and a rate is below `--fail-under` — the page regressed |
 | `2` | The run cannot answer: planned trials have no measurement (re-run with `--resume`), or the arguments were unusable |
 
+In `lint` mode the same codes mean: `0` no findings at or above `--fail-on`, `1` findings at that level, `2` the manifest could not be read.
+
 Incomplete outranks a breach on purpose. Gaps are not random — a judge outage or a page that never loaded can take out one tool's utterances and nothing else — so a rate over a run with holes is a rate over a denominator the run did not choose, and reporting that as a regression would be a lie with a plausible number. Completeness is derived from the plan against the checkpoint, not from the failure log, because a session killed mid-plan logs nothing.
 
 The threshold is compared against the **point rate**, not the Wilson lower bound: 20 of 20 has a lower bound of 83.9%, so gating on the bound would fail a flawless page on sample size alone. The interval is printed beside the rate instead, and the verdict says so when a breach sits inside it.
@@ -87,22 +126,23 @@ A badge is a bare number in a coloured pill — the exact thing this project ref
 
 ### GitHub Action
 
-[`action.yml`](action.yml) wraps both modes. `lint` needs no browser flag, no judge and no key, so it can run on every push; `run` spends a model call per trial and belongs on a schedule or a manual dispatch.
+[`action.yml`](action.yml) wraps both modes and is [on the GitHub Marketplace](https://github.com/marketplace/actions/webmcp-gauge) — pin `@v1` for a stable interface. `lint` needs no browser flag, no judge and no key, so it can run on every push; `run` spends a model call per trial and belongs on a schedule or a manual dispatch.
 
 ```yaml
-- uses: Svishwa2004/webmcp-gauge@main
+- uses: Svishwa2004/webmcp-gauge@v1
   with:
     mode: lint
     url: https://your-page.example
     fail-on: error
 
-- uses: Svishwa2004/webmcp-gauge@main
+- uses: Svishwa2004/webmcp-gauge@v1
   with:
     mode: run
     url: https://your-page.example
     fail-under: '0.9'
     judge: ${{ vars.JUDGE_MODEL }}
     base-url: ${{ vars.JUDGE_BASE_URL }}
+    # judge-shape: anthropic   # only for an anthropic-messages endpoint; default openai
   env:
     WEBMCP_GAUGE_JUDGE_API_KEY: ${{ secrets.JUDGE_API_KEY }}
 ```

package/action.yml

@@ -1,7 +1,7 @@
 name: webmcp-gauge
 description: >-
-  Lint a page's WebMCP manifest, or measure whether an agent actually invokes its
-  tools, and fail the build when it regresses. Emits a badge either way.
+  Lint a page's WebMCP manifest, or measure whether an agent actually invokes
+  its tools, and fail on a regression.
 author: Sahan Vishwa
 
 # Two modes, because they cost different things and mean different things.
@@ -54,7 +54,11 @@ inputs:
     required: false
     default: ''
   base-url:
-    description: 'run mode: OpenAI-compatible endpoint for the judge'
+    description: 'run mode: judge endpoint. OpenAI-compatible by default; set judge-shape for an anthropic-messages endpoint'
+    required: false
+    default: ''
+  judge-shape:
+    description: 'run mode: judge wire protocol, openai (default) or anthropic — the shape is the endpoint''s'
     required: false
     default: ''
   badge-label:
@@ -131,6 +135,7 @@ runs:
           [ -n "${{ inputs.fail-under }}" ] && args+=(--fail-under "${{ inputs.fail-under }}")
           [ -n "${{ inputs.judge }}" ] && args+=(--judge "${{ inputs.judge }}")
           [ -n "${{ inputs.base-url }}" ] && args+=(--base-url "${{ inputs.base-url }}")
+          [ -n "${{ inputs.judge-shape }}" ] && args+=(--judge-shape "${{ inputs.judge-shape }}")
         fi
 
         echo "webmcp-gauge ${args[*]}"

package/bin/webmcp-gauge.mjs

@@ -12,6 +12,7 @@ import { lintManifest, lintToText } from '../core/lint.mjs';
 import { runSessions } from '../core/orchestrate.mjs';
 import { buildPlan, readCheckpoint, readFailures, runSessionSweep, trialKey } from '../core/sweep.mjs';
 import { runTrial } from '../core/trial.mjs';
+import { createJudge as createAnthropicJudge } from '../judges/anthropic-messages.mjs';
 import { createJudge } from '../judges/openai-compatible.mjs';
 import { buildReport, toMarkdown } from '../report/emit.mjs';
 import { buildBadge, renderBadgeSvg } from '../report/badge.mjs';
@@ -39,6 +40,9 @@ error, never a silently ignored default):
                        measurement can run against a fixture page in this repo
   --judge <model>      judge model id (env WEBMCP_GAUGE_JUDGE_MODEL)
   --base-url <url>     judge endpoint (env WEBMCP_GAUGE_JUDGE_BASE_URL)
+  --judge-shape <api>  judge wire protocol: openai (default) or anthropic — the
+                       shape is the endpoint's, and guessing it from the hostname
+                       would be a silent default (env WEBMCP_GAUGE_JUDGE_SHAPE)
   --port <n>           attach to an existing Chrome instead of launching one
 
 lint options (no judge required):
@@ -102,7 +106,7 @@ which one that was. See docs/getting-started.md and .env.example.`;
  */
 const CLI_OPTIONS = {
   values: [
-    'fixture', 'url', 'serve', 'judge', 'base-url', 'port',
+    'fixture', 'url', 'serve', 'judge', 'base-url', 'judge-shape', 'port',
     'manifest', 'variant', 'fail-on', 'min-description', 'max-properties', 'budget-warn',
     'tool', 'utterance',
     'sessions', 'repeats', 'concurrency', 'gap', 'tools', 'out', 'subject', 'fail-under', 'badge-label',
@@ -178,6 +182,17 @@ const judgeModel =
 const judgeBaseUrl =
   (typeof flags['base-url'] === 'string' && flags['base-url']) ||
   process.env.WEBMCP_GAUGE_JUDGE_BASE_URL;
+const judgeShape =
+  (typeof flags['judge-shape'] === 'string' && flags['judge-shape']) ||
+  process.env.WEBMCP_GAUGE_JUDGE_SHAPE ||
+  'openai';
+if (judgeShape !== 'openai' && judgeShape !== 'anthropic') {
+  fail(`--judge-shape must be "openai" or "anthropic", got "${judgeShape}"`);
+}
+const buildJudge = () =>
+  judgeShape === 'anthropic'
+    ? createAnthropicJudge({ baseUrl: judgeBaseUrl, model: judgeModel })
+    : createJudge({ baseUrl: judgeBaseUrl, model: judgeModel });
 // L0 is the free on-ramp: it reads a manifest and calls no model, so demanding a
 // judge for it would put an API key in front of the cheapest useful answer.
 if (needsJudge && (!judgeModel || !judgeBaseUrl)) {
@@ -299,7 +314,7 @@ if (command === 'lint') {
     process.exitCode = blocking > 0 ? EXIT.breach : EXIT.pass;
   }
 } else if (command === 'trial') {
-  const judge = createJudge({ baseUrl: judgeBaseUrl, model: judgeModel });
+  const judge = buildJudge();
   const utteranceId = typeof flags.utterance === 'string' ? flags.utterance : null;
   const toolName =
     typeof flags.tool === 'string' ? flags.tool : utteranceId?.replace(/-\d+$/, '') ?? null;
@@ -348,7 +363,7 @@ if (command === 'lint') {
   }
 } else if (command === 'session') {
   const session = Number(flags.session ?? 1);
-  const judge = createJudge({ baseUrl: judgeBaseUrl, model: judgeModel });
+  const judge = buildJudge();
   const target = await openTarget();
 
   // Each session owns its browser: a cold profile, its own port, its own process
@@ -438,6 +453,8 @@ if (command === 'lint') {
     judgeModel,
     '--base-url',
     judgeBaseUrl,
+    '--judge-shape',
+    judgeShape,
     '--out',
     outDir,
     '--repeats',

package/docs/concept.md

@@ -347,7 +347,7 @@ Verified 2026-08-29. Building any of the left column again would be waste.
 | 3 | Sep 1–3 | 🟡 **Part done, and one part closed unanswered** — Mode B adapters + Mode A↔B correlation study + GitHub Action | **#268 refuted** and the GitHub Action shipped (2026-09-01). Mode B's browser layer is driven; its **agent layer is closed on a recorded negative** (2026-09-02) — the agent reads pages through a Chrome extension bridge into the user's own Chrome, and instrumenting that was declined — so the **correlation study has no result and Gate 3 is unanswered rather than passed** |
 | 4 | by Sep 3, runs Sep 4 | ✅ **Done — staged 2026-09-01, three days early; run complete 2026-09-25/26** — Cohort snapshot script, staged and ready, then run to completion | `core\cohort.mjs` + `core\gallery.mjs` (32 tests), `probes\cohort-snapshot.mjs` + `probes\gallery-harvest.mjs`, rehearsed against three published galleries, three real project pages and a local two-page fixture. The gallery opened Sep 24; all 2,474 submissions were walked (2,185 usable) and **2,183 captured** across three recoverable parts — a browser death and a machine sleep each cost one in-flight target. The census: [`../reports/census-2026-09-26.md`](../reports/census-2026-09-26.md) |
 | 5 | Sep 4–21 | ✅ **Done 2026-09-26** — Private per-builder scorecards; aggregate-only public stats; spec-issue data contributions | **The spec contribution is filed** (2026-09-02, `webmcp#227`, twice — the frame-scope baseline and the cross-origin follow-up). The scorecard **renderer** is built, exercised on 344 real captures, and delivery was decided as **offer on request** — the standing offer is in the published report. The aggregate stats are published: [`../reports/census-2026-09-26.md`](../reports/census-2026-09-26.md) |
-| 6 | Sep 23+ | ✅ **Done 2026-09-26** — Public report + open dataset + harness release | The report is published ([`../reports/public-report-draft.md`](../reports/public-report-draft.md)), the repo went public with `EXPECTED_VISIBILITY=public` landed in the flip commit itself, and the dataset is **aggregate-only by item 8's recorded decision** — the derived tables are the open dataset; the raw JSONL stays local unless revisited. The npm package is prepped and publishes on the maintainer's login |
+| 6 | Sep 23+ | ✅ **Done 2026-09-26** — Public report + open dataset + harness release | The report is published ([`../reports/public-report-draft.md`](../reports/public-report-draft.md)), the repo went public with `EXPECTED_VISIBILITY=public` landed in the flip commit itself, the dataset is **aggregate-only by item 8's recorded decision** (the derived tables are the open dataset; the raw JSONL stays local unless revisited), and the npm package is **published** — `webmcp-gauge@0.1.0`, verified from the registry |
 
 **Milestone 4 is time-critical and cannot be recovered later.** The Devpost gallery was still unpublished as of 2026-08-29 and opens with judging. Those ~165 live demo URLs sit on free hosting tiers; within a few months a large share will 404. A snapshot of that many real `registerTool` implementations, taken on the one day they are all simultaneously live, is not reconstructible after the fact. It is the largest corpus of real-world WebMCP code that will ever exist for this period, and capturing it costs one day of scripting.
 
@@ -402,10 +402,10 @@ Unresolved, and each is answerable with a small spike:
 Ordered, and each independently checkable:
 
 1. A stable, variance-quantified invocation rate for any WebMCP page, in at least three clients. 🟡 **Two measured clients now — both Chromium.** The rate is stable (σ ≤ 0.094 between sessions, ≤ 0.085 at 9–17 h spacing, Chrome 960 trials) and Edge `153.0.4234.13` reproduces it at 160 trials — 99% (139/140), every tool inside or above its Chrome interval ([`reports/airlock-1.3.0-glm-5.3-edge-s1r1.md`](../reports/airlock-1.3.0-glm-5.3-edge-s1r1.md), 2026-09-03) — but that is a point measurement, not a variance one, and both runs share one judge model (`glm-5.3`), so the client coordinate moved and the judge coordinate did not. A third client that is not Chromium is the remaining gap.
-2. A published compatibility matrix that a developer would actually link to when explaining "why does my tool work in Brave but not Chrome?" 🟡 **Assembled 2026-09-03 — [`reports/compatibility-matrix.md`](../reports/compatibility-matrix.md): 20 behaviours across four build columns (Chrome `152.0.7977.65`; the ChatGPT desktop app's Chromium at `151.0.7922.174` and again at `152.0.7977.64` after it updated mid-project; Edge `153.0.4234.13`, where the Chrome activation recipe works unchanged), each cell dated and traced to a re-runnable probe, with a standing Corrections section for the one row that turned out to be our own bug. Publishable as it stands; "published" still waits on the repo going public (criterion 5).** And the honest limit of it: **Brave — one of the two clients the question names — is still unmeasured** (not installed here), and Edge was measured at 153 rather than the 151 that report names, so the matrix answers "why does my tool behave differently in ChatGPT's browser or Edge?" well and the Brave half of the question not at all.
+2. A published compatibility matrix that a developer would actually link to when explaining "why does my tool work in Brave but not Chrome?" 🟡 **Assembled 2026-09-03 — [`reports/compatibility-matrix.md`](../reports/compatibility-matrix.md): 20 behaviours across four build columns (Chrome `152.0.7977.65`; the ChatGPT desktop app's Chromium at `151.0.7922.174` and again at `152.0.7977.64` after it updated mid-project; Edge `153.0.4234.13`, where the Chrome activation recipe works unchanged), each cell dated and traced to a re-runnable probe, with a standing Corrections section for the one row that turned out to be our own bug. Publishable as it stands; "published" still waits on the repo going public (criterion 5).** And the honest limit of it, updated 2026-09-26: **Brave `154.1.96.59` was installed and surface-measured — the matrix now carries a fifth column, and the #227 frame answers reproduce on a third engine — but its invocation-rate sweep is blocked on the judge provider** (agentrouter serves the key no channel for `glm-5.3`), and Edge was measured at 153 rather than the 151 that report names, so the matrix answers "why does my tool behave differently in ChatGPT's browser, Edge or Brave?" at the surface level while the rate comparison still rests on two clients.
 3. At least one reproducible data contribution accepted into a live spec issue. ✅ **Filed 2026-09-02** — two comments on `webmcp#227`, each carrying a build string and a reproduction that was run before it was posted. *Filed* is done; *accepted* is not this project's to decide.
-4. A cohort dataset captured on gallery-publish day and preserved. ⏳ **Tooling staged and rehearsed; the capture waits on the gallery** (expected 2026-09-04).
-5. The linter installed by developers who have never heard of the author. 🟡 **The repo is public (2026-09-26) with `action.yml` live, and the npm package is prepped** — `npx webmcp-gauge lint --url <page>` becomes real the moment `npm publish` runs on the maintainer's login. Criterion 5 is the one this document cannot close by itself: it needs strangers, not commits.
+4. A cohort dataset captured on gallery-publish day and preserved. ✅ **Done 2026-09-25/26** — the gallery opened Sep 24; all 2,474 submissions walked (2,185 usable) and **2,183 captured** across three recoverable parts, reconciled and published as [`../reports/census-2026-09-26.md`](../reports/census-2026-09-26.md). The one-day preservation claim holds: every target was visited exactly once inside the ~13-hour window, with the dead-record composition checked rather than assumed.
+5. The linter installed by developers who have never heard of the author. ✅ **Met 2026-09-26, three ways** — the repo is public with `action.yml` live, **`webmcp-gauge@0.1.0` is on npm** (verified by installing it fresh from the registry and running `npx webmcp-gauge lint --url <page>` against a real page), and **the Action is on the GitHub Marketplace** as `Svishwa2004/webmcp-gauge` (`v1.0.0`, published from a release whose `action.yml` carries the required branding; the first description tripped the Marketplace's 125-character limit and was shortened inside the tag before publishing, since the listing reads `action.yml` at the tag). The moving `v1` tag is the interface promise: fixes move within the major, breaking changes get a `v2`. What this document cannot close by itself is the adoption half: that needs strangers, not commits.
 6. A report that becomes the default citation for "do WebMCP tools actually get called?" 🟡 **Published 2026-09-26 — [`reports/public-report-draft.md`](../reports/public-report-draft.md).** The capture landed (criterion 4) and the repository went public with the report in the same window (criterion 5's on-ramp). Every section is sourced to the write-up that produced it, including Gate 3 stated as unanswered and the complete cohort census. Whether it becomes anyone's default citation is not this project's to decide — see criterion 3's split between *filed* and *accepted*.
 
 Criterion 3 is the cheapest and the most valuable. The WebMCP community is small enough — a few named Chrome engineers, spec editors and library authors — that two or three good field-data contributions to open issues make an unknown solo developer a known one. That is the actual early advantage on offer here, and it does not require the product to succeed commercially. **It is also the one criterion already met, on day five, for the cost of running a test before writing a comment.**

package/docs/getting-started.md

@@ -6,6 +6,8 @@
 
 Verification legend used throughout: ✅ verified on this machine · ⚠️ unverified · ❌ known wrong.
 
+> **Just want to run the tool?** The README's [Usage](../README.md#usage) section is the stranger's path: `npx webmcp-gauge lint --url <page>` needs no install and no API key, and the requirements table there says exactly which modes need a browser and which need a judge key. This document is the author's own machine record — the paths, browsers and credentials below are how *this* project was stood up, kept because the log's verification claims trace to it.
+
 ---
 
 ## 0. What is already in place

package/judges/anthropic-messages.mjs

@@ -0,0 +1,97 @@
+/**
+ * Judge adapter for Anthropic-messages endpoints (POST {base}/v1/messages).
+ *
+ * Same judge-level contract as judges/openai-compatible.mjs — { id, baseUrl,
+ * keySource, complete({system, user, maxTokens}) } — differing only in the wire
+ * shape, so the sweep and trial paths cannot tell the two apart and every
+ * report keeps stamping the same judge fields. Chosen explicitly with
+ * --judge-shape anthropic, because the shape is a property of the endpoint and
+ * guessing it from the hostname would be one more silent default. Just Work
+ * (api.justwoker.icu) serves claude models over this protocol and refuses the
+ * OpenAI shape outright. No retrying and no repair, same rule as the OpenAI
+ * adapter: a judge that cannot follow the response contract is a measurement
+ * result, not an error to paper over.
+ */
+import { resolveApiKey } from './openai-compatible.mjs';
+
+const DEFAULT_TIMEOUT_MS = 60000;
+
+/**
+ * Reasoning budgets sit higher than a tool-choice answer needs, and Anthropic
+ * charges for thinking tokens the same way — the ceiling exists so truncation
+ * is unlikely, and `truncated` says so when it happens anyway.
+ */
+const DEFAULT_MAX_TOKENS = 4096;
+
+export const createJudge = ({
+  baseUrl,
+  model,
+  apiKeyVar,
+  env = process.env,
+  timeoutMs = DEFAULT_TIMEOUT_MS,
+  temperature = 0,
+}) => {
+  if (!baseUrl || !model) throw new Error('createJudge requires baseUrl and model');
+  const { key, source } = resolveApiKey(env, apiKeyVar, baseUrl);
+  const endpoint = `${baseUrl.replace(/\/$/, '')}/v1/messages`;
+
+  return {
+    id: model,
+    baseUrl,
+    keySource: source,
+
+    async complete({ system, user, maxTokens = DEFAULT_MAX_TOKENS }) {
+      const body = {
+        model,
+        temperature,
+        max_tokens: maxTokens,
+        ...(system ? { system } : {}),
+        messages: [{ role: 'user', content: user }],
+      };
+
+      const startedAt = Date.now();
+      const response = await fetch(endpoint, {
+        method: 'POST',
+        headers: {
+          'Content-Type': 'application/json',
+          'x-api-key': key,
+          Authorization: `Bearer ${key}`,
+          'anthropic-version': '2023-06-01',
+        },
+        body: JSON.stringify(body),
+        signal: AbortSignal.timeout(timeoutMs),
+      });
+
+      const text = await response.text();
+      const elapsedMs = Date.now() - startedAt;
+
+      if (!response.ok) {
+        // The body carries the useful part - a quota rejection reads very
+        // differently from a malformed request - so it is surfaced, not swallowed.
+        throw new Error(`judge HTTP ${response.status}: ${text.slice(0, 500)}`);
+      }
+
+      const payload = JSON.parse(text);
+      const content = (payload.content ?? [])
+        .filter((block) => block.type === 'text')
+        .map((block) => block.text)
+        .join('')
+        .trim();
+      const finishReason = payload.stop_reason ?? null;
+
+      return {
+        content,
+        raw: text,
+        model: payload.model ?? model,
+        finishReason,
+        truncated: finishReason === 'max_tokens',
+        // Normalised to the OpenAI field names so a report never carries two
+        // usage shapes depending on which adapter served the trials.
+        usage: payload.usage
+          ? { prompt_tokens: payload.usage.input_tokens, completion_tokens: payload.usage.output_tokens }
+          : null,
+        elapsedMs,
+      };
+    },
+  };
+};

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "webmcp-gauge",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "type": "module",
   "description": "Measures whether an AI agent actually calls the tools a page exposes through WebMCP.",
   "license": "MIT",