readiness-audit 1.0.0 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,54 @@
1
1
  # Changelog
2
2
 
3
+ ## 1.2.0 (2026-10-06)
4
+
5
+ From drifts.bot's switch-over, the third project to adopt the standard. One new warning (D4), no new failures. It also ships 1.1.0, which wasn't published to npm.
6
+
7
+ - **`--base <url>` scores a build before it ships:** a local server or a preview deployment, as the domain. The site's URLs are requested from the base and reported under the domain; a private address is allowed there because you chose it, while everything else the site links to still has to be public. D10 (DNS) and W2's host redirects are skipped.
8
+ - **D4 follows llms.txt's links.** It samples up to 10 of the site's own links and warns about missing pages (404 or 410), server errors and no answer. Endpoints that want a key (401, 403) are fine, and templates like `{id}` are skipped. A warning lowers the required score until fixed, but doesn't change the exit code.
9
+ - **D10 asks public resolvers first** (1.1.1.1, then 8.8.8.8, then the system's), and its failures say that DNS answers stay cached until their TTL runs out.
10
+ - **The status page names the pinned command** (`npx readiness-audit@1 --matrix ...`), as §14 says to run it.
11
+ - **T6 takes structured numbers** in `recorded.json` (period; clicks, impressions, CTR, position, indexed, not indexed, crawled but not indexed; for Google and Bing), and the status page shows them. A free-text `result` still works.
12
+ - **The T5 runner** takes `--attempts`, `--model`, `--max-turns` and `--pause`, runs each attempt in an empty folder with no user settings or MCP servers (and `--bare` when `ANTHROPIC_API_KEY` is set), gives the agent a random test name, and asks for doc mismatches as well as errors.
13
+ - **STANDARD.md:**
14
+ - A9, A10: a limit on key checks answers 429, never 401, and counts only failures; a database error is 503; cache verified keys when the hash is slow; one auth helper for every route.
15
+ - T1, T3: logs outlive a deploy; page through platform logs.
16
+ - T5: keep test accounts off public pages, keep the agent fresh, budget the runs, report doc mismatches.
17
+ - T6: record fields, not prose; it takes a person with console access.
18
+ - §12 gotchas: Next.js drops a `Vary` set in middleware, proxy or `next.config` on App Router pages (tested on 14.2 and 16.3); `robots.ts` can't output `Content-Signal`; Redocly needs `security: []` on public operations; zod-to-openapi v7 with Zod 3 needed `extendZodWithOpenApi`; container logs vanish on deploy.
19
+ - **docs/recipes.md:** one auth helper for every route, and test accounts kept off public pages.
20
+ - **AGENTS.md:** a new warning is a minor version; a new failure is a major one.
21
+
22
+ ## 1.1.0 (2026-10-06, not published to npm; shipped in 1.2.0)
23
+
24
+ From inbed.ai's switch-over, the second project to adopt the standard. Nothing gets stricter.
25
+
26
+ - **D8 (markdown for agents) moves from recommended to next level.** It had no stated reason, our logs can't show `Accept: text/markdown` traffic (Railway's HTTP logs keep the user agent but not `Accept`), and principle 3 says a scanner check alone makes an item recommended at most. It now says why to build it and when (log the `Accept` header to find out), starts with the homepage and docs, and says Cloudflare Pro is optional. The status page gains a "Next level passed" row.
27
+ - **`--compare <file>`** lists what changed since a previous run (a status page from `--matrix`, or a saved `--json` report): better, worse, newly checked, no longer checked, and level changes. A score's denominator grows as items start to apply, so totals alone mislead.
28
+ - **A1 explains two cases.** A field whose description sits inside `anyOf` or `oneOf` (what Zod emits for `.nullable()` when `.describe()` comes first) is still missing a description, and the reason now says so and how to fix it. Operations whose description only repeats the summary are noted on the result line, without failing it.
29
+ - **The Score line carries the date,** so pasted results keep it.
30
+ - **STANDARD.md:**
31
+ - D1: CDN features such as Cloudflare's managed robots.txt prepend their own group.
32
+ - W3: in Next.js, a `loading.tsx` above a detail page makes missing pages answer 200.
33
+ - A1: put descriptions on the field itself, and say when to use each operation.
34
+ - A6: point out unreplaced placeholders such as `{{AGENT_ID}}`.
35
+ - S1 and S2: index the skills the site serves, not every listing; an index makes S1 apply.
36
+ - D10: the AID auth values, with an `a=apikey` example.
37
+ - D5: the template gives llms.txt the type sites actually send (`text/plain`).
38
+ - T5: choose a goal the agent can finish alone.
39
+ - §13: keep the status page in the project's private repo when there is one.
40
+ - **New docs/recipes.md:** a `/.well-known` catch-all for D7 and D11, `GET /api` that sends browsers to the docs, `did_you_mean` by edit distance against the OpenAPI paths, unreplaced placeholders, and markdown routes without a CDN plan.
41
+
42
+ ## 1.0.1 (2026-10-05)
43
+
44
+ Fixes from the first full switch-over (animalhouse.ai, 10 to 19 of 19 required).
45
+
46
+ - **S1 no longer fails skill URLs that don't end in `<name>/SKILL.md`.** It read the folder from the URL's second-to-last segment, so an index pointing at `/skills/<name>` failed every name ("folder is skills"). A name is now checked against its folder only when the URL shows one, and always against its index entry's name.
47
+ - **S1 samples only SKILL.md entries.** It read `archive` entries as text and failed their names and descriptions.
48
+ - **S1 resolves relative skill URLs against the index**, not the site root (RFC 3986).
49
+ - **A CDN-cached copy is named.** When robots.txt, the sitemap, llms.txt or security.txt fails and came from a CDN cache, the reason says how old the copy is, since a recent change may not show yet. Cloudflare caches robots.txt by default.
50
+ - **STANDARD.md:** D1 says to purge robots.txt from the CDN after a change. D8 notes that Next.js 14.2 replaces `Vary` on App Router pages, so `Accept` goes on at the edge.
51
+
3
52
  ## 1.0.0 (2026-10-05)
4
53
 
5
54
  First public release.
package/README.md CHANGED
@@ -18,7 +18,7 @@ npx readiness-audit example.com
18
18
  It needs Node 20 or later and has no dependencies. It only reads: GET requests, plus a few POSTs that create nothing (an MCP `initialize`, `server/discover` and `tools/list`, and an empty POST to `/`). It keeps to about six requests at a time, and it won't follow a link the site supplies (in its sitemap, share images, skills, catalog or DNS record) to a private or local address, at any redirect. It prints a pass, warning or failure for each check, with the reason, and exits with code 1 when a required check fails, so it can gate a deploy.
19
19
 
20
20
  ```
21
- readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json]
21
+ readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json] [--compare previous] [--base url]
22
22
  readiness-audit --matrix <domain> <domain> ... [--recorded results.json]
23
23
  ```
24
24
 
@@ -28,11 +28,15 @@ readiness-audit --matrix <domain> <domain> ... [--recorded results.json]
28
28
  | `--no-mcp` | skip the hosted MCP checks |
29
29
  | `--no-api` | skip the API checks, for a site with no public API |
30
30
  | `--json` | print the results as JSON |
31
+ | `--compare file` | list what changed since a previous run: a status page from `--matrix` or a saved `--json` report |
32
+ | `--base url` | score a local or preview build as the domain before you deploy (`--base http://localhost:3000`); DNS and host redirects are skipped |
31
33
  | `--matrix` | score several sites and print a markdown status page |
32
34
  | `--recorded file` | with `--matrix`, add your recorded T5 and T6 results (format: [examples/recorded.json](examples/recorded.json)) |
33
35
 
34
36
  From a clone, run `node bin/readiness-audit.mjs example.com`.
35
37
 
38
+ To check fixes before they ship, score a local build: `npx readiness-audit@1 example.com --base http://localhost:3000`. The site's own URLs (from its sitemap, llms.txt, skills and catalog) are requested from the base and reported under the domain. A private address is allowed for `--base` because you chose it; anything else the site links to still has to be public.
39
+
36
40
  ## What it checks, and what it can't
37
41
 
38
42
  The scorecard covers what's visible from outside. It checks:
@@ -51,6 +55,8 @@ It can't see whether `next_steps` guide well, whether input is forgiving, whethe
51
55
 
52
56
  Point to [STANDARD.md](STANDARD.md) from your repo instead of copying it (§14 has a short stub), and pin the scorecard to a major version: `npx readiness-audit@1 <domain>`. To run it on a schedule in GitHub Actions, see [examples/github-action.yml](examples/github-action.yml).
53
57
 
58
+ To switch a project over, point its coding agent at [docs/adopting.md](docs/adopting.md): step-by-step instructions, with lessons from the first project that did it.
59
+
54
60
  ## Where it comes from
55
61
 
56
62
  Six projects wrote down what they do for agents and search: [animalhouse.ai](https://animalhouse.ai), [inbed.ai](https://inbed.ai), [magnifica.family](https://magnifica.family), [drifts.bot](https://drifts.bot), [achurch.ai](https://achurch.ai) and [botsmatter.live](https://botsmatter.live). The standard merges their notes and settles where they disagreed, using the specs and production request logs. It was revised after each project's own agent reviewed it.
package/STANDARD.md CHANGED
@@ -121,6 +121,8 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
121
121
  - **Keep the rules the same for every crawler.** A crawler obeys only the group that names it, and falls back to `*` only when no group does (RFC 9309 §2.2.1). A `User-agent: GPTBot` group that lacks a `Disallow` from the `*` group lets that bot crawl what you meant to block. One group listing `*` and every named crawler is simplest (achurch.ai). Repeating the full rules in every group also works (botsmatter.live).
122
122
  - **Disallow what has side effects or private data.** For read-only API endpoints that live-fetch agents need (public stats, a self-documenting `GET /api/auth/register`), allow the fetch and send `X-Robots-Tag: noindex` on the JSON instead, so search engines skip it without blocking agents (botsmatter, inbed).
123
123
  - **Allow your share-card path** (`/api/og/` at animalhouse, `/og/` at botsmatter), and add the `Sitemap:` line.
124
+ - **Purge it from your CDN after a change.** Cloudflare caches robots.txt by default, so crawlers keep the old rules until the cached copy expires. The scorecard says when it read a cached copy.
125
+ - **Watch for CDN features that rewrite it.** Cloudflare's managed robots.txt, and features like it, put their own group at the top of your file, which can break D1 and D15. Leave them off, or check robots.txt after turning one on.
124
126
  - Naming AI crawlers and stating a Content-Signal policy is D15 (recommended).
125
127
  - **Done when** the scorecard passes D1.
126
128
 
@@ -128,7 +130,7 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
128
130
 
129
131
  **D3 Server-rendered pages.** Every page's content is in the HTML the server sends, not loaded later by script, with a real logo through the framework's icon convention and a favicon set (missing icons show up as steady 404s). *Why: crawlers, link-preview bots and agents that fetch pages read the HTML, and many don't run JavaScript.* inbed's chat pages once shipped none of their 48 messages in the HTML. The page-level details (titles, canonicals, structured data, previews, headings, links) are W1 to W7 in [§5](#5-search-people-and-answer-engines).
130
132
 
131
- **D4 `/llms.txt` and `/llms-full.txt`**: a short map (what this is, how to start, links to every other surface) and the whole corpus in one fetch. Generate both and test them against the sitemap. The file is markdown; serve it as `text/markdown` or `text/plain`, and use the same type wherever you link it.
133
+ **D4 `/llms.txt` and `/llms-full.txt`**: a short map (what this is, how to start, links to every other surface) and the whole corpus in one fetch. Generate both and test them against the sitemap: agents follow the links, so every one must open. The scorecard samples them and warns about missing pages (an endpoint that answers 401 for want of a key is fine; templates like `{id}` are skipped). The file is markdown; serve it as `text/markdown` or `text/plain`, and use the same type wherever you link it.
132
134
  - *Why required: traffic, and not from Google.* Google Search ignores llms.txt (it said so in June 2026), but plenty else reads it. At animalhouse, 79 requests in four and a half days came from:
133
135
  - agent directories (AgenstryBot, BrickBlueBot, agentprobe, MCPHarbor, AgentTrustBot);
134
136
  - llms.txt directories and validators;
@@ -159,15 +161,16 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
159
161
 
160
162
  **A1 OpenAPI 3.1 at `/openapi.json`.** *Why: working agents fetch it (curl was the top client in our logs).*
161
163
  - Generated from the same source that validates requests: Zod at animalhouse and inbed, plain JavaScript definitions at botsmatter.
162
- - Every operation has a description saying when and why to call it. A one-line summary isn't enough.
164
+ - Every operation has a description saying when and why to call it. A one-line summary isn't enough, and the scorecard notes descriptions that only repeat the summary. A "When to use" line per operation in your API reference makes a good source, and agents reading the markdown get it too (inbed).
163
165
  - Every parameter and request field is described. Response fields too, once you have response schemas (N7).
164
- - **Done when** the scorecard passes A1 and `npx @redocly/cli lint` passes. Checking the spec against real responses needs response schemas, so that contract test lives with N7.
166
+ - Put each description on the field itself. One inside `anyOf` or `oneOf` counts as missing, because many clients show only the field's own description. Zod puts it there for `.nullable()` when `.describe()` comes first, so call `.describe()` last, after `.optional()` and `.nullable()`.
167
+ - **Done when** the scorecard passes A1 and `npx @redocly/cli lint` passes. Redocly fails public operations that don't declare `security: []` when the spec has a global security scheme. Checking the spec against real responses needs response schemas, so that contract test lives with N7.
165
168
 
166
- **A2 `GET /api`**: a JSON index of every operation, generated from the spec. *Why: agents ask for it (19 requests in four days at animalhouse, curl most often).*
169
+ **A2 `GET /api`**: a JSON index of every operation, generated from the spec. *Why: agents ask for it (19 requests in four days at animalhouse, curl most often).* A browser asking for HTML can be redirected to the docs ([recipe](docs/recipes.md#get-api-json-for-agents-the-docs-for-browsers-a2)).
167
170
 
168
- **A6 Errors that teach**: `next_steps` and a `suggestion` on every response, errors included ([§8](#a6-every-response-teaches)). *Why: working agents rely on it.* The scorecard tests one case: an authenticated operation called without a key must answer JSON that says how to get one.
171
+ **A6 Errors that teach**: `next_steps` and a `suggestion` on every response, errors included ([§8](#a6-every-response-teaches)). *Why: working agents rely on it.* The scorecard tests one case: an authenticated operation called without a key must answer JSON that says how to get one. Point out unreplaced placeholders too: an id like `{{AGENT_ID}}` copied from your docs reaches the real route, not the catch-all, so its not-found answer should say it looks like a placeholder ([recipe](docs/recipes.md#unreplaced-placeholders-in-ids-a6)).
169
172
 
170
- **A7 A JSON catch-all for wrong API paths** with `next_steps`, a `suggestion` or `did_you_mean` ([§8](#forgiving-input-strict-mutations)). *Why: agents guess paths; animalhouse built its catch-all from the 404s in its logs.*
173
+ **A7 A JSON catch-all for wrong API paths** with `next_steps`, a `suggestion` or `did_you_mean` ([§8](#forgiving-input-strict-mutations)). *Why: agents guess paths; animalhouse built its catch-all from the 404s in its logs.* `did_you_mean` can come from edit distance against your OpenAPI paths ([recipe](docs/recipes.md#did_you_mean-from-the-openapi-paths-a7)).
171
174
 
172
175
  ### Recommended
173
176
 
@@ -176,22 +179,14 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
176
179
  ```
177
180
  Link: </openapi.json>; rel="service-desc"; type="application/vnd.oai.openapi+json",
178
181
  </docs/api>; rel="service-doc"; type="text/html",
179
- </llms.txt>; rel="describedby"; type="text/markdown",
182
+ </llms.txt>; rel="describedby"; type="text/plain",
180
183
  </.well-known/api-catalog>; rel="api-catalog"; type="application/linkset+json"
181
184
  ```
185
+ - Give each link the type your server actually sends for that URL. llms.txt is usually `text/plain`.
182
186
  - `service-desc` is the machine-readable description (your OpenAPI). `service-doc` is the human documentation (RFC 8631). Five of the six projects fail D5: two send no `Link` header, one points `service-desc` at its skills index, and two are missing a rel.
183
187
  - Registered relations are `describedby`, `service-desc`, `service-doc`, `service-meta` and `api-catalog`. `ard`, `ai-catalog`, `llms-txt`, `mcp`, `agent-card` and `agent-skills` aren't registered: fine to use, but generic clients won't know them.
184
188
 
185
- **D7 JSON 404s for unknown `/.well-known/*` paths, and a JSON 405 on `POST /`** naming the MCP endpoint and the API (achurch.ai, magnifica). An HTML 404 page is noise to an agent. *Why: cheap, and it helps the agents and crawlers that guess. No project passes all of it yet.*
186
-
187
- **D8 Markdown for agents**
188
- - `Accept: text/markdown` and a `.md` URL both return the page as markdown.
189
- - Send `Vary: Accept` on both representations (RFC 9110 §12.5.5). Otherwise a cache can hand HTML readers the markdown, or the reverse.
190
- - Make links absolute, and link back to the HTML page as canonical.
191
- - Add `x-markdown-tokens` so an agent can budget before reading. It's Cloudflare's header, with no spec behind it.
192
- - Negotiate in front of prerendered pages too, or they ignore it (magnifica).
193
- - Whole-corpus indexes help: `/docs/index.md` and `/docs/index.json` (achurch.ai).
194
- - Behind Cloudflare (Pro and up), its Markdown for Agents feature does the conversion and sends these headers for you.
189
+ **D7 JSON 404s for unknown `/.well-known/*` paths, and a JSON 405 on `POST /`** naming the MCP endpoint and the API (achurch.ai, magnifica). An HTML 404 page is noise to an agent. *Why: cheap, and it helps the agents and crawlers that guess. No project passes all of it yet.* One catch-all route can answer D7 and D11 together ([recipe](docs/recipes.md#one-catch-all-for-unknown-well-known-paths-d7-d11)).
195
190
 
196
191
  **D9 An AI catalog (ARD)**
197
192
  - One list of what an agent can use here, such as the MCP server card, the OpenAPI document, skills and llms.txt, with identifiers in the form `urn:air:<publisher-domain>:<namespace>:<name>`.
@@ -202,7 +197,9 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
202
197
  **D10 A DNS AID record**
203
198
  - One TXT record at `_agent.<domain>` (AID v2.1.1, `v=aid2`). Point it at your main agent entry point: the hosted MCP endpoint if you have one (`p=mcp`), otherwise the OpenAPI document (`p=openapi`).
204
199
  - `v`, `u` and `p` are required and `a` is recommended. `s` is a description of up to 60 bytes; `d` is the docs URL.
200
+ - `a` is the auth hint. AID v2.1.1 allows `none`, `pat`, `apikey`, `basic`, `oauth2_device`, `oauth2_code`, `mtls` and `custom`. For an API that takes keys: `v=aid2;u=https://example.com/openapi.json;p=openapi;a=apikey;s=Example API;d=https://example.com/docs/api`.
205
201
  - Exactly one record: AID clients fail on ambiguity when there are two.
202
+ - After a change, resolvers keep the old answer until its TTL runs out, so an old record can show for minutes. The scorecard asks public resolvers first and says so when D10 fails.
206
203
  - Only name a protocol you serve (allowed: `mcp`, `a2a`, `openapi`, `grpc`, `graphql`, `websocket`, `local`, `zeroconf`, `ucp`). The scorecard fails a record that names a service you don't run.
207
204
  - Scanners check DNS-AID, a different spec ([Only if true](#only-if-true)), so an AID record earns no scanner credit. It's for AID-aware clients.
208
205
 
@@ -220,6 +217,7 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
220
217
  **D15 Name the AI crawlers you welcome, and state a Content-Signal policy.** *Why: it states intent, and scanners check it. It's Cloudflare's policy, not a standard, and Google ignores it.*
221
218
  - Training, search and live-fetch bots are separate user agents: GPTBot, OAI-SearchBot and ChatGPT-User; ClaudeBot, Claude-SearchBot and Claude-User; PerplexityBot and Perplexity-User; Google-Extended; Applebot-Extended; and so on.
222
219
  - `Content-Signal: search=yes, ai-input=yes, ai-train=yes` (or your actual choices) in every group: a line in the `*` group never reaches a bot with its own group.
220
+ - In Next.js, `robots.ts` can't output `Content-Signal`, so serve robots.txt from a route handler (animalhouse, drifts).
223
221
  - Watch the IETF's AIPREF drafts, which define a successor (a `Content-Usage` rule and header with the vocabulary `train-ai`, `ai-use`, `search` and values `y`/`n`).
224
222
 
225
223
  **A3 `/.well-known/api-catalog`** (RFC 9727): a linkset served as `application/linkset+json; profile="https://www.rfc-editor.org/info/rfc9727"`. Serve it from a route, because a prerendered extensionless file loses its content type (magnifica). *Why: baseline hygiene. It's an RFC and scanners check it, but one crawler asked for it in four days.*
@@ -239,7 +237,7 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
239
237
  | `$schema` | `https://schemas.agentskills.io/discovery/0.2.0/schema.json` |
240
238
  | `skills[]` | each entry has `name`, `type` (`skill-md` or `archive`), `description` (up to 1,024 characters), `url`, and `digest` (`sha256:` plus 64 lowercase hex) |
241
239
 
242
- The `$schema` value is an identifier, not a link (it doesn't have to resolve). Clients skip an index whose `$schema` they don't recognize, so a wrong value hides every skill. Extra fields are ignored. Details in [§7](#7-skills-and-plugins).
240
+ List the skills your site serves ([S1](#s1-skill-files)). The `$schema` value is an identifier, not a link (it doesn't have to resolve). Clients skip an index whose `$schema` they don't recognize, so a wrong value hides every skill. Extra fields are ignored. Details in [§7](#7-skills-and-plugins).
243
241
 
244
242
  **M5 An MCP server card** for a hosted endpoint ([§6](#m5-the-server-card)). Recommended, not required, because the extension is experimental.
245
243
 
@@ -313,8 +311,9 @@ The evidence that this work pays: achurch.ai's August batch rewrote titles and s
313
311
 
314
312
  **W3 One rule for what's indexable.** *Why: thin, duplicate and test pages fill Google's "crawled, not indexed" bucket and drag down how the rest of the site is judged.*
315
313
  - One rule decides which pages are indexable, and the sitemap, the page's robots meta and internal recommendation lists all use it (inbed's `indexable()` SQL column; at animalhouse the sitemap's filters are mirrored by `noindex`).
316
- - Not indexable: test and template accounts, empty profiles, unhatched or ephemeral items, filtered views, sign-in pages, and thin pages (inbed: chats with fewer than five messages). They still render, with `noindex`, and stay out of the sitemap. Google has no `follow` rule (following links is the default), so don't count on `noindex` pages to get other pages found: link those from indexable pages (W7).
314
+ - Not indexable: test and template accounts ([recipe](docs/recipes.md#test-accounts-out-of-public-pages-t5-w3)), empty profiles, unhatched or ephemeral items, filtered views, sign-in pages, and thin pages (inbed: chats with fewer than five messages). They still render, with `noindex`, and stay out of the sitemap. Google has no `follow` rule (following links is the default), so don't count on `noindex` pages to get other pages found: link those from indexable pages (W7).
317
315
  - Missing items return a real 404 (or 410; Google treats them the same), never a 200 page that says "not found". Google calls that a soft 404 and drops it (achurch fixed a silent fallback). Empty filter combinations and page numbers past the end get a 404 too.
316
+ - In Next.js, a `loading.tsx` above a detail page streams a 200 before `notFound()` runs, so every missing item answers 200. inbed moved its root `loading.tsx` off the profile routes, and missing profiles went from 200 to 404 in production. Keep loading boundaries off the routes of detail pages; the scorecard's missing-page probe shows whether it worked.
318
317
  - The sitemap lists only indexable canonical URLs that answer 200 without a redirect. Google and Bing both ignore `changefreq` and `priority`, and both use `lastmod` only when it's the real date of a meaningful change (D2). Bing asks for ISO 8601 with a time. Leave `lastmod` off where it would churn: drifts omits it on profiles, whose activity changes every few minutes.
319
318
  - **Done when** the scorecard passes W3.
320
319
 
@@ -475,7 +474,8 @@ The test is whether most of the six projects should do it. These came from one o
475
474
  - **Keep display titles out of `name`.** ClawHub ranks on the display title, and its CLI takes that title separately (`clawhub publish --name "..."`; animalhouse's publish script passes the H1). So the frontmatter `name` can follow the spec. Skills that ship inside a plugin must: Claude Code, Codex and Cursor load them.
476
475
  - **Trigger only on explicit requests, and confirm before anything persistent or public** (registering, publishing, posting, rotating a key). Every skill that publishes needs this, not just most of them. That wording is what passes ClawHub's security audit.
477
476
  - **Link the map.** Every skill links llms.txt and the API or MCP reference (`/docs/api.md`, `/openapi.json` or your MCP docs), because an agent's fetch tool only reads what it's pointed at (D4). The scorecard checks this on a sample of skills from your index. At animalhouse, the core skill and both plugin skills linked neither.
478
- - **Serve the raw file** as `text/plain` at a stable URL so an agent can install it with one fetch, plus a `/skills` page. Index them in S2.
477
+ - **Serve the raw file** at a stable URL so an agent can install it with one fetch (`/.well-known/agent-skills/<name>/SKILL.md` keeps the folder name in the URL), plus a `/skills` page. Index them in S2, with each digest computed from the bytes you serve.
478
+ - **Index the skills the site serves, not every listing.** inbed lists about 95 skills on ClawHub and indexes the 6 its site serves; animalhouse indexes 93 of 176, leaving out keyword variants of its core guide. Publishing an index makes S1 apply, so the required count goes up by one. Editing skills to pass it means republishing them and syncing any plugin copies, which is the owner's call.
479
479
 
480
480
  ### S3 Marketplaces and per-item skills
481
481
 
@@ -540,6 +540,7 @@ The test is whether most of the six projects should do it. These came from one o
540
540
  *Required.*
541
541
 
542
542
  - `Retry-After` on every 429 (RFC 6585 defines 429, RFC 9110 defines `Retry-After`).
543
+ - **A limit on key checks answers 429, never 401.** A 401 tells an agent with a valid key that its key is bad, and it may register again (drifts' busy agents got "include your API key"). Count only failed checks toward that limit.
543
544
  - A usage endpoint (inbed's `GET /api/rate-limits`), and the limits listed in llms.txt and auth.md.
544
545
  - `RateLimit` headers on every response are the next level (N10).
545
546
 
@@ -547,6 +548,10 @@ The test is whether most of the six projects should do it. These came from one o
547
548
 
548
549
  *Required where agents register.* Self-service registration with no human in the loop, a rotation endpoint, and a `source` attribution tag for each entry point plus the User-Agent. Never fail a sign-up over a bad tag.
549
550
 
551
+ - **Failing to check a key isn't a bad key.** A database error while verifying answers 503 with `Retry-After`, not 401.
552
+ - **With a slow hash such as bcrypt, cache verified keys** for a minute or so, keyed by a fast hash of the key, and drop the entry when the key rotates (inbed, drifts). Otherwise every request pays a full compare, and junk keys cost the same.
553
+ - **One auth helper answers for every route.** It returns the user or the response to send (`{ user } | { response }`), so every route answers the same way ([recipe](docs/recipes.md#one-auth-helper-for-every-route-a9-a10)).
554
+
550
555
  ### A11 Honesty and privacy
551
556
 
552
557
  *Required.*
@@ -607,6 +612,16 @@ The test is whether most of the six projects should do it. These came from one o
607
612
 
608
613
  Higher-value features that most of our projects don't have yet. Each one fixes a failure we've seen or can predict. Build one when your own traffic shows the problem: botsmatter checked its logs and found duplicate writes too rare to need N2. (N1, the agent usability test, is now T5 and required.)
609
614
 
615
+ **D8 Markdown for agents** (moved here from recommended; it keeps its ID). Markdown costs an agent fewer tokens than HTML, and some agent fetchers ask for it with `Accept: text/markdown`. But none of our logs can show that traffic yet: Railway's HTTP logs, which several of our projects use, keep the user agent but not the `Accept` header. And a scanner check alone doesn't make an item recommended (principle 3). *Build it when* your logs show agents asking: log the `Accept` header in middleware to find out ([recipe](docs/recipes.md#markdown-for-agents-without-a-cdn-plan-d8)).
616
+ - Start with the homepage and your docs, not every page. `Accept: text/markdown` and a `.md` URL both return the page as markdown. animalhouse serves llms.txt as the homepage's markdown, and its API reference at `/docs/api.md`.
617
+ - No CDN plan is needed: a `.md` route and a few lines of middleware do it ([recipe](docs/recipes.md#markdown-for-agents-without-a-cdn-plan-d8)). Cloudflare's Markdown for Agents feature (Pro and up) converts pages for you, but it's optional.
618
+ - Send `Vary: Accept` on both representations (RFC 9110 §12.5.5). Otherwise a cache can hand HTML readers the markdown, or the reverse.
619
+ - Next.js drops a `Vary` set in middleware, proxy or `next.config` on App Router pages, static or dynamic, while it keeps the other headers (tested on 14.2 and 16.3). So `Accept` has to reach the HTML from the edge (a CDN response header rule), or decline that half in writing. Send it from the markdown route yourself (animalhouse.ai).
620
+ - Make links absolute, and link back to the HTML page as canonical.
621
+ - Add `x-markdown-tokens` so an agent can budget before reading. It's Cloudflare's header, with no spec behind it.
622
+ - Negotiate in front of prerendered pages too, or they ignore it (magnifica).
623
+ - Whole-corpus indexes help: `/docs/index.md` and `/docs/index.json` (achurch.ai).
624
+
610
625
  **N2 `Idempotency-Key` on writes** (IETF draft). Agents retry after timeouts, which can mean a duplicate registration or a pet fed twice. Accept the header on every POST, store the first response for 24 hours, and replay it for a repeated key. *Build it when* your logs show the same write repeated within minutes.
611
626
 
612
627
  **N3 Conditional requests on polled reads.** Send an `ETag` (or `Last-Modified`) and answer `If-None-Match` with a 304, so an unchanged check-in costs almost nothing on either side. *Build it when* scheduled polling is a large share of your traffic.
@@ -638,21 +653,27 @@ Higher-value features that most of our projects don't have yet. Each one fixes a
638
653
  - **T1 Logs** (required):
639
654
  - MCP log lines: for stdio servers, the User-Agent on every API call; for a hosted endpoint, the protocol era, method, tool and client.
640
655
  - Structured request and error logs (drifts writes JSONL, rotated daily) and admin analytics.
656
+ - Logs that outlive a deploy. Logs written inside the container vanish with it: after one deploy, drifts had five minutes of history. Keep them on a volume, ship them out, or use the platform's own HTTP logs.
641
657
  - **T2 Attribution and one come-back measure** (required; per-agent tracking may be declined). Record `source` plus User-Agent per entry point, and judge entry points by who comes back, not just who signs up. At animalhouse the come-back metric is the share of adoptions that get care again 24 hours or more after hatching (about 12%). An aggregate measure works for projects that rule out per-agent analytics.
642
658
  - **T3 Reviews** (required):
643
- - A monthly log review, paged so no window is truncated: requested well-known paths, 404s, refusals, crawler user agents.
659
+ - A monthly log review, paged so no window is truncated (platform log commands often return only the newest window): requested well-known paths, 404s, refusals, crawler user agents.
644
660
  - After each deploy, run the scorecard, lint the OpenAPI spec, and diff generated files against the previous production output. About 24 hours later, review the logs and database query performance.
645
661
  - **T5 The agent usability test** (required; it was N1).
646
662
  - **Why:** it's the only check that measures what the whole standard is for, and without it the scorecard becomes the target.
647
- - **How:** give a fresh agent nothing but your domain and a goal ("adopt a pet and feed it"). Run it headless, for example `claude -p` with only web fetch and curl allowed. Record whether it succeeds, how many calls it takes, and every error it hits.
663
+ - **How:** give a fresh agent nothing but your domain and a goal ("adopt a pet and feed it"). Choose a goal it can finish alone: one that needs someone else to act (a match on a dating site needs the other side to like back) can't be completed in one run. Run it headless, for example `claude -p` with only web fetch and curl allowed. Record whether it succeeds, how many calls it takes, and every error it hits.
648
664
  - **When:** monthly, and after big API changes.
649
665
  - **Report** its first-try success rate next to your score.
650
666
  - **Rules:** run it only against your own site, with a test username so analytics filter it out.
667
+ - **Keep test accounts out of public pages:** one helper recognizes the test prefix; leave those accounts out of public listings, search and counts; `noindex` their pages and keep them out of the sitemap; and guard it with a test ([recipe](docs/recipes.md#test-accounts-out-of-public-pages-t5-w3)).
668
+ - **Keep the agent fresh,** with none of the operator's memory, settings or account details. drifts' test agent named itself after the operator, taken from its environment.
669
+ - **Budget it.** An open-ended run on a large model took drifts 7 minutes, 132 turns and $6.85 for one attempt. Set the model and a turn limit, and keep both the same month to month so results compare.
670
+ - **Report doc mismatches,** not only errors: docs that promise a field or behavior the API doesn't deliver.
651
671
  - **The procedure,** a runner script and the results format are in [docs/usability-test.md](docs/usability-test.md).
652
672
  - **T6 Search numbers** (required):
653
673
  - Verify the site in Google Search Console and Bing Webmaster Tools, and submit the sitemap to both.
654
674
  - Review them monthly: coverage (including "crawled, not indexed"), queries, clicks and click-through rate. Re-check about two weeks after an SEO change (inbed).
655
- - Record clicks, impressions and click-through rate next to the score, as achurch.ai's retrospective did. They're the outcome the W items are for.
675
+ - Record the numbers next to the score as fields, not prose, so months compare: the period, clicks, impressions, click-through rate, average position, pages indexed and not indexed, and "crawled, not indexed", for Google and Bing (format: [examples/recorded.json](examples/recorded.json)). They're the outcome the W items are for.
676
+ - It takes a person, or a browser session, with access to Search Console and Bing Webmaster Tools. There's no unattended way to read them.
656
677
  - In analytics, watch referrals from AI answer engines (chatgpt.com, perplexity.ai, copilot.microsoft.com, gemini.google.com, claude.ai).
657
678
  - Search Console gained a setting on 2026-08-31 that keeps a site out of Google's AI features. It's on by default, so leave it alone unless you mean to opt out.
658
679
 
@@ -683,6 +704,11 @@ A project with no test suite fails T4 however well it scores (botsmatter's own r
683
704
 
684
705
  - **robots.txt groups don't merge with `*`.** A crawler named in its own group ignores the `*` group, including its `Disallow` and `Content-Signal` lines.
685
706
  - **`Vary: Accept` on only one representation** lets caches mix markdown and HTML.
707
+ - **Next.js drops a `Vary` set in middleware, proxy or `next.config`** on App Router pages, while it keeps the other headers (tested on 14.2 and 16.3). It has to come from the CDN.
708
+ - **Next.js `robots.ts` can't output `Content-Signal`** or other unknown lines. Serve robots.txt from a route handler.
709
+ - **Redocly fails public operations without `security: []`** when the spec has a global security scheme.
710
+ - **zod-to-openapi v7 (Zod 3) needed `extendZodWithOpenApi(z)`** before path and query parameters came through with their descriptions (drifts).
711
+ - **A log written inside the container** is gone after the next deploy.
686
712
  - **Prerendered pages skip middleware**, so content negotiation must sit in front of them.
687
713
  - **Static extensionless files lose their content type.** Serve `api-catalog` from a route.
688
714
  - **Build-time `lastmod` dates on every sitemap URL** say nothing to crawlers.
@@ -717,10 +743,12 @@ A project with no test suite fails T4 however well it scores (botsmatter's own r
717
743
  Score each site monthly and after big releases, and keep the results where the project's agents can read them:
718
744
 
719
745
  ```
720
- npx readiness-audit --matrix site-one.com site-two.com --recorded docs/readiness-recorded.json > docs/readiness-status.md
746
+ npx readiness-audit@1 --matrix site-one.com site-two.com --recorded docs/readiness-recorded.json > docs/readiness-status.md
721
747
  ```
722
748
 
723
- The page lists every check for every site, the required and recommended scores, and each failure or warning with its reason. Keep it private if it names live bugs; our projects keep theirs in each project's own repo.
749
+ The page lists every check for every site, the scores at each level, and each failure or warning with its reason. Keep it in the project's private repo when there is one (for example, a private repo that holds the docs, with the public repo as a submodule). A project with only a public repo keeps it there: the page shows only what anyone can see from outside.
750
+
751
+ Score a local or preview build before you deploy: `npx readiness-audit@1 example.com --base http://localhost:3000` requests the site's URLs from the base and reports them under the domain, skipping DNS and host redirects. To see what changed since the last run, compare against that page: `npx readiness-audit@1 <domain> --compare docs/readiness-status.md` lists what was fixed, what newly fails and what newly applies. Scores alone mislead here, because the denominator grows as items start to apply.
724
752
 
725
753
  Read it with three caveats:
726
754
  - **It covers only what's visible from outside.** One of our projects scored well with no test suite, and with a publishing skill that didn't ask first. Neither showed in the score.
@@ -731,6 +759,8 @@ Read it with three caveats:
731
759
 
732
760
  ## 14. Adopting the standard in a repo
733
761
 
762
+ Step-by-step instructions for a project's coding agent: [docs/adopting.md](docs/adopting.md).
763
+
734
764
  - **Point to the standard instead of copying it,** so your agents read one standard, not a fork that drifts:
735
765
  ```markdown
736
766
  # Agent and Search Readiness
@@ -766,7 +796,7 @@ Level: **R** required where it applies, **Rec** recommended, **N** next level. W
766
796
  | D5 | `Link` headers: service-desc, service-doc, describedby, api-catalog | Rec | discovery | score |
767
797
  | D6 | security.txt with Expires once | R | hygiene | score |
768
798
  | D7 | JSON 404s for unknown well-known paths, JSON 405 on `POST /` | Rec | agents | score |
769
- | D8 | Markdown for agents with `Vary: Accept` and x-markdown-tokens | Rec | agents | score |
799
+ | D8 | Markdown for agents with `Vary: Accept` and x-markdown-tokens, once traffic asks for it | N | agents | score |
770
800
  | D9 | AI catalog at ard.json and ai-catalog.json, with `rel` links | Rec | discovery | score |
771
801
  | D10 | One DNS AID record naming a service you run | Rec | discovery | score |
772
802
  | D11 | The A2A path: a valid card or a JSON 404 | R | honesty | score |
@@ -1,13 +1,14 @@
1
1
  #!/usr/bin/env node
2
2
  // readiness-audit: score a live site against the Agent and Search Readiness Standard.
3
3
  //
4
- // readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json]
4
+ // readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json] [--compare previous] [--base url]
5
5
  // readiness-audit --matrix <domain> <domain> ... [--recorded results.json] (a markdown status page)
6
6
  //
7
7
  // Exits 1 when a required check fails, so it can gate a deploy.
8
8
 
9
9
  import { readFileSync } from 'node:fs';
10
10
  import { audit, matrix, score, LEVELS, REVIEW, VERSION } from '../src/audit.mjs';
11
+ import { parsePrevious, compareResults, formatComparison } from '../src/helpers.mjs';
11
12
 
12
13
  const args = process.argv.slice(2);
13
14
  const flag = (n) => args.includes(n);
@@ -18,15 +19,22 @@ if (flag('--version') || flag('-v')) {
18
19
  const valueOf = (name) => (flag(name) ? args[args.indexOf(name) + 1] : undefined);
19
20
  const mcpOpt = valueOf('--mcp');
20
21
  const recordedPath = valueOf('--recorded');
21
- const domains = args.filter((a, i) => !a.startsWith('-') && !['--mcp', '--recorded'].includes(args[i - 1]))
22
+ const comparePath = valueOf('--compare');
23
+ const base = valueOf('--base');
24
+ const domains = args.filter((a, i) => !a.startsWith('-') && !['--mcp', '--recorded', '--compare', '--base'].includes(args[i - 1]))
22
25
  .map((d) => d.replace(/^https?:\/\//, '').replace(/\/.*$/, ''));
23
- const opts = { hasApi: !flag('--no-api'), mcpPath: flag('--no-mcp') ? null : (mcpOpt ?? '/mcp') };
26
+ const opts = { hasApi: !flag('--no-api'), mcpPath: flag('--no-mcp') ? null : (mcpOpt ?? '/mcp'), base: base ?? null };
27
+ if (base) {
28
+ let u;
29
+ try { u = new URL(base); } catch { u = null; }
30
+ if (!u || !['http:', 'https:'].includes(u.protocol)) { console.error(`--base needs an http(s) URL, like http://localhost:3000 (got ${base})`); process.exit(2); }
31
+ }
24
32
 
25
33
  if (!domains.length || flag('--help') || flag('-h')) {
26
34
  console.error(`readiness-audit ${VERSION}
27
35
 
28
36
  Usage:
29
- readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json]
37
+ readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json] [--compare previous] [--base url]
30
38
  readiness-audit --matrix <domain> <domain> ... [--recorded results.json]
31
39
 
32
40
  Options:
@@ -35,31 +43,50 @@ Options:
35
43
  --no-api skip the API checks, for a site with no public API
36
44
  --json print the results as JSON
37
45
  --matrix score several sites and print a markdown status page
38
- --recorded f with --matrix: add recorded T5 and T6 results from a JSON file`);
46
+ --recorded f with --matrix: add recorded T5 and T6 results from a JSON file
47
+ --compare f list what changed since a previous run: a status page from
48
+ --matrix (docs/readiness-status.md) or a saved --json report
49
+ --base url score a local or preview build as the domain, before you
50
+ deploy (http://localhost:3000); DNS and host redirects are
51
+ skipped`);
39
52
  process.exit(domains.length ? 0 : 2);
40
53
  }
41
54
 
42
55
  if (flag('--matrix')) {
56
+ if (comparePath || base) console.error('--compare and --base work with a single domain; ignoring them for --matrix.');
57
+ opts.base = null;
43
58
  let recorded = {};
44
59
  if (recordedPath) {
45
60
  try { recorded = JSON.parse(readFileSync(recordedPath, 'utf8')); }
46
61
  catch (e) { console.error(`Couldn't read ${recordedPath}: ${e.message}`); process.exit(2); }
47
62
  }
48
63
  const reports = await Promise.all(domains.map((d) => audit(d, opts)));
49
- console.log(matrix(reports, `npx readiness-audit --matrix ${domains.join(' ')}${recordedPath ? ` --recorded ${recordedPath}` : ''}`, recorded));
64
+ const pinned = `readiness-audit@${VERSION.split('.')[0]}`; // the major version, as STANDARD.md §14 says to pin it
65
+ console.log(matrix(reports, `npx ${pinned} --matrix ${domains.join(' ')}${recordedPath ? ` --recorded ${recordedPath}` : ''}`, recorded));
50
66
  } else {
67
+ let previous = null;
68
+ if (comparePath) {
69
+ let text;
70
+ try { text = readFileSync(comparePath, 'utf8'); }
71
+ catch (e) { console.error(`Couldn't read ${comparePath}: ${e.message}`); process.exit(2); }
72
+ previous = parsePrevious(text, domains[0]);
73
+ if (!previous) { console.error(`${comparePath} has no results for ${domains[0]}`); process.exit(2); }
74
+ }
51
75
  const rep = await audit(domains[0], opts);
52
- if (flag('--json')) console.log(JSON.stringify(rep, null, 2));
76
+ const since = previous ? `${previous.date ?? 'the previous run'} (${comparePath})` : null;
77
+ const changes = previous ? compareResults(previous.results, rep.results) : null;
78
+ if (flag('--json')) console.log(JSON.stringify(changes ? { ...rep, changes: { since: previous.date, ...changes } } : rep, null, 2));
53
79
  else {
54
80
  const label = { pass: 'PASS', warn: 'WARN', fail: 'FAIL', skip: 'skip' };
55
- console.log(`Agent and search readiness: ${rep.domain} (readiness-audit v${VERSION}, ${rep.checked_at.slice(0, 16)}Z)`);
81
+ console.log(`Agent and search readiness: ${rep.domain}${rep.base ? ` via ${rep.base}` : ''} (readiness-audit v${VERSION}, ${rep.checked_at.slice(0, 16)}Z)`);
56
82
  for (const level of LEVELS) {
57
83
  const rows = rep.results.filter((r) => r.level === level);
58
84
  if (!rows.length) continue;
59
85
  console.log(`\n${{ required: 'Required', recommended: 'Recommended', next: 'Next level' }[level]}`);
60
86
  for (const r of rows) console.log(` ${label[r.status]} ${r.id.padEnd(4)} ${r.name.padEnd(36)} ${r.detail}`);
61
87
  }
62
- console.log(`\nScore: required ${score(rep.results, 'required')}, recommended ${score(rep.results, 'recommended')}, next level ${score(rep.results, 'next')}`);
88
+ console.log(`\nScore on ${rep.checked_at.slice(0, 10)}: required ${score(rep.results, 'required')}, recommended ${score(rep.results, 'recommended')}, next level ${score(rep.results, 'next')}`);
89
+ if (changes) console.log(`\n${formatComparison(changes, since).join('\n')}`);
63
90
  console.log('\nNot scored, so check these yourself:');
64
91
  for (const [id, what] of REVIEW) console.log(` ${id.padEnd(6)} ${what}`);
65
92
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "readiness-audit",
3
- "version": "1.0.0",
3
+ "version": "1.2.0",
4
4
  "description": "Score a live site against the Agent and Search Readiness Standard: discovery files, search, API, MCP and skills, for AI agents and search engines.",
5
5
  "type": "module",
6
6
  "bin": {
package/src/audit.mjs CHANGED
@@ -6,11 +6,13 @@
6
6
  // Read-only: GETs plus a few POSTs that create nothing (an MCP initialize, server/discover and
7
7
  // tools/list, an empty POST to /). Node 20+, no dependencies.
8
8
 
9
- import { lookup, resolveTxt } from 'node:dns/promises';
9
+ import { lookup, resolveTxt, Resolver } from 'node:dns/promises';
10
+ import { request as httpRequest } from 'node:http';
10
11
  import { request as httpsRequest } from 'node:https';
11
12
  import { isIP } from 'node:net';
13
+ import { SKILL_NAME, sampleSkillEntries, skillNameProblem, cdnCacheNote, repeatsSummary, hasVariantDescription, spread, llmsLinks, baseMapper, formatRecorded } from './helpers.mjs';
12
14
 
13
- export const VERSION = '1.0.0';
15
+ export const VERSION = '1.2.0';
14
16
  export const REPO = 'https://github.com/geeks-accelerator/agent-and-search-readiness';
15
17
  const UA = `readiness-audit/${VERSION} (+${REPO})`;
16
18
  const AI_BOTS = ['gptbot', 'oai-searchbot', 'chatgpt-user', 'claudebot', 'claude-user', 'claude-searchbot', 'perplexitybot', 'perplexity-user', 'google-extended', 'applebot-extended'];
@@ -18,7 +20,7 @@ const AID_PROTOCOLS = ['mcp', 'a2a', 'openapi', 'grpc', 'graphql', 'websocket',
18
20
  const SKILLS_SCHEMA = 'https://schemas.agentskills.io/discovery/0.2.0/schema.json';
19
21
  const CARD_SCHEMA = 'https://static.modelcontextprotocol.io/schemas/v1/server-card.schema.json';
20
22
  export const LEVELS = ['required', 'recommended', 'next'];
21
- const ORDER = ['D1', 'D2', 'D4', 'D6', 'D11', 'D12', 'W1', 'W2', 'W3', 'W4', 'W5', 'W6', 'A1', 'A2', 'A6', 'A7', 'M1', 'M4', 'S1', 'D5', 'D7', 'D8', 'D9', 'D10', 'D15', 'A3', 'A4', 'A5', 'M5', 'S2', 'N5', 'N6'];
23
+ const ORDER = ['D1', 'D2', 'D4', 'D6', 'D11', 'D12', 'W1', 'W2', 'W3', 'W4', 'W5', 'W6', 'A1', 'A2', 'A6', 'A7', 'M1', 'M4', 'S1', 'D5', 'D7', 'D9', 'D10', 'D15', 'A3', 'A4', 'A5', 'M5', 'S2', 'D8', 'N5', 'N6'];
22
24
  // Items the scorecard can't see from outside. Each project checks these in review and tests.
23
25
  export const REVIEW = [
24
26
  ['D3', 'pages read well: server-rendered, unique prose on generated pages (the scorecard checks W1 to W6 on a sample)'],
@@ -70,6 +72,19 @@ async function mapLimit(items, limit, fn) {
70
72
  return out;
71
73
  }
72
74
 
75
+ // TXT lookups go to public resolvers first: the machine's own resolver can hold an answer
76
+ // from before a change. Falls back to the system resolver when they can't be reached.
77
+ async function resolveTxtFresh(name) {
78
+ try {
79
+ const resolver = new Resolver({ timeout: 3000, tries: 2 });
80
+ resolver.setServers(['1.1.1.1', '8.8.8.8']);
81
+ return await resolver.resolveTxt(name);
82
+ } catch (e) {
83
+ if (e?.code === 'ENOTFOUND' || e?.code === 'ENODATA') throw e; // a real answer: no record
84
+ return resolveTxt(name);
85
+ }
86
+ }
87
+
73
88
  // Which URLs the audit may request. The site you chose to audit (and its www or bare variant) is
74
89
  // always allowed; anything else must be a public http(s) address, checked again at every redirect hop.
75
90
  export function makeGuard(domain) {
@@ -91,28 +106,32 @@ export function makeGuard(domain) {
91
106
  return { siteHosts, isSafe };
92
107
  }
93
108
 
94
- export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
109
+ export async function audit(domain, { hasApi = true, mcpPath = '/mcp', base = null } = {}) {
95
110
  const BASE = `https://${domain}`;
96
111
  const rand = Math.random().toString(36).slice(2, 10);
97
112
 
98
113
  const { siteHosts, isSafe } = makeGuard(domain);
114
+ // --base scores a local or preview build as the domain: the site's URLs are requested from the
115
+ // base (which may be a local address, because you chose it) and reported under the domain.
116
+ const { baseOrigin, toBase, fromBase } = baseMapper(domain, base);
99
117
 
100
118
  // Redirects are followed here, one hop at a time, so every hop passes isSafe.
101
119
  async function get(path, { headers = {}, method = 'GET', body, redirect = 'follow' } = {}) {
102
- let url = path.startsWith('http') ? path : BASE + path;
120
+ let url = path.startsWith('http') ? path : BASE + path; // always in the domain's terms
103
121
  let m = method, b = body;
104
122
  for (let hop = 0; ; hop++) {
105
- if (!(await isSafe(url))) return { status: 0, headers: new Headers(), text: '', url, type: '', error: 'refused: not a public address' };
123
+ const target = toBase(url);
124
+ if (target === url && !(await isSafe(url))) return { status: 0, headers: new Headers(), text: '', url, type: '', error: 'refused: not a public address' };
106
125
  let res;
107
126
  try {
108
- res = await fetch(url, { method: m, body: b, redirect: 'manual', headers: { 'user-agent': UA, ...headers }, signal: AbortSignal.timeout(15000) });
127
+ res = await fetch(target, { method: m, body: b, redirect: 'manual', headers: { 'user-agent': UA, ...headers }, signal: AbortSignal.timeout(15000) });
109
128
  } catch (e) {
110
129
  return { status: 0, headers: new Headers(), text: '', url, type: '', error: e.message };
111
130
  }
112
131
  const loc = res.headers.get('location');
113
132
  if (redirect === 'follow' && res.status >= 300 && res.status < 400 && loc && hop < 5) {
114
133
  try { await res.body?.cancel(); } catch { /* already closed */ }
115
- url = new URL(loc, url).href;
134
+ url = fromBase(new URL(loc, target).href);
116
135
  if (res.status !== 307 && res.status !== 308) { m = 'GET'; b = undefined; }
117
136
  continue;
118
137
  }
@@ -122,8 +141,10 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
122
141
  }
123
142
  // fetch() always sends an Accept header, so the "no Accept" MCP probe uses node:https.
124
143
  function postNoAccept(path, payload) {
144
+ const target = new URL(toBase(BASE + path));
145
+ const send = target.protocol === 'http:' ? httpRequest : httpsRequest;
125
146
  return new Promise((resolve) => {
126
- const req = httpsRequest(BASE + path, { method: 'POST', timeout: 15000, headers: { 'user-agent': UA, 'content-type': 'application/json', 'content-length': Buffer.byteLength(payload) } }, (res) => {
147
+ const req = send(target, { method: 'POST', timeout: 15000, headers: { 'user-agent': UA, 'content-type': 'application/json', 'content-length': Buffer.byteLength(payload) } }, (res) => {
127
148
  let text = '';
128
149
  res.on('data', (c) => { text += c; });
129
150
  res.on('end', () => resolve({ status: res.statusCode, type: res.headers['content-type'] ?? '', text }));
@@ -136,6 +157,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
136
157
  const json = (r) => { try { return JSON.parse(r.text); } catch { return null; } };
137
158
  const isJson = (r) => /json/.test(r.type) && json(r) !== null;
138
159
  const ok = (r) => r.status >= 200 && r.status < 300;
160
+ const cached = (r) => cdnCacheNote(r.headers);
139
161
  const ct = (r) => r.type.split(';')[0] || (r.status ? 'no body' : 'no answer');
140
162
  const rpc = (r) => {
141
163
  if (/event-stream/.test(r.type)) {
@@ -209,9 +231,9 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
209
231
  const id = 'D1', name = 'robots.txt', level = 'required';
210
232
  const star = groups.find((g) => g.agents.includes('*'));
211
233
  const leaks = star ? groups.filter((g) => g !== star).flatMap((g) => [...star.disallow].filter((d) => !g.disallow.has(d)).map((d) => `${g.agents[0]} skips Disallow ${d}`)) : [];
212
- if (!ok(robots)) miss(id, name, level, `/robots.txt answered ${robots.status}`);
213
- else if (leaks.length) miss(id, name, level, `named groups drop rules from the * group (a crawler obeys only its own group): ${[...new Set(leaks)].slice(0, 3).join('; ')}`);
214
- else if (!sitemapLine) miss(id, name, level, 'no Sitemap line');
234
+ if (!ok(robots)) miss(id, name, level, `/robots.txt answered ${robots.status}${cached(robots)}`);
235
+ else if (leaks.length) miss(id, name, level, `named groups drop rules from the * group (a crawler obeys only its own group): ${[...new Set(leaks)].slice(0, 3).join('; ')}${cached(robots)}`);
236
+ else if (!sitemapLine) miss(id, name, level, `no Sitemap line${cached(robots)}`);
215
237
  else pass(id, name, level, `${groups.length} group${groups.length === 1 ? '' : 's'} with consistent rules, Sitemap listed`);
216
238
  }
217
239
  {
@@ -222,7 +244,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
222
244
  if (unsignaled.length) problems.push(`${unsignaled.length} of ${groups.length} group${groups.length === 1 ? '' : 's'} ${unsignaled.length === 1 ? 'has' : 'have'} no Content-Signal (${unsignaled.slice(0, 3).map((g) => g.agents[0]).join(', ')}${unsignaled.length > 3 ? ', ...' : ''})`);
223
245
  if (named.length < 3) problems.push(`names ${named.length} of the main AI crawlers`);
224
246
  if (!ok(robots)) miss(id, name, level, 'no robots.txt');
225
- else if (problems.length) miss(id, name, level, problems.join('; '));
247
+ else if (problems.length) miss(id, name, level, problems.join('; ') + cached(robots));
226
248
  else pass(id, name, level, `${named.length} AI crawlers named, Content-Signal in every group`);
227
249
  }
228
250
 
@@ -230,18 +252,27 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
230
252
  {
231
253
  const id = 'D2', name = 'sitemap.xml', level = 'required';
232
254
  const lastmods = [...sitemap.text.matchAll(/<lastmod>([^<]+)<\/lastmod>/g)].map((m) => m[1].slice(0, 10));
233
- if (!ok(sitemap) || !/<urlset|<sitemapindex/.test(sitemap.text)) miss(id, name, level, `/sitemap.xml answered ${sitemap.status}`);
234
- else if (!lastmods.length) miss(id, name, level, 'no lastmod dates');
235
- else if (new Set(lastmods).size === 1 && lastmods.length > 5) miss(id, name, level, `all ${lastmods.length} lastmod dates are ${lastmods[0]}; use each page's real change date`);
255
+ if (!ok(sitemap) || !/<urlset|<sitemapindex/.test(sitemap.text)) miss(id, name, level, `/sitemap.xml answered ${sitemap.status}${cached(sitemap)}`);
256
+ else if (!lastmods.length) miss(id, name, level, `no lastmod dates${cached(sitemap)}`);
257
+ else if (new Set(lastmods).size === 1 && lastmods.length > 5) miss(id, name, level, `all ${lastmods.length} lastmod dates are ${lastmods[0]}; use each page's real change date${cached(sitemap)}`);
236
258
  else pass(id, name, level, `${lastmods.length} URLs with ${new Set(lastmods).size} distinct lastmod dates`);
237
259
  }
238
260
 
239
261
  // ---- D4 llms.txt ----
240
262
  {
241
263
  const id = 'D4', name = 'llms.txt and llms-full.txt', level = 'required';
242
- if (!ok(llms) || !/^\s*# \S/.test(llms.text)) miss(id, name, level, ok(llms) ? 'llms.txt has no H1 title on its first line' : `/llms.txt answered ${llms.status}`);
243
- else if (!ok(llmsFull)) miss(id, name, level, '/llms-full.txt missing');
244
- else pass(id, name, level, `llms.txt ${(llms.text.length / 1024).toFixed(1)} KB, llms-full.txt ${(llmsFull.text.length / 1024).toFixed(0)} KB`);
264
+ // Agents follow the links in llms.txt, so a sample of the site's own links must answer 200.
265
+ const links = ok(llms) ? llmsLinks(llms.text, `${BASE}/llms.txt`, siteHosts) : [];
266
+ const sample = spread(links, 10);
267
+ const answers = await mapLimit(sample, 4, async (u) => ({ u, r: await get(u) }));
268
+ // Missing pages, server errors and no answer are broken. A 401, 403 or 405 is an endpoint that
269
+ // exists but wants a key or another method, which is fine to list.
270
+ const broken = answers.filter(({ r }) => !r.status || r.status === 404 || r.status === 410 || r.status >= 500).map(({ u, r }) => `${u.replace(BASE, '') || '/'} (${r.status || 'no answer'})`);
271
+ const sizes = `llms.txt ${(llms.text.length / 1024).toFixed(1)} KB, llms-full.txt ${(llmsFull.text.length / 1024).toFixed(0)} KB`;
272
+ if (!ok(llms) || !/^\s*# \S/.test(llms.text)) miss(id, name, level, (ok(llms) ? 'llms.txt has no H1 title on its first line' : `/llms.txt answered ${llms.status}`) + cached(llms));
273
+ else if (!ok(llmsFull)) miss(id, name, level, `/llms-full.txt missing${cached(llmsFull)}`);
274
+ else if (broken.length) record(id, name, level, 'warn', `broken links in llms.txt: ${broken.slice(0, 3).join(', ')}${broken.length > 3 ? `, +${broken.length - 3} more` : ''} (${broken.length} of ${sample.length} sampled)${cached(llms)}`);
275
+ else pass(id, name, level, `${sizes}${sample.length ? `; ${sample.length} sampled link${sample.length === 1 ? '' : 's'} resolve` : ''}`);
245
276
  }
246
277
 
247
278
  // ---- D5 Link headers (recommended; a wrong service-desc is a false declaration) ----
@@ -267,10 +298,10 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
267
298
  const id = 'D6', name = 'security.txt', level = 'required';
268
299
  const expires = [...security.text.matchAll(/^expires:\s*(.+)$/gim)].map((m) => Date.parse(m[1].trim()));
269
300
  const days = expires.length === 1 ? (expires[0] - Date.now()) / 86_400_000 : NaN;
270
- if (!ok(security) || !/text\/plain/.test(security.type)) miss(id, name, level, `/.well-known/security.txt answered ${security.status}`);
271
- else if (!/^contact:\s*\S/im.test(security.text)) miss(id, name, level, 'no Contact field');
272
- else if (expires.length !== 1) miss(id, name, level, `${expires.length} Expires fields (RFC 9116 requires exactly one)`);
273
- else if (!(days > 0)) miss(id, name, level, `Expires passed ${Math.round(-days)} days ago`);
301
+ if (!ok(security) || !/text\/plain/.test(security.type)) miss(id, name, level, `/.well-known/security.txt answered ${security.status}${cached(security)}`);
302
+ else if (!/^contact:\s*\S/im.test(security.text)) miss(id, name, level, `no Contact field${cached(security)}`);
303
+ else if (expires.length !== 1) miss(id, name, level, `${expires.length} Expires fields (RFC 9116 requires exactly one)${cached(security)}`);
304
+ else if (!(days > 0)) miss(id, name, level, `Expires passed ${Math.round(-days)} days ago${cached(security)}`);
274
305
  else pass(id, name, level, `Contact present, Expires in ${Math.round(days)} days${days > 366 ? ' (the RFC recommends under a year)' : ''}`);
275
306
  }
276
307
 
@@ -286,7 +317,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
286
317
 
287
318
  // ---- D8 markdown for agents ----
288
319
  {
289
- const id = 'D8', name = 'Markdown for agents', level = 'recommended';
320
+ const id = 'D8', name = 'Markdown for agents', level = 'next';
290
321
  const varyAccept = (r) => /(^|,)\s*accept\s*(,|$)/i.test(r.headers.get('vary') ?? '');
291
322
  const tokens = homeMd.headers.get('x-markdown-tokens');
292
323
  if (!/text\/markdown/.test(homeMd.type)) miss(id, name, level, `Accept: text/markdown on / returns ${ct(homeMd)}`);
@@ -343,33 +374,46 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
343
374
  seen.add(s.$ref);
344
375
  return deref(s.$ref.replace(/^#\//, '').split('/').reduce((n, k) => n?.[k], spec), seen);
345
376
  };
346
- const undescribedProps = (schema, seen = new Set()) => {
377
+ // `nested` collects fields whose only description sits inside anyOf, oneOf or allOf.
378
+ const undescribedProps = (schema, seen = new Set(), nested = null) => {
347
379
  const s = deref(schema);
348
380
  if (!s || typeof s !== 'object' || seen.has(s)) return [];
349
381
  seen.add(s);
350
382
  const out = [];
351
383
  for (const [k, v] of Object.entries(s.properties ?? {})) {
352
384
  const d = deref(v);
353
- if (!(v?.description || d?.description)) out.push(k);
354
- out.push(...undescribedProps(d, seen));
385
+ if (!(v?.description || d?.description)) {
386
+ out.push(k);
387
+ if (nested && hasVariantDescription(d, deref)) nested.add(k);
388
+ }
389
+ out.push(...undescribedProps(d, seen, nested));
355
390
  }
356
- for (const key of ['items', 'allOf', 'anyOf', 'oneOf']) for (const sub of [].concat(s[key] ?? [])) out.push(...undescribedProps(sub, seen));
391
+ for (const key of ['items', 'allOf', 'anyOf', 'oneOf']) for (const sub of [].concat(s[key] ?? [])) out.push(...undescribedProps(sub, seen, nested));
357
392
  return out;
358
393
  };
359
394
  const ops = Object.entries(spec.paths ?? {}).flatMap(([p, item]) => Object.entries(item)
360
395
  .filter(([m]) => ['get', 'post', 'put', 'patch', 'delete'].includes(m))
361
396
  .map(([m, o]) => ({ key: `${m.toUpperCase()} ${p}`, o, params: [...(item.parameters ?? []), ...(o.parameters ?? [])] })));
362
397
  const noDesc = ops.filter(({ o }) => !String(o.description ?? '').trim());
363
- const fieldGaps = ops.flatMap(({ key, o, params }) => [
364
- ...params.map((p) => deref(p)).filter((p) => p && !p.description && !deref(p.schema)?.description).map((p) => `${key} ${p.name}`),
365
- ...Object.values(deref(o.requestBody)?.content ?? {}).flatMap((c) => undescribedProps(c.schema)).map((f) => `${key} ${f}`),
366
- ]);
398
+ const inVariant = [];
399
+ const fieldGaps = ops.flatMap(({ key, o, params }) => {
400
+ const nested = new Set();
401
+ const gaps = [
402
+ ...params.map((p) => deref(p)).filter((p) => p && !p.description && !deref(p.schema)?.description).map((p) => `${key} ${p.name}`),
403
+ ...Object.values(deref(o.requestBody)?.content ?? {}).flatMap((c) => undescribedProps(c.schema, new Set(), nested)).map((f) => `${key} ${f}`),
404
+ ];
405
+ inVariant.push(...[...nested].map((f) => `${key} ${f}`));
406
+ return gaps;
407
+ });
408
+ const variantNote = inVariant.length ? `; ${inVariant.length === fieldGaps.length ? 'each has' : `${inVariant.length} of them ${inVariant.length === 1 ? 'has' : 'have'}`} one inside anyOf or oneOf: put it on the field itself (in Zod, call .describe() last, after .optional() and .nullable())` : '';
409
+ const repeats = ops.filter(({ o }) => repeatsSummary(o));
410
+ const repeatNote = repeats.length ? `; ${repeats.length} operation${repeats.length === 1 ? "'s description only repeats its" : "s' descriptions only repeat their"} summary (say when to use ${repeats.length === 1 ? 'it' : 'each'})` : '';
367
411
  const respGaps = new Set(ops.flatMap(({ o }) => Object.values(o.responses ?? {}).flatMap((r) => Object.values(deref(r)?.content ?? {}).flatMap((c) => undescribedProps(c.schema)))));
368
412
  const respNote = respGaps.size ? `; ${respGaps.size} response field${respGaps.size === 1 ? ' has' : 's have'} no description (see N7)` : '';
369
413
  if (!String(spec.openapi).startsWith('3.1')) miss(id, name, level, `OpenAPI ${spec.openapi} (use 3.1)`);
370
- else if (noDesc.length) miss(id, name, level, `${noDesc.length} of ${ops.length} operations ${noDesc.length === 1 ? 'has' : 'have'} no description (${noDesc.slice(0, 3).map((x) => x.key).join(', ')}${noDesc.length > 3 ? ', ...' : ''})${respNote}`);
371
- else if (fieldGaps.length) miss(id, name, level, `${fieldGaps.length} request field${fieldGaps.length === 1 ? ' has' : 's have'} no description (${fieldGaps.slice(0, 3).join(', ')}${fieldGaps.length > 3 ? ', ...' : ''})${respNote}`);
372
- else pass(id, name, level, `OpenAPI ${spec.openapi}; all ${ops.length} operations and their request fields described${respNote}`);
414
+ else if (noDesc.length) miss(id, name, level, `${noDesc.length} of ${ops.length} operations ${noDesc.length === 1 ? 'has' : 'have'} no description (${noDesc.slice(0, 3).map((x) => x.key).join(', ')}${noDesc.length > 3 ? ', ...' : ''})${repeatNote}${respNote}`);
415
+ else if (fieldGaps.length) miss(id, name, level, `${fieldGaps.length} request field${fieldGaps.length === 1 ? ' has' : 's have'} no description (${fieldGaps.slice(0, 3).join(', ')}${fieldGaps.length > 3 ? ', ...' : ''})${variantNote}${repeatNote}${respNote}`);
416
+ else pass(id, name, level, `OpenAPI ${spec.openapi}; all ${ops.length} operations and their request fields described${repeatNote}${respNote}`);
373
417
  }
374
418
  }
375
419
  {
@@ -425,30 +469,34 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
425
469
  // ---- S1 skill files, on a sample from the skills index ----
426
470
  {
427
471
  const id = 'S1', name = 'Skill files', level = 'required';
428
- const entries = (json(skills)?.skills ?? []).filter((e) => typeof e?.url === 'string');
429
- if (!ok(skills) || !entries.length) skip(id, name, level, 'no skills index to sample; check your SKILL.md files in review');
430
- else {
431
- const sample = entries.length <= 5 ? entries : [0, 1, 2, 3, 4].map((i) => entries[Math.floor((i * entries.length) / 5)]);
472
+ const listed = json(skills)?.skills;
473
+ const sample = sampleSkillEntries(listed);
474
+ if (!ok(skills) || !sample.length) {
475
+ skip(id, name, level, Array.isArray(listed) && listed.length
476
+ ? 'the skills index lists no SKILL.md files to sample (archives only); check your SKILL.md files in review'
477
+ : 'no skills index to sample; check your SKILL.md files in review');
478
+ } else {
479
+ // Entry URLs may be relative, and resolve against the index (RFC 3986).
480
+ const indexUrl = `${BASE}/.well-known/agent-skills/index.json`;
432
481
  const files = await Promise.all(sample.map(async (e) => {
433
- const url = new URL(e.url, BASE).href;
482
+ const url = new URL(e.url, indexUrl).href;
434
483
  const r = await get(url);
435
484
  const fm = r.text.match(/^---\r?\n([\s\S]*?)\r?\n---/)?.[1] ?? '';
436
485
  const field = (k) => fm.match(new RegExp(`^${k}:\\s*(?:"([^"]*)"|'([^']*)'|(.*))$`, 'm'))?.slice(1).find((v) => v !== undefined)?.trim() ?? '';
437
- const folder = url.split('/').slice(-2, -1)[0] ?? '';
438
- return { url, status: r.status, text: r.text, name: field('name'), description: field('description'), folder, lines: r.text.split('\n').length };
486
+ const label = typeof e.name === 'string' && e.name ? e.name : url;
487
+ return { url, status: r.status, text: r.text, name: field('name'), description: field('description'), entryName: e.name, label, lines: r.text.split('\n').length };
439
488
  }));
440
- const label = (f) => f.folder || f.url;
441
- const unreachable = files.filter((f) => f.status !== 200).map((f) => `${label(f)} (${f.status})`);
489
+ const unreachable = files.filter((f) => f.status !== 200).map((f) => `${f.label} (${f.status})`);
442
490
  const read = files.filter((f) => f.status === 200);
443
- const badName = read.filter((f) => !/^(?!-)(?!.*--)[a-z0-9-]{1,64}(?<!-)$/.test(f.name) || (f.folder && f.name !== f.folder)).map(label);
444
- const badDesc = read.filter((f) => !f.description || f.description.length > 1024).map(label);
445
- const noMap = read.filter((f) => !/llms\.txt/.test(f.text)).map(label);
446
- const noApi = read.filter((f) => !/openapi|\/docs\/(api|mcp)|https?:\/\/[^\s)]+\/(api|mcp)\b/i.test(f.text)).map(label);
447
- const long = read.filter((f) => f.lines > 500).map((f) => `${label(f)} (${f.lines} lines)`);
491
+ const badName = read.map((f) => [f, skillNameProblem(f)]).filter(([, why]) => why).map(([f, why]) => `${f.label} (${why})`);
492
+ const badDesc = read.filter((f) => !f.description || f.description.length > 1024).map((f) => f.label);
493
+ const noMap = read.filter((f) => !/llms\.txt/.test(f.text)).map((f) => f.label);
494
+ const noApi = read.filter((f) => !/openapi|\/docs\/(api|mcp)|https?:\/\/[^\s)]+\/(api|mcp)\b/i.test(f.text)).map((f) => f.label);
495
+ const long = read.filter((f) => f.lines > 500).map((f) => `${f.label} (${f.lines} lines)`);
448
496
  const list = (xs) => `${xs.slice(0, 3).join(', ')}${xs.length > 3 ? `, +${xs.length - 3} more` : ''}`;
449
497
  const problems = [], warns = [];
450
498
  if (unreachable.length) problems.push(`skill files that don't load: ${list(unreachable)}`);
451
- if (badName.length) problems.push(`names that aren't a lowercase slug matching the folder: ${list(badName)}`);
499
+ if (badName.length) problems.push(`names that break the spec (a lowercase slug, matching its folder and its index entry): ${list(badName)}`);
452
500
  if (badDesc.length) problems.push(`missing or over-long descriptions: ${list(badDesc)}`);
453
501
  if (noMap.length) warns.push(`no link to llms.txt: ${list(noMap)}`);
454
502
  if (noApi.length) warns.push(`no link to the API or MCP reference: ${list(noApi)}`);
@@ -470,7 +518,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
470
518
  if (doc.$schema !== SKILLS_SCHEMA) bad.push(`unrecognized $schema, so clients skip the whole index (${doc.$schema ?? 'none'})`);
471
519
  const types = list.filter((s) => !['skill-md', 'archive'].includes(s.type)).length;
472
520
  const digests = list.filter((s) => !/^sha256:[0-9a-f]{64}$/.test(s.digest ?? '')).length;
473
- const names = list.filter((s) => !/^(?!-)(?!.*--)[a-z0-9-]{1,64}(?<!-)$/.test(s.name ?? '')).length;
521
+ const names = list.filter((s) => !SKILL_NAME.test(s.name ?? '')).length;
474
522
  if (types) bad.push(`${types} entries with a type other than skill-md or archive`);
475
523
  if (digests) bad.push(`${digests} entries without a sha256: digest`);
476
524
  if (names) bad.push(`${names} names that aren't lowercase slugs`);
@@ -654,7 +702,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
654
702
  };
655
703
  const alt = domain.startsWith('www.') ? `https://${domain.slice(4)}/` : `https://www.${domain}/`;
656
704
  const hostProblems = [];
657
- for (const start of [`http://${domain}/`, alt]) {
705
+ for (const start of baseOrigin ? [] : [`http://${domain}/`, alt]) { // --base: the live hosts aren't what's being scored
658
706
  const h = await hostHops(start);
659
707
  if (!h) continue; // that host doesn't resolve or answer
660
708
  if (new URL(h.end).origin !== origin) hostProblems.push(`${start} never reaches ${origin} (ends at ${h.end})`);
@@ -666,7 +714,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
666
714
  if (elsewhere.length) problems.push(`sitemap URLs canonical to another URL: ${list(elsewhere)}`);
667
715
  problems.push(...hostProblems);
668
716
  if (problems.length) miss(id, name, level, problems.join('; '));
669
- else pass(id, name, level, `self-referencing canonicals on ${ok200.length} sampled pages; http and the other host redirect once to ${origin}`);
717
+ else pass(id, name, level, `self-referencing canonicals on ${ok200.length} sampled pages; ${baseOrigin ? 'host redirects not checked with --base' : `http and the other host redirect once to ${origin}`}`);
670
718
  }
671
719
  // W3 only indexable pages in the sitemap; real 404s
672
720
  {
@@ -751,21 +799,25 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
751
799
  // ---- D10 DNS AID (recommended; a record naming a service you don't run is false) ----
752
800
  {
753
801
  const id = 'D10', name = 'DNS AID record', level = 'recommended';
754
- let recs = [];
755
- try { recs = (await resolveTxt(`_agent.${domain}`)).map((r) => r.join('')).filter((r) => r.startsWith('v=aid')); } catch { recs = []; }
756
- if (!recs.length) miss(id, name, level, `no TXT record at _agent.${domain}`);
757
- else if (recs.length > 1) wrong(id, name, level, `${recs.length} records at _agent.${domain}; AID clients fail on ambiguity, so keep exactly one`);
802
+ const recent = ' (DNS answers stay cached until the record\'s TTL runs out, so a change can take a while to show)';
803
+ if (baseOrigin) skip(id, name, level, 'not checked with --base: DNS describes the live domain');
758
804
  else {
759
- const kv = Object.fromEntries(recs[0].split(';').map((p) => p.split('=')).filter((p) => p.length >= 2).map(([k, ...v]) => [k.trim(), v.join('=').trim()]));
760
- if (kv.v !== 'aid2') miss(id, name, level, `version ${kv.v} (current is aid2)`);
761
- else if (!kv.u || !AID_PROTOCOLS.includes(kv.p)) wrong(id, name, level, `invalid record: u=${kv.u ?? 'missing'} p=${kv.p ?? 'missing'}`);
805
+ let recs = [];
806
+ try { recs = (await resolveTxtFresh(`_agent.${domain}`)).map((r) => r.join('')).filter((r) => r.startsWith('v=aid')); } catch { recs = []; }
807
+ if (!recs.length) miss(id, name, level, `no TXT record at _agent.${domain}${recent}`);
808
+ else if (recs.length > 1) wrong(id, name, level, `${recs.length} records at _agent.${domain}; AID clients fail on ambiguity, so keep exactly one${recent}`);
762
809
  else {
763
- let why = '';
764
- if (kv.p === 'a2a' && !a2aValid) why = 'p=a2a, but there is no valid A2A card';
765
- else if (kv.p === 'mcp' && !(await answersMcp(kv.u)).hosted) why = `p=mcp, but ${kv.u} doesn't answer MCP`;
766
- else if (kv.p === 'openapi' && !json(await get(kv.u))?.openapi) why = `p=openapi, but ${kv.u} isn't an OpenAPI document`;
767
- if (why) wrong(id, name, level, why);
768
- else pass(id, name, level, `p=${kv.p} u=${kv.u}`);
810
+ const kv = Object.fromEntries(recs[0].split(';').map((p) => p.split('=')).filter((p) => p.length >= 2).map(([k, ...v]) => [k.trim(), v.join('=').trim()]));
811
+ if (kv.v !== 'aid2') miss(id, name, level, `version ${kv.v} (current is aid2)`);
812
+ else if (!kv.u || !AID_PROTOCOLS.includes(kv.p)) wrong(id, name, level, `invalid record: u=${kv.u ?? 'missing'} p=${kv.p ?? 'missing'}`);
813
+ else {
814
+ let why = '';
815
+ if (kv.p === 'a2a' && !a2aValid) why = 'p=a2a, but there is no valid A2A card';
816
+ else if (kv.p === 'mcp' && !(await answersMcp(kv.u)).hosted) why = `p=mcp, but ${kv.u} doesn't answer MCP`;
817
+ else if (kv.p === 'openapi' && !json(await get(kv.u))?.openapi) why = `p=openapi, but ${kv.u} isn't an OpenAPI document`;
818
+ if (why) wrong(id, name, level, why);
819
+ else pass(id, name, level, `p=${kv.p} u=${kv.u}`);
820
+ }
769
821
  }
770
822
  }
771
823
  }
@@ -795,7 +847,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
795
847
  }
796
848
 
797
849
  results.sort((a, b) => LEVELS.indexOf(a.level) - LEVELS.indexOf(b.level) || ORDER.indexOf(a.id) - ORDER.indexOf(b.id));
798
- return { domain, version: VERSION, checked_at: new Date().toISOString(), hosted_mcp: hosted, results };
850
+ return { domain, base: baseOrigin, version: VERSION, checked_at: new Date().toISOString(), hosted_mcp: hosted, results };
799
851
  }
800
852
 
801
853
  export function score(results, level) {
@@ -803,7 +855,7 @@ export function score(results, level) {
803
855
  return rows.length ? `${rows.filter((r) => r.status === 'pass').length} of ${rows.length}` : 'n/a';
804
856
  }
805
857
 
806
- // Markdown status page for several sites, for docs/guides/agent-readiness-status.md.
858
+ // Markdown status page for several sites, for a project's docs/readiness-status.md.
807
859
  // `recorded` holds what the scorecard can't measure, per domain:
808
860
  // { "example.com": { "T5": { "result": "4 of 5 first try", "date": "2026-10-12" }, "T6": { ... } } }
809
861
  export function matrix(reports, command, recorded = {}) {
@@ -819,9 +871,10 @@ export function matrix(reports, command, recorded = {}) {
819
871
  ...ids.map(([id, name, level]) => `| ${id} | ${name} | ${level} | ${reports.map((rep) => sym[rep.results.find((r) => r.id === id)?.status ?? 'skip']).join(' | ')} |`),
820
872
  `| | **Required passed** | | ${reports.map((rep) => `**${score(rep.results, 'required')}**`).join(' | ')} |`,
821
873
  `| | Recommended passed | | ${reports.map((rep) => score(rep.results, 'recommended')).join(' | ')} |`,
874
+ `| | Next level passed | | ${reports.map((rep) => score(rep.results, 'next')).join(' | ')} |`,
822
875
  `| | Hosted MCP endpoint | | ${reports.map((rep) => (rep.hosted_mcp ? 'yes' : 'no')).join(' | ')} |`,
823
876
  ...[['T5', 'Agent usability test (recorded)'], ['T6', 'Search numbers (recorded)']].map(([id, label]) =>
824
- `| ${id} | ${label} | required | ${reports.map((rep) => { const x = recorded[rep.domain]?.[id]; return x ? `${String(x.result ?? '').replace(/\|/g, '/')}${x.date ? ` (${x.date})` : ''}` : 'not recorded'; }).join(' | ')} |`),
877
+ `| ${id} | ${label} | required | ${reports.map((rep) => formatRecorded(recorded[rep.domain]?.[id])).join(' | ')} |`),
825
878
  '',
826
879
  '## What to fix, per site', '',
827
880
  'Failures and warnings with the scorecard\'s reason, required items first.', '',
@@ -0,0 +1,192 @@
1
+ // Pure helpers for the checks in audit.mjs, kept apart so the tests can run
2
+ // them without a network. Internal: the package exports only audit.mjs.
3
+
4
+ /** The Agent Skills spec's name rule: lowercase letters, digits and single hyphens, 1 to 64 characters. */
5
+ export const SKILL_NAME = /^(?!-)(?!.*--)[a-z0-9-]{1,64}(?<!-)$/;
6
+
7
+ /**
8
+ * Up to five SKILL.md entries from a skills index, spread across it. Archive
9
+ * entries are skipped: the scorecard reads SKILL.md text, not zip bytes. An
10
+ * entry without a type counts when its URL ends in .md (older indexes).
11
+ */
12
+ export function sampleSkillEntries(list) {
13
+ const md = (Array.isArray(list) ? list : []).filter((e) => typeof e?.url === 'string'
14
+ && (e.type === 'skill-md' || (e.type === undefined && /\.md$/i.test(e.url.split(/[?#]/)[0]))));
15
+ return spread(md, 5);
16
+ }
17
+
18
+ /** Up to n items, spread evenly across the list (the first item always included). */
19
+ export function spread(list, n) {
20
+ return list.length <= n ? list : Array.from({ length: n }, (_, i) => list[Math.floor((i * list.length) / n)]);
21
+ }
22
+
23
+ /**
24
+ * The site's own links in an llms.txt: markdown links and bare URLs, resolved
25
+ * against the file's URL, without fragments, kept only when they point at one
26
+ * of the site's hosts. Templates such as /creatures/{id} are skipped.
27
+ */
28
+ export function llmsLinks(text, llmsUrl, hosts) {
29
+ const raw = [
30
+ ...[...String(text ?? '').matchAll(/\]\(\s*<?([^)\s>]+)>?[^)]*\)/g)].map((m) => m[1]),
31
+ ...[...String(text ?? '').matchAll(/\bhttps?:\/\/[^\s)>\]"'`]+/g)].map((m) => m[0]),
32
+ ];
33
+ const out = new Set();
34
+ for (const href of raw) {
35
+ if (href.startsWith('#')) continue; // an anchor in the file itself
36
+ if (/[{}]|%7B|%7D|\/:[a-z_]+|YOUR_|\.\.\./i.test(href)) continue; // a template ({id}, :id), not a link
37
+ let u;
38
+ try { u = new URL(href.replace(/[.,;:!?]+$/, ''), llmsUrl); } catch { continue; }
39
+ if (!['http:', 'https:'].includes(u.protocol) || !hosts.has(u.hostname)) continue;
40
+ u.hash = '';
41
+ out.add(u.href);
42
+ }
43
+ return [...out];
44
+ }
45
+
46
+ /**
47
+ * For --base: request a local or preview build as if it were the domain.
48
+ * toBase() sends the site's URLs (the domain or its www twin) to the base;
49
+ * fromBase() maps them back, so every check keeps seeing the domain's URLs.
50
+ */
51
+ export function baseMapper(domain, base) {
52
+ const hosts = new Set([domain, domain.startsWith('www.') ? domain.slice(4) : `www.${domain}`]);
53
+ const baseOrigin = base ? new URL(base).origin : null;
54
+ const toBase = (url) => {
55
+ if (!baseOrigin) return url;
56
+ let u;
57
+ try { u = new URL(url); } catch { return url; }
58
+ return hosts.has(u.hostname) ? baseOrigin + u.pathname + u.search : url;
59
+ };
60
+ const fromBase = (url) => (baseOrigin && (url === baseOrigin || url.startsWith(`${baseOrigin}/`) || url.startsWith(`${baseOrigin}?`)) ? `https://${domain}${url.slice(baseOrigin.length) || '/'}` : url);
61
+ return { baseOrigin, toBase, fromBase };
62
+ }
63
+
64
+ /**
65
+ * A recorded T5 or T6 result for the status page. T6 may be structured
66
+ * ({ period, google: { clicks, impressions, ctr, position, indexed,
67
+ * not_indexed, crawled_not_indexed }, bing: { ... } }); otherwise the free-text
68
+ * `result` is shown. Pipes are replaced so the table stays intact.
69
+ */
70
+ export function formatRecorded(x) {
71
+ if (!x) return 'not recorded';
72
+ const num = (v, label) => (v === undefined || v === null || v === '' ? null : `${v}${label}`);
73
+ const traffic = (o) => [num(o.clicks, ' clicks'), num(o.impressions, ' impressions'), num(o.ctr, '% CTR'), o.position != null ? `position ${o.position}` : null].filter(Boolean).join(', ');
74
+ const parts = [];
75
+ if (x.google && typeof x.google === 'object') {
76
+ const g = x.google;
77
+ const index = [num(g.indexed, ' indexed'), num(g.not_indexed, ' not indexed'), num(g.crawled_not_indexed, ' crawled but not indexed')].filter(Boolean).join(', ');
78
+ parts.push(`Google: ${[traffic(g), index].filter(Boolean).join('; ')}`);
79
+ }
80
+ if (x.bing && typeof x.bing === 'object') parts.push(`Bing: ${traffic(x.bing)}`);
81
+ const when = [parts.length ? x.period : null, x.date].filter(Boolean).join(', ');
82
+ const text = parts.length ? parts.join('. ') : String(x.result ?? '');
83
+ return `${text}${when ? ` (${when})` : ''}`.replace(/\|/g, '/');
84
+ }
85
+
86
+
87
+ const decode = (s) => { try { return decodeURIComponent(s); } catch { return s; } };
88
+
89
+ /**
90
+ * Why a SKILL.md's frontmatter name breaks the Agent Skills spec, or null.
91
+ * The name must be a lowercase slug, match its folder when the URL shows one
92
+ * (.../<name>/SKILL.md), and match the name its index entry gives. A URL in
93
+ * another shape (/skills/<name>) says nothing about the folder.
94
+ */
95
+ export function skillNameProblem({ name, url, entryName }) {
96
+ if (!SKILL_NAME.test(name ?? '')) return 'not a lowercase slug';
97
+ const path = new URL(url).pathname;
98
+ const folder = /\/SKILL\.md$/i.test(path) ? decode(path.split('/').slice(-2, -1)[0] ?? '') : '';
99
+ if (folder && folder !== name) return `folder is ${folder}`;
100
+ if (typeof entryName === 'string' && entryName && entryName !== name) return `index says ${entryName}`;
101
+ return null;
102
+ }
103
+
104
+ /**
105
+ * A note for checks that read a file a CDN may serve from its cache
106
+ * (Cloudflare caches robots.txt by default): that copy can predate the latest
107
+ * deploy. Empty unless the response says it came from a cache.
108
+ */
109
+ export function cdnCacheNote(headers) {
110
+ const cf = headers.get('cf-cache-status');
111
+ const hit = cf ? /^(hit|stale|updating)$/i.test(cf.trim()) : /\bhit\b/i.test(headers.get('x-cache') ?? '');
112
+ const age = Number(headers.get('age') ?? 0);
113
+ if (!hit || !(age > 0)) return '';
114
+ const old = age < 120 ? `${age} seconds` : `${Math.round(age / 60)} minutes`;
115
+ return ` (this copy came from a CDN cache and is ${old} old, so a recent change may not show yet)`;
116
+ }
117
+
118
+ const normText = (t) => String(t ?? '').toLowerCase().replace(/\s+/g, ' ').replace(/[\s.!:;]+$/, '').trim();
119
+
120
+ /** True when an operation's description adds nothing to its summary (A1 notes these). */
121
+ export function repeatsSummary(op) {
122
+ const summary = normText(op?.summary), description = normText(op?.description);
123
+ return Boolean(summary && description) && summary.includes(description);
124
+ }
125
+
126
+ /**
127
+ * True when a field has no description of its own but one of its anyOf, oneOf
128
+ * or allOf members does: where Zod puts it for `.nullable()` when `.describe()`
129
+ * comes first. `deref` resolves $ref members.
130
+ */
131
+ export function hasVariantDescription(schema, deref = (s) => s) {
132
+ if (!schema || schema.description) return false;
133
+ return [].concat(schema.anyOf ?? [], schema.oneOf ?? [], schema.allOf ?? []).some((m) => deref(m)?.description);
134
+ }
135
+
136
+ const SYMBOLS = { '✓': 'pass', '✗': 'fail', '!': 'warn', '–': 'skip' };
137
+
138
+ /**
139
+ * One domain's results from a previous run: a status page (the --matrix
140
+ * output) or a --json report. Returns { date, results: Map(id -> { status,
141
+ * level }) }, or null when the domain isn't in it.
142
+ */
143
+ export function parsePrevious(text, domain) {
144
+ const trimmed = String(text ?? '').trim();
145
+ if (trimmed.startsWith('{') || trimmed.startsWith('[')) {
146
+ let data;
147
+ try { data = JSON.parse(trimmed); } catch { return null; }
148
+ const rep = [].concat(data).find((r) => r?.domain === domain);
149
+ if (!rep || !Array.isArray(rep.results)) return null;
150
+ return { date: String(rep.checked_at ?? '').slice(0, 10) || null, results: new Map(rep.results.map((r) => [r.id, { status: r.status, level: r.level }])) };
151
+ }
152
+ const lines = trimmed.split('\n');
153
+ const header = lines.find((l) => /^\| ID \| Check \| Level \|/.test(l));
154
+ if (!header) return null;
155
+ const cols = header.split('|').slice(1, -1).map((c) => c.trim());
156
+ const col = cols.indexOf(domain);
157
+ if (col < 3) return null;
158
+ const results = new Map();
159
+ for (const line of lines) {
160
+ const cells = line.split('|').slice(1, -1).map((c) => c.trim());
161
+ if (cells.length !== cols.length || !/^[A-Z]\d+$/.test(cells[0])) continue;
162
+ const status = SYMBOLS[cells[col]]; // recorded rows (T5, T6) hold text, not a symbol
163
+ if (status) results.set(cells[0], { status, level: cells[2] });
164
+ }
165
+ return { date: trimmed.match(/^Generated (\d{4}-\d{2}-\d{2})/m)?.[1] ?? null, results };
166
+ }
167
+
168
+ /** What changed between a previous run's results and this run's, as lists of labeled IDs. */
169
+ export function compareResults(previous, results) {
170
+ const rank = { fail: 0, warn: 1, pass: 2 };
171
+ const out = { better: [], worse: [], newlyChecked: [], noLongerChecked: [], levelChanged: [] };
172
+ for (const r of results) {
173
+ const p = previous.get(r.id);
174
+ const was = p && p.status !== 'skip' ? p.status : null;
175
+ const now = r.status !== 'skip' ? r.status : null;
176
+ if (p && p.level && p.level !== r.level) out.levelChanged.push(`${r.id} (${p.level} to ${r.level})`);
177
+ if (!was && now) out.newlyChecked.push(`${r.id} (${r.level}, ${now})`);
178
+ else if (was && !now) out.noLongerChecked.push(r.id);
179
+ else if (was && now && was !== now) (rank[now] > rank[was] ? out.better : out.worse).push(`${r.id} (${was} to ${now})`);
180
+ }
181
+ const current = new Set(results.map((r) => r.id));
182
+ for (const [id, p] of previous) if (!current.has(id) && p.status !== 'skip') out.noLongerChecked.push(id);
183
+ return out;
184
+ }
185
+
186
+ /** The comparison as printable lines. */
187
+ export function formatComparison(cmp, since) {
188
+ const rows = [['Better', cmp.better], ['Worse', cmp.worse], ['Newly checked', cmp.newlyChecked], ['No longer checked', cmp.noLongerChecked], ['Level changed', cmp.levelChanged]]
189
+ .filter(([, ids]) => ids.length);
190
+ if (!rows.length) return [`Since ${since}: no changes.`];
191
+ return [`Since ${since}:`, ...rows.map(([label, ids]) => ` ${label}: ${ids.join(', ')}`)];
192
+ }