readiness-audit 1.0.1 → 1.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +53 -0
- package/README.md +8 -2
- package/STANDARD.md +84 -51
- package/bin/readiness-audit.mjs +36 -9
- package/package.json +1 -1
- package/src/audit.mjs +89 -42
- package/src/helpers.mjs +146 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,58 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.2.1 (2026-10-06)
|
|
4
|
+
|
|
5
|
+
Principle 7 told skills to have the agent confirm with its person before registering, publishing or rotating a key, and S1 required it of every skill that publishes. That gates the core loop: an agent following it can't finish on its own, and a skill limited to explicit requests isn't used when it would help. No scoring changes.
|
|
6
|
+
|
|
7
|
+
- **Principle 7 is now "Tell agents the consequences, then let them act."** Skills, tool descriptions and responses say what becomes public, what can't be undone and what costs money. MCP annotations carry the same facts, and approval policy belongs to the client and whoever runs the agent.
|
|
8
|
+
- **New principle 10, "Caution has a cost, so count it."** Confirmation steps, ask-first lines, narrow triggers and disclaimers cost installs, engagement and organic discovery. Name the harm before adding a restriction, keep "only when asked" for destructive actions, and undo a change that lowers T5 or the come-back measure. Principles 8 and 9 keep their numbers.
|
|
9
|
+
- **S1:** trigger on the job, specifically enough not to fire on unrelated requests, and say what lasting actions do, with no ask-first gates on the core loop. ClawHub's scan flagged drifts' first plugin for broad triggers ("explore, travel"); animalhouse's plugin, with job-specific triggers and no gates on the core loop, scanned clean. The scorecard's S1 review line says the same. botsmatter had already declined the old rule. If your skills have ask-first lines or explicit-only triggers from the old wording, take them out of the core loop; republishing is the owner's call.
|
|
10
|
+
- **T5:** record every point where the agent couldn't go on by itself (`STOPS` in the runner's report). The runner's prompt no longer tells the agent to read the docs and the API, since finding them is part of what T5 measures, so earlier results aren't directly comparable. Record the full model ID: an alias like `sonnet` moves to each new release.
|
|
11
|
+
- **Fewer examples to copy.** Agents adopting the standard turn its examples into their own skills and copy, so rules teach with a generic example, a `[placeholder]` template or a check question, and name a project only as evidence: W1's title pattern, T2's come-back measure, the T5 goal, A8's placeholders (taken from your own docs, not a fixed list), A11, E2's mechanics (now attributed), E6 (applies when you run related sites), S3 (no skill counts; ClawHub flags near-identical skills as template spam) and §12's timestamp gotcha. S1 gains two check questions. README and §13 no longer list skills that act without asking as a gap. ClawHub observations get a dated row in §16.
|
|
12
|
+
- **docs/recipes.md:** route handlers take `params` as a promise (Next.js 15 and later), and Next.js 16 calls middleware `proxy.ts`.
|
|
13
|
+
- **AGENTS.md:** examples get copied; every restriction names the harm it prevents; the standard describes the current rule, and history goes here and in commits.
|
|
14
|
+
- **Examples that fit any project:** the T5 goal, A7, A8's aliases, N2 and W3 no longer use one project's terms. The principles list now says the first six (not seven) came from all six projects.
|
|
15
|
+
- **docs/adopting.md:** a third switch-over rule, that every restriction names the harm it prevents; step 5 says why it asks first; the intro no longer leads with a score.
|
|
16
|
+
|
|
17
|
+
## 1.2.0 (2026-10-06)
|
|
18
|
+
|
|
19
|
+
From drifts.bot's switch-over, the third project to adopt the standard. One new warning (D4), no new failures. It also ships 1.1.0, which wasn't published to npm.
|
|
20
|
+
|
|
21
|
+
- **`--base <url>` scores a build before it ships:** a local server or a preview deployment, as the domain. The site's URLs are requested from the base and reported under the domain; a private address is allowed there because you chose it, while everything else the site links to still has to be public. D10 (DNS) and W2's host redirects are skipped.
|
|
22
|
+
- **D4 follows llms.txt's links.** It samples up to 10 of the site's own links and warns about missing pages (404 or 410), server errors and no answer. Endpoints that want a key (401, 403) are fine, and templates like `{id}` are skipped. A warning lowers the required score until fixed, but doesn't change the exit code.
|
|
23
|
+
- **D10 asks public resolvers first** (1.1.1.1, then 8.8.8.8, then the system's), and its failures say that DNS answers stay cached until their TTL runs out.
|
|
24
|
+
- **The status page names the pinned command** (`npx readiness-audit@1 --matrix ...`), as §14 says to run it.
|
|
25
|
+
- **T6 takes structured numbers** in `recorded.json` (period; clicks, impressions, CTR, position, indexed, not indexed, crawled but not indexed; for Google and Bing), and the status page shows them. A free-text `result` still works.
|
|
26
|
+
- **The T5 runner** takes `--attempts`, `--model`, `--max-turns` and `--pause`, runs each attempt in an empty folder with no user settings or MCP servers (and `--bare` when `ANTHROPIC_API_KEY` is set), gives the agent a random test name, and asks for doc mismatches as well as errors.
|
|
27
|
+
- **STANDARD.md:**
|
|
28
|
+
- A9, A10: a limit on key checks answers 429, never 401, and counts only failures; a database error is 503; cache verified keys when the hash is slow; one auth helper for every route.
|
|
29
|
+
- T1, T3: logs outlive a deploy; page through platform logs.
|
|
30
|
+
- T5: keep test accounts off public pages, keep the agent fresh, budget the runs, report doc mismatches.
|
|
31
|
+
- T6: record fields, not prose; it takes a person with console access.
|
|
32
|
+
- §12 gotchas: Next.js drops a `Vary` set in middleware, proxy or `next.config` on App Router pages (tested on 14.2 and 16.3); `robots.ts` can't output `Content-Signal`; Redocly needs `security: []` on public operations; zod-to-openapi v7 with Zod 3 needed `extendZodWithOpenApi`; container logs vanish on deploy.
|
|
33
|
+
- **docs/recipes.md:** one auth helper for every route, and test accounts kept off public pages.
|
|
34
|
+
- **AGENTS.md:** a new warning is a minor version; a new failure is a major one.
|
|
35
|
+
|
|
36
|
+
## 1.1.0 (2026-10-06, not published to npm; shipped in 1.2.0)
|
|
37
|
+
|
|
38
|
+
From inbed.ai's switch-over, the second project to adopt the standard. Nothing gets stricter.
|
|
39
|
+
|
|
40
|
+
- **D8 (markdown for agents) moves from recommended to next level.** It had no stated reason, our logs can't show `Accept: text/markdown` traffic (Railway's HTTP logs keep the user agent but not `Accept`), and principle 3 says a scanner check alone makes an item recommended at most. It now says why to build it and when (log the `Accept` header to find out), starts with the homepage and docs, and says Cloudflare Pro is optional. The status page gains a "Next level passed" row.
|
|
41
|
+
- **`--compare <file>`** lists what changed since a previous run (a status page from `--matrix`, or a saved `--json` report): better, worse, newly checked, no longer checked, and level changes. A score's denominator grows as items start to apply, so totals alone mislead.
|
|
42
|
+
- **A1 explains two cases.** A field whose description sits inside `anyOf` or `oneOf` (what Zod emits for `.nullable()` when `.describe()` comes first) is still missing a description, and the reason now says so and how to fix it. Operations whose description only repeats the summary are noted on the result line, without failing it.
|
|
43
|
+
- **The Score line carries the date,** so pasted results keep it.
|
|
44
|
+
- **STANDARD.md:**
|
|
45
|
+
- D1: CDN features such as Cloudflare's managed robots.txt prepend their own group.
|
|
46
|
+
- W3: in Next.js, a `loading.tsx` above a detail page makes missing pages answer 200.
|
|
47
|
+
- A1: put descriptions on the field itself, and say when to use each operation.
|
|
48
|
+
- A6: point out unreplaced placeholders such as `{{AGENT_ID}}`.
|
|
49
|
+
- S1 and S2: index the skills the site serves, not every listing; an index makes S1 apply.
|
|
50
|
+
- D10: the AID auth values, with an `a=apikey` example.
|
|
51
|
+
- D5: the template gives llms.txt the type sites actually send (`text/plain`).
|
|
52
|
+
- T5: choose a goal the agent can finish alone.
|
|
53
|
+
- §13: keep the status page in the project's private repo when there is one.
|
|
54
|
+
- **New docs/recipes.md:** a `/.well-known` catch-all for D7 and D11, `GET /api` that sends browsers to the docs, `did_you_mean` by edit distance against the OpenAPI paths, unreplaced placeholders, and markdown routes without a CDN plan.
|
|
55
|
+
|
|
3
56
|
## 1.0.1 (2026-10-05)
|
|
4
57
|
|
|
5
58
|
Fixes from the first full switch-over (animalhouse.ai, 10 to 19 of 19 required).
|
package/README.md
CHANGED
|
@@ -18,7 +18,7 @@ npx readiness-audit example.com
|
|
|
18
18
|
It needs Node 20 or later and has no dependencies. It only reads: GET requests, plus a few POSTs that create nothing (an MCP `initialize`, `server/discover` and `tools/list`, and an empty POST to `/`). It keeps to about six requests at a time, and it won't follow a link the site supplies (in its sitemap, share images, skills, catalog or DNS record) to a private or local address, at any redirect. It prints a pass, warning or failure for each check, with the reason, and exits with code 1 when a required check fails, so it can gate a deploy.
|
|
19
19
|
|
|
20
20
|
```
|
|
21
|
-
readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json]
|
|
21
|
+
readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json] [--compare previous] [--base url]
|
|
22
22
|
readiness-audit --matrix <domain> <domain> ... [--recorded results.json]
|
|
23
23
|
```
|
|
24
24
|
|
|
@@ -28,11 +28,15 @@ readiness-audit --matrix <domain> <domain> ... [--recorded results.json]
|
|
|
28
28
|
| `--no-mcp` | skip the hosted MCP checks |
|
|
29
29
|
| `--no-api` | skip the API checks, for a site with no public API |
|
|
30
30
|
| `--json` | print the results as JSON |
|
|
31
|
+
| `--compare file` | list what changed since a previous run: a status page from `--matrix` or a saved `--json` report |
|
|
32
|
+
| `--base url` | score a local or preview build as the domain before you deploy (`--base http://localhost:3000`); DNS and host redirects are skipped |
|
|
31
33
|
| `--matrix` | score several sites and print a markdown status page |
|
|
32
34
|
| `--recorded file` | with `--matrix`, add your recorded T5 and T6 results (format: [examples/recorded.json](examples/recorded.json)) |
|
|
33
35
|
|
|
34
36
|
From a clone, run `node bin/readiness-audit.mjs example.com`.
|
|
35
37
|
|
|
38
|
+
To check fixes before they ship, score a local build: `npx readiness-audit@1 example.com --base http://localhost:3000`. The site's own URLs (from its sitemap, llms.txt, skills and catalog) are requested from the base and reported under the domain. A private address is allowed for `--base` because you chose it; anything else the site links to still has to be public.
|
|
39
|
+
|
|
36
40
|
## What it checks, and what it can't
|
|
37
41
|
|
|
38
42
|
The scorecard covers what's visible from outside. It checks:
|
|
@@ -45,12 +49,14 @@ The scorecard covers what's visible from outside. It checks:
|
|
|
45
49
|
- a sample of skill files: spec names and descriptions, and links to llms.txt and the API reference;
|
|
46
50
|
- markdown negotiation, the AI catalog, the DNS AID record, the API catalog, auth.md and the skills index.
|
|
47
51
|
|
|
48
|
-
It can't see whether `next_steps` guide well, whether input is forgiving, whether skills
|
|
52
|
+
It can't see whether `next_steps` guide well, whether input is forgiving, whether an agent can follow your skills through the core loop, or whether you have tests. The standard marks those items "review" or "test". The agent usability test (T5) measures what all of it is for: whether a fresh agent, given only your domain and a goal, succeeds. [docs/usability-test.md](docs/usability-test.md) has the procedure and a runner. A high score is not the goal.
|
|
49
53
|
|
|
50
54
|
## Use it in your project
|
|
51
55
|
|
|
52
56
|
Point to [STANDARD.md](STANDARD.md) from your repo instead of copying it (§14 has a short stub), and pin the scorecard to a major version: `npx readiness-audit@1 <domain>`. To run it on a schedule in GitHub Actions, see [examples/github-action.yml](examples/github-action.yml).
|
|
53
57
|
|
|
58
|
+
To switch a project over, point its coding agent at [docs/adopting.md](docs/adopting.md): step-by-step instructions, with lessons from the first project that did it.
|
|
59
|
+
|
|
54
60
|
## Where it comes from
|
|
55
61
|
|
|
56
62
|
Six projects wrote down what they do for agents and search: [animalhouse.ai](https://animalhouse.ai), [inbed.ai](https://inbed.ai), [magnifica.family](https://magnifica.family), [drifts.bot](https://drifts.bot), [achurch.ai](https://achurch.ai) and [botsmatter.live](https://botsmatter.live). The standard merges their notes and settles where they disagreed, using the specs and production request logs. It was revised after each project's own agent reviewed it.
|
package/STANDARD.md
CHANGED
|
@@ -33,7 +33,7 @@ This is the common standard our projects work toward, published for anyone who b
|
|
|
33
33
|
|
|
34
34
|
## 1. Principles
|
|
35
35
|
|
|
36
|
-
The first
|
|
36
|
+
The first six came from all six projects independently.
|
|
37
37
|
|
|
38
38
|
1. **One source per fact.** The site URL, install lines, counts and request schemas each live in one module. Every discovery file, card, manifest, skills index and MCP tool list is generated from it. (inbed keeps the facts in `agent-discovery.ts`; at animalhouse the OpenAPI registry generates the spec, the MCP tools and the server card.)
|
|
39
39
|
2. **Only declare what's true.** A manifest is a promise a client will act on. An A2A card with no A2A endpoint, OAuth metadata for an authorization server that doesn't exist, or a catalog entry typed as something it isn't sends agents into a dead end, and validators flag it. Answer the paths you don't support with a JSON 404 that points to what you do run. When a scanner's score and the truth disagree, the truth wins ([What scanners check](#what-scanners-check)).
|
|
@@ -41,9 +41,10 @@ The first seven came from all six projects independently.
|
|
|
41
41
|
4. **Guard every surface with a test you've watched fail.** Untested discovery files drift without anyone noticing. A magnifica skills index advertised a parameter the API rejects. The animalhouse `agents.txt` still said "there is no hosted MCP" after the hosted endpoint shipped.
|
|
42
42
|
5. **Make the first call worth it, and point every response at the next one:** `next_steps` with ready-to-send bodies, and errors carrying a `suggestion` that names the call that fixes them.
|
|
43
43
|
6. **Be forgiving at the edge and strict where it counts.** Accept aliases, any `Accept` header and guessable wrong paths. Never guess on a write.
|
|
44
|
-
7. **
|
|
44
|
+
7. **Tell agents the consequences, then let them act.** Skills, tool descriptions and responses say what becomes public, what can't be undone and what costs money. MCP annotations (`readOnlyHint`, `destructiveHint`) carry the same facts in a form clients act on, so any approval policy belongs to the client and whoever runs the agent. An agent should be able to finish your core loop on its own, and T5 measures whether it can.
|
|
45
45
|
8. **Raise the floor together.** When one project finds a better way, it goes into this standard, and the others adopt it. The scorecard and the status page make the gaps visible.
|
|
46
46
|
9. **A project's values come first.** A project may decline items that conflict with what it stands for. achurch.ai rules out engagement mechanics, recommendation algorithms and analytics beyond aggregate traffic, so it declines those parts of E2, E6 and T2. A decision written down with its reason counts as meeting the item. Silence doesn't.
|
|
47
|
+
10. **Caution has a cost, so count it.** Every confirmation step, ask-first line, narrow trigger, disclaimer or feature pulled "to be safe" costs the product something. An agent told to stop and ask can't finish the core loop, and a skill that waits for a request naming your product isn't used when it would help. That shows up as fewer installs, less engagement and less organic discovery, which are the reasons to run the project at all. Name the harm before adding a restriction, and keep "only when asked" for actions that destroy something, like deleting data or rotating a key. If T5's success rate, the come-back measure (T2) or installs drop after a change, undo it. Coding agents lean cautious by default, so when you adopt this standard, add only the restrictions it asks for.
|
|
47
48
|
|
|
48
49
|
---
|
|
49
50
|
|
|
@@ -104,9 +105,10 @@ What it means:
|
|
|
104
105
|
| takes payments from agents | that payment protocol's manifest | |
|
|
105
106
|
| is free | a JSON 404 at the payment paths (`x402`, `mpp`, `payment-manifest`) saying so | payment manifests |
|
|
106
107
|
| has endpoints with side effects | a robots `Disallow` for each one, so a crawler can't trigger it | |
|
|
107
|
-
| has agents that write or post |
|
|
108
|
+
| has agents that write or post | S1's consequence statements and the honesty rules in A11 | ask-first gates on the core loop (principle 10) |
|
|
108
109
|
| has many generated pages (profiles, items, versions) | W3's one indexability rule, and W6's unique paragraph per page | indexing every version or empty profile |
|
|
109
110
|
| serves several languages | W11 | |
|
|
111
|
+
| runs related sites for the same audience | E6 | |
|
|
110
112
|
| rules out engagement mechanics | the items in E2 that fit its values, with the rest declined in writing (principle 9) | |
|
|
111
113
|
|
|
112
114
|
---
|
|
@@ -122,6 +124,7 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
|
|
|
122
124
|
- **Disallow what has side effects or private data.** For read-only API endpoints that live-fetch agents need (public stats, a self-documenting `GET /api/auth/register`), allow the fetch and send `X-Robots-Tag: noindex` on the JSON instead, so search engines skip it without blocking agents (botsmatter, inbed).
|
|
123
125
|
- **Allow your share-card path** (`/api/og/` at animalhouse, `/og/` at botsmatter), and add the `Sitemap:` line.
|
|
124
126
|
- **Purge it from your CDN after a change.** Cloudflare caches robots.txt by default, so crawlers keep the old rules until the cached copy expires. The scorecard says when it read a cached copy.
|
|
127
|
+
- **Watch for CDN features that rewrite it.** Cloudflare's managed robots.txt, and features like it, put their own group at the top of your file, which can break D1 and D15. Leave them off, or check robots.txt after turning one on.
|
|
125
128
|
- Naming AI crawlers and stating a Content-Signal policy is D15 (recommended).
|
|
126
129
|
- **Done when** the scorecard passes D1.
|
|
127
130
|
|
|
@@ -129,7 +132,7 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
|
|
|
129
132
|
|
|
130
133
|
**D3 Server-rendered pages.** Every page's content is in the HTML the server sends, not loaded later by script, with a real logo through the framework's icon convention and a favicon set (missing icons show up as steady 404s). *Why: crawlers, link-preview bots and agents that fetch pages read the HTML, and many don't run JavaScript.* inbed's chat pages once shipped none of their 48 messages in the HTML. The page-level details (titles, canonicals, structured data, previews, headings, links) are W1 to W7 in [§5](#5-search-people-and-answer-engines).
|
|
131
134
|
|
|
132
|
-
**D4 `/llms.txt` and `/llms-full.txt`**: a short map (what this is, how to start, links to every other surface) and the whole corpus in one fetch. Generate both and test them against the sitemap. The file is markdown; serve it as `text/markdown` or `text/plain`, and use the same type wherever you link it.
|
|
135
|
+
**D4 `/llms.txt` and `/llms-full.txt`**: a short map (what this is, how to start, links to every other surface) and the whole corpus in one fetch. Generate both and test them against the sitemap: agents follow the links, so every one must open. The scorecard samples them and warns about missing pages (an endpoint that answers 401 for want of a key is fine; templates like `{id}` are skipped). The file is markdown; serve it as `text/markdown` or `text/plain`, and use the same type wherever you link it.
|
|
133
136
|
- *Why required: traffic, and not from Google.* Google Search ignores llms.txt (it said so in June 2026), but plenty else reads it. At animalhouse, 79 requests in four and a half days came from:
|
|
134
137
|
- agent directories (AgenstryBot, BrickBlueBot, agentprobe, MCPHarbor, AgentTrustBot);
|
|
135
138
|
- llms.txt directories and validators;
|
|
@@ -160,15 +163,16 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
|
|
|
160
163
|
|
|
161
164
|
**A1 OpenAPI 3.1 at `/openapi.json`.** *Why: working agents fetch it (curl was the top client in our logs).*
|
|
162
165
|
- Generated from the same source that validates requests: Zod at animalhouse and inbed, plain JavaScript definitions at botsmatter.
|
|
163
|
-
- Every operation has a description saying when and why to call it. A one-line summary isn't enough.
|
|
166
|
+
- Every operation has a description saying when and why to call it. A one-line summary isn't enough, and the scorecard notes descriptions that only repeat the summary. A "When to use" line per operation in your API reference makes a good source, and agents reading the markdown get it too (inbed).
|
|
164
167
|
- Every parameter and request field is described. Response fields too, once you have response schemas (N7).
|
|
165
|
-
-
|
|
168
|
+
- Put each description on the field itself. One inside `anyOf` or `oneOf` counts as missing, because many clients show only the field's own description. Zod puts it there for `.nullable()` when `.describe()` comes first, so call `.describe()` last, after `.optional()` and `.nullable()`.
|
|
169
|
+
- **Done when** the scorecard passes A1 and `npx @redocly/cli lint` passes. Redocly fails public operations that don't declare `security: []` when the spec has a global security scheme. Checking the spec against real responses needs response schemas, so that contract test lives with N7.
|
|
166
170
|
|
|
167
|
-
**A2 `GET /api`**: a JSON index of every operation, generated from the spec. *Why: agents ask for it (19 requests in four days at animalhouse, curl most often).*
|
|
171
|
+
**A2 `GET /api`**: a JSON index of every operation, generated from the spec. *Why: agents ask for it (19 requests in four days at animalhouse, curl most often).* A browser asking for HTML can be redirected to the docs ([recipe](docs/recipes.md#get-api-json-for-agents-the-docs-for-browsers-a2)).
|
|
168
172
|
|
|
169
|
-
**A6 Errors that teach**: `next_steps` and a `suggestion` on every response, errors included ([§8](#a6-every-response-teaches)). *Why: working agents rely on it.* The scorecard tests one case: an authenticated operation called without a key must answer JSON that says how to get one.
|
|
173
|
+
**A6 Errors that teach**: `next_steps` and a `suggestion` on every response, errors included ([§8](#a6-every-response-teaches)). *Why: working agents rely on it.* The scorecard tests one case: an authenticated operation called without a key must answer JSON that says how to get one. Point out unreplaced placeholders too: an id like `{{AGENT_ID}}` copied from your docs reaches the real route, not the catch-all, so its not-found answer should say it looks like a placeholder ([recipe](docs/recipes.md#unreplaced-placeholders-in-ids-a6)).
|
|
170
174
|
|
|
171
|
-
**A7 A JSON catch-all for wrong API paths** with `next_steps`, a `suggestion` or `did_you_mean` ([§8](#forgiving-input-strict-mutations)). *Why: agents guess paths; animalhouse built its catch-all from the 404s in its logs.*
|
|
175
|
+
**A7 A JSON catch-all for wrong API paths** with `next_steps`, a `suggestion` or `did_you_mean` ([§8](#forgiving-input-strict-mutations)). *Why: agents guess paths; animalhouse built its catch-all from the 404s in its logs.* `did_you_mean` can come from edit distance against your OpenAPI paths ([recipe](docs/recipes.md#did_you_mean-from-the-openapi-paths-a7)).
|
|
172
176
|
|
|
173
177
|
### Recommended
|
|
174
178
|
|
|
@@ -177,23 +181,14 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
|
|
|
177
181
|
```
|
|
178
182
|
Link: </openapi.json>; rel="service-desc"; type="application/vnd.oai.openapi+json",
|
|
179
183
|
</docs/api>; rel="service-doc"; type="text/html",
|
|
180
|
-
</llms.txt>; rel="describedby"; type="text/
|
|
184
|
+
</llms.txt>; rel="describedby"; type="text/plain",
|
|
181
185
|
</.well-known/api-catalog>; rel="api-catalog"; type="application/linkset+json"
|
|
182
186
|
```
|
|
187
|
+
- Give each link the type your server actually sends for that URL. llms.txt is usually `text/plain`.
|
|
183
188
|
- `service-desc` is the machine-readable description (your OpenAPI). `service-doc` is the human documentation (RFC 8631). Five of the six projects fail D5: two send no `Link` header, one points `service-desc` at its skills index, and two are missing a rel.
|
|
184
189
|
- Registered relations are `describedby`, `service-desc`, `service-doc`, `service-meta` and `api-catalog`. `ard`, `ai-catalog`, `llms-txt`, `mcp`, `agent-card` and `agent-skills` aren't registered: fine to use, but generic clients won't know them.
|
|
185
190
|
|
|
186
|
-
**D7 JSON 404s for unknown `/.well-known/*` paths, and a JSON 405 on `POST /`** naming the MCP endpoint and the API (achurch.ai, magnifica). An HTML 404 page is noise to an agent. *Why: cheap, and it helps the agents and crawlers that guess. No project passes all of it yet.*
|
|
187
|
-
|
|
188
|
-
**D8 Markdown for agents**
|
|
189
|
-
- `Accept: text/markdown` and a `.md` URL both return the page as markdown.
|
|
190
|
-
- Send `Vary: Accept` on both representations (RFC 9110 §12.5.5). Otherwise a cache can hand HTML readers the markdown, or the reverse.
|
|
191
|
-
- Next.js 14.2 replaces `Vary` on App Router pages with its own value, so middleware and `next.config` headers can't add `Accept` to the HTML. Add it at the edge (a CDN response header rule), and send it from the markdown route yourself (animalhouse.ai).
|
|
192
|
-
- Make links absolute, and link back to the HTML page as canonical.
|
|
193
|
-
- Add `x-markdown-tokens` so an agent can budget before reading. It's Cloudflare's header, with no spec behind it.
|
|
194
|
-
- Negotiate in front of prerendered pages too, or they ignore it (magnifica).
|
|
195
|
-
- Whole-corpus indexes help: `/docs/index.md` and `/docs/index.json` (achurch.ai).
|
|
196
|
-
- Behind Cloudflare (Pro and up), its Markdown for Agents feature does the conversion and sends these headers for you.
|
|
191
|
+
**D7 JSON 404s for unknown `/.well-known/*` paths, and a JSON 405 on `POST /`** naming the MCP endpoint and the API (achurch.ai, magnifica). An HTML 404 page is noise to an agent. *Why: cheap, and it helps the agents and crawlers that guess. No project passes all of it yet.* One catch-all route can answer D7 and D11 together ([recipe](docs/recipes.md#one-catch-all-for-unknown-well-known-paths-d7-d11)).
|
|
197
192
|
|
|
198
193
|
**D9 An AI catalog (ARD)**
|
|
199
194
|
- One list of what an agent can use here, such as the MCP server card, the OpenAPI document, skills and llms.txt, with identifiers in the form `urn:air:<publisher-domain>:<namespace>:<name>`.
|
|
@@ -204,7 +199,9 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
|
|
|
204
199
|
**D10 A DNS AID record**
|
|
205
200
|
- One TXT record at `_agent.<domain>` (AID v2.1.1, `v=aid2`). Point it at your main agent entry point: the hosted MCP endpoint if you have one (`p=mcp`), otherwise the OpenAPI document (`p=openapi`).
|
|
206
201
|
- `v`, `u` and `p` are required and `a` is recommended. `s` is a description of up to 60 bytes; `d` is the docs URL.
|
|
202
|
+
- `a` is the auth hint. AID v2.1.1 allows `none`, `pat`, `apikey`, `basic`, `oauth2_device`, `oauth2_code`, `mtls` and `custom`. For an API that takes keys: `v=aid2;u=https://example.com/openapi.json;p=openapi;a=apikey;s=Example API;d=https://example.com/docs/api`.
|
|
207
203
|
- Exactly one record: AID clients fail on ambiguity when there are two.
|
|
204
|
+
- After a change, resolvers keep the old answer until its TTL runs out, so an old record can show for minutes. The scorecard asks public resolvers first and says so when D10 fails.
|
|
208
205
|
- Only name a protocol you serve (allowed: `mcp`, `a2a`, `openapi`, `grpc`, `graphql`, `websocket`, `local`, `zeroconf`, `ucp`). The scorecard fails a record that names a service you don't run.
|
|
209
206
|
- Scanners check DNS-AID, a different spec ([Only if true](#only-if-true)), so an AID record earns no scanner credit. It's for AID-aware clients.
|
|
210
207
|
|
|
@@ -222,6 +219,7 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
|
|
|
222
219
|
**D15 Name the AI crawlers you welcome, and state a Content-Signal policy.** *Why: it states intent, and scanners check it. It's Cloudflare's policy, not a standard, and Google ignores it.*
|
|
223
220
|
- Training, search and live-fetch bots are separate user agents: GPTBot, OAI-SearchBot and ChatGPT-User; ClaudeBot, Claude-SearchBot and Claude-User; PerplexityBot and Perplexity-User; Google-Extended; Applebot-Extended; and so on.
|
|
224
221
|
- `Content-Signal: search=yes, ai-input=yes, ai-train=yes` (or your actual choices) in every group: a line in the `*` group never reaches a bot with its own group.
|
|
222
|
+
- In Next.js, `robots.ts` can't output `Content-Signal`, so serve robots.txt from a route handler (animalhouse, drifts).
|
|
225
223
|
- Watch the IETF's AIPREF drafts, which define a successor (a `Content-Usage` rule and header with the vocabulary `train-ai`, `ai-use`, `search` and values `y`/`n`).
|
|
226
224
|
|
|
227
225
|
**A3 `/.well-known/api-catalog`** (RFC 9727): a linkset served as `application/linkset+json; profile="https://www.rfc-editor.org/info/rfc9727"`. Serve it from a route, because a prerendered extensionless file loses its content type (magnifica). *Why: baseline hygiene. It's an RFC and scanners check it, but one crawler asked for it in four days.*
|
|
@@ -241,7 +239,7 @@ Each item says what it is, why it's at its level, and how you know it's done. Se
|
|
|
241
239
|
| `$schema` | `https://schemas.agentskills.io/discovery/0.2.0/schema.json` |
|
|
242
240
|
| `skills[]` | each entry has `name`, `type` (`skill-md` or `archive`), `description` (up to 1,024 characters), `url`, and `digest` (`sha256:` plus 64 lowercase hex) |
|
|
243
241
|
|
|
244
|
-
The `$schema` value is an identifier, not a link (it doesn't have to resolve). Clients skip an index whose `$schema` they don't recognize, so a wrong value hides every skill. Extra fields are ignored. Details in [§7](#7-skills-and-plugins).
|
|
242
|
+
List the skills your site serves ([S1](#s1-skill-files)). The `$schema` value is an identifier, not a link (it doesn't have to resolve). Clients skip an index whose `$schema` they don't recognize, so a wrong value hides every skill. Extra fields are ignored. Details in [§7](#7-skills-and-plugins).
|
|
245
243
|
|
|
246
244
|
**M5 An MCP server card** for a hosted endpoint ([§6](#m5-the-server-card)). Recommended, not required, because the extension is experimental.
|
|
247
245
|
|
|
@@ -300,7 +298,7 @@ The evidence that this work pays: achurch.ai's August batch rewrote titles and s
|
|
|
300
298
|
**W1 Titles and descriptions.** *Why: they're the snippet people choose from (achurch's click-through rate rose after the rewrite).*
|
|
301
299
|
- **One helper builds every page's metadata:** title, description, canonical, Open Graph and Twitter tags (drifts' `pageMetadata()`, achurch's page-meta builder, inbed's `generateMetadata`). Change it in one place.
|
|
302
300
|
- **Unique per URL, generated from the page's data.** botsmatter's 1,937 Ground versions once shared 43 titles.
|
|
303
|
-
- **Titles name what people search for**, with the brand last after a
|
|
301
|
+
- **Titles name what people search for**, with the brand last after a `|`: `[what the page offers] | [brand]`.
|
|
304
302
|
- **Descriptions of about 140 to 160 characters,** built from whole sentences or cut at a word boundary, never mid-word.
|
|
305
303
|
- **Lengths are a guide, not a rule.** Google sets no limit: it cuts titles and snippets to the screen's width and may rewrite them. Titles under about 70 characters usually show in full. The scorecard warns past 70 and outside 50 to 160.
|
|
306
304
|
- **Set them on the server.** A client-side `document.title` overwrites the server's title after load, so people and crawlers see different titles (achurch).
|
|
@@ -315,8 +313,9 @@ The evidence that this work pays: achurch.ai's August batch rewrote titles and s
|
|
|
315
313
|
|
|
316
314
|
**W3 One rule for what's indexable.** *Why: thin, duplicate and test pages fill Google's "crawled, not indexed" bucket and drag down how the rest of the site is judged.*
|
|
317
315
|
- One rule decides which pages are indexable, and the sitemap, the page's robots meta and internal recommendation lists all use it (inbed's `indexable()` SQL column; at animalhouse the sitemap's filters are mirrored by `noindex`).
|
|
318
|
-
- Not indexable: test and template accounts, empty profiles,
|
|
316
|
+
- Not indexable: test and template accounts ([recipe](docs/recipes.md#test-accounts-out-of-public-pages-t5-w3)), empty profiles, unfinished or ephemeral items, filtered views, sign-in pages, and thin pages (inbed: chats with fewer than five messages). They still render, with `noindex`, and stay out of the sitemap. Google has no `follow` rule (following links is the default), so don't count on `noindex` pages to get other pages found: link those from indexable pages (W7).
|
|
319
317
|
- Missing items return a real 404 (or 410; Google treats them the same), never a 200 page that says "not found". Google calls that a soft 404 and drops it (achurch fixed a silent fallback). Empty filter combinations and page numbers past the end get a 404 too.
|
|
318
|
+
- In Next.js, a `loading.tsx` above a detail page streams a 200 before `notFound()` runs, so every missing item answers 200. inbed moved its root `loading.tsx` off the profile routes, and missing profiles went from 200 to 404 in production. Keep loading boundaries off the routes of detail pages; the scorecard's missing-page probe shows whether it worked.
|
|
320
319
|
- The sitemap lists only indexable canonical URLs that answer 200 without a redirect. Google and Bing both ignore `changefreq` and `priority`, and both use `lastmod` only when it's the real date of a meaningful change (D2). Bing asks for ISO 8601 with a time. Leave `lastmod` off where it would churn: drifts omits it on profiles, whose activity changes every few minutes.
|
|
321
320
|
- **Done when** the scorecard passes W3.
|
|
322
321
|
|
|
@@ -387,7 +386,7 @@ magnifica does this for 15 languages.
|
|
|
387
386
|
|
|
388
387
|
The test is whether most of the six projects should do it. These came from one or two projects and don't pass:
|
|
389
388
|
|
|
390
|
-
- **Share actions to X or Moltbook in `next_steps`** (inbed): untested elsewhere
|
|
389
|
+
- **Share actions to X or Moltbook in `next_steps`** (inbed): untested elsewhere. Where a project adds them, the step says the post is public (principle 7).
|
|
391
390
|
- **Commit messages that end with a call to action:** a team habit; neither search nor agents read them.
|
|
392
391
|
- **Keyword-rich image file names** (drifts): little measurable effect.
|
|
393
392
|
- **A web manifest and a light/dark `theme-color`:** nice for installability, no effect on search.
|
|
@@ -406,7 +405,7 @@ The test is whether most of the six projects should do it. These came from one o
|
|
|
406
405
|
|
|
407
406
|
- **Make it a thin adapter over the same operations as the REST API.** One tool per operation, with the same names, generated from the OpenAPI spec where you can (animalhouse) and held to it by parity tests (achurch.ai checks that each tool returns exactly what its REST twin does, in both protocol eras).
|
|
408
407
|
- **Write descriptions for the model:** when to call the tool, what it returns, what becomes public and what stays private. Starting with `Wraps METHOD /path.` lets `next_steps` map straight to a tool.
|
|
409
|
-
- **Annotate:** `readOnlyHint` on reads and `destructiveHint` on irreversible actions, so
|
|
408
|
+
- **Annotate:** `readOnlyHint` on reads and `destructiveHint` on irreversible actions, so each host can apply its own policy: reads run freely, and irreversible actions get whatever its operator chose (principle 7).
|
|
410
409
|
- **Turn API errors into tool results** with `isError: true`, carrying the API's own words. An MCP endpoint should never answer HTTP 401, because a 401 sends clients into OAuth discovery.
|
|
411
410
|
- **Put `next_steps` in every result,** each naming the tool that takes it (achurch.ai).
|
|
412
411
|
- **Add prompts and resources:** prompts for getting started and for the core loop; resources such as `about` and any doc by path.
|
|
@@ -475,15 +474,18 @@ The test is whether most of the six projects should do it. These came from one o
|
|
|
475
474
|
- Keep the body under 500 lines and about 5,000 tokens (the spec's recommendation); ClawHub practice is under 20 KB.
|
|
476
475
|
- One skill per job or intent; never reshuffled duplicates.
|
|
477
476
|
- **Keep display titles out of `name`.** ClawHub ranks on the display title, and its CLI takes that title separately (`clawhub publish --name "..."`; animalhouse's publish script passes the H1). So the frontmatter `name` can follow the spec. Skills that ship inside a plugin must: Claude Code, Codex and Cursor load them.
|
|
478
|
-
- **Trigger
|
|
477
|
+
- **Trigger on the job, and say what lasting actions do.** The description names the job specifically enough that an unrelated request can't match it, and it applies to any request that fits the job. The body says what becomes public, what's saved on the machine and what can't be undone, and an agent can follow it through the core loop on its own (principle 10). Keep "only when the user asks" for actions that destroy something, like deleting data or rotating a key.
|
|
478
|
+
- **Check it:** could a request that has nothing to do with your product match the description? Could an agent with no person present finish the core loop by following the skill?
|
|
479
|
+
- **What ClawHub's scan did:** it rated drifts' first plugin suspicious for broad triggers: general activities that unrelated requests also match. animalhouse's plugin can do the same things (register a persistent account, save a key, create public pages); its triggers name its job, it states the consequences, and an agent runs the core loop without stopping. It scanned clean. That's one case each, so if a scan flags you, make the smallest change that passes and record what it was.
|
|
479
480
|
- **Link the map.** Every skill links llms.txt and the API or MCP reference (`/docs/api.md`, `/openapi.json` or your MCP docs), because an agent's fetch tool only reads what it's pointed at (D4). The scorecard checks this on a sample of skills from your index. At animalhouse, the core skill and both plugin skills linked neither.
|
|
480
|
-
- **Serve the raw file**
|
|
481
|
+
- **Serve the raw file** at a stable URL so an agent can install it with one fetch (`/.well-known/agent-skills/<name>/SKILL.md` keeps the folder name in the URL), plus a `/skills` page. Index them in S2, with each digest computed from the bytes you serve.
|
|
482
|
+
- **Index the skills the site serves, not every listing.** inbed lists about 95 skills on ClawHub and indexes the 6 its site serves; animalhouse indexes 93 of 176, leaving out keyword variants of its core guide. Publishing an index makes S1 apply, so the required count goes up by one. Editing skills to pass it means republishing them and syncing any plugin copies, which is the owner's call.
|
|
481
483
|
|
|
482
484
|
### S3 Marketplaces and per-item skills
|
|
483
485
|
|
|
484
486
|
*Recommended.*
|
|
485
487
|
|
|
486
|
-
- **Per-item skills** can each rank for their own searches: one
|
|
488
|
+
- **Per-item skills** can each rank for their own searches: one for each item people look for by name (animalhouse, inbed and drifts publish them). Give each its own display name and content, with no keyword stuffing: ClawHub flags near-identical skills as template spam.
|
|
487
489
|
- **ClawHub rules:**
|
|
488
490
|
- Skills and plugins share one namespace.
|
|
489
491
|
- New releases stay hidden until the security scan passes.
|
|
@@ -532,16 +534,17 @@ The test is whether most of the six projects should do it. These came from one o
|
|
|
532
534
|
*Required.*
|
|
533
535
|
|
|
534
536
|
- **A7** A JSON catch-all for unknown API paths:
|
|
535
|
-
-
|
|
537
|
+
- An action name used as a path gets a 404 that names the right call (animalhouse answers `POST /api/house/feed` with `POST /api/house/care` and `{"action": "feed"}`).
|
|
536
538
|
- A real operation's name in the wrong place gets `did_you_mean`.
|
|
537
539
|
- A cut-off link gets a 308 to the real path, with a relative `Location`.
|
|
538
|
-
- **A8** Accept the aliases agents guess: `id` or `
|
|
540
|
+
- **A8** Accept the aliases agents guess: `id` or the full field name (`post_id`); a UUID, slug or display name; `Bearer` in any case (one of our projects accepted only the exact capitalization); a bare key; `X-API-Key`. Truncate over-long text with a warning instead of rejecting it, and reject a value copied from a placeholder in your own docs or examples, with a clear message. Reads may default; writes never guess: with several possible targets and none named, return a 400 with the list. Help polling agents with `since` filters, `total_pages` and a recommended next check-in time.
|
|
539
541
|
|
|
540
542
|
### A9 Limits machines can read
|
|
541
543
|
|
|
542
544
|
*Required.*
|
|
543
545
|
|
|
544
546
|
- `Retry-After` on every 429 (RFC 6585 defines 429, RFC 9110 defines `Retry-After`).
|
|
547
|
+
- **A limit on key checks answers 429, never 401.** A 401 tells an agent with a valid key that its key is bad, and it may register again (drifts' busy agents got "include your API key"). Count only failed checks toward that limit.
|
|
545
548
|
- A usage endpoint (inbed's `GET /api/rate-limits`), and the limits listed in llms.txt and auth.md.
|
|
546
549
|
- `RateLimit` headers on every response are the next level (N10).
|
|
547
550
|
|
|
@@ -549,11 +552,15 @@ The test is whether most of the six projects should do it. These came from one o
|
|
|
549
552
|
|
|
550
553
|
*Required where agents register.* Self-service registration with no human in the loop, a rotation endpoint, and a `source` attribution tag for each entry point plus the User-Agent. Never fail a sign-up over a bad tag.
|
|
551
554
|
|
|
555
|
+
- **Failing to check a key isn't a bad key.** A database error while verifying answers 503 with `Retry-After`, not 401.
|
|
556
|
+
- **With a slow hash such as bcrypt, cache verified keys** for a minute or so, keyed by a fast hash of the key, and drop the entry when the key rotates (inbed, drifts). Otherwise every request pays a full compare, and junk keys cost the same.
|
|
557
|
+
- **One auth helper answers for every route.** It returns the user or the response to send (`{ user } | { response }`), so every route answers the same way ([recipe](docs/recipes.md#one-auth-helper-for-every-route-a9-a10)).
|
|
558
|
+
|
|
552
559
|
### A11 Honesty and privacy
|
|
553
560
|
|
|
554
561
|
*Required.*
|
|
555
562
|
|
|
556
|
-
- Say what becomes public. Keep private reads
|
|
563
|
+
- Say what becomes public. Keep private reads, such as a search, separate from public writes, such as a post.
|
|
557
564
|
- One function strips private fields from every response. A hand-rolled strip in one of our projects once let private fields through.
|
|
558
565
|
- Search and answers draw only on pages the site serves, so every citation opens (achurch.ai).
|
|
559
566
|
- Agent contributions that change shared content go through review (achurch.ai's contributions open a pull request).
|
|
@@ -563,11 +570,11 @@ The test is whether most of the six projects should do it. These came from one o
|
|
|
563
570
|
*Required: give agents some reason to return. Which mechanics you use is your choice, and any of them may be declined (principle 9).*
|
|
564
571
|
|
|
565
572
|
- **Time:**
|
|
566
|
-
- A real-time clock, so stats decay while the agent is away.
|
|
567
|
-
- Time-locked steps with a teaser and an unlock time.
|
|
568
|
-
- A daily prompt.
|
|
573
|
+
- A real-time clock, so stats decay while the agent is away (animalhouse).
|
|
574
|
+
- Time-locked steps with a teaser and an unlock time (drifts).
|
|
575
|
+
- A daily prompt (botsmatter).
|
|
569
576
|
- Keep the first steps unlocked: at drifts, a lock right after step 1 is where most travelers left.
|
|
570
|
-
- **Something the agent owns:** a postcard assembled from its own reflections
|
|
577
|
+
- **Something the agent owns:** a public profile, a postcard assembled from its own reflections (drifts), a portrait gallery or a gravestone (animalhouse).
|
|
571
578
|
- **An inhabited world:** platform activity (inbed's `room`, animalhouse's house activity), excerpts from others' reflections on the same step, anonymous popularity.
|
|
572
579
|
- **Progress and anticipation** in varied wording, and recommendations with one slot reserved for something new.
|
|
573
580
|
- **Continuity for agents without memory:** `while_you_were_away`, `your_recent`, a scheduling hint (`recommended_checkin`), and a heartbeat that keeps an active agent ranked as active (inbed).
|
|
@@ -601,15 +608,25 @@ The test is whether most of the six projects should do it. These came from one o
|
|
|
601
608
|
|
|
602
609
|
### E6 Point to the sibling projects
|
|
603
610
|
|
|
604
|
-
*Recommended
|
|
611
|
+
*Recommended when you run related sites for the same audience.* List them in llms.txt and the `/for-agents` page; for some projects (achurch.ai) that's enough. In API responses, mention a sibling only where it's relevant to what the agent just did. inbed links one in about 30% of responses; at that rate a link can read as an ad.
|
|
605
612
|
|
|
606
613
|
---
|
|
607
614
|
|
|
608
615
|
## 9. Next level
|
|
609
616
|
|
|
610
|
-
Higher-value features that most of our projects don't have yet. Each one fixes a failure we've seen or can predict. Build one when your own traffic shows the problem: botsmatter checked its logs and found duplicate writes too rare to need N2.
|
|
617
|
+
Higher-value features that most of our projects don't have yet. Each one fixes a failure we've seen or can predict. Build one when your own traffic shows the problem: botsmatter checked its logs and found duplicate writes too rare to need N2.
|
|
611
618
|
|
|
612
|
-
**
|
|
619
|
+
**D8 Markdown for agents.** Markdown costs an agent fewer tokens than HTML, and some agent fetchers ask for it with `Accept: text/markdown`. But none of our logs can show that traffic yet: Railway's HTTP logs, which several of our projects use, keep the user agent but not the `Accept` header. And a scanner check alone doesn't make an item recommended (principle 3). *Build it when* your logs show agents asking: log the `Accept` header in middleware to find out ([recipe](docs/recipes.md#markdown-for-agents-without-a-cdn-plan-d8)).
|
|
620
|
+
- Start with the homepage and your docs, not every page. `Accept: text/markdown` and a `.md` URL both return the page as markdown. animalhouse serves llms.txt as the homepage's markdown, and its API reference at `/docs/api.md`.
|
|
621
|
+
- No CDN plan is needed: a `.md` route and a few lines of middleware do it ([recipe](docs/recipes.md#markdown-for-agents-without-a-cdn-plan-d8)). Cloudflare's Markdown for Agents feature (Pro and up) converts pages for you, but it's optional.
|
|
622
|
+
- Send `Vary: Accept` on both representations (RFC 9110 §12.5.5). Otherwise a cache can hand HTML readers the markdown, or the reverse.
|
|
623
|
+
- Next.js drops a `Vary` set in middleware, proxy or `next.config` on App Router pages, static or dynamic, while it keeps the other headers (tested on 14.2 and 16.3). So `Accept` has to reach the HTML from the edge (a CDN response header rule), or decline that half in writing. Send it from the markdown route yourself (animalhouse.ai).
|
|
624
|
+
- Make links absolute, and link back to the HTML page as canonical.
|
|
625
|
+
- Add `x-markdown-tokens` so an agent can budget before reading. It's Cloudflare's header, with no spec behind it.
|
|
626
|
+
- Negotiate in front of prerendered pages too, or they ignore it (magnifica).
|
|
627
|
+
- Whole-corpus indexes help: `/docs/index.md` and `/docs/index.json` (achurch.ai).
|
|
628
|
+
|
|
629
|
+
**N2 `Idempotency-Key` on writes** (IETF draft). Agents retry after timeouts, which can mean a duplicate registration or the same action taken twice. Accept the header on every POST, store the first response for 24 hours, and replay it for a repeated key. *Build it when* your logs show the same write repeated within minutes.
|
|
613
630
|
|
|
614
631
|
**N3 Conditional requests on polled reads.** Send an `ETag` (or `Last-Modified`) and answer `If-None-Match` with a 304, so an unchanged check-in costs almost nothing on either side. *Build it when* scheduled polling is a large share of your traffic.
|
|
615
632
|
|
|
@@ -640,21 +657,27 @@ Higher-value features that most of our projects don't have yet. Each one fixes a
|
|
|
640
657
|
- **T1 Logs** (required):
|
|
641
658
|
- MCP log lines: for stdio servers, the User-Agent on every API call; for a hosted endpoint, the protocol era, method, tool and client.
|
|
642
659
|
- Structured request and error logs (drifts writes JSONL, rotated daily) and admin analytics.
|
|
643
|
-
-
|
|
660
|
+
- Logs that outlive a deploy. Logs written inside the container vanish with it: after one deploy, drifts had five minutes of history. Keep them on a volume, ship them out, or use the platform's own HTTP logs.
|
|
661
|
+
- **T2 Attribution and one come-back measure** (required; per-agent tracking may be declined). Record `source` plus User-Agent per entry point, and judge entry points by who comes back, not just who signs up. A come-back measure is the share of agents who return to the core action a day or more after their first success: at animalhouse, the share of adoptions that get care again 24 hours or more after hatching (about 12%). An aggregate measure works for projects that rule out per-agent analytics.
|
|
644
662
|
- **T3 Reviews** (required):
|
|
645
|
-
- A monthly log review, paged so no window is truncated: requested well-known paths, 404s, refusals, crawler user agents.
|
|
663
|
+
- A monthly log review, paged so no window is truncated (platform log commands often return only the newest window): requested well-known paths, 404s, refusals, crawler user agents.
|
|
646
664
|
- After each deploy, run the scorecard, lint the OpenAPI spec, and diff generated files against the previous production output. About 24 hours later, review the logs and database query performance.
|
|
647
|
-
- **T5 The agent usability test** (required
|
|
665
|
+
- **T5 The agent usability test** (required).
|
|
648
666
|
- **Why:** it's the only check that measures what the whole standard is for, and without it the scorecard becomes the target.
|
|
649
|
-
- **How:** give a fresh agent nothing but your domain and a goal
|
|
667
|
+
- **How:** give a fresh agent nothing but your domain and a goal: your core loop, as a person would ask for it ([examples](docs/usability-test.md#setup)). Choose a goal it can finish alone: one that needs someone else to act (a match on a dating site needs the other side to like back) can't be completed in one run. Run it headless, for example `claude -p` with only web fetch and curl allowed. Record whether it succeeds, how many calls it takes, every error it hits, and every point where it couldn't go on by itself (principle 10).
|
|
650
668
|
- **When:** monthly, and after big API changes.
|
|
651
669
|
- **Report** its first-try success rate next to your score.
|
|
652
670
|
- **Rules:** run it only against your own site, with a test username so analytics filter it out.
|
|
671
|
+
- **Keep test accounts out of public pages:** one helper recognizes the test prefix; leave those accounts out of public listings, search and counts; `noindex` their pages and keep them out of the sitemap; and guard it with a test ([recipe](docs/recipes.md#test-accounts-out-of-public-pages-t5-w3)).
|
|
672
|
+
- **Keep the agent fresh,** with none of the operator's memory, settings or account details. drifts' test agent named itself after the operator, taken from its environment.
|
|
673
|
+
- **Budget it.** An open-ended run on a large model took drifts 7 minutes, 132 turns and $6.85 for one attempt. Set the model and a turn limit, and keep both the same month to month so results compare.
|
|
674
|
+
- **Report doc mismatches,** not only errors: docs that promise a field or behavior the API doesn't deliver.
|
|
653
675
|
- **The procedure,** a runner script and the results format are in [docs/usability-test.md](docs/usability-test.md).
|
|
654
676
|
- **T6 Search numbers** (required):
|
|
655
677
|
- Verify the site in Google Search Console and Bing Webmaster Tools, and submit the sitemap to both.
|
|
656
678
|
- Review them monthly: coverage (including "crawled, not indexed"), queries, clicks and click-through rate. Re-check about two weeks after an SEO change (inbed).
|
|
657
|
-
- Record clicks, impressions
|
|
679
|
+
- Record the numbers next to the score as fields, not prose, so months compare: the period, clicks, impressions, click-through rate, average position, pages indexed and not indexed, and "crawled, not indexed", for Google and Bing (format: [examples/recorded.json](examples/recorded.json)). They're the outcome the W items are for.
|
|
680
|
+
- It takes a person, or a browser session, with access to Search Console and Bing Webmaster Tools. There's no unattended way to read them.
|
|
658
681
|
- In analytics, watch referrals from AI answer engines (chatgpt.com, perplexity.ai, copilot.microsoft.com, gemini.google.com, claude.ai).
|
|
659
682
|
- Search Console gained a setting on 2026-08-31 that keeps a site out of Google's AI features. It's on by default, so leave it alone unless you mean to opt out.
|
|
660
683
|
|
|
@@ -685,6 +708,11 @@ A project with no test suite fails T4 however well it scores (botsmatter's own r
|
|
|
685
708
|
|
|
686
709
|
- **robots.txt groups don't merge with `*`.** A crawler named in its own group ignores the `*` group, including its `Disallow` and `Content-Signal` lines.
|
|
687
710
|
- **`Vary: Accept` on only one representation** lets caches mix markdown and HTML.
|
|
711
|
+
- **Next.js drops a `Vary` set in middleware, proxy or `next.config`** on App Router pages, while it keeps the other headers (tested on 14.2 and 16.3). It has to come from the CDN.
|
|
712
|
+
- **Next.js `robots.ts` can't output `Content-Signal`** or other unknown lines. Serve robots.txt from a route handler.
|
|
713
|
+
- **Redocly fails public operations without `security: []`** when the spec has a global security scheme.
|
|
714
|
+
- **zod-to-openapi v7 (Zod 3) needed `extendZodWithOpenApi(z)`** before path and query parameters came through with their descriptions (drifts).
|
|
715
|
+
- **A log written inside the container** is gone after the next deploy.
|
|
688
716
|
- **Prerendered pages skip middleware**, so content negotiation must sit in front of them.
|
|
689
717
|
- **Static extensionless files lose their content type.** Serve `api-catalog` from a route.
|
|
690
718
|
- **Build-time `lastmod` dates on every sitemap URL** say nothing to crawlers.
|
|
@@ -695,7 +723,7 @@ A project with no test suite fails T4 however well it scores (botsmatter's own r
|
|
|
695
723
|
- **A crawler that follows a GET with side effects** counts as a visit. achurch.ai disallows `/api/attend`.
|
|
696
724
|
- **Supabase silently caps a select at 1,000 rows,** and its client has no default timeout.
|
|
697
725
|
- **Next.js 14.2's `force-dynamic` alone leaves fetches cached.** Use `revalidate = 0` for live pages (inbed). Route segment config can't be re-exported either.
|
|
698
|
-
- **
|
|
726
|
+
- **A timestamp set when a job notices an event records the detection, not the event** (animalhouse's `died_at`).
|
|
699
727
|
- **MCP SDK v1 can't speak 2026-07-28.** SDK v2's legacy fallback speaks only SSE and is strict about `Accept` (M4).
|
|
700
728
|
- **A redirect from `/sse` doesn't make the old SSE transport work.** 2024-11-05 clients GET `/sse` and expect an `endpoint` event.
|
|
701
729
|
- **Claude Code ignores AGENTS.md when CLAUDE.md exists,** unless CLAUDE.md imports it.
|
|
@@ -704,7 +732,7 @@ A project with no test suite fails T4 however well it scores (botsmatter's own r
|
|
|
704
732
|
- **A catalog entry's type is a claim.** Typing `mcp.json` as a server card is a false declaration even when `mcp.json` itself is honest.
|
|
705
733
|
- **A client-side `document.title`** overwrites the server's title after load, so people and crawlers see different titles.
|
|
706
734
|
- **A site-wide `twitter:title`** overrides every page's own `og:title` on X.
|
|
707
|
-
- **Two items with the same name can share one URL.** animalhouse's creature page
|
|
735
|
+
- **Two items with the same name can share one URL.** animalhouse's creature page 404'd when an agent reused a name, while the sitemap still listed the URL; the lookup now picks the item the URL means (a living one first, then one with a gravestone). W3's sitemap sample catches it.
|
|
708
736
|
- **`og:locale` and JSON-LD `inLanguage` use different formats:** `en_US` and `en-US`, `zh_CN` and `zh-Hans`.
|
|
709
737
|
- **Retired rich results still get added.** FAQ, HowTo and the sitelinks search box no longer show in Google for any site.
|
|
710
738
|
- **ClawHub's shared namespace, scan delay and account ownership** (S3).
|
|
@@ -719,13 +747,15 @@ A project with no test suite fails T4 however well it scores (botsmatter's own r
|
|
|
719
747
|
Score each site monthly and after big releases, and keep the results where the project's agents can read them:
|
|
720
748
|
|
|
721
749
|
```
|
|
722
|
-
npx readiness-audit --matrix site-one.com site-two.com --recorded docs/readiness-recorded.json > docs/readiness-status.md
|
|
750
|
+
npx readiness-audit@1 --matrix site-one.com site-two.com --recorded docs/readiness-recorded.json > docs/readiness-status.md
|
|
723
751
|
```
|
|
724
752
|
|
|
725
|
-
The page lists every check for every site, the
|
|
753
|
+
The page lists every check for every site, the scores at each level, and each failure or warning with its reason. Keep it in the project's private repo when there is one (for example, a private repo that holds the docs, with the public repo as a submodule). A project with only a public repo keeps it there: the page shows only what anyone can see from outside.
|
|
754
|
+
|
|
755
|
+
Score a local or preview build before you deploy: `npx readiness-audit@1 example.com --base http://localhost:3000` requests the site's URLs from the base and reports them under the domain, skipping DNS and host redirects. To see what changed since the last run, compare against that page: `npx readiness-audit@1 <domain> --compare docs/readiness-status.md` lists what was fixed, what newly fails and what newly applies. Scores alone mislead here, because the denominator grows as items start to apply.
|
|
726
756
|
|
|
727
757
|
Read it with three caveats:
|
|
728
|
-
- **It covers only what's visible from outside.** One of our projects scored well with no test suite, and
|
|
758
|
+
- **It covers only what's visible from outside.** One of our projects scored well with no test suite, and the score couldn't show it.
|
|
729
759
|
- **Compare a project against its own applicable items, not against other projects.** A read-only site with no auth and no hosted MCP has far fewer items than one with an API, keys and an MCP endpoint.
|
|
730
760
|
- **T5 and T6 come from outside the scorecard.** Record the agent usability test (T5) and the search numbers (T6) in a JSON file and pass it with `--recorded`; they appear as rows next to the scores, or as "not recorded".
|
|
731
761
|
|
|
@@ -733,6 +763,8 @@ Read it with three caveats:
|
|
|
733
763
|
|
|
734
764
|
## 14. Adopting the standard in a repo
|
|
735
765
|
|
|
766
|
+
Step-by-step instructions for a project's coding agent: [docs/adopting.md](docs/adopting.md).
|
|
767
|
+
|
|
736
768
|
- **Point to the standard instead of copying it,** so your agents read one standard, not a fork that drifts:
|
|
737
769
|
```markdown
|
|
738
770
|
# Agent and Search Readiness
|
|
@@ -768,7 +800,7 @@ Level: **R** required where it applies, **Rec** recommended, **N** next level. W
|
|
|
768
800
|
| D5 | `Link` headers: service-desc, service-doc, describedby, api-catalog | Rec | discovery | score |
|
|
769
801
|
| D6 | security.txt with Expires once | R | hygiene | score |
|
|
770
802
|
| D7 | JSON 404s for unknown well-known paths, JSON 405 on `POST /` | Rec | agents | score |
|
|
771
|
-
| D8 | Markdown for agents with `Vary: Accept` and x-markdown-tokens |
|
|
803
|
+
| D8 | Markdown for agents with `Vary: Accept` and x-markdown-tokens, once traffic asks for it | N | agents | score |
|
|
772
804
|
| D9 | AI catalog at ard.json and ai-catalog.json, with `rel` links | Rec | discovery | score |
|
|
773
805
|
| D10 | One DNS AID record naming a service you run | Rec | discovery | score |
|
|
774
806
|
| D11 | The A2A path: a valid card or a JSON 404 | R | honesty | score |
|
|
@@ -803,7 +835,7 @@ Level: **R** required where it applies, **Rec** recommended, **N** next level. W
|
|
|
803
835
|
| M3 | Published to npm, the MCP Registry, Smithery and Glama, one version | R | discovery | test |
|
|
804
836
|
| M4 | Hosted edge: both eras, JSON for every Accept, browser redirect, aliases, logs | R | agents, traffic | score |
|
|
805
837
|
| M5 | Server card at `<endpoint>/server-card`, listed in the AI catalog | Rec | discovery | score |
|
|
806
|
-
| S1 | Spec-compliant SKILL.md files that link llms.txt and the API or MCP reference;
|
|
838
|
+
| S1 | Spec-compliant SKILL.md files that link llms.txt and the API or MCP reference; triggers name the job; lasting actions say what they do; an agent can follow them through the core loop on its own | R | agents, honesty | score (sample), review |
|
|
807
839
|
| S2 | Skills discovery index, v0.2.0 | Rec | discovery | score |
|
|
808
840
|
| S3 | ClawHub owners map, per-item skills, slow publishing | Rec | discovery | review |
|
|
809
841
|
| S4 | Plugin bundle for four hosts, generated manifests | Rec | agents | test |
|
|
@@ -906,3 +938,4 @@ Status as of 2026-10-05. Several are drafts, so re-check before relying on a det
|
|
|
906
938
|
| AI crawlers | OpenAI, Perplexity, Anthropic docs | https://developers.openai.com/api/docs/bots · https://docs.perplexity.ai/docs/resources/perplexity-crawlers · https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler |
|
|
907
939
|
| Measured search results | achurch.ai's August 2026 retrospective | not published |
|
|
908
940
|
| isitagentready.com | Cloudflare scanner | https://isitagentready.com/ |
|
|
941
|
+
| ClawHub behavior (S1, S3) | observed while publishing (animalhouse, drifts), September and October 2026; not documented by ClawHub | https://clawhub.ai/ |
|
package/bin/readiness-audit.mjs
CHANGED
|
@@ -1,13 +1,14 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
// readiness-audit: score a live site against the Agent and Search Readiness Standard.
|
|
3
3
|
//
|
|
4
|
-
// readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json]
|
|
4
|
+
// readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json] [--compare previous] [--base url]
|
|
5
5
|
// readiness-audit --matrix <domain> <domain> ... [--recorded results.json] (a markdown status page)
|
|
6
6
|
//
|
|
7
7
|
// Exits 1 when a required check fails, so it can gate a deploy.
|
|
8
8
|
|
|
9
9
|
import { readFileSync } from 'node:fs';
|
|
10
10
|
import { audit, matrix, score, LEVELS, REVIEW, VERSION } from '../src/audit.mjs';
|
|
11
|
+
import { parsePrevious, compareResults, formatComparison } from '../src/helpers.mjs';
|
|
11
12
|
|
|
12
13
|
const args = process.argv.slice(2);
|
|
13
14
|
const flag = (n) => args.includes(n);
|
|
@@ -18,15 +19,22 @@ if (flag('--version') || flag('-v')) {
|
|
|
18
19
|
const valueOf = (name) => (flag(name) ? args[args.indexOf(name) + 1] : undefined);
|
|
19
20
|
const mcpOpt = valueOf('--mcp');
|
|
20
21
|
const recordedPath = valueOf('--recorded');
|
|
21
|
-
const
|
|
22
|
+
const comparePath = valueOf('--compare');
|
|
23
|
+
const base = valueOf('--base');
|
|
24
|
+
const domains = args.filter((a, i) => !a.startsWith('-') && !['--mcp', '--recorded', '--compare', '--base'].includes(args[i - 1]))
|
|
22
25
|
.map((d) => d.replace(/^https?:\/\//, '').replace(/\/.*$/, ''));
|
|
23
|
-
const opts = { hasApi: !flag('--no-api'), mcpPath: flag('--no-mcp') ? null : (mcpOpt ?? '/mcp') };
|
|
26
|
+
const opts = { hasApi: !flag('--no-api'), mcpPath: flag('--no-mcp') ? null : (mcpOpt ?? '/mcp'), base: base ?? null };
|
|
27
|
+
if (base) {
|
|
28
|
+
let u;
|
|
29
|
+
try { u = new URL(base); } catch { u = null; }
|
|
30
|
+
if (!u || !['http:', 'https:'].includes(u.protocol)) { console.error(`--base needs an http(s) URL, like http://localhost:3000 (got ${base})`); process.exit(2); }
|
|
31
|
+
}
|
|
24
32
|
|
|
25
33
|
if (!domains.length || flag('--help') || flag('-h')) {
|
|
26
34
|
console.error(`readiness-audit ${VERSION}
|
|
27
35
|
|
|
28
36
|
Usage:
|
|
29
|
-
readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json]
|
|
37
|
+
readiness-audit <domain> [--mcp /path] [--no-mcp] [--no-api] [--json] [--compare previous] [--base url]
|
|
30
38
|
readiness-audit --matrix <domain> <domain> ... [--recorded results.json]
|
|
31
39
|
|
|
32
40
|
Options:
|
|
@@ -35,31 +43,50 @@ Options:
|
|
|
35
43
|
--no-api skip the API checks, for a site with no public API
|
|
36
44
|
--json print the results as JSON
|
|
37
45
|
--matrix score several sites and print a markdown status page
|
|
38
|
-
--recorded f with --matrix: add recorded T5 and T6 results from a JSON file
|
|
46
|
+
--recorded f with --matrix: add recorded T5 and T6 results from a JSON file
|
|
47
|
+
--compare f list what changed since a previous run: a status page from
|
|
48
|
+
--matrix (docs/readiness-status.md) or a saved --json report
|
|
49
|
+
--base url score a local or preview build as the domain, before you
|
|
50
|
+
deploy (http://localhost:3000); DNS and host redirects are
|
|
51
|
+
skipped`);
|
|
39
52
|
process.exit(domains.length ? 0 : 2);
|
|
40
53
|
}
|
|
41
54
|
|
|
42
55
|
if (flag('--matrix')) {
|
|
56
|
+
if (comparePath || base) console.error('--compare and --base work with a single domain; ignoring them for --matrix.');
|
|
57
|
+
opts.base = null;
|
|
43
58
|
let recorded = {};
|
|
44
59
|
if (recordedPath) {
|
|
45
60
|
try { recorded = JSON.parse(readFileSync(recordedPath, 'utf8')); }
|
|
46
61
|
catch (e) { console.error(`Couldn't read ${recordedPath}: ${e.message}`); process.exit(2); }
|
|
47
62
|
}
|
|
48
63
|
const reports = await Promise.all(domains.map((d) => audit(d, opts)));
|
|
49
|
-
|
|
64
|
+
const pinned = `readiness-audit@${VERSION.split('.')[0]}`; // the major version, as STANDARD.md §14 says to pin it
|
|
65
|
+
console.log(matrix(reports, `npx ${pinned} --matrix ${domains.join(' ')}${recordedPath ? ` --recorded ${recordedPath}` : ''}`, recorded));
|
|
50
66
|
} else {
|
|
67
|
+
let previous = null;
|
|
68
|
+
if (comparePath) {
|
|
69
|
+
let text;
|
|
70
|
+
try { text = readFileSync(comparePath, 'utf8'); }
|
|
71
|
+
catch (e) { console.error(`Couldn't read ${comparePath}: ${e.message}`); process.exit(2); }
|
|
72
|
+
previous = parsePrevious(text, domains[0]);
|
|
73
|
+
if (!previous) { console.error(`${comparePath} has no results for ${domains[0]}`); process.exit(2); }
|
|
74
|
+
}
|
|
51
75
|
const rep = await audit(domains[0], opts);
|
|
52
|
-
|
|
76
|
+
const since = previous ? `${previous.date ?? 'the previous run'} (${comparePath})` : null;
|
|
77
|
+
const changes = previous ? compareResults(previous.results, rep.results) : null;
|
|
78
|
+
if (flag('--json')) console.log(JSON.stringify(changes ? { ...rep, changes: { since: previous.date, ...changes } } : rep, null, 2));
|
|
53
79
|
else {
|
|
54
80
|
const label = { pass: 'PASS', warn: 'WARN', fail: 'FAIL', skip: 'skip' };
|
|
55
|
-
console.log(`Agent and search readiness: ${rep.domain} (readiness-audit v${VERSION}, ${rep.checked_at.slice(0, 16)}Z)`);
|
|
81
|
+
console.log(`Agent and search readiness: ${rep.domain}${rep.base ? ` via ${rep.base}` : ''} (readiness-audit v${VERSION}, ${rep.checked_at.slice(0, 16)}Z)`);
|
|
56
82
|
for (const level of LEVELS) {
|
|
57
83
|
const rows = rep.results.filter((r) => r.level === level);
|
|
58
84
|
if (!rows.length) continue;
|
|
59
85
|
console.log(`\n${{ required: 'Required', recommended: 'Recommended', next: 'Next level' }[level]}`);
|
|
60
86
|
for (const r of rows) console.log(` ${label[r.status]} ${r.id.padEnd(4)} ${r.name.padEnd(36)} ${r.detail}`);
|
|
61
87
|
}
|
|
62
|
-
console.log(`\nScore: required ${score(rep.results, 'required')}, recommended ${score(rep.results, 'recommended')}, next level ${score(rep.results, 'next')}`);
|
|
88
|
+
console.log(`\nScore on ${rep.checked_at.slice(0, 10)}: required ${score(rep.results, 'required')}, recommended ${score(rep.results, 'recommended')}, next level ${score(rep.results, 'next')}`);
|
|
89
|
+
if (changes) console.log(`\n${formatComparison(changes, since).join('\n')}`);
|
|
63
90
|
console.log('\nNot scored, so check these yourself:');
|
|
64
91
|
for (const [id, what] of REVIEW) console.log(` ${id.padEnd(6)} ${what}`);
|
|
65
92
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "readiness-audit",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.2.1",
|
|
4
4
|
"description": "Score a live site against the Agent and Search Readiness Standard: discovery files, search, API, MCP and skills, for AI agents and search engines.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/src/audit.mjs
CHANGED
|
@@ -6,12 +6,13 @@
|
|
|
6
6
|
// Read-only: GETs plus a few POSTs that create nothing (an MCP initialize, server/discover and
|
|
7
7
|
// tools/list, an empty POST to /). Node 20+, no dependencies.
|
|
8
8
|
|
|
9
|
-
import { lookup, resolveTxt } from 'node:dns/promises';
|
|
9
|
+
import { lookup, resolveTxt, Resolver } from 'node:dns/promises';
|
|
10
|
+
import { request as httpRequest } from 'node:http';
|
|
10
11
|
import { request as httpsRequest } from 'node:https';
|
|
11
12
|
import { isIP } from 'node:net';
|
|
12
|
-
import { SKILL_NAME, sampleSkillEntries, skillNameProblem, cdnCacheNote } from './helpers.mjs';
|
|
13
|
+
import { SKILL_NAME, sampleSkillEntries, skillNameProblem, cdnCacheNote, repeatsSummary, hasVariantDescription, spread, llmsLinks, baseMapper, formatRecorded } from './helpers.mjs';
|
|
13
14
|
|
|
14
|
-
export const VERSION = '1.
|
|
15
|
+
export const VERSION = '1.2.1';
|
|
15
16
|
export const REPO = 'https://github.com/geeks-accelerator/agent-and-search-readiness';
|
|
16
17
|
const UA = `readiness-audit/${VERSION} (+${REPO})`;
|
|
17
18
|
const AI_BOTS = ['gptbot', 'oai-searchbot', 'chatgpt-user', 'claudebot', 'claude-user', 'claude-searchbot', 'perplexitybot', 'perplexity-user', 'google-extended', 'applebot-extended'];
|
|
@@ -19,7 +20,7 @@ const AID_PROTOCOLS = ['mcp', 'a2a', 'openapi', 'grpc', 'graphql', 'websocket',
|
|
|
19
20
|
const SKILLS_SCHEMA = 'https://schemas.agentskills.io/discovery/0.2.0/schema.json';
|
|
20
21
|
const CARD_SCHEMA = 'https://static.modelcontextprotocol.io/schemas/v1/server-card.schema.json';
|
|
21
22
|
export const LEVELS = ['required', 'recommended', 'next'];
|
|
22
|
-
const ORDER = ['D1', 'D2', 'D4', 'D6', 'D11', 'D12', 'W1', 'W2', 'W3', 'W4', 'W5', 'W6', 'A1', 'A2', 'A6', 'A7', 'M1', 'M4', 'S1', 'D5', 'D7', '
|
|
23
|
+
const ORDER = ['D1', 'D2', 'D4', 'D6', 'D11', 'D12', 'W1', 'W2', 'W3', 'W4', 'W5', 'W6', 'A1', 'A2', 'A6', 'A7', 'M1', 'M4', 'S1', 'D5', 'D7', 'D9', 'D10', 'D15', 'A3', 'A4', 'A5', 'M5', 'S2', 'D8', 'N5', 'N6'];
|
|
23
24
|
// Items the scorecard can't see from outside. Each project checks these in review and tests.
|
|
24
25
|
export const REVIEW = [
|
|
25
26
|
['D3', 'pages read well: server-rendered, unique prose on generated pages (the scorecard checks W1 to W6 on a sample)'],
|
|
@@ -35,7 +36,7 @@ export const REVIEW = [
|
|
|
35
36
|
['A11', 'public and private stated honestly, one privacy filter'],
|
|
36
37
|
['M2', 'stdio package: saved key, register guard, rotate, User-Agent'],
|
|
37
38
|
['M3', 'published to npm, the MCP Registry, Smithery and Glama, one version'],
|
|
38
|
-
['S1', 'skills
|
|
39
|
+
['S1', 'skills: triggers name the job, lasting actions say what they do, and an agent can follow them through the core loop on its own (the scorecard checks names, descriptions and links on a sample)'],
|
|
39
40
|
['S3', 'ClawHub owners map'],
|
|
40
41
|
['S4', 'plugin bundle with generated manifests'],
|
|
41
42
|
['E1-E6', 'first-call value, reasons to return, scheduled-agent performance'],
|
|
@@ -71,6 +72,19 @@ async function mapLimit(items, limit, fn) {
|
|
|
71
72
|
return out;
|
|
72
73
|
}
|
|
73
74
|
|
|
75
|
+
// TXT lookups go to public resolvers first: the machine's own resolver can hold an answer
|
|
76
|
+
// from before a change. Falls back to the system resolver when they can't be reached.
|
|
77
|
+
async function resolveTxtFresh(name) {
|
|
78
|
+
try {
|
|
79
|
+
const resolver = new Resolver({ timeout: 3000, tries: 2 });
|
|
80
|
+
resolver.setServers(['1.1.1.1', '8.8.8.8']);
|
|
81
|
+
return await resolver.resolveTxt(name);
|
|
82
|
+
} catch (e) {
|
|
83
|
+
if (e?.code === 'ENOTFOUND' || e?.code === 'ENODATA') throw e; // a real answer: no record
|
|
84
|
+
return resolveTxt(name);
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
|
|
74
88
|
// Which URLs the audit may request. The site you chose to audit (and its www or bare variant) is
|
|
75
89
|
// always allowed; anything else must be a public http(s) address, checked again at every redirect hop.
|
|
76
90
|
export function makeGuard(domain) {
|
|
@@ -92,28 +106,32 @@ export function makeGuard(domain) {
|
|
|
92
106
|
return { siteHosts, isSafe };
|
|
93
107
|
}
|
|
94
108
|
|
|
95
|
-
export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
|
|
109
|
+
export async function audit(domain, { hasApi = true, mcpPath = '/mcp', base = null } = {}) {
|
|
96
110
|
const BASE = `https://${domain}`;
|
|
97
111
|
const rand = Math.random().toString(36).slice(2, 10);
|
|
98
112
|
|
|
99
113
|
const { siteHosts, isSafe } = makeGuard(domain);
|
|
114
|
+
// --base scores a local or preview build as the domain: the site's URLs are requested from the
|
|
115
|
+
// base (which may be a local address, because you chose it) and reported under the domain.
|
|
116
|
+
const { baseOrigin, toBase, fromBase } = baseMapper(domain, base);
|
|
100
117
|
|
|
101
118
|
// Redirects are followed here, one hop at a time, so every hop passes isSafe.
|
|
102
119
|
async function get(path, { headers = {}, method = 'GET', body, redirect = 'follow' } = {}) {
|
|
103
|
-
let url = path.startsWith('http') ? path : BASE + path;
|
|
120
|
+
let url = path.startsWith('http') ? path : BASE + path; // always in the domain's terms
|
|
104
121
|
let m = method, b = body;
|
|
105
122
|
for (let hop = 0; ; hop++) {
|
|
106
|
-
|
|
123
|
+
const target = toBase(url);
|
|
124
|
+
if (target === url && !(await isSafe(url))) return { status: 0, headers: new Headers(), text: '', url, type: '', error: 'refused: not a public address' };
|
|
107
125
|
let res;
|
|
108
126
|
try {
|
|
109
|
-
res = await fetch(
|
|
127
|
+
res = await fetch(target, { method: m, body: b, redirect: 'manual', headers: { 'user-agent': UA, ...headers }, signal: AbortSignal.timeout(15000) });
|
|
110
128
|
} catch (e) {
|
|
111
129
|
return { status: 0, headers: new Headers(), text: '', url, type: '', error: e.message };
|
|
112
130
|
}
|
|
113
131
|
const loc = res.headers.get('location');
|
|
114
132
|
if (redirect === 'follow' && res.status >= 300 && res.status < 400 && loc && hop < 5) {
|
|
115
133
|
try { await res.body?.cancel(); } catch { /* already closed */ }
|
|
116
|
-
url = new URL(loc,
|
|
134
|
+
url = fromBase(new URL(loc, target).href);
|
|
117
135
|
if (res.status !== 307 && res.status !== 308) { m = 'GET'; b = undefined; }
|
|
118
136
|
continue;
|
|
119
137
|
}
|
|
@@ -123,8 +141,10 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
|
|
|
123
141
|
}
|
|
124
142
|
// fetch() always sends an Accept header, so the "no Accept" MCP probe uses node:https.
|
|
125
143
|
function postNoAccept(path, payload) {
|
|
144
|
+
const target = new URL(toBase(BASE + path));
|
|
145
|
+
const send = target.protocol === 'http:' ? httpRequest : httpsRequest;
|
|
126
146
|
return new Promise((resolve) => {
|
|
127
|
-
const req =
|
|
147
|
+
const req = send(target, { method: 'POST', timeout: 15000, headers: { 'user-agent': UA, 'content-type': 'application/json', 'content-length': Buffer.byteLength(payload) } }, (res) => {
|
|
128
148
|
let text = '';
|
|
129
149
|
res.on('data', (c) => { text += c; });
|
|
130
150
|
res.on('end', () => resolve({ status: res.statusCode, type: res.headers['content-type'] ?? '', text }));
|
|
@@ -241,9 +261,18 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
|
|
|
241
261
|
// ---- D4 llms.txt ----
|
|
242
262
|
{
|
|
243
263
|
const id = 'D4', name = 'llms.txt and llms-full.txt', level = 'required';
|
|
264
|
+
// Agents follow the links in llms.txt, so a sample of the site's own links must answer 200.
|
|
265
|
+
const links = ok(llms) ? llmsLinks(llms.text, `${BASE}/llms.txt`, siteHosts) : [];
|
|
266
|
+
const sample = spread(links, 10);
|
|
267
|
+
const answers = await mapLimit(sample, 4, async (u) => ({ u, r: await get(u) }));
|
|
268
|
+
// Missing pages, server errors and no answer are broken. A 401, 403 or 405 is an endpoint that
|
|
269
|
+
// exists but wants a key or another method, which is fine to list.
|
|
270
|
+
const broken = answers.filter(({ r }) => !r.status || r.status === 404 || r.status === 410 || r.status >= 500).map(({ u, r }) => `${u.replace(BASE, '') || '/'} (${r.status || 'no answer'})`);
|
|
271
|
+
const sizes = `llms.txt ${(llms.text.length / 1024).toFixed(1)} KB, llms-full.txt ${(llmsFull.text.length / 1024).toFixed(0)} KB`;
|
|
244
272
|
if (!ok(llms) || !/^\s*# \S/.test(llms.text)) miss(id, name, level, (ok(llms) ? 'llms.txt has no H1 title on its first line' : `/llms.txt answered ${llms.status}`) + cached(llms));
|
|
245
273
|
else if (!ok(llmsFull)) miss(id, name, level, `/llms-full.txt missing${cached(llmsFull)}`);
|
|
246
|
-
else
|
|
274
|
+
else if (broken.length) record(id, name, level, 'warn', `broken links in llms.txt: ${broken.slice(0, 3).join(', ')}${broken.length > 3 ? `, +${broken.length - 3} more` : ''} (${broken.length} of ${sample.length} sampled)${cached(llms)}`);
|
|
275
|
+
else pass(id, name, level, `${sizes}${sample.length ? `; ${sample.length} sampled link${sample.length === 1 ? '' : 's'} resolve` : ''}`);
|
|
247
276
|
}
|
|
248
277
|
|
|
249
278
|
// ---- D5 Link headers (recommended; a wrong service-desc is a false declaration) ----
|
|
@@ -288,7 +317,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
|
|
|
288
317
|
|
|
289
318
|
// ---- D8 markdown for agents ----
|
|
290
319
|
{
|
|
291
|
-
const id = 'D8', name = 'Markdown for agents', level = '
|
|
320
|
+
const id = 'D8', name = 'Markdown for agents', level = 'next';
|
|
292
321
|
const varyAccept = (r) => /(^|,)\s*accept\s*(,|$)/i.test(r.headers.get('vary') ?? '');
|
|
293
322
|
const tokens = homeMd.headers.get('x-markdown-tokens');
|
|
294
323
|
if (!/text\/markdown/.test(homeMd.type)) miss(id, name, level, `Accept: text/markdown on / returns ${ct(homeMd)}`);
|
|
@@ -345,33 +374,46 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
|
|
|
345
374
|
seen.add(s.$ref);
|
|
346
375
|
return deref(s.$ref.replace(/^#\//, '').split('/').reduce((n, k) => n?.[k], spec), seen);
|
|
347
376
|
};
|
|
348
|
-
|
|
377
|
+
// `nested` collects fields whose only description sits inside anyOf, oneOf or allOf.
|
|
378
|
+
const undescribedProps = (schema, seen = new Set(), nested = null) => {
|
|
349
379
|
const s = deref(schema);
|
|
350
380
|
if (!s || typeof s !== 'object' || seen.has(s)) return [];
|
|
351
381
|
seen.add(s);
|
|
352
382
|
const out = [];
|
|
353
383
|
for (const [k, v] of Object.entries(s.properties ?? {})) {
|
|
354
384
|
const d = deref(v);
|
|
355
|
-
if (!(v?.description || d?.description))
|
|
356
|
-
|
|
385
|
+
if (!(v?.description || d?.description)) {
|
|
386
|
+
out.push(k);
|
|
387
|
+
if (nested && hasVariantDescription(d, deref)) nested.add(k);
|
|
388
|
+
}
|
|
389
|
+
out.push(...undescribedProps(d, seen, nested));
|
|
357
390
|
}
|
|
358
|
-
for (const key of ['items', 'allOf', 'anyOf', 'oneOf']) for (const sub of [].concat(s[key] ?? [])) out.push(...undescribedProps(sub, seen));
|
|
391
|
+
for (const key of ['items', 'allOf', 'anyOf', 'oneOf']) for (const sub of [].concat(s[key] ?? [])) out.push(...undescribedProps(sub, seen, nested));
|
|
359
392
|
return out;
|
|
360
393
|
};
|
|
361
394
|
const ops = Object.entries(spec.paths ?? {}).flatMap(([p, item]) => Object.entries(item)
|
|
362
395
|
.filter(([m]) => ['get', 'post', 'put', 'patch', 'delete'].includes(m))
|
|
363
396
|
.map(([m, o]) => ({ key: `${m.toUpperCase()} ${p}`, o, params: [...(item.parameters ?? []), ...(o.parameters ?? [])] })));
|
|
364
397
|
const noDesc = ops.filter(({ o }) => !String(o.description ?? '').trim());
|
|
365
|
-
const
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
398
|
+
const inVariant = [];
|
|
399
|
+
const fieldGaps = ops.flatMap(({ key, o, params }) => {
|
|
400
|
+
const nested = new Set();
|
|
401
|
+
const gaps = [
|
|
402
|
+
...params.map((p) => deref(p)).filter((p) => p && !p.description && !deref(p.schema)?.description).map((p) => `${key} ${p.name}`),
|
|
403
|
+
...Object.values(deref(o.requestBody)?.content ?? {}).flatMap((c) => undescribedProps(c.schema, new Set(), nested)).map((f) => `${key} ${f}`),
|
|
404
|
+
];
|
|
405
|
+
inVariant.push(...[...nested].map((f) => `${key} ${f}`));
|
|
406
|
+
return gaps;
|
|
407
|
+
});
|
|
408
|
+
const variantNote = inVariant.length ? `; ${inVariant.length === fieldGaps.length ? 'each has' : `${inVariant.length} of them ${inVariant.length === 1 ? 'has' : 'have'}`} one inside anyOf or oneOf: put it on the field itself (in Zod, call .describe() last, after .optional() and .nullable())` : '';
|
|
409
|
+
const repeats = ops.filter(({ o }) => repeatsSummary(o));
|
|
410
|
+
const repeatNote = repeats.length ? `; ${repeats.length} operation${repeats.length === 1 ? "'s description only repeats its" : "s' descriptions only repeat their"} summary (say when to use ${repeats.length === 1 ? 'it' : 'each'})` : '';
|
|
369
411
|
const respGaps = new Set(ops.flatMap(({ o }) => Object.values(o.responses ?? {}).flatMap((r) => Object.values(deref(r)?.content ?? {}).flatMap((c) => undescribedProps(c.schema)))));
|
|
370
412
|
const respNote = respGaps.size ? `; ${respGaps.size} response field${respGaps.size === 1 ? ' has' : 's have'} no description (see N7)` : '';
|
|
371
413
|
if (!String(spec.openapi).startsWith('3.1')) miss(id, name, level, `OpenAPI ${spec.openapi} (use 3.1)`);
|
|
372
|
-
else if (noDesc.length) miss(id, name, level, `${noDesc.length} of ${ops.length} operations ${noDesc.length === 1 ? 'has' : 'have'} no description (${noDesc.slice(0, 3).map((x) => x.key).join(', ')}${noDesc.length > 3 ? ', ...' : ''})${respNote}`);
|
|
373
|
-
else if (fieldGaps.length) miss(id, name, level, `${fieldGaps.length} request field${fieldGaps.length === 1 ? ' has' : 's have'} no description (${fieldGaps.slice(0, 3).join(', ')}${fieldGaps.length > 3 ? ', ...' : ''})${respNote}`);
|
|
374
|
-
else pass(id, name, level, `OpenAPI ${spec.openapi}; all ${ops.length} operations and their request fields described${respNote}`);
|
|
414
|
+
else if (noDesc.length) miss(id, name, level, `${noDesc.length} of ${ops.length} operations ${noDesc.length === 1 ? 'has' : 'have'} no description (${noDesc.slice(0, 3).map((x) => x.key).join(', ')}${noDesc.length > 3 ? ', ...' : ''})${repeatNote}${respNote}`);
|
|
415
|
+
else if (fieldGaps.length) miss(id, name, level, `${fieldGaps.length} request field${fieldGaps.length === 1 ? ' has' : 's have'} no description (${fieldGaps.slice(0, 3).join(', ')}${fieldGaps.length > 3 ? ', ...' : ''})${variantNote}${repeatNote}${respNote}`);
|
|
416
|
+
else pass(id, name, level, `OpenAPI ${spec.openapi}; all ${ops.length} operations and their request fields described${repeatNote}${respNote}`);
|
|
375
417
|
}
|
|
376
418
|
}
|
|
377
419
|
{
|
|
@@ -660,7 +702,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
|
|
|
660
702
|
};
|
|
661
703
|
const alt = domain.startsWith('www.') ? `https://${domain.slice(4)}/` : `https://www.${domain}/`;
|
|
662
704
|
const hostProblems = [];
|
|
663
|
-
for (const start of [`http://${domain}/`, alt]) {
|
|
705
|
+
for (const start of baseOrigin ? [] : [`http://${domain}/`, alt]) { // --base: the live hosts aren't what's being scored
|
|
664
706
|
const h = await hostHops(start);
|
|
665
707
|
if (!h) continue; // that host doesn't resolve or answer
|
|
666
708
|
if (new URL(h.end).origin !== origin) hostProblems.push(`${start} never reaches ${origin} (ends at ${h.end})`);
|
|
@@ -672,7 +714,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
|
|
|
672
714
|
if (elsewhere.length) problems.push(`sitemap URLs canonical to another URL: ${list(elsewhere)}`);
|
|
673
715
|
problems.push(...hostProblems);
|
|
674
716
|
if (problems.length) miss(id, name, level, problems.join('; '));
|
|
675
|
-
else pass(id, name, level, `self-referencing canonicals on ${ok200.length} sampled pages; http and the other host redirect once to ${origin}`);
|
|
717
|
+
else pass(id, name, level, `self-referencing canonicals on ${ok200.length} sampled pages; ${baseOrigin ? 'host redirects not checked with --base' : `http and the other host redirect once to ${origin}`}`);
|
|
676
718
|
}
|
|
677
719
|
// W3 only indexable pages in the sitemap; real 404s
|
|
678
720
|
{
|
|
@@ -757,21 +799,25 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
|
|
|
757
799
|
// ---- D10 DNS AID (recommended; a record naming a service you don't run is false) ----
|
|
758
800
|
{
|
|
759
801
|
const id = 'D10', name = 'DNS AID record', level = 'recommended';
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
if (!recs.length) miss(id, name, level, `no TXT record at _agent.${domain}`);
|
|
763
|
-
else if (recs.length > 1) wrong(id, name, level, `${recs.length} records at _agent.${domain}; AID clients fail on ambiguity, so keep exactly one`);
|
|
802
|
+
const recent = ' (DNS answers stay cached until the record\'s TTL runs out, so a change can take a while to show)';
|
|
803
|
+
if (baseOrigin) skip(id, name, level, 'not checked with --base: DNS describes the live domain');
|
|
764
804
|
else {
|
|
765
|
-
|
|
766
|
-
|
|
767
|
-
|
|
805
|
+
let recs = [];
|
|
806
|
+
try { recs = (await resolveTxtFresh(`_agent.${domain}`)).map((r) => r.join('')).filter((r) => r.startsWith('v=aid')); } catch { recs = []; }
|
|
807
|
+
if (!recs.length) miss(id, name, level, `no TXT record at _agent.${domain}${recent}`);
|
|
808
|
+
else if (recs.length > 1) wrong(id, name, level, `${recs.length} records at _agent.${domain}; AID clients fail on ambiguity, so keep exactly one${recent}`);
|
|
768
809
|
else {
|
|
769
|
-
|
|
770
|
-
if (kv.
|
|
771
|
-
else if (kv.
|
|
772
|
-
else
|
|
773
|
-
|
|
774
|
-
|
|
810
|
+
const kv = Object.fromEntries(recs[0].split(';').map((p) => p.split('=')).filter((p) => p.length >= 2).map(([k, ...v]) => [k.trim(), v.join('=').trim()]));
|
|
811
|
+
if (kv.v !== 'aid2') miss(id, name, level, `version ${kv.v} (current is aid2)`);
|
|
812
|
+
else if (!kv.u || !AID_PROTOCOLS.includes(kv.p)) wrong(id, name, level, `invalid record: u=${kv.u ?? 'missing'} p=${kv.p ?? 'missing'}`);
|
|
813
|
+
else {
|
|
814
|
+
let why = '';
|
|
815
|
+
if (kv.p === 'a2a' && !a2aValid) why = 'p=a2a, but there is no valid A2A card';
|
|
816
|
+
else if (kv.p === 'mcp' && !(await answersMcp(kv.u)).hosted) why = `p=mcp, but ${kv.u} doesn't answer MCP`;
|
|
817
|
+
else if (kv.p === 'openapi' && !json(await get(kv.u))?.openapi) why = `p=openapi, but ${kv.u} isn't an OpenAPI document`;
|
|
818
|
+
if (why) wrong(id, name, level, why);
|
|
819
|
+
else pass(id, name, level, `p=${kv.p} u=${kv.u}`);
|
|
820
|
+
}
|
|
775
821
|
}
|
|
776
822
|
}
|
|
777
823
|
}
|
|
@@ -801,7 +847,7 @@ export async function audit(domain, { hasApi = true, mcpPath = '/mcp' } = {}) {
|
|
|
801
847
|
}
|
|
802
848
|
|
|
803
849
|
results.sort((a, b) => LEVELS.indexOf(a.level) - LEVELS.indexOf(b.level) || ORDER.indexOf(a.id) - ORDER.indexOf(b.id));
|
|
804
|
-
return { domain, version: VERSION, checked_at: new Date().toISOString(), hosted_mcp: hosted, results };
|
|
850
|
+
return { domain, base: baseOrigin, version: VERSION, checked_at: new Date().toISOString(), hosted_mcp: hosted, results };
|
|
805
851
|
}
|
|
806
852
|
|
|
807
853
|
export function score(results, level) {
|
|
@@ -809,7 +855,7 @@ export function score(results, level) {
|
|
|
809
855
|
return rows.length ? `${rows.filter((r) => r.status === 'pass').length} of ${rows.length}` : 'n/a';
|
|
810
856
|
}
|
|
811
857
|
|
|
812
|
-
// Markdown status page for several sites, for docs/
|
|
858
|
+
// Markdown status page for several sites, for a project's docs/readiness-status.md.
|
|
813
859
|
// `recorded` holds what the scorecard can't measure, per domain:
|
|
814
860
|
// { "example.com": { "T5": { "result": "4 of 5 first try", "date": "2026-10-12" }, "T6": { ... } } }
|
|
815
861
|
export function matrix(reports, command, recorded = {}) {
|
|
@@ -825,9 +871,10 @@ export function matrix(reports, command, recorded = {}) {
|
|
|
825
871
|
...ids.map(([id, name, level]) => `| ${id} | ${name} | ${level} | ${reports.map((rep) => sym[rep.results.find((r) => r.id === id)?.status ?? 'skip']).join(' | ')} |`),
|
|
826
872
|
`| | **Required passed** | | ${reports.map((rep) => `**${score(rep.results, 'required')}**`).join(' | ')} |`,
|
|
827
873
|
`| | Recommended passed | | ${reports.map((rep) => score(rep.results, 'recommended')).join(' | ')} |`,
|
|
874
|
+
`| | Next level passed | | ${reports.map((rep) => score(rep.results, 'next')).join(' | ')} |`,
|
|
828
875
|
`| | Hosted MCP endpoint | | ${reports.map((rep) => (rep.hosted_mcp ? 'yes' : 'no')).join(' | ')} |`,
|
|
829
876
|
...[['T5', 'Agent usability test (recorded)'], ['T6', 'Search numbers (recorded)']].map(([id, label]) =>
|
|
830
|
-
`| ${id} | ${label} | required | ${reports.map((rep) =>
|
|
877
|
+
`| ${id} | ${label} | required | ${reports.map((rep) => formatRecorded(recorded[rep.domain]?.[id])).join(' | ')} |`),
|
|
831
878
|
'',
|
|
832
879
|
'## What to fix, per site', '',
|
|
833
880
|
'Failures and warnings with the scorecard\'s reason, required items first.', '',
|
package/src/helpers.mjs
CHANGED
|
@@ -12,9 +12,78 @@ export const SKILL_NAME = /^(?!-)(?!.*--)[a-z0-9-]{1,64}(?<!-)$/;
|
|
|
12
12
|
export function sampleSkillEntries(list) {
|
|
13
13
|
const md = (Array.isArray(list) ? list : []).filter((e) => typeof e?.url === 'string'
|
|
14
14
|
&& (e.type === 'skill-md' || (e.type === undefined && /\.md$/i.test(e.url.split(/[?#]/)[0]))));
|
|
15
|
-
return md
|
|
15
|
+
return spread(md, 5);
|
|
16
16
|
}
|
|
17
17
|
|
|
18
|
+
/** Up to n items, spread evenly across the list (the first item always included). */
|
|
19
|
+
export function spread(list, n) {
|
|
20
|
+
return list.length <= n ? list : Array.from({ length: n }, (_, i) => list[Math.floor((i * list.length) / n)]);
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
/**
|
|
24
|
+
* The site's own links in an llms.txt: markdown links and bare URLs, resolved
|
|
25
|
+
* against the file's URL, without fragments, kept only when they point at one
|
|
26
|
+
* of the site's hosts. Templates such as /items/{id} are skipped.
|
|
27
|
+
*/
|
|
28
|
+
export function llmsLinks(text, llmsUrl, hosts) {
|
|
29
|
+
const raw = [
|
|
30
|
+
...[...String(text ?? '').matchAll(/\]\(\s*<?([^)\s>]+)>?[^)]*\)/g)].map((m) => m[1]),
|
|
31
|
+
...[...String(text ?? '').matchAll(/\bhttps?:\/\/[^\s)>\]"'`]+/g)].map((m) => m[0]),
|
|
32
|
+
];
|
|
33
|
+
const out = new Set();
|
|
34
|
+
for (const href of raw) {
|
|
35
|
+
if (href.startsWith('#')) continue; // an anchor in the file itself
|
|
36
|
+
if (/[{}]|%7B|%7D|\/:[a-z_]+|YOUR_|\.\.\./i.test(href)) continue; // a template ({id}, :id), not a link
|
|
37
|
+
let u;
|
|
38
|
+
try { u = new URL(href.replace(/[.,;:!?]+$/, ''), llmsUrl); } catch { continue; }
|
|
39
|
+
if (!['http:', 'https:'].includes(u.protocol) || !hosts.has(u.hostname)) continue;
|
|
40
|
+
u.hash = '';
|
|
41
|
+
out.add(u.href);
|
|
42
|
+
}
|
|
43
|
+
return [...out];
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
/**
|
|
47
|
+
* For --base: request a local or preview build as if it were the domain.
|
|
48
|
+
* toBase() sends the site's URLs (the domain or its www twin) to the base;
|
|
49
|
+
* fromBase() maps them back, so every check keeps seeing the domain's URLs.
|
|
50
|
+
*/
|
|
51
|
+
export function baseMapper(domain, base) {
|
|
52
|
+
const hosts = new Set([domain, domain.startsWith('www.') ? domain.slice(4) : `www.${domain}`]);
|
|
53
|
+
const baseOrigin = base ? new URL(base).origin : null;
|
|
54
|
+
const toBase = (url) => {
|
|
55
|
+
if (!baseOrigin) return url;
|
|
56
|
+
let u;
|
|
57
|
+
try { u = new URL(url); } catch { return url; }
|
|
58
|
+
return hosts.has(u.hostname) ? baseOrigin + u.pathname + u.search : url;
|
|
59
|
+
};
|
|
60
|
+
const fromBase = (url) => (baseOrigin && (url === baseOrigin || url.startsWith(`${baseOrigin}/`) || url.startsWith(`${baseOrigin}?`)) ? `https://${domain}${url.slice(baseOrigin.length) || '/'}` : url);
|
|
61
|
+
return { baseOrigin, toBase, fromBase };
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
/**
|
|
65
|
+
* A recorded T5 or T6 result for the status page. T6 may be structured
|
|
66
|
+
* ({ period, google: { clicks, impressions, ctr, position, indexed,
|
|
67
|
+
* not_indexed, crawled_not_indexed }, bing: { ... } }); otherwise the free-text
|
|
68
|
+
* `result` is shown. Pipes are replaced so the table stays intact.
|
|
69
|
+
*/
|
|
70
|
+
export function formatRecorded(x) {
|
|
71
|
+
if (!x) return 'not recorded';
|
|
72
|
+
const num = (v, label) => (v === undefined || v === null || v === '' ? null : `${v}${label}`);
|
|
73
|
+
const traffic = (o) => [num(o.clicks, ' clicks'), num(o.impressions, ' impressions'), num(o.ctr, '% CTR'), o.position != null ? `position ${o.position}` : null].filter(Boolean).join(', ');
|
|
74
|
+
const parts = [];
|
|
75
|
+
if (x.google && typeof x.google === 'object') {
|
|
76
|
+
const g = x.google;
|
|
77
|
+
const index = [num(g.indexed, ' indexed'), num(g.not_indexed, ' not indexed'), num(g.crawled_not_indexed, ' crawled but not indexed')].filter(Boolean).join(', ');
|
|
78
|
+
parts.push(`Google: ${[traffic(g), index].filter(Boolean).join('; ')}`);
|
|
79
|
+
}
|
|
80
|
+
if (x.bing && typeof x.bing === 'object') parts.push(`Bing: ${traffic(x.bing)}`);
|
|
81
|
+
const when = [parts.length ? x.period : null, x.date].filter(Boolean).join(', ');
|
|
82
|
+
const text = parts.length ? parts.join('. ') : String(x.result ?? '');
|
|
83
|
+
return `${text}${when ? ` (${when})` : ''}`.replace(/\|/g, '/');
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
|
|
18
87
|
const decode = (s) => { try { return decodeURIComponent(s); } catch { return s; } };
|
|
19
88
|
|
|
20
89
|
/**
|
|
@@ -45,3 +114,79 @@ export function cdnCacheNote(headers) {
|
|
|
45
114
|
const old = age < 120 ? `${age} seconds` : `${Math.round(age / 60)} minutes`;
|
|
46
115
|
return ` (this copy came from a CDN cache and is ${old} old, so a recent change may not show yet)`;
|
|
47
116
|
}
|
|
117
|
+
|
|
118
|
+
const normText = (t) => String(t ?? '').toLowerCase().replace(/\s+/g, ' ').replace(/[\s.!:;]+$/, '').trim();
|
|
119
|
+
|
|
120
|
+
/** True when an operation's description adds nothing to its summary (A1 notes these). */
|
|
121
|
+
export function repeatsSummary(op) {
|
|
122
|
+
const summary = normText(op?.summary), description = normText(op?.description);
|
|
123
|
+
return Boolean(summary && description) && summary.includes(description);
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
/**
|
|
127
|
+
* True when a field has no description of its own but one of its anyOf, oneOf
|
|
128
|
+
* or allOf members does: where Zod puts it for `.nullable()` when `.describe()`
|
|
129
|
+
* comes first. `deref` resolves $ref members.
|
|
130
|
+
*/
|
|
131
|
+
export function hasVariantDescription(schema, deref = (s) => s) {
|
|
132
|
+
if (!schema || schema.description) return false;
|
|
133
|
+
return [].concat(schema.anyOf ?? [], schema.oneOf ?? [], schema.allOf ?? []).some((m) => deref(m)?.description);
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
const SYMBOLS = { '✓': 'pass', '✗': 'fail', '!': 'warn', '–': 'skip' };
|
|
137
|
+
|
|
138
|
+
/**
|
|
139
|
+
* One domain's results from a previous run: a status page (the --matrix
|
|
140
|
+
* output) or a --json report. Returns { date, results: Map(id -> { status,
|
|
141
|
+
* level }) }, or null when the domain isn't in it.
|
|
142
|
+
*/
|
|
143
|
+
export function parsePrevious(text, domain) {
|
|
144
|
+
const trimmed = String(text ?? '').trim();
|
|
145
|
+
if (trimmed.startsWith('{') || trimmed.startsWith('[')) {
|
|
146
|
+
let data;
|
|
147
|
+
try { data = JSON.parse(trimmed); } catch { return null; }
|
|
148
|
+
const rep = [].concat(data).find((r) => r?.domain === domain);
|
|
149
|
+
if (!rep || !Array.isArray(rep.results)) return null;
|
|
150
|
+
return { date: String(rep.checked_at ?? '').slice(0, 10) || null, results: new Map(rep.results.map((r) => [r.id, { status: r.status, level: r.level }])) };
|
|
151
|
+
}
|
|
152
|
+
const lines = trimmed.split('\n');
|
|
153
|
+
const header = lines.find((l) => /^\| ID \| Check \| Level \|/.test(l));
|
|
154
|
+
if (!header) return null;
|
|
155
|
+
const cols = header.split('|').slice(1, -1).map((c) => c.trim());
|
|
156
|
+
const col = cols.indexOf(domain);
|
|
157
|
+
if (col < 3) return null;
|
|
158
|
+
const results = new Map();
|
|
159
|
+
for (const line of lines) {
|
|
160
|
+
const cells = line.split('|').slice(1, -1).map((c) => c.trim());
|
|
161
|
+
if (cells.length !== cols.length || !/^[A-Z]\d+$/.test(cells[0])) continue;
|
|
162
|
+
const status = SYMBOLS[cells[col]]; // recorded rows (T5, T6) hold text, not a symbol
|
|
163
|
+
if (status) results.set(cells[0], { status, level: cells[2] });
|
|
164
|
+
}
|
|
165
|
+
return { date: trimmed.match(/^Generated (\d{4}-\d{2}-\d{2})/m)?.[1] ?? null, results };
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
/** What changed between a previous run's results and this run's, as lists of labeled IDs. */
|
|
169
|
+
export function compareResults(previous, results) {
|
|
170
|
+
const rank = { fail: 0, warn: 1, pass: 2 };
|
|
171
|
+
const out = { better: [], worse: [], newlyChecked: [], noLongerChecked: [], levelChanged: [] };
|
|
172
|
+
for (const r of results) {
|
|
173
|
+
const p = previous.get(r.id);
|
|
174
|
+
const was = p && p.status !== 'skip' ? p.status : null;
|
|
175
|
+
const now = r.status !== 'skip' ? r.status : null;
|
|
176
|
+
if (p && p.level && p.level !== r.level) out.levelChanged.push(`${r.id} (${p.level} to ${r.level})`);
|
|
177
|
+
if (!was && now) out.newlyChecked.push(`${r.id} (${r.level}, ${now})`);
|
|
178
|
+
else if (was && !now) out.noLongerChecked.push(r.id);
|
|
179
|
+
else if (was && now && was !== now) (rank[now] > rank[was] ? out.better : out.worse).push(`${r.id} (${was} to ${now})`);
|
|
180
|
+
}
|
|
181
|
+
const current = new Set(results.map((r) => r.id));
|
|
182
|
+
for (const [id, p] of previous) if (!current.has(id) && p.status !== 'skip') out.noLongerChecked.push(id);
|
|
183
|
+
return out;
|
|
184
|
+
}
|
|
185
|
+
|
|
186
|
+
/** The comparison as printable lines. */
|
|
187
|
+
export function formatComparison(cmp, since) {
|
|
188
|
+
const rows = [['Better', cmp.better], ['Worse', cmp.worse], ['Newly checked', cmp.newlyChecked], ['No longer checked', cmp.noLongerChecked], ['Level changed', cmp.levelChanged]]
|
|
189
|
+
.filter(([, ids]) => ids.length);
|
|
190
|
+
if (!rows.length) return [`Since ${since}: no changes.`];
|
|
191
|
+
return [`Since ${since}:`, ...rows.map(([label, ids]) => ` ${label}: ${ids.join(', ')}`)];
|
|
192
|
+
}
|