readiness-audit 2.2.1 → 2.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +15 -0
- package/README.md +47 -14
- package/STANDARD.md +1 -1
- package/bin/readiness-audit.mjs +17 -8
- package/package.json +2 -2
- package/src/audit.mjs +14 -10
- package/src/helpers.mjs +20 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,20 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 2.4.0 (2026-11-02)
|
|
4
|
+
|
|
5
|
+
From a review by the agents that run the scorecard daily for two projects. No scoring changes, and no new warnings or failures.
|
|
6
|
+
|
|
7
|
+
- **`--concurrency <n>` sets how many requests the audit sends to a site at once,** for a site that slows under parallel requests. Without it, nothing changes: 6 at a time for the discovery files, 4 for sitemaps and pages. The limit now also covers the sampled skill files and the MCP alias probes, which went out together before. At achurch.ai, a sweep of ten requests at a time slowed the site for minutes (2026-10-12); its own checks now send at most two. The library takes it as `audit(domain, { concurrency })`.
|
|
8
|
+
- **README:** documents `--concurrency` in the options and in "Safe to run", and says that a warning on a required check lowers the required score without changing the exit code (a site can show "required 14 of 15" with no FAIL line, because W6 warns). Two links to the research pointed at a placeholder; they now open [Who requests agent discovery files?](docs/research/who-reads-discovery-files.md).
|
|
9
|
+
- **Plans:** the status lines of the sample-by-template, proving-the-score and dependency-updates plans say 2.0.0 is published, and keep what's still open.
|
|
10
|
+
|
|
11
|
+
## 2.3.0 (2026-10-24)
|
|
12
|
+
|
|
13
|
+
From the sister project's agent, after it reviewed this one. No scoring changes.
|
|
14
|
+
|
|
15
|
+
- **Node 22 and 24 work again,** each at its current security release: the CLI accepts 22.23.4, 24.22.1 or 26.12.1, or later on the same line, and refuses older releases and end-of-life or odd-numbered lines, with advice on updating. 2.1.0 to 2.2.1 needed Node 26, so `npx readiness-audit` failed on the many machines and CI images that run 22 or 24. The audit's own defenses (refusing Brotli bodies, capping body sizes) don't depend on the Node line, and each line's security release has the October 21 fixes. CI tests all three lines with `check-latest`, and a test keeps `engines`, the docs and the CI matrix in step with `NODE_LINES`.
|
|
16
|
+
- **README:** a Quick start with fresh output and the one-line prompt to adopt the standard; "Why this exists", with the evidence up front; a "Safe to run" section for the mechanics; Lighthouse and Google's Rich Results Test in "How it fits"; and a "Next step".
|
|
17
|
+
|
|
3
18
|
## 2.2.1 (2026-10-24)
|
|
4
19
|
|
|
5
20
|
Corrections after an outside review of 2.2.0 and its research. No new checks; D8 is easier to pass.
|
package/README.md
CHANGED
|
@@ -6,35 +6,57 @@
|
|
|
6
6
|
|
|
7
7
|
**Score any site's readiness for AI agents and search:** llms.txt, robots.txt rules for AI crawlers, the sitemap, structured data, link previews, OpenAPI, MCP servers, agent skills and the A2A path. A standard built from production request logs, and a command-line audit that checks a live site against it. Read-only, no account, no API key.
|
|
8
8
|
|
|
9
|
+
## Quick start
|
|
10
|
+
|
|
9
11
|
```
|
|
10
12
|
npx readiness-audit example.com
|
|
11
13
|
```
|
|
12
14
|
|
|
15
|
+
It needs Node 22.23.4, 24.22.1 or 26.12.1, or later on the same line (each supported line's current security release), and has no dependencies.
|
|
16
|
+
|
|
13
17
|
```
|
|
14
|
-
Agent and search readiness: animalhouse.ai (readiness-audit v2.
|
|
18
|
+
Agent and search readiness: animalhouse.ai (readiness-audit v2.3.0)
|
|
15
19
|
|
|
16
20
|
Required
|
|
17
21
|
PASS D1 robots.txt 1 group with consistent rules, Sitemap listed
|
|
18
22
|
PASS D4 llms.txt and llms-full.txt llms.txt 16.2 KB, llms-full.txt 65 KB; 10 sampled links resolve
|
|
19
23
|
PASS D12 No false declarations no OAuth, A2A, plugin or server-card claims without the service
|
|
20
24
|
PASS W1 Titles and descriptions 27 sampled pages (18 static, 3 templates), all with their own
|
|
21
|
-
WARN W5 Link previews og:image size or alt text not declared on /hall, /about, +11 more
|
|
22
25
|
WARN W6 Page structure heading levels skipped on /agent/[name] (h1 to h3)
|
|
23
|
-
PASS A1 OpenAPI 3.1 OpenAPI 3.1.0; all 17 operations and their request fields described
|
|
24
26
|
PASS M4 Hosted MCP edge both eras, JSON for every Accept, browsers redirected
|
|
25
27
|
...
|
|
26
|
-
|
|
28
|
+
Recommended
|
|
29
|
+
PASS D7 JSON 404s and a JSON 405 on POST / JSON 404 for unknown well-known paths, JSON 405 for POST / (Allow: GET, HEAD, OPTIONS)
|
|
30
|
+
WARN D8 Markdown for agents Vary: Accept missing on the HTML response
|
|
31
|
+
...
|
|
32
|
+
Score on 2026-10-24: required 17 of 19, recommended 10 of 11, next level 1 of 2
|
|
27
33
|
```
|
|
28
34
|
|
|
29
|
-
|
|
35
|
+
To switch a project over, give its coding agent one line:
|
|
36
|
+
|
|
37
|
+
> Adopt the Agent and Search Readiness Standard in this repo by following https://github.com/geeks-accelerator/agent-and-search-readiness/blob/main/docs/adopting.md
|
|
38
|
+
|
|
39
|
+
The agent reads the page when it runs. To have it follow the text you reviewed, put the latest release tag in place of `main` (`blob/v<version>/docs/adopting.md`, from the [releases page](https://github.com/geeks-accelerator/agent-and-search-readiness/releases)).
|
|
40
|
+
|
|
41
|
+
## Why this exists
|
|
42
|
+
|
|
43
|
+
What agents and crawlers actually ask a site for, from production logs:
|
|
44
|
+
|
|
45
|
+
- **The AI crawlers read robots.txt and the sitemap, not llms.txt.** In a week across 56 domains on Cloudflare, bots in Cloudflare's AI categories requested robots.txt 3,644 times and the sitemap 2,232 times, and llms.txt 49 times. llms.txt still matters to other readers: directories, tools, scripts and search bots requested it on every site ([research](docs/research/who-reads-discovery-files.md)).
|
|
46
|
+
- **Directories crawl the well-known files; working agents call the API.** At animalhouse.ai, 4 of the 28 addresses that read llms.txt called the API ([§2](STANDARD.md#2-what-the-traffic-says)). Across the 16 sites with an API, 13% of llms.txt readers also got a successful API answer that week.
|
|
47
|
+
- **Bots ask for markdown.** 12,601 requests asked for markdown at ordinary page URLs that week, 92% of them from one search engine's crawler (Exa's).
|
|
48
|
+
- **A hosted MCP endpoint gets found within hours.** Seven MCP directory crawlers connected to animalhouse.ai's in its first two hours, and 14 browser visits reached it in the same two hours ([§2](STANDARD.md#2-what-the-traffic-says)).
|
|
49
|
+
- **No single file reaches every kind of agent.** Directories want the well-known files, scripts want llms.txt and the API, agent tools want clean pages, and search indexes want everything a search engine wants. The standard covers each, at a level its evidence supports.
|
|
50
|
+
|
|
51
|
+
## This repo holds
|
|
30
52
|
|
|
31
53
|
- **[STANDARD.md](STANDARD.md):** what to build, item by item, each with an ID, a level and the reason it's at that level. It covers discovery files, search (SEO, and the AI answer engines that GEO and AEO are about), the API, MCP servers, skills, engagement, measurement and tests.
|
|
32
|
-
- **[Who requests agent discovery files?](docs/research/who-reads-discovery-files.md):**
|
|
54
|
+
- **[Who requests agent discovery files?](docs/research/who-reads-discovery-files.md):** the week across 56 domains, with charts, its queries and its classifier.
|
|
33
55
|
- **`readiness-audit`:** the scorecard. It checks a live site against everything the standard can verify from outside, about 30 checks in one pass, and lists what it can't see for you to review.
|
|
34
56
|
|
|
35
57
|
## Why this one
|
|
36
58
|
|
|
37
|
-
- **Built from evidence.** Each item says why it's there: production request logs
|
|
59
|
+
- **Built from evidence.** Each item says why it's there: production request logs, the specs, and Google's and Bing's own documentation, cited with dates. Items that only scanners look for are recommended at most.
|
|
38
60
|
- **It checks that declarations are true.** An A2A card that isn't a valid A2A card fails D11; OAuth metadata with no token endpoint, or a catalog entry typed as something it isn't, fails D12. Declining an item in writing is fine; faking a pass isn't.
|
|
39
61
|
- **It fits the site.** The API checks run only when a site has or declares an API, and the MCP checks only when one answers. Pages are sampled by type: every static page, and three pages of each generated template.
|
|
40
62
|
- **Before and after you ship.** `--base http://localhost:3000` scores a local or preview build; `--compare` lists what changed since the last run; the exit code gates a deploy.
|
|
@@ -45,16 +67,16 @@ This repo holds:
|
|
|
45
67
|
| Tool | What it does | Use it with this |
|
|
46
68
|
|---|---|---|
|
|
47
69
|
| [Is Your Site Agent-Ready?](https://isitagentready.com/) (Cloudflare) | A web scan for llms.txt, MCP, agent skills and other agent-facing standards | For a quick look in the browser. This runs in your terminal and CI, scores local builds, and checks search as well as agents |
|
|
70
|
+
| [Lighthouse](https://developer.chrome.com/docs/lighthouse/overview) | Audits one page's performance, accessibility, best practices and SEO in a browser | Use it page by page, for Core Web Vitals (W8). This checks a site's discovery files and samples its pages |
|
|
71
|
+
| [Rich Results Test](https://search.google.com/test/rich-results) (Google) | Checks a page's structured data against Google's rich result rules | This lists which pages to run through it, with a link for each (W4) |
|
|
48
72
|
| llms.txt validators | Check that one file is well formed | This checks llms.txt and samples its links, as one of about 30 checks |
|
|
49
73
|
| [security-pranks-audit](https://github.com/geeks-accelerator/agent-security-pranks) | The sister project: checks that a site answers `.env`, `.git` and other secret-hunting probes safely | This welcomes agents and checks what they read; that checks what the scanners get. Same shape, run both |
|
|
50
74
|
| [specification.website](https://specification.website) | A sourced website specification: HTML, accessibility, security, SEO, agent readiness | A spec for the whole site, accessibility and security included. This one focuses on agents and search, and comes with a scorer |
|
|
51
75
|
|
|
52
76
|
## Score a site
|
|
53
77
|
|
|
54
|
-
It needs Node 26 or later, with current security releases, and has no dependencies. It only reads: GET requests, plus a few POSTs that create nothing (an MCP `initialize`, `server/discover` and `tools/list`, and an empty POST to `/`). It keeps to about six requests at a time, and it won't follow a link the site supplies (in its sitemap, share images, skills, catalog or DNS record) to a private or local address, at any redirect. It asks for gzip and decodes it itself; a body in an encoding it didn't ask for, such as Brotli, is reported rather than read, so a site can't reach a decoder bug on the machine running the audit. It prints a pass, warning or failure for each check, with the reason, and exits with code 1 when a required check fails, so it can gate a deploy.
|
|
55
|
-
|
|
56
78
|
```
|
|
57
|
-
readiness-audit <domain> [--mcp /path] [--no-mcp] [--json] [--compare previous] [--base url]
|
|
79
|
+
readiness-audit <domain> [--mcp /path] [--no-mcp] [--json] [--compare previous] [--base url] [--concurrency n]
|
|
58
80
|
readiness-audit --matrix <domain> <domain> ... [--recorded results.json]
|
|
59
81
|
readiness-audit --logs <access-log> [--json]
|
|
60
82
|
```
|
|
@@ -66,15 +88,22 @@ readiness-audit --logs <access-log> [--json]
|
|
|
66
88
|
| `--json` | print the results as JSON |
|
|
67
89
|
| `--compare file` | list what changed since a previous run: a status page from `--matrix` or a saved `--json` report |
|
|
68
90
|
| `--base url` | score a local or preview build as the domain before you deploy (`--base http://localhost:3000`); DNS and host redirects are skipped |
|
|
91
|
+
| `--concurrency n` | send at most n requests to the site at once (default 6 for the discovery files, 4 for sitemaps and pages). Lower it for a site that slows under parallel requests; the checks are the same |
|
|
69
92
|
| `--matrix` | score several sites and print a markdown status page |
|
|
70
93
|
| `--recorded file` | with `--matrix`, add your recorded T5 and T6 results (format: [examples/recorded.json](examples/recorded.json)) |
|
|
71
94
|
| `--logs file` | count requests for the discovery files in an exported access log (combined log format, or JSON lines from Cloudflare, Vercel or Railway), by client family. It prints counts, never addresses |
|
|
72
95
|
|
|
73
|
-
The API checks run when `/openapi.json` or `/api` answers, or when the site declares an API (a `service-desc` link, an API catalog, an API link in llms.txt). A declared API that doesn't answer fails them. A site with neither skips them, the way a site with no hosted MCP endpoint skips the MCP checks.
|
|
96
|
+
It prints a pass, warning or failure for each check, with the reason, and exits with code 1 when a required check fails, so it can gate a deploy. A warning on a required check, such as W6's heading jumps, lowers the required score ("required 14 of 15") without changing the exit code; only a failure does. The API checks run when `/openapi.json` or `/api` answers, or when the site declares an API (a `service-desc` link, an API catalog, an API link in llms.txt). A declared API that doesn't answer fails them. A site with neither skips them, the way a site with no hosted MCP endpoint skips the MCP checks.
|
|
74
97
|
|
|
75
|
-
From a clone, run `node bin/readiness-audit.mjs example.com`.
|
|
98
|
+
To check fixes before they ship, score a local build: `npx readiness-audit@2 example.com --base http://localhost:3000`. The site's own URLs (from its sitemap, llms.txt, skills and catalog) are requested from the base and reported under the domain. From a clone, run `node bin/readiness-audit.mjs example.com`.
|
|
76
99
|
|
|
77
|
-
|
|
100
|
+
### Safe to run
|
|
101
|
+
|
|
102
|
+
- **It only reads:** GET requests, plus a few POSTs that create nothing (an MCP `initialize`, `server/discover` and `tools/list`, and an empty POST to `/`), at most six at a time. With `--matrix`, each site gets its own six.
|
|
103
|
+
- **It can go slower.** A site that slows under parallel requests can be audited with `--concurrency 2`, or `--concurrency 1` for one request at a time. The same checks run, and take longer.
|
|
104
|
+
- **It stays on public addresses.** It won't follow a link the site supplies (in its sitemap, share images, skills, catalog or DNS record) to a private or local address, at any redirect. `--base` may be local, because you chose it.
|
|
105
|
+
- **A hostile site can't reach a decoder bug.** It asks for gzip and decodes it itself; a body in an encoding it didn't ask for, such as Brotli, is reported rather than read, and every body is capped at 16 MB.
|
|
106
|
+
- **Run it only against sites you run,** or with the owner's permission. [SECURITY.md](SECURITY.md) has the details and how to report a problem.
|
|
78
107
|
|
|
79
108
|
## What it checks, and what it can't
|
|
80
109
|
|
|
@@ -96,7 +125,7 @@ It can't see whether `next_steps` guide well, whether input is forgiving, whethe
|
|
|
96
125
|
|
|
97
126
|
Point to [STANDARD.md](STANDARD.md) from your repo instead of copying it (§14 has a short stub), and pin the scorecard to a major version: `npx readiness-audit@2 <domain>`. To run it on a schedule in GitHub Actions, see [examples/github-action.yml](examples/github-action.yml).
|
|
98
127
|
|
|
99
|
-
To switch a project over,
|
|
128
|
+
To switch a project over, give its coding agent the one-line prompt in [Quick start](#quick-start). [docs/adopting.md](docs/adopting.md) has the step-by-step instructions it follows, with lessons from the first projects that did it.
|
|
100
129
|
|
|
101
130
|
## Where it comes from
|
|
102
131
|
|
|
@@ -104,6 +133,10 @@ Six projects wrote down what they do for agents and search: [animalhouse.ai](htt
|
|
|
104
133
|
|
|
105
134
|
An item is in the standard when most projects should do it, not because one project happens to. Specs that are still drafts are marked as such, with dates. If you find something wrong or out of date, open an issue or a pull request.
|
|
106
135
|
|
|
136
|
+
## Next step
|
|
137
|
+
|
|
138
|
+
Score your site with `npx readiness-audit your-domain`, read what it reports against [STANDARD.md](STANDARD.md), and when you're ready to switch a project over, give its coding agent the one-line prompt in [Quick start](#quick-start).
|
|
139
|
+
|
|
107
140
|
## Contributing
|
|
108
141
|
|
|
109
142
|
See [CONTRIBUTING.md](CONTRIBUTING.md). To report a security problem, see [SECURITY.md](SECURITY.md).
|
package/STANDARD.md
CHANGED
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
**Last verified:** 2026-10-05
|
|
6
6
|
|
|
7
7
|
> **Start here**
|
|
8
|
-
> 1. Score your site: `npx readiness-audit <domain>`. It needs Node 26 or later
|
|
8
|
+
> 1. Score your site: `npx readiness-audit <domain>`. It needs Node 22.23.4, 24.22.1 or 26.12.1, or later on the same line, and no dependencies.
|
|
9
9
|
> 2. Check [§3](#3-decide-what-applies) for the items that apply to your project.
|
|
10
10
|
> 3. Fix required failures first. Each line the scorecard prints names an ID and the reason.
|
|
11
11
|
> 4. Then the recommended items. Build a next-level item when your own traffic shows the problem it solves.
|
package/bin/readiness-audit.mjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
// readiness-audit: score a live site against the Agent and Search Readiness Standard.
|
|
3
3
|
//
|
|
4
|
-
// readiness-audit <domain> [--mcp /path] [--no-mcp] [--json] [--compare previous] [--base url]
|
|
4
|
+
// readiness-audit <domain> [--mcp /path] [--no-mcp] [--json] [--compare previous] [--base url] [--concurrency n]
|
|
5
5
|
// readiness-audit --matrix <domain> <domain> ... [--recorded results.json] (a markdown status page)
|
|
6
6
|
// readiness-audit --logs <access-log> [--json] (who requests the discovery files)
|
|
7
7
|
//
|
|
@@ -10,13 +10,12 @@
|
|
|
10
10
|
import { readFileSync, createReadStream } from 'node:fs';
|
|
11
11
|
import { createInterface } from 'node:readline';
|
|
12
12
|
import { audit, matrix, score, LEVELS, REVIEW, VERSION } from '../src/audit.mjs';
|
|
13
|
-
import { parsePrevious, compareResults, formatComparison } from '../src/helpers.mjs';
|
|
13
|
+
import { parsePrevious, compareResults, formatComparison, nodeOk, NODE_REQUIREMENT } from '../src/helpers.mjs';
|
|
14
14
|
import { logTally, formatLogs } from '../src/logs.mjs';
|
|
15
15
|
|
|
16
16
|
// engines in package.json only warns, so say it plainly before anything fails oddly.
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
console.error(`readiness-audit needs Node ${NODE_MAJOR} or later, with current security releases (this is ${process.versions.node}).`);
|
|
17
|
+
if (!nodeOk(process.versions.node)) {
|
|
18
|
+
console.error(`readiness-audit needs ${NODE_REQUIREMENT} (each supported line's current security release); this is ${process.versions.node}. Update with your version manager (nvm install 24), or set node-version: 24 with check-latest: true in CI.`);
|
|
20
19
|
process.exit(2);
|
|
21
20
|
}
|
|
22
21
|
|
|
@@ -32,9 +31,15 @@ const recordedPath = valueOf('--recorded');
|
|
|
32
31
|
const comparePath = valueOf('--compare');
|
|
33
32
|
const base = valueOf('--base');
|
|
34
33
|
const logsPath = valueOf('--logs');
|
|
35
|
-
const
|
|
34
|
+
const concurrencyOpt = valueOf('--concurrency');
|
|
35
|
+
const domains = args.filter((a, i) => !a.startsWith('-') && !['--mcp', '--recorded', '--compare', '--base', '--logs', '--concurrency'].includes(args[i - 1]))
|
|
36
36
|
.map((d) => d.replace(/^https?:\/\//, '').replace(/\/.*$/, ''));
|
|
37
|
-
const opts = { mcpPath: flag('--no-mcp') ? null : (mcpOpt ?? '/mcp'), base: base ?? null };
|
|
37
|
+
const opts = { mcpPath: flag('--no-mcp') ? null : (mcpOpt ?? '/mcp'), base: base ?? null, concurrency: null };
|
|
38
|
+
if (flag('--concurrency')) {
|
|
39
|
+
const n = Number(concurrencyOpt);
|
|
40
|
+
if (!Number.isInteger(n) || n < 1) { console.error(`--concurrency needs a whole number of 1 or more, like --concurrency 2 (got ${concurrencyOpt})`); process.exit(2); }
|
|
41
|
+
opts.concurrency = n;
|
|
42
|
+
}
|
|
38
43
|
if (base) {
|
|
39
44
|
let u;
|
|
40
45
|
try { u = new URL(base); } catch { u = null; }
|
|
@@ -55,7 +60,7 @@ if (!domains.length || flag('--help') || flag('-h')) {
|
|
|
55
60
|
console.error(`readiness-audit ${VERSION}
|
|
56
61
|
|
|
57
62
|
Usage:
|
|
58
|
-
readiness-audit <domain> [--mcp /path] [--no-mcp] [--json] [--compare previous] [--base url]
|
|
63
|
+
readiness-audit <domain> [--mcp /path] [--no-mcp] [--json] [--compare previous] [--base url] [--concurrency n]
|
|
59
64
|
readiness-audit --matrix <domain> <domain> ... [--recorded results.json]
|
|
60
65
|
readiness-audit --logs <access-log> [--json]
|
|
61
66
|
|
|
@@ -70,6 +75,10 @@ Options:
|
|
|
70
75
|
--base url score a local or preview build as the domain, before you
|
|
71
76
|
deploy (http://localhost:3000); DNS and host redirects are
|
|
72
77
|
skipped
|
|
78
|
+
--concurrency n
|
|
79
|
+
at most n requests to the site at once (default 6 for
|
|
80
|
+
the discovery files, 4 for sitemaps and pages); lower it
|
|
81
|
+
for a site that slows under parallel requests
|
|
73
82
|
--logs f count requests for the discovery files in an exported access
|
|
74
83
|
log (combined log format or JSON lines), by client family;
|
|
75
84
|
prints counts, never addresses
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "readiness-audit",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.4.0",
|
|
4
4
|
"description": "Score any site's readiness for AI agents and search (GEO, AEO, AI SEO): llms.txt, robots.txt AI crawler rules, structured data, link previews, OpenAPI, MCP, agent skills and A2A. Read-only CLI, no account.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
"test": "node --test"
|
|
20
20
|
},
|
|
21
21
|
"engines": {
|
|
22
|
-
"node": ">=26"
|
|
22
|
+
"node": "^22.23.4 || ^24.22.1 || >=26.12.1"
|
|
23
23
|
},
|
|
24
24
|
"keywords": [
|
|
25
25
|
"ai-agents",
|
package/src/audit.mjs
CHANGED
|
@@ -4,14 +4,14 @@
|
|
|
4
4
|
// The standard and what each ID means: STANDARD.md in
|
|
5
5
|
// https://github.com/geeks-accelerator/agent-and-search-readiness
|
|
6
6
|
// Read-only: GETs plus a few POSTs that create nothing (an MCP initialize, server/discover and
|
|
7
|
-
// tools/list, an empty POST to /). Node
|
|
7
|
+
// tools/list, an empty POST to /). Node 22, 24 or 26 at its security release, no dependencies.
|
|
8
8
|
|
|
9
9
|
import { lookup, resolveTxt, Resolver } from 'node:dns/promises';
|
|
10
10
|
import { isIP } from 'node:net';
|
|
11
11
|
import { request } from './http.mjs';
|
|
12
12
|
import { SKILL_NAME, sampleSkillEntries, skillNameProblem, cdnCacheNote, repeatsSummary, hasVariantDescription, spread, samplePages, sharedCardText, API_LINK, MCP_LINK, linkRels, apiPresence, DISCOVERY_FILES, AI_BOTS, missingSections, richResultPages, list, norm, pathDepth, decodeText, headings, headingSkips, sitemapEntries, llmsLinks, baseMapper, formatRecorded } from './helpers.mjs';
|
|
13
13
|
|
|
14
|
-
export const VERSION = '2.
|
|
14
|
+
export const VERSION = '2.4.0';
|
|
15
15
|
export const REPO = 'https://github.com/geeks-accelerator/agent-and-search-readiness';
|
|
16
16
|
const UA = `readiness-audit/${VERSION} (+${REPO})`;
|
|
17
17
|
const AID_PROTOCOLS = ['mcp', 'a2a', 'openapi', 'grpc', 'graphql', 'websocket', 'local', 'zeroconf', 'ucp'];
|
|
@@ -105,8 +105,12 @@ export function makeGuard(domain) {
|
|
|
105
105
|
return { siteHosts, isSafe };
|
|
106
106
|
}
|
|
107
107
|
|
|
108
|
-
export async function audit(domain, { mcpPath = '/mcp', base = null } = {}) {
|
|
108
|
+
export async function audit(domain, { mcpPath = '/mcp', base = null, concurrency = null } = {}) {
|
|
109
109
|
const BASE = `https://${domain}`;
|
|
110
|
+
// Requests in flight to the site at once. By default, 6 for the discovery files and 4 for
|
|
111
|
+
// sitemaps and pages; `concurrency` sets one limit for all of them, for a site that slows
|
|
112
|
+
// under parallel requests.
|
|
113
|
+
const limit = (byDefault) => concurrency ?? byDefault;
|
|
110
114
|
const rand = Math.random().toString(36).slice(2, 10);
|
|
111
115
|
|
|
112
116
|
const { siteHosts, isSafe } = makeGuard(domain);
|
|
@@ -178,7 +182,7 @@ export async function audit(domain, { mcpPath = '/mcp', base = null } = {}) {
|
|
|
178
182
|
() => get('/', { method: 'POST', redirect: 'manual' }),
|
|
179
183
|
() => get(D.legacyCard),
|
|
180
184
|
() => get(D.wkCard),
|
|
181
|
-
], 6, (job) => job());
|
|
185
|
+
], limit(6), (job) => job());
|
|
182
186
|
const catalogEntries = [...(json(ard)?.entries ?? []), ...(json(aiCatalog)?.entries ?? [])];
|
|
183
187
|
const rels = linkRels(home.headers.get('link'));
|
|
184
188
|
const api = apiPresence({ openapiSpec: json(openapi), apiIndexJson: ok(apiIndex) && isJson(apiIndex), rels, catalogJson: ok(apiCatalog) ? json(apiCatalog) : null, llmsText: ok(llms) ? llms.text : '' });
|
|
@@ -231,7 +235,7 @@ export async function audit(domain, { mcpPath = '/mcp', base = null } = {}) {
|
|
|
231
235
|
// ---- The sitemap, read once for D2 and the W checks: an index is expanded to its pages ----
|
|
232
236
|
const sitemapIndex = /<sitemapindex/.test(sitemap.text);
|
|
233
237
|
const sitemapLists = sitemapIndex
|
|
234
|
-
? await mapLimit(sitemapEntries(sitemap.text).slice(0, 10), 4, async (e) => sitemapEntries((await get(e.loc)).text))
|
|
238
|
+
? await mapLimit(sitemapEntries(sitemap.text).slice(0, 10), limit(4), async (e) => sitemapEntries((await get(e.loc)).text))
|
|
235
239
|
: [sitemapEntries(sitemap.text)];
|
|
236
240
|
const seenLocs = new Set();
|
|
237
241
|
const entries = sitemapLists.flat().filter((e) => {
|
|
@@ -258,7 +262,7 @@ export async function audit(domain, { mcpPath = '/mcp', base = null } = {}) {
|
|
|
258
262
|
// Agents follow the links in llms.txt, so a sample of the site's own links must answer 200.
|
|
259
263
|
const links = ok(llms) ? llmsLinks(llms.text, `${BASE}/llms.txt`, siteHosts) : [];
|
|
260
264
|
const sample = spread(links, 10);
|
|
261
|
-
const answers = await mapLimit(sample, 4, async (u) => ({ u, r: await get(u) }));
|
|
265
|
+
const answers = await mapLimit(sample, limit(4), async (u) => ({ u, r: await get(u) }));
|
|
262
266
|
llmsAnswers = answers;
|
|
263
267
|
// Missing pages, server errors and no answer are broken. A 401, 403 or 405 is an endpoint that
|
|
264
268
|
// exists but wants a key or another method, which is fine to list.
|
|
@@ -481,14 +485,14 @@ export async function audit(domain, { mcpPath = '/mcp', base = null } = {}) {
|
|
|
481
485
|
} else {
|
|
482
486
|
// Entry URLs may be relative, and resolve against the index (RFC 3986).
|
|
483
487
|
const indexUrl = `${BASE}/.well-known/agent-skills/index.json`;
|
|
484
|
-
const files = await
|
|
488
|
+
const files = await mapLimit(sample, limit(5), async (e) => {
|
|
485
489
|
const url = new URL(e.url, indexUrl).href;
|
|
486
490
|
const r = await get(url);
|
|
487
491
|
const fm = r.text.match(/^---\r?\n([\s\S]*?)\r?\n---/)?.[1] ?? '';
|
|
488
492
|
const field = (k) => fm.match(new RegExp(`^${k}:\\s*(?:"([^"]*)"|'([^']*)'|(.*))$`, 'm'))?.slice(1).find((v) => v !== undefined)?.trim() ?? '';
|
|
489
493
|
const label = typeof e.name === 'string' && e.name ? e.name : url;
|
|
490
494
|
return { url, status: r.status, text: r.text, name: field('name'), description: field('description'), entryName: e.name, label, lines: r.text.split('\n').length };
|
|
491
|
-
})
|
|
495
|
+
});
|
|
492
496
|
const unreachable = files.filter((f) => f.status !== 200).map((f) => `${f.label} (${f.status})`);
|
|
493
497
|
const read = files.filter((f) => f.status === 200);
|
|
494
498
|
const badName = read.map((f) => [f, skillNameProblem(f)]).filter(([, why]) => why).map(([f, why]) => `${f.label} (${why})`);
|
|
@@ -553,7 +557,7 @@ export async function audit(domain, { mcpPath = '/mcp', base = null } = {}) {
|
|
|
553
557
|
variants.push(['(none)', await get(mcpPath, { method: 'POST', headers: { 'content-type': 'application/json', accept: null }, body: INIT })]);
|
|
554
558
|
const discover = await modern('server/discover', 2);
|
|
555
559
|
const browser = await get(mcpPath, { headers: { accept: 'text/html' }, redirect: 'manual' });
|
|
556
|
-
const aliases = await
|
|
560
|
+
const aliases = await mapLimit(['/sse', '/api/mcp'], limit(2), (p) => get(p, { method: 'POST', redirect: 'manual', headers: { 'content-type': 'application/json' }, body: INIT }));
|
|
557
561
|
const problems = [];
|
|
558
562
|
const refused = variants.filter(([, r]) => r.status !== 200).map(([a, r]) => `${a} → ${r.status}`);
|
|
559
563
|
if (refused.length) problems.push(`2025-era initialize refused for Accept ${refused.join(', ')}`);
|
|
@@ -651,7 +655,7 @@ export async function audit(domain, { mcpPath = '/mcp', base = null } = {}) {
|
|
|
651
655
|
ldBroken: ld.filter((x) => x === null).length, types,
|
|
652
656
|
};
|
|
653
657
|
};
|
|
654
|
-
const pages = await mapLimit(pick, 4, async (u) => parsePage(u, await get(u, { headers: { accept: 'text/html' } })));
|
|
658
|
+
const pages = await mapLimit(pick, limit(4), async (u) => parsePage(u, await get(u, { headers: { accept: 'text/html' } })));
|
|
655
659
|
// A body the audit refused (an encoding it doesn't decode) is reported once, in W3, not as missing tags everywhere.
|
|
656
660
|
const ok200 = pages.filter((p) => p.status === 200 && p.html && !p.error);
|
|
657
661
|
if (ok200.length) { const sizes = ok200.map((p) => p.bytes).sort((a, b) => a - b); htmlKb = Math.round(sizes[Math.floor(sizes.length / 2)] / 1024); }
|
package/src/helpers.mjs
CHANGED
|
@@ -20,6 +20,26 @@ export function spread(list, n) {
|
|
|
20
20
|
return list.length <= n ? list : Array.from({ length: n }, (_, i) => list[Math.floor((i * list.length) / n)]);
|
|
21
21
|
}
|
|
22
22
|
|
|
23
|
+
/**
|
|
24
|
+
* Each supported Node line at its current security release (the 2026-10-21 release fixed a
|
|
25
|
+
* Brotli decoder bug and TLS hostname bypasses in all three). TLS checks and the decoders are
|
|
26
|
+
* Node's, so being on a line's latest security release matters more than which line. End-of-life
|
|
27
|
+
* and odd-numbered lines get no security releases and are refused. Raise these after each Node
|
|
28
|
+
* security release, with package.json's engines and the docs (a test checks they agree).
|
|
29
|
+
*/
|
|
30
|
+
export const NODE_LINES = { 22: '22.23.4', 24: '24.22.1', 26: '26.12.1' };
|
|
31
|
+
export const NODE_REQUIREMENT = `Node ${Object.values(NODE_LINES).slice(0, -1).join(', ')} or ${Object.values(NODE_LINES).at(-1)}, or later on the same line`;
|
|
32
|
+
|
|
33
|
+
/** Whether a Node version is a supported line at its security release, or an even line newer than these. */
|
|
34
|
+
export function nodeOk(version) {
|
|
35
|
+
const parts = (v) => v.split('.').map(Number);
|
|
36
|
+
const atLeast = (v, min) => { const [a, b] = [parts(v), parts(min)]; const i = a.findIndex((n, j) => n !== b[j]); return i < 0 || a[i] > b[i]; };
|
|
37
|
+
const major = parts(version)[0];
|
|
38
|
+
const newest = Math.max(...Object.keys(NODE_LINES).map(Number));
|
|
39
|
+
if (major > newest) return major % 2 === 0; // an even line newer than this release knows; odd lines get no long-term support
|
|
40
|
+
return NODE_LINES[major] !== undefined && atLeast(version, NODE_LINES[major]);
|
|
41
|
+
}
|
|
42
|
+
|
|
23
43
|
/** Up to three items, then how many more: "a, b, c, +4 more". */
|
|
24
44
|
export function list(xs, sep = ', ') {
|
|
25
45
|
return `${xs.slice(0, 3).join(sep)}${xs.length > 3 ? `, +${xs.length - 3} more` : ''}`;
|