ax-audit 3.6.0 → 4.1.0

package/CHANGELOG.md

@@ -2,6 +2,157 @@
 
 All notable changes to ax-audit are documented here.
 
+## [4.1.0] - 2026-09-04
+
+**The reference tables are importable.** A consumer could always run an audit and never describe one: `CHECK_WEIGHTS`, `CHECK_CATEGORIES` and the crawler lists sat behind an `exports` map that admits only the entry point, so anything printing what a check is worth, grouping results by area, or naming the crawlers worth naming kept a hand-copied table and a test to catch the day it went stale. That is a duplication this package caused.
+
+### Added
+
+- **`VERSION`, `CHECK_WEIGHTS`, `CHECK_CATEGORIES`, `GRADES`** exported from the entry point.
+- **`AI_CRAWLERS`, `ALL_AI_CRAWLERS`, `CORE_AI_CRAWLERS`, `LEGACY_AI_CRAWLERS`, `CRAWLER_META`, `CONTENT_SIGNALS`**, with `crawlerInfo`, `crawlerPurpose` and `legacyCrawlerNote` for looking a token up, and the `CrawlerInfo` / `CrawlerPurpose` types.
+- **`categoryScore(results, category)`** exported. It was in `cli.ts`, which meant importing `commander` to reach it; it belongs beside `calculateOverallScore` and now lives there. `--fail-on-category` is one caller and a report grouped by area is another.
+- **`test/public-api.test.js`**, which fails both ways: on a promised export that disappeared, and on a new name that leaked out of a refactor. It also holds the tables against the registered checks — every shipped check is weighed and categorised, nothing retired lingers in either, no crawler token sits in two purposes, and a check declaring its own `meta.category` agrees with the table.
+
+Nothing else changed: no check, no weight, no score. The rest of `constants.ts` stays private, because a consumer pinned to a threshold or a required-field list would make every check's internals a breaking change.
+
+## [4.0.0] - 2026-09-04
+
+**Scores change on every site.** This release redistributes the weights, makes protocol checks conditional, and retires a check whose premise did not survive verification. Baselines written by earlier versions are read normally, but regression gating is suspended across the change — a rescore is not something the site did.
+
+### Breaking
+
+- **Weights redistributed by what actually stops an agent.** Content leads at 33, because the failure that breaks the most agents is a page with nothing in its HTML: most crawlers do not run JavaScript, so content that appears only after hydration is invisible no matter how many discovery files a site publishes. `html-rendering` is the highest-weighted check at 11. Access is second at 24, because a firewall rule or a `nosnippet` directive silently undoes everything else. Discovery drops to 21, with llms.txt demoted from 11 to 5 on the evidence that most published files are never fetched by an AI search crawler.
+- **Protocol checks are conditional.** A blog has no API to describe and is not an agent; scoring it zero said something false and made the overall number unreadable. Each protocol check now detects whether the site has the corresponding surface, reports **n/a** when it does not, and leaves the denominator. When the surface *is* present, a missing description is a stronger finding than before, and the wording says so: not "no MCP server discovery found" but "MCP server present but not discoverable".
+- **`well-known-ai` retired.** Three of its five scored files had no consumer: `nlweb.json` appears in no NLWeb release, `genai.txt` has no specification, and `ai-plugin.json` described a product shut down in 2024. Its useful probes moved into the checks that own them in 3.8.
+- **The 3.x scoring freezes are gone.** `robots-txt` scores against all twelve current core crawlers rather than the frozen eight, so the configuration that scored 100 in 3.6 now scores 90. Blocking a crawler deducts by purpose: 5 for a search crawler, whose absence removes the site from that assistant's answers, and 2 for a training crawler, where blocking is a defensible policy choice.
+- **`CheckMeta.weight` is now optional** and nothing uses it. Weights live in `CHECK_WEIGHTS` alone. Having them in two places is what let them drift.
+
+### Added
+
+- **`--profile api|mcp|agent|docs|commerce|all`** forces conditional checks applicable, for auditing against what a site intends to build rather than what it already has.
+- **`--category content|discovery|access|policy|protocols`** runs one area. It narrows an explicit `--checks` selection rather than replacing it.
+- **`--fail-on-category access:70,content:80`** gates CI per area. An overall score hides an area that is entirely broken: a site can score 80 while every access check fails because the other four carry it. Areas with nothing applicable are reported as not evaluated rather than failed.
+- **Versioned baselines.** `schemaVersion` records which scoring model produced a baseline; a mismatch shows the deltas but suspends gating and says how to resume it. Baselines also record which checks were N/A, so gaining or losing a surface is not read as a regression or an improvement.
+
+### Migration
+
+1. Expect a different score. Sites with no API or MCP surface generally rise, because they are no longer marked down for lacking things they do not have. A bare page rose from 20 to 50 in testing; a documentation site with real infrastructure rose from 64 to 77.
+2. Re-save your baseline: `ax-audit <url> --save-baseline .ax-baseline.json`. Until you do, `--fail-on-regression` reports but does not gate.
+3. `--checks agent-json`, `--checks mcp` and `--checks openapi` still work; the ids were renamed in 3.7 and the old names remain aliases.
+4. If you relied on `well-known-ai`, its live probes are now in `usage-policy` (TDMRep) and the checks that own each file.
+
+### Fixed
+
+- **JSON-LD was missed when `type` was not the first attribute** on the script tag. Next.js and several other frameworks emit `<script id="…" type="application/ld+json">`, so a large class of well-marked-up sites scored 0 on a 6-weight check and were told to add markup they already had. Three copies of the pattern now share one order-independent extractor.
+
+### Tests
+
+873 total. New coverage for surface detection, the weight distribution (sums to 100, no check declares its own, content outweighs every area), the CLI end to end via a subprocess, and baseline versioning in both directions.
+
+## [3.9.0] - 2026-09-04
+
+Depth rather than breadth. One new check, and the existing readability checks extended to cover what agents actually do with a page once they reach it. Everything added is informational; scores are unchanged.
+
+### Added
+
+- **agent-operability**: whether a browser agent can work the page or only look at it. Every major browser agent reads the accessibility tree, so a `<div onclick>` styled as a button does not appear at all — the agent does not see a button it cannot press, it sees nothing. Checks accessible names on buttons and links, labels on form controls, elements dressed as controls, dead links, table headers, iframe titles, `<time datetime>`, heading skips, unsized media, `<html lang>`, and entry-page obstacles. Reports proportions with a 90% threshold, and states on every run that it reads markup rather than a rendered tree.
+
+### Changed
+
+- **llms.txt** gained the 2026 specification revision's additions — subpath scoping via `rel="describedby"`, per-page `.md` mirrors — plus link-health sampling. An llms.txt is written once and never revalidated while the site moves underneath it, so an index of dead links is its characteristic failure, and it wastes exactly the budget the file exists to save. Dead links are reported separately from redirecting ones. Every report now states who actually reads this file.
+- **content-negotiation** stopped probing with a header no client sends. A bare `Accept: text/markdown` would pass against an implementation that fails every real request; the probe now sends what Claude Code, Cursor and OpenCode send. Added: origin-reported token counts preferred over a byte ratio, frontmatter validation (stripping the HTML also strips the canonical link, leaving an agent unable to attribute what it quotes), and probes for user-agent negotiation and `.md` suffix URLs before concluding a site has no Markdown.
+- **structured-data** now reads provenance and freshness — `author`, `sameAs`, `publisher`, `dateModified` — and compares `headline` and `name` against the visible text, which is Google's one explicit requirement for structured data and AI features.
+- **crawl-efficiency** reports a page's cost in tokens rather than bytes, naming the markup share an agent pays to receive and then discards, plus response time in agent terms.
+
+### Tests
+
+828 total, up from 777.
+
+## [3.8.0] - 2026-09-04
+
+Eight new checks, all weight 0, plus the reporting machinery they need. Where 3.7 corrected what ax-audit was looking for, 3.8 covers the signals it was not looking at.
+
+### Added — reporting
+
+- **Not-applicable checks.** A commerce-protocol check scoring zero on a personal blog says the blog is badly built; the honest answer is that the question does not arise. `CheckResult.applicable: false` excludes a check from both numerator and denominator, and reports render it as `n/a`. Everything counted against a site is now something the site could have done. A check whose meta exists but produced no result — because it crashed — still counts at full weight, so a broken check cannot inflate a score.
+- **Category grouping.** Terminal, Markdown and HTML output group checks under content, discovery, access, policy and protocols. Twenty-six scores in a flat list do not tell a reader where the problem is.
+- **`renderHtml` / `renderBatchHtml`**, returning the document as a string the way `renderMarkdown` already did. The HTML reporter could previously only print to stdout, which made it untestable and unusable from the programmatic API.
+
+### Added — checks
+
+- **ai-directives**: the page-level controls Google and Microsoft document that they honor — `nosnippet` and `max-snippet` for AI Overviews and AI Mode, `noarchive` and `nocache` for Copilot grounding, `data-nosnippet` judged by what it wraps. The finding it exists for: a site that disallows `Google-Extended` expecting to leave AI Overviews has done no such thing, because that token governs Gemini training and grounding while AI Overviews follow Googlebot and the snippet directives.
+- **usage-policy**: normalises Content Signals, AIPREF, RSL, TDMRep and `noai` onto three questions — train, ground, index — and reports where they disagree. A site whose robots.txt permits training while its RSL licence prohibits it has published two contradictory positions, and which one applies depends on which file a crawler read. Every report states that only robots.txt access rules are documented as honored by major operators.
+- **http-hygiene**: status-code honesty. An agent cannot see that the 200 it received is a "page not found" screen; it stores the apology as the answer. Also redirect depth, `HEAD` support, `Retry-After` on 429, charset, and `<html lang>` against `Content-Language`.
+- **ai-catalog**: the emerging single index of everything callable on a site, discovered the four ways Lighthouse's own audit accepts. Both specifications are drafts, so absence never scores — but a catalog whose entries point at documents that 404 does, because an agent trusts it before it fetches.
+- **agent-skills**: installable procedures (SKILL.md) across all three competing discovery paths, validating the index and fetching a sample of the documents. Conditional on the site having a developer-facing surface.
+- **webmcp**: forms declared as callable tools. Never asks for WebMCP, which is a Community Group draft in a Chrome origin trial; catches the mistakes static analysis can catch with certainty on pages that adopted it.
+- **commerce-discovery**: the Universal Commerce Protocol profile, the one agentic-commerce specification with published site-side discovery. Validates the profile and resolves the schema URLs it promises. The alternatives define no manifest, and the report says so rather than inventing paths.
+- **auth-discovery**: the RFC 9728 chain from `WWW-Authenticate` to the authorization server's metadata. A human hitting a 401 reads the documentation; an agent cannot.
+
+### Added — infrastructure
+
+- `checks/frontmatter.ts`: a focused YAML frontmatter reader for SKILL.md and Markdown mirrors. ax-audit has two runtime dependencies and will not grow a YAML parser, so the reader handles the flat `key: value` block these documents use and records anything more elaborate as skipped rather than guessing.
+
+### Fixed
+
+- **A lone `Offer` is a price, not a storefront.** Found by running commerce-discovery against a real SaaS site, which was told to build a commerce integration because it prices its plans with structured data. A bare `Offer` now counts only alongside a second signal such as a cart link.
+- `buildResult` falls back to `CHECK_CATEGORIES`, so every result carries its category in JSON output too. Previously only checks setting `meta.category` explicitly did, leaving JSON consumers to reimplement the fallback the other reporters already had.
+
+### Scoring
+
+Unchanged. All eight checks carry weight 0 and gain weight in 4.0.
+
+### Tests
+
+755 total, up from 553. New suites: ai-directives, usage-policy, http-hygiene, ai-catalog, agent-skills, webmcp, commerce-discovery, auth-discovery, frontmatter, and the N/A arithmetic including the crashed-check case.
+
+## [3.7.0] - 2026-09-04
+
+A correction release. Three checks were probing paths that are no longer, or never were, the standard, and the crawler catalogue had drifted far enough to contain tokens that do not exist. Everything here was re-verified against vendor documentation, IANA, IETF datatracker and the relevant specification repositories on 2026-09-04.
+
+**No score goes down.** Every correction that would have lowered an existing score is frozen behind a constant that is removed in 4.0, and tests assert it.
+
+### Fixed — wrong paths
+
+- **A2A Agent Card moved.** The check probed `/.well-known/agent.json`. A2A relocated the card to `/.well-known/agent-card.json` in v0.3.0 (2025-07-30), and that path is IANA-registered. Both are probed, the registered one first; a card served only from the old path is validated and flagged. The check also handles both spec generations: A2A 1.0 (2026-03-12) folded `url`, `protocolVersion`, `preferredTransport` and `additionalInterfaces` into `supportedInterfaces[]`, and the generation is detected from the card's own structure rather than a version field. `authentication`, removed in 0.2.x in favour of `securitySchemes`, is now flagged. Renamed `agent-json` → `agent-card`.
+- **`/.well-known/mcp.json` was never an MCP convention.** ax-audit recommended it before the ecosystem settled. Discovery now walks the real chain: `/.well-known/ai-catalog.json`, `/.well-known/mcp/server-card.json`, `<endpoint>/server-card`, then the legacy manifest. Server cards deliberately carry no `tools[]` — tool lists come from a live `tools/list` call. Protocol versions are checked against the five released revisions, with `2026-07-28` current. Renamed `mcp` → `mcp-discovery`.
+- **`/.well-known/openapi.json` is a folk convention**, so a site publishing at `/openapi.json` scored zero. Discovery now runs in order of authority: RFC 9727's registered `/.well-known/api-catalog`, then RFC 8631 `service-desc` relations in headers and HTML, then ten conventional paths. YAML descriptions are recognised and reported as surface-validated only, since ax-audit ships no YAML parser. Renamed `openapi` → `api-discovery`.
+- **`agent-access` was probing with `Google-Extended`.** That string is a robots.txt control token governing how an already-crawled page may be used; no request carries it, so the probe tested nothing. Token-only controls are excluded, leaving 10 real crawlers.
+- **`http-headers` and `meta-tags` penalised correct behaviour**, requiring a discovery link to `agent.json` specifically. Either card path now counts.
+- **An SPA catch-all page is absence, not corruption.** Found by running the new checks against a real site: probing `/mcp/server-card` returned HTTP 200 with the application's index shell, and the check reported a malformed server card on a site with no MCP server. `isHtmlDocument` now gates the speculative probes in `agent-card`, `mcp-discovery` and `api-discovery`.
+
+### Fixed — crawler catalogue
+
+Verified against vendor documentation on 2026-09-04. Removed tokens that were never real user agents (`Gemini`, `GeminiBot`, `DeepSeek-AI`), whose products were discontinued (`NeevaBot`, `Operator`, `GoogleAgent-Mariner`), whose vendor operates no crawlers (Cohere), or that are not AI crawlers at all (the Awario social-listening family). Reclassified five user-triggered fetchers that were listed as training crawlers. Added `meta-webindexer`, `Amzn-SearchBot`, `Amzn-User`, `MistralAI-Index`, `MistralAI-Training`, `Google-GeminiNotebook`, `Applebot`, `ExaSearchBot`, `TikTokSpider`, `OAI-AdsBot`, `YandexAdditional`.
+
+The catalogue is now organised by what a client does with a page — training, search, user-fetch, agent — because that determines the cost of blocking it. `CRAWLER_META` carries per token the vendor, whether robots.txt is honored, the published IP list, whether requests are signed with Web Bot Auth, and one sentence on what blocking costs. Findings quote it: "blocking OAI-SearchBot removes you from ChatGPT search answers", not "1 AI crawler blocked".
+
+### Added
+
+- **Response classification.** `checks/waf.ts` distinguishes a deliberate block from a JavaScript challenge from anti-spoofing, because the remedies are completely different. Signatures verified 2026-09-04: `cf-mitigated: challenge`, `x-vercel-mitigated`, AWS WAF's 202 with `x-amzn-waf-action`, Cloudflare pay-per-crawl's `crawler-price`, x402's `payment-signature`, Web Bot Auth's `Accept-Signature`, RSL's `WWW-Authenticate: License`, plus thirteen body markers. Outcomes an unsigned probe cannot settle are scored at 0.75 and labelled inconclusive with the header observed — never as "blocks AI crawlers".
+- **IETF AIPREF `Content-Usage:`** in robots.txt (draft-ietf-aipref-attach-05), with an optional path scope and a Structured Fields dictionary. The loudest finding is a vocabulary mix-up: AIPREF spells the training token `train-ai` with `y`/`n` while Content Signals and RSL spell it `ai-train` with `yes`/`no`, so `Content-Usage: ai-train=no` looks correct and does nothing.
+- **Content Signals `use=immediate|reference|full`**, the fourth field Cloudflare added on 2026-07-01 and now serves from managed robots.txt. Cloudflare-managed blocks are detected and named.
+- **Discovery link relations** in `http-headers`: `describedby` (llms.txt v2), `api-catalog` (RFC 9727), `service-desc` and `service-doc` (RFC 8631), `ai-catalog`, `c2pa-manifest`, `license`, markdown alternates, and the `X-Llms-Txt` header.
+- **Content parity** in `agent-access` now fingerprints title, h1 and JSON-LD block count, catching cloaking that preserves word count.
+- **Current well-known files** reported but never scored: the Web Bot Auth key directory, TDMRep, the OpenAI Apps verification token, `AGENTS.md`.
+- **Fetcher transport options**: `method: 'HEAD'` and `redirect: 'manual'`, with `elapsedMs`, `redirected` and `redirectLocation` on every response.
+- **Check categories** (content / discovery / access / policy / protocols) and `CheckResult.applicable` for N/A reporting, both consumed by reporters in 3.8.
+- **`docs/roadmap.md`**: the research behind this release and the plan through 4.0.
+
+### Changed
+
+- **robots.txt is parsed once.** `checks/robots-parser.ts` replaces three separate parsers that disagreed about what closes a User-agent group. Two parsing bugs fell out: `Disallow:` with an empty value now correctly means "allow all" per RFC 9309 §2.2.2 instead of being ignored, and a bot named in two groups is merged so a later `Disallow: /` is no longer hidden by an earlier `Allow: /`.
+- **`well-known-ai` tells the truth.** Re-verification found three of its five scored files have no consumer: `/.well-known/nlweb.json` **does not exist** in any NLWeb release, document or commit (NLWeb exposes `/ask` and `/mcp`); `genai.txt` has no specification; `/ai-plugin.json` described ChatGPT plugins, shut down 2024-04-09. Omitting a retired format now reads as a pass with the evidence attached. The formula stays frozen for score stability; the check loses its weight and its retired probes in 4.0.
+- **Renamed check ids keep working.** `CheckMeta.aliases` plus `src/check-ids.ts` resolve former ids in `--checks` selection and in baseline diffing, so a rename does not read as one check removed at a full regression and one added at zero.
+
+### Scoring
+
+Unchanged. `robots-txt` deducts against the frozen eight-token core set and the token list 3.6 recognised; a site whose only MCP document is the legacy manifest is validated with the pre-3.7 rules reproduced exactly; `well-known-ai` keeps its five-file formula; every finding added inside a weighted check is informational. Broadened discovery can only raise a score, never lower one. Both freezes are removed in 4.0.
+
+### Tests
+
+553 total, up from 301. New suites: robots parser, structured fields, crawler catalogue, WAF classification, check-id aliasing, well-known registry, SPA-shell guard, fetcher transport. Two classes of test exist specifically to hold the no-regression promise: score-stability tests asserting a 3.6 perfect configuration still scores 100, and informational-finding tests asserting new findings leave scores untouched.
+
 ## [3.6.0] - 2026-06-06
 
 ### Added

package/LICENSE

@@ -175,7 +175,7 @@
 
    END OF TERMS AND CONDITIONS
 
-   Copyright 2026 Lucio Duran
+   Copyright 2026 Durani Technologies
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

package/README.md

@@ -4,9 +4,9 @@
 
 <h1 align="center">ax-audit</h1>
 
-[![CI](https://github.com/lucioduran/ax-audit/actions/workflows/ci.yml/badge.svg)](https://github.com/lucioduran/ax-audit/actions/workflows/ci.yml)
+[![CI](https://github.com/duranitech/ax-audit/actions/workflows/ci.yml/badge.svg)](https://github.com/duranitech/ax-audit/actions/workflows/ci.yml)
 [![npm version](https://img.shields.io/npm/v/ax-audit.svg)](https://www.npmjs.com/package/ax-audit)
-[![license](https://img.shields.io/npm/l/ax-audit.svg)](https://github.com/lucioduran/ax-audit/blob/main/LICENSE)
+[![license](https://img.shields.io/npm/l/ax-audit.svg)](https://github.com/duranitech/ax-audit/blob/main/LICENSE)
 [![node](https://img.shields.io/node/v/ax-audit.svg)](https://nodejs.org)
 
 **Lighthouse for AI Agents.** Audit any website's AI Agent Experience (AX) readiness in seconds.
@@ -17,47 +17,62 @@ npx ax-audit https://your-site.com
 
 ```
   AX Audit Report
-  https://lucioduran.com
+  https://example.com
 
-  ███████████████████████████████████░░░░░  88/100  Good
+  █████████████████████████████████░░░░░░░  83/100  Good
 
-  LLMs.txt (100/100)
-    PASS  /llms.txt exists
-    PASS  /llms.txt Content-Type OK (text/plain)
-    PASS  H1 heading: "Lucio Duran — Personal Portfolio"
+  ── Content — is there substance an agent can read?
 
-  Robots.txt (100/100)
-    PASS  All 8 core AI crawlers explicitly configured
-    PASS  Content signals declared for User-agent: * — search=yes, ai-train=no
+  Structured Data (90/100)
+  PASS  3 JSON-LD block(s) found
+  WARN  1 structured-data value(s) do not appear in the visible text
+         💡 Google's one explicit requirement for structured data and AI
+            features is that it match what a reader sees.
 
-  Content Negotiation (100/100)
-    PASS  Homepage serves Markdown via content negotiation (Accept: text/markdown)
-    PASS  Markdown is ~95% lighter than the HTML representation
+  ── Access — can an agent actually retrieve it?
+
+  Agent Access (100/100)
+  PASS  All 10 core AI crawler user-agents receive the same page as a regular client
+
+  AI Directives (100/100)
+  PASS  Homepage is indexable
+  PASS  No directive restricts how AI assistants may use this page
+
+  ── Protocols — what can an agent call?
+
+  MCP Discovery (n/a)
+  PASS  No MCP server — MCP discovery does not apply to this site
   ...
 ```
 
 ## Why
 
-AI agents and LLMs are increasingly crawling, indexing, and interacting with websites. Just like Lighthouse audits web performance and axe-core audits accessibility, **ax-audit** tells you how ready your site is for the AI agent ecosystem — discovery files, crawler policy, licensing, content negotiation, and the failure modes invisible to operators (like a WAF blocking crawlers your robots.txt allows).
+AI agents crawl, cite and act on websites. Lighthouse audits performance, axe-core audits accessibility, and **ax-audit** tells you how ready your site is for agents — discovery files, crawler policy, usage rights, content negotiation, and the failures that are invisible from the inside:
+
+- Your robots.txt allows GPTBot and your firewall returns 403 to it.
+- You blocked `Google-Extended` expecting to leave AI Overviews. It does not do that.
+- Your robots.txt permits AI training while your RSL licence prohibits it, so which terms apply depends on which file a crawler read.
+- Your content only exists after hydration, so the crawlers that do not run JavaScript see an empty page.
+- A missing page answers `200 OK`, so an agent stores the apology as the answer.
 
 ## What it checks
 
-18 checks — 14 weighted, 4 informational. Full reference: **[docs/checks.md](docs/checks.md)**.
+26 checks across five areas. Full reference: **[docs/checks.md](docs/checks.md)**.
 
-| Check | Weight | Check | Weight |
-|---|---|---|---|
-| LLMs.txt | 11% | Security.txt | 6% |
-| Robots.txt + [Content Signals](https://contentsignals.org) | 11% | Meta Tags (OG / Twitter / AI) | 6% |
-| HTML Rendering | 9% | OpenAPI | 6% |
-| Structured Data (JSON-LD) | 9% | TLS / HTTPS | 5% |
-| HTTP Headers | 9% | Sitemap | 4% |
-| Agent Card ([A2A](https://a2a-protocol.org)) | 7% | AI Well-Known | 3% |
-| MCP | 7% | Content Negotiation (Markdown for Agents) | 0%* |
-| SEO Basics | 7% | [RSL License](https://rslstandard.org) · Agent Access (cloaking) · Crawl Efficiency | 0%* |
+| Area | Weight | Checks |
+|---|---|---|
+| **Content** — is there substance an agent can read? | 33% | HTML Rendering · Agent Operability · Structured Data · SEO Basics · Content Negotiation |
+| **Access** — can an agent actually retrieve it? | 24% | Agent Access · AI Directives · HTTP Hygiene · TLS/HTTPS · Crawl Efficiency |
+| **Discovery** — can an agent find your machine-readable files? | 21% | Robots.txt · LLMs.txt · HTTP Headers · Sitemap · Meta Tags |
+| **Protocols** — what can an agent call? | 13% | API Discovery · Agent Card · MCP Discovery · Agent Skills · Auth Discovery |
+| **Policy** — what usage rights do you declare? | 9% | Usage Policy · Security.txt · RSL License |
+| Draft specifications, reported but never scored | 0% | AI Catalog · WebMCP · Commerce Discovery |
+
+Content leads because the failure that breaks the most agents is a page with nothing in its HTML. Most crawlers do not run JavaScript, so a site whose content appears only after hydration is invisible to them no matter how many discovery files it publishes.
 
-\* Informational in 3.x: reported in full, no effect on the score. Weighted in v4.0.
+Protocol checks are **conditional**: a blog has no API to describe, so those report `n/a` and are excluded from the score rather than counted as failures. Use `--profile` to audit against what a site intends to build.
 
-Every finding links to a step-by-step **[remediation guide](https://lucioduran.com/projects/ax-audit/guides)**.
+Every finding links to a step-by-step **[remediation guide](https://axrush.com/guides)**.
 
 ## Usage
 
@@ -67,6 +82,9 @@ ax-audit https://a.com https://b.com --concurrency 2  # batch, in parallel
 ax-audit https://example.com --output markdown        # also: json, html
 ax-audit https://example.com --checks llms-txt,rsl    # subset of checks
 ax-audit https://example.com --only-failures          # hide passing findings
+ax-audit https://example.com --category access                 # one area only
+ax-audit https://example.com --profile api                     # audit as though it had an API
+ax-audit https://example.com --fail-on-category access:70      # per-area CI gate
 ax-audit https://example.com --baseline .ax-baseline.json --fail-on-regression 5
 ```
 
@@ -97,15 +115,15 @@ Reference:
 
 | Document | Contents |
 |---|---|
-| [docs/checks.md](docs/checks.md) | All 18 checks with **exact scoring** per finding, weights, scoring model |
-| [docs/cli.md](docs/cli.md) | Every flag, output formats, exit codes, baseline workflow |
+| [docs/checks.md](docs/checks.md) | All 26 checks with **exact scoring** per finding, the weight table, and the conditional-check rule |
+| [docs/cli.md](docs/cli.md) | Every flag, profiles, area filters, per-area CI gates, baseline workflow |
 | [docs/api.md](docs/api.md) | `audit`, `batchAudit`, baselines, reporters, types, API-stability policy |
 | [docs/ci.md](docs/ci.md) | GitHub Actions recipes: gates, PR comments, scheduled drift detection |
 | [docs/architecture.md](docs/architecture.md) | Pipeline design, check anatomy, how to add a check, scoring policy |
 | [docs/faq.md](docs/faq.md) | Troubleshooting, false positives, the `agent-access` verified-bots caveat |
-| [Remediation guides](https://lucioduran.com/projects/ax-audit/guides) | Step-by-step fixes for every finding |
+| [Remediation guides](https://axrush.com/guides) | Step-by-step fixes for every finding |
 
-The same documentation is browsable at [lucioduran.com/projects/ax-audit/docs](https://lucioduran.com/projects/ax-audit/docs), rendered from these files. Contributors: see [CONTRIBUTING.md](CONTRIBUTING.md) and [SECURITY.md](SECURITY.md).
+The same documentation is browsable at [axrush.com/docs](https://axrush.com/docs), rendered from these files. Contributors: see [CONTRIBUTING.md](CONTRIBUTING.md) and [SECURITY.md](SECURITY.md).
 
 ## Scoring
 
@@ -116,9 +134,13 @@ The same documentation is browsable at [lucioduran.com/projects/ax-audit/docs](h
 | Fair | 50–69 | `1` |
 | Poor | 0–49 | `1` |
 
+Checks that do not apply to a site report **n/a** and leave the denominator entirely, rather than scoring 0. A blog is not marked down for having no API to describe. Everything counted against a site is something the site could have done — which is what makes a low score worth acting on.
+
+`--fail-on-category access:70` gates CI per area, because an overall score can hide an area that is entirely broken while the other four carry it.
+
 ## Tech
 
-TypeScript strict mode · 2 runtime dependencies (`chalk`, `commander`) · Node 18+ built-in `fetch` · parallel checks via `Promise.allSettled` · per-run request cache with `Vary`-aware keys · transient-failure retries with backoff · 301 tests on `node:test` with zero test dependencies.
+TypeScript strict mode · 2 runtime dependencies (`chalk`, `commander`) · Node 18+ built-in `fetch` · no HTML, XML or YAML parser dependencies · parallel checks via `Promise.allSettled` · per-run request cache with `Vary`-aware keys · transient-failure retries with backoff · 873 tests on `node:test` with zero test dependencies.
 
 ## Contributing
 
@@ -126,8 +148,8 @@ Contributions are welcome — see **[docs/architecture.md](docs/architecture.md)
 
 ## Related
 
-- **[ax-init](https://github.com/lucioduran/ax-init)** — generate the AX files this tool audits
-- **[ax-cite](https://github.com/lucioduran/ax-cite)** — embed AI-extractable structured data in your pages
+- **[ax-init](https://github.com/duranitech/ax-init)** — generate the AX files this tool audits
+- **[ax-cite](https://github.com/duranitech/ax-cite)** — embed AI-extractable structured data in your pages
 
 ## License
 
@@ -135,4 +157,4 @@ Contributions are welcome — see **[docs/architecture.md](docs/architecture.md)
 
 ---
 
-Built by [Lucio Duran](https://lucioduran.com)
+**[AX Rush](https://axrush.com)** — the agent-experience toolkit, by [Durani Technologies](https://github.com/duranitech).

package/dist/baseline.d.ts

@@ -3,6 +3,8 @@ import type { AuditReport, BaselineData, BaselineDiff } from './types.js';
  * Extract a minimal, stable snapshot from an AuditReport suitable for
  * persistence and future comparison.
  */
+/** Scoring model this build produces. Bumped whenever weights or the scoring rules change. */
+export declare const BASELINE_SCHEMA_VERSION = 2;
 export declare function toBaselineData(report: AuditReport): BaselineData;
 /**
  * Persist a baseline to disk as pretty-printed JSON.

package/dist/baseline.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"baseline.d.ts","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,WAAW,EAAE,YAAY,EAAE,YAAY,EAAa,MAAM,YAAY,CAAC;AAErF;;;GAGG;AACH,wBAAgB,cAAc,CAAC,MAAM,EAAE,WAAW,GAAG,YAAY,CAWhE;AAED;;;GAGG;AACH,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,WAAW,GAAG,IAAI,CAIpE;AAED;;;GAGG;AACH,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,GAAG,YAAY,CAwBvD;AAED;;;GAGG;AACH,wBAAgB,YAAY,CAAC,QAAQ,EAAE,YAAY,EAAE,MAAM,EAAE,WAAW,GAAG,YAAY,CAsCtF"}
+{"version":3,"file":"baseline.d.ts","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EAAE,WAAW,EAAE,YAAY,EAAE,YAAY,EAAa,MAAM,YAAY,CAAC;AAErF;;;GAGG;AACH,8FAA8F;AAC9F,eAAO,MAAM,uBAAuB,IAAI,CAAC;AAEzC,wBAAgB,cAAc,CAAC,MAAM,EAAE,WAAW,GAAG,YAAY,CAiBhE;AAED;;;GAGG;AACH,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,WAAW,GAAG,IAAI,CAIpE;AAED;;;GAGG;AACH,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,GAAG,YAAY,CAwBvD;AAED;;;GAGG;AACH,wBAAgB,YAAY,CAAC,QAAQ,EAAE,YAAY,EAAE,MAAM,EAAE,WAAW,GAAG,YAAY,CAkEtF"}

package/dist/baseline.js

@@ -1,19 +1,28 @@
 import { readFileSync, writeFileSync, mkdirSync } from 'node:fs';
 import { dirname } from 'node:path';
+import { checks as allChecks } from './checks/index.js';
+import { allIdsFor } from './check-ids.js';
 /**
  * Extract a minimal, stable snapshot from an AuditReport suitable for
  * persistence and future comparison.
  */
+/** Scoring model this build produces. Bumped whenever weights or the scoring rules change. */
+export const BASELINE_SCHEMA_VERSION = 2;
 export function toBaselineData(report) {
     const checks = {};
+    const notApplicable = [];
     for (const r of report.results) {
         checks[r.id] = r.score;
+        if (r.applicable === false)
+            notApplicable.push(r.id);
     }
     return {
+        schemaVersion: BASELINE_SCHEMA_VERSION,
         url: report.url,
         timestamp: report.timestamp,
         overallScore: report.overallScore,
         checks,
+        ...(notApplicable.length > 0 ? { notApplicable } : {}),
     };
 }
 /**
@@ -58,19 +67,39 @@ export function loadBaseline(path) {
  * per-check deltas and overall regression/improvement lists.
  */
 export function diffBaseline(baseline, report) {
+    // A baseline saved before a check was renamed still holds the former id, so
+    // look each score up under every id that check answers to. Without this a
+    // rename reads as one check removed (a full regression) and one added at
+    // zero, which would fire `--fail-on-regression` on an unchanged site.
+    const consumedIds = new Set();
+    const previousScoreFor = (id) => {
+        for (const candidate of idsForCheck(id)) {
+            if (Object.hasOwn(baseline.checks, candidate)) {
+                consumedIds.add(candidate);
+                return baseline.checks[candidate];
+            }
+        }
+        return undefined;
+    };
+    const wasNotApplicable = new Set(baseline.notApplicable ?? []);
     const checks = report.results.map((r) => {
-        const previous = baseline.checks[r.id] ?? 0;
+        const previous = previousScoreFor(r.id) ?? 0;
+        const isNotApplicable = r.applicable === false;
+        // A check that is N/A on one side has no comparable score, so its delta
+        // reflects a change in what the site offers rather than in its quality.
+        const applicabilityChanged = isNotApplicable !== wasNotApplicable.has(r.id);
         return {
             id: r.id,
             name: r.name,
             previous,
             current: r.score,
             delta: r.score - previous,
+            ...(applicabilityChanged ? { applicabilityChanged: true } : {}),
         };
     });
     // Include checks that existed in the baseline but were removed from the current run
     for (const [id, score] of Object.entries(baseline.checks)) {
-        if (!checks.some((c) => c.id === id)) {
+        if (!consumedIds.has(id) && !checks.some((c) => c.id === id)) {
             checks.push({
                 id,
                 name: id, // no human-readable name available for removed checks
@@ -81,7 +110,9 @@ export function diffBaseline(baseline, report) {
         }
     }
     const overallDelta = report.overallScore - baseline.overallScore;
+    const scoringModelChanged = (baseline.schemaVersion ?? 1) !== BASELINE_SCHEMA_VERSION;
     return {
+        ...(scoringModelChanged ? { scoringModelChanged: true } : {}),
         url: report.url,
         baselineTimestamp: baseline.timestamp,
         currentTimestamp: report.timestamp,
@@ -89,11 +120,18 @@ export function diffBaseline(baseline, report) {
         overallCurrent: report.overallScore,
         overallDelta,
         checks,
-        regressions: checks.filter((c) => c.delta < 0),
-        improvements: checks.filter((c) => c.delta > 0),
+        // A regression must be something the site did. A check that changed
+        // applicability, or a baseline from a different scoring model, is neither.
+        regressions: scoringModelChanged ? [] : checks.filter((c) => c.delta < 0 && !c.applicabilityChanged),
+        improvements: checks.filter((c) => c.delta > 0 && !c.applicabilityChanged),
     };
 }
 /* ── Internal helpers ─────────────────────────────────────── */
+/** Current id plus every former id a check answers to, for baseline lookup. */
+function idsForCheck(id) {
+    const meta = allChecks.find((c) => c.meta.id === id)?.meta;
+    return meta === undefined ? [id] : allIdsFor(meta);
+}
 function isBaselineData(value) {
     if (typeof value !== 'object' || value === null)
         return false;

package/dist/baseline.js.map

@@ -1 +1 @@
-{"version":3,"file":"baseline.js","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,SAAS,EAAE,MAAM,SAAS,CAAC;AACjE,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAGpC;;;GAGG;AACH,MAAM,UAAU,cAAc,CAAC,MAAmB;IAChD,MAAM,MAAM,GAA2B,EAAE,CAAC;IAC1C,KAAK,MAAM,CAAC,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;QAC/B,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC;IACzB,CAAC;IACD,OAAO;QACL,GAAG,EAAE,MAAM,CAAC,GAAG;QACf,SAAS,EAAE,MAAM,CAAC,SAAS;QAC3B,YAAY,EAAE,MAAM,CAAC,YAAY;QACjC,MAAM;KACP,CAAC;AACJ,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,IAAY,EAAE,MAAmB;IAC5D,MAAM,IAAI,GAAG,cAAc,CAAC,MAAM,CAAC,CAAC;IACpC,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IAC9C,aAAa,CAAC,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,GAAG,IAAI,EAAE,OAAO,CAAC,CAAC;AACrE,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,IAAY;IACvC,IAAI,GAAW,CAAC;IAChB,IAAI,CAAC;QACH,GAAG,GAAG,YAAY,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;IACpC,CAAC;IAAC,OAAO,GAAY,EAAE,CAAC;QACtB,MAAM,IAAI,GAAI,GAA6B,CAAC,IAAI,CAAC;QACjD,IAAI,IAAI,KAAK,QAAQ,EAAE,CAAC;YACtB,MAAM,IAAI,KAAK,CAAC,4BAA4B,IAAI,EAAE,EAAE,EAAE,KAAK,EAAE,GAAG,EAAE,CAAC,CAAC;QACtE,CAAC;QACD,MAAM,GAAG,CAAC;IACZ,CAAC;IAED,IAAI,IAAa,CAAC;IAClB,IAAI,CAAC;QACH,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IACzB,CAAC;IAAC,OAAO,KAAc,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CAAC,oCAAoC,IAAI,EAAE,EAAE,EAAE,KAAK,EAAE,CAAC,CAAC;IACzE,CAAC;IAED,IAAI,CAAC,cAAc,CAAC,IAAI,CAAC,EAAE,CAAC;QAC1B,MAAM,IAAI,KAAK,CAAC,wFAAwF,IAAI,EAAE,CAAC,CAAC;IAClH,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,QAAsB,EAAE,MAAmB;IACtE,MAAM,MAAM,GAAgB,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;QACnD,MAAM,QAAQ,GAAG,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC;QAC5C,OAAO;YACL,EAAE,EAAE,CAAC,CAAC,EAAE;YACR,IAAI,EAAE,CAAC,CAAC,IAAI;YACZ,QAAQ;YACR,OAAO,EAAE,CAAC,CAAC,KAAK;YAChB,KAAK,EAAE,CAAC,CAAC,KAAK,GAAG,QAAQ;SAC1B,CAAC;IACJ,CAAC,CAAC,CAAC;IAEH,oFAAoF;IACpF,KAAK,MAAM,CAAC,EAAE,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC;QAC1D,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,CAAC;YACrC,MAAM,CAAC,IAAI,CAAC;gBACV,EAAE;gBACF,IAAI,EAAE,EAAE,EAAE,sDAAsD;gBAChE,QAAQ,EAAE,KAAK;gBACf,OAAO,EAAE,CAAC;gBACV,KAAK,EAAE,CAAC,KAAK;aACd,CAAC,CAAC;QACL,CAAC;IACH,CAAC;IAED,MAAM,YAAY,GAAG,MAAM,CAAC,YAAY,GAAG,QAAQ,CAAC,YAAY,CAAC;IAEjE,OAAO;QACL,GAAG,EAAE,MAAM,CAAC,GAAG;QACf,iBAAiB,EAAE,QAAQ,CAAC,SAAS;QACrC,gBAAgB,EAAE,MAAM,CAAC,SAAS;QAClC,eAAe,EAAE,QAAQ,CAAC,YAAY;QACtC,cAAc,EAAE,MAAM,CAAC,YAAY;QACnC,YAAY;QACZ,MAAM;QACN,WAAW,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC;QAC9C,YAAY,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC;KAChD,CAAC;AACJ,CAAC;AAED,iEAAiE;AAEjE,SAAS,cAAc,CAAC,KAAc;IACpC,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI;QAAE,OAAO,KAAK,CAAC;IAC9D,MAAM,GAAG,GAAG,KAAgC,CAAC;IAC7C,OAAO,CACL,OAAO,GAAG,CAAC,GAAG,KAAK,QAAQ;QAC3B,OAAO,GAAG,CAAC,SAAS,KAAK,QAAQ;QACjC,OAAO,GAAG,CAAC,YAAY,KAAK,QAAQ;QACpC,OAAO,GAAG,CAAC,MAAM,KAAK,QAAQ;QAC9B,GAAG,CAAC,MAAM,KAAK,IAAI;QACnB,CAAC,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,CAC3B,CAAC;AACJ,CAAC"}
+{"version":3,"file":"baseline.js","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,SAAS,EAAE,MAAM,SAAS,CAAC;AACjE,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,MAAM,IAAI,SAAS,EAAE,MAAM,mBAAmB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAG3C;;;GAGG;AACH,8FAA8F;AAC9F,MAAM,CAAC,MAAM,uBAAuB,GAAG,CAAC,CAAC;AAEzC,MAAM,UAAU,cAAc,CAAC,MAAmB;IAChD,MAAM,MAAM,GAA2B,EAAE,CAAC;IAC1C,MAAM,aAAa,GAAa,EAAE,CAAC;IAEnC,KAAK,MAAM,CAAC,IAAI,MAAM,CAAC,OAAO,EAAE,CAAC;QAC/B,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC;QACvB,IAAI,CAAC,CAAC,UAAU,KAAK,KAAK;YAAE,aAAa,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;IACvD,CAAC;IAED,OAAO;QACL,aAAa,EAAE,uBAAuB;QACtC,GAAG,EAAE,MAAM,CAAC,GAAG;QACf,SAAS,EAAE,MAAM,CAAC,SAAS;QAC3B,YAAY,EAAE,MAAM,CAAC,YAAY;QACjC,MAAM;QACN,GAAG,CAAC,aAAa,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KACvD,CAAC;AACJ,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,IAAY,EAAE,MAAmB;IAC5D,MAAM,IAAI,GAAG,cAAc,CAAC,MAAM,CAAC,CAAC;IACpC,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IAC9C,aAAa,CAAC,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,GAAG,IAAI,EAAE,OAAO,CAAC,CAAC;AACrE,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,IAAY;IACvC,IAAI,GAAW,CAAC;IAChB,IAAI,CAAC;QACH,GAAG,GAAG,YAAY,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;IACpC,CAAC;IAAC,OAAO,GAAY,EAAE,CAAC;QACtB,MAAM,IAAI,GAAI,GAA6B,CAAC,IAAI,CAAC;QACjD,IAAI,IAAI,KAAK,QAAQ,EAAE,CAAC;YACtB,MAAM,IAAI,KAAK,CAAC,4BAA4B,IAAI,EAAE,EAAE,EAAE,KAAK,EAAE,GAAG,EAAE,CAAC,CAAC;QACtE,CAAC;QACD,MAAM,GAAG,CAAC;IACZ,CAAC;IAED,IAAI,IAAa,CAAC;IAClB,IAAI,CAAC;QACH,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IACzB,CAAC;IAAC,OAAO,KAAc,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CAAC,oCAAoC,IAAI,EAAE,EAAE,EAAE,KAAK,EAAE,CAAC,CAAC;IACzE,CAAC;IAED,IAAI,CAAC,cAAc,CAAC,IAAI,CAAC,EAAE,CAAC;QAC1B,MAAM,IAAI,KAAK,CAAC,wFAAwF,IAAI,EAAE,CAAC,CAAC;IAClH,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,YAAY,CAAC,QAAsB,EAAE,MAAmB;IACtE,4EAA4E;IAC5E,0EAA0E;IAC1E,yEAAyE;IACzE,sEAAsE;IACtE,MAAM,WAAW,GAAG,IAAI,GAAG,EAAU,CAAC;IAEtC,MAAM,gBAAgB,GAAG,CAAC,EAAU,EAAsB,EAAE;QAC1D,KAAK,MAAM,SAAS,IAAI,WAAW,CAAC,EAAE,CAAC,EAAE,CAAC;YACxC,IAAI,MAAM,CAAC,MAAM,CAAC,QAAQ,CAAC,MAAM,EAAE,SAAS,CAAC,EAAE,CAAC;gBAC9C,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC;gBAC3B,OAAO,QAAQ,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC;YACpC,CAAC;QACH,CAAC;QACD,OAAO,SAAS,CAAC;IACnB,CAAC,CAAC;IAEF,MAAM,gBAAgB,GAAG,IAAI,GAAG,CAAC,QAAQ,CAAC,aAAa,IAAI,EAAE,CAAC,CAAC;IAE/D,MAAM,MAAM,GAAgB,MAAM,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;QACnD,MAAM,QAAQ,GAAG,gBAAgB,CAAC,CAAC,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC;QAC7C,MAAM,eAAe,GAAG,CAAC,CAAC,UAAU,KAAK,KAAK,CAAC;QAC/C,wEAAwE;QACxE,wEAAwE;QACxE,MAAM,oBAAoB,GAAG,eAAe,KAAK,gBAAgB,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QAE5E,OAAO;YACL,EAAE,EAAE,CAAC,CAAC,EAAE;YACR,IAAI,EAAE,CAAC,CAAC,IAAI;YACZ,QAAQ;YACR,OAAO,EAAE,CAAC,CAAC,KAAK;YAChB,KAAK,EAAE,CAAC,CAAC,KAAK,GAAG,QAAQ;YACzB,GAAG,CAAC,oBAAoB,CAAC,CAAC,CAAC,EAAE,oBAAoB,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;SAChE,CAAC;IACJ,CAAC,CAAC,CAAC;IAEH,oFAAoF;IACpF,KAAK,MAAM,CAAC,EAAE,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC;QAC1D,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,CAAC;YAC7D,MAAM,CAAC,IAAI,CAAC;gBACV,EAAE;gBACF,IAAI,EAAE,EAAE,EAAE,sDAAsD;gBAChE,QAAQ,EAAE,KAAK;gBACf,OAAO,EAAE,CAAC;gBACV,KAAK,EAAE,CAAC,KAAK;aACd,CAAC,CAAC;QACL,CAAC;IACH,CAAC;IAED,MAAM,YAAY,GAAG,MAAM,CAAC,YAAY,GAAG,QAAQ,CAAC,YAAY,CAAC;IACjE,MAAM,mBAAmB,GAAG,CAAC,QAAQ,CAAC,aAAa,IAAI,CAAC,CAAC,KAAK,uBAAuB,CAAC;IAEtF,OAAO;QACL,GAAG,CAAC,mBAAmB,CAAC,CAAC,CAAC,EAAE,mBAAmB,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAC7D,GAAG,EAAE,MAAM,CAAC,GAAG;QACf,iBAAiB,EAAE,QAAQ,CAAC,SAAS;QACrC,gBAAgB,EAAE,MAAM,CAAC,SAAS;QAClC,eAAe,EAAE,QAAQ,CAAC,YAAY;QACtC,cAAc,EAAE,MAAM,CAAC,YAAY;QACnC,YAAY;QACZ,MAAM;QACN,oEAAoE;QACpE,2EAA2E;QAC3E,WAAW,EAAE,mBAAmB,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,oBAAoB,CAAC;QACpG,YAAY,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,oBAAoB,CAAC;KAC3E,CAAC;AACJ,CAAC;AAED,iEAAiE;AAEjE,+EAA+E;AAC/E,SAAS,WAAW,CAAC,EAAU;IAC7B,MAAM,IAAI,GAAG,SAAS,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,IAAI,CAAC;IAC3D,OAAO,IAAI,KAAK,SAAS,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,IAAI,CAAC,CAAC;AACrD,CAAC;AAED,SAAS,cAAc,CAAC,KAAc;IACpC,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI;QAAE,OAAO,KAAK,CAAC;IAC9D,MAAM,GAAG,GAAG,KAAgC,CAAC;IAC7C,OAAO,CACL,OAAO,GAAG,CAAC,GAAG,KAAK,QAAQ;QAC3B,OAAO,GAAG,CAAC,SAAS,KAAK,QAAQ;QACjC,OAAO,GAAG,CAAC,YAAY,KAAK,QAAQ;QACpC,OAAO,GAAG,CAAC,MAAM,KAAK,QAAQ;QAC9B,GAAG,CAAC,MAAM,KAAK,IAAI;QACnB,CAAC,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,CAC3B,CAAC;AACJ,CAAC"}

package/dist/check-ids.d.ts

@@ -0,0 +1,19 @@
+import type { CheckMeta } from './types.js';
+/** Map of alias → current id, built from the registered checks. */
+export declare function buildAliasMap(metas?: CheckMeta[]): Map<string, string>;
+/**
+ * Resolve a possibly-former check id to its current one. Unknown ids are
+ * returned unchanged so callers can report them as invalid with the name the
+ * user actually typed.
+ */
+export declare function resolveCheckId(id: string, metas?: CheckMeta[]): string;
+/** Every id a check answers to, current name first. */
+export declare function allIdsFor(meta: CheckMeta): string[];
+/** Every selectable id across all checks, aliases included. */
+export declare function allSelectableIds(metas?: CheckMeta[]): string[];
+/**
+ * Whether a check should run given a `--checks` selection, matching on its
+ * current id or any alias.
+ */
+export declare function isSelected(meta: CheckMeta, selection: string[]): boolean;
+//# sourceMappingURL=check-ids.d.ts.map

package/dist/check-ids.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"check-ids.d.ts","sourceRoot":"","sources":["../src/check-ids.ts"],"names":[],"mappings":"AAcA,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,YAAY,CAAC;AAE5C,mEAAmE;AACnE,wBAAgB,aAAa,CAAC,KAAK,GAAE,SAAS,EAAiC,GAAG,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,CAQpG;AAED;;;;GAIG;AACH,wBAAgB,cAAc,CAAC,EAAE,EAAE,MAAM,EAAE,KAAK,CAAC,EAAE,SAAS,EAAE,GAAG,MAAM,CAKtE;AAED,uDAAuD;AACvD,wBAAgB,SAAS,CAAC,IAAI,EAAE,SAAS,GAAG,MAAM,EAAE,CAEnD;AAED,+DAA+D;AAC/D,wBAAgB,gBAAgB,CAAC,KAAK,GAAE,SAAS,EAAiC,GAAG,MAAM,EAAE,CAE5F;AAED;;;GAGG;AACH,wBAAgB,UAAU,CAAC,IAAI,EAAE,SAAS,EAAE,SAAS,EAAE,MAAM,EAAE,GAAG,OAAO,CAGxE"}

package/dist/check-ids.js

@@ -0,0 +1,53 @@
+/**
+ * Check-id resolution across renames.
+ *
+ * Check ids are a public interface twice over: they appear in `--checks`
+ * invocations that live in CI configs, and as keys in saved baseline files. So
+ * a rename cannot be a silent break — `--checks mcp` must keep working, and a
+ * baseline recorded when the check was called `mcp` must still compare against
+ * the check now called `mcp-discovery` rather than reporting one check removed
+ * and another added.
+ *
+ * A check declares its former names in `meta.aliases`; everything here reads
+ * from that, so adding an alias is a one-line change on the check itself.
+ */
+import { checks as allChecks } from './checks/index.js';
+/** Map of alias → current id, built from the registered checks. */
+export function buildAliasMap(metas = allChecks.map((c) => c.meta)) {
+    const map = new Map();
+    for (const meta of metas) {
+        for (const alias of meta.aliases ?? []) {
+            map.set(alias.toLowerCase(), meta.id);
+        }
+    }
+    return map;
+}
+/**
+ * Resolve a possibly-former check id to its current one. Unknown ids are
+ * returned unchanged so callers can report them as invalid with the name the
+ * user actually typed.
+ */
+export function resolveCheckId(id, metas) {
+    const current = metas ?? allChecks.map((c) => c.meta);
+    const lower = id.toLowerCase();
+    if (current.some((m) => m.id.toLowerCase() === lower))
+        return current.find((m) => m.id.toLowerCase() === lower).id;
+    return buildAliasMap(current).get(lower) ?? id;
+}
+/** Every id a check answers to, current name first. */
+export function allIdsFor(meta) {
+    return [meta.id, ...(meta.aliases ?? [])];
+}
+/** Every selectable id across all checks, aliases included. */
+export function allSelectableIds(metas = allChecks.map((c) => c.meta)) {
+    return metas.flatMap(allIdsFor);
+}
+/**
+ * Whether a check should run given a `--checks` selection, matching on its
+ * current id or any alias.
+ */
+export function isSelected(meta, selection) {
+    const wanted = new Set(selection.map((s) => s.toLowerCase()));
+    return allIdsFor(meta).some((id) => wanted.has(id.toLowerCase()));
+}
+//# sourceMappingURL=check-ids.js.map

package/dist/check-ids.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"check-ids.js","sourceRoot":"","sources":["../src/check-ids.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AACH,OAAO,EAAE,MAAM,IAAI,SAAS,EAAE,MAAM,mBAAmB,CAAC;AAGxD,mEAAmE;AACnE,MAAM,UAAU,aAAa,CAAC,QAAqB,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;IAC7E,MAAM,GAAG,GAAG,IAAI,GAAG,EAAkB,CAAC;IACtC,KAAK,MAAM,IAAI,IAAI,KAAK,EAAE,CAAC;QACzB,KAAK,MAAM,KAAK,IAAI,IAAI,CAAC,OAAO,IAAI,EAAE,EAAE,CAAC;YACvC,GAAG,CAAC,GAAG,CAAC,KAAK,CAAC,WAAW,EAAE,EAAE,IAAI,CAAC,EAAE,CAAC,CAAC;QACxC,CAAC;IACH,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,cAAc,CAAC,EAAU,EAAE,KAAmB;IAC5D,MAAM,OAAO,GAAG,KAAK,IAAI,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;IACtD,MAAM,KAAK,GAAG,EAAE,CAAC,WAAW,EAAE,CAAC;IAC/B,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,WAAW,EAAE,KAAK,KAAK,CAAC;QAAE,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,WAAW,EAAE,KAAK,KAAK,CAAE,CAAC,EAAE,CAAC;IACpH,OAAO,aAAa,CAAC,OAAO,CAAC,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC;AACjD,CAAC;AAED,uDAAuD;AACvD,MAAM,UAAU,SAAS,CAAC,IAAe;IACvC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,GAAG,CAAC,IAAI,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC,CAAC;AAC5C,CAAC;AAED,+DAA+D;AAC/D,MAAM,UAAU,gBAAgB,CAAC,QAAqB,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;IAChF,OAAO,KAAK,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;AAClC,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,UAAU,CAAC,IAAe,EAAE,SAAmB;IAC7D,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC;IAC9D,OAAO,SAAS,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC;AACpE,CAAC"}

package/dist/checks/agent-access.d.ts

@@ -1,5 +1,26 @@
 import type { CheckContext, CheckResult, CheckMeta } from '../types.js';
-import { type BotEntry } from './robots-txt.js';
+/**
+ * "agent-access" — the gap between what robots.txt permits and what the server
+ * actually does.
+ *
+ * The failure this catches is invisible from the inside: robots.txt says
+ * `Allow: /` for GPTBot, and the WAF returns 403 to anything whose user agent
+ * says GPTBot. The operator sees a permissive robots.txt and assumes the site
+ * is reachable. Cloudflare's "Block AI Crawlers" toggle produces exactly this,
+ * and so does a hand-written rule that outlived its reason.
+ *
+ * The check probes the homepage once per core crawler and compares the result
+ * against the default-user-agent baseline. Since 3.7 it classifies *how* a
+ * request was turned away, because the remedies are completely different: a
+ * JavaScript challenge needs a bot-management exception, a hard 403 needs a
+ * firewall rule change, a 402 is a deliberate price, and a signature demand is
+ * the site working as designed.
+ *
+ * Honesty constraint: this probe is unsigned and comes from the auditor's own
+ * network. An edge that verifies crawlers by IP range or Web Bot Auth signature
+ * will reject it while admitting the genuine crawler. Those outcomes are scored
+ * as inconclusive and labelled as such, never as "blocks AI crawlers".
+ */
 export declare const meta: CheckMeta;
 /**
  * Build a realistic crawler User-Agent for a given bot token. WAF and
@@ -7,10 +28,6 @@ export declare const meta: CheckMeta;
  * to trigger the same code path the real crawler would hit.
  */
 export declare function crawlerUserAgent(token: string): string;
-/**
- * Whether robots.txt expresses the intent to block this crawler: an explicit
- * full Disallow for it, or a full wildcard Disallow with no explicit entry.
- */
-export declare function intentBlocked(entries: BotEntry[], crawler: string): boolean;
+export { intentBlocked } from './robots-parser.js';
 export default function check(ctx: CheckContext): Promise<CheckResult>;
 //# sourceMappingURL=agent-access.d.ts.map

package/dist/checks/agent-access.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"agent-access.d.ts","sourceRoot":"","sources":["../../src/checks/agent-access.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,YAAY,EAAE,WAAW,EAAE,SAAS,EAAW,MAAM,aAAa,CAAC;AAGjF,OAAO,EAAmB,KAAK,QAAQ,EAAE,MAAM,iBAAiB,CAAC;AAEjE,eAAO,MAAM,IAAI,EAAE,SAKlB,CAAC;AASF;;;;GAIG;AACH,wBAAgB,gBAAgB,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,CAEtD;AAED;;;GAGG;AACH,wBAAgB,aAAa,CAAC,OAAO,EAAE,QAAQ,EAAE,EAAE,OAAO,EAAE,MAAM,GAAG,OAAO,CAK3E;AAED,wBAA8B,KAAK,CAAC,GAAG,EAAE,YAAY,GAAG,OAAO,CAAC,WAAW,CAAC,CAiF3E"}
+{"version":3,"file":"agent-access.d.ts","sourceRoot":"","sources":["../../src/checks/agent-access.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,YAAY,EAAE,WAAW,EAAE,SAAS,EAAW,MAAM,aAAa,CAAC;AAMjF;;;;;;;;;;;;;;;;;;;;;GAqBG;AACH,eAAO,MAAM,IAAI,EAAE,SAKlB,CAAC;AA+BF;;;;GAIG;AACH,wBAAgB,gBAAgB,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,CAEtD;AAGD,OAAO,EAAE,aAAa,EAAE,MAAM,oBAAoB,CAAC;AAoCnD,wBAA8B,KAAK,CAAC,GAAG,EAAE,YAAY,GAAG,OAAO,CAAC,WAAW,CAAC,CA8E3E"}