ax-audit 3.1.0 → 4.0.0

package/docs/cli.md

@@ -0,0 +1,103 @@
+# CLI Reference
+
+```bash
+ax-audit <urls...> [options]
+```
+
+One or more fully qualified URLs (scheme required). A single URL produces a full report; multiple URLs run in batch mode with a summary table.
+
+## Options
+
+| Flag | Default | Description |
+| --- | --- | --- |
+| `--output <format>` | `terminal` | Output format: `terminal`, `json`, `html`, `markdown`. Invalid values error out. |
+| `--json` | — | Shorthand for `--output json`. |
+| `--checks <list>` | all | Comma-separated check IDs to run (see [checks.md](./checks.md)). Unknown IDs error with the list of valid ones. |
+| `--timeout <ms>` | `10000` | Per-request timeout in milliseconds. |
+| `--retries <n>` | `2` | Retry attempts for transient fetch failures (network errors, timeouts, 408/425/429/5xx) with exponential backoff from 250ms. `0` disables retries. |
+| `--concurrency <n>` | `1` | Batch mode only: maximum URLs audited in parallel. Output order always matches input order. |
+| `--verbose` | — | Log every HTTP request, cache hit, retry, and per-check score to stderr. |
+| `--only-failures` | — | Hide passing findings; checks with only passes are omitted entirely. |
+| `--save-baseline <path>` | — | Save this audit as a baseline JSON file. |
+| `--baseline <path>` | — | Compare against a saved baseline; shows per-check deltas (▲/▼). Single-URL mode only. |
+| `--fail-on-regression <points>` | — | Exit 1 if any check regresses more than N points vs the baseline. Requires `--baseline`. |
+| `-v, --version` | — | Print version. |
+
+## Output formats
+
+- **terminal** — colored report with score bar, per-check sections, and PASS/WARN/FAIL findings.
+- **json** — the full `AuditReport` (plus `baselineDiff` when `--baseline` is used). Stable shape for CI pipelines.
+- **html** — self-contained page (score gauge, dark/light mode, collapsible sections). Pipe to a file: `ax-audit <url> --output html > report.html`.
+- **markdown** — summary table + per-check findings with status emoji. Built for CI logs and PR comments: `ax-audit <url> --output markdown > report.md`.
+
+## Exit codes
+
+| Code | Meaning |
+| --- | --- |
+| `0` | Score ≥ 70 (single), or all URLs ≥ 70 (batch), and no regression beyond the `--fail-on-regression` threshold. |
+| `1` | Score < 70, any batch URL < 70, invalid arguments, or regression beyond threshold. |
+| `2` | Fatal error (network failure on the audit itself, unreadable baseline file). |
+
+## Baseline workflow
+
+```bash
+# First run — record the baseline
+ax-audit https://your-site.com --save-baseline .ax-baseline.json
+
+# Subsequent runs — compare and gate
+ax-audit https://your-site.com --baseline .ax-baseline.json --fail-on-regression 5
+```
+
+The baseline stores the overall score and per-check scores. Checks added after the baseline was saved appear as new (no delta); removed checks are ignored.
+
+## Examples
+
+```bash
+# Quick audit
+npx ax-audit https://your-site.com
+
+# Only the AI-licensing surface
+npx ax-audit https://your-site.com --checks robots-txt,rsl,content-negotiation
+
+# Batch, 4 at a time, machine-readable
+npx ax-audit $(cat urls.txt) --concurrency 4 --json > batch.json
+
+# Show me only what is broken
+npx ax-audit https://your-site.com --only-failures
+```
+
+## Agent-surface profiles
+
+Protocol checks (`api-discovery`, `agent-card`, `mcp-discovery`, `agent-skills`, `auth-discovery`, `commerce-discovery`) apply only where the site has the corresponding surface. A blog has no API to describe, so scoring it zero would say something false; those checks report **n/a** and leave the denominator.
+
+`--profile` overrides that, for auditing against what a site intends to build rather than what it already has.
+
+```bash
+ax-audit https://example.com --profile api        # audit as though it offered an API
+ax-audit https://example.com --profile all        # every protocol check applies
+```
+
+Values: `auto` (default, detect from the site), `api`, `mcp`, `agent`, `docs`, `commerce`, `all`.
+
+## Focusing on one area
+
+```bash
+ax-audit https://example.com --category access              # only access checks
+ax-audit https://example.com --category content,discovery   # two areas
+```
+
+`--category` narrows an explicit `--checks` selection rather than replacing it, and errors if the intersection is empty.
+
+## Per-area CI gates
+
+An overall score hides an area that is entirely broken: a site can score 80 while every access check fails, because the other four carry it.
+
+```bash
+ax-audit https://example.com --fail-on-category access:70,content:80
+```
+
+Each area is reported against its threshold on stderr, so the output survives `--output json` being piped elsewhere. An area with no applicable checks is reported as not evaluated rather than failed: a build must not break because a site has no commerce surface.
+
+## Baselines across a scoring change
+
+Baselines record which scoring model produced them. Comparing a baseline written by an older version shows the deltas but suspends regression gating, because a rescore is not something the site did. Re-save with `--save-baseline` to resume gating. Checks that changed applicability are excluded from regressions and improvements for the same reason.

package/docs/concepts.md

@@ -0,0 +1,101 @@
+# Concepts: the AX standards landscape
+
+"AI Agent Experience" (AX) is the sum of the conventions a site uses to be discovered, read, governed, and transacted with by autonomous AI agents and crawlers — the way "web accessibility" is the sum of conventions for assistive technology. This page maps the standards ax-audit checks against, why each exists, and how they relate. It's the conceptual companion to the mechanical detail in [checks.md](./checks.md).
+
+## Why AX is its own discipline
+
+Agents are not browsers. Three differences drive every check:
+
+1. **They mostly don't run JavaScript.** GPTBot, ClaudeBot, CCBot and most crawlers fetch raw HTML. A client-rendered SPA that returns an empty `<div id="root">` is, to them, a blank page. (`html-rendering`, `content-negotiation`)
+2. **They look for declared structure, not visual layout.** An agent would rather read a `/llms.txt` summary or a JSON-LD graph than infer meaning from your CSS grid. (`llms-txt`, `structured-data`, `meta-tags`, `agent-card`, `mcp-discovery`, `api-discovery`)
+3. **Their access is a policy and economic question, not just a technical one.** Who may crawl, for what use, at what price, under what license — these now have machine-readable answers. (`robots-txt`, Content Signals, `rsl`, `agent-access`)
+
+Bot traffic is projected to exceed human traffic by 2029. AX is the interface layer for that shift.
+
+## The four families of standards
+
+### 1. Content discovery & readability
+
+| Standard | What it is | Check |
+| --- | --- | --- |
+| **[llms.txt](https://llmstxt.org)** | A Markdown file at your root summarizing your site for LLMs, with curated links. The "sitemap for AI." | `llms-txt` |
+| **Server-side rendering** | Delivering real content in the HTML response, not assembling it client-side. | `html-rendering` |
+| **[Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/)** | Content negotiation: serve clean Markdown when a client sends `Accept: text/markdown`. ~80% fewer tokens than HTML. | `content-negotiation` |
+| **schema.org / JSON-LD** | Structured data describing entities (Person, Organization, Product) in a graph agents can parse. | `structured-data` |
+| **Sitemaps** | The classic XML index, still how crawlers enumerate your URLs. | `sitemap` |
+
+These answer: *can an agent find your content and actually read it?*
+
+### 2. Agent interaction surface
+
+| Standard | What it is | Check |
+| --- | --- | --- |
+| **[A2A — Agent2Agent](https://a2a-protocol.org)** | An Agent Card at `/.well-known/agent-card.json` advertising your identity and skills, so other agents can interoperate. The path moved there in v0.3.0 and is IANA-registered; v1.0 changed the card's shape. | `agent-card` |
+| **[MCP — Model Context Protocol](https://modelcontextprotocol.io)** | A **server card** identifying your MCP server and its remote endpoints. Note it carries no tool list: tools come from a live `tools/list` call, and a static copy drifts. `/.well-known/mcp.json` was never part of the specification. | `mcp-discovery` |
+| **[OpenAPI](https://www.openapis.org)** and **[RFC 9727](https://www.rfc-editor.org/rfc/rfc9727.html)** | The machine-readable API description, plus the registered `/.well-known/api-catalog` that points at it. Agents use both to call your endpoints without a human reading your docs. | `api-discovery` |
+| **[Agent Skills](https://agentskills.io)** | SKILL.md documents an agent installs and follows — setup steps, argument shapes, mistakes to avoid. Answers "how do I do the thing this site is for", where llms.txt answers "what is here". | `agent-skills` |
+| **[RFC 9728 / RFC 8414](https://www.rfc-editor.org/rfc/rfc9728.html)** | OAuth metadata. A human hitting a 401 reads your docs; an agent cannot, so the answer has to be in the response. | `auth-discovery` |
+| **[UCP](https://developers.google.com/merchant/ucp/guides/ucp-profile)** | The Universal Commerce Protocol profile at `/.well-known/ucp` — the one agentic-commerce specification with published site-side discovery. | `commerce-discovery` |
+| **[WebMCP](https://webmachinelearning.github.io/webmcp/)** | Declaring a form as a callable tool, so an agent invokes it rather than driving it pixel by pixel. A Community Group draft in a Chrome origin trial. | `webmcp` |
+| **AI catalogs** | `ai-catalog.json` and `ard.json`, two competing drafts for one index listing everything above, so a client stops probing four conventions. | `ai-catalog` |
+| **AI meta tags & discovery links** | `ai:*` meta tags and the `rel` relations — `alternate`, `describedby`, `service-desc`, `api-catalog` — that point agents at your files instead of making them guess paths. | `meta-tags`, `http-headers` |
+
+These answer: *once an agent arrives, can it understand what you offer and act on it?*
+
+### 3. Access governance & licensing
+
+This is the newest and fastest-moving family — the response to "AI scraped my content and now competes with me."
+
+| Standard | What it is | Check |
+| --- | --- | --- |
+| **[Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309.html)** | The original robots.txt — *who* may crawl *what*. ax-audit knows 57 AI clients, grouped by what they do with a page, because blocking a search crawler costs citations while blocking a training crawler is a policy choice. | `robots-txt` |
+| **[Content Signals](https://contentsignals.org)** | A robots.txt extension (Cloudflare, CC0) declaring *how* content may be used after access: `search`, `ai-input`, `ai-train`, plus the `use` field added in 2026. Served by default on millions of Cloudflare domains. | `robots-txt`, `usage-policy` |
+| **[IETF AIPREF](https://datatracker.ietf.org/wg/aipref/documents/)** | The standards-track answer to the same question: a `Content-Usage` directive or header over `train-ai` and `search`. Still pre-last-call. Note the token inversion against Content Signals — `train-ai` here, `ai-train` there. | `robots-txt`, `usage-policy` |
+| **[RSL — Really Simple Licensing](https://rslstandard.org)** | A full machine-readable licensing layer: permits/prohibits vocabularies, payment models, and an Open Licensing Protocol for negotiating access. | `rsl`, `usage-policy` |
+| **[TDMRep](https://www.w3.org/community/reports/tdmrep/CG-FINAL-tdmrep-20240510/)** | A text-and-data-mining reservation, named in the EU GPAI Code of Practice. Its weight is legal rather than technical. | `usage-policy` |
+| **Page-level AI directives** | `nosnippet` and `max-snippet` for Google AI Overviews, `noarchive` and `nocache` for Copilot grounding. Unlike everything above, the vendors document that they honor these. | `ai-directives` |
+| **Cloaking and blocking integrity** | Not a standard but a failure mode: robots.txt allows GPTBot and the firewall returns 403. | `agent-access` |
+| **Signal consistency** | Also not a standard: whether your Content Signals, AIPREF, RSL and TDMRep declarations agree with each other. Five documents maintained by hand do not stay in sync, and the terms that apply then depend on which file a crawler read. | `usage-policy` |
+
+These answer: *have you expressed your access and usage policy in a form agents can honor — and does your infrastructure actually match it?*
+
+The progression is one of increasing expressiveness: robots.txt says **who and where**, Content Signals and AIPREF add **how it may be used**, RSL adds **under what licence and price**.
+
+One caveat worth stating plainly: only robots.txt access rules are documented as honored by Google, OpenAI, Anthropic and Microsoft. Content Signals, AIPREF, RSL and TDMRep are declarations. Their weight is legal rather than technical, and ax-audit says so on every run rather than implying a crawler will obey.
+
+### 4. Transport, efficiency & hygiene
+
+| Standard | What it is | Check |
+| --- | --- | --- |
+| **TLS / HSTS** | HTTPS everywhere; many agents refuse plaintext origins. | `tls-https` |
+| **HTTP security & discovery headers** | Security headers plus `Link` headers advertising your AI files. | `http-headers` |
+| **Compression & conditional GET** | Brotli/gzip and `ETag`/`304` — crawl cost matters when bots dominate traffic. | `crawl-efficiency` |
+| **[RFC 9116 security.txt](https://www.rfc-editor.org/rfc/rfc9116)** | A machine-readable security contact. | `security-txt` |
+| **SEO basics** | Title, description, canonical, lang, hreflang — agents use the same head-tag fundamentals search engines do. | `seo-basics` |
+
+These answer: *is the connection trustworthy, cheap, and well-formed?*
+
+## On the horizon (not yet scored)
+
+Two standards are maturing and worth watching:
+
+- **[Web Bot Auth](https://datatracker.ietf.org/doc/draft-meunier-web-bot-auth-architecture/)** — cryptographic crawler verification via HTTP Message Signatures (RFC 9421). Bots sign requests with a key published at `/.well-known/http-message-signatures-directory`; sites verify identity instead of guessing from user-agent strings. Already implemented by Cloudflare and Google (`agent.bot.goog`). It directly affects the `agent-access` check: a WAF using Web Bot Auth may pass a real, signed crawler while rejecting ax-audit's unsigned probe — which is why that check's findings carry an explicit verified-bots caveat.
+- **Pay-per-crawl / HTTP 402** — Cloudflare and the RSL payment vocabulary point toward metered, paid agent access. RSL already encodes the terms; enforcement protocols (Open License Protocol, x402) are emerging.
+
+## How the families compose
+
+A fully AX-ready site tells a coherent story across all four:
+
+> "Here's my content in a form you can read **(family 1)**, here's the interface to interact with me **(family 2)**, here's exactly who may use it and how, for what license **(family 3)**, over a fast and trustworthy connection **(family 4)**."
+
+ax-audit's weighting follows what actually stops an agent. **Content** carries the most weight, led by `html-rendering`, because a page whose content only exists after hydration is invisible to the crawlers that do not run JavaScript, no matter how many discovery files it publishes. **Access** is next, because a firewall rule or a `nosnippet` directive silently undoes everything else.
+
+`llms-txt` sits at 5 rather than the 11 it carried in 3.x. Adoption studies find most published files are never fetched by an AI search crawler, and Google has stated that Search ignores them; the vendors that do read them are coding agents like Claude Code and Cursor. It is worth publishing, as developer tooling, and it is not worth twice what having content is worth.
+
+Checks resting on draft specifications — `ai-catalog`, `webmcp`, `commerce-discovery` — are reported but never scored. A specification that may be renamed next quarter should not move your number.
+
+## See also
+
+- [getting-started.md](./getting-started.md) — run your first audit
+- [checks.md](./checks.md) — exact scoring per standard
+- The [remediation guides](https://axrush.com/guides) — how to implement each one

package/docs/faq.md

@@ -0,0 +1,89 @@
+# FAQ & Troubleshooting
+
+## Scores & results
+
+### Why did my score change after upgrading ax-audit?
+
+In 3.x, score changes on the same site are treated as **breaking** and only happen in major or minor releases that explicitly say so. The 3.0.0 release redistributed weights across 14 checks and added Content-Type penalties — see its CHANGELOG entry. Every check added since (3.1.0–3.6.0) ships at **weight 0** precisely so your score and baselines don't move. To track changes deliberately, use `--baseline` (see [cli.md](./cli.md)).
+
+### Why is my score lower than my Lighthouse / SEO score?
+
+ax-audit measures the *AI-agent* surface, not performance, accessibility, or human SEO. A fast, beautiful site can still score poorly if it has no `llms.txt`, ships an empty SPA shell to non-JS crawlers, and exposes no structured data. That gap is the reason the tool exists.
+
+### A check shows 0 but the file exists — why?
+
+Most "not found" hard-fails mean the request didn't return a 2xx. Common causes: the file is served with a redirect chain that breaks, a non-2xx status, or — most often — a WAF/bot-rule blocking ax-audit's request (see below). Re-run with `--verbose` to see the exact status per request.
+
+### What's the difference between a weighted and an informational check?
+
+Weighted checks (14) sum to 100% and determine your overall score. Informational checks (4: `content-negotiation`, `rsl`, `agent-access`, `crawl-efficiency`) run and report full findings but contribute 0 to the score in 3.x. They gain weight in v4.0. The Content Signals findings inside `robots-txt` are likewise informational.
+
+## False positives & caveats
+
+### `agent-access` flags crawlers as blocked, but my real crawlers work fine
+
+This is the most important caveat in the tool. ax-audit's probe sends a user-agent *containing* the crawler token (e.g. `...GPTBot/1.0`) but it is **not** the real, verified crawler. If your WAF verifies bots cryptographically ([Web Bot Auth](./concepts.md)) or by IP range, it will correctly pass the genuine GPTBot while rejecting ax-audit's unverified probe. **Before changing any WAF rule, confirm against your WAF logs** whether real crawler traffic is actually being served. If it is, this finding is a false positive for your setup.
+
+### Where did `well-known-ai` go?
+
+Removed in 4.0. Three of the five files it scored turned out to have no consumer at all: `/.well-known/nlweb.json` appears in no NLWeb release, document or commit; `genai.txt` has no specification; and `/ai-plugin.json` described ChatGPT plugins, shut down in April 2024. Marking a site down for omitting a file nobody specified is worse than not checking it.
+
+### A check says `n/a` — is that bad?
+
+No, it is the point. A blog has no API to describe and no MCP server to advertise, so those checks report `n/a` and leave the score entirely rather than counting as failures. Everything counted against your site is something your site could have done.
+
+If you are *planning* to build one of those surfaces and want it audited anyway, use `--profile api`, `--profile mcp`, or `--profile all`.
+
+### My score changed after upgrading to 4.0
+
+Expected. 4.0 redistributed the weights and made protocol checks conditional. Sites with no API or MCP surface generally rise, because they are no longer marked down for lacking things they do not have.
+
+Your saved baseline still works, but regression gating is suspended until you re-save it: comparing across a scoring change would report regressions your site did not cause. Run `ax-audit <url> --save-baseline .ax-baseline.json` to resume gating.
+
+### `crawl-efficiency` says no compression, but my CDN compresses
+
+The check reads the `Content-Encoding` header on the response it received. If a proxy between ax-audit and your origin strips or fails to negotiate compression, you'll see this. Verify directly: `curl -sI -H 'Accept-Encoding: br, gzip' https://your-site.com | grep -i content-encoding`.
+
+### `content-negotiation` fails but I don't serve Markdown
+
+That's expected — most sites don't yet. It's informational (weight 0). Adopt it when you're ready; the [guide](https://axrush.com/guides/content-negotiation) covers Cloudflare/Vercel zero-code options.
+
+## Running the tool
+
+### My WAF is blocking ax-audit itself
+
+ax-audit sends a `User-Agent` of `ax-audit/<version> (+https://github.com/duranitech/ax-audit)`. If your firewall challenges unknown agents, allowlist that UA (or the IP you run from) for the duration of the audit. Note that several checks deliberately send *other* user-agents (`agent-access`) and unusual `Accept` headers (`content-negotiation`) — a WAF rejecting those is itself a finding, not a tool bug.
+
+### How do I audit a staging site behind auth?
+
+ax-audit has no auth support today. Options: run it from inside the network perimeter, temporarily allowlist its UA/IP, or audit a public preview deployment (the typical CI pattern — see [ci.md](./ci.md)).
+
+### Audits are slow / flaky on cold deployments
+
+Transient failures (timeouts, 5xx) retry automatically with backoff — raise `--retries` (default 2) for very cold preview environments and `--timeout` (default 10000ms) for slow origins. In batch mode, `--concurrency` speeds up multi-URL runs.
+
+### Can I run only some checks?
+
+Yes: `--checks llms-txt,robots-txt,rsl`. Note the overall score then averages *only* those checks, so a subset run isn't comparable to a full-audit score. Unknown IDs error out with the valid list.
+
+### Is there rate limiting I should know about?
+
+The tool itself doesn't rate-limit, but it makes several requests per audit (one per check, plus follow-ups for conditional GET, content negotiation, and the 8 `agent-access` probes). All responses are cached per run, so repeated checks of the same URL don't re-fetch. Be considerate auditing sites you don't own.
+
+## Integration
+
+### Does it work in CI?
+
+Yes — exit codes gate the build (`0` = Good/Excellent, `1` = Fair/Poor). See [ci.md](./ci.md) for GitHub Actions recipes including PR comments via `--output markdown` and regression gates via `--baseline`.
+
+### Can I consume results programmatically?
+
+Yes — `import { audit } from 'ax-audit'` returns a typed `AuditReport`. See [api.md](./api.md).
+
+### How do I generate the files ax-audit checks for?
+
+Use [ax-init](https://github.com/duranitech/ax-init) — it generates `llms.txt`, `robots.txt`, an Agent Card, `security.txt`, structured data, and header snippets, then you verify with `npx ax-audit`. Check its output against this tool: the standards moved in 2026, and a generator written against the older paths will produce files agents no longer look for.
+
+## Still stuck?
+
+Open an issue at [github.com/duranitech/ax-audit/issues](https://github.com/duranitech/ax-audit/issues) with the output of `npx ax-audit <url> --verbose`.

package/docs/getting-started.md

@@ -0,0 +1,108 @@
+# Getting Started
+
+This walkthrough takes you from zero to a passing AX score: run your first audit, learn to read the report, and fix findings in the order that moves your score most.
+
+## 1. Run your first audit
+
+No install needed:
+
+```bash
+npx ax-audit https://your-site.com
+```
+
+You get a report like:
+
+```
+  AX Audit Report
+  https://your-site.com
+
+  ██████████████████████░░░░░░░░░░░░░░░░░░  56/100  Fair
+
+  LLMs.txt (0/100)
+    FAIL  /llms.txt not found
+  ...
+```
+
+Three things to locate immediately:
+
+- **The overall score and grade.** 0–100, weighted across 14 checks. Grades: Excellent (≥90), Good (≥70), Fair (≥50), Poor (<50). The CLI exits `0` at Good or better — that is the CI gate.
+- **Per-check scores.** Each check is independent and scored 0–100. The weight of each check is in [checks.md](./checks.md).
+- **Findings.** Every `WARN`/`FAIL` line carries a hint and a `learnMoreUrl` to a remediation guide with copy-pasteable fixes.
+
+To see only what needs fixing:
+
+```bash
+npx ax-audit https://your-site.com --only-failures
+```
+
+## 2. Understand what you're optimizing
+
+AI agents interact with your site differently than browsers: most don't execute JavaScript, they look for machine-readable discovery files, and they respect (or at least read) your declared crawler policy. The audit measures three layers — if you're new to the standards involved (llms.txt, A2A, MCP, RSL, Content Signals), read [concepts.md](./concepts.md) first:
+
+1. **Content** — is there substance an agent can read, and can a browser agent act on it? (`html-rendering`, `agent-operability`, `structured-data`, `seo-basics`, `content-negotiation`)
+2. **Access** — can an agent actually retrieve it? (`agent-access`, `ai-directives`, `http-hygiene`, `tls-https`, `crawl-efficiency`)
+3. **Discovery** — can an agent find your machine-readable files? (`robots-txt`, `llms-txt`, `http-headers`, `sitemap`, `meta-tags`)
+4. **Policy** — what usage rights do you declare, and do they agree? (`usage-policy`, `security-txt`, `rsl`)
+5. **Protocols** — what can an agent call? (`api-discovery`, `agent-card`, `mcp-discovery`, `agent-skills`, `auth-discovery`)
+
+Protocol checks are conditional: if your site has no API to describe and no MCP server, they report **n/a** and leave the score alone rather than counting as failures.
+
+## 3. Fix in impact order
+
+The fastest path from Fair to Good, by weight and typical effort:
+
+| Step | Check | Weight | Typical effort |
+| --- | --- | --- | --- |
+| 1 | Verify server-rendered content | 11% | Free if you already render on the server; significant if you ship an SPA shell. Nothing else matters if an agent sees an empty page. |
+| 2 | Confirm nothing blocks AI crawlers | 9% | 15 minutes in your WAF. Check `agent-access` first: your robots.txt may say one thing and your firewall another. |
+| 3 | Configure `robots.txt` for the 12 core AI crawlers | 9% | 15 minutes; `npx ax-init` generates it |
+| 4 | Name your buttons and label your inputs | 7% | Hours to days, and it is accessibility work you owed anyway |
+| 5 | Check your page-level AI directives | 6% | 10 minutes. A stray `nosnippet` removes you from AI Overviews. |
+| 6 | Add JSON-LD structured data | 6% | 1–2 hours |
+| 7 | Create `/llms.txt` | 5% | 30 minutes — it is a Markdown file. Read the check's note on who actually fetches it first. |
+
+Steps 1 and 2 are the ones worth doing today. They are also the two most likely to be silently broken: a hydration-only page and a firewall rule are both invisible from inside the site.
+
+The remaining checks (`seo-basics`, `content-negotiation`, `usage-policy`, `http-hygiene`, `http-headers`, `security-txt`, `tls-https`, `sitemap`, `crawl-efficiency`, `rsl`, `meta-tags`) are mostly configuration; the remediation guides give exact snippets for Nginx, Vercel, Netlify, and Express.
+
+Re-run after each fix — all requests are cached per run, so audits are fast and cheap.
+
+## 4. Lock in your progress with a baseline
+
+Once you reach a score you're happy with, freeze it:
+
+```bash
+npx ax-audit https://your-site.com --save-baseline .ax-baseline.json
+git add .ax-baseline.json && git commit -m "chore: AX baseline"
+```
+
+From then on, compare every run against it:
+
+```bash
+npx ax-audit https://your-site.com --baseline .ax-baseline.json --fail-on-regression 5
+```
+
+This catches drift you didn't cause — a CDN toggle, a WAF rule, a header dropped in a refactor. Wire it into CI with the recipes in [ci.md](./ci.md).
+
+## 5. Look at the informational checks
+
+Four checks report findings without affecting your score yet (they will in v4.0): `content-negotiation`, `rsl`, `agent-access`, `crawl-efficiency`. Treat them as the early-warning lane — they cover the newest standards, and fixing them now means v4.0 changes nothing for you.
+
+The one to check first is `agent-access`: it detects the failure mode you cannot see — your robots.txt allows GPTBot while your WAF returns it a 403:
+
+```bash
+npx ax-audit https://your-site.com --checks agent-access
+```
+
+## Common first-run questions
+
+- **"My score seems harsh."** The audit measures the AI-agent surface, not site quality. A beautiful SPA with no llms.txt, no structured data, and an empty `#root` div is genuinely poor AX — that's the point of the tool.
+- **"A check crashed / network error."** Transient failures retry automatically (`--retries`, default 2). For slow staging environments raise `--timeout`.
+- **"Which findings are safe to ignore?"** See the [FAQ](./faq.md) — notably the `agent-access` verified-bots caveat, and the checks resting on draft specifications, which never affect your score.
+
+## Next steps
+
+- [checks.md](./checks.md) — exact scoring of all 26 checks, with the weight table
+- [concepts.md](./concepts.md) — the AX standards landscape explained
+- [cli.md](./cli.md) — every flag · [ci.md](./ci.md) — CI recipes · [api.md](./api.md) — programmatic use
+- [ax-init](https://github.com/duranitech/ax-init) — generates most of the files this tool audits