ax-audit 3.0.0 → 3.6.0

package/docs/faq.md

@@ -0,0 +1,77 @@
+# FAQ & Troubleshooting
+
+## Scores & results
+
+### Why did my score change after upgrading ax-audit?
+
+In 3.x, score changes on the same site are treated as **breaking** and only happen in major or minor releases that explicitly say so. The 3.0.0 release redistributed weights across 14 checks and added Content-Type penalties — see its CHANGELOG entry. Every check added since (3.1.0–3.6.0) ships at **weight 0** precisely so your score and baselines don't move. To track changes deliberately, use `--baseline` (see [cli.md](./cli.md)).
+
+### Why is my score lower than my Lighthouse / SEO score?
+
+ax-audit measures the *AI-agent* surface, not performance, accessibility, or human SEO. A fast, beautiful site can still score poorly if it has no `llms.txt`, ships an empty SPA shell to non-JS crawlers, and exposes no structured data. That gap is the reason the tool exists.
+
+### A check shows 0 but the file exists — why?
+
+Most "not found" hard-fails mean the request didn't return a 2xx. Common causes: the file is served with a redirect chain that breaks, a non-2xx status, or — most often — a WAF/bot-rule blocking ax-audit's request (see below). Re-run with `--verbose` to see the exact status per request.
+
+### What's the difference between a weighted and an informational check?
+
+Weighted checks (14) sum to 100% and determine your overall score. Informational checks (4: `content-negotiation`, `rsl`, `agent-access`, `crawl-efficiency`) run and report full findings but contribute 0 to the score in 3.x. They gain weight in v4.0. The Content Signals findings inside `robots-txt` are likewise informational.
+
+## False positives & caveats
+
+### `agent-access` flags crawlers as blocked, but my real crawlers work fine
+
+This is the most important caveat in the tool. ax-audit's probe sends a user-agent *containing* the crawler token (e.g. `...GPTBot/1.0`) but it is **not** the real, verified crawler. If your WAF verifies bots cryptographically ([Web Bot Auth](./concepts.md)) or by IP range, it will correctly pass the genuine GPTBot while rejecting ax-audit's unverified probe. **Before changing any WAF rule, confirm against your WAF logs** whether real crawler traffic is actually being served. If it is, this finding is a false positive for your setup.
+
+### `well-known-ai` is low — should I worry?
+
+No. It's scored as *coverage bonus* over five emerging, partly-competing files (`ai.txt`, `genai.txt`, `ai-plugin.json`, `agents.json`, `nlweb.json`). None is universally adopted; a low score here is not a defect. Implement the ones relevant to your stack.
+
+### `crawl-efficiency` says no compression, but my CDN compresses
+
+The check reads the `Content-Encoding` header on the response it received. If a proxy between ax-audit and your origin strips or fails to negotiate compression, you'll see this. Verify directly: `curl -sI -H 'Accept-Encoding: br, gzip' https://your-site.com | grep -i content-encoding`.
+
+### `content-negotiation` fails but I don't serve Markdown
+
+That's expected — most sites don't yet. It's informational (weight 0). Adopt it when you're ready; the [guide](https://lucioduran.com/projects/ax-audit/guides/content-negotiation) covers Cloudflare/Vercel zero-code options.
+
+## Running the tool
+
+### My WAF is blocking ax-audit itself
+
+ax-audit sends a `User-Agent` of `ax-audit/<version> (+https://github.com/lucioduran/ax-audit)`. If your firewall challenges unknown agents, allowlist that UA (or the IP you run from) for the duration of the audit. Note that several checks deliberately send *other* user-agents (`agent-access`) and unusual `Accept` headers (`content-negotiation`) — a WAF rejecting those is itself a finding, not a tool bug.
+
+### How do I audit a staging site behind auth?
+
+ax-audit has no auth support today. Options: run it from inside the network perimeter, temporarily allowlist its UA/IP, or audit a public preview deployment (the typical CI pattern — see [ci.md](./ci.md)).
+
+### Audits are slow / flaky on cold deployments
+
+Transient failures (timeouts, 5xx) retry automatically with backoff — raise `--retries` (default 2) for very cold preview environments and `--timeout` (default 10000ms) for slow origins. In batch mode, `--concurrency` speeds up multi-URL runs.
+
+### Can I run only some checks?
+
+Yes: `--checks llms-txt,robots-txt,rsl`. Note the overall score then averages *only* those checks, so a subset run isn't comparable to a full-audit score. Unknown IDs error out with the valid list.
+
+### Is there rate limiting I should know about?
+
+The tool itself doesn't rate-limit, but it makes several requests per audit (one per check, plus follow-ups for conditional GET, content negotiation, and the 8 `agent-access` probes). All responses are cached per run, so repeated checks of the same URL don't re-fetch. Be considerate auditing sites you don't own.
+
+## Integration
+
+### Does it work in CI?
+
+Yes — exit codes gate the build (`0` = Good/Excellent, `1` = Fair/Poor). See [ci.md](./ci.md) for GitHub Actions recipes including PR comments via `--output markdown` and regression gates via `--baseline`.
+
+### Can I consume results programmatically?
+
+Yes — `import { audit } from 'ax-audit'` returns a typed `AuditReport`. See [api.md](./api.md).
+
+### How do I generate the files ax-audit checks for?
+
+Use [ax-init](https://github.com/lucioduran/ax-init) — it generates `llms.txt`, `robots.txt`, `agent.json`, `mcp.json`, `security.txt`, structured data, and header snippets, then you verify with `npx ax-audit`.
+
+## Still stuck?
+
+Open an issue at [github.com/lucioduran/ax-audit/issues](https://github.com/lucioduran/ax-audit/issues) with the output of `npx ax-audit <url> --verbose`.

package/docs/getting-started.md

@@ -0,0 +1,101 @@
+# Getting Started
+
+This walkthrough takes you from zero to a passing AX score: run your first audit, learn to read the report, and fix findings in the order that moves your score most.
+
+## 1. Run your first audit
+
+No install needed:
+
+```bash
+npx ax-audit https://your-site.com
+```
+
+You get a report like:
+
+```
+  AX Audit Report
+  https://your-site.com
+
+  ██████████████████████░░░░░░░░░░░░░░░░░░  56/100  Fair
+
+  LLMs.txt (0/100)
+    FAIL  /llms.txt not found
+  ...
+```
+
+Three things to locate immediately:
+
+- **The overall score and grade.** 0–100, weighted across 14 checks. Grades: Excellent (≥90), Good (≥70), Fair (≥50), Poor (<50). The CLI exits `0` at Good or better — that is the CI gate.
+- **Per-check scores.** Each check is independent and scored 0–100. The weight of each check is in [checks.md](./checks.md).
+- **Findings.** Every `WARN`/`FAIL` line carries a hint and a `learnMoreUrl` to a remediation guide with copy-pasteable fixes.
+
+To see only what needs fixing:
+
+```bash
+npx ax-audit https://your-site.com --only-failures
+```
+
+## 2. Understand what you're optimizing
+
+AI agents interact with your site differently than browsers: most don't execute JavaScript, they look for machine-readable discovery files, and they respect (or at least read) your declared crawler policy. The audit measures three layers — if you're new to the standards involved (llms.txt, A2A, MCP, RSL, Content Signals), read [concepts.md](./concepts.md) first:
+
+1. **Can agents find and read your content?** (`html-rendering`, `robots-txt`, `sitemap`, `tls-https`, `agent-access`)
+2. **Did you publish the AI-specific surface?** (`llms-txt`, `agent-json`, `mcp`, `openapi`, `well-known-ai`, `meta-tags`, `structured-data`)
+3. **Is the interaction efficient and well-governed?** (`content-negotiation`, `crawl-efficiency`, `rsl`, Content Signals, `http-headers`, `security-txt`, `seo-basics`)
+
+## 3. Fix in impact order
+
+The fastest path from Fair to Good, by weight and typical effort:
+
+| Step | Check | Weight | Typical effort |
+| --- | --- | --- | --- |
+| 1 | Create `/llms.txt` | 11% | 30 minutes — it's a Markdown file. `npx ax-init` generates it. |
+| 2 | Configure `robots.txt` for the 8 core AI crawlers | 11% | 15 minutes; `npx ax-init` generates this too |
+| 3 | Verify server-rendered content | 9% | Free if you SSR; significant if you ship an SPA shell |
+| 4 | Add JSON-LD structured data | 9% | 1–2 hours |
+| 5 | Security + discovery headers | 9% | 30 minutes of server config |
+| 6 | `agent.json` + `mcp.json` | 14% combined | An hour with the spec links in the guides |
+
+The remaining weighted checks (`seo-basics`, `security-txt`, `meta-tags`, `openapi`, `tls-https`, `sitemap`, `well-known-ai`) are mostly configuration; the remediation guides give exact snippets for Nginx, Vercel, Netlify, and Express.
+
+Re-run after each fix — all requests are cached per run, so audits are fast and cheap.
+
+## 4. Lock in your progress with a baseline
+
+Once you reach a score you're happy with, freeze it:
+
+```bash
+npx ax-audit https://your-site.com --save-baseline .ax-baseline.json
+git add .ax-baseline.json && git commit -m "chore: AX baseline"
+```
+
+From then on, compare every run against it:
+
+```bash
+npx ax-audit https://your-site.com --baseline .ax-baseline.json --fail-on-regression 5
+```
+
+This catches drift you didn't cause — a CDN toggle, a WAF rule, a header dropped in a refactor. Wire it into CI with the recipes in [ci.md](./ci.md).
+
+## 5. Look at the informational checks
+
+Four checks report findings without affecting your score yet (they will in v4.0): `content-negotiation`, `rsl`, `agent-access`, `crawl-efficiency`. Treat them as the early-warning lane — they cover the newest standards, and fixing them now means v4.0 changes nothing for you.
+
+The one to check first is `agent-access`: it detects the failure mode you cannot see — your robots.txt allows GPTBot while your WAF returns it a 403:
+
+```bash
+npx ax-audit https://your-site.com --checks agent-access
+```
+
+## Common first-run questions
+
+- **"My score seems harsh."** The audit measures the AI-agent surface, not site quality. A beautiful SPA with no llms.txt, no structured data, and an empty `#root` div is genuinely poor AX — that's the point of the tool.
+- **"A check crashed / network error."** Transient failures retry automatically (`--retries`, default 2). For slow staging environments raise `--timeout`.
+- **"Which findings are safe to ignore?"** See the [FAQ](./faq.md) — notably the `agent-access` verified-bots caveat and `well-known-ai`, which is coverage bonus rather than baseline.
+
+## Next steps
+
+- [checks.md](./checks.md) — exact scoring of all 18 checks
+- [concepts.md](./concepts.md) — the AX standards landscape explained
+- [cli.md](./cli.md) — every flag · [ci.md](./ci.md) — CI recipes · [api.md](./api.md) — programmatic use
+- [ax-init](https://github.com/lucioduran/ax-init) — generates most of the files this tool audits

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ax-audit",
-  "version": "3.0.0",
+  "version": "3.6.0",
   "description": "Audit websites for AI Agent Experience (AX) readiness. Lighthouse for AI Agents.",
   "type": "module",
   "license": "Apache-2.0",
@@ -40,6 +40,7 @@
   "files": [
     "bin/",
     "dist/",
+    "docs/",
     "LICENSE",
     "README.md",
     "CHANGELOG.md"