@vercel/agent-readability 0.3.0 → 0.4.0

package/README.md

@@ -9,7 +9,7 @@ Detect AI agents. Serve them markdown. Audit your site against the
 npm install @vercel/agent-readability
 ```
 
-Or run the audit CLI directly without installing:
+Or audit without installing:
 
 ```bash
 npx @vercel/agent-readability audit https://vercel.com/docs
@@ -17,40 +17,55 @@ npx @vercel/agent-readability audit https://vercel.com/docs
 
 ## Quick Start
 
-Add AI agent detection to your Next.js middleware in three lines:
-
+**Next.js** `middleware.ts`:
 ```ts
-// middleware.ts
 import { withAgentReadability } from '@vercel/agent-readability/next'
 
 export default withAgentReadability({
   rewrite: (pathname) => `/api/docs-md${pathname}`,
 })
 
-export const config = {
-  matcher: ['/docs/:path*'],
-}
+export const config = { matcher: ['/docs/:path*'] }
+```
+
+**SvelteKit** `hooks.server.ts`:
+```ts
+import { handleAgentReadability } from '@vercel/agent-readability/sveltekit'
+import { sequence } from '@sveltejs/kit/hooks'
+
+export const handle = sequence(
+  handleAgentReadability({ rewrite: (p) => `/api/docs-md${p}` }),
+)
+```
+
+**Nuxt** `server/middleware/agent.ts`:
+```ts
+import { defineAgentMiddleware } from '@vercel/agent-readability/nuxt'
+
+export default defineAgentMiddleware({
+  getMarkdown: async (pathname) => {
+    const doc = await fetchDoc(pathname)
+    return doc.markdown
+  },
+})
 ```
 
-AI agents hitting `/docs/*` now receive markdown instead of HTML.
+Agents hitting `/docs/*` get markdown instead of HTML.
 
-## What It Does
+## How Detection Works
 
-Three-layer detection identifies AI agents from HTTP request headers:
+Three layers, checked in order:
 
-1. **Known UA patterns.** 40+ agents including ClaudeBot, GPTBot, Cursor, and Perplexity.
-2. **Signature-Agent header.** Catches ChatGPT agent (RFC 9421).
-3. **sec-fetch-mode heuristic.** Catches unknown bots that lack browser fingerprints.
+1. **Known UA patterns.** 30+ agents (ClaudeBot, GPTBot, Cursor, Perplexity, etc.)
+2. **Signature-Agent header.** ChatGPT agent via RFC 9421.
+3. **sec-fetch-mode heuristic.** Unknown bots lacking browser fingerprints.
 
-The detection optimizes for recall over precision. Serving markdown to a non-AI bot
-is low cost. Missing an AI agent means a worse experience.
+Optimizes for recall over precision. Serving markdown to a non-AI bot is cheap. Missing an AI agent is not.
 
 ## Core API
 
 ### `isAIAgent(request)`
 
-Detect AI agents from request headers.
-
 ```ts
 import { isAIAgent } from '@vercel/agent-readability'
 
@@ -58,19 +73,14 @@ const result = isAIAgent(request)
 // { detected: true, method: 'ua-match' }
 ```
 
-Accepts any object with a `headers.get()` method: `Request`, `NextRequest`,
-or a custom wrapper.
+Accepts any object with `headers.get()` (`Request`, `NextRequest`, etc.)
 
-Returns a discriminated union:
+Returns:
 - `{ detected: true, method: 'ua-match' | 'signature-agent' | 'heuristic' }`
 - `{ detected: false, method: null }`
 
-When `detected` is `true`, `method` is always non-null. TypeScript narrows automatically.
-
 ### `acceptsMarkdown(request)`
 
-Check if the request prefers markdown via the `Accept` header.
-
 ```ts
 import { acceptsMarkdown } from '@vercel/agent-readability'
 
@@ -83,7 +93,7 @@ if (acceptsMarkdown(request)) {
 
 ### `shouldServeMarkdown(request)`
 
-Combines agent detection and content negotiation into one call.
+Combines detection + content negotiation.
 
 ```ts
 import { shouldServeMarkdown } from '@vercel/agent-readability'
@@ -94,8 +104,8 @@ const { serve, reason } = shouldServeMarkdown(request)
 
 ### `generateNotFoundMarkdown(path, options?)`
 
-Generates a markdown body for missing pages. Return this with a 200 status
-(not 404) because agents discard 404 response bodies.
+Markdown body for missing pages. Return with 200 on the canonical URL
+(agents discard 404 bodies).
 
 ```ts
 import { generateNotFoundMarkdown } from '@vercel/agent-readability'
@@ -103,23 +113,25 @@ import { generateNotFoundMarkdown } from '@vercel/agent-readability'
 const md = generateNotFoundMarkdown('/docs/missing', {
   baseUrl: 'https://example.com',
 })
-// Return as 200 so agents read the body
 return new Response(md, {
-  headers: { 'Content-Type': 'text/markdown' },
+  headers: { 'Content-Type': 'text/markdown', 'Vary': 'Accept' },
 })
 ```
 
+Keep suggested page links canonical (`/docs/page`) and negotiate markdown with
+`Accept: text/markdown` rather than exposing `.md` page URLs in not-found
+responses.
+
 ### Pattern Exports
 
 `AI_AGENT_UA_PATTERNS`, `TRADITIONAL_BOT_PATTERNS`, `SIGNATURE_AGENT_DOMAINS`,
-and `BOT_LIKE_REGEX` are exported for consumers who need to extend or inspect them.
+and `BOT_LIKE_REGEX` are all exported.
 
 ## Next.js Adapter
 
 ### `withAgentReadability(options, handler?)`
 
-Middleware wrapper that detects AI agents and rewrites matching requests
-to markdown routes. Compatible with Next.js 14 and 15 (Pages and App Router).
+Works with Next.js 14 and 15 (Pages and App Router).
 
 ```ts
 import { withAgentReadability } from '@vercel/agent-readability/next'
@@ -133,18 +145,13 @@ export default withAgentReadability({
 })
 ```
 
-The `onDetection` callback runs via `event.waitUntil()` and does not block
-the response.
+`onDetection` runs via `event.waitUntil()` and does not block the response.
 
-#### Composing with other middleware
-
-Pass your existing middleware as the second argument:
+#### Composing with existing middleware
 
 ```ts
 export default withAgentReadability(
-  {
-    rewrite: (p) => `/md${p}`,
-  },
+  { rewrite: (p) => `/md${p}` },
   (req, event) => i18nMiddleware(req, event),
 )
 ```
@@ -159,8 +166,7 @@ export default withAgentReadability(
 
 #### `agentReadabilityMatcher`
 
-A pre-built matcher that excludes Next.js internals and static files.
-Use this for site-wide agent detection instead of scoping to a prefix:
+Excludes Next.js internals and static files. Use for site-wide detection:
 
 ```ts
 import { withAgentReadability, agentReadabilityMatcher } from '@vercel/agent-readability/next'
@@ -175,27 +181,90 @@ export const config = {
 }
 ```
 
-## Audit CLI
+## SvelteKit Adapter
+
+### `handleAgentReadability(options)`
+
+Returns a `Handle` function. Requires SvelteKit 2+. Uses `event.fetch()`
+for zero-cost internal routing to your `+server.ts` markdown routes.
+
+```ts
+// hooks.server.ts
+import { handleAgentReadability } from '@vercel/agent-readability/sveltekit'
+import { sequence } from '@sveltejs/kit/hooks'
+
+export const handle = sequence(
+  handleAgentReadability({
+    docsPrefix: '/docs',
+    rewrite: (pathname) => `/api/docs-md${pathname}`,
+    onDetection: ({ path, method }) => {
+      console.log(`Agent detected: ${method} on ${path}`)
+    },
+  }),
+)
+```
 
-Check your site against the Agent Readability Spec.
+Automatically guards against infinite loops (`isSubRequest`), skips client
+navigation requests (`isDataRequest`), and falls through if the rewrite
+target returns non-OK.
+
+#### Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `docsPrefix` | `string` | `'/docs'` | URL prefix to intercept |
+| `rewrite` | `(pathname: string) => string` | required | Maps request path to `+server.ts` route |
+| `onDetection` | `(info) => void \| Promise<void>` | - | Fire-and-forget analytics callback |
+
+## Nuxt Adapter
+
+### `defineAgentMiddleware(options)`
+
+Wraps `defineEventHandler`. Requires h3 1.8+ (ships with Nuxt 3/4).
+Uses a `getMarkdown` callback instead of rewrite since Nuxt has no
+zero-cost internal fetch.
+
+```ts
+// server/middleware/agent.ts
+import { defineAgentMiddleware } from '@vercel/agent-readability/nuxt'
+
+export default defineAgentMiddleware({
+  docsPrefix: '/docs',
+  getMarkdown: async (pathname, event) => {
+    const doc = await queryContent(pathname).findOne()
+    return doc.body
+  },
+})
+```
+
+`getMarkdown` can return a `string` (auto-wrapped with `text/markdown` headers)
+or a `Response` for full control.
+
+#### Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `docsPrefix` | `string` | `'/docs'` | URL prefix to intercept |
+| `getMarkdown` | `(pathname, event) => string \| Response \| Promise<...>` | required | Returns markdown content |
+| `onDetection` | `(info) => void \| Promise<void>` | - | Fire-and-forget analytics callback |
+
+## Audit CLI
 
 ```bash
 npx @vercel/agent-readability audit https://sdk.vercel.ai
 ```
 
-Runs 16 weighted checks across three categories. Returns a score from 0 to 100.
-Failed checks include fix suggestions that can be copy-pasted into your coding agent.
+25 weighted checks across 4 categories. Score 0-100. Failed checks include
+fix suggestions you can paste into your coding agent.
 
-### CI Integration
+### CI
 
 ```yaml
 - name: Audit agent readability
   run: npx @vercel/agent-readability audit ${{ env.SITE_URL }} --min-score 70 --json
 ```
 
-Exits with code 1 if the score is below the threshold.
-
-### CLI Options
+Exit code 1 if score is below threshold.
 
 | Flag | Description |
 |------|-------------|
@@ -204,23 +273,19 @@ Exits with code 1 if the score is below the threshold.
 
 ## Caching
 
-When the same URL serves HTML to browsers and markdown to AI agents,
-CDN caching must include the `Accept` header in the cache key.
-
-Set `Vary: Accept` on all markdown responses. Without it, CDNs may serve
-cached HTML to agents or cached markdown to browsers.
+Set `Vary: Accept` on markdown responses so CDNs don't serve cached
+HTML to agents (or cached markdown to browsers).
 
-The `withAgentReadability` middleware handles detection and rewriting.
-Your markdown route handler is responsible for setting response headers
-including `Vary: Accept` and `Content-Type: text/markdown`.
+The SvelteKit and Nuxt adapters set `Vary: Accept` automatically. The
+Next.js adapter rewrites the URL, so your markdown route handler must
+set `Vary: Accept` and `Content-Type: text/markdown`.
 
 ## Edge Runtime
 
-The core library and Next.js adapter use only Web APIs. They work in
-Vercel Edge Runtime, Cloudflare Workers, and any standard `Request`/`Response`
-environment.
+Core library and all adapters use Web APIs only. Works in Vercel Edge
+Runtime, Cloudflare Workers, and any `Request`/`Response` environment.
 
-The CLI runs on Node.js 20+.
+CLI requires Node.js 20+.
 
 ## License