@vercel/agent-readability 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (17) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +227 -0
  3. package/dist/cli/index.cjs +228 -0
  4. package/dist/index.cjs +175 -0
  5. package/dist/index.cjs.map +1 -0
  6. package/dist/index.d.cts +82 -0
  7. package/dist/index.d.ts +82 -0
  8. package/dist/index.js +141 -0
  9. package/dist/index.js.map +1 -0
  10. package/dist/next/index.cjs +166 -0
  11. package/dist/next/index.cjs.map +1 -0
  12. package/dist/next/index.d.cts +35 -0
  13. package/dist/next/index.d.ts +35 -0
  14. package/dist/next/index.js +140 -0
  15. package/dist/next/index.js.map +1 -0
  16. package/package.json +91 -0
  17. package/skill/SKILL.md +171 -0
package/skill/SKILL.md ADDED
@@ -0,0 +1,171 @@
1
+ ---
2
+ name: agent-readability
3
+ description: Make a website readable by AI agents. Adds llms.txt, content negotiation, structured data, and runs the readability audit.
4
+ ---
5
+
6
+ # Agent Readability
7
+
8
+ Make a website readable by AI agents by adding discovery files, content negotiation, and structured data.
9
+
10
+ ## When to Use
11
+
12
+ - user asks to make their site agent-friendly or AI-readable
13
+ - user wants to serve markdown to AI agents
14
+ - user wants to add llms.txt, content negotiation, or structured data
15
+ - user asks about AI agent readability or the agent readability spec
16
+ - user wants to run an agent readability audit
17
+
18
+ ## Steps
19
+
20
+ ### 1. Install
21
+
22
+ ```bash
23
+ pnpm add @vercel/agent-readability
24
+ ```
25
+
26
+ ### 2. Add middleware (Next.js)
27
+
28
+ Create or update `middleware.ts`:
29
+
30
+ ```ts
31
+ import { withAgentReadability } from '@vercel/agent-readability/next'
32
+
33
+ export default withAgentReadability({
34
+ docsPrefix: '/docs',
35
+ rewrite: (pathname) => `/api/docs-md${pathname}`,
36
+ })
37
+
38
+ export const config = {
39
+ matcher: ['/docs/:path*'],
40
+ }
41
+ ```
42
+
43
+ If middleware already exists, wrap it:
44
+
45
+ ```ts
46
+ import { withAgentReadability } from '@vercel/agent-readability/next'
47
+ import { existingMiddleware } from './lib/middleware'
48
+
49
+ export default withAgentReadability(
50
+ { rewrite: (pathname) => `/api/docs-md${pathname}` },
51
+ existingMiddleware,
52
+ )
53
+ ```
54
+
55
+ For non-Next.js frameworks, use the core API directly:
56
+
57
+ ```ts
58
+ import { isAIAgent, acceptsMarkdown } from '@vercel/agent-readability'
59
+
60
+ // In your request handler:
61
+ if (isAIAgent(request).detected || acceptsMarkdown(request)) {
62
+ return new Response(markdownContent, {
63
+ headers: { 'Content-Type': 'text/markdown', 'Vary': 'Accept' },
64
+ })
65
+ }
66
+ ```
67
+
68
+ ### 3. Add /llms.txt
69
+
70
+ Create a route that returns a markdown index of your site content. Format follows the llms.txt spec (llmstxt.org):
71
+
72
+ ```ts
73
+ // Next.js: app/llms.txt/route.ts
74
+ export async function GET() {
75
+ const content = `# Site Name
76
+
77
+ > Brief description of the site
78
+
79
+ ## Docs
80
+ - [Getting Started](/docs/getting-started): How to get started
81
+ - [API Reference](/docs/api): Complete API documentation
82
+ `
83
+ return new Response(content, {
84
+ headers: { 'Content-Type': 'text/plain; charset=utf-8' },
85
+ })
86
+ }
87
+ ```
88
+
89
+ ### 4. Add /sitemap.md
90
+
91
+ Create a markdown sitemap with headings and links for AI navigation:
92
+
93
+ ```ts
94
+ // Next.js: app/sitemap.md/route.ts
95
+ export async function GET() {
96
+ const content = `# Sitemap
97
+
98
+ ## Getting Started
99
+ - [Introduction](/docs/introduction): Overview and setup
100
+ - [Quick Start](/docs/quickstart): Get running in 5 minutes
101
+
102
+ ## API Reference
103
+ - [Authentication](/docs/api/auth): Auth endpoints
104
+ - [Users](/docs/api/users): User management
105
+ `
106
+ return new Response(content, {
107
+ headers: { 'Content-Type': 'text/markdown; charset=utf-8' },
108
+ })
109
+ }
110
+ ```
111
+
112
+ ### 5. Add structured data
113
+
114
+ Add JSON-LD to pages with Schema.org types (Article, TechArticle, WebPage):
115
+
116
+ ```html
117
+ <script type="application/ld+json">
118
+ {
119
+ "@context": "https://schema.org",
120
+ "@type": "TechArticle",
121
+ "headline": "Page Title",
122
+ "description": "Page description"
123
+ }
124
+ </script>
125
+ ```
126
+
127
+ ### 6. Add link alternate to HTML pages
128
+
129
+ Add to `<head>` on all pages that have markdown versions:
130
+
131
+ ```html
132
+ <link rel="alternate" type="text/markdown" href="/docs/page.md">
133
+ ```
134
+
135
+ ### 7. Add frontmatter to markdown responses
136
+
137
+ All markdown responses should include YAML frontmatter:
138
+
139
+ ```markdown
140
+ ---
141
+ title: Page Title
142
+ description: Brief page description
143
+ canonical_url: https://example.com/docs/page
144
+ last_updated: 2026-03-30
145
+ ---
146
+ ```
147
+
148
+ ### 8. Check robots.txt
149
+
150
+ Verify these AI bots are not blocked: GPTBot, ClaudeBot, CCBot, Google-Extended. Remove any `Disallow: /` rules targeting them.
151
+
152
+ ### 9. Run the audit
153
+
154
+ ```bash
155
+ npx @vercel/agent-readability audit https://your-site.com
156
+ ```
157
+
158
+ The audit scores your site on 16 checks across discovery, content delivery, and HTML quality. Failed checks include fix suggestions.
159
+
160
+ ### 10. Add CI check (optional)
161
+
162
+ ```yaml
163
+ - name: Agent readability
164
+ run: npx @vercel/agent-readability audit ${{ env.SITE_URL }} --min-score 70 --json
165
+ ```
166
+
167
+ ## Notes
168
+
169
+ - Set `Vary: Accept` on all markdown responses for correct CDN caching
170
+ - For missing pages, return 200 with markdown body (agents discard 404 bodies)
171
+ - The `onDetection` callback in the Next.js adapter runs via `event.waitUntil()`