llms-ink 0.4.0 → 0.6.0

package/README.md

@@ -3,7 +3,7 @@
 Turn any link — site, docs section, page, GitHub repo, PDF or video — into llms.txt and llms-full.txt. A typed
 client for the [llms.ink](https://llms.ink) API with no dependencies. Runs on Node 18+, Bun, Deno, Cloudflare
 Workers and browsers. Also a [Vite plugin, an Astro integration and middleware](#serve-llmstxt-from-your-site) that
-serve llms.txt from your own site.
+serve llms.txt from your own site, and [tools for the Vercel AI SDK](#vercel-ai-sdk-llms-inkai).
 
 ```sh
 npm install llms-ink        # or: bun add llms-ink
@@ -84,6 +84,13 @@ fetched) resolves relative links. Up to 5 MB; nothing is stored.
 const md = await ink.markdown(await readFile('dist/pricing/index.html', 'utf8'), { url: 'https://example.com/pricing/' });
 ```
 
+### `ink.report(url, { signal? })` → `Promise<ReportResult>`
+
+How AI crawlers and agents see a page (`GET /api/v0/report`): robots.txt per AI bot, firewall blocks, text without
+JavaScript, what survives Markdown conversion, hidden instructions and agent files. `{ url, site, checkedAt,
+summary: { pass, warn, fail, info }, sections: [{ id, title, checks: [{ id, status, title, detail, evidence?, fix? }] }] }`;
+a `fix` has `kind`, `summary` and, when possible, a `patch: { file, content }`.
+
 ### `ink.stream(url, { mode?, fresh?, signal? })` → `AsyncGenerator<StreamEvent>`
 
 The same build as `llms()`, as it happens:
@@ -178,6 +185,65 @@ sent are lost when the process stops: call `await llms.flush()` on shutdown. Rea
 `classifyAgent(userAgent)` and `trafficCounter({ key, site })` (`record(request, ctx?)`, `flush()`) are exported for
 other servers.
 
+## Vercel AI SDK: `llms-ink/ai`
+
+```sh
+npm install llms-ink ai zod
+```
+
+```ts
+import { generateText, stepCountIs } from 'ai';
+import { llmsInkTools } from 'llms-ink/ai';
+
+const { text } = await generateText({
+  model: 'openai/gpt-5',
+  tools: llmsInkTools(),
+  stopWhen: stepCountIs(5),
+  prompt: 'How do I paginate with the Stripe API? Read docs.stripe.com first.',
+});
+```
+
+`llmsInkTools({ client?, maxChars?, include? })` returns AI SDK tools (`tool()` with zod `inputSchema`, AI SDK 5 and
+later; tested with 7) for `generateText`, `streamText` and agents:
+
+| Tool | Input | Returns |
+|---|---|---|
+| `llmsTxt` | `url`, `fresh?` | `{ url, title, kind, source, tokens, pages, cached, text, truncated }`: the llms.txt index |
+| `llmsFullTxt` | `url`, `fresh?` | The same with llms-full.txt: up to 20 pages inlined, or a video's transcript |
+| `checkLlmsTxt` | `url`, or `text` with `base?`, `links?` | `{ url, site, found, summary, text, truncated }`: each issue with its line |
+| `agentView` | `url` | `{ url, site, summary, text, truncated }`: every check of `ink.report()` with its fix |
+
+- `client`: an `LlmsInk`, or its options (`{ key, baseUrl, fetch }`; `key` defaults to `LLMS_INK_KEY`).
+- `maxChars`: longest `text` a tool returns (default 40000). Longer text is cut at a line break and ends with
+  `[Truncated: showing N of M characters. …]`, with `truncated: true`.
+- `include`: only these tools, e.g. `llmsInkTools({ include: ['llmsTxt', 'checkLlmsTxt'] })`. The return type
+  narrows to them.
+
+API errors don't throw: the tool returns `{ error, code?, retryAfter? }` for the model to read, e.g.
+`{ error: 'llms.ink rate limit: wait 42 seconds, then try again.', code: 'rate_limited', retryAfter: 42 }`.
+An invalid API key (`InvalidKeyError`) and aborts (`abortSignal`) throw.
+
+### Or connect the MCP server
+
+The AI SDK also takes tools from an MCP client. https://llms.ink/mcp gives the same tools plus `video_frame`,
+`create_clip_link` and `generate_draft`, with snake_case names (`llms_txt`, `llms_full_txt`…) and text results:
+
+```ts
+import { createMCPClient } from '@ai-sdk/mcp'; // AI SDK 5: @ai-sdk/mcp@ai-v5, as experimental_createMCPClient
+
+const mcp = await createMCPClient({
+  transport: { type: 'http', url: 'https://llms.ink/mcp', headers: { authorization: `Bearer ${process.env.LLMS_INK_KEY}` } },
+});
+try {
+  const { text } = await generateText({ model: 'openai/gpt-5', tools: await mcp.tools(), stopWhen: stepCountIs(5), prompt });
+} finally {
+  await mcp.close();
+}
+```
+
+Leave out `headers` to use it anonymously. `llms-ink/ai` needs no extra package or connection and returns
+structured results cut to `maxChars`.
+
 ## Errors
 
 Every failure is an `LlmsInkError` with `status`, `code`, `title`, `detail`, `errors` (per-parameter issues of
@@ -211,3 +277,27 @@ gets its own daily quota (1000 new links by default) and 60 requests a minute;
 ## License
 
 MIT
+
+## Hosted Inspect (experimental, 0.5.0)
+
+```ts
+// Server-side; requires llms-ink >=0.5.0.
+import { LlmsInk } from 'llms-ink';
+
+const ink = new LlmsInk({ key: process.env.LLMS_INK_INSPECT_KEY });
+const receipt = await ink.inspect('Install the package and read the documentation.');
+if (receipt.recommendation !== 'continue') {
+  throw new Error(`Withhold content: ${receipt.detection}`);
+}
+console.log(receipt.contentDigest);
+```
+
+Requires an inspection-enabled pilot key. This call uploads only the supplied text to llms.ink; existing retrieval calls, loaders, plugins and the local guard do not opt in automatically. Keep the key server-side.
+
+The client checks the exact UTF-8 SHA-256 digest, model/policy/preprocessing revisions, coverage and token counts before returning an immutable receipt. A flagged result can use HTTP 200; incomplete 422/503 receipts return `unknown`. HTTP problems and invalid receipts raise an error. Withhold on either `withhold` or any exception. `no_finding` is not a safety guarantee, and receipts are unsigned.
+
+Requests are limited to 128 KiB after JSON encoding and responses to 16 KiB. There are no automatic retries: admitted attempts consume pilot allowance even when incomplete. Redirects are refused. Inspection ignores `LLMS_INK_API`; an explicit client base URL may select an HTTPS endpoint or localhost for development. The key follows that explicitly configured endpoint.
+
+For JSON, serialize it yourself and pass the resulting string to `inspect()`. The SDK sends `format: "text"`, so its digest covers those exact bytes; it does not promise cross-language JSON canonicalization. See [pilot limits and privacy](https://llms.ink/inspect/).
+
+JavaScript uses a 65-second total request deadline; pass `{ signal }` to cancel earlier.

package/dist/ai.d.ts

@@ -0,0 +1,111 @@
+/**
+ * llms.ink as Vercel AI SDK tools (`ai` 5 and later, with `zod`):
+ *
+ * ```ts
+ * import { generateText, stepCountIs } from 'ai';
+ * import { llmsInkTools } from 'llms-ink/ai';
+ *
+ * const { text } = await generateText({
+ *   model: 'openai/gpt-5',
+ *   tools: llmsInkTools(),
+ *   stopWhen: stepCountIs(5),
+ *   prompt: 'How do I paginate with the Stripe API? Read docs.stripe.com first.',
+ * });
+ * ```
+ *
+ * Results are cut to `maxChars` with a note saying so. API errors come back as `{ error, code, retryAfter? }`
+ * for the model to read; an invalid API key throws.
+ *
+ * @module
+ */
+import { type Tool } from 'ai';
+import { LlmsInk, type Kind, type LlmsInkOptions, type Source } from './index.ts';
+/** The tools {@link llmsInkTools} can return, by name. */
+export type LlmsInkToolName = 'llmsTxt' | 'llmsFullTxt' | 'checkLlmsTxt' | 'agentView';
+/** Options for {@link llmsInkTools}. */
+export interface LlmsInkToolsOptions<K extends LlmsInkToolName = LlmsInkToolName> {
+    /** A client, or options for a new one (`key` defaults to `LLMS_INK_KEY`). */
+    client?: LlmsInk | LlmsInkOptions;
+    /** Longest text a tool returns, in characters; longer text is cut and says so. Default 40000. */
+    maxChars?: number;
+    /** Only these tools. Default: all four. */
+    include?: readonly K[];
+}
+/** Input of `llmsTxt` and `llmsFullTxt`. */
+export interface LinkInput {
+    url: string;
+    fresh?: boolean;
+}
+/** Input of `checkLlmsTxt`: `url`, or `text` with optional `base` and `links`. */
+export interface CheckInput {
+    url?: string;
+    text?: string;
+    base?: string;
+    links?: boolean;
+}
+/** Input of `agentView`. */
+export interface AgentViewInput {
+    url: string;
+}
+/** What a tool returns instead of throwing when llms.ink answers with an error or can't be reached. */
+export interface LlmsInkToolError {
+    /** A sentence for the model, e.g. "llms.ink rate limit: wait 42 seconds, then try again." */
+    error: string;
+    /** The API's error code (`rate_limited`, `invalid_url`…); absent when llms.ink couldn't be reached. */
+    code?: string;
+    /** Seconds to wait before trying again, when llms.ink said so. */
+    retryAfter?: number;
+}
+/** `llmsTxt` / `llmsFullTxt` result. */
+export interface LlmsTextOutput {
+    url: string;
+    title: string;
+    kind: Kind;
+    source: Source;
+    /** Approximate tokens of the whole file, before any cut. */
+    tokens: number;
+    /** Links in llms.txt, or pages inlined in llms-full.txt. */
+    pages: number;
+    cached: boolean;
+    text: string;
+    /** `text` was cut to `maxChars`; it ends with a note saying so. */
+    truncated: boolean;
+}
+/** `checkLlmsTxt` result: `text` is a short report with each issue and its line. */
+export interface CheckOutput {
+    url: string | null;
+    site: string | null;
+    found: boolean;
+    summary: {
+        errors: number;
+        warnings: number;
+    };
+    text: string;
+    truncated: boolean;
+}
+/** `agentView` result: `text` lists every check with its status, evidence and fix. */
+export interface AgentViewOutput {
+    url: string;
+    site: string;
+    summary: {
+        pass: number;
+        warn: number;
+        fail: number;
+        info: number;
+    };
+    text: string;
+    truncated: boolean;
+}
+/** Every tool, by name. */
+export interface LlmsInkTools {
+    llmsTxt: Tool<LinkInput, LlmsTextOutput | LlmsInkToolError>;
+    llmsFullTxt: Tool<LinkInput, LlmsTextOutput | LlmsInkToolError>;
+    checkLlmsTxt: Tool<CheckInput, CheckOutput | LlmsInkToolError>;
+    agentView: Tool<AgentViewInput, AgentViewOutput | LlmsInkToolError>;
+}
+export declare const DEFAULT_MAX_CHARS = 40000;
+/**
+ * llms.ink tools for `generateText` / `streamText` / agents of the Vercel AI SDK: `llmsTxt`, `llmsFullTxt`,
+ * `checkLlmsTxt` and `agentView`, or the ones in `include`.
+ */
+export declare function llmsInkTools<K extends LlmsInkToolName = LlmsInkToolName>(options?: LlmsInkToolsOptions<K>): Pick<LlmsInkTools, K>;

package/dist/ai.js

@@ -0,0 +1,170 @@
+/**
+ * llms.ink as Vercel AI SDK tools (`ai` 5 and later, with `zod`):
+ *
+ * ```ts
+ * import { generateText, stepCountIs } from 'ai';
+ * import { llmsInkTools } from 'llms-ink/ai';
+ *
+ * const { text } = await generateText({
+ *   model: 'openai/gpt-5',
+ *   tools: llmsInkTools(),
+ *   stopWhen: stepCountIs(5),
+ *   prompt: 'How do I paginate with the Stripe API? Read docs.stripe.com first.',
+ * });
+ * ```
+ *
+ * Results are cut to `maxChars` with a note saying so. API errors come back as `{ error, code, retryAfter? }`
+ * for the model to read; an invalid API key throws.
+ *
+ * @module
+ */
+import { tool } from 'ai';
+import { z } from 'zod';
+import { InvalidKeyError, LlmsInk, LlmsInkError, QuotaExceededError, RateLimitedError, } from "./index.js";
+export const DEFAULT_MAX_CHARS = 40_000;
+const NAMES = ['llmsTxt', 'llmsFullTxt', 'checkLlmsTxt', 'agentView'];
+/**
+ * llms.ink tools for `generateText` / `streamText` / agents of the Vercel AI SDK: `llmsTxt`, `llmsFullTxt`,
+ * `checkLlmsTxt` and `agentView`, or the ones in `include`.
+ */
+export function llmsInkTools(options = {}) {
+    const max = options.maxChars ?? DEFAULT_MAX_CHARS;
+    if (!Number.isInteger(max) || max < 1)
+        throw new TypeError(`maxChars must be a positive integer, not ${max}`);
+    const include = options.include ?? NAMES;
+    for (const name of include) {
+        if (!NAMES.includes(name))
+            throw new TypeError(`Unknown llms.ink tool ${JSON.stringify(name)}: use ${NAMES.join(', ')}.`);
+    }
+    const given = options.client;
+    const ink = given && typeof given.llms === 'function' ? given : new LlmsInk(given);
+    const llmsTool = (mode, description, hint) => tool({
+        description,
+        inputSchema: z.object({
+            url: z.string().describe('Any public http(s) link, e.g. "docs.stripe.com/payments", "github.com/honojs/hono", a PDF or a video URL. The scheme is optional.'),
+            fresh: z.boolean().optional().describe('Ignore the 24h cache and rebuild. Default false.'),
+        }),
+        execute: (input, { abortSignal }) => attempt(async () => {
+            const r = await ink.llms(input.url, { mode, fresh: input.fresh, signal: abortSignal });
+            return { url: r.url, title: r.title, kind: r.kind, source: r.source, tokens: r.tokens, pages: r.pages, cached: r.cached, ...cut(r.text, max, hint) };
+        }),
+    });
+    const all = {
+        llmsTxt: llmsTool('llms', 'Get an llms.txt index for any link (site, docs section, page, GitHub repo, PDF or video): title, one-line summary, and sections of the most useful links with short descriptions. ' +
+            'Returns the site’s own llms.txt when it has one. Cached 24h.', 'Open only the linked pages you need.'),
+        llmsFullTxt: llmsTool('full', 'Get llms-full.txt for any link: the llms.txt index followed by the full Markdown of up to 20 pages it links to ' +
+            '(or the page/document itself; for a video, its description and timestamped transcript). Can be large — prefer llmsTxt when you only need to find pages. Cached 24h.', 'Call llmsTxt for the index, then llmsFullTxt on the one page or section you need.'),
+        checkLlmsTxt: tool({
+            description: 'Check an llms.txt against the format (H1 title, blockquote summary, H2 sections of `- [name](url): notes` links) and size. ' +
+                'Pass `url` to check the file a site publishes, including whether its first 25 links answer; or pass `text` to check a file you have (a draft, a local copy), ' +
+                'with `base` for where it lives and `links` to also test its links. Returns a short report with each issue and its line, and the sections with their link counts.',
+            inputSchema: z.object({
+                url: z.string().optional().describe('A site, a docs section, or the llms.txt itself, e.g. "llmstxt.org" or "docs.astro.build/llms.txt". Use url or text.'),
+                text: z.string().optional().describe('The llms.txt itself (up to 2 MB), instead of url.'),
+                base: z.string().optional().describe('With text: the site or llms.txt link it belongs to, e.g. "example.com". Resolves relative links.'),
+                links: z.boolean().optional().describe('With text: also check that its first 25 links answer (and llms-full.txt next to base). Default false.'),
+            }),
+            execute: (input, { abortSignal }) => attempt(async () => {
+                if ((input.text === undefined) === !input.url) {
+                    return { error: 'Pass url (a published llms.txt) or text (the file itself), not both.', code: 'invalid_request' };
+                }
+                const r = input.text === undefined
+                    ? await ink.check(input.url, { signal: abortSignal })
+                    : await ink.checkText(input.text, { base: input.base, links: input.links, signal: abortSignal });
+                return { url: r.url, site: r.site, found: r.found, summary: r.summary, ...cut(checkReport(r), max, 'Fix the issues shown, then check again.') };
+            }),
+        }),
+        agentView: tool({
+            description: 'Report how AI crawlers and agents see a page: robots.txt per AI bot (search, user fetches, training) and Content-Signal, firewall blocks found by fetching with bot User-Agents, text present without JavaScript, what survives Markdown conversion, ' +
+                'title/first paragraph/facts/noindex/canonical/sitemap, hidden instructions, and llms.txt, .md URLs, Accept: text/markdown, Link headers, ard.json. Each check is pass/warn/fail/info with a fix. ' +
+                'Use it to tell a site owner why assistants can’t read or cite a page and what to change.',
+            inputSchema: z.object({
+                url: z.string().describe('A site or any page on it, e.g. "example.com" or "docs.example.com/guide". The scheme is optional.'),
+            }),
+            execute: (input, { abortSignal }) => attempt(async () => {
+                const r = await ink.report(input.url, { signal: abortSignal });
+                return { url: r.url, site: r.site, summary: r.summary, ...cut(reportText(r), max, 'Fix the failures shown first.') };
+            }),
+        }),
+    };
+    const picked = {};
+    for (const name of include)
+        Object.assign(picked, { [name]: all[name] });
+    return picked;
+}
+/** `text` cut to `max` characters (at a line break when one is near), with a note saying what was left out. */
+function cut(text, max, hint) {
+    if (text.length <= max)
+        return { text, truncated: false };
+    let end = max;
+    const last = text.charCodeAt(end - 1);
+    if (last >= 0xd800 && last <= 0xdbff)
+        end--; // don't split a surrogate pair
+    const newline = text.lastIndexOf('\n', end);
+    if (newline > end * 0.9)
+        end = newline;
+    return {
+        text: `${text.slice(0, end)}\n\n[Truncated: showing ${end} of ${text.length} characters. ${hint}]`,
+        truncated: true,
+    };
+}
+/** Runs a tool body; llms.ink errors become a {@link LlmsInkToolError}, an invalid key and anything else throw. */
+async function attempt(run) {
+    try {
+        return await run();
+    }
+    catch (err) {
+        if (!(err instanceof LlmsInkError) || err instanceof InvalidKeyError)
+            throw err;
+        const wait = err.retryAfter;
+        const out = { error: errorText(err, wait) };
+        if (err.code)
+            out.code = err.code;
+        if (wait !== undefined)
+            out.retryAfter = wait;
+        return out;
+    }
+}
+function errorText(err, wait) {
+    if (err instanceof RateLimitedError)
+        return `llms.ink rate limit: wait ${wait ?? 60} seconds, then try again.`;
+    if (err instanceof QuotaExceededError) {
+        return `llms.ink daily quota of new links is used up${wait === undefined ? '' : `; it resets in ${wait} seconds (midnight UTC)`}. Cached links still work.`;
+    }
+    return `llms.ink: ${err.detail}`;
+}
+const plural = (count, word) => `${count} ${word}${count === 1 ? '' : 's'}`;
+/** The check as a short report (the same as the MCP tool's). */
+function checkReport(r) {
+    const lines = r.issues.map((i) => `- ${i.level}${i.line ? ` (line ${i.line})` : ''}: ${i.message}`);
+    if (!r.found)
+        return [`No llms.txt found for ${r.site ?? 'this text'}.`, '', ...lines].join('\n');
+    const facts = [
+        `${plural(r.summary.errors, 'error')}, ${plural(r.summary.warnings, 'warning')}`,
+        plural(r.sections, 'section'),
+        `${plural(r.links.total, 'link')} (${r.links.checked || !r.links.total ? `${r.links.checked} checked, ${r.links.broken.length} broken` : 'not checked'})`,
+        `~${r.tokens} tokens`,
+        ...(r.llmsFull === null ? [] : [`llms-full.txt ${r.llmsFull ? 'found' : 'not found'}`]),
+    ];
+    const outline = r.parsed?.sections.length
+        ? ['', `Sections: ${r.parsed.sections.map((s) => `${s.name} (${plural(s.links.length, 'link')})`).join(', ')}`]
+        : [];
+    const broken = r.links.broken.map((b) => `- broken link (${b.status || 'no answer'}): ${b.url}`);
+    return [`${r.url ?? 'llms.txt'}: ${facts.join(' · ')}`, ...outline, '', ...lines, ...broken].join('\n');
+}
+/** The agent's-eye report as text (the same as the MCP tool's). */
+function reportText(r) {
+    const mark = { pass: '✓', warn: '!', fail: '✕', info: '·' };
+    const lines = [`${r.url}: ${r.summary.fail} fail, ${r.summary.warn} warn, ${r.summary.pass} pass, ${r.summary.info} info`];
+    for (const s of r.sections) {
+        lines.push('', `## ${s.title}`);
+        for (const c of s.checks) {
+            lines.push(`${mark[c.status]} ${c.status} · ${c.title}: ${c.detail}`);
+            if (c.evidence && c.status !== 'pass')
+                lines.push(...c.evidence.split('\n').slice(0, 8).map((l) => `    ${l}`));
+            if (c.fix && c.status !== 'pass')
+                lines.push(`  fix (${c.fix.kind}): ${c.fix.summary}${c.fix.patch ? ` [patch for ${c.fix.patch.file}]` : ''}`);
+        }
+    }
+    return lines.join('\n');
+}

package/dist/index.d.ts

@@ -13,7 +13,9 @@
  *
  * @module
  */
-export declare const VERSION = "0.4.0";
+import { type InspectionReceipt } from './inspect.ts';
+export type { InspectionReceipt } from './inspect.ts';
+export declare const VERSION = "0.6.0";
 export declare const DEFAULT_BASE_URL = "https://llms.ink";
 /** `llms` for the index (llms.txt), `full` for llms-full.txt. */
 export type Mode = 'llms' | 'full';
@@ -136,8 +138,55 @@ export interface CheckResult {
     /** ISO 8601 date-time. */
     checkedAt: string;
 }
+/** `pass` is fine; `warn` is worth fixing; `fail` keeps agents out or empties the page for them; `info` is a fact. */
+export type ReportStatus = 'pass' | 'warn' | 'fail' | 'info';
+/** How to fix a check that didn't pass. */
+export interface ReportFix {
+    /** Where the change goes. */
+    kind: 'robots' | 'markdown' | 'llms-txt' | 'indexnow' | 'content' | 'html' | 'headers' | 'other';
+    /** What to change, in a sentence. */
+    summary: string;
+    /** A concrete change, when one is possible: `file` from the site root and its new content or lines to add. */
+    patch?: {
+        file: string;
+        content: string;
+    };
+}
+/** One check of the agent's-eye report. */
+export interface ReportCheck {
+    /** Stable name of the check, e.g. `bots-search`, `raw-text`. */
+    id: string;
+    status: ReportStatus;
+    title: string;
+    /** What was found, for people. */
+    detail: string;
+    /** What it's based on: lines, headers, snippets, sizes. */
+    evidence?: string;
+    fix?: ReportFix;
+}
+/** `GET /api/v0/report`: how AI crawlers and agents see a page, each problem with its fix. */
+export interface ReportResult {
+    /** The page reported on, after normalization and redirects. */
+    url: string;
+    /** Its origin. */
+    site: string;
+    /** ISO 8601 date-time. */
+    checkedAt: string;
+    summary: {
+        pass: number;
+        warn: number;
+        fail: number;
+        info: number;
+    };
+    /** In order: access, rendering, extraction, retrieval, hidden instructions, agent files. */
+    sections: Array<{
+        id: string;
+        title: string;
+        checks: ReportCheck[];
+    }>;
+}
 /** Every error code the API answers with (https://llms.ink/docs/#errors). */
-export type ErrorCode = 'invalid_request' | 'invalid_url' | 'invalid_key' | 'key_required' | 'site_claimed' | 'not_found' | 'unknown_version' | 'method_not_allowed' | 'payload_too_large' | 'too_large' | 'unsupported_media_type' | 'unprocessable' | 'rate_limited' | 'quota_exceeded' | 'internal' | 'upstream_error' | 'unavailable' | 'timeout' | 'loop_detected';
+export type ErrorCode = 'inspection_not_enabled' | 'inspection_quota' | 'inspection_unavailable' | 'invalid_request' | 'invalid_url' | 'invalid_key' | 'key_required' | 'site_claimed' | 'not_found' | 'unknown_version' | 'method_not_allowed' | 'payload_too_large' | 'too_large' | 'unsupported_media_type' | 'unprocessable' | 'rate_limited' | 'quota_exceeded' | 'internal' | 'upstream_error' | 'unavailable' | 'timeout' | 'loop_detected';
 /** `invalid_request` only: one invalid parameter. */
 export interface Issue {
     path: string;
@@ -236,6 +285,10 @@ export interface LlmsOptions {
 export interface CheckOptions {
     signal?: AbortSignal;
 }
+/** Options for {@link LlmsInk.report}. */
+export interface ReportOptions {
+    signal?: AbortSignal;
+}
 /** Options for {@link LlmsInk.checkText}. */
 export interface CheckTextOptions {
     /** Public URL the text lives at: resolves relative links and is reported as `url` / `site`. */
@@ -268,8 +321,22 @@ export declare class LlmsInk {
      * any public link.
      */
     llms(url: string, options?: LlmsOptions): Promise<Result>;
+    /**
+     * Explicitly upload supplied text to the hosted Inspect pilot. Requires an enabled key.
+     * The receipt binds to these exact UTF-8 bytes; no finding is not a safety guarantee.
+     * No retries. 65-second default deadline. LLMS_INK_API is ignored; baseUrl must be explicit.
+     * Keep keys server-side. Malformed receipts and HTTP problems throw LlmsInkError.
+     */
+    inspect(text: string, options?: {
+        signal?: AbortSignal;
+    }): Promise<InspectionReceipt>;
     /** Checks a site's published llms.txt against the llmstxt.org format and that its links answer. */
     check(url: string, options?: CheckOptions): Promise<CheckResult>;
+    /**
+     * How AI crawlers and agents see a page (`GET /api/v0/report`): robots.txt per AI bot, firewall blocks, text
+     * without JavaScript, Markdown conversion, hidden instructions and agent files, each problem with its fix.
+     */
+    report(url: string, options?: ReportOptions): Promise<ReportResult>;
     /** Parses and checks llms.txt text you already have (`POST /api/v0/check`); nothing is published. */
     checkText(text: string, options?: CheckTextOptions): Promise<CheckResult>;
     /**

package/dist/index.js

@@ -13,8 +13,9 @@
  *
  * @module
  */
+import { prepareInspection, inspectionJson, inspectionReceipt } from "./inspect.js";
 import { env } from "./env.js";
-export const VERSION = '0.4.0';
+export const VERSION = '0.6.0';
 export const DEFAULT_BASE_URL = 'https://llms.ink';
 /**
  * An API error, or llms.ink couldn't be reached (then `status` and `code` are undefined).
@@ -83,10 +84,12 @@ export class LlmsInk {
     key;
     /** Origin of the API, without a trailing slash. */
     baseUrl;
+    #inspectBaseUrl;
     #fetch;
     constructor(options = {}) {
         const key = (options.key ?? env('LLMS_INK_KEY'))?.trim();
         this.key = key || undefined;
+        this.#inspectBaseUrl = (options.baseUrl ?? DEFAULT_BASE_URL).replace(/\/+$/, '');
         this.baseUrl = (options.baseUrl ?? env('LLMS_INK_API') ?? DEFAULT_BASE_URL).replace(/\/+$/, '');
         if (!options.fetch && typeof globalThis.fetch !== 'function') {
             throw new TypeError('No global fetch: use Node 18+ or pass options.fetch.');
@@ -102,11 +105,57 @@ export class LlmsInk {
         const res = await this.#request('/api/v0/llms', llmsParams(url, options), 'application/json', options.signal);
         return (await this.#json(res));
     }
+    /**
+     * Explicitly upload supplied text to the hosted Inspect pilot. Requires an enabled key.
+     * The receipt binds to these exact UTF-8 bytes; no finding is not a safety guarantee.
+     * No retries. 65-second default deadline. LLMS_INK_API is ignored; baseUrl must be explicit.
+     * Keep keys server-side. Malformed receipts and HTTP problems throw LlmsInkError.
+     */
+    async inspect(text, options = {}) {
+        if (!this.key)
+            throw new LlmsInkError('Inspect requires an inspection-enabled API key.', { code: 'key_required' });
+        const { body, digest } = await prepareInspection(text, this.#inspectBaseUrl);
+        const controller = new AbortController();
+        const abort = () => controller.abort(options.signal?.reason);
+        const timer = setTimeout(() => controller.abort(new DOMException('Inspection timed out.', 'TimeoutError')), 65000);
+        if (options.signal?.aborted)
+            abort();
+        options.signal?.addEventListener('abort', abort, { once: true });
+        try {
+            const res = await this.#fetch(`${this.#inspectBaseUrl}/api/v0/inspect`, {
+                method: 'POST', redirect: 'error', signal: controller.signal,
+                headers: { authorization: `Bearer ${this.key}`, 'content-type': 'application/json', accept: 'application/json' }, body,
+            });
+            const data = await inspectionJson(res);
+            if (!res.ok && data && typeof data === 'object' && !('detection' in data)) {
+                const p = data;
+                throw errorFor(p.detail || p.title || `Inspect returned HTTP ${res.status}.`, { status: res.status, code: p.code, retryAfter: retryAfter(res.headers.get('retry-after')) });
+            }
+            return inspectionReceipt(data, digest, res.status);
+        }
+        catch (err) {
+            if (err instanceof LlmsInkError || isAbort(err))
+                throw err;
+            throw new LlmsInkError('Inspection did not return a valid receipt. Withhold the content.', { code: 'inspection_unavailable' });
+        }
+        finally {
+            clearTimeout(timer);
+            options.signal?.removeEventListener('abort', abort);
+        }
+    }
     /** Checks a site's published llms.txt against the llmstxt.org format and that its links answer. */
     async check(url, options = {}) {
         const res = await this.#request('/api/v0/check', { url }, 'application/json', options.signal);
         return (await this.#json(res));
     }
+    /**
+     * How AI crawlers and agents see a page (`GET /api/v0/report`): robots.txt per AI bot, firewall blocks, text
+     * without JavaScript, Markdown conversion, hidden instructions and agent files, each problem with its fix.
+     */
+    async report(url, options = {}) {
+        const res = await this.#request('/api/v0/report', { url }, 'application/json', options.signal);
+        return (await this.#json(res));
+    }
     /** Parses and checks llms.txt text you already have (`POST /api/v0/check`); nothing is published. */
     async checkText(text, options = {}) {
         const params = {};

package/dist/inspect.d.ts

@@ -0,0 +1,21 @@
+/** Hosted Inspect pilot. Only explicit inspect() calls upload content. */
+export interface InspectionReceipt {
+    readonly detection: 'no_finding' | 'flagged' | 'unknown';
+    readonly coverage: 'complete' | 'partial' | 'none';
+    readonly recommendation: 'continue' | 'withhold';
+    readonly reason: string;
+    readonly modelRevision: string;
+    readonly policyRevision: string;
+    readonly preprocessingRevision: string;
+    readonly contentDigest: string;
+    readonly inputTokens: number;
+    readonly inspectedTokens: number;
+    readonly windows: number;
+    readonly enforced: false;
+}
+export declare function prepareInspection(text: string, baseUrl: string): Promise<{
+    body: string;
+    digest: string;
+}>;
+export declare function inspectionJson(res: Response): Promise<unknown>;
+export declare function inspectionReceipt(value: unknown, digest: string, status: number): InspectionReceipt;

package/dist/inspect.js

@@ -0,0 +1,67 @@
+export async function prepareInspection(text, baseUrl) {
+    const url = new URL(baseUrl);
+    if (url.username || url.password || url.search || url.hash || (url.protocol !== 'https:' && !(url.protocol === 'http:' && ['localhost', '127.0.0.1', '[::1]'].includes(url.hostname))))
+        throw new TypeError('Inspect requires HTTPS (or localhost for development).');
+    if (typeof text !== 'string' || !text.length)
+        throw new TypeError('Inspect requires nonempty text.');
+    const bytes = new TextEncoder().encode(text);
+    if (new TextDecoder('utf-8', { ignoreBOM: true }).decode(bytes) !== text)
+        throw new TypeError('Inspect requires valid Unicode.');
+    const body = JSON.stringify({ format: 'text', content: text });
+    if (new TextEncoder().encode(body).length > 131072)
+        throw new TypeError('Inspect request exceeds 128 KiB after JSON encoding.');
+    const crypto = globalThis.crypto ?? (await import('node:crypto')).webcrypto;
+    const digest = [...new Uint8Array(await crypto.subtle.digest('SHA-256', bytes))].map(b => b.toString(16).padStart(2, '0')).join('');
+    return { body, digest };
+}
+export async function inspectionJson(res) {
+    const reader = res.body?.getReader();
+    if (!reader)
+        throw new Error('Inspect returned an empty response.');
+    const chunks = [];
+    let size = 0;
+    try {
+        while (true) {
+            const { value, done } = await reader.read();
+            if (done)
+                break;
+            size += value.length;
+            if (size > 16384) {
+                await reader.cancel();
+                throw new Error('Inspect response exceeds 16 KiB.');
+            }
+            chunks.push(value);
+        }
+    }
+    finally {
+        reader.releaseLock();
+    }
+    const bytes = new Uint8Array(size);
+    let offset = 0;
+    for (const chunk of chunks) {
+        bytes.set(chunk, offset);
+        offset += chunk.length;
+    }
+    return JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(bytes));
+}
+export function inspectionReceipt(value, digest, status) {
+    const fail = () => { throw new Error('Inspect returned an invalid or unsupported receipt. Withhold the content.'); };
+    if (!value || typeof value !== 'object')
+        return fail();
+    const r = value;
+    if (![200, 422, 503].includes(status) || (status !== 200 && r.detection !== 'unknown') || r.contentDigest !== digest || r.modelRevision !== '4a43fec65a670e48835d8e8c85c705b4b0d1c907' || r.policyRevision !== 'llms-ink-inspect-1' || r.preprocessingRevision !== 'utf8-json-stringify-1' || r.enforced !== false)
+        return fail();
+    if (!Number.isSafeInteger(r.inputTokens) || r.inputTokens < 0 || !Number.isSafeInteger(r.inspectedTokens) || r.inspectedTokens < 0 || r.inspectedTokens > r.inputTokens || !Number.isInteger(r.windows) || r.windows < 0 || r.windows > 43)
+        return fail();
+    let valid = false;
+    if (r.detection === 'no_finding')
+        valid = r.coverage === 'complete' && r.recommendation === 'continue' && r.reason === 'no_finding' && r.inputTokens <= 8192 && r.inspectedTokens === r.inputTokens && (r.inputTokens === 0 || r.windows > 0);
+    if (r.detection === 'flagged')
+        valid = ['complete', 'partial'].includes(r.coverage) && r.recommendation === 'withhold' && ['instruction_pattern', 'model_detection'].includes(r.reason) && r.inputTokens <= 8192 && r.inspectedTokens > 0 && r.windows > 0 && (r.coverage !== 'complete' || r.inspectedTokens === r.inputTokens);
+    if (r.detection === 'unknown')
+        valid = r.coverage === 'none' && r.recommendation === 'withhold' && r.inspectedTokens === 0 && r.windows === 0 && ['token_limit', 'model_unavailable', 'model_failure', 'inspection_timeout'].includes(r.reason);
+    if (!valid)
+        return fail();
+    const { detection, coverage, recommendation, reason, modelRevision, policyRevision, preprocessingRevision, contentDigest, inputTokens, inspectedTokens, windows, enforced } = r;
+    return Object.freeze({ detection, coverage, recommendation, reason, modelRevision, policyRevision, preprocessingRevision, contentDigest, inputTokens, inspectedTokens, windows, enforced });
+}

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "llms-ink",
-  "version": "0.4.0",
-  "description": "Turn any link — site, docs section, page, GitHub repo, PDF or video — into llms.txt and llms-full.txt. Typed client for llms.ink, plus a Vite plugin, an Astro integration and middleware that serve the files from your site. No dependencies.",
+  "version": "0.6.0",
+  "description": "Turn any link — site, docs section, page, GitHub repo, PDF or video — into llms.txt and llms-full.txt. Typed client for llms.ink, plus a Vite plugin, an Astro integration and middleware that serve the files from your site, and Vercel AI SDK tools. No dependencies.",
   "type": "module",
   "exports": {
     ".": {
@@ -19,6 +19,10 @@
     "./middleware": {
       "types": "./dist/middleware.d.ts",
       "default": "./dist/middleware.js"
+    },
+    "./ai": {
+      "types": "./dist/ai.d.ts",
+      "default": "./dist/ai.js"
     }
   },
   "main": "./dist/index.js",
@@ -31,27 +35,33 @@
   "scripts": {
     "build": "rm -rf dist && tsc -p tsconfig.json",
     "prepublishOnly": "npm run build && npm test",
-    "test": "npm run build && node --test test/client.test.mjs test/plugins.test.mjs test/middleware.test.mjs test/traffic.test.mjs",
+    "test": "npm run build && tsc -p test/tsconfig.json && node --test test/client.test.mjs test/plugins.test.mjs test/middleware.test.mjs test/traffic.test.mjs test/inspect.test.mjs test/ai.test.mjs",
     "test:live": "npm run build && node --test test/live.test.mjs",
-    "test:bun": "npm run build && bun test ./test/client.test.mjs ./test/plugins.test.mjs ./test/middleware.test.mjs ./test/traffic.test.mjs",
+    "test:bun": "npm run build && bun test ./test/client.test.mjs ./test/plugins.test.mjs ./test/middleware.test.mjs ./test/traffic.test.mjs ./test/inspect.test.mjs ./test/ai.test.mjs",
     "test:bun:live": "npm run build && bun test --timeout 300000 ./test/live.test.mjs"
   },
-  "keywords": ["llms.txt", "llms-full.txt", "llm", "ai", "markdown", "docs", "context", "rag", "sdk", "llms-ink", "vite-plugin", "astro-integration", "middleware"],
+  "keywords": ["llms.txt", "llms-full.txt", "llm", "ai", "markdown", "docs", "context", "rag", "sdk", "llms-ink", "vite-plugin", "astro-integration", "middleware", "ai-sdk", "vercel-ai-sdk", "tools"],
   "homepage": "https://llms.ink/docs/",
   "author": "Lucas Espindola (https://lucasespindola.com)",
   "license": "MIT",
   "peerDependencies": {
+    "ai": ">=5",
     "astro": ">=4",
-    "vite": ">=5"
+    "vite": ">=5",
+    "zod": "^3.25.76 || ^4.1.8"
   },
   "peerDependenciesMeta": {
+    "ai": { "optional": true },
     "astro": { "optional": true },
-    "vite": { "optional": true }
+    "vite": { "optional": true },
+    "zod": { "optional": true }
   },
   "devDependencies": {
     "@types/node": "^22.20.5",
-    "astro": "^7.4.1",
+    "ai": "^7.0.164",
+    "astro": "^7.4.7",
     "typescript": "^5.9.3",
-    "vite": "^8.3.8"
+    "vite": "^8.3.8",
+    "zod": "^4.6.7"
   }
 }