@andesine/pages 0.0.0-stage → 0.4.0

package/CHANGELOG.md

@@ -0,0 +1,99 @@
+# Changelog
+
+## 0.4.0
+
+First release. The version follows the other Andesine packages.
+
+- Add `defineConfig()` with config validation.
+- Add the `andesine()` Astro integration. It reads `andesine.config.ts`, sets the
+  Astro site options, and renders pages with the template layout.
+- Add the files source: Markdown and MDX pages from a directory, with folder
+  groups, index landing pages, frontmatter validation, and slugs.
+- Add sections, source mounts, URL collision checks, and redirects to the first
+  page.
+- Add `PageContext` with navigation, breadcrumbs, headings, and previous and next
+  links.
+- Add MDX content components: a Sätteri plugin prepares `Tabs`, `CodeGroup`, and
+  `FileTree`, and a Shiki transformer adds code block titles. Pages render with the
+  template's component map.
+- Add Andesine sources: build-time loading with copied images, entry property
+  mapping, collection navigation, and content rendering with the MDX component map
+  and Andesine heading anchors.
+- Add search: a Pagefind index of local pages (build and development), a server
+  endpoint for Andesine published search, and a browser client with results
+  grouped by source.
+- Add AI answers: a streaming server endpoint for Andesine sources, with cited
+  pages mapped to page URLs and follow-up history, the `answers` source option,
+  a browser answer client, and `renderAnswer` for safe answer Markdown.
+- Add outputs: Markdown alternatives at `index.md`, `sitemap.xml`, `llms.txt`,
+  `robots.txt`, and a not-found page.
+- Add canonical URLs, Markdown URLs, and page actions to `PageContext`, and the
+  `pageActions` option.
+- Add the `summary` and `aside` slots for Andesine entry fragments.
+- Remove the `slug`, `navigationLabel`, and `navigationHidden` Andesine properties.
+  URLs and labels come from the publication tree, and an `index` entry is its
+  collection's landing page.
+- Support Cloudflare Workers for build-time Andesine sources: no social renderer
+  externals with the Cloudflare adapter, and `@iconify/json` loads on first use.
+- Read the API key with `getSecret()` from `astro:env/server`, so each adapter
+  supplies it from its platform. Hashing uses Web Crypto, fonts decode with
+  `atob`, and the build-time image cache is a separate module, so server routes
+  do not import Node.js modules.
+- Add the OpenAPI source (`type: "openapi"`): OpenAPI 3.0 and 3.1 specs from a
+  file or an HTTPS URL, validated and normalized at build time, with overview,
+  tag, and operation pages, navigation, Markdown alternatives, and search.
+- Add `OperationView` for the template's `Operation` component: descriptions,
+  parameters, request bodies, responses with field trees, and generated cURL,
+  JavaScript, and Python examples, which `x-codeSamples` can replace. Add
+  `method` to navigation items.
+- Add operation embeds: `<Operation source id />` in MDX and the `Operation`
+  element in Andesine content, resolved with `getOperationView()` from
+  `@andesine/pages/astro`. Markdown alternatives link to the operations. Boolean
+  props show or hide the description, authentication, parameters, request body,
+  responses, and examples; embeds hide the examples by default.
+- Add OpenAPI operation pages to the development search index.
+- Add the `tagPages` option of OpenAPI sources, to leave out tag landing pages.
+- Add `links` to sections, shown after the site's links on the section's pages, and
+  `PageContext.links` with the links of the current page.
+- Replace the git commit dates of file pages with an optional `updatedAt`
+  frontmatter field. Pages without it have no date.
+- Add agent outputs: `llms-full.txt`, `/.well-known/llms.txt`, `.md` page aliases,
+  an index note in each Markdown page, the `agents.instructions` option,
+  absolute links in file page Markdown, an "Optional" section in `llms.txt`, and
+  `lastmod` in the sitemap.
+- Serve Markdown for requests with `Accept: text/markdown` in development.
+- Add `<Visibility for="agents">` and `<Visibility for="humans">` for MDX files
+  and Andesine content.
+- Add schema.org JSON-LD, the last update date, and `noindex` for pages hidden
+  from search to `PageContext`.
+- Add social images: a generated image for each page from the template's social
+  card, the `social` option, and `page.image`. `satori` and `@resvg/resvg-js`
+  are optional peer dependencies.
+- The integration no longer sets `trailingSlash: "always"`, so pages answer with
+  and without the trailing slash. Package URLs still end with a slash.
+- The answers endpoint allows 10 questions a minute for each client address.
+- The integration inlines a script into each page's `<head>` that restores saved
+  navigation tree groups before the first paint.
+- Add the `socialLinks` option for footer icon links. `links` are icon links at the
+  top of the navigation, and need an `icon`.
+- Add `PageContext.meta`, `PageContext.fragments`, and `site.storageKey`;
+  `renderTabs` and `readFileTreeItems` in `@andesine/pages/astro` (replacing
+  `renderPanels`); and `groupAnswerSources`. The answer client takes earlier
+  turns and enforces the API history limits. Social card fonts accept data URLs, and
+  `SocialCardData.showSiteName` follows `logo.title`.
+- Add the `html` template tag for social card markup: it escapes values and
+  leaves out `false`, `null`, and `undefined`. Card markup can use `tw` classes.
+- Add `format` (`icon` or `full`) and `alt` to the `logo` option, also in social
+  cards.
+- Add the `sectionsDisplay` option: sections as header tabs, or as links at the
+  top of the navigation.
+- The development search index uses the plain text of MDX pages, without JSX tags and
+  Markdown syntax, like the built index.
+- Add a generated `skill.md` with agentskills.io and `skills` CLI discovery
+  indexes, and the `agents.skill` option for your own skill file.
+- Add the `publicKey` option to Andesine sources. With a publishable key, search and
+  AI answers call the Andesine API from browsers, and the build writes a JSON map of
+  entry URLs in place of the `/_andesine/` server routes, so the site stays static.
+  Without a secret key, the build also reads the content with the publishable key.
+- Search and AI answers show "Too many…" only for short rate limits. Used-up monthly
+  quotas and spending limits show that the feature is not available now.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Vrite, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,93 @@
-# Temporary Holding Version
+# @andesine/pages
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The Astro integration and data layer of Andesine Pages. Use it with the Andesine
+Pages template, which supplies the layouts, styles, and components.
+
+> **0.x.** The config and the page context can change in minor releases; the
+> changelog lists the changes. Use the version that the template pins.
+
+## Exports
+
+| Export                        | Contents                                 |
+| ----------------------------- | ---------------------------------------- |
+| `@andesine/pages`             | `defineConfig()`, config and page types. |
+| `@andesine/pages/integration` | The `andesine()` Astro integration.      |
+| `@andesine/pages/collections` | Content collections for the sources.     |
+| `@andesine/pages/astro`       | Helpers for the template's components.   |
+| `@andesine/pages/client`      | The browser search client.               |
+
+## Setup
+
+`andesine.config.ts`:
+
+```ts
+import { defineConfig } from "@andesine/pages";
+
+export default defineConfig({
+  name: "Example",
+  site: "https://docs.example.com"
+});
+```
+
+`astro.config.ts`:
+
+```ts
+import { defineConfig } from "astro/config";
+import { andesine } from "@andesine/pages/integration";
+
+export default defineConfig({ integrations: [andesine()] });
+```
+
+`src/content.config.ts`:
+
+```ts
+export { collections } from "@andesine/pages/collections";
+```
+
+The integration reads `andesine.config.ts` and sets the Astro `site`, `base`, and
+`trailingSlash` options. It adds one static route for the pages of all files
+sources. Each page renders with `src/layouts/page.astro`. The layout receives one
+`page` prop with the type `PageContext`, and the page body in its default slot.
+
+A source mount or the site root without a page redirects to the first page. The
+build stops when two pages use the same URL.
+
+| Option       | Default                           | Description                              |
+| ------------ | --------------------------------- | ---------------------------------------- |
+| `config`     | `andesine.config.ts`              | Config file, relative to the root.       |
+| `layout`     | `src/layouts/page.astro`          | Page layout, relative to the root.       |
+| `components` | `src/components/content/index.ts` | MDX component map, relative to the root. |
+
+Andesine sources read the latest publication with `@andesine/sdk` at build time.
+They load in a content collection and copy their images into the build.
+Content converts with `@andesine/converters`, so heading IDs match Andesine search
+and answers, and renders with the same component map as MDX.
+
+OpenAPI sources read an OpenAPI 3.0 or 3.1 spec at build time, from a file or an
+HTTPS URL, and make overview, tag, and operation pages. The template's `Operation`
+component renders operations with `getOperationView()` from `@andesine/pages/astro`,
+on reference pages and in embeds (`<Operation source="api" id="..." />`).
+
+## Search
+
+Local pages are in a Pagefind index. The build writes it to `pagefind/`, and the dev
+server builds it in memory from the source text. Andesine sources use the published
+search API. With a `publicKey` (a publishable `adn_pk_` key), the build can read
+the content without a secret key, browsers call the API
+directly, and map results to pages with `/_andesine/pages/<source>.json`, which the
+build writes. Without one, they use `/_andesine/search/`, a server route that keeps
+the API key on the server, so the site needs a server adapter. Results
+of build-time sources link only to pages in the build; results of pages that were
+published after the build are left out. Pages with `searchHidden` are left out of
+results. This is not access control.
+
+`createSearchClient(page.site.search)` from `@andesine/pages/client` searches all
+sources and returns results grouped by source.
+
+The component map module exports `components`. MDX pages can use these components
+without imports. The integration adds a plugin to the Sätteri Markdown processor
+that prepares `<Tabs>`, `<CodeGroup>`, and `<FileTree>` content, and a Shiki
+transformer that reads `title="..."` from code fences.
+
+Relative `logo.src` and `favicon` paths in the config are relative to the config
+file. The integration bundles them as assets.

package/dist/answers.d.ts

@@ -0,0 +1,6 @@
+import { APIRoute } from "astro";
+//#region src/astro/routes/answers.d.ts
+export declare const prerender = false;
+export declare const POST: APIRoute;
+//#endregion
+//# sourceMappingURL=answers.d.ts.map

package/dist/astro/content.astro

@@ -0,0 +1,118 @@
+---
+import type { Element, RootContent } from "hast";
+import { toHtml } from "hast-util-to-html";
+import { isElement, readElement, toFileTreeItems } from "../runtime.js";
+
+interface Props {
+  nodes: RootContent[];
+  /** The template's content components, keyed by element name, and `pre` for code. */
+  components: Record<string, unknown>;
+}
+
+const { nodes, components } = Astro.props;
+const marker = "@@andesine-children@@";
+// Subtrees without custom elements or code blocks render as one HTML string.
+const isPlain = (node: RootContent): boolean =>
+  !isElement(node) ||
+  (!node.properties.dataAndesineElement && node.tagName !== "pre" && node.children.every(isPlain));
+const getComponent = (name: string): any => {
+  if (!components[name]) throw new Error(`The template has no component for <${name}>.`);
+
+  return components[name];
+};
+const getLabels = (items: Element[], label: (item: Element) => string): string[] => {
+  const labels = items.map(label);
+
+  if (new Set(labels).size !== labels.length) {
+    throw new Error(`Tabs need a unique label for each tab: ${labels.join(", ")}.`);
+  }
+
+  return labels;
+};
+// Andesine code blocks have no titles, so a repeated language gets a number, e.g. `bash 2`.
+const getCodeLabels = (items: Element[]): string[] => {
+  const counts = new Map<string, number>();
+
+  return items.map((pre) => {
+    const language = String(pre.properties.dataLanguage || "Code");
+    const count = (counts.get(language) ?? 0) + 1;
+
+    counts.set(language, count);
+
+    return count > 1 ? `${language} ${count}` : language;
+  });
+};
+---
+
+{
+  nodes.map((node) => {
+    if (isPlain(node)) return <Fragment set:html={toHtml(node)} />;
+
+    const element = node as Element;
+    const custom = readElement(element);
+    const children = element.children.filter(isElement);
+
+    if (custom?.name === "Tabs") {
+      const Component = getComponent("Tabs");
+      const labels = getLabels(children, (tab) => String(readElement(tab)?.props.label));
+
+      return (
+        <Component values={JSON.stringify(labels)} syncKey={custom.props.syncKey}>
+          {(index: number) => (
+            <Astro.self nodes={children[index]!.children} components={components} />
+          )}
+        </Component>
+      );
+    }
+
+    if (custom?.name === "CodeGroup") {
+      const Component = getComponent("CodeGroup");
+      const labels = getCodeLabels(children);
+
+      return (
+        <Component values={JSON.stringify(labels)}>
+          {(index: number) => <Astro.self nodes={[children[index]!]} components={components} />}
+        </Component>
+      );
+    }
+
+    if (custom?.name === "FileTree") {
+      const Component = getComponent("FileTree");
+
+      return <Component items={JSON.stringify(toFileTreeItems(children[0]!))} />;
+    }
+
+    if (custom) {
+      const Component = getComponent(custom.name);
+
+      return (
+        <Component {...custom.props}>
+          <Astro.self nodes={element.children} components={components} />
+        </Component>
+      );
+    }
+
+    if (element.tagName === "pre" && components.pre) {
+      const Code = getComponent("pre");
+
+      return (
+        <Code data-language={element.properties.dataLanguage}>
+          <Fragment set:html={toHtml(element.children)} />
+        </Code>
+      );
+    }
+
+    // Keeps the element, and renders its children with this component.
+    const [open, close] = toHtml({ ...element, children: [{ type: "text", value: marker }] }).split(
+      marker
+    );
+
+    return (
+      <>
+        <Fragment set:html={open} />
+        <Astro.self nodes={element.children} components={components} />
+        <Fragment set:html={close} />
+      </>
+    );
+  })
+}

package/dist/astro/middleware.js

@@ -0,0 +1,39 @@
+import { defineMiddleware } from "astro:middleware";
+//#region src/astro/middleware.ts
+/** The highest quality of the given media types in an `Accept` header. */
+const getQuality = (accept, types) => {
+	return Math.max(0, ...accept.split(",").flatMap((range) => {
+		const [type = "", ...parameters] = range.split(";").map((part) => part.trim());
+		const quality = parameters.find((parameter) => parameter.startsWith("q="));
+		return types.includes(type.toLowerCase()) ? [quality ? Number(quality.slice(2)) || 0 : 1] : [];
+	}));
+};
+/** Agents ask for Markdown with `text/markdown`, or with `text/plain` instead of HTML. */
+const prefersMarkdown = (accept) => {
+	const markdown = getQuality(accept, [
+		"text/markdown",
+		"text/x-markdown",
+		"text/plain"
+	]);
+	return markdown > 0 && markdown > getQuality(accept, ["text/html", "application/xhtml+xml"]);
+};
+const withVary = (response) => {
+	const result = new Response(response.body, response);
+	result.headers.append("Vary", "Accept");
+	return result;
+};
+/**
+* Serves a page's Markdown to requests that prefer it. Hosts serve built pages as files, so this
+* works in development only; a host can add the same rule for `Accept: text/markdown`.
+*/
+const onRequest = defineMiddleware(async (context, next) => {
+	const { pathname } = context.url;
+	const isPage = !/\.[^/]+$/.test(pathname);
+	if (context.isPrerendered || !isPage) return next();
+	const accept = context.request.headers.get("accept") ?? "";
+	return withVary(prefersMarkdown(accept) ? await context.rewrite(`${pathname.replace(/\/?$/, "/")}index.md`) : await next());
+});
+//#endregion
+export { onRequest };
+
+//# sourceMappingURL=middleware.js.map

package/dist/astro/middleware.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"middleware.js","names":[],"sources":["../../src/astro/middleware.ts"],"sourcesContent":["import { defineMiddleware } from \"astro:middleware\";\n\n/** The highest quality of the given media types in an `Accept` header. */\nconst getQuality = (accept: string, types: string[]): number => {\n  return Math.max(\n    0,\n    ...accept.split(\",\").flatMap((range) => {\n      const [type = \"\", ...parameters] = range.split(\";\").map((part) => part.trim());\n      const quality = parameters.find((parameter) => parameter.startsWith(\"q=\"));\n\n      return types.includes(type.toLowerCase())\n        ? [quality ? Number(quality.slice(2)) || 0 : 1]\n        : [];\n    })\n  );\n};\n/** Agents ask for Markdown with `text/markdown`, or with `text/plain` instead of HTML. */\nconst prefersMarkdown = (accept: string): boolean => {\n  const markdown = getQuality(accept, [\"text/markdown\", \"text/x-markdown\", \"text/plain\"]);\n\n  return markdown > 0 && markdown > getQuality(accept, [\"text/html\", \"application/xhtml+xml\"]);\n};\nconst withVary = (response: Response): Response => {\n  const result = new Response(response.body, response);\n\n  result.headers.append(\"Vary\", \"Accept\");\n\n  return result;\n};\n\n/**\n * Serves a page's Markdown to requests that prefer it. Hosts serve built pages as files, so this\n * works in development only; a host can add the same rule for `Accept: text/markdown`.\n */\nexport const onRequest = defineMiddleware(async (context, next) => {\n  const { pathname } = context.url;\n  // Page URLs have no file extension; they can end with or without a slash.\n  const isPage = !/\\.[^/]+$/.test(pathname);\n\n  if (context.isPrerendered || !isPage) return next();\n\n  const accept = context.request.headers.get(\"accept\") ?? \"\";\n\n  return withVary(\n    prefersMarkdown(accept)\n      ? await context.rewrite(`${pathname.replace(/\\/?$/, \"/\")}index.md`)\n      : await next()\n  );\n});\n"],"mappings":";;;AAGA,MAAM,cAAc,QAAgB,UAA4B;CAC9D,OAAO,KAAK,IACV,GACA,GAAG,OAAO,MAAM,GAAG,CAAC,CAAC,SAAS,UAAU;EACtC,MAAM,CAAC,OAAO,IAAI,GAAG,cAAc,MAAM,MAAM,GAAG,CAAC,CAAC,KAAK,SAAS,KAAK,KAAK,CAAC;EAC7E,MAAM,UAAU,WAAW,MAAM,cAAc,UAAU,WAAW,IAAI,CAAC;EAEzE,OAAO,MAAM,SAAS,KAAK,YAAY,CAAC,IACpC,CAAC,UAAU,OAAO,QAAQ,MAAM,CAAC,CAAC,KAAK,IAAI,CAAC,IAC5C,CAAC;CACP,CAAC,CACH;AACF;;AAEA,MAAM,mBAAmB,WAA4B;CACnD,MAAM,WAAW,WAAW,QAAQ;EAAC;EAAiB;EAAmB;CAAY,CAAC;CAEtF,OAAO,WAAW,KAAK,WAAW,WAAW,QAAQ,CAAC,aAAa,uBAAuB,CAAC;AAC7F;AACA,MAAM,YAAY,aAAiC;CACjD,MAAM,SAAS,IAAI,SAAS,SAAS,MAAM,QAAQ;CAEnD,OAAO,QAAQ,OAAO,QAAQ,QAAQ;CAEtC,OAAO;AACT;;;;;AAMA,MAAa,YAAY,iBAAiB,OAAO,SAAS,SAAS;CACjE,MAAM,EAAE,aAAa,QAAQ;CAE7B,MAAM,SAAS,CAAC,WAAW,KAAK,QAAQ;CAExC,IAAI,QAAQ,iBAAiB,CAAC,QAAQ,OAAO,KAAK;CAElD,MAAM,SAAS,QAAQ,QAAQ,QAAQ,IAAI,QAAQ,KAAK;CAExD,OAAO,SACL,gBAAgB,MAAM,IAClB,MAAM,QAAQ,QAAQ,GAAG,SAAS,QAAQ,QAAQ,GAAG,EAAE,SAAS,IAChE,MAAM,KAAK,CACjB;AACF,CAAC"}

package/dist/astro/routes/answers.js

@@ -0,0 +1,129 @@
+import { r as toAnswerSources } from "../../chunks/limits-Ppc7BQPG.js";
+import { i as toErrorResponse, n as json, r as readRequest, t as createPageResolver } from "../../chunks/request-DOuBHGQs.js";
+import { z } from "zod";
+import config from "virtual:andesine/config";
+//#region src/search/rate-limit.ts
+/**
+* A small in-memory rate limit per client, e.g. by IP address. Each server instance counts on its
+* own, so it only limits abuse; the Andesine API applies its own limits too.
+*/
+const createRateLimit = (options) => {
+	const requests = /* @__PURE__ */ new Map();
+	let prunedAt = 0;
+	return (client) => {
+		const now = Date.now();
+		const recent = (requests.get(client) ?? []).filter((time) => now - time < options.window);
+		if (requests.size > 1e4 && now - prunedAt >= options.window) {
+			prunedAt = now;
+			for (const [key, times] of requests) if (times.every((time) => now - time >= options.window)) requests.delete(key);
+		}
+		if (recent.length >= options.limit) {
+			requests.set(client, recent);
+			return false;
+		}
+		requests.set(client, [...recent, now]);
+		return true;
+	};
+};
+//#endregion
+//#region src/search/answers.ts
+/** The API limits for questions and history. */
+const bodySchema = z.object({
+	source: z.string().min(1),
+	question: z.string().trim().min(1).max(1e3),
+	history: z.array(z.object({
+		role: z.enum(["user", "assistant"]),
+		content: z.string().trim().min(1).max(4e3)
+	}).strict()).max(10).default([])
+}).strict();
+const encoder = new TextEncoder();
+const allowQuestion = createRateLimit({
+	limit: 10,
+	window: 6e4
+});
+const encode = (event) => encoder.encode(`${JSON.stringify(event)}\n`);
+/**
+* Streams an AI answer about the published collection of an Andesine source, as one JSON
+* event per line. Cited pages are mapped to page URLs; pages without one are left out.
+*/
+const handleAnswer = async (request, config, clientAddress) => {
+	if (clientAddress && !allowQuestion(clientAddress)) return json({ error: "Too many questions. Try again soon." }, 429, { "Retry-After": "60" });
+	const result = await readRequest(request, config, {
+		schema: bodySchema,
+		maxSize: 64e3,
+		timeout: 12e4
+	});
+	if ("response" in result) return result.response;
+	const { body, source, client, signal } = result.request;
+	if (!source.answers) return json({ error: "AI answers are off for this source." }, 403);
+	let resolve;
+	let events;
+	try {
+		[resolve, events] = await Promise.all([createPageResolver(source, config.base), client.search.askPublishedStream({
+			question: body.question,
+			history: body.history,
+			collectionID: source.collection,
+			channel: "published"
+		}, { signal })]);
+	} catch (error) {
+		return toErrorResponse(error, {
+			limited: "Too many questions. Try again soon.",
+			failed: "AI answers are not available now."
+		});
+	}
+	const iterator = events[Symbol.asyncIterator]();
+	const stream = new ReadableStream({
+		async pull(controller) {
+			try {
+				const next = await iterator.next();
+				if (next.done) {
+					controller.close();
+					return;
+				}
+				const event = next.value;
+				if (event.type === "textDelta") controller.enqueue(encode(event));
+				else if (event.type === "sources") controller.enqueue(encode({
+					type: "sources",
+					sources: toAnswerSources(event.sources, resolve)
+				}));
+				else controller.enqueue(encode({
+					type: "completed",
+					answer: event.answer,
+					sources: toAnswerSources(event.sources, resolve)
+				}));
+			} catch (error) {
+				if (!signal.aborted) console.error(error);
+				controller.enqueue(encode({
+					type: "error",
+					error: "The answer could not be completed."
+				}));
+				controller.close();
+				await iterator.return?.();
+			}
+		},
+		async cancel() {
+			await iterator.return?.();
+		}
+	}, { highWaterMark: 0 });
+	return new Response(stream, { headers: {
+		"Content-Type": "application/x-ndjson; charset=utf-8",
+		"Cache-Control": "private, no-store, no-transform",
+		"X-Accel-Buffering": "no"
+	} });
+};
+//#endregion
+//#region src/astro/routes/answers.ts
+const prerender = false;
+const POST = (context) => {
+	let clientAddress;
+	try {
+		clientAddress = context.clientAddress;
+	} catch {
+		clientAddress = void 0;
+	}
+	return handleAnswer(context.request, config, clientAddress);
+};
+//#endregion
+export { POST, prerender };
+
+//# sourceMappingURL=answers.js.map

package/dist/astro/routes/answers.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"answers.js","names":[],"sources":["../../../src/search/rate-limit.ts","../../../src/search/answers.ts","../../../src/astro/routes/answers.ts"],"sourcesContent":["interface RateLimitOptions {\n  /** Requests allowed in each window. */\n  limit: number;\n  /** Window length, in milliseconds. */\n  window: number;\n}\n\n/**\n * A small in-memory rate limit per client, e.g. by IP address. Each server instance counts on its\n * own, so it only limits abuse; the Andesine API applies its own limits too.\n */\nconst createRateLimit = (options: RateLimitOptions): ((client: string) => boolean) => {\n  const requests = new Map<string, number[]>();\n\n  let prunedAt = 0;\n\n  return (client) => {\n    const now = Date.now();\n    const recent = (requests.get(client) ?? []).filter((time) => now - time < options.window);\n    const shouldPrune = requests.size > 10_000 && now - prunedAt >= options.window;\n\n    // Forgets clients without recent requests, at most once a window, so memory stays small.\n    if (shouldPrune) {\n      prunedAt = now;\n\n      for (const [key, times] of requests) {\n        if (times.every((time) => now - time >= options.window)) requests.delete(key);\n      }\n    }\n\n    if (recent.length >= options.limit) {\n      requests.set(client, recent);\n\n      return false;\n    }\n\n    requests.set(client, [...recent, now]);\n\n    return true;\n  };\n};\n\nexport { createRateLimit };\nexport type { RateLimitOptions };\n","import type { PublishedAnswerEvent } from \"@andesine/sdk\";\nimport { z } from \"zod\";\nimport type { PagesConfig } from \"../config\";\nimport { createRateLimit } from \"./rate-limit\";\nimport { toAnswerSources, type PageResolver } from \"./items\";\nimport { createPageResolver, json, readRequest, toErrorResponse } from \"./request\";\nimport type { AnswerEvent } from \"./types\";\n\n/** The API limits for questions and history. */\nconst bodySchema = z\n  .object({\n    source: z.string().min(1),\n    question: z.string().trim().min(1).max(1000),\n    history: z\n      .array(\n        z\n          .object({\n            role: z.enum([\"user\", \"assistant\"]),\n            content: z.string().trim().min(1).max(4000)\n          })\n          .strict()\n      )\n      .max(10)\n      .default([])\n  })\n  .strict();\nconst encoder = new TextEncoder();\n// Each client can ask 10 questions a minute.\nconst allowQuestion = createRateLimit({ limit: 10, window: 60_000 });\n\nconst encode = (event: AnswerEvent): Uint8Array => encoder.encode(`${JSON.stringify(event)}\\n`);\n/**\n * Streams an AI answer about the published collection of an Andesine source, as one JSON\n * event per line. Cited pages are mapped to page URLs; pages without one are left out.\n */\nconst handleAnswer = async (\n  request: Request,\n  config: PagesConfig,\n  clientAddress?: string\n): Promise<Response> => {\n  if (clientAddress && !allowQuestion(clientAddress)) {\n    return json({ error: \"Too many questions. Try again soon.\" }, 429, { \"Retry-After\": \"60\" });\n  }\n\n  const result = await readRequest(request, config, {\n    schema: bodySchema,\n    maxSize: 64_000,\n    timeout: 120_000\n  });\n\n  if (\"response\" in result) return result.response;\n\n  const { body, source, client, signal } = result.request;\n\n  if (!source.answers) return json({ error: \"AI answers are off for this source.\" }, 403);\n\n  let resolve: PageResolver;\n  let events: AsyncIterable<PublishedAnswerEvent>;\n\n  try {\n    [resolve, events] = await Promise.all([\n      createPageResolver(source, config.base),\n      client.search.askPublishedStream(\n        {\n          question: body.question,\n          history: body.history,\n          collectionID: source.collection,\n          channel: \"published\"\n        },\n        { signal }\n      )\n    ]);\n  } catch (error) {\n    return toErrorResponse(error, {\n      limited: \"Too many questions. Try again soon.\",\n      failed: \"AI answers are not available now.\"\n    });\n  }\n\n  const iterator = events[Symbol.asyncIterator]();\n  const stream = new ReadableStream<Uint8Array>(\n    {\n      async pull(controller) {\n        try {\n          const next = await iterator.next();\n\n          if (next.done) {\n            controller.close();\n\n            return;\n          }\n\n          const event = next.value;\n\n          if (event.type === \"textDelta\") {\n            controller.enqueue(encode(event));\n          } else if (event.type === \"sources\") {\n            controller.enqueue(\n              encode({ type: \"sources\", sources: toAnswerSources(event.sources, resolve) })\n            );\n          } else {\n            controller.enqueue(\n              encode({\n                type: \"completed\",\n                answer: event.answer,\n                sources: toAnswerSources(event.sources, resolve)\n              })\n            );\n          }\n        } catch (error) {\n          if (!signal.aborted) console.error(error);\n\n          controller.enqueue(\n            encode({ type: \"error\", error: \"The answer could not be completed.\" })\n          );\n          controller.close();\n          await iterator.return?.();\n        }\n      },\n      async cancel() {\n        await iterator.return?.();\n      }\n    },\n    { highWaterMark: 0 }\n  );\n\n  return new Response(stream, {\n    headers: {\n      \"Content-Type\": \"application/x-ndjson; charset=utf-8\",\n      \"Cache-Control\": \"private, no-store, no-transform\",\n      \"X-Accel-Buffering\": \"no\"\n    }\n  });\n};\n\nexport { handleAnswer };\n","import type { APIRoute } from \"astro\";\nimport config from \"virtual:andesine/config\";\nimport { handleAnswer } from \"../../search/answers\";\n\nexport const prerender = false;\n\nexport const POST: APIRoute = (context) => {\n  let clientAddress: string | undefined;\n\n  // Adapters that do not know the client address throw here; the endpoint then has no limit.\n  try {\n    clientAddress = context.clientAddress;\n  } catch {\n    clientAddress = undefined;\n  }\n\n  return handleAnswer(context.request, config, clientAddress);\n};\n"],"mappings":";;;;;;;;;AAWA,MAAM,mBAAmB,YAA6D;CACpF,MAAM,2BAAW,IAAI,IAAsB;CAE3C,IAAI,WAAW;CAEf,QAAQ,WAAW;EACjB,MAAM,MAAM,KAAK,IAAI;EACrB,MAAM,UAAU,SAAS,IAAI,MAAM,KAAK,CAAC,EAAA,CAAG,QAAQ,SAAS,MAAM,OAAO,QAAQ,MAAM;EAIxF,IAHoB,SAAS,OAAO,OAAU,MAAM,YAAY,QAAQ,QAGvD;GACf,WAAW;GAEX,KAAK,MAAM,CAAC,KAAK,UAAU,UACzB,IAAI,MAAM,OAAO,SAAS,MAAM,QAAQ,QAAQ,MAAM,GAAG,SAAS,OAAO,GAAG;EAEhF;EAEA,IAAI,OAAO,UAAU,QAAQ,OAAO;GAClC,SAAS,IAAI,QAAQ,MAAM;GAE3B,OAAO;EACT;EAEA,SAAS,IAAI,QAAQ,CAAC,GAAG,QAAQ,GAAG,CAAC;EAErC,OAAO;CACT;AACF;;;;AC/BA,MAAM,aAAa,EAChB,OAAO;CACN,QAAQ,EAAE,OAAO,CAAC,CAAC,IAAI,CAAC;CACxB,UAAU,EAAE,OAAO,CAAC,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,GAAI;CAC3C,SAAS,EACN,MACC,EACG,OAAO;EACN,MAAM,EAAE,KAAK,CAAC,QAAQ,WAAW,CAAC;EAClC,SAAS,EAAE,OAAO,CAAC,CAAC,KAAK,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,GAAI;CAC5C,CAAC,CAAC,CACD,OAAO,CACZ,CAAC,CACA,IAAI,EAAE,CAAC,CACP,QAAQ,CAAC,CAAC;AACf,CAAC,CAAC,CACD,OAAO;AACV,MAAM,UAAU,IAAI,YAAY;AAEhC,MAAM,gBAAgB,gBAAgB;CAAE,OAAO;CAAI,QAAQ;AAAO,CAAC;AAEnE,MAAM,UAAU,UAAmC,QAAQ,OAAO,GAAG,KAAK,UAAU,KAAK,EAAE,GAAG;;;;;AAK9F,MAAM,eAAe,OACnB,SACA,QACA,kBACsB;CACtB,IAAI,iBAAiB,CAAC,cAAc,aAAa,GAC/C,OAAO,KAAK,EAAE,OAAO,sCAAsC,GAAG,KAAK,EAAE,eAAe,KAAK,CAAC;CAG5F,MAAM,SAAS,MAAM,YAAY,SAAS,QAAQ;EAChD,QAAQ;EACR,SAAS;EACT,SAAS;CACX,CAAC;CAED,IAAI,cAAc,QAAQ,OAAO,OAAO;CAExC,MAAM,EAAE,MAAM,QAAQ,QAAQ,WAAW,OAAO;CAEhD,IAAI,CAAC,OAAO,SAAS,OAAO,KAAK,EAAE,OAAO,sCAAsC,GAAG,GAAG;CAEtF,IAAI;CACJ,IAAI;CAEJ,IAAI;EACF,CAAC,SAAS,UAAU,MAAM,QAAQ,IAAI,CACpC,mBAAmB,QAAQ,OAAO,IAAI,GACtC,OAAO,OAAO,mBACZ;GACE,UAAU,KAAK;GACf,SAAS,KAAK;GACd,cAAc,OAAO;GACrB,SAAS;EACX,GACA,EAAE,OAAO,CACX,CACF,CAAC;CACH,SAAS,OAAO;EACd,OAAO,gBAAgB,OAAO;GAC5B,SAAS;GACT,QAAQ;EACV,CAAC;CACH;CAEA,MAAM,WAAW,OAAO,OAAO,cAAc,CAAC;CAC9C,MAAM,SAAS,IAAI,eACjB;EACE,MAAM,KAAK,YAAY;GACrB,IAAI;IACF,MAAM,OAAO,MAAM,SAAS,KAAK;IAEjC,IAAI,KAAK,MAAM;KACb,WAAW,MAAM;KAEjB;IACF;IAEA,MAAM,QAAQ,KAAK;IAEnB,IAAI,MAAM,SAAS,aACjB,WAAW,QAAQ,OAAO,KAAK,CAAC;SAC3B,IAAI,MAAM,SAAS,WACxB,WAAW,QACT,OAAO;KAAE,MAAM;KAAW,SAAS,gBAAgB,MAAM,SAAS,OAAO;IAAE,CAAC,CAC9E;SAEA,WAAW,QACT,OAAO;KACL,MAAM;KACN,QAAQ,MAAM;KACd,SAAS,gBAAgB,MAAM,SAAS,OAAO;IACjD,CAAC,CACH;GAEJ,SAAS,OAAO;IACd,IAAI,CAAC,OAAO,SAAS,QAAQ,MAAM,KAAK;IAExC,WAAW,QACT,OAAO;KAAE,MAAM;KAAS,OAAO;IAAqC,CAAC,CACvE;IACA,WAAW,MAAM;IACjB,MAAM,SAAS,SAAS;GAC1B;EACF;EACA,MAAM,SAAS;GACb,MAAM,SAAS,SAAS;EAC1B;CACF,GACA,EAAE,eAAe,EAAE,CACrB;CAEA,OAAO,IAAI,SAAS,QAAQ,EAC1B,SAAS;EACP,gBAAgB;EAChB,iBAAiB;EACjB,qBAAqB;CACvB,EACF,CAAC;AACH;;;ACjIA,MAAa,YAAY;AAEzB,MAAa,QAAkB,YAAY;CACzC,IAAI;CAGJ,IAAI;EACF,gBAAgB,QAAQ;CAC1B,QAAQ;EACN,gBAAgB,KAAA;CAClB;CAEA,OAAO,aAAa,QAAQ,SAAS,QAAQ,aAAa;AAC5D"}

package/dist/astro/routes/entry-pages.js

@@ -0,0 +1,22 @@
+import { t as loadAndesineSource } from "../../chunks/stored-CabomV-C.js";
+import { t as textResponse } from "../../chunks/response-CtzaKMWH.js";
+import config from "virtual:andesine/config";
+//#region src/astro/routes/entry-pages.ts
+const sources = config.sources.filter((source) => {
+	return source.type === "andesine" && Boolean(source.publicKey);
+});
+const getStaticPaths = () => sources.map((source) => ({ params: { source: source.id } }));
+/**
+* Maps the entry IDs of a source with a publishable key to page URLs, so the browser can link
+* API search results and answer sources. Pages hidden from search are left out.
+*/
+const GET = async ({ params }) => {
+	const source = sources.find((item) => item.id === params.source);
+	const { pages } = await loadAndesineSource(source, config.base);
+	const hrefs = Object.fromEntries(pages.filter((page) => !page.searchHidden).map((page) => [page.id, page.href]));
+	return textResponse(JSON.stringify(hrefs), "application/json");
+};
+//#endregion
+export { GET, getStaticPaths };
+
+//# sourceMappingURL=entry-pages.js.map

package/dist/astro/routes/entry-pages.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"entry-pages.js","names":[],"sources":["../../../src/astro/routes/entry-pages.ts"],"sourcesContent":["import type { APIRoute } from \"astro\";\nimport config from \"virtual:andesine/config\";\nimport type { AndesineSourceConfig } from \"../../config\";\nimport { textResponse } from \"../../output/response\";\nimport { loadAndesineSource } from \"../../sources/andesine/stored\";\n\nconst sources = config.sources.filter((source): source is AndesineSourceConfig => {\n  return source.type === \"andesine\" && Boolean(source.publicKey);\n});\n\nexport const getStaticPaths = () => sources.map((source) => ({ params: { source: source.id } }));\n\n/**\n * Maps the entry IDs of a source with a publishable key to page URLs, so the browser can link\n * API search results and answer sources. Pages hidden from search are left out.\n */\nexport const GET: APIRoute = async ({ params }) => {\n  const source = sources.find((item) => item.id === params.source)!;\n  const { pages } = await loadAndesineSource(source, config.base);\n  const hrefs = Object.fromEntries(\n    pages.filter((page) => !page.searchHidden).map((page) => [page.id, page.href])\n  );\n\n  return textResponse(JSON.stringify(hrefs), \"application/json\");\n};\n"],"mappings":";;;;AAMA,MAAM,UAAU,OAAO,QAAQ,QAAQ,WAA2C;CAChF,OAAO,OAAO,SAAS,cAAc,QAAQ,OAAO,SAAS;AAC/D,CAAC;AAED,MAAa,uBAAuB,QAAQ,KAAK,YAAY,EAAE,QAAQ,EAAE,QAAQ,OAAO,GAAG,EAAE,EAAE;;;;;AAM/F,MAAa,MAAgB,OAAO,EAAE,aAAa;CACjD,MAAM,SAAS,QAAQ,MAAM,SAAS,KAAK,OAAO,OAAO,MAAM;CAC/D,MAAM,EAAE,UAAU,MAAM,mBAAmB,QAAQ,OAAO,IAAI;CAC9D,MAAM,QAAQ,OAAO,YACnB,MAAM,QAAQ,SAAS,CAAC,KAAK,YAAY,CAAC,CAAC,KAAK,SAAS,CAAC,KAAK,IAAI,KAAK,IAAI,CAAC,CAC/E;CAEA,OAAO,aAAa,KAAK,UAAU,KAAK,GAAG,kBAAkB;AAC/D"}

package/dist/astro/routes/llms-full.js

@@ -0,0 +1,12 @@
+import { t as textResponse } from "../../chunks/response-CtzaKMWH.js";
+import { t as loadCatalog } from "../../chunks/catalog-DU4BUrpw.js";
+import { n as createLLMsFull } from "../../chunks/files-BxqiQzFj.js";
+import config from "virtual:andesine/config";
+//#region src/astro/routes/llms-full.ts
+const GET = async () => {
+	return textResponse(createLLMsFull(config, await loadCatalog(config)), "text/plain");
+};
+//#endregion
+export { GET };
+
+//# sourceMappingURL=llms-full.js.map

package/dist/astro/routes/llms-full.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"llms-full.js","names":[],"sources":["../../../src/astro/routes/llms-full.ts"],"sourcesContent":["import type { APIRoute } from \"astro\";\nimport config from \"virtual:andesine/config\";\nimport { loadCatalog } from \"../../output/catalog\";\nimport { createLLMsFull } from \"../../output/files\";\nimport { textResponse } from \"../../output/response\";\n\nexport const GET: APIRoute = async () => {\n  return textResponse(createLLMsFull(config, await loadCatalog(config)), \"text/plain\");\n};\n"],"mappings":";;;;;AAMA,MAAa,MAAgB,YAAY;CACvC,OAAO,aAAa,eAAe,QAAQ,MAAM,YAAY,MAAM,CAAC,GAAG,YAAY;AACrF"}

package/dist/astro/routes/llms.js

@@ -0,0 +1,12 @@
+import { t as textResponse } from "../../chunks/response-CtzaKMWH.js";
+import { t as loadCatalog } from "../../chunks/catalog-DU4BUrpw.js";
+import { t as createLLMs } from "../../chunks/files-BxqiQzFj.js";
+import config from "virtual:andesine/config";
+//#region src/astro/routes/llms.ts
+const GET = async () => {
+	return textResponse(createLLMs(config, await loadCatalog(config)), "text/plain");
+};
+//#endregion
+export { GET };
+
+//# sourceMappingURL=llms.js.map

package/dist/astro/routes/llms.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"llms.js","names":[],"sources":["../../../src/astro/routes/llms.ts"],"sourcesContent":["import type { APIRoute } from \"astro\";\nimport config from \"virtual:andesine/config\";\nimport { loadCatalog } from \"../../output/catalog\";\nimport { createLLMs } from \"../../output/files\";\nimport { textResponse } from \"../../output/response\";\n\nexport const GET: APIRoute = async () => {\n  return textResponse(createLLMs(config, await loadCatalog(config)), \"text/plain\");\n};\n"],"mappings":";;;;;AAMA,MAAa,MAAgB,YAAY;CACvC,OAAO,aAAa,WAAW,QAAQ,MAAM,YAAY,MAAM,CAAC,GAAG,YAAY;AACjF"}

package/dist/astro/routes/markdown-alias.js

@@ -0,0 +1,14 @@
+import { t as getMarkdownPaths } from "../../chunks/site-DY-uZ0PP.js";
+import { t as textResponse } from "../../chunks/response-CtzaKMWH.js";
+import config from "virtual:andesine/config";
+//#region src/astro/routes/markdown-alias.ts
+const getStaticPaths = async () => {
+	return (await getMarkdownPaths(config)).filter((path) => path.params.path);
+};
+const GET = ({ props }) => {
+	return textResponse(props.markdown, "text/markdown");
+};
+//#endregion
+export { GET, getStaticPaths };
+
+//# sourceMappingURL=markdown-alias.js.map

package/dist/astro/routes/markdown-alias.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"markdown-alias.js","names":[],"sources":["../../../src/astro/routes/markdown-alias.ts"],"sourcesContent":["import type { APIRoute } from \"astro\";\nimport config from \"virtual:andesine/config\";\nimport { textResponse } from \"../../output/response\";\nimport { getMarkdownPaths } from \"../../site\";\n\n// Agents often add `.md` to a page URL: `/guide/` becomes `/guide.md`. The home page has no alias.\nexport const getStaticPaths = async () => {\n  return (await getMarkdownPaths(config)).filter((path) => path.params.path);\n};\n\nexport const GET: APIRoute = ({ props }) => {\n  return textResponse((props as { markdown: string }).markdown, \"text/markdown\");\n};\n"],"mappings":";;;;AAMA,MAAa,iBAAiB,YAAY;CACxC,QAAQ,MAAM,iBAAiB,MAAM,EAAA,CAAG,QAAQ,SAAS,KAAK,OAAO,IAAI;AAC3E;AAEA,MAAa,OAAiB,EAAE,YAAY;CAC1C,OAAO,aAAc,MAA+B,UAAU,eAAe;AAC/E"}

package/dist/astro/routes/markdown.js

@@ -0,0 +1,12 @@
+import { t as getMarkdownPaths } from "../../chunks/site-DY-uZ0PP.js";
+import { t as textResponse } from "../../chunks/response-CtzaKMWH.js";
+import config from "virtual:andesine/config";
+//#region src/astro/routes/markdown.ts
+const getStaticPaths = () => getMarkdownPaths(config);
+const GET = ({ props }) => {
+	return textResponse(props.markdown, "text/markdown");
+};
+//#endregion
+export { GET, getStaticPaths };
+
+//# sourceMappingURL=markdown.js.map

package/dist/astro/routes/markdown.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"markdown.js","names":[],"sources":["../../../src/astro/routes/markdown.ts"],"sourcesContent":["import type { APIRoute } from \"astro\";\nimport config from \"virtual:andesine/config\";\nimport { textResponse } from \"../../output/response\";\nimport { getMarkdownPaths } from \"../../site\";\n\nexport const getStaticPaths = () => getMarkdownPaths(config);\n\nexport const GET: APIRoute = ({ props }) => {\n  return textResponse((props as { markdown: string }).markdown, \"text/markdown\");\n};\n"],"mappings":";;;;AAKA,MAAa,uBAAuB,iBAAiB,MAAM;AAE3D,MAAa,OAAiB,EAAE,YAAY;CAC1C,OAAO,aAAc,MAA+B,UAAU,eAAe;AAC/E"}