@stackql/docusaurus-plugin-aeo 0.4.2 → 0.5.0

package/CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## 0.5.0
+
+Plain-markdown companions, one rule for where a companion lives, and the companion advertised from its page. Raised against stackql.io, where the Ask AI prompt on the homepage read `https://stackql.io.md` and every `.md` URL returned raw MDX (imports, MUI components, tab definitions and inline SVG path data before the first sentence), and from an independent agent-readiness audit of the built site (alternate link, `llms-full.txt` reference and size, `Source:` URLs, the `how-tos` alias).
+
+### Changed (breaking)
+
+- **`companions.format` now defaults to `'plain'`.** The companion is plain markdown rendered from the MDX source rather than the source itself. The source is parsed with the remark toolchain Docusaurus compiles it with (`remark-mdx`, `remark-gfm`, `remark-directive`, `@slorber/remark-comment`) to locate every MDX construct, and the original text is edited by source position: imports, exports and `{expressions}` are removed; presentational elements (`svg`, `video`, `iframe`, icons) are dropped; `<Tabs>`/`<TabItem>` become a bold label per tab with the tab's content under it (the label is read from the `label` attribute or from the `<Tabs values={[...]}>` expression); `<a>`, `<img>`, `<b>`, `<i>`, `<code>`, `<h1>`-`<h6>`, `<br>`, `<details>`/`<summary>` become their markdown equivalent; a self-closing component that carries a link (`to`/`href` plus `text`/`label`/`title`) becomes a markdown link; every other element is unwrapped and the indentation its author added inside it is removed. HTML comments (including `<!-- truncate -->`) and `{#custom-id}` heading ids are removed; fenced code is never touched. Nothing outside MDX syntax is re-stringified, so `snake_case` stays unescaped and tables, admonitions and code fences are byte for byte the author's. A `# Title` / `> description` block from the page metadata is prepended (the title only when the body does not already open with an H1). A source that fails to parse falls back to the v0.4 regex stripper with a build warning. `format: md` files (front matter or `siteConfig.markdown.format`) are parsed as CommonMark and their raw HTML stripped. Set `companions: { format: 'raw' }` to keep emitting the source.
+- **Default `askAi.promptTemplate` is `'Read {companionUrl} and help me understand it. ...'`.** `{companionUrl}` is the page's `.md` file by the same rule feature 1 writes it. `{pageUrl}` is unchanged (the page, no trailing slash). A custom template still using the v0.4 idiom `{pageUrl}.md` keeps working: that sequence now resolves to `{companionUrl}` instead of being computed from the pathname, which produced `https://site.md` on the homepage and `/foo.md` on sites that build `/foo/index.html`.
+- **The Ask AI button is only rendered on pages that have a companion.** Generated category index pages, React pages and blog list pages have no source markdown and so no `.md` file; the button used to point readers at a 404 there. The route list is set as plugin global data from `allContentLoaded` (Docusaurus 3 merges it with the `contentLoaded` data); if it is absent the button shows everywhere as before. With `companions.enabled: false` the button shows everywhere and the prompt uses the page URL.
+- **`llms.txt` links are absolute** (`https://site/foo.md`), as the llms.txt spec's examples are. The file is read away from the site, where a relative link has no base.
+
+### Added
+
+- **`<link rel="alternate" type="text/markdown" href="...">` in every page that has a companion** (`companions.alternateLink`, default `true`), inserted into the built HTML in `postBuild`, with the absolute companion URL from the same rule the file is written by. `companions.linkHeader` (default `false`) also writes the relation as a `Link:` header per route in a Netlify-style `_headers` file, appended to one the site already ships.
+- **`## Optional` section in `llms.txt`** linking `llms-full.txt` (`llmsTxt.linkFullTxt`, default `true`), the llmstxt.org convention for secondary resources.
+- **`llmsTxt.fullTxt` object form** `{ include: ['<plugin>@<id>', ...], maxBytes }` to limit `llms-full.txt` to chosen content instances and/or a byte budget; the file stops before the page that would cross the cap. The boolean form is unchanged (`true` = everything).
+- Plain companions carry a `Source: <page URL>` line under the title and description, and `llms-full.txt` reuses it rather than adding a second one.
+- `companions.format: 'clean'` is accepted as a synonym for `'plain'`.
+- `src/features/aiRoutes.js`: `how-tos` and `how-to` directory names map to the `howTo` payload (only `howto` / `howtos` did; stackql.io uses `how-tos`).
+
+### Fixed
+
+- Feature 1, feature 2 and the Ask AI button each computed the companion location on their own; the button's version was wrong for the site root and for `trailingSlash: true`/undefined sites. All three now use `src/companionPath.js`.
+- `llms-full.txt` no longer doubles the page title (the plain companion already opens with it) and now puts a blank line after the `Source:` line, which used to run into the first paragraph.
+- Companions are written with LF line endings regardless of the source file's.
+
+### Added
+
+- `src/companionPath.js` (`companionPath`, `companionUrl`, `normalizeRoute`) and `src/askAiPrompt.js` (`fillPrompt`), plain CommonJS shared by the Node side and the theme component.
+- Runtime dependencies on `unified`, `remark-parse`, `remark-mdx`, `remark-gfm`, `remark-directive` and `@slorber/remark-comment` at the versions `@docusaurus/mdx-loader` 3.x already ships, so npm dedupes them.
+- A test suite (`npm test`, `node --test`): the companion path and HTML path rules, the prompt template, MDX fixtures for the converter (tabs, nested containers and dedent, link recovery, inline elements, details/summary, comments and heading ids, string-literal expressions, `md` format, the parse-failure fallback, CRLF sources), `llms.txt` rendering (sections, absolute links, the Optional block, the `fullTxt` forms), the alternate-link insertion (both `trailingSlash` layouts, idempotence, `_headers`), and the `/ai/*` directory classifier.
+
 ## 0.4.2
 
 ### Fixed

package/README.md

@@ -1,6 +1,8 @@
+[![NPM Version](https://img.shields.io/npm/v/%40stackql%2Fdocusaurus-plugin-aeo)](https://www.npmjs.com/package/@stackql/docusaurus-plugin-aeo)
+
 # @stackql/docusaurus-plugin-aeo
 
-AEO (Answer Engine Optimization) helpers for Docusaurus 3.x sites: emit raw `.md` companion files for every page, generate `llms.txt` and `llms-full.txt` at the site root, drop an "Ask AI" dropdown into doc and blog footers, and document a `/ai/*` routing convention for machine-readable companion content. This is a sibling to `@stackql/docusaurus-plugin-structured-data` (which emits JSON-LD); the two plugins compose without overlap.
+AEO (Answer Engine Optimization) helpers for Docusaurus 3.x sites: emit plain-markdown `.md` companions for docs and blog posts, generate `llms.txt` and `llms-full.txt` at the build root, add an "Ask AI" dropdown above doc and blog content, and document a `/ai/*` routing convention for machine-readable companion content. This is a sibling to `@stackql/docusaurus-plugin-structured-data` (which emits JSON-LD); the two plugins compose without overlap.
 
 ## Installation
 
@@ -25,7 +27,7 @@ module.exports = {
 };
 ```
 
-All four features are on by default. To configure, pass options:
+Companions, `llms.txt` / `llms-full.txt`, and the Ask AI button are enabled by default. The `/ai/*` docs instance requires site configuration, and its optional validator is off by default. To configure, pass options:
 
 ```js
 module.exports = {
@@ -45,12 +47,23 @@ module.exports = {
 
 ## Feature 1: `.md` companion files
 
-For every emitted HTML route from the docs and blog plugins, this plugin writes a sibling `.md` file. A page at `/docs/intro` gets a companion at `/docs/intro/index.md` (or `/docs/intro.md` if `siteConfig.trailingSlash` is `false`). The file contains the raw markdown source.
+For each doc or blog post with a readable source file, this plugin writes a sibling `.md` file unless its permalink matches `companions.exclude`. The companion mirrors the HTML layout: a page at `/docs/intro` gets `/docs/intro/index.md` (or `/docs/intro.md` if `siteConfig.trailingSlash` is `false`). A doc or blog post at the site root gets `/index.md`. The same URL rule (`src/companionPath.js`) is used by `llms.txt` and the Ask AI button.
 
 Two modes:
 
-- `companions.format: 'raw'` (default): the MDX source is emitted as-is. MDX `<Component />` tags pass through. LLMs handle them fine, and the file is a faithful representation of the page.
-- `companions.format: 'plain'`: a best-effort regex pass strips `import` / `export` lines and JSX tags, then prepends a `# Title\n\n> description` block from the page frontmatter. Not a full remark pipeline - use `'raw'` unless an MDX-heavy page is causing concrete problems.
+- `companions.format: 'plain'` (default since 0.5.0; `'clean'` is accepted as a synonym): plain markdown rendered from the MDX source, with a `# Title`, `> description` and `Source: <page URL>` block from the page metadata at the top, so an LLM can cite the page without parsing YAML. The converter uses remark-mdx, GFM, directives and comments to locate MDX constructs, then edits the original text by source position. It does not load the site's custom remark/rehype plugins:
+  - `import` / `export` statements and `{expressions}` are removed (a string literal expression keeps its value)
+  - presentational elements (`<svg>`, `<video>`, `<iframe>`, icons, ...) are dropped with their subtree
+  - `<Tabs>` / `<TabItem>` become a bold label per tab followed by the tab's content; the label comes from the `label` attribute or from the `<Tabs values={[...]}>` expression
+  - `<a>`, `<img>`, `<b>`, `<i>`, `<code>`, `<h2>`, `<br>`, `<details>` / `<summary>` become their markdown equivalent
+  - a self-closing component that plainly carries a link (`to` or `href` plus `text`, `label` or `title`) becomes a markdown link, so download buttons survive
+  - everything else (`<div>`, `<Box>`, `<span>`, your own layout components) is unwrapped: the tags go, the children stay, and indentation the author added inside the container is removed
+  - HTML comments (including `<!-- truncate -->`) and `{#custom-id}` heading ids are removed outside ordinary code fences; `mdx-code-block` fences are unwrapped and their contents are processed as markup
+
+  Markdown is not re-stringified, so `snake_case` is not escaped and Markdown syntax is retained. Output is not byte-for-byte identical: line endings become LF, surplus blank lines and outer whitespace are trimmed, leading/trailing horizontal rules are removed, and content inside unwrapped containers is dedented. An existing opening H1 is retained instead of adding another title. A Markdown/MDX parse or conversion failure uses a regex fallback and logs a warning; front matter parsing errors are not covered by that fallback.
+
+  Source format follows front matter `format`, then `siteConfig.markdown.format`; `detect` selects by extension (`.md` as Markdown, `.mdx` as MDX). Files resolved as `md` use a CommonMark parser with GFM and directives, with HTML tags stripped or converted.
+- `companions.format: 'raw'`: the MDX source is emitted as-is, front matter included. MDX `<Component />` tags, imports and inline SVG data all pass through.
 
 Fetch example:
 
@@ -58,15 +71,30 @@ Fetch example:
 curl https://your-site.example/docs/intro/index.md
 ```
 
-**MIME type note.** Setting `Content-Type: text/markdown` from a static-site plugin is not possible. The file simply gets a `.md` extension and the host's MIME table handles it. Vercel, Netlify, Cloudflare Pages, GitHub Pages, and S3+CloudFront all serve `.md` as `text/markdown` or `text/plain` out of the box. If you serve from Nginx, ensure `text/markdown md;` is in your `mime.types`.
+**MIME type note.** The plugin writes `.md` files; the host determines their `Content-Type`. Check a deployed companion with `curl -I` and configure the host if needed. The alternate link below advertises the companion as `text/markdown` but does not set its HTTP response type.
+
+### Advertising the companion from the page
 
-Non-content routes (custom React pages, redirects, the 404 page, search) are skipped silently. Enable `verbose: true` to log skips.
+Every page that has a companion gets, in its `<head>`:
+
+```html
+<link rel="alternate" type="text/markdown" href="https://your-site.example/docs/intro/index.md">
+```
+
+inserted into the built HTML in `postBuild` (the same pattern `@stackql/docusaurus-plugin-structured-data` uses). An agent that lands on the HTML can find the markdown without guessing the URL, and agent-readiness checkers score it. The href is absolute and follows the same rule as the file on disk. Options:
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `companions.alternateLink` | `true` | Insert the `<link rel="alternate" type="text/markdown">` tag into each page that has a companion. |
+| `companions.linkHeader` | `false` | Also write the relation as an HTTP header for Netlify-style hosts: a `_headers` file in the build root with `Link: <url>; rel="alternate"; type="text/markdown"` per route (appended to a `_headers` your `static/` already ships). Requires `companions.alternateLink: true`. |
+
+Generated category indexes, custom React pages and blog listing pages do not get companions. `verbose: true` logs unreadable source files and missing HTML during alternate-link insertion.
 
 ## Feature 2: `llms.txt` and `llms-full.txt`
 
-After feature 1 emits all companions, the plugin writes two files to the build root:
+With the defaults enabled, the plugin writes two files to the build root after emitting companions. Both use the emitted companion list, filtered by `llmsTxt.include` and `llmsTxt.exclude`. If companions are disabled, `llms.txt` contains only the site title, tagline and optional header; no page entries or `llms-full.txt` are emitted.
 
-- `llms.txt` - sectioned index of every doc/blog page, in the format described at <https://llmstxt.org>:
+- `llms.txt` - sectioned index of included doc/blog companions, in the format described at <https://llmstxt.org>. Links are absolute (`https://site/foo.md`) since 0.5.0, as the spec's examples are; the file is read away from the site, where a relative link has no base:
 
   ```text
   # StackQL
@@ -75,15 +103,21 @@ After feature 1 emits all companions, the plugin writes two files to the build r
 
   ## Documentation
 
-  - [Getting started](/docs/intro/index.md): Install StackQL and run your first query.
-  - [Providers](/docs/providers/index.md): Catalog of supported cloud APIs.
+  - [Getting started](https://stackql.io/docs/intro/index.md): Install StackQL and run your first query.
+  - [Providers](https://stackql.io/docs/providers/index.md): Catalog of supported cloud APIs.
 
   ## Blog
 
-  - [Querying Snowflake with StackQL](/blog/snowflake/index.md): ...
+  - [Querying Snowflake with StackQL](https://stackql.io/blog/snowflake/index.md): ...
+
+  ## Optional
+
+  - [Full text of every page](https://stackql.io/llms-full.txt): one file with the content of every page listed above
   ```
 
-- `llms-full.txt` - every companion concatenated with a `\n---\n\n` separator, each block prefixed with the page's title and URL so an LLM can cite individual sections.
+  The `## Optional` section is the llmstxt.org convention for secondary resources a reader can skip; it points at `llms-full.txt` whenever that file is emitted (`llmsTxt.linkFullTxt`, default `true`).
+
+- `llms-full.txt` - the included companions concatenated with a `\n---\n\n` separator, each block carrying a `Source: <page URL>` line and the page's title (a plain companion already opens with both; a raw one has them added) so an LLM can cite individual sections. Limit it to chosen content instances or a byte budget with the object form of `llmsTxt.fullTxt` (below).
 
 Options:
 
@@ -92,16 +126,17 @@ Options:
 | `llmsTxt.enabled` | `true` | Master switch. |
 | `llmsTxt.exclude` | `["/search", "/404", "/blog/tags/**", "/blog/page/**", "/blog/archive", "/blog/authors/**"]` | Route glob patterns to skip. |
 | `llmsTxt.include` | `null` | If set, only routes matching at least one pattern are included. Use for opt-in mode. |
-| `llmsTxt.header` | `null` | String prepended to `llms.txt` above the section list. Good for a project intro paragraph or links to key external resources. |
+| `llmsTxt.header` | `null` | String inserted after the site title/tagline and before the section list. Good for a project intro paragraph or links to key external resources. |
 | `llmsTxt.sections` | `{ docs: 'Documentation', blog: 'Blog', pages: 'Pages' }` | Section title overrides for the default per-type grouping. |
 | `llmsTxt.instanceSections` | `null` | Per-content-plugin-instance section map. When set, supersedes `sections` (see below). |
-| `llmsTxt.fullTxt` | `true` | Emit `llms-full.txt`. |
+| `llmsTxt.fullTxt` | `true` | Emit `llms-full.txt`. `true` is every section, uncapped. The object form `{ include?: string[], maxBytes?: number }` limits it: `include` names the content instances to concatenate (`'<pluginName>@<pluginId>'` keys, e.g. `'docusaurus-plugin-content-docs@ai'`; with per-type grouping the keys are `docs`, `blog`, `pages`), in `llms.txt` section order; `maxBytes` stops appending once the next page would cross the cap, so the file always ends on a whole page. The budget includes section headings and separators. If the first page exceeds it, the file is empty. An omitted or empty `include` selects all sections. |
+| `llmsTxt.linkFullTxt` | `true` | Add the `## Optional` section to `llms.txt` with a link to `llms-full.txt`. |
 
 Glob patterns are matched against route permalinks. `**` matches across path segments; `*` matches within a single segment.
 
 ### Per-instance sectioning
 
-By default, `llms.txt` groups by content-plugin TYPE: every `@docusaurus/plugin-content-docs` instance lands under one `## Documentation` heading, every blog instance under `## Blog`, and so on. This is fine for sites with a single docs instance, but it breaks down for sites that run multiple docs instances with different audiences — for example, one for human-facing docs at `/docs/*` and one for AI-targeted reference content at `/ai/*`. In that case, `llms.txt` and `llms-full.txt` interleave both surfaces under one heading, hiding the corpus shape from the crawlers and agents the file exists to serve.
+By default, `llms.txt` groups by content-plugin TYPE: every `@docusaurus/plugin-content-docs` instance lands under one `## Documentation` heading, every blog instance under `## Blog`, and so on. This is fine for sites with a single docs instance, but it breaks down for sites that run multiple docs instances with different audiences - for example, one for human-facing docs at `/docs/*` and one for AI-targeted reference content at `/ai/*`. In that case, `llms.txt` and `llms-full.txt` interleave both surfaces under one heading, hiding the corpus shape from the crawlers and agents the file exists to serve.
 
 Set `llmsTxt.instanceSections` to switch to per-instance grouping. Keys are `"${pluginName}@${pluginId}"`; each value is `{ title, order? }`.
 
@@ -146,31 +181,35 @@ Resulting `llms.txt`:
 
 ## Documentation
 
-- [Getting started](/docs/intro/index.md): Install and run your first query.
-- [Providers](/docs/providers/index.md): Catalog of supported cloud APIs.
+- [Getting started](https://your-site.example/docs/intro/index.md): Install and run your first query.
+- [Providers](https://your-site.example/docs/providers/index.md): Catalog of supported cloud APIs.
 
 ## AI Reference
 
-- [What is StackQL](/ai/faqs/what-is-stackql/index.md): Canonical one-paragraph definition.
-- [Connecting to AWS](/ai/howto/connect-to-aws/index.md): Step-by-step authentication.
+- [What is StackQL](https://your-site.example/ai/faqs/what-is-stackql/index.md): Canonical one-paragraph definition.
+- [Connecting to AWS](https://your-site.example/ai/howto/connect-to-aws/index.md): Step-by-step authentication.
 
 ## Blog
 
-- [Querying Snowflake](/blog/snowflake/index.md): ...
+- [Querying Snowflake](https://your-site.example/blog/snowflake/index.md): ...
+
+## Optional
+
+- [Full text of every page](https://your-site.example/llms-full.txt): one file with the content of every page listed above
 ```
 
 `llms-full.txt` mirrors the same section structure: each section's title is emitted as an `## H2` heading above its concatenated companion blocks, in the same order.
 
 Behavior:
 
-- Sections appear in ascending `order`. Ties break alphabetically by title. Entries with no `order` go last.
+- Non-empty sections appear in ascending `order`. Ties break alphabetically by title. Sections with no `order` go last. Pages within each mapped section are sorted by permalink.
 - Instances not present in `instanceSections` are collected into a single appended `## Other` section, sorted alphabetically by page title. When `verbose: true`, each unmapped instance name is logged once so you notice and can map it.
 - `llmsTxt.sections` (the per-type titles) is ignored when `instanceSections` is set. When `verbose: true`, the plugin logs a one-line notice if both are configured.
-- When `instanceSections` is `null` (default), behavior is identical to v0.1.x: per-type grouping using `llmsTxt.sections` titles.
+- When `instanceSections` is `null` (default), grouping is by content type using `llmsTxt.sections` titles.
 
 ## Feature 3: "Ask AI" button
 
-An outlined pill button with a caret reading "Ask AI about this page" is injected at the top of every doc and blog-post content area, right-aligned in the breadcrumb row. Each item in the dropdown opens the corresponding AI surface in a new tab with a prefilled prompt that references the current page's `.md` companion. The button is hidden on viewports under 997px to keep the breadcrumb row uncluttered on mobile.
+An outlined pill button with a caret reading "Ask AI about this page" appears above eligible doc and blog-post content. Each item in the dropdown opens the corresponding AI surface in a new tab with a prefilled prompt that references the current page's `.md` companion. The button is hidden on viewports under 997px to keep the breadcrumb row uncluttered on mobile.
 
 Placement specifics:
 
@@ -188,11 +227,21 @@ Providers and URL patterns:
 Default prompt:
 
 ```text
-Read https://your-site.example/path/to/page.md and help me understand it. Summarize the key points, then ask me one clarifying question to dig deeper.
+Read https://your-site.example/path/to/page/index.md and help me understand it. Summarize the key points, then ask me one clarifying question to dig deeper.
 ```
 
 The default is self-contained - submitting it as-is yields a useful summary plus a follow-up question. The prompt is prefilled in each provider's input box, so users can still edit or replace it before sending.
 
+Template placeholders:
+
+| Placeholder | Value |
+| --- | --- |
+| `{companionUrl}` | The page's `.md` companion, by the same rule feature 1 writes it: `https://site/index.md` for the homepage, `https://site/foo.md` or `https://site/foo/index.md` depending on `trailingSlash`. With feature 1 disabled, the page URL. |
+| `{pageUrl}` | The page itself: canonical URL, no trailing slash. |
+| `{pageUrl}.md` | Still honoured for templates written against v0.4, and resolves to `{companionUrl}`. (On its own it produced `https://site.md` for the homepage and the wrong path on sites that build `/foo/index.html`.) |
+
+The button only renders on pages that have a companion. Generated category index pages, React pages and blog list pages have no `.md` file, so on those the button is not shown rather than sending the reader to a 404. (The list of routes comes from `allContentLoaded`; with feature 1 disabled, the companion-route filter is disabled and the prompt uses the page URL wherever the button is rendered.)
+
 Provider icons are hand-rolled inline SVGs in each provider's brand color (Claude `#D97757`, ChatGPT `#000000`, Perplexity `#21808D`). The Claude and Perplexity SVG paths come from [simple-icons](https://simpleicons.org) (CC0-1.0); the OpenAI mark is sourced from the @lobehub/icons project. Bundle cost is ~3KB across the three components combined.
 
 > **Gemini was previously supported but was removed in 0.4.2** because `gemini.google.com` silently ignores URL-encoded prompts, leaving users on an empty prompt box. There is no documented Gemini URL prefill API and Google has not indicated one is coming.
@@ -204,15 +253,15 @@ Options:
 | Option | Default | Description |
 | --- | --- | --- |
 | `askAi.enabled` | `true` | When `false`, the theme components are not registered. |
-| `askAi.providerOrder` | `['claude', 'chatgpt', 'perplexity']` | Order of items in the dropdown. Drop entries to hide them. Valid values: `'claude'`, `'chatgpt'`, `'perplexity'`. |
-| `askAi.promptTemplate` | `'Read {pageUrl}.md and help me understand it. Summarize the key points, then ask me one clarifying question to dig deeper.'` | Prompt sent to each provider. `{pageUrl}` is the page's canonical URL (no trailing slash). If feature 1 is disabled, the consumer should remove the `.md` from the template. |
-| `askAi.placement` | `'breadcrumb-row'` | `'breadcrumb-row'` puts the button at the top of every doc/blog page (docs breadcrumb row, or above the blog title). `'none'` does not register any theme components - swizzle the button into your preferred location manually. |
+| `askAi.providerOrder` | `['claude', 'chatgpt', 'perplexity']` | Order of items in the dropdown. Omit providers to hide them; an empty array falls back to the default order. Valid values: `'claude'`, `'chatgpt'`, `'perplexity'`. |
+| `askAi.promptTemplate` | `'Read {companionUrl} and help me understand it. Summarize the key points, then ask me one clarifying question to dig deeper.'` | Prompt sent to each provider. `{companionUrl}` is the page's `.md` companion (the page URL when feature 1 is disabled); `{pageUrl}` is the page's canonical URL (no trailing slash). See the placeholder table above. |
+| `askAi.placement` | `'breadcrumb-row'` | `'breadcrumb-row'` puts the button at the top of every doc/blog page (docs breadcrumb row, or above the blog title). `'none'` disables all bundled theme components, including the `@theme/AskAiButton` alias. See custom placement below. |
 
 The button is built from [MUI](https://mui.com) primitives - outlined `Button` with a `KeyboardArrowDownIcon` caret as the trigger, and a `Menu` of `MenuItem` rows for the providers. Theming reads `--ifm-color-primary` and `--ifm-font-family-base` via MUI's `sx` prop, so dark/light mode work automatically. The MUI `Menu` handles click-outside-to-close and Esc-to-close natively.
 
 ### Peer dependencies
 
-When `askAi.enabled` is `true` (the default), the following peer dependencies must be installed by the consumer site:
+The package declares these MUI/Emotion peers as required, in addition to Docusaurus `^3.0.0` and React/React DOM `^18.0.0 || ^19.0.0`:
 
 | Package | Range |
 | --- | --- |
@@ -221,19 +270,21 @@ When `askAi.enabled` is `true` (the default), the following peer dependencies mu
 | `@emotion/react` | `^11.0.0` |
 | `@emotion/styled` | `^11.0.0` |
 
-These are declared as peer dependencies (not direct dependencies) so consumers that already use MUI - common in Docusaurus sites - get a single deduped copy at install time. Consumers without MUI will get a clean npm peer-dependency error pointing at exactly what to install.
+These are peer dependencies so the plugin can share the site's MUI installation. npm can install missing peers automatically; other package managers may require you to add them explicitly.
 
-Consumers who disable the Ask AI button (`askAi.enabled: false`) can ignore these peers; the theme components are not registered in that case and MUI is never imported.
+Disabling Ask AI (`askAi.enabled: false`) prevents the plugin from registering its theme components, so they do not import MUI at runtime. It does not change the package's required peer declarations or the package manager's installation requirements.
 
 ### Customizing placement
 
-To put the button somewhere other than the breadcrumb row - sidebar, header, a specific page region - set `askAi.placement: 'none'` to disable the bundled swizzles, then import and place the component manually wherever you want:
+`askAi.placement: 'none'` disables the entire theme path, so `@theme/AskAiButton` is unavailable with that setting.
+
+To reuse the bundled button in a custom location, keep Ask AI enabled with `placement: 'breadcrumb-row'`. Add site-level theme overrides for `DocBreadcrumbs` and `BlogPostItem/Header/Title` that omit the default button, then import it in your chosen site component:
 
 ```jsx
 import AskAiButton from '@theme/AskAiButton';
 ```
 
-Wrap an existing theme component (e.g. `@theme/Layout`) the same way Docusaurus documents for any swizzle.
+The button retains its companion-route filter and desktop-only styling in a custom location.
 
 ## Feature 4: `/ai/*` routing convention
 
@@ -241,7 +292,7 @@ This plugin documents but does not auto-configure a second docs instance for mac
 
 ```js
 // docusaurus.config.js
-const { sitemapExclude, AI_ROUTE_PATTERNS } = require('@stackql/docusaurus-plugin-aeo/helpers');
+const { sitemapExclude } = require('@stackql/docusaurus-plugin-aeo/helpers');
 
 module.exports = {
   plugins: [
@@ -292,11 +343,11 @@ ai-content/
     └── stackql-cli.md         # frontmatter: softwareApplication: { ... }
 ```
 
-The companion files emitted by feature 1 will live at `/ai/faqs/<slug>/index.md` (etc.), and they will appear in `llms.txt` and `llms-full.txt` just like any other doc, which is the point.
+With default filters, these docs appear in `llms.txt` and `llms-full.txt`. Their companions follow the same URL rule: `/ai/faqs/<slug>/index.md`, or `/ai/faqs/<slug>.md` when `trailingSlash: false`.
 
 ### Optional validation
 
-Set `aiRoutes.validate: true` to fail the build (with warnings, not errors) when files under `ai/faqs/`, `ai/howto/`, or `ai/apps/` are missing the corresponding frontmatter payload (`faq`, `howTo`, `softwareApplication`). Off by default.
+Set `aiRoutes.validate: true` to warn when docs at `/ai/faqs/*`, `/ai/howto/*`, or `/ai/apps/*` lack the corresponding frontmatter payload (`faq`, `howTo`, `softwareApplication`). Validation uses route permalinks, not source directories, and does not fail the build. Set `verbose: true` for per-page details. Off by default.
 
 ### Helpers
 
@@ -304,7 +355,7 @@ Set `aiRoutes.validate: true` to fail the build (with warnings, not errors) when
 const {
   AI_ROUTE_PATTERNS,             // ['/ai', '/ai/**'] - glob patterns
   sitemapExclude,                // { ignorePatterns: AI_ROUTE_PATTERNS }
-  isAiRoute,                     // (permalink) => boolean
+  isAiRoute,                     // true for /ai/*; false for /ai itself
   buildStructuredDataAiExcludes, // (routePaths[]) => string[]
 } = require('@stackql/docusaurus-plugin-aeo/helpers');
 ```
@@ -335,8 +386,10 @@ The `sitemapExclude` helper, by contrast, does work with globs because `@docusau
   // Feature 1
   companions: {
     enabled: true,            // emit .md siblings
-    format: 'raw',            // 'raw' | 'plain'
+    format: 'plain',          // 'plain' ('clean' is a synonym) | 'raw'
     exclude: [],              // route glob patterns to skip
+    alternateLink: true,      // <link rel="alternate" type="text/markdown"> in each page
+    linkHeader: false,        // also a Link: header via a Netlify _headers file
   },
 
   // Feature 2
@@ -351,7 +404,7 @@ The `sitemapExclude` helper, by contrast, does work with globs because `@docusau
       '/blog/authors/**',
     ],
     include: null,            // null = all not-excluded
-    header: null,             // optional string prepended to llms.txt
+    header: null,             // optional text after the site title/tagline
     sections: {               // per-type section titles (used when
       docs: 'Documentation',  // instanceSections is null)
       blog: 'Blog',
@@ -361,14 +414,16 @@ The `sitemapExclude` helper, by contrast, does work with globs because `@docusau
                               // sections when set. Shape:
                               //   { '${pluginName}@${pluginId}':
                               //       { title: string, order?: number } }
-    fullTxt: true,            // emit llms-full.txt
+    fullTxt: true,            // emit llms-full.txt: true | false |
+                              //   { include: ['<plugin>@<id>'], maxBytes: 65536 }
+    linkFullTxt: true,        // "## Optional" section in llms.txt -> llms-full.txt
   },
 
   // Feature 3
   askAi: {
     enabled: true,
     providerOrder: ['claude', 'chatgpt', 'perplexity'],
-    promptTemplate: 'Read {pageUrl}.md and help me understand it. Summarize the key points, then ask me one clarifying question to dig deeper.',
+    promptTemplate: 'Read {companionUrl} and help me understand it. Summarize the key points, then ask me one clarifying question to dig deeper.',
     placement: 'breadcrumb-row',  // 'breadcrumb-row' | 'none'
   },
 
@@ -384,12 +439,27 @@ The `sitemapExclude` helper, by contrast, does work with globs because `@docusau
 
 Options are validated by a small handwritten validator at plugin construction. Invalid options throw a clear error before the build starts.
 
+## Development
+
+Use Node.js 24.x to match CI. From the repository root:
+
+```bash
+npm ci
+npm test
+```
+
+`npm test` runs every `test/*.test.js` file with Node's test runner: companion paths, Ask AI prompts, plain Markdown, alternate links/headers, `llms.txt`, and AI route classification.
+
+The [Tests workflow](.github/workflows/tests.yml) runs on every pull request targeting `main` and on pushes to `main`, using the committed lockfile and Node 24.x. To make it a merge gate, require the **Tests (Node 24)** status check in the branch protection rule or ruleset for `main`. Until that rule is enabled, a failed check does not block merging.
+
+The plain-markdown converter lives in `src/features/plainMarkdown.js` and is exercised by `test/plainMarkdown.test.js` with small MDX fixtures; add a fixture there when a new construct needs handling.
+
 ## Composing with `@stackql/docusaurus-plugin-structured-data`
 
 The two plugins are complementary and intended to be used together:
 
 - `@stackql/docusaurus-plugin-structured-data` emits JSON-LD into page `<head>`. <https://github.com/stackql/docusaurus-plugin-structured-data>
-- `@stackql/docusaurus-plugin-aeo` does everything else AEO-adjacent: raw `.md` companions, `llms.txt`, the Ask AI button, the `/ai/*` convention.
+- `@stackql/docusaurus-plugin-aeo` provides plain-markdown `.md` companions by default (raw mode is optional), `llms.txt` / `llms-full.txt`, the Ask AI button, and the `/ai/*` convention.
 
 There is no overlap and no shared state. Add both:
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@stackql/docusaurus-plugin-aeo",
-  "version": "0.4.2",
+  "version": "0.5.0",
   "description": "AEO (Answer Engine Optimization) helpers for Docusaurus: .md companion files, llms.txt / llms-full.txt, an Ask AI dropdown, and /ai/* route conventions.",
   "main": "src/index.js",
   "exports": {
@@ -17,7 +17,7 @@
     "access": "public"
   },
   "scripts": {
-    "test": "echo \"Error: no test specified\" && exit 1"
+    "test": "node --test test/*.test.js"
   },
   "peerDependencies": {
     "@docusaurus/core": "^3.0.0",
@@ -43,11 +43,21 @@
     }
   },
   "dependencies": {
-    "gray-matter": "^4.0.3"
+    "@slorber/remark-comment": "^1.0.0",
+    "gray-matter": "npm:@11ty/gray-matter@1.0.0",
+    "remark-directive": "^3.0.0",
+    "remark-gfm": "^4.0.0",
+    "remark-mdx": "^3.0.0",
+    "remark-parse": "^11.0.0",
+    "unified": "^11.0.3"
   },
   "overrides": {
     "serialize-javascript": "^7.0.5",
-    "uuid": "^11.1.1"
+    "uuid": "^11.1.1",
+    "@docusaurus/core": {
+      "tinypool": "2.1.2"
+    },
+    "postcss-selector-parser": "7.1.6"
   },
   "repository": {
     "type": "git",
@@ -67,5 +77,8 @@
   "bugs": {
     "url": "https://github.com/stackql-labs/docusaurus-plugin-aeo/issues"
   },
-  "homepage": "https://github.com/stackql-labs/docusaurus-plugin-aeo#readme"
+  "homepage": "https://github.com/stackql-labs/docusaurus-plugin-aeo#readme",
+  "engines": {
+    "node": ">=18.0"
+  }
 }

package/src/askAiPrompt.js

@@ -0,0 +1,21 @@
+// Fills the Ask AI prompt template. Plain CommonJS so the theme component
+// can bundle it and the tests can require it.
+//
+//   {companionUrl}  the page's .md companion (absolute URL)
+//   {pageUrl}       the page itself (canonical, no trailing slash)
+//
+// The v0.4 idiom `{pageUrl}.md` is still honoured so a custom template
+// keeps working. It was wrong for the homepage (`https://site.md`) and for
+// sites that build /foo/index.html; it now resolves to the real companion
+// URL.
+function fillPrompt(template, { pageUrl, companionUrl }) {
+  return String(template || '')
+    .split('{pageUrl}.md')
+    .join(companionUrl)
+    .split('{companionUrl}')
+    .join(companionUrl)
+    .split('{pageUrl}')
+    .join(pageUrl);
+}
+
+module.exports = { fillPrompt };

package/src/companionPath.js

@@ -0,0 +1,55 @@
+// The one rule for where a page's `.md` companion lives, shared by the
+// emitter (feature 1), llms.txt (feature 2) and the Ask AI button (feature 3)
+// so the three can never disagree.
+//
+// Docusaurus emits either /foo/index.html (trailingSlash true or undefined)
+// or /foo.html (trailingSlash false); the companion mirrors the HTML:
+//
+//   route            trailingSlash: false   trailingSlash: true / undefined
+//   /                /index.md              /index.md
+//   /foo             /foo.md                /foo/index.md
+//   /foo/            /foo.md                /foo/index.md
+//   /docs/intro      /docs/intro.md         /docs/intro/index.md
+//
+// Plain CommonJS with no dependencies: required by the Node-side features
+// and bundled into the theme component by webpack.
+
+function normalizeRoute(routePath) {
+  const cleaned = String(routePath || '').replace(/\/+$/, '');
+  return cleaned === '' ? '/' : cleaned;
+}
+
+// Site-relative companion path for a route, e.g. '/foo.md'. Always starts
+// with '/'.
+function companionPath(routePath, trailingSlash) {
+  const route = normalizeRoute(routePath);
+  if (route === '/') return '/index.md';
+  return trailingSlash === false ? `${route}.md` : `${route}/index.md`;
+}
+
+// Absolute companion URL for a route, e.g. 'https://example.com/foo.md'.
+// `siteUrl` is siteConfig.url; the route already carries siteConfig.baseUrl
+// (Docusaurus permalinks and location.pathname both include it).
+function companionUrl(siteUrl, routePath, trailingSlash) {
+  return `${String(siteUrl || '').replace(/\/+$/, '')}${companionPath(routePath, trailingSlash)}`;
+}
+
+// Site-relative path of the HTML file Docusaurus writes for a route, the
+// same rule as @docusaurus/core's pathnameToFilename once the route has
+// been normalised: '/foo.html' when trailingSlash is false, '/foo/index.html'
+// otherwise, '/index.html' for the root.
+function htmlPath(routePath, trailingSlash) {
+  const route = normalizeRoute(routePath);
+  if (route === '/') return '/index.html';
+  return trailingSlash === false ? `${route}.html` : `${route}/index.html`;
+}
+
+// Absolute URL of the page itself: canonical, no trailing slash except on
+// the root.
+function pageUrl(siteUrl, routePath) {
+  const base = String(siteUrl || '').replace(/\/+$/, '');
+  const route = normalizeRoute(routePath);
+  return route === '/' ? `${base}/` : `${base}${route}`;
+}
+
+module.exports = { companionPath, companionUrl, htmlPath, pageUrl, normalizeRoute };

package/src/features/aiRoutes.js

@@ -5,6 +5,7 @@
 //   ai/faqs/<slug>.md          -> must declare a `faq` frontmatter object
 //   ai/howto/<slug>.md         -> must declare a `howTo` frontmatter object
 //   ai/howtos/<slug>.md        -> same as above
+//   ai/how-tos/<slug>.md       -> same as above
 //   ai/apps/<slug>.md          -> must declare a `softwareApplication`
 //                                   frontmatter object
 //
@@ -16,6 +17,8 @@ const KIND_FOR_DIR = {
   faq: 'faq',
   howto: 'howTo',
   howtos: 'howTo',
+  'how-tos': 'howTo',
+  'how-to': 'howTo',
   apps: 'softwareApplication',
   app: 'softwareApplication',
   applications: 'softwareApplication',