@stackql/docusaurus-plugin-aeo 0.5.0 → 0.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +23 -0
- package/LICENSE +0 -0
- package/README.md +76 -12
- package/package.json +4 -1
- package/src/askAiPrompt.js +2 -1
- package/src/companionPath.js +51 -17
- package/src/features/aiRoutes.js +12 -6
- package/src/features/alternateLinks.js +47 -14
- package/src/features/companions.js +16 -43
- package/src/features/llmsTxt.js +14 -40
- package/src/features/plainMarkdown.js +0 -0
- package/src/helpers.js +34 -10
- package/src/index.js +8 -1
- package/src/routeGlob.js +60 -0
- package/src/theme/AskAiButton/brand-icons/ChatGpt.jsx +0 -0
- package/src/theme/AskAiButton/brand-icons/Claude.jsx +0 -0
- package/src/theme/AskAiButton/brand-icons/Perplexity.jsx +0 -0
- package/src/theme/AskAiButton/icons.js +0 -0
- package/src/theme/AskAiButton/index.jsx +37 -11
- package/src/theme/AskAiButton/styles.module.css +0 -0
- package/src/theme/BlogPostItem/Header/Title/index.jsx +0 -0
- package/src/theme/BlogPostItem/Header/Title/styles.module.css +0 -0
- package/src/theme/DocBreadcrumbs/index.jsx +0 -0
- package/src/theme/DocBreadcrumbs/styles.module.css +0 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,28 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.5.1
|
|
4
|
+
|
|
5
|
+
Localized sites (Docusaurus i18n) and sites with a non-root `baseUrl`. Raised against stackql.io when it gained a `de` locale.
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
- **Companions and alternate links on a site whose `baseUrl` is not `/`.** Docusaurus permalinks carry `siteConfig.baseUrl` but the build output tree is rooted at it, and in a localized build the plugin runs once per locale with `baseUrl: '/<locale>/'` and `outDir: build/<locale>`. Feature 1 joined `outDir` with the full permalink, so a `de` site wrote its companions to `build/de/de/foo.md`, and feature 1b looked for the page's HTML at the same wrong path and injected nothing. Both now strip the base URL from the route before touching the file system (`outputRoute` in `src/companionPath.js`); the companion URLs, the `Source:` lines, `llms.txt` and the `_headers` routes keep the full route as before. A localized build therefore emits `/<locale>/foo.md`, `/<locale>/llms.txt` and `/<locale>/llms-full.txt` from the translated sources with no configuration.
|
|
10
|
+
- **Route patterns in a localized build or under a non-root `baseUrl`.** `companions.exclude`, `llmsTxt.exclude` and `llmsTxt.include` were matched against the full permalink, which carries `siteConfig.baseUrl`: `exclude: ['/docs/internal/**']` skipped the page in the default locale and kept `/de/docs/internal/...` in the `de` build, and the default `llmsTxt.exclude` list had the same gap. A pattern now matches a route with or without the base URL in front of it (`src/routeGlob.js`), so one configuration covers every locale. A pattern that spells the base out (`/de/blog/**`, or `/docs/guide/**` on a site built at `/docs/`) matches as before and can single out one locale. The Ask AI button follows, since it shows on the routes that have a companion.
|
|
11
|
+
- **`aiRoutes.validate` under a base URL.** The `/ai/*` convention was looked for at the start of the permalink, so pages at `/de/ai/...` or `/docs/ai/...` were never validated. It is now relative to `siteConfig.baseUrl`.
|
|
12
|
+
- **`_headers` for a non-default locale** (`companions.linkHeader`). Each build wrote its rules to `<outDir>/_headers`, which for a non-default locale is `build/<locale>/_headers`. Netlify and Cloudflare Pages document the file at the root of the published directory, so rules in a sub-directory cannot be relied on. In a build of several locales, the rules of every locale now go in the one `_headers` at the build root (Docusaurus builds the default locale first, so the file is there). A locale built on its own (`docusaurus build --locale de`) or given its own `baseUrl` keeps the file in its `outDir`. A rule already in the file is not appended a second time.
|
|
13
|
+
|
|
14
|
+
### Changed
|
|
15
|
+
|
|
16
|
+
- The Ask AI button's visible strings ("Ask AI about this page", "Ask Claude", "Ask ChatGPT", "Ask Perplexity") go through `translate()` with the ids `aeo.askAi.button` and `aeo.askAi.provider.<key>`, so `docusaurus write-translations` extracts them into `i18n/<locale>/code.json` and a localized site can translate them.
|
|
17
|
+
- `{pageUrl}` in the Ask AI prompt ends with a slash on a root: the site root (`https://site/`, previously `https://site`) and a base URL such as `https://site/de/`, as the `Source:` lines already did. Other pages are unchanged (no trailing slash), and so is the default prompt, which uses `{companionUrl}`.
|
|
18
|
+
- The Ask AI button's hover background is a 4% tint of the site's `--ifm-color-primary` instead of a fixed colour.
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
|
|
22
|
+
- `aiRoutePatterns(baseUrl)` in `@stackql/docusaurus-plugin-aeo/helpers`, and an optional `baseUrl` argument on `isAiRoute` and `buildStructuredDataAiExcludes`. `AI_ROUTE_PATTERNS` and `sitemapExclude` are unchanged and describe a site at the root with one locale; a site with a non-root `baseUrl` or several locales builds its list with `['/', '/de/'].flatMap(aiRoutePatterns)`.
|
|
23
|
+
- README: hosting notes. Which hosts `companions.linkHeader` applies to (those that read `_headers`: Netlify, Cloudflare Pages, with Cloudflare's 100-rule limit), where the file is written, `.nojekyll` on GitHub Pages, and absolute URLs on preview deployments.
|
|
24
|
+
- Tests for all of the above and for the rest of this release: companion files and `llms.txt` under a base URL, the plugin run once per locale, and the Ask AI button rendered in Node (`test/companions.test.js`, `test/localizedBuild.test.js`, `test/askAiButton.test.js`, `test/routeGlob.test.js`, `test/aiHelpers.test.js`).
|
|
25
|
+
|
|
3
26
|
## 0.5.0
|
|
4
27
|
|
|
5
28
|
Plain-markdown companions, one rule for where a companion lives, and the companion advertised from its page. Raised against stackql.io, where the Ask AI prompt on the homepage read `https://stackql.io.md` and every `.md` URL returned raw MDX (imports, MUI components, tab definitions and inline SVG path data before the first sentence), and from an independent agent-readiness audit of the built site (alternate link, `llms-full.txt` reference and size, `Source:` URLs, the `how-tos` alias).
|
package/LICENSE
CHANGED
|
File without changes
|
package/README.md
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
|
-
[](https://www.npmjs.com/package/@stackql/docusaurus-plugin-aeo)
|
|
1
|
+
[](https://www.npmjs.com/package/@stackql/docusaurus-plugin-aeo)
|
|
2
|
+
[](https://www.npmjs.com/package/@stackql/docusaurus-plugin-aeo)
|
|
2
3
|
|
|
3
4
|
# @stackql/docusaurus-plugin-aeo
|
|
4
5
|
|
|
@@ -47,7 +48,7 @@ module.exports = {
|
|
|
47
48
|
|
|
48
49
|
## Feature 1: `.md` companion files
|
|
49
50
|
|
|
50
|
-
For each doc or blog post with a readable source file, this plugin writes a sibling `.md` file unless its
|
|
51
|
+
For each doc or blog post with a readable source file, this plugin writes a sibling `.md` file unless its route matches `companions.exclude` (see [Route patterns](#route-patterns)). The companion mirrors the HTML layout: a page at `/docs/intro` gets `/docs/intro/index.md` (or `/docs/intro.md` if `siteConfig.trailingSlash` is `false`). A doc or blog post at the site root gets `/index.md`. The same URL rule (`src/companionPath.js`) is used by `llms.txt` and the Ask AI button.
|
|
51
52
|
|
|
52
53
|
Two modes:
|
|
53
54
|
|
|
@@ -86,10 +87,33 @@ inserted into the built HTML in `postBuild` (the same pattern `@stackql/docusaur
|
|
|
86
87
|
| Option | Default | Description |
|
|
87
88
|
| --- | --- | --- |
|
|
88
89
|
| `companions.alternateLink` | `true` | Insert the `<link rel="alternate" type="text/markdown">` tag into each page that has a companion. |
|
|
89
|
-
| `companions.linkHeader` | `false` | Also
|
|
90
|
+
| `companions.linkHeader` | `false` | Also send the relation as an HTTP response header, on hosts that read a `_headers` file (Netlify, Cloudflare Pages): one `Link: <url>; rel="alternate"; type="text/markdown"` rule per page, written to `_headers` in the build root (appended to a `_headers` your `static/` already ships). It has no effect on other hosts; see [Hosting notes](#hosting-notes). Requires `companions.alternateLink: true`. |
|
|
90
91
|
|
|
91
92
|
Generated category indexes, custom React pages and blog listing pages do not get companions. `verbose: true` logs unreadable source files and missing HTML during alternate-link insertion.
|
|
92
93
|
|
|
94
|
+
### Localized sites and non-root `baseUrl`
|
|
95
|
+
|
|
96
|
+
The plugin works from `siteConfig.url` and `siteConfig.baseUrl` as Docusaurus reports them for the build, so a site at the root, a site built at `/docs/` and a site with several locales are configured the same way.
|
|
97
|
+
|
|
98
|
+
On a site with Docusaurus i18n the plugin runs once per locale, like every plugin, and needs no extra configuration: each locale gets its own companions (`/de/docs/intro.md` from the translated source, or from the default-locale source when a page has no translation), its own `/de/llms.txt` and `/de/llms-full.txt`, and alternate links and `Source:` lines that carry the locale prefix. The localized home page `/de/` is a root, so its companion is `/de/index.md`. The same holds for a site built at a non-root `baseUrl`: files are written relative to the build directory, URLs carry the base.
|
|
99
|
+
|
|
100
|
+
### Route patterns
|
|
101
|
+
|
|
102
|
+
`companions.exclude`, `llmsTxt.exclude` and `llmsTxt.include` take glob patterns for routes. `**` matches across path segments; `*` and `?` match within a single segment.
|
|
103
|
+
|
|
104
|
+
Write a pattern for the route as it is on the site, without the base URL: `/docs/internal/**`. Docusaurus permalinks carry the base URL (`/de/docs/internal/notes` in a `de` build, `/docs/guide/setup` on a site built at `/docs/`), so a pattern is matched against the route both with and without it. One list therefore applies to every locale, and a pattern that includes the base still works and can target a single locale:
|
|
105
|
+
|
|
106
|
+
```js
|
|
107
|
+
companions: {
|
|
108
|
+
exclude: [
|
|
109
|
+
'/docs/internal/**', // every locale: /docs/internal/..., /de/docs/internal/...
|
|
110
|
+
'/de/blog/**', // the de locale only
|
|
111
|
+
],
|
|
112
|
+
},
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
A page excluded from companions has no `.md` file, no alternate link, no `llms.txt` entry and no Ask AI button.
|
|
116
|
+
|
|
93
117
|
## Feature 2: `llms.txt` and `llms-full.txt`
|
|
94
118
|
|
|
95
119
|
With the defaults enabled, the plugin writes two files to the build root after emitting companions. Both use the emitted companion list, filtered by `llmsTxt.include` and `llmsTxt.exclude`. If companions are disabled, `llms.txt` contains only the site title, tagline and optional header; no page entries or `llms-full.txt` are emitted.
|
|
@@ -132,7 +156,7 @@ Options:
|
|
|
132
156
|
| `llmsTxt.fullTxt` | `true` | Emit `llms-full.txt`. `true` is every section, uncapped. The object form `{ include?: string[], maxBytes?: number }` limits it: `include` names the content instances to concatenate (`'<pluginName>@<pluginId>'` keys, e.g. `'docusaurus-plugin-content-docs@ai'`; with per-type grouping the keys are `docs`, `blog`, `pages`), in `llms.txt` section order; `maxBytes` stops appending once the next page would cross the cap, so the file always ends on a whole page. The budget includes section headings and separators. If the first page exceeds it, the file is empty. An omitted or empty `include` selects all sections. |
|
|
133
157
|
| `llmsTxt.linkFullTxt` | `true` | Add the `## Optional` section to `llms.txt` with a link to `llms-full.txt`. |
|
|
134
158
|
|
|
135
|
-
|
|
159
|
+
`exclude` and `include` follow the rules in [Route patterns](#route-patterns): a pattern matches a route with or without the base URL, so the defaults apply to every locale.
|
|
136
160
|
|
|
137
161
|
### Per-instance sectioning
|
|
138
162
|
|
|
@@ -237,7 +261,7 @@ Template placeholders:
|
|
|
237
261
|
| Placeholder | Value |
|
|
238
262
|
| --- | --- |
|
|
239
263
|
| `{companionUrl}` | The page's `.md` companion, by the same rule feature 1 writes it: `https://site/index.md` for the homepage, `https://site/foo.md` or `https://site/foo/index.md` depending on `trailingSlash`. With feature 1 disabled, the page URL. |
|
|
240
|
-
| `{pageUrl}` | The page itself: canonical URL, no trailing slash. |
|
|
264
|
+
| `{pageUrl}` | The page itself: canonical URL, no trailing slash, except on a root (`https://site/`, or a base URL such as `https://site/de/`). |
|
|
241
265
|
| `{pageUrl}.md` | Still honoured for templates written against v0.4, and resolves to `{companionUrl}`. (On its own it produced `https://site.md` for the homepage and the wrong path on sites that build `/foo/index.html`.) |
|
|
242
266
|
|
|
243
267
|
The button only renders on pages that have a companion. Generated category index pages, React pages and blog list pages have no `.md` file, so on those the button is not shown rather than sending the reader to a 404. (The list of routes comes from `allContentLoaded`; with feature 1 disabled, the companion-route filter is disabled and the prompt uses the page URL wherever the button is rendered.)
|
|
@@ -254,10 +278,23 @@ Options:
|
|
|
254
278
|
| --- | --- | --- |
|
|
255
279
|
| `askAi.enabled` | `true` | When `false`, the theme components are not registered. |
|
|
256
280
|
| `askAi.providerOrder` | `['claude', 'chatgpt', 'perplexity']` | Order of items in the dropdown. Omit providers to hide them; an empty array falls back to the default order. Valid values: `'claude'`, `'chatgpt'`, `'perplexity'`. |
|
|
257
|
-
| `askAi.promptTemplate` | `'Read {companionUrl} and help me understand it. Summarize the key points, then ask me one clarifying question to dig deeper.'` | Prompt sent to each provider. `{companionUrl}` is the page's `.md` companion (the page URL when feature 1 is disabled); `{pageUrl}` is the page's canonical URL (no trailing slash). See the placeholder table above. |
|
|
281
|
+
| `askAi.promptTemplate` | `'Read {companionUrl} and help me understand it. Summarize the key points, then ask me one clarifying question to dig deeper.'` | Prompt sent to each provider. `{companionUrl}` is the page's `.md` companion (the page URL when feature 1 is disabled); `{pageUrl}` is the page's canonical URL (no trailing slash except on a root). See the placeholder table above. |
|
|
258
282
|
| `askAi.placement` | `'breadcrumb-row'` | `'breadcrumb-row'` puts the button at the top of every doc/blog page (docs breadcrumb row, or above the blog title). `'none'` disables all bundled theme components, including the `@theme/AskAiButton` alias. See custom placement below. |
|
|
259
283
|
|
|
260
|
-
The button is built from [MUI](https://mui.com) primitives - outlined `Button` with a `KeyboardArrowDownIcon` caret as the trigger, and a `Menu` of `MenuItem` rows for the providers. Theming reads `--ifm-color-primary` and `--ifm-font-family-base` via MUI's `sx` prop, so dark/light mode work automatically. The MUI `Menu` handles click-outside-to-close and Esc-to-close natively.
|
|
284
|
+
The button is built from [MUI](https://mui.com) primitives - outlined `Button` with a `KeyboardArrowDownIcon` caret as the trigger, and a `Menu` of `MenuItem` rows for the providers. Theming reads `--ifm-color-primary` and `--ifm-font-family-base` via MUI's `sx` prop, so the button takes the site's own colours and dark/light mode work automatically. The MUI `Menu` handles click-outside-to-close and Esc-to-close natively.
|
|
285
|
+
|
|
286
|
+
### Translating the button
|
|
287
|
+
|
|
288
|
+
The visible strings go through Docusaurus `translate()`, so `docusaurus write-translations --locale <locale>` extracts them into `i18n/<locale>/code.json` with the rest of the theme's strings:
|
|
289
|
+
|
|
290
|
+
| Id | Default |
|
|
291
|
+
| --- | --- |
|
|
292
|
+
| `aeo.askAi.button` | Ask AI about this page |
|
|
293
|
+
| `aeo.askAi.provider.claude` | Ask Claude |
|
|
294
|
+
| `aeo.askAi.provider.chatgpt` | Ask ChatGPT |
|
|
295
|
+
| `aeo.askAi.provider.perplexity` | Ask Perplexity |
|
|
296
|
+
|
|
297
|
+
A locale with no translation for an id shows the default. The prompt itself is `askAi.promptTemplate`; in a localized build its URLs point at that locale's page and companion.
|
|
261
298
|
|
|
262
299
|
### Peer dependencies
|
|
263
300
|
|
|
@@ -345,21 +382,36 @@ ai-content/
|
|
|
345
382
|
|
|
346
383
|
With default filters, these docs appear in `llms.txt` and `llms-full.txt`. Their companions follow the same URL rule: `/ai/faqs/<slug>/index.md`, or `/ai/faqs/<slug>.md` when `trailingSlash: false`.
|
|
347
384
|
|
|
385
|
+
The convention is relative to the site's base URL: on a site built at `/docs/` the routes are `/docs/ai/*`, and in a `de` build they are `/de/ai/*`.
|
|
386
|
+
|
|
348
387
|
### Optional validation
|
|
349
388
|
|
|
350
|
-
Set `aiRoutes.validate: true` to warn when docs at `/ai/faqs/*`, `/ai/howto/*`, or `/ai/apps/*` lack the corresponding frontmatter payload (`faq`, `howTo`, `softwareApplication`). Validation uses route permalinks, not source directories, and does not fail the build. Set `verbose: true` for per-page details. Off by default.
|
|
389
|
+
Set `aiRoutes.validate: true` to warn when docs at `/ai/faqs/*`, `/ai/howto/*`, or `/ai/apps/*` lack the corresponding frontmatter payload (`faq`, `howTo`, `softwareApplication`). Validation uses route permalinks (relative to the base URL, so every locale is checked), not source directories, and does not fail the build. Set `verbose: true` for per-page details. Off by default.
|
|
351
390
|
|
|
352
391
|
### Helpers
|
|
353
392
|
|
|
354
393
|
```js
|
|
355
394
|
const {
|
|
356
395
|
AI_ROUTE_PATTERNS, // ['/ai', '/ai/**'] - glob patterns
|
|
396
|
+
aiRoutePatterns, // (baseUrl) => the same patterns under a base URL
|
|
357
397
|
sitemapExclude, // { ignorePatterns: AI_ROUTE_PATTERNS }
|
|
358
|
-
isAiRoute, // true for /ai/*; false for /ai itself
|
|
359
|
-
buildStructuredDataAiExcludes, // (routePaths[]) => string[]
|
|
398
|
+
isAiRoute, // (route, baseUrl?) true for /ai/*; false for /ai itself
|
|
399
|
+
buildStructuredDataAiExcludes, // (routePaths[], baseUrl?) => string[]
|
|
360
400
|
} = require('@stackql/docusaurus-plugin-aeo/helpers');
|
|
361
401
|
```
|
|
362
402
|
|
|
403
|
+
`AI_ROUTE_PATTERNS` and `sitemapExclude` describe a site at the root with one locale. These helpers run in your config, where other plugins match the full route path, base URL included (the Docusaurus docs say as much of the sitemap plugin's `ignorePatterns`). With a non-root `baseUrl` or several locales, build the list from `aiRoutePatterns`, one call per base URL:
|
|
404
|
+
|
|
405
|
+
```js
|
|
406
|
+
const { aiRoutePatterns } = require('@stackql/docusaurus-plugin-aeo/helpers');
|
|
407
|
+
|
|
408
|
+
// a site with en at / and de at /de/
|
|
409
|
+
const ignorePatterns = ['/', '/de/'].flatMap(aiRoutePatterns);
|
|
410
|
+
// ['/ai', '/ai/**', '/de/ai', '/de/ai/**']
|
|
411
|
+
```
|
|
412
|
+
|
|
413
|
+
`isAiRoute` and `buildStructuredDataAiExcludes` take the base URL as an optional second argument.
|
|
414
|
+
|
|
363
415
|
**Heads-up on the structured-data plugin.** `@stackql/docusaurus-plugin-structured-data`'s `themeConfig.structuredData.excludedRoutes` compares routes with strict equality, so glob patterns like `/ai/**` do not work there. Use one of these approaches:
|
|
364
416
|
|
|
365
417
|
- List the exact `/ai/*` permalinks you want skipped by hand (works if your `/ai/*` tree is small and stable).
|
|
@@ -379,6 +431,16 @@ const {
|
|
|
379
431
|
|
|
380
432
|
The `sitemapExclude` helper, by contrast, does work with globs because `@docusaurus/plugin-sitemap` uses `ignorePatterns` (micromatch under the hood).
|
|
381
433
|
|
|
434
|
+
## Hosting notes
|
|
435
|
+
|
|
436
|
+
Everything the plugin produces is a static file or an edit to the built HTML, so companions, `llms.txt`, `llms-full.txt`, the alternate link tag and the Ask AI button work on any static host with no host configuration. A few things do depend on where the site is deployed:
|
|
437
|
+
|
|
438
|
+
- **`Content-Type` of `.md` files.** Set by the host; see the MIME type note under Feature 1.
|
|
439
|
+
- **GitHub Pages.** GitHub Pages runs published files through Jekyll by default. Jekyll builds sites from Markdown, so `.md` companions may be turned into pages instead of being published as they are. Add an empty `.nojekyll` file to `static/`, as the [Docusaurus deployment guide](https://docusaurus.io/docs/deployment) recommends. A project site is served under `/<repo>/`: set `baseUrl` as usual and the plugin follows it.
|
|
440
|
+
- **`companions.linkHeader` is for hosts that read `_headers`.** That is Netlify and Cloudflare Pages. Vercel documents response headers through `vercel.json`, and GitHub Pages has no server-side configuration for them, so the option does nothing there. The alternate link tag, on by default, needs nothing from the host and is the portable way to advertise a companion. The file gets one rule per page that has a companion; Cloudflare Pages documents a limit of 100 header rules, so leave the option off there on a larger site.
|
|
441
|
+
- **Where `_headers` is written.** At the root of the build directory, which is where both hosts document the file. A build of several locales (`docusaurus build`, one deployment serving `/` and `/de/`) puts every locale's rules in that one file. A locale built on its own (`docusaurus build --locale de`) or given its own `baseUrl` is a separate deployment and gets the file in its own output directory.
|
|
442
|
+
- **Preview deployments.** Companion URLs, `Source:` lines, `llms.txt` links and the Ask AI prompt are absolute and come from `siteConfig.url`, so on a deploy preview they point at the production site.
|
|
443
|
+
|
|
382
444
|
## Full options reference
|
|
383
445
|
|
|
384
446
|
```js
|
|
@@ -389,7 +451,7 @@ The `sitemapExclude` helper, by contrast, does work with globs because `@docusau
|
|
|
389
451
|
format: 'plain', // 'plain' ('clean' is a synonym) | 'raw'
|
|
390
452
|
exclude: [], // route glob patterns to skip
|
|
391
453
|
alternateLink: true, // <link rel="alternate" type="text/markdown"> in each page
|
|
392
|
-
linkHeader: false, // also a Link: header via a Netlify
|
|
454
|
+
linkHeader: false, // also a Link: header via a _headers file (Netlify, Cloudflare Pages)
|
|
393
455
|
},
|
|
394
456
|
|
|
395
457
|
// Feature 2
|
|
@@ -448,10 +510,12 @@ npm ci
|
|
|
448
510
|
npm test
|
|
449
511
|
```
|
|
450
512
|
|
|
451
|
-
`npm test` runs every `test/*.test.js` file with Node's test runner: companion paths, Ask AI prompts, plain Markdown, alternate links/headers, `llms.txt`,
|
|
513
|
+
`npm test` runs every `test/*.test.js` file with Node's test runner: companion paths and files, route patterns, Ask AI prompts and the rendered button, plain Markdown, alternate links/headers, `llms.txt`, AI route classification and helpers, and the plugin run once per locale as Docusaurus runs it (`test/localizedBuild.test.js`). Shared fixtures and the loader that renders theme components in Node are in `test/helpers/`. There are no dev dependencies: the tests use the peer dependencies and the Babel setup that comes with `@docusaurus/core`.
|
|
452
514
|
|
|
453
515
|
The [Tests workflow](.github/workflows/tests.yml) runs on every pull request targeting `main` and on pushes to `main`, using the committed lockfile and Node 24.x. To make it a merge gate, require the **Tests (Node 24)** status check in the branch protection rule or ruleset for `main`. Until that rule is enabled, a failed check does not block merging.
|
|
454
516
|
|
|
517
|
+
When `npm audit` reports advisories, they are almost always in the Docusaurus toolchain, which is installed here only because it is a peer dependency. Do not run `npm audit fix`: it clears the report by downgrading the `@docusaurus/core` peer to an older release with a different set of advisories. Update the named packages instead (`npm update <package> ...`), which moves them within the ranges Docusaurus already allows, then re-run `npm audit` and `npm test`.
|
|
518
|
+
|
|
455
519
|
The plain-markdown converter lives in `src/features/plainMarkdown.js` and is exercised by `test/plainMarkdown.test.js` with small MDX fixtures; add a fixture there when a new construct needs handling.
|
|
456
520
|
|
|
457
521
|
## Composing with `@stackql/docusaurus-plugin-structured-data`
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@stackql/docusaurus-plugin-aeo",
|
|
3
|
-
"version": "0.5.
|
|
3
|
+
"version": "0.5.1",
|
|
4
4
|
"description": "AEO (Answer Engine Optimization) helpers for Docusaurus: .md companion files, llms.txt / llms-full.txt, an Ask AI dropdown, and /ai/* route conventions.",
|
|
5
5
|
"main": "src/index.js",
|
|
6
6
|
"exports": {
|
|
@@ -80,5 +80,8 @@
|
|
|
80
80
|
"homepage": "https://github.com/stackql-labs/docusaurus-plugin-aeo#readme",
|
|
81
81
|
"engines": {
|
|
82
82
|
"node": ">=18.0"
|
|
83
|
+
},
|
|
84
|
+
"allowScripts": {
|
|
85
|
+
"core-js": false
|
|
83
86
|
}
|
|
84
87
|
}
|
package/src/askAiPrompt.js
CHANGED
|
@@ -2,7 +2,8 @@
|
|
|
2
2
|
// can bundle it and the tests can require it.
|
|
3
3
|
//
|
|
4
4
|
// {companionUrl} the page's .md companion (absolute URL)
|
|
5
|
-
// {pageUrl} the page itself (canonical
|
|
5
|
+
// {pageUrl} the page itself (canonical: no trailing slash, except
|
|
6
|
+
// on a root - the site root or a base URL such as '/de/')
|
|
6
7
|
//
|
|
7
8
|
// The v0.4 idiom `{pageUrl}.md` is still honoured so a custom template
|
|
8
9
|
// keeps working. It was wrong for the homepage (`https://site.md`) and for
|
package/src/companionPath.js
CHANGED
|
@@ -19,37 +19,71 @@ function normalizeRoute(routePath) {
|
|
|
19
19
|
return cleaned === '' ? '/' : cleaned;
|
|
20
20
|
}
|
|
21
21
|
|
|
22
|
+
// Every function below takes an optional `baseUrl` (siteConfig.baseUrl).
|
|
23
|
+
// Docusaurus permalinks and location.pathname carry it, and the part of
|
|
24
|
+
// the route that is the base is a root, not a page: on a site built at
|
|
25
|
+
// '/docs/' or in a localized build (baseUrl '/de/') the route '/de/' is the
|
|
26
|
+
// home page, whose companion is '/de/index.md', not '/de.md'. With the
|
|
27
|
+
// default '/' nothing changes.
|
|
28
|
+
function withBase(baseUrl, sitePath) {
|
|
29
|
+
return `${String(baseUrl || '/').replace(/\/+$/, '')}${sitePath}`;
|
|
30
|
+
}
|
|
31
|
+
|
|
22
32
|
// Site-relative companion path for a route, e.g. '/foo.md'. Always starts
|
|
23
33
|
// with '/'.
|
|
24
|
-
function companionPath(routePath, trailingSlash) {
|
|
25
|
-
const route =
|
|
26
|
-
|
|
27
|
-
|
|
34
|
+
function companionPath(routePath, trailingSlash, baseUrl) {
|
|
35
|
+
const route = outputRoute(routePath, baseUrl);
|
|
36
|
+
const rel = route === '/'
|
|
37
|
+
? '/index.md'
|
|
38
|
+
: trailingSlash === false ? `${route}.md` : `${route}/index.md`;
|
|
39
|
+
return withBase(baseUrl, rel);
|
|
28
40
|
}
|
|
29
41
|
|
|
30
42
|
// Absolute companion URL for a route, e.g. 'https://example.com/foo.md'.
|
|
31
|
-
// `siteUrl` is siteConfig.url
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
return `${String(siteUrl || '').replace(/\/+$/, '')}${companionPath(routePath, trailingSlash)}`;
|
|
43
|
+
// `siteUrl` is siteConfig.url.
|
|
44
|
+
function companionUrl(siteUrl, routePath, trailingSlash, baseUrl) {
|
|
45
|
+
return `${String(siteUrl || '').replace(/\/+$/, '')}${companionPath(routePath, trailingSlash, baseUrl)}`;
|
|
35
46
|
}
|
|
36
47
|
|
|
37
48
|
// Site-relative path of the HTML file Docusaurus writes for a route, the
|
|
38
49
|
// same rule as @docusaurus/core's pathnameToFilename once the route has
|
|
39
50
|
// been normalised: '/foo.html' when trailingSlash is false, '/foo/index.html'
|
|
40
51
|
// otherwise, '/index.html' for the root.
|
|
41
|
-
function htmlPath(routePath, trailingSlash) {
|
|
42
|
-
const route =
|
|
43
|
-
|
|
44
|
-
|
|
52
|
+
function htmlPath(routePath, trailingSlash, baseUrl) {
|
|
53
|
+
const route = outputRoute(routePath, baseUrl);
|
|
54
|
+
const rel = route === '/'
|
|
55
|
+
? '/index.html'
|
|
56
|
+
: trailingSlash === false ? `${route}.html` : `${route}/index.html`;
|
|
57
|
+
return withBase(baseUrl, rel);
|
|
45
58
|
}
|
|
46
59
|
|
|
47
60
|
// Absolute URL of the page itself: canonical, no trailing slash except on
|
|
48
|
-
// the root.
|
|
49
|
-
function pageUrl(siteUrl, routePath) {
|
|
61
|
+
// the root (the site root, or the localized root '/de/').
|
|
62
|
+
function pageUrl(siteUrl, routePath, baseUrl) {
|
|
50
63
|
const base = String(siteUrl || '').replace(/\/+$/, '');
|
|
51
|
-
const route =
|
|
52
|
-
return
|
|
64
|
+
const route = outputRoute(routePath, baseUrl);
|
|
65
|
+
return `${base}${withBase(baseUrl, route)}`;
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
// The route relative to the build output tree. Docusaurus permalinks carry
|
|
69
|
+
// siteConfig.baseUrl, but the output tree is rooted at it: a site with
|
|
70
|
+
// baseUrl '/docs/' writes /docs/intro to build/intro.html, and a localized
|
|
71
|
+
// build (baseUrl '/de/', outDir build/de) writes /de/intro to
|
|
72
|
+
// build/de/intro.html. URLs keep the full route (pass baseUrl to the
|
|
73
|
+
// functions above); file paths under outDir use this route with no base.
|
|
74
|
+
function outputRoute(routePath, baseUrl) {
|
|
75
|
+
return stripBase(normalizeRoute(routePath), baseUrl);
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
// A route with siteConfig.baseUrl removed and nothing else changed (a
|
|
79
|
+
// trailing slash stays): '/de/blog/post/' under '/de/' is '/blog/post/', and
|
|
80
|
+
// the base itself is '/'. A route outside the base is returned as is.
|
|
81
|
+
function stripBase(routePath, baseUrl) {
|
|
82
|
+
const route = String(routePath || '');
|
|
83
|
+
const base = String(baseUrl || '/').replace(/\/+$/, '');
|
|
84
|
+
if (base === '') return route;
|
|
85
|
+
if (route === base) return '/';
|
|
86
|
+
return route.startsWith(`${base}/`) ? route.slice(base.length) : route;
|
|
53
87
|
}
|
|
54
88
|
|
|
55
|
-
module.exports = { companionPath, companionUrl, htmlPath, pageUrl, normalizeRoute };
|
|
89
|
+
module.exports = { companionPath, companionUrl, htmlPath, pageUrl, outputRoute, stripBase, normalizeRoute };
|
package/src/features/aiRoutes.js
CHANGED
|
@@ -12,6 +12,8 @@
|
|
|
12
12
|
// Anything else under /ai/* is permitted without a payload (consumer can
|
|
13
13
|
// extend this with additional checks if needed).
|
|
14
14
|
|
|
15
|
+
const { outputRoute } = require('../companionPath');
|
|
16
|
+
|
|
15
17
|
const KIND_FOR_DIR = {
|
|
16
18
|
faqs: 'faq',
|
|
17
19
|
faq: 'faq',
|
|
@@ -24,10 +26,14 @@ const KIND_FOR_DIR = {
|
|
|
24
26
|
applications: 'softwareApplication',
|
|
25
27
|
};
|
|
26
28
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
29
|
+
// `baseUrl` is siteConfig.baseUrl. The convention is relative to it: on a
|
|
30
|
+
// site built at '/docs/' the routes are '/docs/ai/...', and in a localized
|
|
31
|
+
// build '/de/ai/...'.
|
|
32
|
+
function classify(permalink, baseUrl) {
|
|
33
|
+
// route like "/ai/faqs/foo"
|
|
34
|
+
const route = outputRoute(permalink, baseUrl);
|
|
35
|
+
if (!route.startsWith('/ai/')) return null;
|
|
36
|
+
const segments = route.split('/').filter(Boolean); // ["ai", "faqs", "foo"]
|
|
31
37
|
if (segments.length < 2) return null;
|
|
32
38
|
const dir = segments[1].toLowerCase();
|
|
33
39
|
return KIND_FOR_DIR[dir] || null;
|
|
@@ -45,7 +51,7 @@ function collectDocs(content) {
|
|
|
45
51
|
return out;
|
|
46
52
|
}
|
|
47
53
|
|
|
48
|
-
function validateAiRoutes({ loadedContentByPlugin, verbose }) {
|
|
54
|
+
function validateAiRoutes({ loadedContentByPlugin, baseUrl, verbose }) {
|
|
49
55
|
const problems = [];
|
|
50
56
|
for (const [, entry] of loadedContentByPlugin) {
|
|
51
57
|
if (entry.pluginName !== 'docusaurus-plugin-content-docs') continue;
|
|
@@ -53,7 +59,7 @@ function validateAiRoutes({ loadedContentByPlugin, verbose }) {
|
|
|
53
59
|
for (const doc of docs) {
|
|
54
60
|
const permalink = doc.permalink;
|
|
55
61
|
if (!permalink) continue;
|
|
56
|
-
const expectedKey = classify(permalink);
|
|
62
|
+
const expectedKey = classify(permalink, baseUrl);
|
|
57
63
|
if (!expectedKey) continue;
|
|
58
64
|
const fm = doc.frontMatter || {};
|
|
59
65
|
const value = fm[expectedKey];
|
|
@@ -6,9 +6,12 @@
|
|
|
6
6
|
//
|
|
7
7
|
// inserted into its built HTML before </head>, so an agent that lands on
|
|
8
8
|
// the page can find the markdown without guessing the URL. Optionally the
|
|
9
|
-
// same relation is written as an HTTP header
|
|
10
|
-
//
|
|
11
|
-
//
|
|
9
|
+
// same relation is written as an HTTP header via a `_headers` file
|
|
10
|
+
// (`Link: <url>; rel="alternate"; type="text/markdown"`), for agents that
|
|
11
|
+
// read headers before bodies. Only hosts that read `_headers` use it
|
|
12
|
+
// (Netlify, Cloudflare Pages); elsewhere the file is inert and the <link>
|
|
13
|
+
// tag is what advertises the companion. The file sits at the root of the
|
|
14
|
+
// build directory, shared by every locale built into it.
|
|
12
15
|
//
|
|
13
16
|
// Same approach as @stackql/docusaurus-plugin-structured-data, which also
|
|
14
17
|
// post-processes the built HTML in postBuild: the file is a string, the
|
|
@@ -16,7 +19,7 @@
|
|
|
16
19
|
|
|
17
20
|
const path = require('path');
|
|
18
21
|
const fs = require('fs').promises;
|
|
19
|
-
const { htmlPath, companionUrl } = require('../companionPath');
|
|
22
|
+
const { htmlPath, companionUrl, outputRoute } = require('../companionPath');
|
|
20
23
|
|
|
21
24
|
const MARK = 'rel="alternate" type="text/markdown"';
|
|
22
25
|
|
|
@@ -28,6 +31,25 @@ function headerBlock(route, href) {
|
|
|
28
31
|
return `${route}\n Link: <${href}>; rel="alternate"; type="text/markdown"\n`;
|
|
29
32
|
}
|
|
30
33
|
|
|
34
|
+
// Where `_headers` goes. The hosts that read the file (Netlify, Cloudflare
|
|
35
|
+
// Pages) document it at the root of the published directory, so that is
|
|
36
|
+
// the only place it can be relied on. A build of several locales puts
|
|
37
|
+
// every locale but the default in a sub-directory of the build directory
|
|
38
|
+
// named after the locale (baseUrl '/de/' or '/docs/de/' -> build/de), so
|
|
39
|
+
// that locale's rules belong one level up, in the file the default
|
|
40
|
+
// locale's build wrote (Docusaurus builds the default locale first). This
|
|
41
|
+
// assumes the build directory is deployed as one site. Every other build
|
|
42
|
+
// is its own root: one locale, a locale built on its own (`--locale de`,
|
|
43
|
+
// baseUrl '/'), a locale given its own baseUrl, or a core that passes no
|
|
44
|
+
// i18n.
|
|
45
|
+
function headersDir({ outDir, siteConfig, i18n }) {
|
|
46
|
+
const locale = i18n && i18n.currentLocale;
|
|
47
|
+
if (!locale || locale === i18n.defaultLocale) return outDir;
|
|
48
|
+
const base = String(siteConfig.baseUrl || '/').replace(/\/+$/, '');
|
|
49
|
+
const nested = base.endsWith(`/${locale}`) && path.basename(outDir) === locale;
|
|
50
|
+
return nested ? path.dirname(outDir) : outDir;
|
|
51
|
+
}
|
|
52
|
+
|
|
31
53
|
async function readIfExists(file) {
|
|
32
54
|
try {
|
|
33
55
|
return await fs.readFile(file, 'utf8');
|
|
@@ -52,8 +74,13 @@ module.exports = async function injectAlternateLinks({
|
|
|
52
74
|
const headerBlocks = [];
|
|
53
75
|
|
|
54
76
|
for (const item of emittedCompanions) {
|
|
55
|
-
const href = companionUrl(siteUrl, item.permalink, trailingSlash);
|
|
56
|
-
|
|
77
|
+
const href = companionUrl(siteUrl, item.permalink, trailingSlash, siteConfig.baseUrl);
|
|
78
|
+
// The HTML lives under outDir at the route without siteConfig.baseUrl
|
|
79
|
+
// (see outputRoute); the URL and the _headers route keep the full one.
|
|
80
|
+
const file = path.join(
|
|
81
|
+
outDir,
|
|
82
|
+
htmlPath(outputRoute(item.permalink, siteConfig.baseUrl), trailingSlash),
|
|
83
|
+
);
|
|
57
84
|
const html = await readIfExists(file);
|
|
58
85
|
if (html === null) {
|
|
59
86
|
skipped++;
|
|
@@ -75,20 +102,26 @@ module.exports = async function injectAlternateLinks({
|
|
|
75
102
|
injected++;
|
|
76
103
|
}
|
|
77
104
|
if (linkHeader) {
|
|
78
|
-
//
|
|
79
|
-
// that is the bare route, otherwise the route
|
|
80
|
-
|
|
105
|
+
// A `_headers` rule matches the path as requested; with
|
|
106
|
+
// trailingSlash false that is the bare route, otherwise the route
|
|
107
|
+
// with its slash.
|
|
108
|
+
const route = htmlPath(item.permalink, trailingSlash, siteConfig.baseUrl).replace(/index\.html$/, '').replace(/\.html$/, '');
|
|
81
109
|
headerBlocks.push(headerBlock(route === '' ? '/' : route, href));
|
|
82
110
|
}
|
|
83
111
|
}
|
|
84
112
|
|
|
85
113
|
if (linkHeader && headerBlocks.length > 0) {
|
|
86
|
-
// Append to a `_headers` the site may already ship from static
|
|
87
|
-
|
|
114
|
+
// Append to a `_headers` the site may already ship from static/, or
|
|
115
|
+
// that another locale's build has already added its rules to. A rule
|
|
116
|
+
// already in the file is not written again.
|
|
117
|
+
const file = path.join(headersDir(props), '_headers');
|
|
88
118
|
const existing = (await readIfExists(file)) || '';
|
|
89
|
-
const
|
|
90
|
-
|
|
91
|
-
|
|
119
|
+
const fresh = headerBlocks.filter((block) => !existing.includes(block));
|
|
120
|
+
if (fresh.length > 0) {
|
|
121
|
+
const banner = '# @stackql/docusaurus-plugin-aeo: .md companions as Link headers\n';
|
|
122
|
+
const sep = existing && !existing.endsWith('\n') ? '\n\n' : existing ? '\n' : '';
|
|
123
|
+
await fs.writeFile(file, `${existing}${sep}${banner}${fresh.join('')}`, 'utf8');
|
|
124
|
+
}
|
|
92
125
|
}
|
|
93
126
|
|
|
94
127
|
if (verbose) {
|
|
@@ -1,45 +1,10 @@
|
|
|
1
1
|
const path = require('path');
|
|
2
2
|
const fs = require('fs').promises;
|
|
3
3
|
const matter = require('gray-matter');
|
|
4
|
-
const { companionPath, pageUrl } = require('../companionPath');
|
|
4
|
+
const { companionPath, pageUrl, outputRoute } = require('../companionPath');
|
|
5
|
+
const { matchAny, matchesRoute } = require('../routeGlob');
|
|
5
6
|
const { toPlainMarkdown, resolveFormat } = require('./plainMarkdown');
|
|
6
7
|
|
|
7
|
-
// Minimal glob matcher for route paths. Patterns like "/blog/tags/**",
|
|
8
|
-
// "/search". Avoids pulling micromatch directly.
|
|
9
|
-
function matchAny(value, patterns) {
|
|
10
|
-
if (!patterns || patterns.length === 0) return false;
|
|
11
|
-
for (const p of patterns) {
|
|
12
|
-
if (toRegExp(p).test(value)) return true;
|
|
13
|
-
}
|
|
14
|
-
return false;
|
|
15
|
-
}
|
|
16
|
-
|
|
17
|
-
function toRegExp(glob) {
|
|
18
|
-
// Minimal globstar -> regex translation. Supports: **, *, ?, character
|
|
19
|
-
// classes left untouched. Good enough for route paths like
|
|
20
|
-
// "/blog/tags/**" and "/search".
|
|
21
|
-
let re = '';
|
|
22
|
-
for (let i = 0; i < glob.length; i++) {
|
|
23
|
-
const c = glob[i];
|
|
24
|
-
if (c === '*') {
|
|
25
|
-
if (glob[i + 1] === '*') {
|
|
26
|
-
re += '.*';
|
|
27
|
-
i++;
|
|
28
|
-
if (glob[i + 1] === '/') i++;
|
|
29
|
-
} else {
|
|
30
|
-
re += '[^/]*';
|
|
31
|
-
}
|
|
32
|
-
} else if (c === '?') {
|
|
33
|
-
re += '[^/]';
|
|
34
|
-
} else if ('.+^$()|{}[]\\'.includes(c)) {
|
|
35
|
-
re += '\\' + c;
|
|
36
|
-
} else {
|
|
37
|
-
re += c;
|
|
38
|
-
}
|
|
39
|
-
}
|
|
40
|
-
return new RegExp('^' + re + '$');
|
|
41
|
-
}
|
|
42
|
-
|
|
43
8
|
function collectFromDocs(content, pluginName, pluginId) {
|
|
44
9
|
// Docs plugin loaded content shape:
|
|
45
10
|
// { loadedVersions: [{ docs: [{ id, permalink, source, title, description, frontMatter }] }] }
|
|
@@ -91,8 +56,10 @@ function collectFromBlog(content, pluginName, pluginId) {
|
|
|
91
56
|
// minus `companions.exclude`. Generated category indexes, React pages and
|
|
92
57
|
// blog list/tag/author pages have no source and are not in this list. Used
|
|
93
58
|
// by postBuild to emit the files and by allContentLoaded to tell the Ask AI
|
|
94
|
-
// button which routes have one.
|
|
95
|
-
|
|
59
|
+
// button which routes have one. `baseUrl` is siteConfig.baseUrl: an exclude
|
|
60
|
+
// pattern matches a route with or without it (see src/routeGlob.js), so the
|
|
61
|
+
// same patterns apply in every locale's build.
|
|
62
|
+
function collectCompanionItems(loadedContentByPlugin, options, baseUrl) {
|
|
96
63
|
const items = [];
|
|
97
64
|
for (const [, entry] of loadedContentByPlugin) {
|
|
98
65
|
const { pluginName, pluginId, content } = entry;
|
|
@@ -104,7 +71,7 @@ function collectCompanionItems(loadedContentByPlugin, options) {
|
|
|
104
71
|
// Custom pages plugin emits React components, not markdown. Skip silently.
|
|
105
72
|
}
|
|
106
73
|
const exclude = (options && options.exclude) || [];
|
|
107
|
-
return items.filter((item) => !
|
|
74
|
+
return items.filter((item) => !matchesRoute(item.permalink, exclude, baseUrl));
|
|
108
75
|
}
|
|
109
76
|
|
|
110
77
|
function resolveSourcePath(sourceRef, siteDir) {
|
|
@@ -162,7 +129,7 @@ module.exports = async function emitCompanions({
|
|
|
162
129
|
const format = options.format === 'clean' ? 'plain' : options.format;
|
|
163
130
|
const markdownConfigFormat = siteConfig.markdown && siteConfig.markdown.format;
|
|
164
131
|
|
|
165
|
-
const items = collectCompanionItems(loadedContentByPlugin, options);
|
|
132
|
+
const items = collectCompanionItems(loadedContentByPlugin, options, siteConfig.baseUrl);
|
|
166
133
|
|
|
167
134
|
const emitted = [];
|
|
168
135
|
let fallbacks = 0;
|
|
@@ -211,11 +178,17 @@ module.exports = async function emitCompanions({
|
|
|
211
178
|
result.markdown,
|
|
212
179
|
item.title,
|
|
213
180
|
item.description,
|
|
214
|
-
pageUrl(siteConfig.url, item.permalink),
|
|
181
|
+
pageUrl(siteConfig.url, item.permalink, siteConfig.baseUrl),
|
|
215
182
|
);
|
|
216
183
|
}
|
|
217
184
|
|
|
218
|
-
|
|
185
|
+
// The permalink carries siteConfig.baseUrl (localized to /<locale>/ in a
|
|
186
|
+
// localized build, whose outDir is build/<locale>); the file goes under
|
|
187
|
+
// outDir at the route without it.
|
|
188
|
+
const target = path.join(
|
|
189
|
+
outDir,
|
|
190
|
+
companionPath(outputRoute(item.permalink, siteConfig.baseUrl), trailingSlash),
|
|
191
|
+
);
|
|
219
192
|
await ensureDir(target);
|
|
220
193
|
await fs.writeFile(target, body, 'utf8');
|
|
221
194
|
|
package/src/features/llmsTxt.js
CHANGED
|
@@ -1,37 +1,7 @@
|
|
|
1
1
|
const path = require('path');
|
|
2
2
|
const fs = require('fs').promises;
|
|
3
3
|
const { companionUrl, pageUrl } = require('../companionPath');
|
|
4
|
-
|
|
5
|
-
function matchAny(value, patterns) {
|
|
6
|
-
if (!patterns || patterns.length === 0) return false;
|
|
7
|
-
for (const p of patterns) {
|
|
8
|
-
if (toRegExp(p).test(value)) return true;
|
|
9
|
-
}
|
|
10
|
-
return false;
|
|
11
|
-
}
|
|
12
|
-
|
|
13
|
-
function toRegExp(glob) {
|
|
14
|
-
let re = '';
|
|
15
|
-
for (let i = 0; i < glob.length; i++) {
|
|
16
|
-
const c = glob[i];
|
|
17
|
-
if (c === '*') {
|
|
18
|
-
if (glob[i + 1] === '*') {
|
|
19
|
-
re += '.*';
|
|
20
|
-
i++;
|
|
21
|
-
if (glob[i + 1] === '/') i++;
|
|
22
|
-
} else {
|
|
23
|
-
re += '[^/]*';
|
|
24
|
-
}
|
|
25
|
-
} else if (c === '?') {
|
|
26
|
-
re += '[^/]';
|
|
27
|
-
} else if ('.+^$()|{}[]\\'.includes(c)) {
|
|
28
|
-
re += '\\' + c;
|
|
29
|
-
} else {
|
|
30
|
-
re += c;
|
|
31
|
-
}
|
|
32
|
-
}
|
|
33
|
-
return new RegExp('^' + re + '$');
|
|
34
|
-
}
|
|
4
|
+
const { matchesRoute } = require('../routeGlob');
|
|
35
5
|
|
|
36
6
|
function groupKind(kind, pluginId) {
|
|
37
7
|
if (kind === 'docs') return 'docs';
|
|
@@ -172,8 +142,8 @@ function renderLlmsTxt({ siteConfig, siteUrl, trailingSlash, sections, options,
|
|
|
172
142
|
// Absolute, as the llms.txt spec's examples are: the file is read
|
|
173
143
|
// away from the site, where a relative link has no base.
|
|
174
144
|
const url = companionsEnabled
|
|
175
|
-
? companionUrl(siteUrl, item.permalink, trailingSlash)
|
|
176
|
-
: pageUrl(siteUrl, item.permalink);
|
|
145
|
+
? companionUrl(siteUrl, item.permalink, trailingSlash, siteConfig.baseUrl)
|
|
146
|
+
: pageUrl(siteUrl, item.permalink, siteConfig.baseUrl);
|
|
177
147
|
const desc = item.description || siteConfig.tagline || '';
|
|
178
148
|
const titleText = item.title || item.permalink;
|
|
179
149
|
if (desc) {
|
|
@@ -191,8 +161,10 @@ function renderLlmsTxt({ siteConfig, siteUrl, trailingSlash, sections, options,
|
|
|
191
161
|
if (options.fullTxt && options.linkFullTxt !== false && companionsEnabled) {
|
|
192
162
|
lines.push('## Optional');
|
|
193
163
|
lines.push('');
|
|
164
|
+
// Next to this file: under the locale's base in a localized build.
|
|
165
|
+
const base = String(siteConfig.baseUrl || '/').replace(/\/+$/, '');
|
|
194
166
|
lines.push(
|
|
195
|
-
`- [Full text of every page](${siteUrl}/llms-full.txt): one file with the content of every page listed above`,
|
|
167
|
+
`- [Full text of every page](${siteUrl}${base}/llms-full.txt): one file with the content of every page listed above`,
|
|
196
168
|
);
|
|
197
169
|
lines.push('');
|
|
198
170
|
}
|
|
@@ -204,8 +176,8 @@ function renderLlmsTxt({ siteConfig, siteUrl, trailingSlash, sections, options,
|
|
|
204
176
|
// the title, when the body does not already open with it). A plain
|
|
205
177
|
// companion already carries both, so the header is skipped and the body
|
|
206
178
|
// is emitted as is.
|
|
207
|
-
function fullTxtBlock({ body, item, siteUrl }) {
|
|
208
|
-
const url = pageUrl(siteUrl, item.permalink);
|
|
179
|
+
function fullTxtBlock({ body, item, siteUrl, baseUrl }) {
|
|
180
|
+
const url = pageUrl(siteUrl, item.permalink, baseUrl);
|
|
209
181
|
const head = body.trimStart().split('\n', 8);
|
|
210
182
|
const hasH1 = /^#\s+\S/.test(head[0] || '');
|
|
211
183
|
const hasSource = head.some((l) => /^Source: \S/.test(l));
|
|
@@ -228,12 +200,14 @@ module.exports = async function emitLlmsTxt({
|
|
|
228
200
|
const trailingSlash = siteConfig.trailingSlash;
|
|
229
201
|
const siteUrl = (siteConfig.url || '').replace(/\/+$/, '');
|
|
230
202
|
|
|
231
|
-
// Filter the companions through include/exclude rules.
|
|
203
|
+
// Filter the companions through include/exclude rules. A pattern matches
|
|
204
|
+
// a route with or without siteConfig.baseUrl (see src/routeGlob.js), so
|
|
205
|
+
// the same patterns apply in every locale's build.
|
|
232
206
|
const filtered = emittedCompanions.filter((item) => {
|
|
233
|
-
if (options.include && !options.include
|
|
207
|
+
if (options.include && !matchesRoute(item.permalink, options.include, siteConfig.baseUrl)) {
|
|
234
208
|
return false;
|
|
235
209
|
}
|
|
236
|
-
if (
|
|
210
|
+
if (matchesRoute(item.permalink, options.exclude, siteConfig.baseUrl)) {
|
|
237
211
|
return false;
|
|
238
212
|
}
|
|
239
213
|
return true;
|
|
@@ -297,7 +271,7 @@ module.exports = async function emitLlmsTxt({
|
|
|
297
271
|
}
|
|
298
272
|
continue;
|
|
299
273
|
}
|
|
300
|
-
const block = fullTxtBlock({ body, item, siteUrl });
|
|
274
|
+
const block = fullTxtBlock({ body, item, siteUrl, baseUrl: siteConfig.baseUrl });
|
|
301
275
|
const chunk = `${sectionOpen ? SEP : `${out ? SEP : ''}## ${section.title}\n\n`}${block}`;
|
|
302
276
|
const bytes = Buffer.byteLength(chunk, 'utf8');
|
|
303
277
|
if (full.maxBytes && size + bytes > full.maxBytes) {
|
|
File without changes
|
package/src/helpers.js
CHANGED
|
@@ -3,18 +3,36 @@
|
|
|
3
3
|
// These are pure data helpers - consumers spread them into other plugin
|
|
4
4
|
// configs so that /ai/* routes don't pollute artifacts aimed at humans
|
|
5
5
|
// (sitemaps, breadcrumb graphs, etc.).
|
|
6
|
+
//
|
|
7
|
+
// The convention is relative to the site's base URL. Docusaurus routes
|
|
8
|
+
// carry it, so on a site built at '/docs/' the routes are '/docs/ai/...'
|
|
9
|
+
// and in a localized build they are '/de/ai/...'. Every helper takes the
|
|
10
|
+
// base URL as an optional argument; without one it means '/'.
|
|
11
|
+
|
|
12
|
+
const { stripBase } = require('./companionPath');
|
|
6
13
|
|
|
7
|
-
// Glob patterns matching the /ai/* convention
|
|
8
|
-
//
|
|
9
|
-
//
|
|
10
|
-
|
|
14
|
+
// Glob patterns matching the /ai/* convention under a base URL:
|
|
15
|
+
// aiRoutePatterns('/de/') is ['/de/ai', '/de/ai/**']. Suitable for plugins
|
|
16
|
+
// whose exclude lists accept globs and match full route paths (notably
|
|
17
|
+
// @docusaurus/plugin-sitemap's `ignorePatterns`). One list can cover every
|
|
18
|
+
// locale of a site:
|
|
19
|
+
// ignorePatterns: ['/', '/de/'].flatMap(aiRoutePatterns)
|
|
20
|
+
function aiRoutePatterns(baseUrl) {
|
|
21
|
+
const base = String(baseUrl || '/').replace(/\/+$/, '');
|
|
22
|
+
return [`${base}/ai`, `${base}/ai/**`];
|
|
23
|
+
}
|
|
11
24
|
|
|
12
|
-
//
|
|
13
|
-
|
|
25
|
+
// The patterns for a site at the root with one locale.
|
|
26
|
+
const AI_ROUTE_PATTERNS = aiRoutePatterns('/');
|
|
27
|
+
|
|
28
|
+
// Sitemap slice for @docusaurus/plugin-sitemap (standalone or via preset),
|
|
29
|
+
// for a site at the root with one locale. Usage:
|
|
14
30
|
// [
|
|
15
31
|
// '@docusaurus/plugin-sitemap',
|
|
16
32
|
// { ...sitemapExclude, /* your other sitemap options */ },
|
|
17
33
|
// ]
|
|
34
|
+
// With a base URL or several locales, build `ignorePatterns` from
|
|
35
|
+
// aiRoutePatterns instead.
|
|
18
36
|
const sitemapExclude = {
|
|
19
37
|
ignorePatterns: [...AI_ROUTE_PATTERNS],
|
|
20
38
|
};
|
|
@@ -23,8 +41,8 @@ const sitemapExclude = {
|
|
|
23
41
|
// excludedRoutes list for plugins that don't accept globs (e.g.
|
|
24
42
|
// @stackql/docusaurus-plugin-structured-data, whose `excludedRoutes`
|
|
25
43
|
// is an exact-match array).
|
|
26
|
-
function isAiRoute(permalink) {
|
|
27
|
-
return typeof permalink === 'string' && permalink.startsWith('/ai/');
|
|
44
|
+
function isAiRoute(permalink, baseUrl) {
|
|
45
|
+
return typeof permalink === 'string' && stripBase(permalink, baseUrl).startsWith('/ai/');
|
|
28
46
|
}
|
|
29
47
|
|
|
30
48
|
// Build an exact-match exclude list for @stackql/docusaurus-plugin-
|
|
@@ -48,13 +66,19 @@ function isAiRoute(permalink) {
|
|
|
48
66
|
// If you don't have the route list at config time, list the known
|
|
49
67
|
// /ai/* parents by hand (e.g. ['/ai', '/ai/faqs/install', ...]); the
|
|
50
68
|
// glob-based AI_ROUTE_PATTERNS will NOT work for that plugin.
|
|
51
|
-
|
|
69
|
+
//
|
|
70
|
+
// With a base URL, pass it as the second argument: the routes are matched
|
|
71
|
+
// under it and the parent route returned is '<baseUrl>ai'.
|
|
72
|
+
function buildStructuredDataAiExcludes(allRoutePaths, baseUrl) {
|
|
52
73
|
if (!Array.isArray(allRoutePaths)) return [];
|
|
53
|
-
return allRoutePaths
|
|
74
|
+
return allRoutePaths
|
|
75
|
+
.filter((permalink) => isAiRoute(permalink, baseUrl))
|
|
76
|
+
.concat([aiRoutePatterns(baseUrl)[0]]);
|
|
54
77
|
}
|
|
55
78
|
|
|
56
79
|
module.exports = {
|
|
57
80
|
AI_ROUTE_PATTERNS,
|
|
81
|
+
aiRoutePatterns,
|
|
58
82
|
sitemapExclude,
|
|
59
83
|
isAiRoute,
|
|
60
84
|
buildStructuredDataAiExcludes,
|
package/src/index.js
CHANGED
|
@@ -182,6 +182,12 @@ module.exports = function pluginAeo(context, rawOptions) {
|
|
|
182
182
|
const options = normalizeOptions(rawOptions);
|
|
183
183
|
validateOptions(options);
|
|
184
184
|
|
|
185
|
+
// siteConfig.baseUrl for this build. Docusaurus constructs the plugin
|
|
186
|
+
// once per locale, and a localized build has its own base URL ('/de/');
|
|
187
|
+
// routes carry it, and the route conventions below are relative to it.
|
|
188
|
+
const baseUrl =
|
|
189
|
+
(context && ((context.siteConfig && context.siteConfig.baseUrl) || context.baseUrl)) || '/';
|
|
190
|
+
|
|
185
191
|
// Captured from contentLoaded for downstream postBuild use.
|
|
186
192
|
// Map<pluginName, { plugin: { name, id }, content: any }>
|
|
187
193
|
const loadedContentByPlugin = new Map();
|
|
@@ -229,6 +235,7 @@ module.exports = function pluginAeo(context, rawOptions) {
|
|
|
229
235
|
if (options.aiRoutes.validate) {
|
|
230
236
|
validateAiRoutes({
|
|
231
237
|
loadedContentByPlugin,
|
|
238
|
+
baseUrl,
|
|
232
239
|
verbose: options.verbose,
|
|
233
240
|
});
|
|
234
241
|
}
|
|
@@ -238,7 +245,7 @@ module.exports = function pluginAeo(context, rawOptions) {
|
|
|
238
245
|
// category indexes, React pages). Same list postBuild emits from.
|
|
239
246
|
if (actions && typeof actions.setGlobalData === 'function') {
|
|
240
247
|
const companionRoutes = options.companions.enabled
|
|
241
|
-
? collectCompanionItems(loadedContentByPlugin, options.companions).map(
|
|
248
|
+
? collectCompanionItems(loadedContentByPlugin, options.companions, baseUrl).map(
|
|
242
249
|
(item) => item.permalink,
|
|
243
250
|
)
|
|
244
251
|
: [];
|
package/src/routeGlob.js
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
// Route glob matching for the `exclude` / `include` options.
|
|
2
|
+
//
|
|
3
|
+
// A pattern describes a route of the site: '/blog/tags/**', '/search'.
|
|
4
|
+
// Docusaurus permalinks carry siteConfig.baseUrl, and that differs between
|
|
5
|
+
// builds of the same site: a localized build runs with baseUrl '/de/', so
|
|
6
|
+
// the page a pattern names is '/de/blog/tags/x' there. A pattern is
|
|
7
|
+
// therefore tried against the route with the base URL removed as well as
|
|
8
|
+
// the route as Docusaurus reports it:
|
|
9
|
+
//
|
|
10
|
+
// - one configuration covers every locale and any baseUrl, and
|
|
11
|
+
// - a pattern that spells the base out keeps working ('/docs/guide/**' on
|
|
12
|
+
// a site built at '/docs/') and can single out one locale
|
|
13
|
+
// ('/de/blog/**').
|
|
14
|
+
//
|
|
15
|
+
// Minimal matcher, to avoid pulling micromatch in directly: `**` matches
|
|
16
|
+
// across path segments, `*` and `?` within one.
|
|
17
|
+
|
|
18
|
+
const { stripBase } = require('./companionPath');
|
|
19
|
+
|
|
20
|
+
function toRegExp(glob) {
|
|
21
|
+
let re = '';
|
|
22
|
+
for (let i = 0; i < glob.length; i++) {
|
|
23
|
+
const c = glob[i];
|
|
24
|
+
if (c === '*') {
|
|
25
|
+
if (glob[i + 1] === '*') {
|
|
26
|
+
re += '.*';
|
|
27
|
+
i++;
|
|
28
|
+
if (glob[i + 1] === '/') i++;
|
|
29
|
+
} else {
|
|
30
|
+
re += '[^/]*';
|
|
31
|
+
}
|
|
32
|
+
} else if (c === '?') {
|
|
33
|
+
re += '[^/]';
|
|
34
|
+
} else if ('.+^$()|{}[]\\'.includes(c)) {
|
|
35
|
+
re += '\\' + c;
|
|
36
|
+
} else {
|
|
37
|
+
re += c;
|
|
38
|
+
}
|
|
39
|
+
}
|
|
40
|
+
return new RegExp('^' + re + '$');
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
// True when `value` matches at least one glob. No patterns, no match.
|
|
44
|
+
function matchAny(value, patterns) {
|
|
45
|
+
if (!patterns || patterns.length === 0) return false;
|
|
46
|
+
for (const p of patterns) {
|
|
47
|
+
if (toRegExp(p).test(value)) return true;
|
|
48
|
+
}
|
|
49
|
+
return false;
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
// True when a permalink matches at least one glob, with or without
|
|
53
|
+
// `baseUrl` (siteConfig.baseUrl) in front of it.
|
|
54
|
+
function matchesRoute(permalink, patterns, baseUrl) {
|
|
55
|
+
if (matchAny(permalink, patterns)) return true;
|
|
56
|
+
const relative = stripBase(permalink, baseUrl);
|
|
57
|
+
return relative !== permalink && matchAny(relative, patterns);
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
module.exports = { toRegExp, matchAny, matchesRoute };
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
@@ -8,6 +8,7 @@ import KeyboardArrowDownIcon from '@mui/icons-material/KeyboardArrowDown';
|
|
|
8
8
|
import { useLocation } from '@docusaurus/router';
|
|
9
9
|
import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
|
|
10
10
|
import { usePluginData } from '@docusaurus/useGlobalData';
|
|
11
|
+
import { translate } from '@docusaurus/Translate';
|
|
11
12
|
import icons from './icons.js';
|
|
12
13
|
import styles from './styles.module.css';
|
|
13
14
|
// CommonJS, shared with the Node-side features so the button and the
|
|
@@ -15,14 +16,31 @@ import styles from './styles.module.css';
|
|
|
15
16
|
import companionPathModule from '../../companionPath.js';
|
|
16
17
|
import askAiPromptModule from '../../askAiPrompt.js';
|
|
17
18
|
|
|
18
|
-
const { companionUrl, normalizeRoute } = companionPathModule;
|
|
19
|
+
const { companionUrl, pageUrl: pageUrlFor, normalizeRoute } = companionPathModule;
|
|
19
20
|
const { fillPrompt } = askAiPromptModule;
|
|
20
21
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
22
|
+
// Visible strings go through translate() so a localized site can override
|
|
23
|
+
// them in i18n/<locale>/code.json (`docusaurus write-translations` extracts
|
|
24
|
+
// them from this theme component).
|
|
25
|
+
function providerLabels() {
|
|
26
|
+
return {
|
|
27
|
+
claude: translate({
|
|
28
|
+
id: 'aeo.askAi.provider.claude',
|
|
29
|
+
message: 'Ask Claude',
|
|
30
|
+
description: 'Ask AI menu entry',
|
|
31
|
+
}),
|
|
32
|
+
chatgpt: translate({
|
|
33
|
+
id: 'aeo.askAi.provider.chatgpt',
|
|
34
|
+
message: 'Ask ChatGPT',
|
|
35
|
+
description: 'Ask AI menu entry',
|
|
36
|
+
}),
|
|
37
|
+
perplexity: translate({
|
|
38
|
+
id: 'aeo.askAi.provider.perplexity',
|
|
39
|
+
message: 'Ask Perplexity',
|
|
40
|
+
description: 'Ask AI menu entry',
|
|
41
|
+
}),
|
|
42
|
+
};
|
|
43
|
+
}
|
|
26
44
|
|
|
27
45
|
const PROVIDER_URLS = {
|
|
28
46
|
claude: 'https://claude.ai/new?q=',
|
|
@@ -47,7 +65,9 @@ export default function AskAiButton() {
|
|
|
47
65
|
|
|
48
66
|
const siteUrl = (siteConfig.url || '').replace(/\/+$/, '');
|
|
49
67
|
const route = normalizeRoute(location.pathname);
|
|
50
|
-
|
|
68
|
+
// siteConfig.baseUrl is the locale's base (/de/) in a localized build, so
|
|
69
|
+
// the localized home page resolves like the site root.
|
|
70
|
+
const pageUrl = pageUrlFor(siteUrl, route, siteConfig.baseUrl);
|
|
51
71
|
|
|
52
72
|
// Only pages with a companion get the button: a generated category index
|
|
53
73
|
// or a React page has no .md file, so a prompt pointing at one would
|
|
@@ -62,7 +82,7 @@ export default function AskAiButton() {
|
|
|
62
82
|
if (!known) return null;
|
|
63
83
|
}
|
|
64
84
|
const mdUrl = companionsEnabled
|
|
65
|
-
? companionUrl(siteUrl, route, siteConfig.trailingSlash)
|
|
85
|
+
? companionUrl(siteUrl, route, siteConfig.trailingSlash, siteConfig.baseUrl)
|
|
66
86
|
: pageUrl;
|
|
67
87
|
|
|
68
88
|
const promptTemplate = cfg.promptTemplate || DEFAULT_PROMPT_TEMPLATE;
|
|
@@ -76,6 +96,7 @@ export default function AskAiButton() {
|
|
|
76
96
|
|
|
77
97
|
const handleOpen = (e) => setAnchorEl(e.currentTarget);
|
|
78
98
|
const handleClose = () => setAnchorEl(null);
|
|
99
|
+
const labels = providerLabels();
|
|
79
100
|
|
|
80
101
|
return (
|
|
81
102
|
<div className={styles.dropdownWrapper}>
|
|
@@ -95,11 +116,16 @@ export default function AskAiButton() {
|
|
|
95
116
|
color: 'var(--ifm-color-primary)',
|
|
96
117
|
'&:hover': {
|
|
97
118
|
borderColor: 'var(--ifm-color-primary)',
|
|
98
|
-
|
|
119
|
+
// a 4% tint of the site's own primary colour
|
|
120
|
+
backgroundColor: 'color-mix(in srgb, var(--ifm-color-primary) 4%, transparent)',
|
|
99
121
|
},
|
|
100
122
|
}}
|
|
101
123
|
>
|
|
102
|
-
|
|
124
|
+
{translate({
|
|
125
|
+
id: 'aeo.askAi.button',
|
|
126
|
+
message: 'Ask AI about this page',
|
|
127
|
+
description: 'Label of the Ask AI dropdown button',
|
|
128
|
+
})}
|
|
103
129
|
</Button>
|
|
104
130
|
<Menu
|
|
105
131
|
anchorEl={anchorEl}
|
|
@@ -116,7 +142,7 @@ export default function AskAiButton() {
|
|
|
116
142
|
>
|
|
117
143
|
{order.map((key) => {
|
|
118
144
|
const Icon = icons[key];
|
|
119
|
-
const label =
|
|
145
|
+
const label = labels[key];
|
|
120
146
|
const base = PROVIDER_URLS[key];
|
|
121
147
|
if (!base || !Icon) return null;
|
|
122
148
|
const href = `${base}${encoded}`;
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|