@f5-sales-demo/starlight-llms-txt 1.3.1

package/CHANGELOG.md

@@ -0,0 +1,171 @@
+# starlight-llms-txt
+
+## 1.3.1
+
+### Patch Changes
+
+- [#195](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/195) [`5b734d4`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/5b734d4affd4e288238703f774f1215c7197847d) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Widen `peerDependencies.astro` from `^6.0.0` to `>=6.0.0` so consumers can use astro 7+.
+
+## 1.3.0
+
+### Minor Changes
+
+- [#41](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/41) [`193dd0d`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/193dd0d1bc4b8870a40f7fdf4fbf5435b04fba96) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Add auto-tiered `.txt` hierarchy for progressive AI discovery. Every hint link in `llms.txt` now points to a `.txt` file. Content lives only at leaf nodes; intermediate tiers contain metadata hints. Replaces `customSets` and `perPageMarkdown` with a single `tieredHierarchy` option (enabled by default).
+
+## 1.2.2
+
+### Patch Changes
+
+- [#30](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/30) [`a4e3c1c`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/a4e3c1c5b9135f7b311733e6982a5e7ae0ff49e0) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Deduplicate Sections entries when a node matches a custom set — prevents duplicate Guides/Functions lines in the llms.txt output for subcategory-aware sites.
+
+## 1.2.1
+
+### Patch Changes
+
+- [#25](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/25) [`068535a`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/068535ab4d725cc0c9095f0162031c306a0675a3) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Make ## Sections link to custom set \_llms-txt/\*.txt files instead of individual page URLs when a section node matches a custom set label. When sidebarNav is enabled, custom sets are moved from Documentation Sets into Sections to avoid duplication.
+
+## 1.2.0
+
+### Minor Changes
+
+- [#18](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/18) [`f850ddd`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/f850dddf7f7791c11ac15c0d1dfc379bb4262351) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Add subcategory-aware grouping to section tree and auto-generate custom sets from frontmatter
+
+## 1.1.0
+
+### Minor Changes
+
+- [#3](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/3) [`c59c96d`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/c59c96d3a3569552c3b70dec2cbf253d2bc9ccaa) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Add categorized federation support. Federated sites can now be grouped by category with optional section headings and descriptions. Fully backwards-compatible — when no categories are defined, rendering is unchanged.
+
+## 1.0.0
+
+### Major Changes
+
+- [#1](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/1) [`9ef7e93`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/9ef7e93d1685e1d5ccc0dd43f0b4de6dded1522f) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Initial release of the F5 XC Sales Demos fork, derived from `starlight-llms-txt@0.8.1`. Adds:
+
+  - `perPageMarkdown` — per-page `.md` endpoints (Tier 4 routing in the xcsh#223 cascading knowledge hierarchy). Rebased from [delucis#32](https://github.com/delucis/starlight-llms-txt/pull/32) by Matthias Vallentin, with `excludePages` extended to accept glob patterns.
+  - `sidebarNav` — sidebar hierarchy in `llms.txt` (Tier 2 routing), with frontmatter descriptions inlined automatically.
+  - `federatedSites` — cross-repo links block in `llms.txt` (Tier 1 routing).
+
+  The `starlight-llms-txt` package is authored by Chris Swithinbank (delucis). This fork exists to ship features needed by the f5xc-salesdemos documentation federation; we intend to upstream compatible features once they have been validated in production.
+
+### Minor Changes
+
+- [#1](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/1) [`940de8c`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/940de8ce11bd6d7124b6e41708c991adc576dc59) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Add `federatedSites` option. When set to a non-empty array, the plugin includes a `## Federated Sites` block in `llms.txt` listing links to other sites' `llms.txt` entry points. Intended for docs portals that federate out to product-specific documentation.
+
+- [#1](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/1) [`a8e692c`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/a8e692c82c96bbfe36d3fc993d2fca4d0d5dc74e) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Add individual Markdown file generation for each documentation page
+
+  Implements the second part of the llmstxt.org standard proposal by generating clean Markdown versions of each documentation page. These individual `.md` files are accessible at the same URL path with `.md` appended, allowing LLMs to fetch specific documentation pages on-demand.
+
+  New configuration option:
+
+  - `perPageMarkdown`: Enable individual page generation. Set to `true` for defaults, or an object for advanced configuration:
+    - `extensionStrategy`: Control URL pattern - `'append'` or `'replace'` (default: `'append'`)
+    - `excludePages`: Exclude specific pages from .md generation (default: `['404']`)
+
+- [#1](https://github.com/f5xc-salesdemos/starlight-llms-txt/pull/1) [`46d0562`](https://github.com/f5xc-salesdemos/starlight-llms-txt/commit/46d0562410dd4ddea05fd6f3b9b4bfb732253891) Thanks [@robinmordasiewicz](https://github.com/robinmordasiewicz)! - Add `sidebarNav` option. When enabled, the plugin includes a `## Sections` block in `llms.txt` with the site's pages grouped hierarchically. Entries include frontmatter descriptions inline when present.
+
+## 0.8.1
+
+### Patch Changes
+
+- [`29e5efb`](https://github.com/delucis/starlight-llms-txt/commit/29e5efb3a7d4d8b2167696e87b7f4d60db1ac93a) Thanks [@mvanhorn](https://github.com/mvanhorn)! - Strips HTML comments from llms.txt files. `<!-- ...  -->` style comments in `.md` files are no longer emitted in the generated files.
+
+## 0.8.0
+
+### Minor Changes
+
+- [#80](https://github.com/delucis/starlight-llms-txt/pull/80) [`dea7b22`](https://github.com/delucis/starlight-llms-txt/commit/dea7b22b6821168b9982fa9d8840163c6f55b70e) Thanks [@nonoakij](https://github.com/nonoakij)! - Adds support for Astro v6 and Starlight v0.38, drops support for lower versions.
+
+## 0.7.0
+
+### Minor Changes
+
+- [#43](https://github.com/delucis/starlight-llms-txt/pull/43) [`1db4591`](https://github.com/delucis/starlight-llms-txt/commit/1db45917d329e7e2ec00630a0c517b973c08fe5f) Thanks [@sanscontext](https://github.com/sanscontext)! - Enforces llms.txt files to be prerendered at build time.
+  Previously, sites using Astro’s `output: server` configuration would generate llms.txt files on-demand, which can be slow, and additionally was incompatible with the [custom sets](https://delucis.github.io/starlight-llms-txt/configuration/#customsets) feature.
+  This change means that llms.txt files are statically generated even for sites using `output: server`.
+
+  ⚠️ **Potentially breaking change:** If you were relying on on-demand rendered llms.txt files, for example by using middleware to gate access, this may be a breaking change. Please [share your use case](https://github.com/delucis/starlight-llms-txt/issues) to let us know if you need this.
+
+## 0.6.1
+
+### Patch Changes
+
+- [#49](https://github.com/delucis/starlight-llms-txt/pull/49) [`56b8233`](https://github.com/delucis/starlight-llms-txt/commit/56b823325bd42374300597a82b0f04e289be4b25) Thanks [@delucis](https://github.com/delucis)! - No code changes. This release is the first published using OIDC trusted publisher configuration for improved security.
+
+## 0.6.0
+
+### Minor Changes
+
+- [#30](https://github.com/delucis/starlight-llms-txt/pull/30) [`a1650c9`](https://github.com/delucis/starlight-llms-txt/commit/a1650c92b16377d9abdcccc8b2a68b34bc695796) Thanks [@alvinometric](https://github.com/alvinometric)! - Adds a new `rawContent` option to skip the Markdown processing pipeline
+
+## 0.5.1
+
+### Patch Changes
+
+- [#22](https://github.com/delucis/starlight-llms-txt/pull/22) [`2a8102a`](https://github.com/delucis/starlight-llms-txt/commit/2a8102a2554ac80495568b89acba2bb5a437d206) Thanks [@florian-lefebvre](https://github.com/florian-lefebvre)! - Fixes output of Expressive Code `diff` codeblocks with a `lang` attribute
+
+## 0.5.0
+
+### Minor Changes
+
+- [#18](https://github.com/delucis/starlight-llms-txt/pull/18) [`52838f6`](https://github.com/delucis/starlight-llms-txt/commit/52838f63fc5280436982880744921dac923d4be2) Thanks [@pelikhan](https://github.com/pelikhan)! - Add page separator configuration object
+
+## 0.4.1
+
+### Patch Changes
+
+- [#13](https://github.com/delucis/starlight-llms-txt/pull/13) [`629fa9b`](https://github.com/delucis/starlight-llms-txt/commit/629fa9b00444a70ebf9aac3e15375f400f8a10cc) Thanks [@jumski](https://github.com/jumski)! - Filters out draft pages from output
+
+## 0.4.0
+
+### Minor Changes
+
+- [#10](https://github.com/delucis/starlight-llms-txt/pull/10) [`7f914f5`](https://github.com/delucis/starlight-llms-txt/commit/7f914f526dbe504ed3e3763864fd9ec1d5150d0d) Thanks [@delucis](https://github.com/delucis)! - Adds a new `customSets` option to support breaking up large docs into multiple custom document sets
+
+### Patch Changes
+
+- [`0c0678d`](https://github.com/delucis/starlight-llms-txt/commit/0c0678da19f1f9981f808a1fa0e1a01c77a26b4d) Thanks [@delucis](https://github.com/delucis)! - Improves rendering of Starlight’s `<FileTree>` component by removing “Directory” labels
+
+## 0.3.0
+
+### Minor Changes
+
+- [#7](https://github.com/delucis/starlight-llms-txt/pull/7) [`cba125e`](https://github.com/delucis/starlight-llms-txt/commit/cba125ed259601895ba78f6da95a55564b914470) Thanks [@hippotastic](https://github.com/hippotastic)! - Adds options to promote or demote pages in the order of the `llms-full.txt` and `llms-small.txt` output files
+
+## 0.2.1
+
+### Patch Changes
+
+- [`39760e7`](https://github.com/delucis/starlight-llms-txt/commit/39760e70e921b685bc6dc6a5338f8f80bf79e57e) Thanks [@delucis](https://github.com/delucis)! - Removes Expressive Code’s “Terminal Window” labels from output
+
+- [`26fa616`](https://github.com/delucis/starlight-llms-txt/commit/26fa616793798bda41911bfe7dc229475f89db26) Thanks [@delucis](https://github.com/delucis)! - Fixes a bug where pages excluded using the `exclude` configuration option were excluded in `llms-full.txt` instead of only in `llms-small.txt`
+
+## 0.2.0
+
+### Minor Changes
+
+- [#4](https://github.com/delucis/starlight-llms-txt/pull/4) [`a4a77ca`](https://github.com/delucis/starlight-llms-txt/commit/a4a77ca433b7cee7cbeb3c603498e760cd037867) Thanks [@delucis](https://github.com/delucis)! - Adds support for generating a smaller `llms-small.txt` file for smaller context windows
+
+- [`618fa88`](https://github.com/delucis/starlight-llms-txt/commit/618fa882d29bc4b7ce054392c9b65d97ce1ceb82) Thanks [@delucis](https://github.com/delucis)! - Adds support for including additional optional links in the main `llms.txt` entrypoint
+
+### Patch Changes
+
+- [#4](https://github.com/delucis/starlight-llms-txt/pull/4) [`a4a77ca`](https://github.com/delucis/starlight-llms-txt/commit/a4a77ca433b7cee7cbeb3c603498e760cd037867) Thanks [@delucis](https://github.com/delucis)! - Sort pages in llms.txt output
+
+## 0.1.2
+
+### Patch Changes
+
+- [`d5ca030`](https://github.com/delucis/starlight-llms-txt/commit/d5ca0307192585f141164dd8328f244f32db5a90) Thanks [@delucis](https://github.com/delucis)! - Improves rendering of Starlight Tabs component in output
+
+## 0.1.1
+
+### Patch Changes
+
+- [`04f641c`](https://github.com/delucis/starlight-llms-txt/commit/04f641c48dd70acf480c80df26d9e2f774510428) Thanks [@delucis](https://github.com/delucis)! - Preserves language metadata on code blocks
+
+## 0.1.0
+
+### Minor Changes
+
+- [`249438b`](https://github.com/delucis/starlight-llms-txt/commit/249438b23d2998ef79a1bbb19ac7a532938f7ade) Thanks [@delucis](https://github.com/delucis)! - Initial release

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025-present, delucis
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,19 @@
+# @f5-sales-demo/starlight-llms-txt
+
+Fork of [`starlight-llms-txt`](https://github.com/delucis/starlight-llms-txt) by Chris Swithinbank, extended for the [f5-sales-demo](https://github.com/f5-sales-demo) documentation federation.
+
+## Additions over upstream
+
+- `perPageMarkdown` — per-page `.md` endpoints
+- `sidebarNav` — sidebar hierarchy in `llms.txt`, with frontmatter descriptions inlined automatically
+- `federatedSites` — cross-repo links for federated doc portals
+
+See the [configuration docs](https://f5-sales-demo.github.io/starlight-llms-txt/configuration/) for the full option reference.
+
+## Relationship to upstream
+
+Compatible features are intended to land upstream at `delucis/starlight-llms-txt` after production validation. Until then, this package tracks the f5-sales-demo integration needs.
+
+## License
+
+MIT — copyright Chris Swithinbank, fork modifications by f5-sales-demo contributors.

package/entryToSimpleMarkdown.ts

@@ -0,0 +1,209 @@
+import { type CollectionEntry, render } from 'astro:content';
+import { starlightLllmsTxtContext } from 'virtual:starlight-llms-txt/context';
+import mdxServer from '@astrojs/mdx/server.js';
+import type { APIContext } from 'astro';
+import { experimental_AstroContainer } from 'astro/container';
+import type { RootContent } from 'hast';
+import { matches, select, selectAll } from 'hast-util-select';
+import rehypeParse from 'rehype-parse';
+import rehypeRemark from 'rehype-remark';
+import remarkGfm from 'remark-gfm';
+import remarkStringify from 'remark-stringify';
+import { unified } from 'unified';
+import { remove } from 'unist-util-remove';
+
+/** Minification defaults */
+const minifyDefaults = {
+  note: true,
+  tip: true,
+  caution: false,
+  danger: false,
+  details: true,
+  whitespace: true,
+  customSelectors: [],
+};
+/** Resolved minification options */
+const minify = { ...minifyDefaults, ...starlightLllmsTxtContext.minify };
+/** Selectors for elements to remove during minification. */
+const selectors = [...minify.customSelectors];
+if (minify.details) selectors.unshift('details');
+
+const astroContainer = await experimental_AstroContainer.create({
+  renderers: [{ name: 'astro:jsx', ssr: mdxServer }],
+});
+
+const htmlToMarkdownPipeline = unified()
+  .use(rehypeParse, { fragment: true })
+  .use(function minifyLlmsTxt() {
+    return (tree, file) => {
+      if (!file.data.starlightLlmsTxt.minify) {
+        return;
+      }
+      remove(tree, (_node) => {
+        const node = _node as RootContent;
+
+        // Remove elements matching any selectors to be minified:
+        for (const selector of selectors) {
+          if (matches(selector, node)) {
+            return true;
+          }
+        }
+
+        // Remove aside components:
+        if (matches('.starlight-aside', node)) {
+          for (const variant of ['note', 'tip', 'caution', 'danger'] as const) {
+            if (minify[variant] && matches(`.starlight-aside--${variant}`, node)) {
+              return true;
+            }
+          }
+        }
+
+        return false;
+      });
+      return tree;
+    };
+  })
+  .use(function improveExpressiveCodeHandling() {
+    return (tree) => {
+      const ecInstances = selectAll('.expressive-code', tree as Parameters<typeof selectAll>[1]);
+      for (const instance of ecInstances) {
+        // Remove the “Terminal Window” label from Expressive Code terminal frames.
+        const figcaption = select('figcaption', instance);
+        if (figcaption) {
+          const terminalWindowTextIndex = figcaption.children.findIndex((child) => matches('span.sr-only', child));
+          if (terminalWindowTextIndex > -1) {
+            figcaption.children.splice(terminalWindowTextIndex, 1);
+          }
+        }
+        const pre = select('pre', instance);
+        const code = select('code', instance);
+        // Use Expressive Code’s `data-language=*` attribute to set a `language-*` class name.
+        // This is what `hast-util-to-mdast` checks for code language metadata.
+        if (pre?.properties.dataLanguage && code) {
+          if (!Array.isArray(code.properties.className)) code.properties.className = [];
+
+          const diffLines =
+            pre.properties.dataLanguage === 'diff'
+              ? []
+              : code.children.filter((child) => matches('div.ec-line.ins, div.ec-line.del', child));
+          if (diffLines.length === 0) {
+            code.properties.className.push(`language-${pre.properties.dataLanguage}`);
+          } else {
+            code.properties.className.push('language-diff');
+            for (const line of diffLines) {
+              if (line.type !== 'element') continue;
+              const classes = line.properties?.className;
+              if (typeof classes !== 'string' && !Array.isArray(classes)) continue;
+              const marker = classes.includes('ins') ? '+' : '-';
+              const span = select('span:not(.indent)', line);
+              const firstChild = span?.children[0];
+              if (firstChild?.type === 'text') {
+                firstChild.value = `${marker}${firstChild.value}`;
+              }
+            }
+          }
+        }
+      }
+    };
+  })
+  .use(function improveTabsHandling() {
+    return (tree) => {
+      const tabInstances = selectAll('starlight-tabs', tree as Parameters<typeof selectAll>[1]);
+      for (const instance of tabInstances) {
+        const tabs = selectAll('[role="tab"]', instance);
+        const panels = selectAll('[role="tabpanel"]', instance);
+        // Convert parent `<starlight-tabs>` element to empty unordered list.
+        instance.tagName = 'ul';
+        instance.properties = {};
+        instance.children = [];
+        // Iterate over tabs and panels to build a list with tab label as initial list text.
+        for (let i = 0; i < Math.min(tabs.length, panels.length); i++) {
+          const tab = tabs[i];
+          const panel = panels[i];
+          if (!tab || !panel) continue;
+          // Filter out extra whitespace and icons from tab contents.
+          const tabLabel = tab.children
+            .filter((child) => child.type === 'text' && child.value.trim())
+            .map((child) => child.type === 'text' && child.value.trim())
+            .join('');
+          // Add list entry for this tab and panel.
+          instance.children.push({
+            type: 'element',
+            tagName: 'li',
+            properties: {},
+            children: [
+              {
+                type: 'element',
+                tagName: 'p',
+                children: [{ type: 'text', value: tabLabel }],
+                properties: {},
+              },
+              panel,
+            ],
+          });
+        }
+      }
+    };
+  })
+  .use(function improveFileTreeHandling() {
+    return (tree) => {
+      const trees = selectAll('starlight-file-tree', tree as Parameters<typeof selectAll>[1]);
+      for (const tree of trees) {
+        // Remove “Directory” screen reader labels from <FileTree> entries.
+        remove(tree, (_node) => {
+          const node = _node as RootContent;
+          return matches('.sr-only', node);
+        });
+      }
+    };
+  })
+  .use(function removeHtmlComments() {
+    return (tree) => {
+      remove(tree, ({ type }) => type === 'comment');
+    };
+  })
+  .use(function removeHeadingAnchorLinks() {
+    return (tree) => {
+      // Remove Starlight's heading anchor links (e.g. "Section titled …").
+      // These are sibling <a> elements next to headings inside .sl-heading-wrapper divs.
+      remove(tree, (_node) => {
+        const node = _node as RootContent;
+        return matches('a.sl-anchor-link', node);
+      });
+    };
+  })
+  .use(rehypeRemark)
+  .use(remarkGfm)
+  .use(remarkStringify, {
+    emphasis: '*',
+    strong: '*',
+  });
+
+/** Render a content collection entry to HTML and back to Markdown to support rendering and simplifying MDX components */
+export async function entryToSimpleMarkdown(
+  entry: CollectionEntry<'docs'>,
+  context: APIContext,
+  shouldMinify: boolean = false,
+) {
+  const { rawContent } = starlightLllmsTxtContext;
+
+  // Skip processing if the `rawContent` option is enabled.
+  // This helps with UI framework components (we don't support renderers for these yet)
+  // and speeds up processing for large sites.
+  if (rawContent) {
+    // Return the raw body content directly.
+    return entry.body || '';
+  }
+
+  const { Content } = await render(entry);
+  const html = await astroContainer.renderToString(Content, context);
+  const file = await htmlToMarkdownPipeline.process({
+    value: html,
+    data: { starlightLlmsTxt: { minify: shouldMinify } },
+  });
+  let markdown = String(file).trim();
+  if (shouldMinify && minify.whitespace) {
+    markdown = markdown.replace(/\s+/g, ' ');
+  }
+  return markdown;
+}

package/env.d.ts

@@ -0,0 +1,13 @@
+/// <reference types="../../docs/.astro/types.d.ts" />
+
+declare module 'virtual:starlight-llms-txt/context' {
+  export const starlightLllmsTxtContext: import('./types').ProjectContext;
+}
+
+declare module 'vfile' {
+  interface DataMap {
+    starlightLlmsTxt: {
+      minify: boolean;
+    };
+  }
+}

package/federated-sites.ts

@@ -0,0 +1,64 @@
+export interface FederatedSite {
+  label: string;
+  url: string;
+  description?: string;
+  category?: string;
+}
+
+export interface FederatedSiteCategory {
+  id: string;
+  label: string;
+  description?: string;
+}
+
+function renderSiteList(sites: FederatedSite[]): string[] {
+  return sites.map((site) => {
+    const desc = site.description ? `: ${site.description}` : '';
+    return `- [${site.label}](${site.url})${desc}`;
+  });
+}
+
+export function renderFederatedSites(sites: FederatedSite[], categories?: FederatedSiteCategory[]): string {
+  if (sites.length === 0) return '';
+
+  if (!categories || categories.length === 0) {
+    const lines: string[] = ['## Federated Sites', ''];
+    lines.push(...renderSiteList(sites));
+    return lines.join('\n');
+  }
+
+  const sections: string[] = [];
+  const sitesByCategory = new Map<string, FederatedSite[]>();
+
+  for (const site of sites) {
+    const key = site.category || '__uncategorized__';
+    const list = sitesByCategory.get(key) || [];
+    list.push(site);
+    sitesByCategory.set(key, list);
+  }
+
+  for (const cat of categories) {
+    const catSites = sitesByCategory.get(cat.id);
+    if (!catSites || catSites.length === 0) continue;
+    const lines: string[] = [`## ${cat.label}`];
+    if (cat.description) lines.push(cat.description);
+    lines.push('');
+    lines.push(...renderSiteList(catSites));
+    sections.push(lines.join('\n'));
+    sitesByCategory.delete(cat.id);
+  }
+
+  const uncategorized = sitesByCategory.get('__uncategorized__') || [];
+  const remaining = Array.from(sitesByCategory.entries())
+    .filter(([key]) => key !== '__uncategorized__')
+    .flatMap(([, s]) => s);
+  const otherSites = [...remaining, ...uncategorized];
+
+  if (otherSites.length > 0) {
+    const lines: string[] = ['## Other', ''];
+    lines.push(...renderSiteList(otherSites));
+    sections.push(lines.join('\n'));
+  }
+
+  return sections.join('\n\n');
+}

package/generator.ts

@@ -0,0 +1,73 @@
+import type { CollectionEntry } from 'astro:content';
+import { getCollection } from 'astro:content';
+import { starlightLllmsTxtContext } from 'virtual:starlight-llms-txt/context';
+import type { APIContext } from 'astro';
+import micromatch from 'micromatch';
+import { entryToSimpleMarkdown } from './entryToSimpleMarkdown';
+import { defaultLang, isDefaultLocale, isLocale } from './utils';
+
+/** Collator to compare two strings in the default language. */
+const collator = new Intl.Collator(defaultLang);
+
+/**
+ * Generates a single plaintext Markdown document from the full website content.
+ */
+export async function generateLlmsTxt(
+  context: APIContext,
+  {
+    minify,
+    description,
+    exclude,
+    include,
+    locale,
+  }: {
+    /** Generate a smaller file to fit within smaller context windows. */
+    minify: boolean;
+    /** Description of the document being generated. Prepended to output inside `<SYSTEM>` tags. */
+    description: string | undefined;
+    exclude?: string[] | undefined;
+    include?: string[] | undefined;
+    locale?: string | undefined;
+  },
+): Promise<string> {
+  const docFilter = locale
+    ? (doc: { id: string; data: { draft?: boolean } }) =>
+        isLocale(doc as CollectionEntry<'docs'>, locale) && !doc.data.draft
+    : (doc: { id: string; data: { draft?: boolean } }) =>
+        isDefaultLocale(doc as CollectionEntry<'docs'>) && !doc.data.draft;
+  let docs = await getCollection('docs', docFilter);
+  if (include) {
+    docs = docs.filter((doc) => micromatch.isMatch(doc.id, include));
+  }
+  if (exclude) {
+    docs = docs.filter((doc) => !micromatch.isMatch(doc.id, exclude));
+  }
+  const { promote, demote, pageSeparator } = starlightLllmsTxtContext;
+  /** Processes page IDs by prepending underscores to influence the sorting order. */
+  const prioritizePages = (id: string) => {
+    // Match the page ID against the patterns listed in the `promote` and `demote`
+    // config options and return the index of the first match. If a page matches
+    // a `demote` pattern, we don't check `promote` as demotions take precedence.
+    const demoted = demote.findIndex((expr) => micromatch.isMatch(id, expr));
+    const promoted = demoted > -1 ? -1 : promote.findIndex((expr) => micromatch.isMatch(id, expr));
+    // Calculate the number of underscores to prefix the page ID with
+    // to influence the sorting order. The more underscores, the earlier
+    // the page will appear in the list. The amount of underscores added by
+    // a pattern is determined by the respective array length and the match index.
+    const prefixLength = (promoted > -1 ? promote.length - promoted : 0) + demote.length - demoted - 1;
+    return '_'.repeat(prefixLength) + id;
+  };
+  docs.sort((a, b) => collator.compare(prioritizePages(a.id), prioritizePages(b.id)));
+  const segments: string[] = [];
+  for (const doc of docs) {
+    const docSegments = [`# ${doc.data.hero?.title || doc.data.title}`];
+    const description = doc.data.hero?.tagline || doc.data.description;
+    if (description) docSegments.push(`> ${description}`);
+    docSegments.push(await entryToSimpleMarkdown(doc, context, minify));
+    segments.push(docSegments.join('\n\n'));
+  }
+  if (description) {
+    segments.unshift(`<SYSTEM>${description}</SYSTEM>`);
+  }
+  return segments.join(pageSeparator);
+}

package/index.ts

@@ -0,0 +1,111 @@
+import type { StarlightPlugin } from '@astrojs/starlight/types';
+import { AstroError } from 'astro/errors';
+import type { ProjectContext, StarlightLllmsTextOptions } from './types';
+
+export default function starlightLlmsTxt(opts: StarlightLllmsTextOptions = {}): StarlightPlugin {
+  return {
+    name: 'starlight-llms-txt',
+    hooks: {
+      setup({ astroConfig, addIntegration, config }) {
+        if (!astroConfig.site) {
+          throw new AstroError(
+            '`site` not set in Astro configuration',
+            'The `starlight-llms-txt` plugin requires setting `site` in your Astro configuration file.',
+          );
+        }
+        addIntegration({
+          name: 'starlight-llms-txt',
+          hooks: {
+            'astro:config:setup'({ injectRoute, updateConfig }) {
+              injectRoute({
+                entrypoint: new URL('./llms.txt.ts', import.meta.url),
+                pattern: '/llms.txt',
+                prerender: true,
+              });
+              injectRoute({
+                entrypoint: new URL('./llms-full.txt.ts', import.meta.url),
+                pattern: '/llms-full.txt',
+                prerender: true,
+              });
+              injectRoute({
+                entrypoint: new URL('./llms-small.txt.ts', import.meta.url),
+                pattern: '/llms-small.txt',
+                prerender: true,
+              });
+
+              injectRoute({
+                entrypoint: new URL('./llms-locale-full.txt.ts', import.meta.url),
+                pattern: '/[locale]/llms-full.txt',
+                prerender: true,
+              });
+              injectRoute({
+                entrypoint: new URL('./llms-locale-small.txt.ts', import.meta.url),
+                pattern: '/[locale]/llms-small.txt',
+                prerender: true,
+              });
+
+              const tieredHierarchy = opts.tieredHierarchy ?? true;
+              if (tieredHierarchy) {
+                injectRoute({
+                  entrypoint: new URL('./llms-tiered.txt.ts', import.meta.url),
+                  pattern: '/_llms-txt/[...path].txt',
+                  prerender: true,
+                });
+              }
+
+              const projectContext: ProjectContext = {
+                base: astroConfig.base,
+                title: opts.projectName ?? config.title,
+                description: opts.description ?? config.description,
+                details: opts.details,
+                optionalLinks: opts.optionalLinks ?? [],
+                minify: opts.minify ?? {},
+                promote: opts.promote ?? ['index*'],
+                demote: opts.demote ?? [],
+                exclude: opts.exclude ?? [],
+                defaultLocale: config.defaultLocale,
+                locales: config.locales,
+                pageSeparator: opts.pageSeparator ?? '\n\n',
+                rawContent: opts.rawContent ?? false,
+                sidebarNav: opts.sidebarNav ?? false,
+                tieredHierarchy,
+                federatedSites: opts.federatedSites ?? [],
+                federatedSiteCategories: opts.federatedSiteCategories ?? [],
+              };
+
+              const modules = {
+                'virtual:starlight-llms-txt/context': `export const starlightLllmsTxtContext = ${JSON.stringify(
+                  projectContext,
+                )}`,
+              };
+              const resolutionMap = Object.fromEntries(
+                (Object.keys(modules) as (keyof typeof modules)[]).map((key) => [resolveVirtualModuleId(key), key]),
+              );
+
+              updateConfig({
+                vite: {
+                  plugins: [
+                    {
+                      name: 'vite-plugin-starlight-llms-text',
+                      resolveId(id): string | undefined {
+                        if (id in modules) return resolveVirtualModuleId(id);
+                      },
+                      load(id): string | undefined {
+                        const resolution = resolutionMap[id];
+                        if (resolution) return modules[resolution];
+                      },
+                    },
+                  ],
+                },
+              });
+            },
+          },
+        });
+      },
+    },
+  };
+}
+
+function resolveVirtualModuleId<T extends string>(id: T): `\0${T}` {
+  return `\0${id}`;
+}

package/llms-full.txt.ts

@@ -0,0 +1,18 @@
+import type { APIRoute } from 'astro';
+import { generateLlmsTxt } from './generator';
+import { getSiteTitle } from './utils';
+
+// Explicitly set this to prerender so it works the same way for sites in `server` mode.
+
+export const prerender = true;
+
+/**
+ * Route that generates a single plaintext Markdown document from the full website content.
+ */
+export const GET: APIRoute = async (context) => {
+  const body = await generateLlmsTxt(context, {
+    minify: false,
+    description: `This is the full developer documentation for ${getSiteTitle()}`,
+  });
+  return new Response(body);
+};

package/llms-locale-full.txt.ts

@@ -0,0 +1,19 @@
+import type { APIRoute, GetStaticPaths } from 'astro';
+import { generateLlmsTxt } from './generator';
+import { getLocaleKeys, getSiteTitle } from './utils';
+
+export const prerender = true;
+
+export const getStaticPaths: GetStaticPaths = () => {
+  return getLocaleKeys().map((locale) => ({ params: { locale } }));
+};
+
+export const GET: APIRoute = async (context) => {
+  const locale = context.params.locale as string;
+  const body = await generateLlmsTxt(context, {
+    minify: false,
+    description: `This is the full developer documentation for ${getSiteTitle()} (${locale})`,
+    locale,
+  });
+  return new Response(body);
+};

package/llms-locale-small.txt.ts

@@ -0,0 +1,21 @@
+import { starlightLllmsTxtContext } from 'virtual:starlight-llms-txt/context';
+import type { APIRoute, GetStaticPaths } from 'astro';
+import { generateLlmsTxt } from './generator';
+import { getLocaleKeys, getSiteTitle } from './utils';
+
+export const prerender = true;
+
+export const getStaticPaths: GetStaticPaths = () => {
+  return getLocaleKeys().map((locale) => ({ params: { locale } }));
+};
+
+export const GET: APIRoute = async (context) => {
+  const locale = context.params.locale as string;
+  const body = await generateLlmsTxt(context, {
+    minify: true,
+    description: `This is the abridged developer documentation for ${getSiteTitle()} (${locale})`,
+    exclude: starlightLllmsTxtContext.exclude,
+    locale,
+  });
+  return new Response(body);
+};

package/llms-small.txt.ts

@@ -0,0 +1,19 @@
+import { starlightLllmsTxtContext } from 'virtual:starlight-llms-txt/context';
+import type { APIRoute } from 'astro';
+import { generateLlmsTxt } from './generator';
+import { getSiteTitle } from './utils';
+
+// Explicitly set this to prerender so it works the same way for sites in `server` mode.
+
+export const prerender = true;
+/**
+ * Route that generates a single plaintext Markdown document from the full website content.
+ */
+export const GET: APIRoute = async (context) => {
+  const body = await generateLlmsTxt(context, {
+    minify: true,
+    description: `This is the abridged developer documentation for ${getSiteTitle()}`,
+    exclude: starlightLllmsTxtContext.exclude,
+  });
+  return new Response(body);
+};

package/llms-tiered.txt.ts

@@ -0,0 +1,69 @@
+import { type CollectionEntry, getCollection } from 'astro:content';
+import { starlightLllmsTxtContext } from 'virtual:starlight-llms-txt/context';
+import type { APIRoute, GetStaticPaths, InferGetStaticParamsType, InferGetStaticPropsType } from 'astro';
+import { entryToSimpleMarkdown } from './entryToSimpleMarkdown';
+import { buildTierTree, type DirectoryNode, getAllTierPaths, type LeafNode } from './tier-tree';
+import { renderDirectoryIndex, renderLeafContent } from './tier-tree-render';
+import { isDefaultLocale } from './utils';
+
+export const prerender = true;
+
+let cachedTree: DirectoryNode | undefined;
+
+async function getTree(): Promise<DirectoryNode> {
+  if (cachedTree) return cachedTree;
+  const docs = await getCollection('docs', (doc) => isDefaultLocale(doc) && !doc.data.draft);
+  cachedTree = buildTierTree(docs, {
+    promote: starlightLllmsTxtContext.promote,
+    demote: starlightLllmsTxtContext.demote,
+  });
+  return cachedTree;
+}
+
+function findNode(root: DirectoryNode, pathSegments: string[]): DirectoryNode | LeafNode | undefined {
+  let current: DirectoryNode = root;
+  for (let i = 0; i < pathSegments.length; i++) {
+    const seg = pathSegments[i] ?? '';
+    const child = current.children.get(seg);
+    if (!child) return undefined;
+    if (i === pathSegments.length - 1) return child;
+    if (child.type !== 'directory') return undefined;
+    current = child;
+  }
+  return current;
+}
+
+export const getStaticPaths = (async () => {
+  const tree = await getTree();
+  const tierPaths = getAllTierPaths(tree);
+  return tierPaths.map((tp) => ({
+    params: { path: tp.path },
+    props: { tierType: tp.type },
+  }));
+}) satisfies GetStaticPaths;
+
+type Props = InferGetStaticPropsType<typeof getStaticPaths>;
+type Params = InferGetStaticParamsType<typeof getStaticPaths>;
+
+export const GET: APIRoute<Props, Params> = async (context) => {
+  const tree = await getTree();
+  const pathStr = context.params.path;
+  const segments = pathStr.split('/');
+  const node = findNode(tree, segments);
+
+  if (!node) {
+    return new Response('Not found', { status: 404 });
+  }
+
+  const site = new URL(
+    starlightLllmsTxtContext.base.endsWith('/') ? starlightLllmsTxtContext.base : `${starlightLllmsTxtContext.base}/`,
+    context.site,
+  );
+
+  if (node.type === 'directory') {
+    return new Response(renderDirectoryIndex(node, site));
+  }
+
+  const renderedContent = await entryToSimpleMarkdown(node.entry as unknown as CollectionEntry<'docs'>, context, false);
+  return new Response(renderLeafContent(node, renderedContent));
+};

package/llms.txt.ts

@@ -0,0 +1,58 @@
+import { getCollection } from 'astro:content';
+import { starlightLllmsTxtContext } from 'virtual:starlight-llms-txt/context';
+import type { APIRoute } from 'astro';
+import { renderFederatedSites } from './federated-sites';
+import { buildSectionTree, renderSectionTree } from './sidebar-nav';
+import { ensureTrailingSlash, getSiteTitle, isDefaultLocale } from './utils';
+
+export const prerender = true;
+
+export const GET: APIRoute = async (context) => {
+  const title = getSiteTitle();
+  const description = starlightLllmsTxtContext.description ? `> ${starlightLllmsTxtContext.description}` : '';
+  const site = new URL(ensureTrailingSlash(starlightLllmsTxtContext.base), context.site);
+  const llmsFullLink = new URL('./llms-full.txt', site);
+  const llmsSmallLink = new URL('./llms-small.txt', site);
+
+  const segments = [`# ${title}`];
+  if (description) segments.push(description);
+  if (starlightLllmsTxtContext.details) segments.push(starlightLllmsTxtContext.details);
+
+  segments.push(`## Documentation Sets`);
+  segments.push(
+    [
+      `- [Abridged documentation](${llmsSmallLink}): a compact version of the documentation for ${getSiteTitle()}, with non-essential content removed`,
+      `- [Complete documentation](${llmsFullLink}): the full documentation for ${getSiteTitle()}`,
+    ].join('\n'),
+  );
+
+  if (starlightLllmsTxtContext.sidebarNav) {
+    const docs = await getCollection('docs', (doc) => isDefaultLocale(doc) && !doc.data.draft);
+    const tree = buildSectionTree(docs, starlightLllmsTxtContext.promote, starlightLllmsTxtContext.demote);
+    const rendered = renderSectionTree(tree, site);
+    if (rendered) segments.push(rendered);
+  }
+
+  {
+    const rendered = renderFederatedSites(
+      starlightLllmsTxtContext.federatedSites,
+      starlightLllmsTxtContext.federatedSiteCategories,
+    );
+    if (rendered) segments.push(rendered);
+  }
+
+  segments.push(`## Notes`);
+  segments.push(`- The complete documentation includes all content from the official documentation
+- The content is automatically generated from the same source as the official documentation`);
+
+  if (starlightLllmsTxtContext.optionalLinks.length > 0) {
+    segments.push('## Optional');
+    segments.push(
+      starlightLllmsTxtContext.optionalLinks
+        .map((link) => `- [${link.label}](${link.url})${link.description ? `: ${link.description}` : ''}`)
+        .join('\n'),
+    );
+  }
+
+  return new Response(`${segments.join('\n\n')}\n`);
+};

package/package.json

@@ -0,0 +1,59 @@
+{
+  "name": "@f5-sales-demo/starlight-llms-txt",
+  "version": "1.3.1",
+  "license": "MIT",
+  "description": "Generate llms.txt files to train large language models on your Starlight documentation website",
+  "author": "f5-sales-demo (fork of delucis/starlight-llms-txt)",
+  "type": "module",
+  "exports": {
+    ".": "./index.ts"
+  },
+  "files": [
+    "*.ts",
+    "!*.test.ts",
+    "!vitest.config.ts",
+    "CHANGELOG.md"
+  ],
+  "dependencies": {
+    "@astrojs/mdx": "^5.0.3",
+    "@types/hast": "^3.0.4",
+    "@types/micromatch": "^4.0.10",
+    "github-slugger": "^2.0.0",
+    "hast-util-select": "^6.0.4",
+    "micromatch": "^4.0.8",
+    "rehype-parse": "^9.0.1",
+    "rehype-remark": "^10.0.1",
+    "remark-gfm": "^4.0.1",
+    "remark-stringify": "^11.0.0",
+    "unified": "^11.0.5",
+    "unist-util-remove": "^4.0.0"
+  },
+  "devDependencies": {
+    "@astrojs/starlight": "^0.38.3",
+    "astro": "^6.1.5",
+    "vitest": "^3.2.0"
+  },
+  "scripts": {
+    "test": "vitest run",
+    "test:watch": "vitest"
+  },
+  "peerDependencies": {
+    "@astrojs/starlight": ">=0.38.0",
+    "astro": ">=6.0.0"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "homepage": "https://f5-sales-demo.github.io/starlight-llms-txt/",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/f5-sales-demo/starlight-llms-txt.git",
+    "directory": "packages/starlight-llms-txt"
+  },
+  "bugs": "https://github.com/f5-sales-demo/starlight-llms-txt/issues",
+  "keywords": [
+    "llms.txt",
+    "withastro",
+    "starlight"
+  ]
+}

package/sidebar-nav.ts

@@ -0,0 +1,29 @@
+import { buildTierTree, type DirectoryNode } from './tier-tree';
+
+type DocLike = {
+  id: string;
+  data: {
+    title: string;
+    description?: string;
+    draft?: boolean;
+    sidebar?: { order?: number };
+  };
+};
+
+export function buildSectionTree(docs: DocLike[], promote: string[] = [], demote: string[] = []): DirectoryNode {
+  return buildTierTree(docs, { promote, demote });
+}
+
+export function renderSectionTree(tree: DirectoryNode, site: URL): string {
+  if (tree.children.size === 0) return '';
+
+  const lines: string[] = ['## Sections', ''];
+
+  for (const [, child] of tree.children) {
+    const descSuffix = child.meta.description ? `: ${child.meta.description}` : '';
+    const url = new URL(`./_llms-txt/${child.slug}.txt`, site);
+    lines.push(`- [${child.meta.title}](${url})${descSuffix}`);
+  }
+
+  return lines.join('\n');
+}

package/tier-tree-render.ts

@@ -0,0 +1,43 @@
+import type { DirectoryNode, LeafNode } from './tier-tree';
+
+export function renderDirectoryIndex(node: DirectoryNode, site: URL): string {
+  const segments: string[] = [];
+
+  segments.push(`# ${node.meta.title}`);
+  if (node.meta.description) {
+    segments.push(`> ${node.meta.description}`);
+  }
+
+  segments.push('## Contents');
+
+  const lines: string[] = [];
+  for (const [, child] of node.children) {
+    const childPath = child.slug;
+    const url = new URL(`./_llms-txt/${childPath}.txt`, site);
+    const title = child.meta.title;
+    const desc = child.meta.description ? `: ${child.meta.description}` : '';
+    lines.push(`- [${title}](${url})${desc}`);
+  }
+
+  segments.push(lines.join('\n'));
+
+  return segments.join('\n\n');
+}
+
+export function renderLeafContent(leaf: LeafNode, renderedContent?: string): string {
+  const segments: string[] = [];
+
+  const title = leaf.entry.data.hero?.title || leaf.entry.data.title;
+  segments.push(`# ${title}`);
+
+  const description = leaf.entry.data.hero?.tagline || leaf.entry.data.description;
+  if (description) {
+    segments.push(`> ${description}`);
+  }
+
+  if (renderedContent !== undefined) {
+    segments.push(renderedContent);
+  }
+
+  return segments.join('\n\n');
+}

package/tier-tree.ts

@@ -0,0 +1,190 @@
+// packages/starlight-llms-txt/tier-tree.ts
+import micromatch from 'micromatch';
+
+export interface DirectoryNode {
+  type: 'directory';
+  slug: string;
+  segment: string;
+  meta: { title: string; description?: string };
+  children: Map<string, DirectoryNode | LeafNode>;
+}
+
+export interface LeafNode {
+  type: 'leaf';
+  slug: string;
+  segment: string;
+  meta: { title: string; description?: string };
+  entry: {
+    id: string;
+    data: {
+      title: string;
+      description?: string;
+      draft?: boolean;
+      sidebar?: { order?: number };
+      hero?: { title?: string; tagline?: string };
+    };
+  };
+}
+
+export type TierNode = DirectoryNode | LeafNode;
+
+export interface TierTreeOptions {
+  promote?: string[];
+  demote?: string[];
+}
+
+function titleCase(segment: string): string {
+  return segment
+    .split(/[-_]/)
+    .filter(Boolean)
+    .map((s) => s.charAt(0).toUpperCase() + s.slice(1))
+    .join(' ');
+}
+
+function sortKey(id: string, promote: string[], demote: string[]): string {
+  const demoted = demote.findIndex((expr) => micromatch.isMatch(id, expr));
+  const promoted = demoted > -1 ? -1 : promote.findIndex((expr) => micromatch.isMatch(id, expr));
+  const prefixLength = (promoted > -1 ? promote.length - promoted : 0) + demote.length - demoted - 1;
+  return '_'.repeat(prefixLength) + id;
+}
+
+type DocLike = {
+  id: string;
+  data: {
+    title: string;
+    description?: string;
+    draft?: boolean;
+    sidebar?: { order?: number };
+  };
+};
+
+export function buildTierTree(entries: DocLike[], options: TierTreeOptions = {}): DirectoryNode {
+  const { promote = [], demote = [] } = options;
+
+  const filtered = entries.filter((e) => !e.data.draft);
+
+  const sorted = [...filtered].sort((a, b) => {
+    const keyA = sortKey(a.id, promote, demote);
+    const keyB = sortKey(b.id, promote, demote);
+    if (keyA !== keyB) return keyA.localeCompare(keyB);
+    const orderA = a.data.sidebar?.order ?? Infinity;
+    const orderB = b.data.sidebar?.order ?? Infinity;
+    if (orderA !== orderB) return orderA - orderB;
+    return a.data.title.localeCompare(b.data.title);
+  });
+
+  const root: DirectoryNode = {
+    type: 'directory',
+    slug: '',
+    segment: '',
+    meta: { title: '' },
+    children: new Map(),
+  };
+
+  for (const entry of sorted) {
+    const segments = entry.id.split('/');
+
+    if (segments.length === 1) {
+      const seg0 = segments[0];
+      if (!seg0) continue;
+      root.children.set(seg0, {
+        type: 'leaf',
+        slug: entry.id,
+        segment: seg0,
+        meta: { title: entry.data.title, description: entry.data.description },
+        entry,
+      });
+      continue;
+    }
+
+    let current = root;
+    for (let i = 0; i < segments.length - 1; i++) {
+      const seg = segments[i];
+      if (!seg) continue;
+      const existingChild = current.children.get(seg);
+      if (!existingChild) {
+        const slugPath = segments.slice(0, i + 1).join('/');
+        const dirNode: DirectoryNode = {
+          type: 'directory',
+          slug: slugPath,
+          segment: seg,
+          meta: { title: titleCase(seg) },
+          children: new Map(),
+        };
+        current.children.set(seg, dirNode);
+        current = dirNode;
+      } else if (existingChild.type === 'leaf') {
+        const slugPath = segments.slice(0, i + 1).join('/');
+        const dirNode: DirectoryNode = {
+          type: 'directory',
+          slug: slugPath,
+          segment: seg,
+          meta: { title: existingChild.meta.title, description: existingChild.meta.description },
+          children: new Map(),
+        };
+        dirNode.children.set('index', {
+          ...existingChild,
+          slug: `${slugPath}/index`,
+          segment: 'index',
+        });
+        current.children.set(seg, dirNode);
+        current = dirNode;
+      } else {
+        current = existingChild;
+      }
+    }
+
+    const leafSegment = segments[segments.length - 1];
+    if (!leafSegment) continue;
+    const leaf: LeafNode = {
+      type: 'leaf',
+      slug: entry.id,
+      segment: leafSegment,
+      meta: { title: entry.data.title, description: entry.data.description },
+      entry,
+    };
+    current.children.set(leafSegment, leaf);
+
+    if (leafSegment === 'index') {
+      current.meta.title = entry.data.title;
+      current.meta.description = entry.data.description;
+    }
+  }
+
+  pruneEmptyDirectories(root);
+
+  return root;
+}
+
+function pruneEmptyDirectories(node: DirectoryNode): boolean {
+  for (const [key, child] of node.children) {
+    if (child.type === 'directory') {
+      const isEmpty = pruneEmptyDirectories(child);
+      if (isEmpty) node.children.delete(key);
+    }
+  }
+  return node.children.size === 0;
+}
+
+export interface TierPath {
+  path: string;
+  type: 'directory' | 'leaf';
+}
+
+export function getAllTierPaths(root: DirectoryNode): TierPath[] {
+  const paths: TierPath[] = [];
+
+  function walk(node: DirectoryNode): void {
+    for (const [, child] of node.children) {
+      if (child.type === 'directory') {
+        paths.push({ path: child.slug, type: 'directory' });
+        walk(child);
+      } else {
+        paths.push({ path: child.slug, type: 'leaf' });
+      }
+    }
+  }
+
+  walk(root);
+  return paths;
+}

package/types.ts

@@ -0,0 +1,62 @@
+import type { StarlightUserConfig } from '@astrojs/starlight/types';
+import type { AstroConfig } from 'astro';
+
+interface FederatedSiteCategoryUserConfig {
+  id: string;
+  label: string;
+  description?: string;
+}
+
+export interface ProjectContext {
+  base: AstroConfig['base'];
+  defaultLocale: StarlightUserConfig['defaultLocale'];
+  locales: StarlightUserConfig['locales'];
+  title: StarlightUserConfig['title'];
+  description: StarlightUserConfig['description'];
+  details: StarlightLllmsTextOptions['details'];
+  optionalLinks: NonNullable<StarlightLllmsTextOptions['optionalLinks']>;
+  minify: NonNullable<StarlightLllmsTextOptions['minify']>;
+  promote: NonNullable<StarlightLllmsTextOptions['promote']>;
+  demote: NonNullable<StarlightLllmsTextOptions['demote']>;
+  exclude: NonNullable<StarlightLllmsTextOptions['exclude']>;
+  pageSeparator: NonNullable<StarlightLllmsTextOptions['pageSeparator']>;
+  rawContent: NonNullable<StarlightLllmsTextOptions['rawContent']>;
+  sidebarNav: NonNullable<StarlightLllmsTextOptions['sidebarNav']>;
+  tieredHierarchy: NonNullable<StarlightLllmsTextOptions['tieredHierarchy']>;
+  federatedSites: NonNullable<StarlightLllmsTextOptions['federatedSites']>;
+  federatedSiteCategories: NonNullable<StarlightLllmsTextOptions['federatedSiteCategories']>;
+}
+
+export interface StarlightLllmsTextOptions {
+  projectName?: string;
+  description?: string;
+  details?: string;
+  optionalLinks?: Array<{ label: string; url: string; description?: string }>;
+  minify?: {
+    note?: boolean;
+    tip?: boolean;
+    caution?: boolean;
+    danger?: boolean;
+    details?: boolean;
+    whitespace?: boolean;
+    customSelectors?: string[];
+  };
+  promote?: string[];
+  demote?: string[];
+  exclude?: string[];
+  pageSeparator?: string;
+  rawContent?: boolean;
+  sidebarNav?: boolean;
+  /**
+   * When enabled, auto-generates a tiered `.txt` hierarchy under `/_llms-txt/`.
+   * @default true
+   */
+  tieredHierarchy?: boolean;
+  federatedSites?: Array<{
+    label: string;
+    url: string;
+    description?: string;
+    category?: string;
+  }>;
+  federatedSiteCategories?: Array<FederatedSiteCategoryUserConfig>;
+}

package/utils.ts

@@ -0,0 +1,34 @@
+import type { CollectionEntry } from 'astro:content';
+import { starlightLllmsTxtContext } from 'virtual:starlight-llms-txt/context';
+
+const { defaultLocale, locales, title } = starlightLllmsTxtContext;
+export const defaultLang = (defaultLocale === 'root' ? locales?.root?.lang : defaultLocale) || 'en';
+
+/** Get the site title from the Starlight config. */
+export function getSiteTitle(): string {
+  return typeof title === 'string' ? title : (title[defaultLang] as string);
+}
+
+const localeKeys = Object.keys(locales || {}).filter((key) => key !== 'root' && key !== defaultLang);
+const startsWithLocaleRE = new RegExp(`^(${localeKeys.join('|')})/`);
+
+/** Check if a content collection entry is part of the default locale or not. */
+export function isDefaultLocale(doc: CollectionEntry<'docs'>): boolean {
+  return !(localeKeys.includes(doc.id) || startsWithLocaleRE.test(doc.id));
+}
+
+/** Check if a content collection entry belongs to a specific locale. */
+export function isLocale(doc: CollectionEntry<'docs'>, targetLocale: string): boolean {
+  if (targetLocale === defaultLang || targetLocale === 'root') return isDefaultLocale(doc);
+  return doc.id === targetLocale || doc.id.startsWith(`${targetLocale}/`);
+}
+
+/** Get all non-default locale keys. */
+export function getLocaleKeys(): string[] {
+  return localeKeys;
+}
+
+/** Append a `/` to the passed string if it doesn’t already end with one. */
+export function ensureTrailingSlash(path: string) {
+  return path.at(-1) === '/' ? path : `${path}/`;
+}