nuxt-ai-ready 2.5.0 → 2.5.2

package/node_modules/mdream/README.md

@@ -0,0 +1,1091 @@
+# mdream
+
+[![npm version](https://img.shields.io/npm/v/mdream?color=yellow)](https://npmjs.com/package/mdream)
+[![npm downloads](https://img.shields.io/npm/dm/mdream?color=yellow)](https://npm.chart.dev/mdream)
+[![license](https://img.shields.io/github/license/harlan-zw/mdream?color=yellow)](https://github.com/harlan-zw/mdream/blob/main/LICENSE.md)
+<a href="https://skilld.dev/gh/harlan-zw/mdream">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="https://skilld.dev/b/harlan-zw/mdream?theme=dark">
+    <source media="(prefers-color-scheme: light)" srcset="https://skilld.dev/b/harlan-zw/mdream?theme=light">
+    <img alt="Skill repository on skilld.dev" src="https://skilld.dev/b/harlan-zw/mdream?theme=light">
+  </picture>
+</a>
+
+## Installation
+
+```bash
+# npm
+npm install mdream
+
+# pnpm
+pnpm add mdream
+
+# yarn
+yarn add mdream
+```
+
+> [!TIP]
+> Using an AI agent? Get the mdream Skill on [skilld.dev/gh/harlan-zw/mdream](https://skilld.dev/gh/harlan-zw/mdream).
+
+For the JavaScript-only engine (hook-based plugins, splitter, pure HTML parser):
+
+```bash
+pnpm add @mdream/js
+```
+
+### Bundler Compatibility
+
+The `mdream` package uses native Node.js bindings (NAPI-RS) which cannot be statically bundled. If your bundler fails to resolve `mdream`, mark it as external:
+
+**Next.js / Turbopack:**
+```js
+// next.config.js
+const nextConfig = {
+  serverExternalPackages: ['mdream'],
+}
+```
+
+**Webpack / other bundlers:**
+```js
+externals: ['mdream']
+```
+
+> [!TIP]
+> [`@mdream/js`](https://github.com/harlan-zw/mdream/tree/main/packages/js) has zero native dependencies and works with all bundlers without configuration.
+
+> [!TIP]
+> Using Vite? [`@mdream/vite`](https://github.com/harlan-zw/mdream/tree/main/packages/vite) handles this automatically.
+
+## Table of Contents
+
+- [API Reference](#api-reference)
+  - [htmlToMarkdown()](#htmltomarkdown)
+  - [streamHtmlToMarkdown()](#streamhtmltomarkdown)
+- [Engines](#engines)
+- [Options](#options)
+  - [MdreamOptions (Rust engine)](#mdreamoptions-rust-engine)
+  - [MdreamOptions (JS engine)](#mdreamoptions-js-engine)
+  - [CleanOptions](#cleanoptions)
+  - [FrontmatterConfig](#frontmatterconfig)
+  - [TagOverride](#tagoverride)
+  - [FilterOptions](#filteroptions)
+- [Presets](#presets)
+  - [Minimal Preset](#minimal-preset)
+- [Built-in Plugins](#built-in-plugins)
+  - [Frontmatter](#frontmatter-plugin)
+  - [Isolate Main](#isolate-main-plugin)
+  - [Tailwind](#tailwind-plugin)
+  - [Filter](#filter-plugin)
+  - [Extraction](#extraction-plugin)
+- [Hook-Based Plugins (JS Engine)](#hook-based-plugins-js-engine)
+  - [Plugin Hooks](#plugin-hooks)
+  - [createPlugin()](#createplugin)
+- [Markdown Splitting (JS Engine)](#markdown-splitting-js-engine)
+  - [Basic Chunking](#basic-chunking)
+  - [Streaming Chunks](#streaming-chunks-memory-efficient)
+  - [Splitter Options](#splitter-options)
+  - [Chunk Metadata](#chunk-metadata)
+- [Content Negotiation](#content-negotiation)
+- [Pure HTML Parser (JS Engine)](#pure-html-parser-js-engine)
+- [CLI Usage](#cli-usage)
+- [Browser and Edge Usage](#browser-and-edge-usage)
+  - [Edge / Cloudflare Workers](#edge--cloudflare-workers)
+  - [Browser CDN (IIFE)](#browser-cdn-iife)
+  - [Web Worker](#web-worker)
+- [llms.txt Generation](#llmstxt-generation)
+- [Related Packages](#related-packages)
+
+## API Reference
+
+### `htmlToMarkdown()`
+
+Converts a complete HTML string to Markdown, plain text, or HTML. It is synchronous in Node and on edge runtimes.
+
+**Rust engine** (`mdream`):
+
+```ts
+import { htmlToMarkdown } from 'mdream'
+
+function htmlToMarkdown(html: string, options?: Partial<MdreamOptions>): string
+
+// Browser bundles and the CDN script load the WASM binary first
+function htmlToMarkdown(html: string, options?: Partial<MdreamOptions>): Promise<string>
+```
+
+**JS engine** (`@mdream/js`):
+
+```ts
+import { htmlToMarkdown } from '@mdream/js'
+
+function htmlToMarkdown(html: string, options?: Partial<MdreamOptions>): string
+```
+
+**Example:**
+
+```ts
+import { htmlToMarkdown } from 'mdream'
+
+const markdown = htmlToMarkdown('<h1>Hello World</h1><p>Some content.</p>')
+// # Hello World
+//
+// Some content.
+
+const text = htmlToMarkdown('<h1>Hello <strong>World</strong></h1>', {
+  format: 'text',
+})
+// Hello World
+
+const html = htmlToMarkdown('<h1>Hello <strong>World</strong></h1>', {
+  format: 'html',
+})
+// <h1 id="hello-world">Hello <strong>World</strong></h1>
+```
+
+### `streamHtmlToMarkdown()`
+
+Converts an HTML `ReadableStream` to Markdown, plain text, or HTML incrementally. Returns an `AsyncIterable<string>` that yields output chunks as they are processed.
+
+It takes the same options as `htmlToMarkdown()`. The `frontmatter` callback and the `extraction` handlers run once, after the stream ends and before the last chunk is yielded.
+
+
+```ts
+import { streamHtmlToMarkdown } from 'mdream'
+
+function streamHtmlToMarkdown(
+  htmlStream: ReadableStream<Uint8Array | string> | null,
+  options?: Partial<MdreamOptions>,
+): AsyncIterable<string>
+```
+
+
+**Example:**
+
+```ts
+import { streamHtmlToMarkdown } from 'mdream'
+
+const response = await fetch('https://example.com')
+const stream = response.body
+
+for await (const chunk of streamHtmlToMarkdown(stream, {
+  origin: 'https://example.com',
+})) {
+  process.stdout.write(chunk)
+}
+```
+
+## Engines
+
+Mdream includes two rendering engines, automatically selecting the best one for your environment:
+
+| Engine | Package | Plugins | Use case |
+|--------|---------|---------|----------|
+| **Rust** (NAPI) | `mdream` | Declarative config only | Node.js (default) |
+| **Rust** (WASM) | `mdream` | Declarative config only | Edge, browser |
+| **JavaScript** | `@mdream/js` | Hook-based + declarative | Custom plugins, splitter |
+
+```ts
+// JavaScript engine (required for hook-based plugins)
+import { htmlToMarkdown } from '@mdream/js'
+
+// Rust NAPI engine (auto-selected in Node.js)
+import { htmlToMarkdown } from 'mdream'
+```
+
+Both engines support the same declarative plugins, but the option shape differs. The Rust engine takes `minimal`, `frontmatter`, `isolateMain`, `tailwind`, `filter`, `extraction`, and `tagOverrides` at the top level. The JS engine reads the plugins from `options.plugins` and has no `minimal` option: use `withMinimalPreset()` from `@mdream/js/preset/minimal`. If the JS engine gets one of these options at the top level, it throws a `TypeError` that names the fix. The JS engine also supports `hooks` for imperative plugin transforms.
+
+## Options
+
+### MdreamOptions (Rust engine)
+
+Defined in `mdream`:
+
+```ts
+interface MdreamOptions {
+  /** Base URL for resolving relative links and images. */
+  origin?: string
+
+  /**
+   * Enable minimal preset (frontmatter, isolateMain, tailwind, filter).
+   * Default: false
+   */
+  minimal?: boolean
+
+  /**
+   * Post-processing cleanup. Pass `true` for all cleanup, or an object for specific features.
+   * Enabled by default when `minimal` is true.
+   */
+  clean?: boolean | CleanOptions
+
+  /**
+   * Extract frontmatter from HTML <head>.
+   * - `true`: enable with defaults
+   * - `(fm) => void`: enable and receive structured data via callback
+   * - `FrontmatterConfig`: enable with config and optional callback
+   */
+  frontmatter?: boolean | ((frontmatter: Record<string, string>) => void) | FrontmatterConfig
+
+  /** Isolate main content area. Default when minimal: true */
+  isolateMain?: boolean
+
+  /** Convert Tailwind utility classes to Markdown. Default when minimal: true */
+  tailwind?: boolean
+
+  /** Filter elements by CSS selectors. Default when minimal: excludes form, nav, footer, etc. */
+  filter?: false | { include?: string[], exclude?: string[], processChildren?: boolean }
+
+  /** Extract elements matching CSS selectors during conversion. */
+  extraction?: Record<string, (element: ExtractedElement) => void>
+
+  /** Override tag rendering behavior. String values act as aliases. */
+  tagOverrides?: Record<string, TagOverride | string>
+
+  /**
+   * Hard-wrap prose at this many characters, breaking on word boundaries.
+   * Applied inline during conversion (zero-cost when unset). Code blocks,
+   * tables, and headings are never wrapped. `0` (or unset) disables wrapping.
+   */
+  wrapWidth?: number
+
+  /** Output Markdown, plain text, or HTML. Default: 'markdown' */
+  format?: 'markdown' | 'text' | 'html'
+}
+```
+
+### MdreamOptions (JS engine)
+
+The JS engine extends the shared `EngineOptions` with hook-based plugin support:
+
+```ts
+interface MdreamOptions extends EngineOptions {
+  /** Imperative hook-based transform plugins. JS engine only. */
+  hooks?: TransformPlugin[]
+}
+
+interface EngineOptions {
+  origin?: string
+  clean?: boolean | CleanOptions
+  plugins?: BuiltinPlugins
+
+  /**
+   * Hard-wrap prose at this many characters, breaking on word boundaries.
+   * Code blocks, tables, and headings are never wrapped. `0` (or unset)
+   * disables wrapping.
+   */
+  wrapWidth?: number
+
+  /** Output Markdown, plain text, or HTML. Default: 'markdown' */
+  format?: 'markdown' | 'text' | 'html'
+}
+
+interface BuiltinPlugins {
+  filter?: false | { include?: (string | number)[], exclude?: (string | number)[], processChildren?: boolean }
+  frontmatter?: boolean | ((fm: Record<string, string>) => void) | FrontmatterConfig
+  isolateMain?: boolean
+  tailwind?: boolean
+  extraction?: Record<string, (element: ExtractedElement) => void>
+  tagOverrides?: Record<string, TagOverride | string>
+}
+```
+
+Note: The JS engine uses `options.plugins.filter` while the Rust engine uses `options.filter` directly.
+
+### CleanOptions
+
+Post-processing cleanup applied to the final Markdown output. All options default to `false` unless `clean: true` is set.
+
+```ts
+interface CleanOptions {
+  /** Strip tracking query parameters (utm_*, fbclid, gclid, etc.) from URLs */
+  urls?: boolean
+  /** Strip fragment-only links that don't match any heading in the output */
+  fragments?: boolean
+  /** Strip links with meaningless hrefs (#, javascript:void(0)) to plain text */
+  emptyLinks?: boolean
+  /** Collapse 3+ consecutive blank lines to 2 */
+  blankLines?: boolean
+  /** Strip links where text equals URL: [https://x.com](https://x.com) becomes https://x.com */
+  redundantLinks?: boolean
+  /** Strip self-referencing heading anchors: ## [Title](#title) becomes ## Title */
+  selfLinkHeadings?: boolean
+  /** Strip images with no alt text (decorative/tracking pixels) */
+  emptyImages?: boolean
+  /** Drop links that produce no visible text: [](url) is removed entirely */
+  emptyLinkText?: boolean
+}
+```
+
+**Example:**
+
+```ts
+const markdown = htmlToMarkdown(html, {
+  clean: {
+    urls: true,
+    emptyLinks: true,
+    emptyImages: true,
+  },
+})
+```
+
+### FrontmatterConfig
+
+```ts
+interface FrontmatterConfig {
+  /** Additional static fields to include in frontmatter */
+  additionalFields?: Record<string, string>
+  /**
+   * Meta tag names to extract beyond the defaults.
+   * Defaults: description, keywords, author, date,
+   * og:title, og:description, twitter:title, twitter:description
+   */
+  metaFields?: string[]
+  /** Callback to receive structured frontmatter data after conversion */
+  onExtract?: (frontmatter: Record<string, string>) => void
+}
+```
+
+### TagOverride
+
+Override how specific HTML tags are rendered in Markdown. String values act as aliases.
+
+> **Unknown tags pass through as plain text.** Tag matching is strict: only the standard HTML tags ship with built-in Markdown semantics. Custom elements (`<my-widget>`), web components, and any non-standard tag emit their text content verbatim, with the surrounding tag dropped. To render a custom tag with Markdown semantics, alias it with `tagOverrides`:
+>
+> ```ts
+> htmlToMarkdown('<my-em>hi</my-em>', { tagOverrides: { 'my-em': 'em' } })
+> // → "*hi*"
+> ```
+
+```ts
+interface TagOverride {
+  /** Markdown string to insert when entering this tag */
+  enter?: string
+  /** Markdown string to insert when exiting this tag */
+  exit?: string
+  /** Spacing: [newlines before, newlines after] */
+  spacing?: number[]
+  /** Whether this tag should be treated as inline */
+  isInline?: boolean
+  /** Whether this tag is self-closing */
+  isSelfClosing?: boolean
+  /** Whether whitespace inside this tag should be collapsed */
+  collapsesInnerWhiteSpace?: boolean
+  /** Alias this tag to another tag's handler */
+  alias?: string
+}
+```
+
+**Example:**
+
+```ts
+const markdown = htmlToMarkdown(html, {
+  tagOverrides: {
+    // Treat <x-heading> like <h2>
+    'x-heading': 'h2',
+    // Custom rendering for <callout>
+    'callout': {
+      enter: '> **Note:** ',
+      exit: '',
+      spacing: [2, 2],
+    },
+  },
+})
+```
+
+**Output is GitHub Flavored Markdown.** Mdream emits a fixed GFM dialect tuned for LLM input: ATX headings (`#`), fenced code blocks (` ``` `), `-` bullets, `*` emphasis, `**` strong, `---` horizontal rules, inline links. These are not configurable. For simple delimiter swaps you can use `tagOverrides`:
+
+```ts
+htmlToMarkdown(html, {
+  tagOverrides: {
+    em: { enter: '_', exit: '_', isInline: true }, // *x*  →  _x_
+    strong: { enter: '__', exit: '__', isInline: true }, // **x** →  __x__
+    hr: { enter: '* * *', exit: '' }, // ---  →  * * *
+  },
+})
+```
+
+Structural style differences (setext headings, indented code blocks, reference-style links, `~~~` fences, dynamic list markers) are out of scope. If you need turndown-style configurability, use [turndown](https://github.com/mixmark-io/turndown). If you have a use case for these in mdream, please open an issue.
+
+### FilterOptions
+
+```ts
+interface FilterOptions {
+  /** CSS selectors, tag names, or TAG_* constants for elements to include (all others excluded) */
+  include?: (string | number)[]
+  /** CSS selectors, tag names, or TAG_* constants for elements to exclude */
+  exclude?: (string | number)[]
+  /** Whether to also process children of matched elements. Default: true */
+  processChildren?: boolean
+}
+```
+
+## Presets
+
+### Minimal Preset
+
+The `minimal` preset enables the following plugins together:
+
+- **frontmatter**: Extracts metadata from HTML `<head>` into YAML frontmatter
+- **isolateMain**: Extracts the main content area, skipping navigation, headers, and footers
+- **tailwind**: Converts Tailwind utility classes to Markdown formatting
+- **filter**: Excludes `form`, `fieldset`, `object`, `embed`, `footer`, `aside`, `iframe`, `input`, `textarea`, `select`, `button`, `nav`
+- **clean**: All post-processing cleanup enabled
+
+A `filter` passed with `minimal` adds to the preset's excludes. It does not replace them. To turn the preset's filter off, pass `filter: false`. `withMinimalPreset()` in the JS engine works the same way.
+
+```ts
+// Excludes h1 as well as form, nav, footer, and the rest
+htmlToMarkdown(html, { minimal: true, filter: { exclude: ['h1'] } })
+
+// Keeps forms and nav
+htmlToMarkdown(html, { minimal: true, filter: false })
+```
+
+**Rust engine:**
+
+```ts
+import { htmlToMarkdown } from 'mdream'
+
+const markdown = htmlToMarkdown(html, {
+  origin: 'https://example.com',
+  minimal: true,
+})
+```
+
+**JS engine (using `withMinimalPreset`):**
+
+```ts
+import { htmlToMarkdown } from '@mdream/js'
+import { withMinimalPreset } from '@mdream/js/preset/minimal'
+
+const markdown = htmlToMarkdown(html, withMinimalPreset({
+  origin: 'https://example.com',
+}))
+```
+
+`withMinimalPreset()` returns an `EngineOptions` object with all plugin defaults applied. You can override individual plugins. A `filter` adds to the preset's excludes:
+
+```ts
+const markdown = htmlToMarkdown(html, withMinimalPreset({
+  plugins: {
+    frontmatter: false,
+    filter: { exclude: ['.cookie-banner'] },
+  },
+}))
+```
+
+## Built-in Plugins
+
+All built-in plugins work with both the Rust and JS engines through declarative configuration.
+
+### Frontmatter Plugin
+
+Extracts metadata from the HTML `<head>` element and generates YAML frontmatter.
+
+**Extracted fields by default:** `title`, `description`, `keywords`, `author`, `date`, `og:title`, `og:description`, `twitter:title`, `twitter:description`.
+
+```ts
+// Enable with defaults
+htmlToMarkdown(html, { frontmatter: true })
+
+// With callback to receive structured data
+htmlToMarkdown(html, {
+  frontmatter: (fm) => {
+    console.log(fm.title)
+    console.log(fm.description)
+  },
+})
+
+// With full config
+htmlToMarkdown(html, {
+  frontmatter: {
+    additionalFields: { source: 'https://example.com' },
+    metaFields: ['robots', 'viewport'],
+    onExtract: fm => console.log(fm),
+  },
+})
+```
+
+**Output example:**
+
+```yaml
+---
+title: My Page Title
+meta:
+  description: A page description
+  'og:title': My Page Title
+---
+```
+
+### Isolate Main Plugin
+
+Isolates the main content area using the following priority:
+
+1. If an explicit `<main>` element exists (within 5 depth levels), use its content exclusively
+2. Otherwise, find content between the first header tag (`h1`-`h6`) and the first `<footer>`
+3. Headings inside `<header>` tags are skipped during fallback detection
+4. The `<head>` section is always passed through for other plugins (e.g., frontmatter)
+
+```ts
+htmlToMarkdown(html, { isolateMain: true })
+```
+
+### Tailwind Plugin
+
+Converts Tailwind CSS utility classes to semantic Markdown formatting:
+
+| Tailwind Class | Markdown Output |
+|---|---|
+| `font-bold`, `font-semibold`, `font-medium`, `font-extrabold`, `font-black` | `**bold**` |
+| `italic`, `font-italic` | `*italic*` |
+| `line-through` | `~~strikethrough~~` |
+| `hidden`, `invisible` | Content removed |
+| `absolute`, `fixed`, `sticky` | Content removed |
+
+Supports responsive breakpoint prefixes (`sm:`, `md:`, `lg:`, `xl:`, `2xl:`) with mobile-first resolution.
+
+```ts
+htmlToMarkdown(html, { tailwind: true })
+```
+
+### Filter Plugin
+
+Filters HTML elements by CSS selectors, tag names, or `TAG_*` constants.
+
+```ts
+// Exclude navigation, sidebar, footer
+htmlToMarkdown(html, {
+  filter: {
+    exclude: ['nav', '#sidebar', '.footer', 'aside'],
+  },
+})
+
+// Include only specific elements
+htmlToMarkdown(html, {
+  filter: {
+    include: ['article', 'main'],
+  },
+})
+```
+
+The JS engine also supports `TAG_*` integer constants for filtering:
+
+```ts
+import { TAG_FOOTER, TAG_NAV } from '@mdream/js'
+
+htmlToMarkdown(html, {
+  plugins: {
+    filter: { exclude: [TAG_NAV, TAG_FOOTER] },
+  },
+})
+```
+
+Elements with `style="position: absolute"` or `style="position: fixed"` are also automatically excluded when the filter plugin is active.
+
+### Extraction Plugin
+
+Extracts elements matching CSS selectors during conversion. Callbacks receive the matched element with its accumulated text content and attributes.
+
+```ts
+htmlToMarkdown(html, {
+  extraction: {
+    'h2': (el) => {
+      console.log('Heading:', el.textContent)
+    },
+    'img[alt]': (el) => {
+      console.log('Image:', el.attributes.src, el.attributes.alt)
+    },
+    'a[href]': (el) => {
+      console.log('Link:', el.textContent, el.attributes.href)
+    },
+  },
+})
+```
+
+The `ExtractedElement` interface:
+
+```ts
+interface ExtractedElement {
+  selector: string
+  tagName: string
+  textContent: string
+  attributes: Record<string, string>
+}
+```
+
+## Hook-Based Plugins (JS Engine)
+
+The JS engine (`@mdream/js`) supports imperative hook-based plugins for custom transform logic. These allow you to intercept and modify the conversion pipeline at multiple stages.
+
+```ts
+import { htmlToMarkdown } from '@mdream/js'
+import { createPlugin } from '@mdream/js/plugins'
+
+const myPlugin = createPlugin({
+  onNodeEnter(node) {
+    if (node.name === 'h1')
+      return '** '
+  },
+  processTextNode(textNode) {
+    if (textNode.parent?.attributes?.id === 'highlight') {
+      return { content: `**${textNode.value}**`, skip: false }
+    }
+  },
+})
+
+const markdown = htmlToMarkdown(html, { hooks: [myPlugin] })
+```
+
+### Plugin Hooks
+
+```ts
+interface TransformPlugin {
+  /**
+   * Called before any node processing. Return { skip: true } to skip the node.
+   */
+  beforeNodeProcess?: (
+    event: NodeEvent,
+    state: MdreamRuntimeState,
+  ) => undefined | void | { skip: boolean }
+
+  /**
+   * Called when entering an element node.
+   * Return a string to prepend to the output.
+   */
+  onNodeEnter?: (
+    node: ElementNode,
+    state: MdreamRuntimeState,
+  ) => string | undefined | void
+
+  /**
+   * Called when exiting an element node.
+   * Return a string to append to the output.
+   */
+  onNodeExit?: (
+    node: ElementNode,
+    state: MdreamRuntimeState,
+  ) => string | undefined | void
+
+  /**
+   * Called to process element attributes (e.g., extracting Tailwind classes).
+   */
+  processAttributes?: (
+    node: ElementNode,
+    state: MdreamRuntimeState,
+  ) => void
+
+  /**
+   * Called for each text node. Return { content, skip } to transform text.
+   * Return undefined for no transformation.
+   */
+  processTextNode?: (
+    node: TextNode,
+    state: MdreamRuntimeState,
+  ) => { content: string, skip: boolean } | undefined
+}
+```
+
+### `createPlugin()`
+
+A typed identity function for creating plugins with full TypeScript inference:
+
+```ts
+import { createPlugin } from '@mdream/js/plugins'
+
+const plugin = createPlugin({
+  beforeNodeProcess({ node }) {
+    // Skip all div elements with class "ad"
+    if (node.type === 1 && node.attributes?.class?.includes('ad')) {
+      return { skip: true }
+    }
+  },
+})
+```
+
+### Built-in Plugin Functions (JS Engine)
+
+The following plugin factory functions are available from `@mdream/js/plugins`:
+
+```ts
+import {
+  createPlugin,
+  extractionCollectorPlugin,
+  extractionPlugin,
+  filterPlugin,
+  frontmatterPlugin,
+  isolateMainPlugin,
+  tailwindPlugin,
+} from '@mdream/js/plugins'
+```
+
+## Markdown Splitting (JS Engine)
+
+Split HTML into Markdown chunks during conversion. Compatible with the LangChain `Document` structure.
+
+Available from `@mdream/js/splitter`.
+
+### Basic Chunking
+
+```ts
+import { TAG_H2 } from '@mdream/js'
+import { htmlToMarkdownSplitChunks } from '@mdream/js/splitter'
+
+const html = `
+  <h1>Documentation</h1>
+  <h2>Installation</h2>
+  <p>Install via npm...</p>
+  <h2>Usage</h2>
+  <p>Use it like this...</p>
+`
+
+const chunks = htmlToMarkdownSplitChunks(html, {
+  headersToSplitOn: [TAG_H2],
+  chunkSize: 1000,
+  chunkOverlap: 200,
+  stripHeaders: true,
+})
+
+chunks.forEach((chunk) => {
+  console.log(chunk.content)
+  console.log(chunk.metadata.headers) // { h1: "Documentation", h2: "Installation" }
+  console.log(chunk.metadata.code) // Language if chunk contains code
+  console.log(chunk.metadata.loc) // { lines: { from: 1, to: 5 } }
+})
+```
+
+### Streaming Chunks (Memory Efficient)
+
+For large documents, use the generator version to process chunks one at a time:
+
+```ts
+import { htmlToMarkdownSplitChunksStream } from '@mdream/js/splitter'
+
+for (const chunk of htmlToMarkdownSplitChunksStream(html, options)) {
+  await processChunk(chunk)
+
+  // Early termination supported
+  if (foundTarget)
+    break
+}
+```
+
+### Splitter Options
+
+```ts
+interface SplitterOptions {
+  // --- Structural splitting ---
+
+  /**
+   * Header tag IDs to split on (TAG_H1 through TAG_H6).
+   * Default: [TAG_H2, TAG_H3, TAG_H4, TAG_H5, TAG_H6]
+   */
+  headersToSplitOn?: number[]
+
+  // --- Size-based splitting ---
+
+  /** Maximum chunk size in characters. Default: 1000 */
+  chunkSize?: number
+
+  /** Overlap between chunks for context preservation. Default: 200 */
+  chunkOverlap?: number
+
+  /**
+   * Custom length function (e.g., a token counter for LLM applications).
+   * Default: (text) => text.length
+   */
+  lengthFunction?: (text: string) => number
+
+  // --- Output formatting ---
+
+  /** Remove headers from chunk content. Default: true */
+  stripHeaders?: boolean
+
+  /** Split into individual lines. Default: false */
+  returnEachLine?: boolean
+
+  /** Keep separators in the split chunks. Default: false */
+  keepSeparator?: boolean
+
+  // --- Standard options ---
+
+  /** Base URL for resolving relative links/images */
+  origin?: string
+
+  /** Declarative built-in plugin config */
+  plugins?: BuiltinPlugins
+
+  /** Hook-based plugins (JS engine only) */
+  hooks?: TransformPlugin[]
+
+  /** Post-processing cleanup */
+  clean?: boolean | CleanOptions
+}
+```
+
+### Chunk Metadata
+
+Each chunk includes metadata for context:
+
+```ts
+interface MarkdownChunk {
+  content: string
+  metadata: {
+    /** Header hierarchy at this chunk position */
+    headers?: Record<string, string> // { h1: "Title", h2: "Section" }
+    /** Code block language if chunk contains code */
+    code?: string
+    /** Line number range in the original document */
+    loc?: {
+      lines: { from: number, to: number }
+    }
+  }
+}
+```
+
+### Use with Presets
+
+Combine splitting with presets:
+
+```ts
+import { TAG_H2 } from '@mdream/js'
+import { withMinimalPreset } from '@mdream/js/preset/minimal'
+import { htmlToMarkdownSplitChunks } from '@mdream/js/splitter'
+
+const chunks = htmlToMarkdownSplitChunks(html, withMinimalPreset({
+  headersToSplitOn: [TAG_H2],
+  chunkSize: 500,
+  origin: 'https://example.com',
+}))
+```
+
+## Content Negotiation
+
+The `@mdream/js/negotiate` module provides HTTP content negotiation utilities for serving Markdown to LLM clients:
+
+```ts
+import { parseAcceptHeader, shouldServeMarkdown } from '@mdream/js/negotiate'
+
+// Check if client prefers markdown
+const serveMarkdown = shouldServeMarkdown(
+  request.headers.get('accept'),
+  request.headers.get('sec-fetch-dest'),
+)
+
+if (serveMarkdown) {
+  return new Response(markdown, {
+    headers: { 'Content-Type': 'text/markdown' },
+  })
+}
+```
+
+`shouldServeMarkdown()` uses Accept header quality weights and position ordering. It returns `true` when `text/markdown` or `text/plain` has higher priority than `text/html`. Browser navigation requests (`sec-fetch-dest: document`) always return `false`.
+
+## Pure HTML Parser (JS Engine)
+
+If you only need to parse HTML into a DOM-like event stream without converting to Markdown, use `parseHtml` from the JS engine:
+
+```ts
+import { parseHtml } from '@mdream/js/parse'
+
+const html = '<div><h1>Title</h1><p>Content</p></div>'
+const { events, remainingHtml } = parseHtml(html)
+
+events.forEach((event) => {
+  if (event.type === 0 && event.node.type === 1) { // Enter + Element
+    console.log('Entering element:', event.node.name)
+  }
+})
+```
+
+The parser provides:
+- Pure AST event stream with no markdown generation overhead
+- Enter/exit events for each element and text node
+- Plugin support during parsing
+- Streaming compatible via `parseHtmlStream()`
+
+## CLI Usage
+
+Mdream provides a CLI that works with Unix pipes.
+
+**Pipe site to Markdown:**
+
+```bash
+curl -s https://en.wikipedia.org/wiki/Markdown \
+  | npx mdream --origin https://en.wikipedia.org --preset minimal \
+  | tee output.md
+```
+
+**Local file to Markdown:**
+
+```bash
+cat index.html \
+  | npx mdream --preset minimal \
+  | tee output.md
+```
+
+**Plain text output:**
+
+```bash
+cat index.html \
+  | npx mdream --format text \
+  | tee output.txt
+```
+
+**HTML output:**
+
+```bash
+cat index.html \
+  | npx mdream --format html \
+  | tee output.html
+```
+
+### CLI Options
+
+| Option | Description |
+|--------|-------------|
+| `--origin <url>` | Base URL for resolving relative links and images |
+| `--preset minimal` | Enable the minimal preset |
+| `--wrap-width <n>` | Hard-wrap prose at `n` characters (code, tables, and headings are never wrapped) |
+| `--format <format>` | Output format: `markdown`, `text`, `html` |
+| `--text` | Alias for `--format text` |
+| `-h`, `--help` | Display help information |
+
+The CLI reads HTML from stdin and writes Markdown, plain text, or HTML to stdout. It uses the streaming API internally.
+
+## Browser and Edge Usage
+
+Every entry takes the same `MdreamOptions` and produces the same Markdown string. In Node and edge runtimes, `htmlToMarkdown` is synchronous. In browser bundles, the CDN script, and `mdream/worker`, it returns `Promise<string>`, because the WASM binary loads first. Each export condition ships its own types, so TypeScript shows the right return type when it resolves the `browser` condition (for example, with `customConditions: ["browser"]`).
+
+```ts
+import { htmlToMarkdown } from 'mdream'
+
+// In a browser bundle
+const markdown = await htmlToMarkdown('<h1>Hello</h1>', { minimal: true })
+```
+
+### Edge / Cloudflare Workers
+
+For edge runtimes (Cloudflare Workers, Vercel Edge), `mdream` automatically selects the WASM build via export conditions (`workerd`, `edge-light`). Both `htmlToMarkdown` and `streamHtmlToMarkdown` are available:
+
+```ts
+import { htmlToMarkdown, streamHtmlToMarkdown } from 'mdream'
+
+// WASM engine auto-selected via export conditions
+const markdown = htmlToMarkdown('<h1>Hello World</h1>')
+
+// Streaming works the same as Node.js
+const response = await fetch('https://example.com')
+for await (const chunk of streamHtmlToMarkdown(response.body)) {
+  // process chunk
+}
+```
+
+Conversion errors are catchable. WASM has no unwinding, so an internal Rust panic aborts the call, but the message is preserved and re-thrown as a normal `Error` (with the raw WASM trap as `cause`), and the instance keeps working for later requests:
+
+```ts
+try {
+  return new Response(htmlToMarkdown(html))
+}
+catch (error) {
+  // "mdream WASM panic, please report this at ...\npanicked at ..."
+  console.error(error)
+  return new Response('conversion failed', { status: 500 })
+}
+```
+
+If your toolchain doesn't resolve the export conditions, the wasm-bindgen build (web target) is exposed at `mdream/wasm` for manual initialization. Its `htmlToMarkdown` takes the same `MdreamOptions` as the `mdream` entry. Its `MarkdownStream` runs the callbacks in `finish()`.
+
+```ts
+import init, { htmlToMarkdown } from 'mdream/wasm'
+import wasmModule from 'mdream/wasm/mdream_edge_bg.wasm'
+
+await init({ module_or_path: wasmModule })
+const markdown = htmlToMarkdown('<h1>Hello</h1>', { minimal: true })
+```
+
+`mdream/worker` is not an edge entry. It runs the conversion in a browser Web Worker; see [Web Worker](#web-worker).
+
+### Browser CDN (IIFE)
+
+Use mdream directly via CDN with no build step. The script inlines the WASM binary and initializes it on load, so `window.mdream.htmlToMarkdown()` is ready at once. It takes the same `MdreamOptions` as the `mdream` entry and returns `Promise<string>`, the same as the browser bundle.
+
+```html
+<script src="https://unpkg.com/mdream/dist/iife.js"></script>
+<script>
+  window.mdream.htmlToMarkdown('<h1>Hello</h1><p>World</p>')
+    .then(markdown => console.log(markdown)) // # Hello\n\nWorld
+</script>
+```
+
+**CDN Options:**
+- **unpkg**: `https://unpkg.com/mdream/dist/iife.js`
+- **jsDelivr**: `https://cdn.jsdelivr.net/npm/mdream/dist/iife.js`
+
+### Web Worker
+
+For browser environments, `mdream/worker` runs conversions off the main thread using a Web Worker:
+
+```ts
+import { htmlToMarkdown, initWorker, terminateWorker } from 'mdream/worker'
+
+await initWorker('/path/to/mdream_edge_bg.wasm')
+
+// Takes the same options as the mdream entry. Callbacks run on this thread.
+const markdown = await htmlToMarkdown('<h1>Hello</h1>', { minimal: true })
+
+// Clean up
+terminateWorker()
+```
+
+## Content Extraction with Readability
+
+For advanced content extraction (article detection, boilerplate removal), use [@mozilla/readability](https://github.com/mozilla/readability) before mdream:
+
+```ts
+import { Readability } from '@mozilla/readability'
+import { JSDOM } from 'jsdom'
+import { htmlToMarkdown } from 'mdream'
+
+const dom = new JSDOM(html, { url: 'https://example.com' })
+const article = new Readability(dom.window.document).parse()
+
+if (article) {
+  const markdown = htmlToMarkdown(article.content)
+  // article.title, article.excerpt, article.byline also available
+}
+```
+
+## llms.txt Generation
+
+For llms.txt artifact generation, use `@mdream/js/llms-txt` from the `@mdream/js` package. It accepts pre-converted Markdown and generates `llms.txt` and `llms-full.txt` artifacts.
+
+```ts
+import { generateLlmsTxtArtifacts } from '@mdream/js/llms-txt'
+import { htmlToMarkdown } from 'mdream'
+
+const result = await generateLlmsTxtArtifacts({
+  files: [
+    { title: 'Home', url: '/', content: htmlToMarkdown(homeHtml) },
+    { title: 'About', url: '/about', content: htmlToMarkdown(aboutHtml) },
+  ],
+  siteName: 'My Site',
+  origin: 'https://example.com',
+  generateFull: true,
+})
+
+console.log(result.llmsTxt) // llms.txt content
+console.log(result.llmsFullTxt) // llms-full.txt content
+```
+
+## Related Packages
+
+| Package | Description |
+|---------|-------------|
+| [`mdream`](https://npmjs.com/package/mdream) | Core HTML to Markdown converter (Rust + WASM engine) |
+| [`@mdream/js`](https://npmjs.com/package/@mdream/js) | JavaScript engine with hook-based plugins, splitter, and llms.txt generation (`@mdream/js/llms-txt`) |
+| [`@mdream/crawl`](https://github.com/harlan-zw/mdream/tree/main/packages/crawl) | Site-wide crawler for llms.txt generation |
+| [`@mdream/vite`](https://github.com/harlan-zw/mdream/tree/main/packages/vite) | Vite plugin integration |
+| [`@mdream/nuxt`](https://github.com/harlan-zw/mdream/tree/main/packages/nuxt) | Nuxt module integration |
+| [`@mdream/action`](https://github.com/harlan-zw/mdream/tree/main/packages/action) | GitHub Actions integration |
+
+## License
+
+Licensed under the [MIT license](https://github.com/harlan-zw/mdream/blob/main/LICENSE.md).