aem-live-docs-mcp 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jigar Karangiya
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,163 @@
+# AEM Live Docs MCP
+
+An MCP (Model Context Protocol) server that gives AI assistants (Cursor, Claude, etc.) direct access to the official **Adobe Experience Manager (AEM)** documentation at [aem.live](https://www.aem.live).
+
+Every page on aem.live is available as a native markdown file. This MCP fetches the source markdown directly — no HTML scraping, no conversion artefacts, just clean content.
+
+---
+
+## What's Indexed
+
+~185 pages from [www.aem.live](https://www.aem.live), organised into sections:
+
+| Section | Contents |
+|---|---|
+| `docs` | Main documentation — authoring, CDN setup, sidekick, redirects, metadata, security, etc. |
+| `developer` | Developer guides — tutorial, block collection, anatomy of a project, markup/sections, indexing, forms, etc. |
+| `blog` | AEM team blog posts |
+| `business` | Business & community pages |
+| `community` | Community hub |
+
+---
+
+## Tools
+
+| Tool | Description |
+|---|---|
+| `search_aem_docs` | BM25 search with synonym expansion and fuzzy matching |
+| `get_aem_doc_content` | Fetch full page as clean markdown |
+| `get_aem_page_toc` | Table of contents (headings only) for cheap page preview |
+| `get_aem_code_examples` | Extract only code blocks from a page |
+| `get_related_aem_docs` | Find sibling pages in the same folder |
+| `list_aem_doc_sections` | List all sections with page counts |
+| `lookup_aem_topic` | Lookup a topic and auto-fetch the top result |
+| `multi_aem_search` | Run multiple queries at once, de-duplicated |
+| `refresh_aem_index` | Force-refresh the cached page index |
+
+## Prompts
+
+| Prompt | Description |
+|---|---|
+| `aem-site-setup` | Step-by-step guide to set up a new AEM site |
+| `aem-block-creator` | Design and scaffold an AEM block/component |
+| `aem-performance-guide` | Achieve a Lighthouse score of 100 |
+| `aem-cdn-setup` | Configure a CDN for AEM production delivery |
+
+## Resources
+
+| Resource | URI | Description |
+|---|---|---|
+| Sections | `aem-docs://sections` | All sections with page counts |
+| Stats | `aem-docs://stats` | Server status, uptime, index size |
+| Section docs | `aem-docs://docs/{section}` | Browse pages in a section |
+
+---
+
+## Quick Setup (Cursor)
+
+```bash
+bash setup-cursor.sh
+```
+
+Or add manually to `~/.cursor/mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "aem-live-docs": {
+      "command": "npx",
+      "args": ["-y", "aem-live-docs-mcp"]
+    }
+  }
+}
+```
+
+### From local build
+
+```bash
+npm install && npm run build
+
+# Add to ~/.cursor/mcp.json:
+{
+  "mcpServers": {
+    "aem-live-docs": {
+      "command": "node",
+      "args": ["/path/to/aem-live-docs-mcp/dist/index.js"]
+    }
+  }
+}
+```
+
+---
+
+## Example Prompts
+
+```
+"How do I set up a new AEM site with Google Drive as content source?"
+"Show me how to create an AEM hero block"
+"How do I achieve a Lighthouse score of 100 in AEM?"
+"Configure Cloudflare for AEM with push invalidation"
+"What is the AEM Sidekick and how do I install it?"
+"How does the AEM query-index.json work for indexing?"
+"What are AEM fragments and how do I use them?"
+"Explain the AEM block collection and auto-blocking"
+```
+
+---
+
+## How Content Fetching Works
+
+AEM live serves every page as clean markdown at `<page-url>.md`. For example:
+
+- `https://www.aem.live/docs/faq` → `https://www.aem.live/docs/faq.md`
+- `https://www.aem.live/developer/tutorial` → `https://www.aem.live/developer/tutorial.md`
+
+This means content is always fresh and authoritative — no GitHub repo mapping required.
+
+The page index is loaded from the [query-index.json](https://www.aem.live/query-index.json) endpoint which provides `path`, `title`, `description`, and `lastModified` for all ~185 pages in a single request.
+
+---
+
+## Caching
+
+| Cache | TTL | Location |
+|---|---|---|
+| Page index | 24 h | `~/.cache/aem-live-docs-mcp/index-cache.json` |
+| Page content (disk) | 7 days | `~/.cache/aem-live-docs-mcp/pages/<hash>.md` |
+| Page content (memory) | 1 h | In-process LRU (100 entries) |
+
+Use `refresh_aem_index` to force a re-fetch of the page index.
+
+## Environment Variables
+
+| Variable | Default | Description |
+|---|---|---|
+| `AEM_BASE_URL` | `https://www.aem.live` | AEM site base URL |
+| `QUERY_INDEX_URL` | `https://www.aem.live/query-index.json` | Query index endpoint |
+| `CACHE_DIR` | `~/.cache/aem-live-docs-mcp` | Cache directory |
+| `MAX_CONTENT_LENGTH` | `15000` | Max chars per page (smart truncation at heading boundaries) |
+| `PORT` | `3000` | HTTP port (when using `--http` mode) |
+| `LOG_LEVEL` | `info` | Log level: debug/info/warn/error |
+
+## HTTP Mode
+
+```bash
+npm run start:http
+# or
+node dist/index.js --http
+```
+
+Server listens on `http://localhost:3000` with CORS enabled.
+
+---
+
+## Related MCPs
+
+- [adobe-commerce-docs-mcp](https://github.com/jigarkkarangiya/adobe-commerce-docs-mcp) — Adobe Commerce merchant/user docs
+- [adobe-commerce-dev-docs-mcp](https://github.com/jigarkkarangiya/adobe-commerce-dev-docs-mcp) — Adobe Commerce developer docs
+
+---
+
+## License
+
+MIT © Jigar Karangiya

package/dist/config.d.ts

@@ -0,0 +1,19 @@
+export declare const config: {
+    version: string;
+    baseUrl: string;
+    queryIndexUrl: string;
+    cacheDir: string;
+    indexCacheFile: string;
+    pageCacheDir: string;
+    indexCacheTtlMs: number;
+    pageCacheMemoryMax: number;
+    pageCacheMemoryTtlMs: number;
+    pageCacheDiskTtlMs: number;
+    maxContentLength: number;
+    maxConcurrentFetches: number;
+    httpPort: number;
+    logLevel: "debug" | "info" | "warn" | "error";
+    userAgent: string;
+    docPathPrefixes: string[];
+};
+//# sourceMappingURL=config.d.ts.map

package/dist/config.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"config.d.ts","sourceRoot":"","sources":["../src/config.ts"],"names":[],"mappings":"AA8BA,eAAO,MAAM,MAAM;;;;;;;;;;;;;;cA0BwB,OAAO,GAAG,MAAM,GAAG,MAAM,GAAG,OAAO;;;CAM7E,CAAC"}

package/dist/config.js

@@ -0,0 +1,45 @@
+import { join } from "node:path";
+import { homedir } from "node:os";
+function envInt(key, fallback) {
+    const v = process.env[key];
+    if (!v)
+        return fallback;
+    const n = parseInt(v, 10);
+    return isNaN(n) ? fallback : n;
+}
+function envStr(key, fallback) {
+    return process.env[key] || fallback;
+}
+const cacheDir = envStr("CACHE_DIR", join(homedir(), ".cache", "aem-live-docs-mcp"));
+// Path prefixes considered part of the AEM documentation.
+// Pages outside these prefixes (footer, gnav, tools/organizer, experiments, etc.)
+// are excluded from the index.
+const docPathPrefixes = [
+    "/docs",
+    "/developer",
+    "/blog",
+    "/business",
+    "/community",
+];
+export const config = {
+    version: "1.0.0",
+    // AEM live base URL — all page content is available as markdown at <baseUrl><path>.md
+    baseUrl: envStr("AEM_BASE_URL", "https://www.aem.live"),
+    // AEM live query-index.json endpoint — provides the full page listing with
+    // path, title, description, lastModified in one request.
+    queryIndexUrl: envStr("QUERY_INDEX_URL", "https://www.aem.live/query-index.json"),
+    cacheDir,
+    indexCacheFile: join(cacheDir, "index-cache.json"),
+    pageCacheDir: join(cacheDir, "pages"),
+    indexCacheTtlMs: envInt("INDEX_CACHE_TTL_MS", 24 * 60 * 60 * 1000),
+    pageCacheMemoryMax: envInt("PAGE_CACHE_MAX", 100),
+    pageCacheMemoryTtlMs: envInt("PAGE_CACHE_TTL_MS", 60 * 60 * 1000),
+    pageCacheDiskTtlMs: envInt("PAGE_DISK_CACHE_TTL_MS", 7 * 24 * 60 * 60 * 1000),
+    maxContentLength: envInt("MAX_CONTENT_LENGTH", 15000),
+    maxConcurrentFetches: envInt("MAX_CONCURRENT_FETCHES", 5),
+    httpPort: envInt("PORT", 3000),
+    logLevel: envStr("LOG_LEVEL", "info"),
+    userAgent: "Mozilla/5.0 (compatible; AEMLiveDocsMCP/1.0; +https://github.com/jigarkkarangiya/aem-live-docs-mcp)",
+    docPathPrefixes,
+};
+//# sourceMappingURL=config.js.map

package/dist/config.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"config.js","sourceRoot":"","sources":["../src/config.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAC;AACjC,OAAO,EAAE,OAAO,EAAE,MAAM,SAAS,CAAC;AAElC,SAAS,MAAM,CAAC,GAAW,EAAE,QAAgB;IAC3C,MAAM,CAAC,GAAG,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IAC3B,IAAI,CAAC,CAAC;QAAE,OAAO,QAAQ,CAAC;IACxB,MAAM,CAAC,GAAG,QAAQ,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;IAC1B,OAAO,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;AACjC,CAAC;AAED,SAAS,MAAM,CAAC,GAAW,EAAE,QAAgB;IAC3C,OAAO,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,QAAQ,CAAC;AACtC,CAAC;AAED,MAAM,QAAQ,GAAG,MAAM,CACrB,WAAW,EACX,IAAI,CAAC,OAAO,EAAE,EAAE,QAAQ,EAAE,mBAAmB,CAAC,CAC/C,CAAC;AAEF,0DAA0D;AAC1D,kFAAkF;AAClF,+BAA+B;AAC/B,MAAM,eAAe,GAAG;IACtB,OAAO;IACP,YAAY;IACZ,OAAO;IACP,WAAW;IACX,YAAY;CACb,CAAC;AAEF,MAAM,CAAC,MAAM,MAAM,GAAG;IACpB,OAAO,EAAE,OAAO;IAEhB,sFAAsF;IACtF,OAAO,EAAE,MAAM,CAAC,cAAc,EAAE,sBAAsB,CAAC;IAEvD,2EAA2E;IAC3E,yDAAyD;IACzD,aAAa,EAAE,MAAM,CACnB,iBAAiB,EACjB,uCAAuC,CACxC;IAED,QAAQ;IACR,cAAc,EAAE,IAAI,CAAC,QAAQ,EAAE,kBAAkB,CAAC;IAClD,YAAY,EAAE,IAAI,CAAC,QAAQ,EAAE,OAAO,CAAC;IAErC,eAAe,EAAE,MAAM,CAAC,oBAAoB,EAAE,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,IAAI,CAAC;IAClE,kBAAkB,EAAE,MAAM,CAAC,gBAAgB,EAAE,GAAG,CAAC;IACjD,oBAAoB,EAAE,MAAM,CAAC,mBAAmB,EAAE,EAAE,GAAG,EAAE,GAAG,IAAI,CAAC;IACjE,kBAAkB,EAAE,MAAM,CAAC,wBAAwB,EAAE,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,IAAI,CAAC;IAE7E,gBAAgB,EAAE,MAAM,CAAC,oBAAoB,EAAE,KAAK,CAAC;IACrD,oBAAoB,EAAE,MAAM,CAAC,wBAAwB,EAAE,CAAC,CAAC;IAEzD,QAAQ,EAAE,MAAM,CAAC,MAAM,EAAE,IAAI,CAAC;IAC9B,QAAQ,EAAE,MAAM,CAAC,WAAW,EAAE,MAAM,CAAwC;IAE5E,SAAS,EACP,qGAAqG;IAEvG,eAAe;CAChB,CAAC"}

package/dist/content.d.ts

@@ -0,0 +1,16 @@
+export interface TocEntry {
+    level: number;
+    title: string;
+}
+export declare function clearMemoryCache(): void;
+export declare function buildMarkdownUrl(pageUrl: string): string;
+export declare function cleanAemMarkdown(raw: string): string;
+export declare function smartTruncate(content: string, maxLen: number): string;
+export declare function fetchPageContent(url: string): Promise<string>;
+export declare function fetchRawContent(url: string): Promise<string>;
+export declare function extractCodeExamples(markdown: string): {
+    language: string;
+    code: string;
+}[];
+export declare function extractPageToc(markdown: string): TocEntry[];
+//# sourceMappingURL=content.d.ts.map

package/dist/content.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"content.d.ts","sourceRoot":"","sources":["../src/content.ts"],"names":[],"mappings":"AAOA,MAAM,WAAW,QAAQ;IACvB,KAAK,EAAE,MAAM,CAAC;IACd,KAAK,EAAE,MAAM,CAAC;CACf;AAgCD,wBAAgB,gBAAgB,IAAI,IAAI,CAEvC;AAqCD,wBAAgB,gBAAgB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,CASxD;AA4ED,wBAAgB,gBAAgB,CAAC,GAAG,EAAE,MAAM,GAAG,MAAM,CAgBpD;AAID,wBAAgB,aAAa,CAAC,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG,MAAM,CAgCrE;AAWD,wBAAsB,gBAAgB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAmBnE;AAED,wBAAsB,eAAe,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAOlE;AAID,wBAAgB,mBAAmB,CACjC,QAAQ,EAAE,MAAM,GACf;IAAE,QAAQ,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,EAAE,CAWtC;AAED,wBAAgB,cAAc,CAAC,QAAQ,EAAE,MAAM,GAAG,QAAQ,EAAE,CAW3D"}

package/dist/content.js

@@ -0,0 +1,239 @@
+import { readFile, writeFile, mkdir, stat } from "node:fs/promises";
+import { join } from "node:path";
+import { createHash } from "node:crypto";
+import { config } from "./config.js";
+const memoryCache = new Map();
+function getFromMemoryCache(url) {
+    const entry = memoryCache.get(url);
+    if (!entry)
+        return null;
+    if (Date.now() - entry.timestamp > config.pageCacheMemoryTtlMs) {
+        memoryCache.delete(url);
+        return null;
+    }
+    // LRU: move to end
+    memoryCache.delete(url);
+    memoryCache.set(url, entry);
+    return entry.content;
+}
+function setMemoryCache(url, content) {
+    if (memoryCache.size >= config.pageCacheMemoryMax) {
+        const oldest = memoryCache.keys().next().value;
+        if (oldest !== undefined)
+            memoryCache.delete(oldest);
+    }
+    memoryCache.set(url, { content, timestamp: Date.now() });
+}
+export function clearMemoryCache() {
+    memoryCache.clear();
+}
+// --- Persistent disk page cache ---
+function urlToHash(url) {
+    return createHash("sha256").update(url).digest("hex").slice(0, 16);
+}
+async function getFromDiskCache(url) {
+    try {
+        const filePath = join(config.pageCacheDir, `${urlToHash(url)}.md`);
+        const info = await stat(filePath);
+        if (Date.now() - info.mtimeMs > config.pageCacheDiskTtlMs)
+            return null;
+        return await readFile(filePath, "utf-8");
+    }
+    catch {
+        return null;
+    }
+}
+async function setDiskCache(url, content) {
+    try {
+        await mkdir(config.pageCacheDir, { recursive: true });
+        await writeFile(join(config.pageCacheDir, `${urlToHash(url)}.md`), content, "utf-8");
+    }
+    catch {
+        // non-critical
+    }
+}
+// --- AEM live markdown URL builder ---
+//
+// AEM live serves every page as clean markdown at <path>.md.
+// E.g. https://www.aem.live/docs/faq  →  https://www.aem.live/docs/faq.md
+export function buildMarkdownUrl(pageUrl) {
+    try {
+        const u = new URL(pageUrl);
+        // Strip trailing slash then append .md
+        u.pathname = u.pathname.replace(/\/+$/, "") + ".md";
+        return u.toString();
+    }
+    catch {
+        return pageUrl + ".md";
+    }
+}
+// --- Fetch helpers ---
+const FETCH_HEADERS = { "User-Agent": config.userAgent };
+async function fetchMarkdown(pageUrl) {
+    const mdUrl = buildMarkdownUrl(pageUrl);
+    try {
+        const res = await fetch(mdUrl, {
+            headers: { ...FETCH_HEADERS, Accept: "text/plain, text/markdown" },
+            redirect: "follow",
+        });
+        if (!res.ok)
+            return null;
+        const text = await res.text();
+        return text.length > 0 ? text : null;
+    }
+    catch {
+        return null;
+    }
+}
+async function fetchAndParseHtml(url) {
+    const res = await fetch(url, {
+        headers: { ...FETCH_HEADERS, Accept: "text/html" },
+    });
+    if (!res.ok) {
+        throw new Error(`Failed to fetch page: ${res.status} ${res.statusText}`);
+    }
+    return extractMainContent(await res.text());
+}
+// --- HTML → Markdown fallback ---
+function extractMainContent(html) {
+    let content = html;
+    const mainMatch = content.match(/<main[^>]*>([\s\S]*?)<\/main>/i) ??
+        content.match(/<article[^>]*>([\s\S]*?)<\/article>/i) ??
+        content.match(/<div[^>]*class="[^"]*content[^"]*"[^>]*>([\s\S]*?)<\/div>/i);
+    if (mainMatch)
+        content = mainMatch[1];
+    return content
+        .replace(/<script[^>]*>[\s\S]*?<\/script>/gi, "")
+        .replace(/<style[^>]*>[\s\S]*?<\/style>/gi, "")
+        .replace(/<nav[^>]*>[\s\S]*?<\/nav>/gi, "")
+        .replace(/<footer[^>]*>[\s\S]*?<\/footer>/gi, "")
+        .replace(/<header[^>]*>[\s\S]*?<\/header>/gi, "")
+        .replace(/<h1[^>]*>([\s\S]*?)<\/h1>/gi, "\n# $1\n")
+        .replace(/<h2[^>]*>([\s\S]*?)<\/h2>/gi, "\n## $1\n")
+        .replace(/<h3[^>]*>([\s\S]*?)<\/h3>/gi, "\n### $1\n")
+        .replace(/<h4[^>]*>([\s\S]*?)<\/h4>/gi, "\n#### $1\n")
+        .replace(/<pre[^>]*><code[^>]*>([\s\S]*?)<\/code><\/pre>/gi, "\n```\n$1\n```\n")
+        .replace(/<code[^>]*>([\s\S]*?)<\/code>/gi, "`$1`")
+        .replace(/<a[^>]*href="([^"]*)"[^>]*>([\s\S]*?)<\/a>/gi, "[$2]($1)")
+        .replace(/<li[^>]*>([\s\S]*?)<\/li>/gi, "- $1\n")
+        .replace(/<\/?[uo]l[^>]*>/gi, "\n")
+        .replace(/<p[^>]*>([\s\S]*?)<\/p>/gi, "\n$1\n")
+        .replace(/<br\s*\/?>/gi, "\n")
+        .replace(/<\/div>/gi, "\n")
+        .replace(/<[^>]+>/g, "")
+        .replace(/&amp;/g, "&")
+        .replace(/&lt;/g, "<")
+        .replace(/&gt;/g, ">")
+        .replace(/&quot;/g, '"')
+        .replace(/&#39;/g, "'")
+        .replace(/&nbsp;/g, " ")
+        .replace(/\n{3,}/g, "\n\n")
+        .trim();
+}
+// --- AEM markdown cleanup ---
+//
+// AEM live markdown files use a table-based block syntax (not MDX) and include
+// metadata rows. We normalise the content to clean, readable markdown.
+export function cleanAemMarkdown(raw) {
+    let out = raw;
+    // Strip YAML-style frontmatter if present (some pages have it)
+    const frontmatterMatch = out.match(/^---\s*\r?\n[\s\S]*?\r?\n---\s*\r?\n/);
+    if (frontmatterMatch) {
+        out = out.slice(frontmatterMatch[0].length);
+    }
+    // AEM uses pipe-delimited Markdown tables for blocks. Keep them as-is since
+    // they are valid Markdown — no transformation needed.
+    // Collapse runs of 3+ blank lines down to 2
+    out = out.replace(/\n{3,}/g, "\n\n");
+    return out.trim();
+}
+// --- Smart truncation (heading-boundary aware) ---
+export function smartTruncate(content, maxLen) {
+    if (content.length <= maxLen)
+        return content;
+    const lines = content.split("\n");
+    let charCount = 0;
+    let lastHeadingIdx = -1;
+    let lastBlankIdx = -1;
+    for (let i = 0; i < lines.length; i++) {
+        charCount += lines[i].length + 1;
+        if (charCount > maxLen)
+            break;
+        if (/^#{1,6}\s/.test(lines[i]))
+            lastHeadingIdx = i;
+        if (lines[i].trim() === "")
+            lastBlankIdx = i;
+    }
+    const threshold = lines.length * 0.3;
+    const cutIdx = lastHeadingIdx > threshold
+        ? lastHeadingIdx
+        : lastBlankIdx > threshold
+            ? lastBlankIdx
+            : -1;
+    if (cutIdx > 0) {
+        const remaining = lines.length - cutIdx;
+        return (lines.slice(0, cutIdx).join("\n").trim() +
+            `\n\n... [truncated — ${remaining} more lines]`);
+    }
+    return content.substring(0, maxLen) + "\n\n... [content truncated]";
+}
+// --- Fetch orchestration ---
+async function fetchAndClean(url) {
+    const md = await fetchMarkdown(url);
+    if (md)
+        return cleanAemMarkdown(md);
+    // Fallback to HTML parsing
+    return fetchAndParseHtml(url);
+}
+export async function fetchPageContent(url) {
+    const memoryCached = getFromMemoryCache(url);
+    if (memoryCached)
+        return memoryCached;
+    const diskCached = await getFromDiskCache(url);
+    if (diskCached) {
+        const truncated = smartTruncate(diskCached, config.maxContentLength);
+        const result = `Source: ${url}\n\n${truncated}`;
+        setMemoryCache(url, result);
+        return result;
+    }
+    const rawContent = await fetchAndClean(url);
+    await setDiskCache(url, rawContent);
+    const truncated = smartTruncate(rawContent, config.maxContentLength);
+    const result = `Source: ${url}\n\n${truncated}`;
+    setMemoryCache(url, result);
+    return result;
+}
+export async function fetchRawContent(url) {
+    const diskCached = await getFromDiskCache(url);
+    if (diskCached)
+        return diskCached;
+    const rawContent = await fetchAndClean(url);
+    await setDiskCache(url, rawContent);
+    return rawContent;
+}
+// --- Content extraction helpers ---
+export function extractCodeExamples(markdown) {
+    const examples = [];
+    const fenced = /```(\w*)\n([\s\S]*?)```/g;
+    let m;
+    while ((m = fenced.exec(markdown)) !== null) {
+        const code = m[2].trim();
+        if (code.length > 0) {
+            examples.push({ language: m[1] || "text", code });
+        }
+    }
+    return examples;
+}
+export function extractPageToc(markdown) {
+    const entries = [];
+    const heading = /^(#{1,6})\s+(.+)$/gm;
+    let m;
+    while ((m = heading.exec(markdown)) !== null) {
+        entries.push({
+            level: m[1].length,
+            title: m[2].trim().replace(/[`*_]/g, ""),
+        });
+    }
+    return entries;
+}
+//# sourceMappingURL=content.js.map

package/dist/content.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"content.js","sourceRoot":"","sources":["../src/content.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,SAAS,EAAE,KAAK,EAAE,IAAI,EAAE,MAAM,kBAAkB,CAAC;AACpE,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAC;AACjC,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACzC,OAAO,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAgBrC,MAAM,WAAW,GAAG,IAAI,GAAG,EAAyB,CAAC;AAErD,SAAS,kBAAkB,CAAC,GAAW;IACrC,MAAM,KAAK,GAAG,WAAW,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IACnC,IAAI,CAAC,KAAK;QAAE,OAAO,IAAI,CAAC;IACxB,IAAI,IAAI,CAAC,GAAG,EAAE,GAAG,KAAK,CAAC,SAAS,GAAG,MAAM,CAAC,oBAAoB,EAAE,CAAC;QAC/D,WAAW,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;QACxB,OAAO,IAAI,CAAC;IACd,CAAC;IACD,mBAAmB;IACnB,WAAW,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;IACxB,WAAW,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;IAC5B,OAAO,KAAK,CAAC,OAAO,CAAC;AACvB,CAAC;AAED,SAAS,cAAc,CAAC,GAAW,EAAE,OAAe;IAClD,IAAI,WAAW,CAAC,IAAI,IAAI,MAAM,CAAC,kBAAkB,EAAE,CAAC;QAClD,MAAM,MAAM,GAAG,WAAW,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC;QAC/C,IAAI,MAAM,KAAK,SAAS;YAAE,WAAW,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IACvD,CAAC;IACD,WAAW,CAAC,GAAG,CAAC,GAAG,EAAE,EAAE,OAAO,EAAE,SAAS,EAAE,IAAI,CAAC,GAAG,EAAE,EAAE,CAAC,CAAC;AAC3D,CAAC;AAED,MAAM,UAAU,gBAAgB;IAC9B,WAAW,CAAC,KAAK,EAAE,CAAC;AACtB,CAAC;AAED,qCAAqC;AAErC,SAAS,SAAS,CAAC,GAAW;IAC5B,OAAO,UAAU,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;AACrE,CAAC;AAED,KAAK,UAAU,gBAAgB,CAAC,GAAW;IACzC,IAAI,CAAC;QACH,MAAM,QAAQ,GAAG,IAAI,CAAC,MAAM,CAAC,YAAY,EAAE,GAAG,SAAS,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;QACnE,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,QAAQ,CAAC,CAAC;QAClC,IAAI,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC,kBAAkB;YAAE,OAAO,IAAI,CAAC;QACvE,OAAO,MAAM,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC;IAC3C,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,IAAI,CAAC;IACd,CAAC;AACH,CAAC;AAED,KAAK,UAAU,YAAY,CAAC,GAAW,EAAE,OAAe;IACtD,IAAI,CAAC;QACH,MAAM,KAAK,CAAC,MAAM,CAAC,YAAY,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;QACtD,MAAM,SAAS,CACb,IAAI,CAAC,MAAM,CAAC,YAAY,EAAE,GAAG,SAAS,CAAC,GAAG,CAAC,KAAK,CAAC,EACjD,OAAO,EACP,OAAO,CACR,CAAC;IACJ,CAAC;IAAC,MAAM,CAAC;QACP,eAAe;IACjB,CAAC;AACH,CAAC;AAED,wCAAwC;AACxC,EAAE;AACF,6DAA6D;AAC7D,0EAA0E;AAE1E,MAAM,UAAU,gBAAgB,CAAC,OAAe;IAC9C,IAAI,CAAC;QACH,MAAM,CAAC,GAAG,IAAI,GAAG,CAAC,OAAO,CAAC,CAAC;QAC3B,uCAAuC;QACvC,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC,QAAQ,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,GAAG,KAAK,CAAC;QACpD,OAAO,CAAC,CAAC,QAAQ,EAAE,CAAC;IACtB,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,OAAO,GAAG,KAAK,CAAC;IACzB,CAAC;AACH,CAAC;AAED,wBAAwB;AAExB,MAAM,aAAa,GAAG,EAAE,YAAY,EAAE,MAAM,CAAC,SAAS,EAAE,CAAC;AAEzD,KAAK,UAAU,aAAa,CAAC,OAAe;IAC1C,MAAM,KAAK,GAAG,gBAAgB,CAAC,OAAO,CAAC,CAAC;IACxC,IAAI,CAAC;QACH,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,KAAK,EAAE;YAC7B,OAAO,EAAE,EAAE,GAAG,aAAa,EAAE,MAAM,EAAE,2BAA2B,EAAE;YAClE,QAAQ,EAAE,QAAQ;SACnB,CAAC,CAAC;QACH,IAAI,CAAC,GAAG,CAAC,EAAE;YAAE,OAAO,IAAI,CAAC;QACzB,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC;QAC9B,OAAO,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC;IACvC,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,IAAI,CAAC;IACd,CAAC;AACH,CAAC;AAED,KAAK,UAAU,iBAAiB,CAAC,GAAW;IAC1C,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,GAAG,EAAE;QAC3B,OAAO,EAAE,EAAE,GAAG,aAAa,EAAE,MAAM,EAAE,WAAW,EAAE;KACnD,CAAC,CAAC;IACH,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;QACZ,MAAM,IAAI,KAAK,CAAC,yBAAyB,GAAG,CAAC,MAAM,IAAI,GAAG,CAAC,UAAU,EAAE,CAAC,CAAC;IAC3E,CAAC;IACD,OAAO,kBAAkB,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC;AAC9C,CAAC;AAED,mCAAmC;AAEnC,SAAS,kBAAkB,CAAC,IAAY;IACtC,IAAI,OAAO,GAAG,IAAI,CAAC;IACnB,MAAM,SAAS,GACb,OAAO,CAAC,KAAK,CAAC,gCAAgC,CAAC;QAC/C,OAAO,CAAC,KAAK,CAAC,sCAAsC,CAAC;QACrD,OAAO,CAAC,KAAK,CAAC,4DAA4D,CAAC,CAAC;IAE9E,IAAI,SAAS;QAAE,OAAO,GAAG,SAAS,CAAC,CAAC,CAAC,CAAC;IAEtC,OAAO,OAAO;SACX,OAAO,CAAC,mCAAmC,EAAE,EAAE,CAAC;SAChD,OAAO,CAAC,iCAAiC,EAAE,EAAE,CAAC;SAC9C,OAAO,CAAC,6BAA6B,EAAE,EAAE,CAAC;SAC1C,OAAO,CAAC,mCAAmC,EAAE,EAAE,CAAC;SAChD,OAAO,CAAC,mCAAmC,EAAE,EAAE,CAAC;SAChD,OAAO,CAAC,6BAA6B,EAAE,UAAU,CAAC;SAClD,OAAO,CAAC,6BAA6B,EAAE,WAAW,CAAC;SACnD,OAAO,CAAC,6BAA6B,EAAE,YAAY,CAAC;SACpD,OAAO,CAAC,6BAA6B,EAAE,aAAa,CAAC;SACrD,OAAO,CAAC,kDAAkD,EAAE,kBAAkB,CAAC;SAC/E,OAAO,CAAC,iCAAiC,EAAE,MAAM,CAAC;SAClD,OAAO,CAAC,8CAA8C,EAAE,UAAU,CAAC;SACnE,OAAO,CAAC,6BAA6B,EAAE,QAAQ,CAAC;SAChD,OAAO,CAAC,mBAAmB,EAAE,IAAI,CAAC;SAClC,OAAO,CAAC,2BAA2B,EAAE,QAAQ,CAAC;SAC9C,OAAO,CAAC,cAAc,EAAE,IAAI,CAAC;SAC7B,OAAO,CAAC,WAAW,EAAE,IAAI,CAAC;SAC1B,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC;SACvB,OAAO,CAAC,QAAQ,EAAE,GAAG,CAAC;SACtB,OAAO,CAAC,OAAO,EAAE,GAAG,CAAC;SACrB,OAAO,CAAC,OAAO,EAAE,GAAG,CAAC;SACrB,OAAO,CAAC,SAAS,EAAE,GAAG,CAAC;SACvB,OAAO,CAAC,QAAQ,EAAE,GAAG,CAAC;SACtB,OAAO,CAAC,SAAS,EAAE,GAAG,CAAC;SACvB,OAAO,CAAC,SAAS,EAAE,MAAM,CAAC;SAC1B,IAAI,EAAE,CAAC;AACZ,CAAC;AAED,+BAA+B;AAC/B,EAAE;AACF,+EAA+E;AAC/E,uEAAuE;AAEvE,MAAM,UAAU,gBAAgB,CAAC,GAAW;IAC1C,IAAI,GAAG,GAAG,GAAG,CAAC;IAEd,+DAA+D;IAC/D,MAAM,gBAAgB,GAAG,GAAG,CAAC,KAAK,CAAC,sCAAsC,CAAC,CAAC;IAC3E,IAAI,gBAAgB,EAAE,CAAC;QACrB,GAAG,GAAG,GAAG,CAAC,KAAK,CAAC,gBAAgB,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAC9C,CAAC;IAED,4EAA4E;IAC5E,sDAAsD;IAEtD,4CAA4C;IAC5C,GAAG,GAAG,GAAG,CAAC,OAAO,CAAC,SAAS,EAAE,MAAM,CAAC,CAAC;IAErC,OAAO,GAAG,CAAC,IAAI,EAAE,CAAC;AACpB,CAAC;AAED,oDAAoD;AAEpD,MAAM,UAAU,aAAa,CAAC,OAAe,EAAE,MAAc;IAC3D,IAAI,OAAO,CAAC,MAAM,IAAI,MAAM;QAAE,OAAO,OAAO,CAAC;IAE7C,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAClC,IAAI,SAAS,GAAG,CAAC,CAAC;IAClB,IAAI,cAAc,GAAG,CAAC,CAAC,CAAC;IACxB,IAAI,YAAY,GAAG,CAAC,CAAC,CAAC;IAEtB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACtC,SAAS,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC;QACjC,IAAI,SAAS,GAAG,MAAM;YAAE,MAAM;QAC9B,IAAI,WAAW,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;YAAE,cAAc,GAAG,CAAC,CAAC;QACnD,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,KAAK,EAAE;YAAE,YAAY,GAAG,CAAC,CAAC;IAC/C,CAAC;IAED,MAAM,SAAS,GAAG,KAAK,CAAC,MAAM,GAAG,GAAG,CAAC;IACrC,MAAM,MAAM,GACV,cAAc,GAAG,SAAS;QACxB,CAAC,CAAC,cAAc;QAChB,CAAC,CAAC,YAAY,GAAG,SAAS;YACxB,CAAC,CAAC,YAAY;YACd,CAAC,CAAC,CAAC,CAAC,CAAC;IAEX,IAAI,MAAM,GAAG,CAAC,EAAE,CAAC;QACf,MAAM,SAAS,GAAG,KAAK,CAAC,MAAM,GAAG,MAAM,CAAC;QACxC,OAAO,CACL,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE;YACxC,wBAAwB,SAAS,cAAc,CAChD,CAAC;IACJ,CAAC;IAED,OAAO,OAAO,CAAC,SAAS,CAAC,CAAC,EAAE,MAAM,CAAC,GAAG,6BAA6B,CAAC;AACtE,CAAC;AAED,8BAA8B;AAE9B,KAAK,UAAU,aAAa,CAAC,GAAW;IACtC,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,GAAG,CAAC,CAAC;IACpC,IAAI,EAAE;QAAE,OAAO,gBAAgB,CAAC,EAAE,CAAC,CAAC;IACpC,2BAA2B;IAC3B,OAAO,iBAAiB,CAAC,GAAG,CAAC,CAAC;AAChC,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,gBAAgB,CAAC,GAAW;IAChD,MAAM,YAAY,GAAG,kBAAkB,CAAC,GAAG,CAAC,CAAC;IAC7C,IAAI,YAAY;QAAE,OAAO,YAAY,CAAC;IAEtC,MAAM,UAAU,GAAG,MAAM,gBAAgB,CAAC,GAAG,CAAC,CAAC;IAC/C,IAAI,UAAU,EAAE,CAAC;QACf,MAAM,SAAS,GAAG,aAAa,CAAC,UAAU,EAAE,MAAM,CAAC,gBAAgB,CAAC,CAAC;QACrE,MAAM,MAAM,GAAG,WAAW,GAAG,OAAO,SAAS,EAAE,CAAC;QAChD,cAAc,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;QAC5B,OAAO,MAAM,CAAC;IAChB,CAAC;IAED,MAAM,UAAU,GAAG,MAAM,aAAa,CAAC,GAAG,CAAC,CAAC;IAC5C,MAAM,YAAY,CAAC,GAAG,EAAE,UAAU,CAAC,CAAC;IAEpC,MAAM,SAAS,GAAG,aAAa,CAAC,UAAU,EAAE,MAAM,CAAC,gBAAgB,CAAC,CAAC;IACrE,MAAM,MAAM,GAAG,WAAW,GAAG,OAAO,SAAS,EAAE,CAAC;IAChD,cAAc,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;IAC5B,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,eAAe,CAAC,GAAW;IAC/C,MAAM,UAAU,GAAG,MAAM,gBAAgB,CAAC,GAAG,CAAC,CAAC;IAC/C,IAAI,UAAU;QAAE,OAAO,UAAU,CAAC;IAElC,MAAM,UAAU,GAAG,MAAM,aAAa,CAAC,GAAG,CAAC,CAAC;IAC5C,MAAM,YAAY,CAAC,GAAG,EAAE,UAAU,CAAC,CAAC;IACpC,OAAO,UAAU,CAAC;AACpB,CAAC;AAED,qCAAqC;AAErC,MAAM,UAAU,mBAAmB,CACjC,QAAgB;IAEhB,MAAM,QAAQ,GAAyC,EAAE,CAAC;IAC1D,MAAM,MAAM,GAAG,0BAA0B,CAAC;IAC1C,IAAI,CAAC,CAAC;IACN,OAAO,CAAC,CAAC,GAAG,MAAM,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;QAC5C,MAAM,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QACzB,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YACpB,QAAQ,CAAC,IAAI,CAAC,EAAE,QAAQ,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;QACpD,CAAC;IACH,CAAC;IACD,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED,MAAM,UAAU,cAAc,CAAC,QAAgB;IAC7C,MAAM,OAAO,GAAe,EAAE,CAAC;IAC/B,MAAM,OAAO,GAAG,qBAAqB,CAAC;IACtC,IAAI,CAAC,CAAC;IACN,OAAO,CAAC,CAAC,GAAG,OAAO,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC;QAC7C,OAAO,CAAC,IAAI,CAAC;YACX,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM;YAClB,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,OAAO,CAAC,QAAQ,EAAE,EAAE,CAAC;SACzC,CAAC,CAAC;IACL,CAAC;IACD,OAAO,OAAO,CAAC;AACjB,CAAC"}

package/dist/index.d.ts

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+export {};
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":""}