wiki-plugin-similarity 0.4.4 → 0.5.1

package/pages/about-similarity-plugin

@@ -3,70 +3,66 @@
   "story": [
     {
       "type": "markdown",
-      "id": "61ccb2d2cdb60c71",
-      "text": "The Similarity plugin adds semantic similarity search to a Federated Wiki farm. One item type — **similarity** — with behaviour controlled by a DSL: interactive search, ambient similar-pages panel, domain listing, ranked search reports, ghost-page authoring, index building, and generic report buttons."
+      "id": "aec1dfcd5f2bc42b",
+      "text": "The Similarity plugin adds search to a Federated Wiki farm — semantic similarity over prebuilt vector indexes, ranked search reports, and galactic keyword search over each site's own MiniSearch index. One item type — **similarity** — with a line-oriented DSL, plus a server component that serves every search same-origin. Works on any host: embedding runs in-process (native ONNX where available, WASM everywhere else)."
     },
     {
       "type": "markdown",
-      "id": "a3f7c8d2e1b94056",
-      "text": "**Version:** 0.4.3 — [GitHub](https://github.com/Hitchhikers-Guide-to-the-Galaxy/wiki-plugin-similarity) · [npm](https://www.npmjs.com/package/wiki-plugin-similarity)"
-    },
-    {
-      "type": "similarity",
-      "id": "ad3c41667c285efd",
-      "text": "SIMILAR: medium\nLIMIT: 5"
+      "id": "28edff984cc5334d",
+      "text": "## The DSL\n\nThe first ALL-CAPS keyword sets the mode; other lines are domain specs (globs, explicit domains, or the scope keywords PUBLIC / LOCAL / PRIVATE):\n\n- (no keyword) — interactive semantic search form\n- `SIMILAR: high|medium|low` — ambient: pages similar to this page\n- `REPORT` — server-side ranked, stub-filtered, fork-bundled report\n- `KEYWORD` — galactic MiniSearch over live site-index.json files\n- `LIST` — table of indexed domains\n- `AUTHOR` — search results as a draggable ghost page\n- `BUILD` / `FORCE` — proxy to the farm indexer, where configured\n- `GHOST url` / `BUTTON label` — open any page-json as a ghost page\n- `THRESHOLD: 0.72` · `LIMIT: 8` · `LIVE` — tuning"
     },
     {
       "type": "markdown",
-      "id": "bdf93e3860d64ebb",
-      "text": "## The DSL"
+      "id": "251793ca0dae4810",
+      "text": "## Try It\n\nSemantic search across the farm:"
     },
     {
-      "type": "markdown",
-      "id": "0d628d43edf9f8e8",
-      "text": "The item's `text` field configures what the item does. Mode commands come first; the rest are options and domain specs:\n\n```\nSIMILAR              ambient mode — similar pages to this one\nSIMILAR high         threshold preset (high=0.78 / medium=0.68 / low=0.58)\nLIST                 table of indexed domains with page counts\nAUTHOR               search → ghost page of [[wiki links]] + references\nREPORT               server-side ranked report (stubs dropped, forks bundled)\nBUILD                button that builds semantic indexes\nGHOST <url>          button that opens any page-json URL as a ghost page\n\nTHRESHOLD 0.72       exact cosine threshold\nLIMIT 8              max results (default 10)\nLIVE                 bypass the localStorage cache\nFORCE                BUILD: re-embed even when the index is fresh\nBUTTON <text>        custom button caption (BUILD / GHOST)\n```\n\nColons and extra whitespace are optional — `LIMIT: 8`, `LIMIT 8`, and `LIMIT:8` all work. Leave `text` empty for an interactive search form on the current domain."
+      "type": "similarity",
+      "id": "86aeab8be16dd8e1",
+      "text": "*"
     },
     {
       "type": "markdown",
-      "id": "1e2f3a4b5c6d7e8f",
-      "text": "## Domain Specs and Scopes"
+      "id": "b6a9be0145bae9bd",
+      "text": "Pages similar to this one:"
     },
     {
-      "type": "markdown",
-      "id": "2f3a4b5c6d7e8f90",
-      "text": "Any other line is a domain spec. Scope keywords expand server-side:\n\n```\n*                        all indexed domains, all farms\nPUBLIC                   the Nextcloud mirror farm\nLOCAL                    the localhost farm\nPRIVATE                  public wikis with restricted: true in the farm config\ndavid.*                  glob pattern\ndavid.hitchhikers.earth  explicit domain\n# comment                ignored\n```\n\nSpecs combine — `LOCAL` and `david.*` on separate lines searches both."
+      "type": "similarity",
+      "id": "cf558a16395d7d8c",
+      "text": "SIMILAR: medium\nLIMIT: 5"
     },
     {
       "type": "markdown",
-      "id": "c0775c94a895e9d2",
-      "text": "## Modes"
+      "id": "1aceb7035c7537b2",
+      "text": "A ranked report (opens as a ghost page):"
     },
     {
-      "type": "markdown",
-      "id": "4f91a3b2d8e05c11",
-      "text": "**Search** (default) — interactive form; results appear inline as scored links. Shift-click a result to append it at the end of the lineup instead of truncating.\n\n**Similar** — runs on page load using the current page's pre-computed vector (no embed call if already indexed). Ranked *Similar Pages* list.\n\n**List** — table of indexed domains with page counts; supports scopes, globs, and `LIMIT`.\n\n**Author** — same search, but opens a ghost page with `# Similar Pages` ([[wiki links]], deduplicated) and `# Reference Links` (all results as reference items with scores).\n\n**Report** — posts to the farm API's `/search-report`: candidates from semantic scan ∪ title matching, stubs dropped, forks and near-duplicates bundled under the strongest copy with a one-click lineup link, ranked by semantic × richness × recency + title boost. Opens as a ghost page. *(localhost only — needs the FastAPI service.)*\n\n**Build** — a button that triggers semantic index building via `/build-index.json`; the build report opens as a ghost page. Add `FORCE` to re-embed fresh domains. *(localhost only.)*\n\n**Ghost** — a generic button: `GHOST <url>` fetches any endpoint returning fedwiki page JSON and opens it as a ghost page. Caption set with `BUTTON <text>`. Example: the *Farm Roster Report* button fetches `/farm-rosters.json`, which rosters every farm domain by scope with per-section error rosters for unserved domains."
+      "type": "similarity",
+      "id": "59bfe1e3968743a1",
+      "text": "REPORT\nLIMIT: 5"
     },
     {
       "type": "markdown",
-      "id": "c0775c94a895e9d3",
-      "text": "## Caching"
+      "id": "31b81a31f675c4e1",
+      "text": "Keyword search over the sites' own indexes:"
     },
     {
-      "type": "markdown",
-      "id": "9a1b2c3d4e5f6001",
-      "text": "Results are cached in `localStorage` keyed by item id and invalidated whenever the item's `text` changes. Add `LIVE` anywhere in the DSL to bypass the cache."
+      "type": "similarity",
+      "id": "0c3fb9298ba8fd23",
+      "text": "KEYWORD\nLIMIT: 5"
     },
     {
       "type": "markdown",
-      "id": "c0775c94a895e9d4",
-      "text": "## How It Works"
+      "id": "d9a96dfebb9d1692",
+      "text": "## Server Routes\n\nAll same-origin, registered by the plugin's server component:\n\n- `GET /system/indexed-domains.json?pattern=`\n- `GET /system/semantic-vectors.json?domain=`\n- `GET /system/embed.json?text=`\n- `POST /system/search-report.json`\n- `GET /system/farm-search.json?q=&pattern=&limit=`\n- `GET /system/build-index.json?domains=` (proxied to the farm indexer)\n\nIndexes are `status/semantic-vectors.json` per domain — built by the farm indexer, distributed by sync, read from farm disk. Keyword search reads each site's per-edit `status/site-index.json`; no separate index is built. Query embedding is BAAI bge-small-en-v1.5, 384-dim, CLS-pooled, unit-normalised — byte-compatible with the index builder."
     },
     {
       "type": "markdown",
-      "id": "4b1bbe95544548eb",
-      "text": "Pages are pre-indexed as 384-dimensional embedding vectors using BAAI/bge-small-en-v1.5 via fastembed. The plugin loads `semantic-vectors.json` index files directly in the browser and scores them with cosine similarity in JavaScript — no remote search engine required.\n\nThe server component registers `/system/indexed-domains.json`, `/system/semantic-vectors.json`, and `/system/embed.json` routes, and understands the scope keywords. See [[Similarity Indexing]] for building indices and [[Build Semantic Index]] for triggering builds from the wiki."
+      "id": "21eb5354d2adb29e",
+      "text": "**Version:** 0.5.1\n\n# Links\n\n- [https://github.com/Hitchhikers-Guide-to-the-Galaxy/wiki-plugin-similarity GitHub repository]\n- [https://www.npmjs.com/package/wiki-plugin-similarity npm package]\n- [https://plugin.fedwiki.club/similarity-plugin.html Plugin documentation]\n- [https://search.fedwiki.club/ Search hub] — architecture, indexing, freshness."
     }
   ],
+  "plugin": "similarity",
   "journal": [
     {
       "type": "create",
@@ -75,14 +71,68 @@
         "story": [
           {
             "type": "markdown",
-            "id": "61ccb2d2cdb60c71",
-            "text": "The Similarity plugin adds semantic similarity search to a Federated Wiki farm."
+            "id": "aec1dfcd5f2bc42b",
+            "text": "The Similarity plugin adds search to a Federated Wiki farm — semantic similarity over prebuilt vector indexes, ranked search reports, and galactic keyword search over each site's own MiniSearch index. One item type — **similarity** — with a line-oriented DSL, plus a server component that serves every search same-origin. Works on any host: embedding runs in-process (native ONNX where available, WASM everywhere else)."
+          },
+          {
+            "type": "markdown",
+            "id": "28edff984cc5334d",
+            "text": "## The DSL\n\nThe first ALL-CAPS keyword sets the mode; other lines are domain specs (globs, explicit domains, or the scope keywords PUBLIC / LOCAL / PRIVATE):\n\n- (no keyword) — interactive semantic search form\n- `SIMILAR: high|medium|low` — ambient: pages similar to this page\n- `REPORT` — server-side ranked, stub-filtered, fork-bundled report\n- `KEYWORD` — galactic MiniSearch over live site-index.json files\n- `LIST` — table of indexed domains\n- `AUTHOR` — search results as a draggable ghost page\n- `BUILD` / `FORCE` — proxy to the farm indexer, where configured\n- `GHOST url` / `BUTTON label` — open any page-json as a ghost page\n- `THRESHOLD: 0.72` · `LIMIT: 8` · `LIVE` — tuning"
+          },
+          {
+            "type": "markdown",
+            "id": "251793ca0dae4810",
+            "text": "## Try It\n\nSemantic search across the farm:"
+          },
+          {
+            "type": "similarity",
+            "id": "86aeab8be16dd8e1",
+            "text": "*"
+          },
+          {
+            "type": "markdown",
+            "id": "b6a9be0145bae9bd",
+            "text": "Pages similar to this one:"
+          },
+          {
+            "type": "similarity",
+            "id": "cf558a16395d7d8c",
+            "text": "SIMILAR: medium\nLIMIT: 5"
+          },
+          {
+            "type": "markdown",
+            "id": "1aceb7035c7537b2",
+            "text": "A ranked report (opens as a ghost page):"
+          },
+          {
+            "type": "similarity",
+            "id": "59bfe1e3968743a1",
+            "text": "REPORT\nLIMIT: 5"
+          },
+          {
+            "type": "markdown",
+            "id": "31b81a31f675c4e1",
+            "text": "Keyword search over the sites' own indexes:"
+          },
+          {
+            "type": "similarity",
+            "id": "0c3fb9298ba8fd23",
+            "text": "KEYWORD\nLIMIT: 5"
+          },
+          {
+            "type": "markdown",
+            "id": "d9a96dfebb9d1692",
+            "text": "## Server Routes\n\nAll same-origin, registered by the plugin's server component:\n\n- `GET /system/indexed-domains.json?pattern=`\n- `GET /system/semantic-vectors.json?domain=`\n- `GET /system/embed.json?text=`\n- `POST /system/search-report.json`\n- `GET /system/farm-search.json?q=&pattern=&limit=`\n- `GET /system/build-index.json?domains=` (proxied to the farm indexer)\n\nIndexes are `status/semantic-vectors.json` per domain — built by the farm indexer, distributed by sync, read from farm disk. Keyword search reads each site's per-edit `status/site-index.json`; no separate index is built. Query embedding is BAAI bge-small-en-v1.5, 384-dim, CLS-pooled, unit-normalised — byte-compatible with the index builder."
+          },
+          {
+            "type": "markdown",
+            "id": "21eb5354d2adb29e",
+            "text": "**Version:** 0.5.1\n\n# Links\n\n- [https://github.com/Hitchhikers-Guide-to-the-Galaxy/wiki-plugin-similarity GitHub repository]\n- [https://www.npmjs.com/package/wiki-plugin-similarity npm package]\n- [https://plugin.fedwiki.club/similarity-plugin.html Plugin documentation]\n- [https://search.fedwiki.club/ Search hub] — architecture, indexing, freshness."
           }
         ]
       },
-      "date": 1781133629000,
-      "certificate": "from marvin"
+      "date": 1783782734556,
+      "provenance": "Authored by an AI agent — Claude Fable 5 via the [[fedwiki-page]] skill. [https://claude.com/claude-code Claude Code]"
     }
-  ],
-  "plugin": "similarity"
-}
+  ]
+}

package/server/embedder.js

@@ -0,0 +1,71 @@
+// wiki-plugin-similarity — in-process query embedder (CommonJS)
+//
+// Reproduces the farm indexer's embeddings (fastembed, BAAI/bge-small-en-v1.5,
+// 384-dim, L2-normalised, NO query prefix) using transformers.js — pure
+// JS/WASM ONNX, no native dependencies, so it runs inside any wiki-server
+// install including the public farm's container.
+//
+// Parity notes (verified against the FastAPI /embed reference):
+//   - pooling and quantisation are configurable via env because they are the
+//     two knobs that can desync scores from the prebuilt semantic-vectors.json:
+//       WIKI_EMBED_POOLING   cls | mean   (default cls — BGE models use CLS)
+//       WIKI_EMBED_QUANTIZED 1 | 0        (default 0 — fp32 matches fastembed)
+//   - never add an instruction prefix: the index was built with plain .embed().
+//
+// Model files download on first use (~130MB fp32) into WIKI_MODEL_CACHE if set,
+// else transformers.js's default cache. Subsequent runs are offline.
+
+const MODEL     = process.env.WIKI_EMBED_MODEL || 'Xenova/bge-small-en-v1.5'
+const POOLING   = process.env.WIKI_EMBED_POOLING || 'cls'
+const QUANTIZED = process.env.WIKI_EMBED_QUANTIZED === '1'
+
+// transformers.js statically imports onnxruntime-node, whose native binary
+// needs glibc. On musl containers (node:alpine — the public farm) that import
+// throws "Error loading shared library ld-linux-x86-64.so.2". Probe the
+// native runtime with a CJS require first; if it can't load, register a
+// module hook (Node >= 18.19) that redirects the onnxruntime-node import to
+// onnxruntime-web — the WASM backend runs everywhere and benchmarks ~25ms a
+// query. WIKI_EMBED_FORCE_WASM=1 forces the WASM path for testing.
+
+const nativeOrtLoads = () => {
+  if (process.env.WIKI_EMBED_FORCE_WASM === '1') return false
+  try { require('onnxruntime-node'); return true } catch { return false }
+}
+
+let extractorPromise = null
+
+const getExtractor = () => {
+  if (!extractorPromise) {
+    extractorPromise = (async () => {
+      if (!nativeOrtLoads()) {
+        const { register } = require('node:module')
+        if (typeof register !== 'function') {
+          throw new Error('native onnxruntime unavailable and Node lacks ' +
+            'module.register (need >= 18.19) for the WASM fallback')
+        }
+        register('./ort-hooks.mjs', require('node:url').pathToFileURL(__filename))
+        console.log('[wiki-plugin-similarity] native onnxruntime unavailable — using WASM backend')
+      }
+      // Dynamic import: @xenova/transformers is ESM-only; this file must stay CJS.
+      const { pipeline, env } = await import('@xenova/transformers')
+      if (process.env.WIKI_MODEL_CACHE) env.cacheDir = process.env.WIKI_MODEL_CACHE
+      if (env.backends?.onnx?.wasm) env.backends.onnx.wasm.numThreads = 1
+      return pipeline('feature-extraction', MODEL, { quantized: QUANTIZED })
+    })()
+    extractorPromise.catch(() => { extractorPromise = null }) // allow retry
+  }
+  return extractorPromise
+}
+
+// Embed one text → plain number[384], unit-normalised.
+const embed = async text => {
+  const extractor = await getExtractor()
+  const out = await extractor(text, { pooling: POOLING, normalize: true })
+  return Array.from(out.data)
+}
+
+// Warm the model in the background (call at startup; failures just defer to
+// first request).
+const warm = () => { getExtractor().catch(() => {}) }
+
+module.exports = { embed, warm }

package/server/farm-lib.js

@@ -0,0 +1,102 @@
+// wiki-plugin-similarity — shared farm helpers (CommonJS, see server/package.json)
+//
+// Used by server.js, search-report.js and farm-search.js. A "farms" list is
+// [[rootPath, kind], ...] where kind is 'local' (the primary farm this wiki
+// serves from) or 'public' (extra farms, e.g. the Nextcloud mirror).
+
+const fs   = require('node:fs')
+const path = require('node:path')
+
+// ── Glob matching — supports * and ?, no path separator semantics ─────────────
+
+const globMatch = (pattern, str) => {
+  const p = pattern.length
+  const s = str.length
+  const dp = Array.from({ length: p + 1 }, () => new Array(s + 1).fill(false))
+  dp[0][0] = true
+  for (let i = 1; i <= p; i++) {
+    if (pattern[i - 1] === '*') dp[i][0] = dp[i - 1][0]
+  }
+  for (let i = 1; i <= p; i++) {
+    for (let j = 1; j <= s; j++) {
+      if (pattern[i - 1] === '*') {
+        dp[i][j] = dp[i - 1][j] || dp[i][j - 1]
+      } else if (pattern[i - 1] === '?' || pattern[i - 1] === str[j - 1]) {
+        dp[i][j] = dp[i - 1][j - 1]
+      }
+    }
+  }
+  return dp[p][s]
+}
+
+// ── Scope keywords (exact uppercase, per DSL convention) ──────────────────────
+//   *        all farms
+//   PUBLIC   domains in 'public' farms (Nextcloud mirror)
+//   LOCAL    domains in the primary ('local') farm
+//   PRIVATE  public domains marked "restricted": true in a farm config
+
+const loadRestricted = publicFarms => {
+  const restricted = new Set()
+  for (const farm of publicFarms) {
+    let files
+    try { files = fs.readdirSync(farm) } catch { continue }
+    for (const f of files) {
+      if (!/^config-.*\.json$/.test(f)) continue
+      try {
+        const cfg = JSON.parse(fs.readFileSync(path.join(farm, f), 'utf8'))
+        for (const [domain, opts] of Object.entries(cfg.wikiDomains || {})) {
+          if (opts && opts.restricted) restricted.add(domain)
+        }
+      } catch { /* ignore malformed config */ }
+    }
+  }
+  return restricted
+}
+
+const matchesAny = (domain, kind, patterns, restricted) =>
+  patterns.some(p => {
+    if (p === '*') return true
+    if (p === 'PUBLIC') return kind === 'public'
+    if (p === 'LOCAL') return kind === 'local'
+    if (p === 'PRIVATE') return kind === 'public' && restricted.has(domain)
+    return globMatch(p, domain)
+  })
+
+// ── Domain listing ────────────────────────────────────────────────────────────
+// List domains across farms matching the patterns, optionally requiring a file
+// (relative to the domain dir) to exist. First farm wins on duplicate names.
+// Returns [{farm, kind, domain}].
+
+const listDomains = (farms, patterns, restricted, requireFile = null) => {
+  const seen = new Set()
+  const out = []
+  for (const [farm, kind] of farms) {
+    let entries
+    try { entries = fs.readdirSync(farm, { withFileTypes: true }) } catch { continue }
+    for (const ent of entries) {
+      if (!ent.isDirectory()) continue
+      const domain = ent.name
+      if (seen.has(domain)) continue
+      if (!matchesAny(domain, kind, patterns, restricted)) continue
+      if (requireFile) {
+        try { fs.accessSync(path.join(farm, domain, requireFile), fs.constants.F_OK) }
+        catch { continue }
+      }
+      seen.add(domain)
+      out.push({ farm, kind, domain })
+    }
+  }
+  out.sort((a, b) => a.domain.localeCompare(b.domain))
+  return out
+}
+
+// First existing path for domain + relative sub-path across farm roots.
+const findInFarms = (farms, domain, relPath) => {
+  for (const [farm] of farms) {
+    const full = path.join(farm, domain, relPath)
+    try { fs.accessSync(full, fs.constants.F_OK); return full } catch { /* next */ }
+  }
+  return null
+}
+
+module.exports = { globMatch, loadRestricted, matchesAny, listDomains, findInFarms }

package/server/farm-search.js

@@ -0,0 +1,122 @@
+// wiki-plugin-similarity — galactic keyword search (CommonJS)
+//
+// Fedwiki maintains a serialized MiniSearch index per site: the wiki-server
+// updates {domain}/status/site-index.json on every edit. This module READS
+// those indexes across every domain on the farm(s) — no index building of any
+// kind. On the farm server the files are per-edit fresh because the server
+// itself writes them.
+//
+// The server-built index stores no fields (storedFields is empty), so display
+// titles come from each domain's status/sitemap.json.
+//
+// Loaded indexes are cached per domain and invalidated by file mtime.
+
+const fs   = require('node:fs')
+const path = require('node:path')
+const crypto = require('node:crypto')
+
+const MiniSearch = require('minisearch')
+const { listDomains } = require('./farm-lib')
+
+// site-index.json was serialized with these options; loadJSON needs them.
+const MS_OPTIONS = { fields: ['title', 'content'] }
+
+const makeId = () => crypto.randomBytes(8).toString('hex')
+
+// ── Per-domain index cache, mtime-invalidated ─────────────────────────────────
+
+const cache = new Map() // domain → {mtime, ms, titles, synopses}
+
+const loadDomainIndex = (farm, domain) => {
+  const indexPath = path.join(farm, domain, 'status', 'site-index.json')
+  let mtime
+  try { mtime = fs.statSync(indexPath).mtimeMs } catch { return null }
+
+  const hit = cache.get(domain)
+  if (hit && hit.mtime === mtime) return hit
+
+  let ms
+  try {
+    ms = MiniSearch.loadJSON(fs.readFileSync(indexPath, 'utf8'), MS_OPTIONS)
+  } catch { return null }
+
+  // Titles + synopses from the sitemap (the index stores no fields).
+  const titles = new Map()
+  const synopses = new Map()
+  try {
+    const sitemap = JSON.parse(fs.readFileSync(
+      path.join(farm, domain, 'status', 'sitemap.json'), 'utf8'))
+    for (const p of sitemap) {
+      titles.set(p.slug, p.title)
+      if (p.synopsis) synopses.set(p.slug, p.synopsis)
+    }
+  } catch { /* titles fall back to slugs */ }
+
+  const entry = { mtime, ms, titles, synopses }
+  cache.set(domain, entry)
+  return entry
+}
+
+// ── Search across domains ─────────────────────────────────────────────────────
+
+const searchFarm = (farms, patterns, restricted, query, limit) => {
+  const domains = listDomains(farms, patterns, restricted, 'status/site-index.json')
+  const results = []
+  let searched = 0
+  for (const { farm, domain } of domains) {
+    const entry = loadDomainIndex(farm, domain)
+    if (!entry) continue
+    searched += 1
+    let hits
+    try { hits = entry.ms.search(query, { boost: { title: 2 } }) } catch { continue }
+    for (const h of hits) {
+      results.push({
+        domain,
+        slug: String(h.id),
+        title: entry.titles.get(String(h.id)) || String(h.id),
+        synopsis: entry.synopses.get(String(h.id)) || '',
+        score: h.score,
+      })
+    }
+  }
+  results.sort((a, b) => b.score - a.score)
+  return { results: results.slice(0, limit), searched, matched: results.length }
+}
+
+// ── Report page JSON (ghost-page ready, mirrors search-report style) ──────────
+
+const keywordReportPage = (query, { results, searched, matched }, limit, specs) => {
+  const story = [{
+    type: 'markdown', id: makeId(),
+    text: `Keyword search for **${query}** — searched the live site indexes of ` +
+      `${searched} domains; ${matched} matches, top ${Math.min(limit, results.length)} shown.\n\n` +
+      `<small>Config — domains: ${(specs || ['*']).join(', ')}; limit: ${limit}. ` +
+      `Reads each site's own per-edit MiniSearch index (site-index.json) — ` +
+      `no separate index is built.</small>`,
+  }]
+
+  const seenTitles = new Set()
+  const uniqueTitles = results.map(r => r.title).filter(t => {
+    if (seenTitles.has(t)) return false
+    seenTitles.add(t)
+    return true
+  })
+  if (uniqueTitles.length) {
+    story.push({
+      type: 'markdown', id: makeId(),
+      text: uniqueTitles.map(t => `- [[${t}]]`).join('\n'),
+    })
+  }
+
+  story.push({ type: 'markdown', id: makeId(), text: '# Results' })
+  for (const r of results) {
+    story.push({
+      type: 'reference', id: makeId(),
+      site: r.domain, slug: r.slug, title: r.title,
+      text: r.synopsis || `score ${r.score.toFixed(2)}`,
+    })
+  }
+  return { title: `${query} Keyword Search`, story }
+}
+
+module.exports = { searchFarm, keywordReportPage }

package/server/ort-hooks.mjs

@@ -0,0 +1,6 @@
+export async function resolve(specifier, context, nextResolve) {
+  if (specifier === 'onnxruntime-node') {
+    return nextResolve(new URL('./ort-shim.mjs', import.meta.url).href, context)
+  }
+  return nextResolve(specifier, context)
+}

package/server/ort-shim.mjs

@@ -0,0 +1,2 @@
+import ort from 'onnxruntime-web'
+export default ort