@sdsrs/llm-wiki 0.6.5 → 0.6.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,40 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.6.6 (2026-07-11)
4
+
5
+ Audit remediation batch (roadmap M2 complete + M3/M4). No breaking changes, no KB
6
+ migration; new config keys default to prior behavior. Suite 185 → 208.
7
+
8
+ - **feat(cli): `llm-wiki --version`** prints the package version (was an unknown
9
+ option).
10
+ - **fix(net): LLM and embedding calls now time out and retry.** `chat/completions`
11
+ and `embeddings` requests go through a shared helper with an `AbortSignal.timeout`
12
+ ceiling (120s) and exponential-backoff retry on 429 / 5xx / network errors — a
13
+ hung or rate-limited provider no longer hangs the CLI or aborts a whole `embed`
14
+ run on one transient failure.
15
+ - **fix(embed): incremental durability.** `embed` now persists the vector store
16
+ after each batch, so a later batch failing keeps earlier batches' work (the
17
+ re-run reuses them by content hash — no repeated API cost). The store is written
18
+ atomically (temp + rename), as is `.manifest.json`.
19
+ - **fix(manifest): tolerate a hand-edited manifest** missing or mistyping the
20
+ `files` key instead of throwing deep in the diff.
21
+ - **fix(index): no line injection from frontmatter.** Newlines in a page's
22
+ title/description are collapsed before they are written into `index.md` /
23
+ `llms.txt`, so a crafted value can't inject headings or spoof wikilinks. Stale
24
+ `topics/*.md` are pruned when a type empties or the KB drops back below the split
25
+ threshold.
26
+ - **fix(export): escape newlines in Cypher** node titles so they can't split a
27
+ statement.
28
+ - **fix(docs): `@0` pin in the generated per-KB README**; corrected a fabricated
29
+ `supersedes` relation type in the main README to the real vocabulary (with a note
30
+ that `superseded_by` is a structural edge, not a `relations:` type); command
31
+ reference now lists `connect` / `install-skills` and documents `--version` /
32
+ `--follow-symlinks`; added an "Operational envelope" section (scale, no concurrent
33
+ writers, size cap, kana/hangul BM25 gap).
34
+ - Test coverage: `installSkills` + CLI-layer smokes for `index` / `lint` / `status`
35
+ / `connect` / `install-skills` / `export` / `graph` and the `-k` / `--depth` /
36
+ `--top` guards; a timeout on the MCP stdio handshake test.
37
+
3
38
  ## 0.6.5 (2026-07-11)
4
39
 
5
40
  Three security/robustness fixes from a full architecture-and-security audit
package/README.md CHANGED
@@ -65,8 +65,9 @@ npx @sdsrs/llm-wiki@0 ask "what did we decide about X?"
65
65
  (28 probes), enabling vectors lifted Recall@5 from 0.73 to 0.97 — the gap
66
66
  was almost entirely cross-language queries.
67
67
  - **A real link graph, queried without an LLM** — `graph path | neighbors |
68
- hubs` traverse wikilinks and typed relations (`implements`, `supersedes`, …)
69
- instantly.
68
+ hubs` traverse wikilinks and typed relations (`implements`, `contrasts_with`,
69
+ … — the `relationTypes` vocabulary; `superseded_by` is a separate structural
70
+ edge the CLI derives from frontmatter, not a `relations:` type) instantly.
70
71
  - **Agent-native, tool-agnostic** — the same KB serves a standalone CLI,
71
72
  Claude Code / Codex skills, an MCP server (Cursor, Windsurf, …), and opens
72
73
  directly as an Obsidian vault.
@@ -91,14 +92,31 @@ npx @sdsrs/llm-wiki@0 ask "what did we decide about X?"
91
92
  | `graph` | query `graph.json` with `path` / `neighbors` / `hubs` (zero-LLM traversal) |
92
93
  | `embed` | compute/update page embeddings (`wiki/.vectors.json`) for optional vector location |
93
94
  | `export` | export the graph as GraphML, Cypher, JSON Canvas, or an interactive HTML viewer; or the wiki as standard-markdown copies |
95
+ | `connect <projectDir>` | register a KB into a project's `CLAUDE.md` (sentinel block) so coding agents use it |
96
+ | `install-skills` | copy the bundled `wiki-*` skills into a `.claude` directory |
94
97
  | `mcp` | run a read-only MCP server (stdio) over the KB |
95
98
 
96
- All commands take `--kb <dir>` (default `.`). `ask` supports `-k <n>` (pages to
97
- load) and `--retrieve-only` (locate pages without calling the LLM).
99
+ Run `llm-wiki --version` to print the version. All commands take `--kb <dir>`
100
+ (default `.`). `ask` supports `-k <n>` (pages to load) and `--retrieve-only`
101
+ (locate pages without calling the LLM). `scan` supports `--exclude <pattern...>`
102
+ and `--follow-symlinks` (follow symlinked source files that resolve outside the
103
+ source dir — off by default, since such a link can read an arbitrary file into
104
+ the KB).
98
105
 
99
106
  `scan` also warns when the source looks multi-domain (mixed-language files or
100
107
  dispersed wiki tags) and suggests splitting — one KB per domain.
101
108
 
109
+ ### Operational envelope
110
+
111
+ llm-wiki targets **single-user, single-domain KBs at the hundreds-of-pages
112
+ scale**. Within that envelope everything is in-memory and re-read per operation
113
+ (no persistent index); there is no locking, so **do not run two writers against
114
+ one KB concurrently** (a CLI build alongside a long-lived MCP server is fine —
115
+ MCP is read-only). Source files above `maxFileBytes` (`wiki.config.json`, default
116
+ 50 MB) are skipped. `wiki/log.md` is append-only and not rotated. BM25 tokenizes
117
+ CJK by single char + bigram and ASCII by word, so purely **kana/hangul** queries
118
+ won't match — enable vector location for robust cross-language retrieval.
119
+
102
120
  ### Asking questions
103
121
 
104
122
  Retrieval is lexical (BM25) by default. When it finds nothing — typically a
package/bin/llm-wiki.mjs CHANGED
@@ -1,4 +1,5 @@
1
1
  #!/usr/bin/env node
2
+ import fs from 'node:fs'
2
3
  import path from 'node:path'
3
4
  import { fileURLToPath } from 'node:url'
4
5
  import { program } from 'commander'
@@ -15,7 +16,9 @@ import { exportGraph, loadGraph, exportMarkdownPages } from '../src/export.mjs'
15
16
  import { shortestPath, neighborhood, hubs } from '../src/graph.mjs'
16
17
  import { runMcpServer } from '../src/mcp.mjs'
17
18
 
18
- program.name('llm-wiki').description('Compile messy directories into an llm_wiki knowledge base')
19
+ const pkg = JSON.parse(fs.readFileSync(new URL('../package.json', import.meta.url), 'utf8'))
20
+
21
+ program.name('llm-wiki').description('Compile messy directories into an llm_wiki knowledge base').version(pkg.version)
19
22
 
20
23
  program.command('init [dir]').description('scaffold a knowledge base').action((dir = '.') => {
21
24
  const { created, skipped } = initKb(dir)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@sdsrs/llm-wiki",
3
- "version": "0.6.5",
3
+ "version": "0.6.6",
4
4
  "description": "Compile messy document directories into Karpathy-style llm_wiki knowledge bases — standalone Q&A CLI + Claude Code/Codex skills",
5
5
  "directories": {
6
6
  "test": "test"
package/src/ask.mjs CHANGED
@@ -8,6 +8,7 @@ import { estimateTokens } from './scanner.mjs'
8
8
  import { loadLlmConfig, makeTransport } from './llm-config.mjs'
9
9
  import { loadVectorStore, normalize, cosineTopK } from './vector.mjs'
10
10
  import { embedTexts } from './embed.mjs'
11
+ import { fetchWithRetry } from './retry.mjs'
11
12
 
12
13
  export function retrievePages(kbRoot, question, k = 6) {
13
14
  const pages = listWikiPages(kbRoot).filter(p => !p.error && !isInvalidated(p))
@@ -39,7 +40,7 @@ export function rrfFuse(lists, k) {
39
40
  // stderr warning. 'bm25': lexical only, returns before any vector access.
40
41
  // 'hybrid': explicit BM25+vector fusion — every missing prerequisite (and any
41
42
  // vector error) throws instead of degrading, and vectorEnabled is ignored.
42
- export async function locatePages(kbRoot, question, { k = 6, fetchImpl, retrieval = 'auto' } = {}) {
43
+ export async function locatePages(kbRoot, question, { k = 6, fetchImpl, retrieval = 'auto', retry } = {}) {
43
44
  if (!['auto', 'bm25', 'hybrid'].includes(retrieval)) throw new Error(`unknown retrieval mode: ${retrieval}`)
44
45
  const bm25 = retrievePages(kbRoot, question, k)
45
46
  const asBm25 = () => ({ hits: bm25.map(h => ({ ...h, sources: ['bm25'] })), usedVector: false })
@@ -59,7 +60,7 @@ export async function locatePages(kbRoot, question, { k = 6, fetchImpl, retrieva
59
60
  // fusing it produces silent garbage, so treat it as missing (not a failure).
60
61
  if (store.model !== cfg.embeddingModel) return unavailable(`vector store was built with "${store.model}" but config says "${cfg.embeddingModel}" — re-run \`llm-wiki embed\``)
61
62
  try {
62
- const t = fetchImpl ? { fetchImpl, dispatcher: undefined } : await makeTransport()
63
+ const t = fetchImpl ? { fetchImpl, dispatcher: undefined, retry } : { ...(await makeTransport()), retry }
63
64
  const [qv] = await embedTexts(cfg, t, [question])
64
65
  const qn = normalize(qv)
65
66
  const vecHits = qn ? cosineTopK(qn, store, k).map(v => ({ relPath: v.id, score: v.score })) : []
@@ -78,12 +79,11 @@ export async function locatePages(kbRoot, question, { k = 6, fetchImpl, retrieva
78
79
  }
79
80
 
80
81
  export async function chatCompletion(cfg, t, messages) {
81
- const res = await t.fetchImpl(`${cfg.baseURL.replace(/\/$/, '')}/chat/completions`, {
82
+ const res = await fetchWithRetry(t.fetchImpl, `${cfg.baseURL.replace(/\/$/, '')}/chat/completions`, {
82
83
  method: 'POST',
83
84
  headers: { 'content-type': 'application/json', authorization: `Bearer ${cfg.apiKey}` },
84
85
  body: JSON.stringify({ model: cfg.model, messages }),
85
- ...(t.dispatcher ? { dispatcher: t.dispatcher } : {}),
86
- })
86
+ }, { dispatcher: t.dispatcher, ...(t.retry ?? {}) })
87
87
  if (!res.ok) {
88
88
  let body = ''
89
89
  try { body = (await res.text()).slice(0, 200) } catch { /* body unreadable; status alone */ }
@@ -130,9 +130,9 @@ async function pickPagesFromListing(p, question, k, cfg, t, validIds) {
130
130
  return picked.map(id => ({ relPath: `${id}.md`, score: 0 }))
131
131
  }
132
132
 
133
- export async function askKb(kbRoot, question, { k = 6, retrieveOnly = false, fetchImpl, retrieval = 'auto' } = {}) {
133
+ export async function askKb(kbRoot, question, { k = 6, retrieveOnly = false, fetchImpl, retrieval = 'auto', retry } = {}) {
134
134
  const p = kbPaths(kbRoot)
135
- let { hits } = await locatePages(kbRoot, question, { k, fetchImpl, retrieval })
135
+ let { hits } = await locatePages(kbRoot, question, { k, fetchImpl, retrieval, retry })
136
136
  if (retrieveOnly) return { pages: hits, answer: null }
137
137
  let validIds
138
138
  if (hits.length === 0) {
@@ -145,7 +145,7 @@ export async function askKb(kbRoot, question, { k = 6, retrieveOnly = false, fet
145
145
  if (!cfg) throw new Error('No LLM configured. Create ~/.llm-wiki/config.json with {"baseURL","apiKey","model"} (OpenAI-compatible).')
146
146
  // Injected fetchImpl (tests) is used as-is with no dispatcher; otherwise
147
147
  // pick the proxy-aware transport (undici fetch + agent, or global fetch).
148
- const t = fetchImpl ? { fetchImpl, dispatcher: undefined } : await makeTransport()
148
+ const t = fetchImpl ? { fetchImpl, dispatcher: undefined, retry } : { ...(await makeTransport()), retry }
149
149
  let fallback = false
150
150
  if (hits.length === 0) {
151
151
  hits = await pickPagesFromListing(p, question, k, cfg, t, validIds)
package/src/embed.mjs CHANGED
@@ -2,6 +2,7 @@ import { listWikiPages, isInvalidated } from './pages.mjs'
2
2
  import { loadLlmConfig, makeTransport } from './llm-config.mjs'
3
3
  import { sha256Text } from './hashing.mjs'
4
4
  import { normalize, pageEmbedText, loadVectorStore, saveVectorStore } from './vector.mjs'
5
+ import { fetchWithRetry } from './retry.mjs'
5
6
 
6
7
  const BATCH = 64
7
8
  // Safety margin under the 8192-token input limit of common embedding models
@@ -44,12 +45,11 @@ function capEmbedText(text) {
44
45
  }
45
46
 
46
47
  export async function embedTexts(cfg, t, texts) {
47
- const res = await t.fetchImpl(`${cfg.baseURL.replace(/\/$/, '')}/embeddings`, {
48
+ const res = await fetchWithRetry(t.fetchImpl, `${cfg.baseURL.replace(/\/$/, '')}/embeddings`, {
48
49
  method: 'POST',
49
50
  headers: { 'content-type': 'application/json', authorization: `Bearer ${cfg.apiKey}` },
50
51
  body: JSON.stringify({ model: cfg.embeddingModel, input: texts }),
51
- ...(t.dispatcher ? { dispatcher: t.dispatcher } : {}),
52
- })
52
+ }, { dispatcher: t.dispatcher, ...(t.retry ?? {}) })
53
53
  if (!res.ok) {
54
54
  let body = ''
55
55
  try { body = (await res.text()).slice(0, 200) } catch { /* status alone */ }
@@ -62,7 +62,7 @@ export async function embedTexts(cfg, t, texts) {
62
62
  return [...data.data].sort((a, b) => a.index - b.index).map(d => d.embedding)
63
63
  }
64
64
 
65
- export async function embedKb(kbRoot, { fetchImpl } = {}) {
65
+ export async function embedKb(kbRoot, { fetchImpl, retry } = {}) {
66
66
  const cfg = loadLlmConfig(kbRoot)
67
67
  if (!cfg) throw new Error('No LLM configured. Create ~/.llm-wiki/config.json (OpenAI-compatible).')
68
68
  if (!cfg.embeddingModel) throw new Error('No embedding model configured. Add "embeddingModel" to your provider (or flat config) in ~/.llm-wiki/config.json, e.g. "text-embedding-3-small".')
@@ -84,7 +84,7 @@ export async function embedKb(kbRoot, { fetchImpl } = {}) {
84
84
  let dim = (prev && prev.model === cfg.embeddingModel) ? prev.dim : null
85
85
  let embedded = 0
86
86
  if (jobs.length > 0) {
87
- const t = fetchImpl ? { fetchImpl, dispatcher: undefined } : await makeTransport()
87
+ const t = fetchImpl ? { fetchImpl, dispatcher: undefined, retry } : { ...(await makeTransport()), retry }
88
88
  for (let i = 0; i < jobs.length; i += BATCH) {
89
89
  const batch = jobs.slice(i, i + BATCH)
90
90
  const vecs = await embedTexts(cfg, t, batch.map(j => j.text))
@@ -96,6 +96,10 @@ export async function embedKb(kbRoot, { fetchImpl } = {}) {
96
96
  nextPages[j.relPath] = { hash: j.hash, vec: n }
97
97
  embedded++
98
98
  })
99
+ // Incremental durability: persist after each batch so a later batch failing
100
+ // (rate-limit, network) doesn't discard this batch's work — the re-run reuses
101
+ // it by hash (zero repeat API cost). saveVectorStore is atomic (temp+rename).
102
+ saveVectorStore(kbRoot, { model: cfg.embeddingModel, dim: dim ?? 0, pages: nextPages })
99
103
  }
100
104
  }
101
105
  const pruned = prev ? Object.keys(prev.pages).filter(id => !(id in nextPages)).length : 0
package/src/export.mjs CHANGED
@@ -9,7 +9,7 @@ const xmlEscape = (s) => String(s)
9
9
  .replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;')
10
10
  .replace(/"/g, '&quot;').replace(/'/g, '&apos;')
11
11
 
12
- const cyEscape = (s) => String(s).replace(/\\/g, '\\\\').replace(/'/g, "\\'")
12
+ const cyEscape = (s) => String(s).replace(/\\/g, '\\\\').replace(/\n/g, '\\n').replace(/\r/g, '\\r').replace(/'/g, "\\'")
13
13
 
14
14
  // graph.json only lists wiki pages as nodes, but source-type edges point at raw/
15
15
  // paths. Exports need every edge endpoint to exist, so synthesize raw nodes.
package/src/indexer.mjs CHANGED
@@ -17,6 +17,11 @@ export function extractWikilinks(body) {
17
17
 
18
18
  const SECTION_TITLES = { source: 'Sources', entity: 'Entities', concept: 'Concepts', comparison: 'Comparisons', other: 'Other' }
19
19
 
20
+ // Frontmatter title/description are interpolated raw into line-based markdown
21
+ // (index.md, llms.txt). A value with newlines would inject extra lines, fake
22
+ // headings or spoof wikilinks (audit LOW-1). Collapse any newline run to a space.
23
+ const inline = (s) => String(s ?? '').replace(/[\r\n]+/g, ' ')
24
+
20
25
  export function buildIndex(kbRoot) {
21
26
  const p = kbPaths(kbRoot)
22
27
  const cfg = loadKbConfig(kbRoot)
@@ -42,7 +47,7 @@ export function buildIndex(kbRoot) {
42
47
  const byType = { source: [], entity: [], concept: [], comparison: [], other: [] }
43
48
  for (const pg of pages) (byType[pg.data.type] ?? byType.other).push(pg)
44
49
  const line = (pg) => {
45
- const base = `- [[${pg.relPath.replace(/\.md$/, '')}]] — ${pg.data.description ?? ''}`
50
+ const base = `- [[${pg.relPath.replace(/\.md$/, '')}]] — ${inline(pg.data.description)}`
46
51
  if (!isInvalidated(pg)) return base
47
52
  return pg.data.superseded_by
48
53
  ? `${base} ⚠ invalidated, superseded by [[${pg.data.superseded_by}]]`
@@ -50,15 +55,27 @@ export function buildIndex(kbRoot) {
50
55
  }
51
56
 
52
57
  let indexBody = '# Index\n\n> Auto-generated by `llm-wiki index`. Only the "Pending concepts" section may be edited by the LLM.\n'
58
+ // Prune stale topics/*.md so a type that emptied (or a drop back below the split
59
+ // threshold) can't leave an orphaned list that exportMarkdownPages would copy.
60
+ const pruneTopics = (keep = new Set()) => {
61
+ if (!fs.existsSync(p.topics)) return
62
+ for (const f of fs.readdirSync(p.topics)) {
63
+ if (f.endsWith('.md') && !keep.has(f)) fs.rmSync(path.join(p.topics, f))
64
+ }
65
+ }
53
66
  const topicsSplit = pages.length > cfg.indexSplitAt
54
67
  if (topicsSplit) {
55
68
  fs.mkdirSync(p.topics, { recursive: true })
69
+ const written = new Set()
56
70
  for (const [type, list] of Object.entries(byType)) {
57
71
  if (!list.length) continue
58
72
  fs.writeFileSync(path.join(p.topics, `${type}.md`), `# ${SECTION_TITLES[type]}\n\n${list.map(line).join('\n')}\n`)
73
+ written.add(`${type}.md`)
59
74
  indexBody += `\n## ${SECTION_TITLES[type]}\nSee [[topics/${type}]] (${list.length} pages)\n`
60
75
  }
76
+ pruneTopics(written)
61
77
  } else {
78
+ pruneTopics() // no longer split: index.md inlines everything; drop any prior topic files
62
79
  for (const [type, list] of Object.entries(byType)) {
63
80
  if (type === 'other' && !list.length) continue
64
81
  indexBody += `\n## ${SECTION_TITLES[type]}\n${list.map(line).join('\n')}\n`
@@ -101,7 +118,7 @@ export function buildIndex(kbRoot) {
101
118
 
102
119
  const kbName = path.basename(path.resolve(kbRoot))
103
120
  const llms = [`# ${kbName}`, '', `> llm_wiki knowledge base. Read wiki/index.md first, then open full pages.`, '', '## Pages',
104
- ...pages.filter(pg => !isInvalidated(pg)).map(pg => `- [${pg.data.title}](wiki/${pg.relPath}): ${pg.data.description ?? ''}`)].join('\n') + '\n'
121
+ ...pages.filter(pg => !isInvalidated(pg)).map(pg => `- [${inline(pg.data.title)}](wiki/${pg.relPath}): ${inline(pg.data.description)}`)].join('\n') + '\n'
105
122
  fs.writeFileSync(p.llmsTxt, llms)
106
123
 
107
124
  return { pageCount: pages.length, topicsSplit }
package/src/manifest.mjs CHANGED
@@ -5,12 +5,19 @@ import { readJsonFile } from './json.mjs'
5
5
  export function loadManifest(kbRoot) {
6
6
  const p = kbPaths(kbRoot)
7
7
  if (!fs.existsSync(p.manifest)) return { files: {} }
8
- return readJsonFile(p.manifest)
8
+ const m = readJsonFile(p.manifest)
9
+ // Shape guard: a hand-edited or half-written manifest lacking `files` (or with a
10
+ // non-object there) would make diffManifest's `Object.keys(manifest.files)` throw.
11
+ return (m && typeof m.files === 'object' && m.files !== null && !Array.isArray(m.files)) ? m : { files: {} }
9
12
  }
10
13
 
11
14
  export function saveManifest(kbRoot, manifest) {
12
15
  const p = kbPaths(kbRoot)
13
- fs.writeFileSync(p.manifest, JSON.stringify(manifest, null, 2) + '\n')
16
+ // Atomic write: manifest is non-derived state (hash->raw map). A crash mid-write
17
+ // must not leave a truncated/corrupt file. Write a temp sibling, then rename.
18
+ const tmp = `${p.manifest}.tmp`
19
+ fs.writeFileSync(tmp, JSON.stringify(manifest, null, 2) + '\n')
20
+ fs.renameSync(tmp, p.manifest)
14
21
  }
15
22
 
16
23
  export function diffManifest(manifest, entries) {
package/src/mcp.mjs CHANGED
@@ -17,7 +17,7 @@ const pkg = JSON.parse(fs.readFileSync(new URL('../package.json', import.meta.ur
17
17
  function textResult(text) { return { content: [{ type: 'text', text }] } }
18
18
  function errorResult(text) { return { content: [{ type: 'text', text }], isError: true } }
19
19
 
20
- export function createMcpServer(kbRoot, { fetchImpl } = {}) {
20
+ export function createMcpServer(kbRoot, { fetchImpl, retry } = {}) {
21
21
  const p = kbPaths(kbRoot)
22
22
  const server = new McpServer({ name: 'llm-wiki', version: pkg.version })
23
23
 
@@ -38,7 +38,7 @@ export function createMcpServer(kbRoot, { fetchImpl } = {}) {
38
38
  }, async ({ query, k = 6 }) => {
39
39
  // 'auto' mode: opt-in via the KB's vectorEnabled, fail-open on any vector
40
40
  // error — KBs without embeddings keep the exact pre-v2.5 BM25 behavior.
41
- const { hits, usedVector } = await locatePages(kbRoot, query, { k, ...(fetchImpl ? { fetchImpl } : {}) })
41
+ const { hits, usedVector } = await locatePages(kbRoot, query, { k, retry, ...(fetchImpl ? { fetchImpl } : {}) })
42
42
  if (hits.length === 0) {
43
43
  return textResult(usedVector
44
44
  ? 'No match from BM25 or vector retrieval. Call wiki_overview and pick pages from the catalog yourself.'
@@ -79,7 +79,7 @@ export function createMcpServer(kbRoot, { fetchImpl } = {}) {
79
79
  inputSchema: { question: z.string(), k: z.number().int().min(1).max(20).optional() },
80
80
  }, async ({ question, k = 6 }) => {
81
81
  try {
82
- const r = await askKb(kbRoot, question, { k, ...(fetchImpl ? { fetchImpl } : {}) })
82
+ const r = await askKb(kbRoot, question, { k, retry, ...(fetchImpl ? { fetchImpl } : {}) })
83
83
  const parts = [DATA_NOTICE, '', r.answer, '', `--- pages used: ${r.pages.map(h => h.relPath).join(', ')}`]
84
84
  if (r.fallback) parts.push('(BM25 had no lexical match; pages were selected from the KB listing by the model)')
85
85
  if (r.trimmed?.length) parts.push(`(token budget: dropped ${r.trimmed.length} lower-ranked page(s): ${r.trimmed.join(', ')})`)
package/src/retry.mjs ADDED
@@ -0,0 +1,29 @@
1
+ const RETRYABLE_STATUS = new Set([429, 500, 502, 503, 504])
2
+ const sleep = (ms) => new Promise(r => setTimeout(r, ms))
3
+
4
+ // One network call to an OpenAI-compatible endpoint, hardened for a CLI that must
5
+ // not hang forever or die on a transient rate-limit. Per attempt: an
6
+ // AbortSignal.timeout ceiling (its timer is unref'd, so it never keeps the process
7
+ // alive); retry on 429/5xx and on network/abort errors with exponential backoff.
8
+ // On a persistent 429/5xx the final response is returned so the caller raises its
9
+ // own "API error <status>"; on a persistent network/abort error the last error is
10
+ // thrown. retries=0 disables retry (immediate surface — used by tests).
11
+ export async function fetchWithRetry(fetchImpl, url, opts, { dispatcher, timeoutMs = 120000, retries = 3, backoffMs = 500 } = {}) {
12
+ for (let attempt = 0; ; attempt++) {
13
+ try {
14
+ const res = await fetchImpl(url, {
15
+ ...opts,
16
+ signal: AbortSignal.timeout(timeoutMs),
17
+ ...(dispatcher ? { dispatcher } : {}),
18
+ })
19
+ if (RETRYABLE_STATUS.has(res.status) && attempt < retries) {
20
+ await sleep(backoffMs * 2 ** attempt)
21
+ continue
22
+ }
23
+ return res
24
+ } catch (err) {
25
+ if (attempt >= retries) throw err
26
+ await sleep(backoffMs * 2 ** attempt)
27
+ }
28
+ }
29
+ }
package/src/templates.mjs CHANGED
@@ -118,7 +118,7 @@ export function readmeTemplate(name) {
118
118
 
119
119
  An llm_wiki knowledge base. Layers: \`raw/\` (immutable sources), \`wiki/\` (LLM-maintained pages).
120
120
 
121
- - Ask questions standalone: \`npx @sdsrs/llm-wiki ask "your question"\` (configure the LLM API in \`~/.llm-wiki/config.json\`).
121
+ - Ask questions standalone: \`npx @sdsrs/llm-wiki@0 ask "your question"\` (configure the LLM API in \`~/.llm-wiki/config.json\`).
122
122
  - Browse: open \`wiki/index.md\`, or open this folder as an Obsidian vault.
123
123
  - Maintain with a coding agent: see \`AGENTS.md\`.
124
124
  `
package/src/vector.mjs CHANGED
@@ -36,7 +36,13 @@ export function loadVectorStore(kbRoot) {
36
36
  export function saveVectorStore(kbRoot, store) {
37
37
  const rounded = { ...store, pages: Object.fromEntries(Object.entries(store.pages).map(([id, e]) =>
38
38
  [id, { hash: e.hash, vec: e.vec.map(x => Number(x.toFixed(5))) }])) }
39
- fs.writeFileSync(vectorStorePath(kbRoot), JSON.stringify(rounded) + '\n')
39
+ // Atomic write: the per-batch flush in embedKb re-saves repeatedly; a crash
40
+ // mid-write must not leave a truncated store (loadVectorStore fails open, but
41
+ // temp+rename avoids the corruption entirely).
42
+ const f = vectorStorePath(kbRoot)
43
+ const tmp = `${f}.tmp`
44
+ fs.writeFileSync(tmp, JSON.stringify(rounded) + '\n')
45
+ fs.renameSync(tmp, f)
40
46
  }
41
47
 
42
48
  // queryVec must already be normalized; stored vecs are normalized at save time,