llm-wiki-compiler 0.5.0 → 0.6.0

package/README.md

@@ -112,6 +112,23 @@ The OpenAI SDK defaults to a 10-minute per-request timeout, which can cut off lo
 
 Defaults: 10 minutes for `openai`, 30 minutes for `ollama` (local models commonly need more).
 
+### Output language
+
+Generated wiki content defaults to whatever language the model produces from the source material — typically English. Override with either:
+
+- `LLMWIKI_OUTPUT_LANG` — e.g. `zh-CN`, `Chinese`, `ja`, `Japanese`. Applies to every prompt the compile and query pipelines make.
+- `--lang <code>` on `llmwiki compile` and `llmwiki query` — same effect, scoped to one invocation. Wins over the env var.
+
+Unset preserves prior behaviour byte-for-byte.
+
+### Per-concept prompt budget
+
+When many sources contribute to the same compiled concept, `compile` enforces a per-concept character cap on the combined source content sent to the LLM so popular shared concepts don't blow past the model's context window. Each contributing source gets a fair share when truncation kicks in.
+
+- `LLMWIKI_PROMPT_BUDGET_CHARS` — character ceiling for the combined per-concept prompt. Defaults to `200000` (~50k tokens), which fits modern context windows with headroom. Raise it for larger-context models, lower it for local small-context models.
+
+A truncation warning prints to stderr when the cap fires so you know which concept hit the budget.
+
 ## Why not just RAG?
 
 RAG retrieves chunks at query time. Every question re-discovers the same relationships from scratch. Nothing accumulates.
@@ -167,8 +184,10 @@ Pages include source attribution in frontmatter. Paragraphs are annotated with `
 | Command | What it does |
 |---------|-------------|
 | `llmwiki ingest <url\|file>` | Fetch a URL or copy a local file into `sources/` |
+| `llmwiki ingest-session <path>` | Import a Claude/Codex/Cursor session export (single file or whole directory) into `sources/` |
 | `llmwiki compile` | Incremental compile: extract concepts, generate wiki pages |
 | `llmwiki compile --review` | Write candidate pages to `.llmwiki/candidates/` instead of `wiki/` so you can review before they land |
+| `llmwiki compile --lang <code>` | Generate wiki content in the given language (e.g. `Chinese`, `ja`, `zh-CN`); also works on `query` |
 | `llmwiki review list` | List pending candidate pages |
 | `llmwiki review show <id>` | Print a candidate's title, summary, and body |
 | `llmwiki review approve <id>` | Promote a candidate into `wiki/` and refresh index/MOC/embeddings |
@@ -177,6 +196,7 @@ Pages include source attribution in frontmatter. Paragraphs are annotated with `
 | `llmwiki schema show` | Print the resolved schema for the current project |
 | `llmwiki query "question"` | Ask questions against your compiled wiki |
 | `llmwiki query "question" --save` | Answer and save the result as a wiki page |
+| `llmwiki export [--target <name>]` | Export the wiki to portable formats — `llms.txt`, `llms-full.txt`, JSON, JSON-LD, GraphML, Marp slides |
 | `llmwiki lint` | Check wiki quality (broken links, orphans, empty pages, low confidence, contradictions, etc.) |
 | `llmwiki watch` | Auto-recompile when `sources/` changes |
 | `llmwiki serve [--root <dir>]` | Start an MCP server exposing wiki tools to AI agents |
@@ -229,17 +249,16 @@ confidence: 0.82           # 0–1, LLM-reported confidence in the synthesized p
 provenanceState: merged    # extracted | merged | inferred | ambiguous
 contradictedBy:
   - slug: probabilistic-reasoning
-inferredParagraphs: 1      # paragraphs the LLM marked as inferred (vs cited)
 ---
 ```
 
-When multiple sources merge into one slug, metadata is reconciled: `min` confidence, `provenanceState = 'merged'`, union of `contradictedBy` (deduped by slug), `max` `inferredParagraphs`.
+When multiple sources merge into one slug, metadata is reconciled: `min` confidence, `provenanceState = 'merged'`, union of `contradictedBy` (deduped by slug).
 
 `llmwiki lint` adds three rules that surface this metadata:
 
 - `low-confidence` — flags pages with `confidence` below a threshold
 - `contradicted-page` — flags pages with non-empty `contradictedBy`
-- `excess-inferred-paragraphs` — flags pages with too many inferred paragraphs without citations
+- `excess-inferred-paragraphs` — flags pages whose body has too many uncited prose paragraphs (counted directly from the rendered text — the body is the single source of truth, no frontmatter field involved)
 
 ## Claim-level provenance
 
@@ -364,12 +383,19 @@ Karpathy describes an abstract pattern for turning raw data into compiled knowle
 | Auto-recompile | `llmwiki watch` | Implemented |
 | Linting / health-check pass | `llmwiki lint` | Implemented |
 | Agent integration | `llmwiki serve` (MCP server) | Implemented |
-| Image support | — | Not yet implemented |
-| Marp slides | — | Not yet implemented |
+| Image support | `llmwiki ingest <image>` | Implemented |
+| Marp slides | `llmwiki export --target marp` | Implemented |
 | Fine-tuning | — | Not yet implemented |
 
 ## Roadmap
 
+Shipped in 0.6.0:
+
+- ✅ Export bundle (`llms.txt`, JSON, JSON-LD, GraphML, Marp slides)
+- ✅ Session-history adapters — `llmwiki ingest-session` for Claude, Codex, and Cursor exports
+- ✅ Configurable output language — `--lang <code>` and `LLMWIKI_OUTPUT_LANG`
+- ✅ Defensive per-concept prompt budget so popular shared concepts don't crash compile
+
 Shipped in 0.5.0:
 
 - ✅ Multimodal ingest (images, PDFs, transcripts)
@@ -395,11 +421,6 @@ Shipped in 0.2.0:
 - ✅ Deeper Obsidian integration (tags, aliases, Map of Content)
 - ✅ MCP server for agent integration
 
-Next up:
-
-- Export bundle (`llms.txt`, JSON, JSON-LD, GraphML, Marp)
-- Session-history adapters (Claude, Codex, Cursor exports)
-
 Future ideas (open to discussion):
 
 - Recurring source refresh jobs — re-ingest URLs on a schedule, diff against the prior snapshot, re-compile only what changed