llm-wiki-compiler 0.6.0 → 0.7.0

package/README.md

@@ -25,9 +25,20 @@ export ANTHROPIC_API_KEY=sk-...
 llmwiki ingest https://some-article.com
 llmwiki compile
 llmwiki query "what is X?"
+llmwiki view --open
 ```
 
-## Configuration
+
+<br>
+
+---
+
+<br>
+
+
+<details>
+<summary><span style="font-size: 1.4em;"><strong>Configuration — click to expand</strong></span></summary>
+
 
 llmwiki configures providers via environment variables. The default provider is Anthropic.
 
@@ -103,6 +114,35 @@ export OLLAMA_HOST=http://ollama_host:11434/v1
 export OLLAMA_EMBEDDINGS_HOST=http://ollama_host:11435/v1
 ```
 
+### GitHub Copilot
+
+Uses the GitHub Copilot API (`https://api.githubcopilot.com`), an
+OpenAI-compatible endpoint available to Copilot subscribers. Requires a GitHub
+OAuth token with the `copilot` scope — **classic PATs are not supported**.
+
+First, ensure your `gh` CLI token has the required scope:
+
+```bash
+gh auth refresh --scopes copilot
+```
+
+Then run:
+
+```bash
+export LLMWIKI_PROVIDER=copilot
+export GITHUB_TOKEN=$(gh auth token)  # OAuth token required; PATs will not work
+export LLMWIKI_MODEL=gpt-4o           # optional; gpt-4o is the default
+```
+
+Available models (names use dots, not dashes): `gpt-4o`, `gpt-4o-mini`,
+`claude-sonnet-4.5`, `claude-sonnet-4.6`, `claude-opus-4.5`, `gemini-2.5-pro`,
+and others — availability depends on your Copilot plan.
+
+**Embeddings:** The GitHub Copilot API does not expose an embeddings endpoint.
+Semantic search (used by `llmwiki query` with chunked retrieval) will fall back
+to full-index selection without embeddings. For embedding-dependent workflows,
+switch to the `openai` provider and provide `OPENAI_API_KEY`.
+
 ### Request timeouts
 
 The OpenAI SDK defaults to a 10-minute per-request timeout, which can cut off long compile-time completions on slower local models. Override per provider:
@@ -129,6 +169,16 @@ When many sources contribute to the same compiled concept, `compile` enforces a
 
 A truncation warning prints to stderr when the cap fires so you know which concept hit the budget.
 
+</details>
+
+
+<br>
+
+---
+
+<br>
+
+
 ## Why not just RAG?
 
 RAG retrieves chunks at query time. Every question re-discovers the same relationships from scratch. Nothing accumulates.
@@ -179,6 +229,18 @@ Related concepts: [[Propositional Logic]], [[Model Counting]]
 
 Pages include source attribution in frontmatter. Paragraphs are annotated with `^[filename.md]` markers pointing back to the source file that contributed the content; specific claims can use line ranges like `^[filename.md:42-58]` or `^[filename.md#L42-L58]`.
 
+
+<br>
+
+---
+
+<br>
+
+
+<details>
+<summary><span style="font-size: 1.4em;"><strong>CLI and wiki model — click to expand</strong></span></summary>
+
+
 ## Commands
 
 | Command | What it does |
@@ -197,6 +259,7 @@ Pages include source attribution in frontmatter. Paragraphs are annotated with `
 | `llmwiki query "question"` | Ask questions against your compiled wiki |
 | `llmwiki query "question" --save` | Answer and save the result as a wiki page |
 | `llmwiki export [--target <name>]` | Export the wiki to portable formats — `llms.txt`, `llms-full.txt`, JSON, JSON-LD, GraphML, Marp slides |
+| `llmwiki view [--open]` | Start a read-only local web viewer for browsing, searching, and inspecting the compiled wiki |
 | `llmwiki lint` | Check wiki quality (broken links, orphans, empty pages, low confidence, contradictions, etc.) |
 | `llmwiki watch` | Auto-recompile when `sources/` changes |
 | `llmwiki serve [--root <dir>]` | Start an MCP server exposing wiki tools to AI agents |
@@ -216,6 +279,17 @@ wiki/
 
 Obsidian-compatible. `[[wikilinks]]` resolve to concept titles.
 
+## Local web viewer
+
+Run `llmwiki view` from a project root to browse the compiled wiki in a local browser without Obsidian. The viewer is read-only: it renders `wiki/`, exposes sidebar navigation, search, page metadata, health counts, and provenance/citation chips, but does not mutate sources or generated pages.
+
+```bash
+llmwiki view          # prints Viewer ready at http://127.0.0.1:<port>
+llmwiki view --open   # also opens the URL in your default browser
+```
+
+The server is private by default. It binds to `127.0.0.1` unless you explicitly provide both `--host <host>` and `--allow-lan`; wildcard hosts are rejected. Viewer responses use a strict local-asset CSP and path-confinement checks so the UI can safely render local markdown content.
+
 ## Review queue
 
 By default, `compile` writes pages directly to `wiki/`. Add `--review` to write candidate JSON records to `.llmwiki/candidates/` instead, so you can inspect each generated page before it lands.
@@ -295,6 +369,16 @@ The schema supports four page kinds:
 
 Schema rules can set per-kind `minWikilinks` and optional `seedPages`. Compile can materialize seed pages such as overviews, lint enforces page-kind-specific cross-link minimums, and review candidates surface schema violations before approval.
 
+</details>
+
+
+<br>
+
+---
+
+<br>
+
+
 ## Demo
 
 Try it on any article or document:
@@ -304,10 +388,23 @@ mkdir my-wiki && cd my-wiki
 llmwiki ingest https://en.wikipedia.org/wiki/Andrej_Karpathy
 llmwiki compile
 llmwiki query "What terms did Andrej coin?"
+llmwiki view --open
 ```
 
 See `examples/basic/` in the repo for pre-generated output you can browse without an API key.
 
+
+<br>
+
+---
+
+<br>
+
+
+<details>
+<summary><span style="font-size: 1.4em;"><strong>MCP Server — click to expand</strong></span></summary>
+
+
 ## MCP Server
 
 llmwiki ships an MCP (Model Context Protocol) server so AI agents (Claude Desktop, Cursor, Claude Code, etc.) can drive the full pipeline directly: ingest sources, compile, query, search, lint, and read pages — without scraping CLI output.
@@ -364,6 +461,16 @@ Tools that need an LLM (`compile_wiki`, `query_wiki`, `search_pages`) check for
 | `llmwiki://sources` | List of ingested source files with metadata. |
 | `llmwiki://state` | Compilation state (per-source hashes, last compile times). |
 
+</details>
+
+
+<br>
+
+---
+
+<br>
+
+
 ## Limitations
 
 Early software. Best for small, high-signal corpora (a few dozen sources). Query routing is index-based.
@@ -389,6 +496,12 @@ Karpathy describes an abstract pattern for turning raw data into compiled knowle
 
 ## Roadmap
 
+Shipped in 0.7.0:
+
+- ✅ Read-only local web viewer — `llmwiki view` with sidebar navigation, markdown rendering, search, metadata, health counts, and provenance/citation chips
+- ✅ GitHub Copilot provider — `LLMWIKI_PROVIDER=copilot` with `GITHUB_TOKEN=$(gh auth token)` for Copilot chat/tool calls
+- ✅ Cached lint health summary — `llmwiki lint` writes `.llmwiki/last-lint.json` so viewer health can show the latest lint counts without re-running lint
+
 Shipped in 0.6.0:
 
 - ✅ Export bundle (`llms.txt`, JSON, JSON-LD, GraphML, Marp slides)
@@ -421,17 +534,24 @@ Shipped in 0.2.0:
 - ✅ Deeper Obsidian integration (tags, aliases, Map of Content)
 - ✅ MCP server for agent integration
 
-Future ideas (open to discussion):
+Next up:
+
+- **Graph/context layer** — page-neighborhood tools, graph paths, gap detection, and token-budgeted context packs for agents.
+- **Evaluation harness** — benchmark answer quality, citation accuracy, update drift, retrieval recall, and scale curves against serious retrieval baselines.
+- **Task and decision ledger** — turn session ingest into durable agent memory: goals, decisions, open questions, outcomes, and next-agent handoffs.
+- **Rollback, audit, and source lifecycle** — undo/reverse ingest, compile diff reports, stale-claim checks, freshness reports, and a durable operation log.
+- **Domain templates** — schema/prompt packs for research, codebase docs, team handbooks, decision logs, and standards/regulations.
+
+Later / open to discussion:
 
 - Recurring source refresh jobs — re-ingest URLs on a schedule, diff against the prior snapshot, re-compile only what changed
-- Graph export and a lightweight read-only graph browser for the concept network
-- A local read-only web UI for browsing the compiled wiki without Obsidian
-- MCP prompt resources — curated agent prompts (review the wiki, propose new sources, draft a comparison page) shipped as MCP resources
-- Maintenance log + log rotation so long-running watch sessions don't grow unbounded
+- MCP prompt resources — curated agent prompts such as "review the wiki", "propose new sources", and "draft a comparison page"
+- Codex OAuth provider — ChatGPT subscription auth as a dedicated provider, with clear token refresh and embedding-limit behavior
+- Team-chat connectors for Slack/Discord/Teams-style institutional memory
 
-If you like ambitious problems: **graph export with a browser**, **recurring source refresh**, and **MCP prompt resources** are the meatiest of the futures. Open an issue to claim one or kick off a design discussion.
+If you like ambitious problems: **local web UI**, **graph/context packs**, and **eval harness** are the meatiest next contributions. Open an issue to claim one or kick off a design discussion.
 
-Explicitly not planned (good ideas, just not for this repo): full static-site generator, desktop or mobile apps, fine-tuning, a formal ontology engine, heavy graph reasoning.
+Explicitly not planned (good ideas, just not for this repo): full static-site generator, desktop or mobile apps, fine-tuning, a formal ontology engine, heavy graph database infrastructure.
 
 ## Requirements