@wanshi-kg/wanshi 0.2.0 → 0.2.1

package/README.md

@@ -6,6 +6,9 @@
   <img alt="wanshi" src="docs/assets/readme-banner-light.png">
 </picture>
 
+[![npm version](https://img.shields.io/npm/v/@wanshi-kg/wanshi)](https://www.npmjs.com/package/@wanshi-kg/wanshi)
+[![CI](https://github.com/wanshi-kg/wanshi/actions/workflows/ci.yml/badge.svg)](https://github.com/wanshi-kg/wanshi/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
 
 > A local-first CLI that reads ten thousand things — code, docs, PDFs, audio, transcripts — and builds one knowledge graph that remembers where every fact came from.
 
@@ -15,7 +18,11 @@ It's a working CLI and a research platform in equal measure — the long game is
 
 ---
 
-> **Command shorthand:** examples below write `wanshi` for the run command. Installed from npm (`@wanshi-kg/wanshi`) that's literally `wanshi`; from a source checkout it's `npm start --` (i.e. `npx ts-node ./src/cli/index.ts`) in dev, or `node ./dist/cli/index.js` after `npm run build`.
+> **Command shorthand:** examples below write `wanshi` for the run command — the global CLI once you've run `npm i -g @wanshi-kg/wanshi`. From a source checkout it's `npm start --` (i.e. `npx ts-node ./src/cli/index.ts`) in dev, or `node ./dist/cli/index.js` after `npm run build`.
+
+## Contents
+
+[What's distinctive](#whats-distinctive) · [Supported inputs](#supported-inputs) · [Install](#install) · [Quick start](#quick-start) · [CLI reference](#cli-reference) · [Output formats](#output-formats) · [Local model guidance](#local-model-guidance) · [Quality metrics](#quality-metrics) · [Architecture](#architecture) · [Development](#development)
 
 ## What's distinctive
 
@@ -25,6 +32,7 @@ Most text→KG tools stop at "extract triples." `wanshi` is built around the par
 - **A grounding gate (opt-in).** Each extracted fact can be scored against its source chunk and flagged or dropped before it reaches the output — keyword overlap as a cheap pre-filter, with an optional local NLI checker (MiniCheck) for the uncertain cases. Enabled (`--grounding flag|drop`), it won't record what it can't verify against the source — but it's `disabled` by default.
 - **Closed-vocabulary extraction.** An optional corpus pre-pass builds a glossary of canonical entity/relation types, which then *constrains* extraction — so a large corpus doesn't fragment into hundreds of one-off types.
 - **Transcript-aware ingestion.** Speaker-labeled transcripts and chat exports are split into speaker-pure chunks, so a speaker becomes per-fact provenance rather than a polluting entity.
+- **Beyond plain text.** A structured source can map straight to graph — a SQLite `.db` becomes tables→types, rows→entities, foreign-keys→edges with no LLM — and a document's own links and citations become deterministic edges, optionally fetching the cited work to ground the claim.
 - **Memory-store interop.** `mcp-jsonl` output is byte-compatible with the official [MCP memory server](https://github.com/modelcontextprotocol/servers/tree/main/src/memory) — point it at the file and query your graph from Claude Code/Desktop. No store to build.
 - **Training-data exports.** Emit KBLaM `(entity, property, value)` triples or quality-filtered LoRA/SFT chat examples straight from a graph.
 - **Resumable runs.** Per-chunk checkpoints survive interrupts and exhausted API credits; re-run the same command to continue.
@@ -35,9 +43,15 @@ Most text→KG tools stop at "extract triples." `wanshi` is built around the par
 | ------ | ---------- | -------- |
 | Text / source code | `.txt`, `.ts`, `.js`, `.py`, `.go`, `.rs`, … | Direct / code-aware extraction |
 | Markdown | `.md` | Markdown-aware parsing |
+| LaTeX | `.tex` | De-TeX'd to readable prose; `\cite{}` keys feed the citation pipeline |
+| EPUB | `.epub` | Unzipped and parsed per chapter (adm-zip + cheerio + html-to-text) |
+| Jupyter | `.ipynb` | Cell-aware (markdown narrative + fenced code); cell outputs opt-in |
 | Transcripts | speaker-labeled `*.parakeet.txt`/`*.whisper.txt`, transcript/turn JSON, Claude/ChatGPT exports | Speaker-pure chunks with per-fact `speaker`/`occurredAt` |
+| Email | `.eml`, `.mbox` | Per-message turns (sender → `speaker`, `Date` → `validAt`); thread-aware; quoted replies stripped |
+| Chat exports | WhatsApp `.txt`, Telegram/Discord/Slack `.json` | Per-message speaker-pure turns via a per-platform parser |
+| Subtitles | `.srt`, `.vtt` | Caption text (timecodes/styling stripped); VTT `<v>` voice tags → speakers |
 | JSON | `.json`, `.jsonl`, `.geojson` | Structure-aware chunking (splits on JSON structure, never mid-object) |
-| PDF | `.pdf` | Page text (`pdf2json`), or a richer engine via `--pdf-engine docling\|marker\|mistral` |
+| PDF | `.pdf` | Page text (`pdf2json`), or a richer engine via `--pdf-engine tesseract\|docling\|marker\|chandra\|mistral` |
 | Office | `.docx`, `.xlsx`, `.pptx` | Via officeparser |
 | HTML / RTF | `.html`, `.htm`, `.rtf` | cheerio / RTF parsing |
 | Images | `.jpg`, `.png`, `.gif`, `.webp`, `.tiff`, `.heic`, `.avif` | Vision model required |
@@ -48,14 +62,20 @@ Most text→KG tools stop at "extract triples." `wanshi` is built around the par
 Requires **Node.js 18+** and **[Ollama](https://ollama.ai)** running locally (needed for the default local generation + embeddings path; optional only if you point *both* at an OpenAI-compatible provider).
 
 ```bash
-git clone https://github.com/wanshi-kg/wanshi
-cd wanshi
-npm install
+# Install the published CLI (gives you the `wanshi` command)
+npm install -g @wanshi-kg/wanshi
 
 # Default local models
 ollama pull llama3.2                 # generation
-ollama pull nomic-embed-text   # embeddings
+ollama pull nomic-embed-text         # embeddings
+```
+
+Or run from a source checkout (for development / contributing):
 
+```bash
+git clone https://github.com/wanshi-kg/wanshi
+cd wanshi
+npm install
 npm run build   # optional; ts-node works directly
 ```
 
@@ -145,98 +165,43 @@ wanshi --export-only -i ./knowledge-graph.json --export-format kblam -o ./kb.jso
 
 ## CLI reference
 
-### Core
+The most-used flags are below. Run **`wanshi --help`** for the full list and **`wanshi schema`** for the complete, authoritative config (generated from the Zod schema, so it never drifts from the code); the prose reference lives in [`website/docs/reference/cli.md`](website/docs/reference/cli.md).
 
 | Option | Default | Description |
 | ------ | ------- | ----------- |
 | `-i, --input <path>` | `.` | Input directory |
-| `-f, --filter <glob>` | `**/*` | Include pattern |
-| `-e, --exclude <glob...>` | — | Exclude patterns |
 | `-o, --output <path>` | `knowledge-graph.json` | Output file |
-| `-d, --description <text>` | — | Content description for LLM context |
-| `--config <file>` | — | YAML/JSON config file |
-
-### LLM
-
-| Option | Default | Description |
-| ------ | ------- | ----------- |
+| `-f, --filter` / `-e, --exclude <glob…>` | `**/*` | Include / exclude patterns |
+| `--config <file>` | — | YAML/JSON config (recommended; nested shape — `wanshi schema`) |
 | `--provider <name>` | `ollama` | `ollama` or `openai` (any OpenAI-compatible endpoint) |
-| `-m, --model <name>` | `llama3.2` | Ollama tag or provider model id |
-| `-h, --host <url>` | `http://localhost:11434` | Ollama host, or OpenAI-compatible base URL |
-| `--api-key <key>` | — | Falls back to `$OPENAI_API_KEY` / `$WANSHI_API_KEY` |
-| `--temperature <n>` | `0.1` | Sampling temperature |
-| `--repeat-penalty <n>` | `1.1` | Ollama only (>1.0 discourages repetition) |
-| `--context-length <n>` | `8192` | Context window (Ollama only) |
-| `--max-tokens <n>` | provider default | Raise (or lower `--chunk-size`) if graph JSON truncates mid-output |
-| `--seed <n>` | — | Reproducibility seed (Ollama only) |
-| `-s, --system <prompt\|path>` | — | Custom system prompt or template path |
-
-### Embeddings (independent from generation)
-
-| Option | Default | Description |
-| ------ | ------- | ----------- |
-| `--embeddings-provider <name>` | `ollama` | `ollama` or `openai` |
-| `--embeddings-model <name>` | `nomic-embed-text` | Embeddings model |
-| `--embeddings-host <url>` | `http://localhost:11434` | Host / base URL |
-| `--embeddings-max-input-chars <n>` | `1024` | Truncate embedding inputs (safe for 512-token models; raise for cloud) |
-
-### Processing & retrieval
-
-| Option | Default | Description |
-| ------ | ------- | ----------- |
-| `--chunking <mode>` | `enabled` | `enabled\|disabled\|auto` |
+| `-m, --model <name>` | `llama3.2` | Generation model |
+| `-h, --host <url>` | `localhost:11434` | Ollama host / OpenAI base URL |
+| `--embeddings-model <name>` | `nomic-embed-text` | Embeddings model (chosen independently from generation) |
 | `-c, --chunk-size <n>` | `2000` | Max chunk size (chars) |
-| `--overlap-size <n>` | `100` | Chunk overlap |
-| `--retrieval <mode>` | `enabled` | `enabled\|disabled\|auto` |
-| `--retrieval-limit <n>` | `3` | Retrieved context entities per chunk |
-| `--retrieval-scope <mode>` | `chunk` | `chunk` (per-chunk) or `file` (once, reused) |
-| `--json-strategy <mode>` | `structural` | `structural` (split on JSON structure) or `raw` |
-
-### Media & classification
-
-| Option | Default | Description |
-| ------ | ------- | ----------- |
-| `--asr <mode>` | `enabled` | `enabled\|disabled\|auto` |
-| `--whisper-model <name>` | `medium` | `tiny\|base\|small\|medium\|large` |
-| `--language <lang>` | `auto` | Language code or `auto` |
-| `--translate` | `false` | Translate audio to English |
-| `--images <mode>` | `auto` | `enabled\|disabled\|auto` (vision model required) |
-| `--pdf-engine <engine>` | `pdf2json` | `pdf2json\|docling\|marker\|mistral` — PDF reading engine (non-default engines degrade to `pdf2json` on failure) |
-| `--asr-engine <engine>` | `whisper` | `whisper\|dual` — `dual` = vendored Python VAD + Parakeet/Whisper dual-STT + diarization (Apple-Silicon) |
-| `--classifier <mode>` | `disabled` | `disabled\|heuristic\|llm\|cascade` — drives domain prompt hints and scopes `entityType` to a per-domain enum *(experimental)* |
-| `--trace` | `false` | Emit a structured decision run-trace to `<output>.trace.jsonl` *(debug/observability)* |
-
-### Merging, grounding, corpus glossary
-
-| Option | Default | Description |
-| ------ | ------- | ----------- |
-| `--entity-similarity-threshold <n>` | `0.9` | Jaro-Winkler entity dedup (0–1) |
-| `--observation-similarity-threshold <n>` | `0.9` | Embedding similarity (0–1) |
-| `--enable-similarity-merging` | `true` | Enable entity deduplication |
-| `--grounding <mode>` | `disabled` | `disabled` · `flag` (annotate `grounded`/`groundingScore`) · `drop` (remove below threshold) |
-| `--grounding-min-score <n>` | `0.5` | Min grounding score; also gates which facts the `lora` export keeps |
-| `--corpus-profiling <mode>` | `disabled` | Pre-pass that builds an authoritative corpus glossary (closed vocab under v5) *(experimental)* |
-| `--prompt-version <version>` | `v5` | `v5` (closed-vocab + topology hygiene) or `v4.5` (legacy) |
-
-### Export, resume, logging
-
-| Option | Default | Description |
-| ------ | ------- | ----------- |
-| `--export-format <format>` | `json` | `json\|jsonl\|mcp-jsonl\|dot\|kblam\|lora\|graphiti` |
-| `--export-only` | `false` | Convert an existing graph (`--input`) to `--export-format` — no extraction |
+| `--export-format <fmt>` | `json` | `json·jsonl·mcp-jsonl·dot·kblam·lora·graphiti` |
+| `--export-only` | `false` | Convert an existing graph — no extraction |
 | `--resume` | `false` | Checkpoint chunks; skip done ones on re-run |
-| `--checkpoint <path>` | `<output>.checkpoint.jsonl` | Checkpoint sidecar |
-| `-L, --log-level <level>` | `info` | `debug\|info\|warning\|error` |
-| `-l, --log-file <path>` | — | Write logs to file |
-| `-w, --watch` | `false` | Watch mode |
+| `--grounding <mode>` | `disabled` | `flag` / `drop` ungrounded facts (opt-in) |
+| `--pdf-engine <engine>` | `pdf2json` | `pdf2json·tesseract·docling·marker·chandra·mistral` |
+| `-w, --watch` | `false` | Update the graph as files change |
 
-> Document-outline injection (`readers.outline`) and DOT styling (`export.dot`) are config-only (no CLI flags) — see the config schema.
+**Opt-in subsystems** — all default **off** (an otherwise byte-identical, offline run): reference + citation resolution (`--reference-links`, `--reference-citations`, `--reference-web`, `--reference-citation-fetch`, plus GROBID / Unpaywall / title-resolver), image enrichment (`--exif`, `--c2pa`, `--object-detection`), structured-source adapters (`--sqlite`), AST code seeding (`--ast`), the dual-STT ASR engine (`--asr-engine dual`), and cost metering (`--cost` / `--max-cost`). Run `wanshi --help` for each.
 
 ## Output formats
 
-### JSON (`json`)
+Pick with `--export-format`:
 
-Observations are **objects**, not bare strings — each carries provenance and the bi-temporal axis. The LLM emits plain text; `wanshi` stamps the metadata deterministically from what it knows about the chunk. Unknown fields are omitted; legacy string-observation graphs still load.
+| Format | What it's for |
+| ------ | ------------- |
+| `json` (default) | Full graph; observations are **objects** carrying provenance + the bi-temporal axis |
+| `jsonl` | Streamable JSON Lines |
+| `mcp-jsonl` | Byte-compatible with the [MCP memory server](https://github.com/modelcontextprotocol/servers/tree/main/src/memory) — point it at the file, query from Claude. No store to build |
+| `dot` | Styled GraphViz (colors, legend, clustering — config-only `export.dot:`); render `dot -Tsvg graph.dot -o graph.svg` |
+| `kblam` | Microsoft [KBLaM](https://github.com/microsoft/KBLaM) `(entity, property, value)` triples for knowledge-token training |
+| `lora` | Chat SFT examples, **quality-filtered** (drops facts below `--grounding-min-score`) |
+| `graphiti` | `add_triplet`-shaped `{ nodes, edges }` for a [Graphiti](https://github.com/getzep/graphiti) temporal graph |
+
+The default `json` keeps observations as provenance-stamped **objects** — the LLM emits plain text; `wanshi` stamps `source`/`speaker` and the bi-temporal axis deterministically from what it knows about each chunk:
 
 ```json
 {
@@ -245,11 +210,8 @@ Observations are **objects**, not bare strings — each carries provenance and t
       "name": "knowledge_graph_builder",
       "entityType": "class",
       "observations": [
-        {
-          "text": "Extracts entities and relations from file content using an LLM",
-          "source": "src/core/knowledge/KnowledgeGraphBuilder.ts",
-          "createdAt": "2026-06-05T15:57:59.856Z"
-        }
+        { "text": "Extracts entities and relations from file content using an LLM",
+          "source": "src/core/knowledge/KnowledgeGraphBuilder.ts", "createdAt": "2026-06-05T15:57:59.856Z" }
       ],
       "files": ["src/core/knowledge/KnowledgeGraphBuilder.ts"]
     },
@@ -257,13 +219,9 @@ Observations are **objects**, not bare strings — each carries provenance and t
       "name": "SPEAKER_01",
       "entityType": "person",
       "observations": [
-        {
-          "text": "Explains that a Naïve Bayes classifier assumes word independence",
-          "speaker": "SPEAKER_01",
-          "source": "Olga Lesson P.parakeet.txt",
-          "validAt": "2026-05-28T00:00:00Z",
-          "createdAt": "2026-06-05T15:57:59.856Z"
-        }
+        { "text": "Explains that a Naïve Bayes classifier assumes word independence",
+          "speaker": "SPEAKER_01", "source": "Olga Lesson P.parakeet.txt",
+          "validAt": "2026-05-28T00:00:00Z", "createdAt": "2026-06-05T15:57:59.856Z" }
       ],
       "files": ["Olga Lesson P.parakeet.txt"]
     }
@@ -274,37 +232,7 @@ Observations are **objects**, not bare strings — each carries provenance and t
 }
 ```
 
-### MCP-compatible JSONL (`mcp-jsonl`)
-
-```jsonl
-{"type":"entity","name":"knowledge_graph_builder","entityType":"class","observations":["Extracts entities and relations from file content using an LLM"]}
-{"type":"relation","from":"knowledge_graph_builder","to":"ollama_service","relationType":"uses,depends_on"}
-```
-
-### GraphViz DOT (`dot`)
-
-Styled, colored graph (one node per entity, colored edges per relation type, legend, config summary). Render with `dot -Tsvg graph.dot -o graph.svg` (or `neato`/`fdp`/`sfdp`/`circo`/`twopi`). Styling is config-only under `export.dot:` — layout, `rankdir`, `colorScheme` (`default\|scientific\|code\|minimal`), clustering by type or file, etc.
-
-### KBLaM triples (`kblam`)
-
-JSONL in the shape Microsoft [KBLaM](https://github.com/microsoft/KBLaM)'s `dataset_generation` ingests — **one `(entity, property, value)` per line**, each with the derived `Q`/`A`/`key_string` it encodes into a knowledge token. Property names are distinct per entity (relations contribute their predicate as the property), and keys are unique per `(name, property)` so rectangular-attention lookup is unambiguous.
-
-```jsonl
-{"name":"Recursion","property":"definition","value":"a function that calls itself","Q":"What is the definition of Recursion?","A":"The definition of Recursion is a function that calls itself.","key_string":"the definition of Recursion"}
-{"name":"Recursion","property":"terminates_at","value":"BaseCase","Q":"What is the terminates_at of Recursion?","A":"The terminates_at of Recursion is BaseCase.","key_string":"the terminates_at of Recursion"}
-```
-
-### LoRA / SFT (`lora`)
-
-Chat-format instruction examples derived from the same triples, **quality-filtered**: observations whose grounding score is below `--grounding-min-score` are dropped, so only grounded facts become training data.
-
-```jsonl
-{"messages":[{"role":"user","content":"What is the definition of Recursion?"},{"role":"assistant","content":"The definition of Recursion is a function that calls itself."}]}
-```
-
-### Graphiti (`graphiti`)
-
-`add_triplet`-shaped `{ nodes, edges }` for ingestion into a [Graphiti](https://github.com/getzep/graphiti) temporal graph — entities → nodes (summary from observations), relations → `UPPER_SNAKE` edges with stable uuids. Per-fact valid-time rides along in the `json`/`kblam` exports.
+Per-format shapes + examples (KBLaM / LoRA / Graphiti / DOT): [`website/docs/guides/output-formats.md`](website/docs/guides/output-formats.md).
 
 ## Local model guidance
 
@@ -320,15 +248,16 @@ Quality/speed trade-off for local selection. For measured numbers see the benchm
 
 Default embeddings: `nomic-embed-text`.
 
-The table above is qualitative guidance. For measured, comparative numbers (wanshi vs KGGen on gold-labeled datasets) see **[Benchmarks](#benchmarks)** below — note those run on **cloud** models; local-model benchmarks are planned.
+The table above is qualitative guidance. For measured, comparative numbers (wanshi vs KGGen on gold-labeled datasets) see **[Benchmarks](#benchmarks)** below — both a cloud arm and a **local (M4 + L4) arm**.
 
 ## Benchmarks
 
-> **Scope & honesty (read first).** Every number here is **cloud inference via OpenRouter** —
-> **local-model (offline-first) benchmarks are planned and not yet run** (see [What's not yet
-> measured](#whats-not-yet-measured)). Comparative baselines are **re-scored under one identical
-> harness, not the published figures**. The document-level result rests on **one dataset** so far.
-> **MINE** is a recall-only, LLM-judge-mediated axis, reported as *context*, not a load-bearing claim.
+> **Scope & honesty (read first).** Cloud numbers are **OpenRouter inference**; the
+> **local (offline-first) arm is now measured too** — see [Local arm](#local-arm-offline-first).
+> Comparative baselines are **re-scored under one identical harness**
+> ([pre-registered methodology](docs/benchmark/SCORING.md)), not the published figures. The
+> document-level result rests on **one dataset** so far. **MINE** is a recall-only, LLM-judge-mediated
+> axis, reported as *context*, not a load-bearing claim.
 
 wanshi vs **KGGen** (its real Python package), **same model for both tools**, on gold-labeled datasets.
 The fair cross-tool metric is **entity-capture F1** (did the tool recover the gold entities) — both
@@ -399,13 +328,29 @@ npx ts-node scripts/gold-compare.ts --dataset redocred --limit 100 \
 # add --relation-vocab @data/redocred/compare/relation-vocab.txt for the schema-aware (H4) cell
 ```
 
+### Local arm (offline-first)
+
+The deployment-target floor is now measured: wanshi vs KGGen on the **same local Ollama model**
+(`gemma3:4b`, `qwen3:8b`), gold corpora, on a **16 GB M4 laptop** *and* a rented **L4 GPU**. The
+precision-collapse holds at the 4B *local* tier — biored KGGen node-precision **0.26**, matching the
+cloud's ~0.24 — so the precision-stability claim is **model-invariant across 4B→70B and three hardware
+tiers**, not just cloud.
+
+| `gemma3:4b` · biored | wanshi node-F1 | KGGen node-F1 | conformance | throughput |
+| -------------------- | -------------- | ------------- | ----------- | ---------- |
+| M4 (16 GB laptop) | 0.49 | 0.39 | 1.000 | ~25 tok/s |
+| L4 (rented GPU) | 0.49 | 0.39 | 1.000 | ~63 tok/s |
+
+**Quality is hardware-independent** — M4 and L4 node-F1 differ only by sampling noise, and JSON-conformance
+is **1.000** on both dense models — at **~40% of the rental GPU's throughput**. wanshi wins node-F1 in
+**8/8 M4 cells and 11/12 L4 cells** (sole loss: redocred/qwen3:8b). *(qwen3:8b runs on 16 GB only
+serialized; a full 8B comparison sweep isn't a realistic laptop workload.)*
+
 ### What's not yet measured
 
-- **Local-model (offline-first) benchmarks** — the deployment-target floor (`gemma3:4b`-class) is *owed*;
-  every number above is cloud inference. This is the next benchmark priority. *(An earlier indicative
-  n=20 single-domain run hinted small `gemma3:4b` ≈ larger Gemmas on entity extraction — to be confirmed
-  in the local arm.)*
 - **A second document-level dataset** (SciERC / BioRED) to close the single-dataset caveat on claim (a).
+- **A clean wanshi-alone cell + the redocred/qwen3:8b document cell** (the one local loss) on that second
+  corpus — to settle whether the doc-level arc weakens at 8B or it's noise.
 
 ## Quality metrics
 
@@ -415,18 +360,25 @@ Importable evaluators in `src/quality/` (also wired into `npm run benchmark`): *
 
 ```text
 src/
-├── cli/          # Commander.js CLI (process/watch/export; --export-only)
+├── cli/           # Commander.js CLI (process/watch/export; --export-only)
+├── config/        # Single nested Zod schema — defaults, validation, `wanshi schema`
 ├── core/
-│   ├── di/        # Async DI container + service registrations
-│   ├── processor/ # File readers (transcript, JSON, PDF, Office, audio, …) + chunking + classifiers
-│   ├── checkpoint/# Per-chunk resume sidecar
-│   ├── llm/       # Ollama / OpenAI-compatible providers, embeddings, Handlebars prompts
-│   ├── knowledge/ # KG building (LLM+Zod, provenance + grounding gate), 3-level merge, vector search
-│   └── export/    # Strategy pattern: json, jsonl, mcp-jsonl, dot, kblam, lora, graphiti
-├── quality/      # Importable metrics (structural, semantic, factual, consistency, composite)
-├── evaluation/   # Benchmark harness (CrossRE / REBEL / RE-DocRED)
-├── types/        # Interfaces and data models
-└── shared/       # Logger, graceful shutdown, utilities (Jaro-Winkler, cosine, config)
+│   ├── di/         # Async DI container + service registrations
+│   ├── processor/  # File readers (transcript, email, chat, PDF/OCR, audio, …) + chunking + classifiers + AST seed
+│   ├── corpus/     # Corpus pre-pass: term frequency + LLM glossary (closed vocab)
+│   ├── checkpoint/ # Per-chunk resume sidecar
+│   ├── llm/        # Ollama / OpenAI-compatible providers, embeddings, Handlebars prompts
+│   ├── knowledge/  # KG build (LLM+Zod, provenance + grounding gate), 3-level merge, canon, references, images, vector search
+│   ├── adapters/   # Structured-emit adapters (SQLite → graph fragments, no LLM)
+│   ├── cv/         # Object-detection pre-pass (a signal for the model, not a verdict)
+│   ├── cost/       # Token/cost metering + `--max-cost` cap
+│   ├── trace/      # Debug run-trace sidecar (observability, off by default)
+│   ├── pipeline/   # Post-merge transform stages
+│   └── export/     # Strategy pattern: json, jsonl, mcp-jsonl, dot, kblam, lora, graphiti
+├── quality/       # Importable metrics (structural, semantic, factual, consistency, composite)
+├── evaluation/    # Benchmark harness (CrossRE / REBEL / RE-DocRED / SemEval-2010 T8 / MINE)
+├── types/         # Interfaces and data models
+└── shared/        # Logger, graceful shutdown, utilities (Jaro-Winkler, cosine, config)
 ```
 
 Tests use Jest (`npm test`); mock the LLM via `ILLMProvider` for network-free unit tests.
@@ -439,7 +391,7 @@ npm start -- --config config.yaml                            # run directly (ts-
 npm run build && node ./dist/cli/index.js --config config.yaml   # or build first
 ```
 
-See `examples/kg-mail-assistant/` for a full integration (Gmail OAuth + Telegram bot + continuous email→KG pipeline) and programmatic usage via `ContainerFactory`.
+See [`examples/`](examples/) for integrations — `kg-telegram-sink` (Telegram → graph bot with an A/B canon config) and the legacy `kg-mail-assistant` (Gmail OAuth + email→KG prototype, reference-only) — plus programmatic usage via `ContainerFactory`.
 
 ## Acknowledgments
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@wanshi-kg/wanshi",
-  "version": "0.2.0",
+  "version": "0.2.1",
   "description": "Local-first CLI that turns files, code, PDFs, audio and transcripts into a provenance-tracked knowledge graph — via local Ollama or any OpenAI-compatible LLM.",
   "keywords": [
     "knowledge-graph",