@biaoo/tiangong-wiki 0.2.0

package/package.json

@@ -0,0 +1,64 @@
+{
+  "name": "@biaoo/tiangong-wiki",
+  "version": "0.2.0",
+  "description": "Local-first wiki index and query engine for Markdown knowledge pages (Tiangong Wiki).",
+  "type": "module",
+  "publishConfig": {
+    "access": "public"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/Biaoo/tiangong-wiki.git"
+  },
+  "homepage": "https://github.com/Biaoo/tiangong-wiki#readme",
+  "bugs": {
+    "url": "https://github.com/Biaoo/tiangong-wiki/issues"
+  },
+  "license": "MIT",
+  "bin": {
+    "tiangong-wiki": "./dist/index.js"
+  },
+  "main": "./dist/index.js",
+  "files": [
+    "dist",
+    "assets",
+    "references",
+    "agents",
+    "SKILL.md"
+  ],
+  "engines": {
+    "node": ">=18"
+  },
+  "scripts": {
+    "build": "npm run build:cli && npm run build:dashboard",
+    "build:cli": "tsc -p tsconfig.json",
+    "build:dashboard": "vite build --config dashboard/vite.config.ts",
+    "clean": "rm -rf dist node_modules/.vitest node_modules/.vite",
+    "dev": "tsx src/index.ts",
+    "dev:dashboard": "vite --config dashboard/vite.config.ts",
+    "test": "npm run build && vitest run",
+    "test:watch": "vitest"
+  },
+  "dependencies": {
+    "@antv/g6": "^5.1.0",
+    "@fontsource/jetbrains-mono": "^5.2.8",
+    "@fontsource/space-grotesk": "^5.2.10",
+    "@openai/codex-sdk": "^0.118.0",
+    "adm-zip": "^0.5.17",
+    "better-sqlite3": "^12.8.0",
+    "commander": "^14.0.3",
+    "gray-matter": "^4.0.3",
+    "preact": "^10.29.1",
+    "sqlite-vec": "^0.1.9"
+  },
+  "devDependencies": {
+    "@preact/preset-vite": "^2.10.5",
+    "@types/adm-zip": "^0.5.8",
+    "@types/better-sqlite3": "^7.6.13",
+    "@types/node": "^24.5.2",
+    "tsx": "^4.21.0",
+    "typescript": "^6.0.2",
+    "vite": "^8.0.7",
+    "vitest": "^4.1.2"
+  }
+}

package/references/cli-interface.md

@@ -0,0 +1,312 @@
+# CLI Reference
+
+All commands are invoked through a single entry point:
+
+```bash
+# Global install
+tiangong-wiki <command> [options]
+
+# npx
+npx @biaoo/tiangong-wiki <command> [options]
+
+# Development
+npm run dev -- <command> [options]
+```
+
+---
+
+## Command Overview
+
+| Command | Description |
+| --- | --- |
+| `setup` | Interactive configuration wizard — writes `.wiki.env` and scaffolds workspace |
+| `doctor` | Diagnose configuration, paths, embedding, and daemon health |
+| `init` | Initialize workspace assets and run the first sync |
+| `sync` | Incrementally sync pages, embeddings, and vault metadata |
+| `check-config` | Validate environment variables, config, and templates |
+| `find` | Query pages by structured metadata filters |
+| `search` | Semantic search over page summary embeddings |
+| `fts` | Full-text search over title, tags, and summary text |
+| `graph` | Traverse the knowledge graph from a root node |
+| `page-info` | Show full metadata and edges for a single page |
+| `list` | List wiki pages |
+| `stat` | Show aggregate index statistics |
+| `create` | Create a new page from a registered template |
+| `template` | List, show, or create wiki templates |
+| `type` | Inspect and recommend page types |
+| `vault` | Inspect vault files and changelog entries |
+| `lint` | Validate pages, references, and graph integrity |
+| `export-graph` | Export graph nodes and edges as JSON |
+| `export-index` | Export a human-readable Markdown index |
+| `daemon` | Manage the background HTTP daemon |
+| `dashboard` | Open the web dashboard in a browser |
+
+---
+
+## Command Details
+
+### setup
+
+```
+tiangong-wiki setup
+```
+
+Interactive step-by-step wizard that:
+
+- Records `WIKI_PATH`, `VAULT_PATH`, `WIKI_DB_PATH`, `WIKI_CONFIG_PATH`, `WIKI_TEMPLATES_PATH`
+- Optionally configures `EMBEDDING_*` and Synology vault settings
+- Writes `.wiki.env` in the current working directory
+- Scaffolds `wiki/pages/`, `vault/`, `wiki.config.json`, and `templates/`
+
+After setup, run `tiangong-wiki doctor` then `tiangong-wiki init` to complete initialization.
+
+### doctor
+
+```
+tiangong-wiki doctor [--probe] [--format text|json]
+```
+
+| Option | Description |
+| --- | --- |
+| `--probe` | Additionally test remote services (embedding endpoint, Synology NAS) |
+| `--format` | Output format: `text` (default) or `json` |
+
+Checks: `.wiki.env` loading, path existence, database accessibility, config validity, template completeness, embedding configuration, and daemon status. Exit code `2` on errors.
+
+### init
+
+```
+tiangong-wiki init [--force]
+```
+
+| Option | Description |
+| --- | --- |
+| `--force` | Force a full rebuild of the index |
+
+Creates `index.db` (if needed), builds tables according to `wiki.config.json` (including dynamic columns), and runs the first full sync.
+
+### sync
+
+```
+tiangong-wiki sync [options]
+```
+
+| Option | Description |
+| --- | --- |
+| `--path <pagePath>` | Sync only a single page (page-only, no vault scan) |
+| `--force` | Force a full rebuild (ignore content_hash) |
+| `--skip-embedding` | Skip embedding generation |
+| `--process` | Process vault queue items after sync |
+| `--vault-file <fileId>` | Process only one vault queue item |
+
+When `--path` is used, vault scanning is skipped. If a global config or embedding profile change is detected, `--path` automatically upgrades to a full sync.
+
+### check-config
+
+```
+tiangong-wiki check-config [--probe] [--format text|json]
+```
+
+Validates environment variables, `wiki.config.json`, and template files. With `--probe`, also tests embedding API connectivity.
+
+### find
+
+```
+tiangong-wiki find [options]
+```
+
+| Option | Description |
+| --- | --- |
+| `--type <pageType>` | Filter by page type |
+| `--status <status>` | Filter by status |
+| `--visibility <vis>` | Filter by visibility |
+| `--tag <tag>` | Filter by tag |
+| `--node-id <nodeId>` | Filter by node ID |
+| `--updated-after <date>` | Filter by updatedAt >= date |
+| `--sort <column>` | Sort column |
+| `--limit <n>` | Max rows (default: 50) |
+
+Supports custom columns declared in `wiki.config.json` as additional `--<column> <value>` filters.
+
+Output: JSON array to stdout.
+
+### search
+
+```
+tiangong-wiki search <query> [--type <pageType>] [--limit <n>]
+```
+
+Semantic similarity search. The query is embedded via the configured embedding API and matched against `vec_pages`. Requires `EMBEDDING_*` environment variables.
+
+Output: JSON array with `similarity` scores.
+
+### fts
+
+```
+tiangong-wiki fts <query> [--type <pageType>] [--limit <n>]
+```
+
+Full-text search against the `pages_fts` table (title, tags, summary_text). Default limit: 20.
+
+### graph
+
+```
+tiangong-wiki graph <root> [options]
+```
+
+| Option | Description |
+| --- | --- |
+| `--depth <n>` | Traversal depth (default: 1) |
+| `--edge-type <type>` | Filter by edge type |
+| `--direction <dir>` | `outgoing`, `incoming`, or `both` (default: both) |
+
+Returns `{ root, nodes[], edges[] }` as JSON.
+
+### page-info
+
+```
+tiangong-wiki page-info <pageId>
+```
+
+Returns full indexed metadata for one page: all frontmatter fields, incoming/outgoing edges, embedding status, and content hash.
+
+### list
+
+```
+tiangong-wiki list [--type <pageType>] [--sort <column>] [--limit <n>]
+```
+
+Compact listing with title, pageType, status, updatedAt, and filePath. Default sort: `updatedAt`, default limit: 50.
+
+### stat
+
+```
+tiangong-wiki stat
+```
+
+Aggregate statistics: total pages, breakdown by type/status, total edges, orphan count, embedding status, vault file count, last sync time, and registered template count.
+
+### create
+
+```
+tiangong-wiki create --type <pageType> --title <title> [--node-id <nodeId>]
+```
+
+Creates a page from the corresponding template in `wiki/templates/`, fills frontmatter fields (title, createdAt, updatedAt, etc.), writes to `wiki/pages/`, and immediately indexes it.
+
+Output: `{ created, filePath }`.
+
+### template
+
+```
+tiangong-wiki template list [--format text|json]
+tiangong-wiki template show <pageType> [--format text|json]
+tiangong-wiki template create --type <pageType> --title <title>
+```
+
+- `list` — Show registered templates
+- `show` — Display template content for a specific type
+- `create` — Generate a new template file in `wiki/templates/` and register it in `wiki.config.json`
+
+### type
+
+```
+tiangong-wiki type list [--format text|json]
+tiangong-wiki type show <pageType> [--format text|json]
+tiangong-wiki type recommend [--text <text>] [--keywords <kw>] [--limit <n>] [--format text|json]
+```
+
+- `list` — List all registered page types with their columns, edges, and summary fields
+- `show` — Show full schema for one type
+- `recommend` — Suggest page types based on vector similarity against existing pages (requires embeddings)
+
+### vault
+
+```
+tiangong-wiki vault list [--path <prefix>] [--ext <ext>]
+tiangong-wiki vault diff [--since <date>] [--path <prefix>]
+tiangong-wiki vault queue [--status pending|processing|done|skipped|error]
+```
+
+- `list` — List indexed vault files; `--path` does prefix matching on relative paths
+- `diff` — Show changes since the last sync (or since a given date with `--since`)
+- `queue` — Show processing queue status and item details
+
+### lint
+
+```
+tiangong-wiki lint [--path <pagePath>] [--level error|warning|info] [--format text|json]
+```
+
+Validates all pages (or a single page with `--path`) for integrity issues at three severity levels:
+
+- **error** — Missing required fields, unregistered pageType, broken references
+- **warning** — Orphan pages, empty sourceRefs, stale active pages, references to archived pages
+- **info** — Unregistered frontmatter fields, draft count, pending embeddings
+
+### export-graph
+
+```
+tiangong-wiki export-graph [--output <filePath>]
+```
+
+Exports all graph nodes (pages with node IDs) and edges as JSON. Prints to stdout by default; `--output` writes to a file.
+
+### export-index
+
+```
+tiangong-wiki export-index [--output <filePath>] [--group-by pageType|tags]
+```
+
+Generates a human-readable Markdown index of all pages. Default grouping: `pageType`.
+
+### daemon
+
+```
+tiangong-wiki daemon run               # Foreground (for process managers)
+tiangong-wiki daemon start             # Background (detached process)
+tiangong-wiki daemon stop
+tiangong-wiki daemon status [--format text|json]
+```
+
+The daemon provides a local HTTP server (binds to `127.0.0.1` only) for the web dashboard and accelerated query routing. When running, query commands automatically use HTTP instead of direct database access.
+
+### dashboard
+
+```
+tiangong-wiki dashboard [--no-open] [--format text|json]
+```
+
+Opens the web dashboard in the default browser. Starts the daemon automatically if it is not already running. Use `--no-open` to print the URL without opening a browser.
+
+---
+
+## Output Conventions
+
+### Exit Codes
+
+| Code | Meaning |
+| --- | --- |
+| `0` | Success |
+| `1` | Runtime error |
+| `2` | Configuration error |
+
+### Output Formats
+
+| Type | Commands | Default Output | Machine-Readable Option |
+| --- | --- | --- | --- |
+| Query | find, search, fts, graph, page-info, list, stat, vault list/diff/queue | JSON to stdout | — (default is JSON) |
+| Mutation | init, sync, create, template create | JSON to stdout | — (default is JSON) |
+| Wizard | setup | Interactive text | — |
+| Validation | lint | Human-readable text | `--format json` |
+| Export | export-graph | JSON | — |
+| Export | export-index | Markdown | — |
+| Info | doctor, check-config, template list/show, type list/show/recommend, daemon status, dashboard | Human-readable text | `--format json` |
+
+### Error Output
+
+Runtime and configuration errors are written to stderr as JSON:
+
+```json
+{ "error": "...", "type": "config | runtime | not_found | not_configured" }
+```

package/references/env.md

@@ -0,0 +1,122 @@
+# Environment Variables
+
+All configuration is managed through `.wiki.env` (created by `tiangong-wiki setup`). Variables can also be set in shell environment or a `.env` file.
+
+---
+
+## Core
+
+| Variable | Required | Description |
+| --- | --- | --- |
+| `WIKI_PATH` | Yes | Path to the Markdown pages directory |
+| `WIKI_DB_PATH` | Yes | Path to the SQLite index database |
+| `WIKI_CONFIG_PATH` | Yes | Path to `wiki.config.json` |
+| `WIKI_TEMPLATES_PATH` | Yes | Path to the templates directory |
+| `WIKI_SYNC_INTERVAL` | No | Auto-sync interval in seconds (default: `86400`) |
+
+## Vault
+
+| Variable | Required | Description |
+| --- | --- | --- |
+| `VAULT_PATH` | Yes | Path to the local vault directory |
+| `VAULT_SOURCE` | Yes | Vault source type (`local` or `synology`) |
+| `VAULT_HASH_MODE` | No | Hash mode for change detection (`content` or `mtime`, default: `mtime`) |
+| `VAULT_SYNOLOGY_REMOTE_PATH` | If `synology` | Remote path on Synology NAS |
+
+## Synology (when `VAULT_SOURCE=synology`)
+
+| Variable | Required | Description |
+| --- | --- | --- |
+| `SYNOLOGY_BASE_URL` | Yes | Synology DSM base URL |
+| `SYNOLOGY_USERNAME` | Yes | DSM username |
+| `SYNOLOGY_PASSWORD` | Yes | DSM password |
+| `SYNOLOGY_VERIFY_SSL` | No | Verify SSL certificates (default: `true`) |
+| `SYNOLOGY_READONLY` | No | Read-only mode (default: `false`) |
+
+## Embedding
+
+| Variable | Required | Description |
+| --- | --- | --- |
+| `EMBEDDING_BASE_URL` | Yes | Embedding API base URL |
+| `EMBEDDING_API_KEY` | Yes | Embedding API key |
+| `EMBEDDING_MODEL` | Yes | Embedding model name |
+| `EMBEDDING_DIMENSIONS` | No | Vector dimensions (default: model-dependent) |
+
+## Agent (Agentic Workflow)
+
+The agent uses [Codex SDK](https://www.npmjs.com/package/@openai/codex-sdk) to process vault items. When `WIKI_AGENT_BASE_URL` is set, a custom `model_provider` is injected to override any global `~/.codex/config.toml` settings, ensuring requests go to the correct endpoint.
+
+| Variable | Required | Description |
+| --- | --- | --- |
+| `WIKI_AGENT_ENABLED` | No | Enable agentic workflow (`true` / `false`, default: `false`) |
+| `WIKI_AGENT_BASE_URL` | No | LLM API base URL (e.g. `https://api.openai.com/v1`). When set, overrides global Codex config |
+| `WIKI_AGENT_API_KEY` | If enabled | API key for the LLM provider |
+| `WIKI_AGENT_MODEL` | No | Model name (e.g. `gpt-5.4`, `Qwen/Qwen3.5-397B-A17B-GPTQ-Int4`) |
+| `WIKI_AGENT_BATCH_SIZE` | No | Max concurrent vault items per batch (default: `5`) |
+| `WIKI_PARSER_SKILLS` | No | Comma-separated parser skill list (e.g. `pdf,docx,pptx,xlsx`) |
+
+### OpenAI (default)
+
+No special setup required. Set `WIKI_AGENT_BASE_URL` to `https://api.openai.com/v1` (or leave empty) and provide your API key.
+
+```env
+WIKI_AGENT_ENABLED=true
+WIKI_AGENT_API_KEY=sk-...
+WIKI_AGENT_MODEL=gpt-5.4
+```
+
+### vLLM
+
+vLLM can serve as a self-hosted LLM provider via its OpenAI-compatible API. Wiki's agentic workflow communicates through the [Responses API](https://platform.openai.com/docs/api-reference/responses) (`/v1/responses`), which requires vLLM **v0.8.5+**.
+
+```env
+WIKI_AGENT_ENABLED=true
+WIKI_AGENT_BASE_URL=http://<host>:<port>/v1
+WIKI_AGENT_API_KEY=<your-token>
+WIKI_AGENT_MODEL=Qwen/Qwen3.5-397B-A17B-GPTQ-Int4
+```
+
+#### Chat template: `developer` role support
+
+The Codex CLI sends `developer`-role messages (the OpenAI equivalent of `system` that can appear mid-conversation). Most model chat templates only recognize `system`, `user`, `assistant`, and `tool` — they will reject `developer` with:
+
+```
+400 Bad Request: "Unexpected message role."
+```
+
+**Fix:** Use a modified chat template that maps `developer` → `system`. A ready-to-use template for Qwen3.5 is included at:
+
+```
+assets/vllm/qwen3_5_openai_developer.jinja
+```
+
+Launch vLLM with the custom template:
+
+```bash
+vllm serve <model> \
+  --chat-template /path/to/qwen3_5_openai_developer.jinja \
+  --port 7730
+```
+
+The template is derived from the official Qwen3.5 template with a single semantic change: `developer` messages are treated as `system` messages. Key modifications:
+
+1. **Instruction prefix detection** — counts both `system` and `developer` at the start of the conversation (line 56)
+2. **Role normalization** — `developer` → `system` throughout the conversation loop (lines 111, 160, 166)
+
+> For other model families (LLaMA, Mistral, etc.), apply the same pattern: find `message.role == "system"` checks in the template and extend them to also match `"developer"`.
+
+#### Verifying vLLM compatibility
+
+```bash
+# Test the /v1/responses endpoint directly
+curl -s http://<host>:<port>/v1/responses \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer <token>" \
+  -d '{
+    "model": "<model-name>",
+    "input": "Say hello.",
+    "reasoning": {"effort": "low"}
+  }' | head -c 500
+```
+
+A successful response returns a JSON object with `output` containing the model's reply.