@joekytc/dsh-swarm 0.1.2 → 0.3.0

package/skills/llm-wiki-skill/CLAUDE.md

@@ -0,0 +1,152 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## 🧭 仓库导航（任何人 / AI 进来先读这里）
+
+本仓库是 **llm-wiki monorepo**，含三个区：
+
+| 区 | 位置 | 状态 |
+|---|---|---|
+| **agent 工作台**（开发主线） | `workbench/`（server + web） | 🚧 活跃开发 |
+| **共享图谱引擎** | `packages/graph-engine/` | 🚧 随 agent 演进 |
+| **Skill 形态** | 根目录 `SKILL.md` / `scripts/` / `templates/` / `platforms/` | ❄️ 成熟·维护冻结 |
+
+➡️ **开发 agent 工作台（日常主线）**：先读 [workbench/CLAUDE.md](workbench/CLAUDE.md) + [workbench/PRODUCT.md](workbench/PRODUCT.md)——当前阶段、ADR、协作规则都在那里。
+
+➡️ **本文件以下内容**：Skill 形态的安装与维护规则，**仅当你在维护 Skill 时才看**（install.sh / SKILL.md / scripts / templates / platforms）。Skill 已功能成熟、进入维护冻结，不再追加新功能。
+
+---
+
+## 🏗️ monorepo 怎么连
+
+```
+workbench/web (React 19, SSE) ──HTTP POST + SSE──▶ workbench/server (Hono + @earendil-works/pi-coding-agent)
+                                                          │
+                                                          ├─ spawn 根目录 scripts/（Skill 已有能力，ADR-16 能力归属）
+                                                          └─ 依赖 @llm-wiki/graph-engine
+packages/graph-engine ──ESM + IIFE 双产物──▶ workbench/web 图谱视图 + Skill 离线 HTML（一个引擎、两个宿主，ADR-21）
+```
+
+- 权威架构图、技术栈与全部 ADR 见 [workbench/PRODUCT.md §3 / §7](workbench/PRODUCT.md)；当前阶段与协作铁律见 [workbench/CLAUDE.md](workbench/CLAUDE.md)。
+- **三类数据彻底分离**（别写错位置）：知识库 `~/llm-wiki/<name>/`、应用数据 `~/.llm-wiki-agent/`、模型凭证 `~/.pi/agent/auth.json`（pi-agent 管，权限 0600）。应用自己的 `config.json` **绝不存** API key。
+
+## ⚙️ 开发命令速查
+
+npm workspaces，三个包（根 `package.json` 不设 `"type": "module"`——Skill 的 CommonJS 测试要兼容，ESM 声明在各子包；ADR-20）：
+
+| 包 | 路径 | npm 名 |
+|---|---|---|
+| 前端 | `workbench/web` | `@llm-wiki-agent/web` |
+| 后端 | `workbench/server` | `@llm-wiki-agent/server` |
+| 图谱引擎 | `packages/graph-engine` | `@llm-wiki/graph-engine` |
+
+| 操作 | 命令（从仓库根） |
+|---|---|
+| 一行启动（后端 `8787` + 前端 `5180`，strictPort） | `npm run dev` |
+| 全仓类型检查 | `npm run typecheck` |
+| 前端 lint | `npm run lint -w @llm-wiki-agent/web` |
+| 前端单测（unit + dom） | `npm run test -w @llm-wiki-agent/web` |
+| 前端 Paper 视觉回归（playwright） | `npm run visual:paper -w @llm-wiki-agent/web` |
+| 引擎单测 | `npm run test -w @llm-wiki/graph-engine` |
+| 后端单测（server 无聚合 test 脚本，用 node:test） | `node --import tsx --test "workbench/server/src/**/*.test.ts"` |
+
+要点：
+
+- 测试统一用 Node 内置 `node --test`（不是 jest/vitest）。前端 DOM 测试走 jsdom + @testing-library/react，视觉回归用 playwright（仅 dev 依赖，不进运行时）。
+- `web` / `server` 的 `build` 与 `typecheck` 带 `prebuild` / `pretypecheck` 钩子，会**自动先 build `@llm-wiki/graph-engine`**。改了引擎代码后，跑前端/后端的 typecheck 或 build 会自动带上最新引擎产物；单跑引擎自己的 `tsc --noEmit` 不会刷新 `dist/`。
+- Node `>=22.19.0`（pi-coding-agent 硬要求，`.mise.toml` / `.nvmrc` 锁定）。
+
+---
+
+## Skill 形态：安装与维护
+
+先看这三个文件：
+
+- [README.md](README.md)：多平台总说明
+- [platforms/claude/CLAUDE.md](platforms/claude/CLAUDE.md)：Claude 专属入口提示
+- [SKILL.md](SKILL.md)：核心能力和工作流
+
+## Claude 安装动作
+
+如果当前任务是安装这个 skill，优先执行：
+
+```bash
+bash install.sh --platform claude
+```
+
+> `setup.sh` 是 `install.sh --platform claude` 的兼容包装，老用户可以继续用。
+
+默认只准备知识库核心主线。如果这次要自动提取网页 / X / 微信公众号 / YouTube / 知乎，再执行：
+
+```bash
+bash install.sh --platform claude --with-optional-adapters
+```
+
+安装完成后，还会一并带上 `/llm-wiki-upgrade`。以后要更新核心主线，可以直接让 Claude 执行这个命令。
+
+## 分支管理规则
+
+改动代码（非纯文档/注释）时，按以下流程操作：
+
+1. 开新分支：从 main 创建，命名用 feat/ 或 fix/ 前缀（如 fix/cache-reliability-write-through）
+2. 分步 commit：每完成一个逻辑单元就提交（脚本实现 → 测试 → 文档更新，分开 commit）
+3. 推送并创建 PR：推到远端后用 `gh pr create` 创建 PR
+4. 合并：确认测试通过后在 GitHub 上合并
+
+不需要开分支的情况：
+- 只改了 CLAUDE.md、文档、注释
+- 只是探索性阅读代码
+
+设计文档或 plan 写完准备动手改代码时，也先开分支再开始实现。
+
+## 推送前测试规则
+
+每次 `git push` 前必须验证，按改动范围选深度：
+
+### 第一层：快速检查（Claude Code 直接跑，1 分钟内）
+
+不管改了什么都跑这 3 项：
+
+1. `bash install.sh --dry-run --platform codex` — 安装脚本不报错
+2. 改过的脚本如果有 `tests/fixtures/`，跑一下 diff 预期输出
+3. `grep -r '本机用户路径\\|真实姓名\\|私有素材路径' scripts/ templates/ tests/ SKILL.md` — 没泄露隐私路径
+
+### 第二/三层：工作流测试（你在 codex 终端手动跑）
+
+- **第二层**（只改了 SKILL.md 里个别工作流）：Claude Code 生成测试提示词写到文件，告诉你路径，你复制到 codex 跑涉及的工作流
+- **第三层**（多工作流改动 / 版本号升级）：Claude Code 生成全量回归提示词，你在 codex 跑完整流程（init → ingest → lint → digest → graph）
+
+素材复用本机私有测试素材，不上传到远端。
+
+codex 跑完后把 `test-report.md` 发回来，Claude Code 确认无阻塞问题后才执行 `git push`。
+
+## 推送前文档更新规则
+
+每次 commit 含功能改动（feat/fix）后、`git push` 前，**必须**主动检查并更新以下文档，不需要用户提醒：
+
+1. **CHANGELOG.md**：在顶部加新版本条目（日期、新增/改进/修复分类）
+2. **README.md 功能列表**：新增功能或行为变化时，在"功能"章节补一条
+3. **版本号**：如果改动涉及新功能，在 CHANGELOG 条目里用新版本号（按 v当前+1 递增）
+
+跳过条件：纯文档/排版/注释改动不需要更新。
+
+## Skill routing
+
+When the user's request matches an available skill, ALWAYS invoke it using the Skill
+tool as your FIRST action. Do NOT answer directly, do NOT use other tools first.
+The skill has specialized workflows that produce better results than ad-hoc answers.
+
+Key routing rules:
+- Product ideas, "is this worth building", brainstorming → invoke office-hours
+- Bugs, errors, "why is this broken", 500 errors → invoke investigate
+- Ship, deploy, push, create PR → invoke ship
+- QA, test the site, find bugs → invoke qa
+- Code review, check my diff → invoke review
+- Update docs after shipping → invoke document-release
+- Weekly retro → invoke retro
+- Design system, brand → invoke design-consultation
+- Visual audit, design polish → invoke design-review
+- Architecture review → invoke plan-eng-review
+- Save progress, checkpoint, resume → invoke checkpoint
+- Code quality, health check → invoke health

package/skills/llm-wiki-skill/HERMES.md

@@ -0,0 +1,44 @@
+# HERMES.md
+
+这是 llm-wiki 在 Hermes 下的入口文件。
+
+先看这三个文件：
+
+- [README.md](README.md)：多平台总说明
+- [platforms/hermes/README.md](platforms/hermes/README.md)：Hermes 专属入口提示
+- [SKILL.md](SKILL.md)：核心能力和工作流
+
+## Hermes 安装动作
+
+如果当前任务是安装这个 skill，执行：
+
+```bash
+bash install.sh --platform hermes
+```
+
+默认安装到 `~/.hermes/skills/llm-wiki`。
+
+默认只准备知识库核心主线。如果这次要自动提取网页 / X / 微信公众号 / YouTube / 知乎，再执行：
+
+```bash
+bash install.sh --platform hermes --with-optional-adapters
+```
+
+## 重要提醒
+
+- 不要把这个仓库当成 Hermes 专属仓库；Claude Code、Codex、OpenClaw 也共用同一套核心内容
+- Hermes 会优先读取仓库根的 `HERMES.md`；这里负责安装入口，知识库能力本身仍以 [SKILL.md](SKILL.md) 为准
+- 安装完成后，再按 [SKILL.md](SKILL.md) 的工作流继续做事
+
+## 使用顺序
+
+安装完成后，按 [SKILL.md](SKILL.md) 中的工作流继续执行：
+
+1. `init`
+2. `ingest`
+3. `batch-ingest`
+4. `query`
+5. `digest`
+6. `lint`
+7. `status`
+8. `graph`

package/skills/llm-wiki-skill/README.en.md

@@ -0,0 +1,261 @@
+English | [中文](README.md)
+
+<div align="center">
+
+# llm-wiki
+
+Based on [Andrej Karpathy](https://karpathy.ai/)'s [llm-wiki methodology](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)
+
+**The personal knowledge base that grows with you**
+
+Turn scattered information into a growing, interconnected knowledge base
+
+[![version](https://img.shields.io/badge/v3.6.4-Digital%20Landscape%20Graph-E8D5B5?style=flat-square&labelColor=3a3026&color=E8D5B5)](https://github.com/sdyckjq-lab/llm-wiki-skill/releases)
+[![license](https://img.shields.io/badge/MIT-license-5a6e5c?style=flat-square&labelColor=3a3026)](LICENSE)
+[![platforms](https://img.shields.io/badge/Claude·Codex·OpenClaw·Hermes-multi--platform-7a96a6?style=flat-square&labelColor=3a3026)]
+
+</div>
+
+---
+
+## Preview
+
+<div align="center">
+<img src="assets/graph-demo.gif" width="100%" alt="Knowledge Graph Demo">
+</div>
+
+Oriental editorial × digital landscape interactive knowledge graph — double-click the HTML file to explore in your browser. Search, community legend, focus filters, layered node grammar, hover previews, click-to-read, Shift multi-select, canvas zoom/pan, and minimap navigation all run offline.
+
+---
+
+## 30-Second Start
+
+Give this repo link to your agent and let it install itself.
+
+```bash
+# Claude Code
+bash install.sh --platform claude
+
+# Codex
+bash install.sh --platform codex
+
+# OpenClaw
+bash install.sh --platform openclaw
+
+# Hermes
+bash install.sh --platform hermes
+```
+
+Then just say:
+
+> "Help me initialize a knowledge base"
+> "Digest this article: <url>"
+
+The key difference: knowledge is **compiled once, maintained continuously** — not re-derived from scratch every query.
+
+---
+
+## Highlights
+
+| | Feature | Description |
+|---|---|---|
+| 🗺️ | **Digital Landscape Graph** | Self-contained HTML with a three-column oriental editorial layout, draggable and zoomable canvas, minimap navigation, and readable side panels |
+| ✨ | **Graph Reading Polish** | Nodes now separate map labels, index slips, and cinnabar annotations; the default view is lighter, hover previews stay available, and clicks open reading mode |
+| 🎓 | **Local Reading Flow** | Community legend, focus filters, scoped search, the reader drawer, and the selection drawer stay connected to the visible graph; offline HTML keeps selected facts visible too |
+| 📦 | **Zero-config Init** | One sentence to create a full knowledge base with directory structure and templates |
+| 🔗 | **Structured Wiki** | Auto-generates entity pages, topic pages, source summaries with `[[bidirectional links]]` |
+| 🏷️ | **Confidence Annotation** | EXTRACTED / INFERRED / AMBIGUOUS / UNVERIFIED — see at a glance what needs verification |
+| 🔄 | **Smart Caching** | SHA256 deduplication + write-through cache + self-healing safety net |
+| 🧠 | **Conversation Crystallization** | Turn valuable conversations into knowledge base pages directly |
+| 📡 | **Auto Context Injection** | SessionStart hook makes the agent automatically sense the knowledge base every session |
+| 📊 | **Multi-format Analysis** | Deep reports, comparison tables, and timeline views |
+
+---
+
+## Source Support
+
+| Category | Sources | How |
+|---|---|---|
+| Core | PDF, Markdown, Text, HTML, Plain text | Direct ingestion, no external dependencies |
+| Optional | Web articles, X/Twitter, WeChat, YouTube, Zhihu | Auto-extract via adapters; manual fallback if extraction fails |
+| Manual | Xiaohongshu | Paste content directly |
+
+Optional adapters require one extra flag:
+
+```bash
+bash install.sh --platform claude --with-optional-adapters
+```
+
+---
+
+## Platform Entry Points
+
+Each platform has its own setup guide:
+
+- [Claude Code](platforms/claude/CLAUDE.md)
+- [Codex](platforms/codex/AGENTS.md)
+- [OpenClaw](platforms/openclaw/README.md)
+- [Hermes](platforms/hermes/README.md)
+
+---
+
+<details>
+<summary><strong>Full Feature List</strong></summary>
+
+- **Research Direction Guidance** — `purpose.md` gives the agent a clear direction for organizing and querying
+- **Two-step Ingestion** — Analyze first, then generate; long content uses chained thinking
+- **Ingest Format Validation** — Scripts auto-validate analysis results; even weak models won't produce broken data
+- **Smart Material Routing** — Auto-selects the best extraction method based on URL domain
+- **Core-first Install** — Default setup only includes the core pipeline; optional extractors enabled explicitly
+- **Claude Companion Upgrade Command** — `/llm-wiki-upgrade` included after installation
+- **Material Deletion** — Cascade-delete with automatic cleanup of associated pages, broken links, and cache
+- **Query Persistence** — Save valuable comprehensive answers back to the knowledge base
+- **Batch Digestion** — Give a folder path, process all files at once
+- **Knowledge Base Health Check** — Scripts detect orphan pages, broken links, index consistency; plus AI-level contradiction and cross-reference checks
+- **Ingest Privacy Check** — First-time ingestion reminds you to check for phone numbers, API keys, etc.
+- **Graph Relationship Vocabulary** — Optional manual annotation vocabulary for more precise graph diagrams
+- **Obsidian Compatible** — All content is local markdown, open directly in Obsidian
+
+</details>
+
+<details>
+<summary><strong>Installation Details</strong></summary>
+
+### Default Install Locations
+
+| Platform | Path |
+|---|---|
+| Claude Code | `~/.claude/skills/llm-wiki` |
+| Codex | `~/.codex/skills/llm-wiki` |
+| OpenClaw | `~/.openclaw/skills/llm-wiki` |
+| Hermes | `~/.hermes/skills/llm-wiki` |
+
+### Updating
+
+Already installed? Run from the repo directory:
+
+```bash
+bash install.sh --upgrade
+```
+
+Automatically: `git pull` → detect installed platforms → re-copy core files → existing hooks preserved.
+
+For Claude Code with default install, you can also use `/llm-wiki-upgrade` directly.
+
+Custom directories:
+
+```bash
+bash install.sh --upgrade --platform openclaw --target-dir <your-skill-dir>/llm-wiki
+```
+
+```bash
+bash install.sh --upgrade --platform hermes --target-dir <your-skill-dir>/llm-wiki
+```
+
+### Prerequisites
+
+- Core: your agent can run shell commands and read/write local files
+- Optional: `uv` for WeChat extraction; `bun` or `npm` for web extraction; Chrome debug mode (port 9222) for authenticated sessions
+
+</details>
+
+<details>
+<summary><strong>Directory Structure</strong></summary>
+
+```
+your-knowledge-base/
+├── raw/                    # Raw materials (immutable)
+│   ├── articles/           # Web articles
+│   ├── tweets/             # X/Twitter
+│   ├── wechat/             # WeChat articles
+│   ├── xiaohongshu/        # Xiaohongshu
+│   ├── zhihu/              # Zhihu
+│   ├── pdfs/               # PDF
+│   ├── notes/              # Notes
+│   └── assets/             # Attachments
+├── wiki/                   # AI-generated knowledge base
+│   ├── entities/           # Entity pages
+│   ├── topics/             # Topic pages
+│   ├── sources/            # Source summaries
+│   ├── comparisons/        # Comparisons
+│   ├── synthesis/          # Synthesis
+│   │   └── sessions/       # Conversation crystallization
+│   └── queries/            # Saved queries
+├── purpose.md              # Research direction
+├── index.md                # Index
+├── log.md                  # Operation log
+├── .wiki-schema.md         # Config
+└── .wiki-cache.json        # Dedup cache
+```
+
+</details>
+
+<details>
+<summary><strong>FAQ</strong></summary>
+
+**Is this Claude-only?**
+No. Claude is one of multiple entry points. The same repo can be installed by Claude Code, Codex, OpenClaw, or Hermes.
+
+**Why does Hermes care about `HERMES.md`?**
+Hermes loads the repo root `HERMES.md` as its highest-priority project context. That file only defines the Hermes entry and install path; the shared workflow still lives in `SKILL.md`.
+
+**Can I update from within Claude Code?**
+Yes. `/llm-wiki-upgrade` updates the core. Add `--with-optional-adapters` to also refresh web/X/WeChat/YouTube/Zhihu extraction.
+
+**X/Twitter extraction failing?**
+Ensure optional adapters are installed (`--with-optional-adapters`). For authenticated content, start Chrome with debug port 9222. You can also paste content directly.
+
+**WeChat extraction failing?**
+Requires `uv`. Install it, then re-run with `--with-optional-adapters`.
+
+</details>
+
+---
+
+## Windows Users
+
+Windows PowerShell 5.1 (bundled with Win10 / Win11) defaults to GB2312 for console encoding, ASCII (CP1252) for `$OutputEncoding`, and `gbk` for Python subprocess `sys.stdout.encoding` on Chinese locale. Running `bash install.sh` directly under PowerShell 5.1 garbles Chinese output and hook JSON ([#16](https://github.com/sdyckjq-lab/llm-wiki-skill/issues/16)).
+
+**Option A — Use `install.ps1` (recommended)**
+
+From the repo root:
+
+```powershell
+powershell -ExecutionPolicy Bypass -File install.ps1 --platform claude
+powershell -ExecutionPolicy Bypass -File install.ps1 --platform codex --dry-run
+```
+
+`install.ps1` sets console / `$OutputEncoding` / `PYTHONIOENCODING` all to UTF-8, then delegates to `bash install.sh`.
+
+**Option B — Manually set PowerShell encoding**
+
+```powershell
+chcp 65001
+[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
+$OutputEncoding = [System.Text.Encoding]::UTF8
+$env:PYTHONIOENCODING = 'utf-8'
+bash install.sh --platform claude
+```
+
+**Option C — Upgrade to PowerShell 7+**
+
+PowerShell 7 defaults to UTF-8. Install: `winget install Microsoft.PowerShell`, then run `bash install.sh --platform claude` under `pwsh` directly.
+
+### Python command
+
+Windows typically installs Python as `python.exe`, not `python3.exe` (the `python3` on PATH via Microsoft Store is a stub that prompts installation instead of running). `scripts/shared-config.sh` auto-detects: **tries `python3` first, falls back to `python`**. As long as Python 3.8+ is on PATH under either name, all scripts work.
+
+---
+
+## Credits
+
+This project builds on:
+
+- **[Andrej Karpathy](https://karpathy.ai/)** — [llm-wiki gist](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f), the core methodology
+- **[baoyu-url-to-markdown](https://github.com/JimLiu/baoyu-skills#baoyu-url-to-markdown)** by [JimLiu](https://github.com/JimLiu) — Web/X/Twitter content extraction
+- **youtube-transcript** — YouTube subtitle extraction
+- **[wechat-article-to-markdown](https://github.com/jackwener/wechat-article-to-markdown)** — WeChat article extraction
+
+## License
+
+MIT