agent-knowledge 1.5.2 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/INGEST.md ADDED
@@ -0,0 +1,189 @@
1
+ # Codebase Ingestion
2
+
3
+ The **knowledge-ingest** skill populates or updates agent-knowledge from a codebase directory. It uses tree-sitter for zero-token structural extraction, then the agent distills the results into knowledge entries and graph edges.
4
+
5
+ **Same command for both initial ingest and updates** — the skill detects whether a `.knowledge-ingest-cache.json` exists in the target directory and behaves accordingly:
6
+
7
+ | Mode | Trigger | Behavior |
8
+ | ---------------------- | ------------------- | ----------------------------------------------------------------------------------------------- |
9
+ | **Full ingest** | No cache file found | Scans all files, creates all entries from scratch |
10
+ | **Incremental update** | Cache file exists | Only reprocesses changed files (SHA256 diff), adds new files, removes entries for deleted files |
11
+
12
+ ## How It Works
13
+
14
+ ```
15
+ Codebase → tree-sitter AST → clusters → knowledge entries + graph edges
16
+ (zero tokens) (agent) (via existing MCP tools)
17
+ ↕
18
+ .knowledge-ingest-cache.json
19
+ (tracks SHA256 per file for incremental updates)
20
+ ```
21
+
22
+ **Phase 1 — Structural extraction**: A standalone Node.js script (`scripts/tree-sitter-extract.mjs`) parses source files via web-tree-sitter (WASM). Extracts classes, functions, imports, exports, call graphs, rationale comments, and SHA256 hashes. No LLM tokens consumed.
23
+
24
+ **Phase 2 — Clustering**: The agent groups files into subsystems based on the dependency graph and directory structure. Identifies god nodes (most-imported symbols), entry points, and cross-cutting concerns.
25
+
26
+ **Phase 3 — Knowledge entries**: One entry per subsystem (not per file), plus project overview, architecture decisions from rationale comments, and workflow entries from CI configs. All created via `knowledge({ action: "write" })`.
27
+
28
+ **Phase 4 — Graph edges**: Subsystem relationships mapped via `knowledge_graph({ action: "link" })` — `part_of`, `depends_on`, `builds_on`. Auto-linking adds `related_to` edges automatically.
29
+
30
+ **Phase 5 — Multi-modal**: PDFs, architecture diagrams, and URLs are processed if present.
31
+
32
+ ## Prerequisites
33
+
34
+ The tree-sitter dependencies ship with agent-knowledge (v1.6.0+):
35
+
36
+ - `web-tree-sitter` — WASM-based parser (no native compilation)
37
+ - `tree-sitter-wasms` — pre-built grammar files for 8 languages
38
+
39
+ No additional installation needed.
40
+
41
+ ## Supported Languages
42
+
43
+ | Language | Extensions |
44
+ | ---------- | ----------------------------------- |
45
+ | TypeScript | `.ts`, `.tsx` |
46
+ | JavaScript | `.js`, `.jsx`, `.mjs`, `.cjs` |
47
+ | Python | `.py` |
48
+ | Go | `.go` |
49
+ | Rust | `.rs` |
50
+ | Java | `.java` |
51
+ | C | `.c`, `.h` |
52
+ | C++ | `.cpp`, `.cc`, `.cxx`, `.hpp`, etc. |
53
+
54
+ ## Usage
55
+
56
+ ### Supported Platforms
57
+
58
+ The skill uses the [Agent Skills standard](https://agentskills.io) (`SKILL.md`) and works on Claude Code, OpenCode, Cursor, Codex CLI, and Gemini CLI. `setup.js` installs to both `~/.claude/skills/` and `~/.agents/skills/` to cover all platforms.
59
+
60
+ ```
61
+ /knowledge-ingest .
62
+ /knowledge-ingest ~/projects/my-api
63
+ /knowledge-ingest ./libs/auth
64
+ ```
65
+
66
+ Or use natural language: "ingest this codebase", "scan this project", "load this into knowledge", "update knowledge", "refresh knowledge".
67
+
68
+ ### Direct Script Usage
69
+
70
+ Run the tree-sitter extraction script directly (any platform):
71
+
72
+ ```bash
73
+ node ~/.claude/mcp-servers/agent-knowledge/scripts/tree-sitter-extract.mjs ./my-project --json
74
+ ```
75
+
76
+ Then use the JSON output to create knowledge entries via the REST API or MCP tools.
77
+
78
+ ## Extraction Script
79
+
80
+ The tree-sitter extraction script is a standalone CLI:
81
+
82
+ ```bash
83
+ node scripts/tree-sitter-extract.mjs <path> [options]
84
+ ```
85
+
86
+ ### Options
87
+
88
+ | Flag | Default | Description |
89
+ | ------------- | ------------------------------ | ------------------------------- |
90
+ | `--include` | all supported extensions | Glob patterns (comma-separated) |
91
+ | `--exclude` | node_modules, dist, .git, etc. | Directories to skip |
92
+ | `--max-files` | 2000 | Cap on files to process |
93
+ | `--json` | compact | Pretty-print JSON output |
94
+
95
+ ### Output Schema
96
+
97
+ ```json
98
+ {
99
+ "version": 1,
100
+ "timestamp": "2026-04-12T...",
101
+ "root": "/path/to/project",
102
+ "fileCount": 42,
103
+ "files": [
104
+ {
105
+ "path": "src/foo.ts",
106
+ "language": "typescript",
107
+ "sha256": "abc123...",
108
+ "size": 4200,
109
+ "symbols": [
110
+ {
111
+ "kind": "class",
112
+ "name": "FooService",
113
+ "line": 10,
114
+ "endLine": 85,
115
+ "params": null,
116
+ "docstring": "..."
117
+ }
118
+ ],
119
+ "imports": [{ "source": "./bar", "names": ["BarService"] }],
120
+ "exports": ["FooService"],
121
+ "rationale": [{ "line": 22, "tag": "WHY", "text": "Debounce due to upstream rate limit" }],
122
+ "calls": [{ "caller": "processFoo", "callee": "BarService.create" }]
123
+ }
124
+ ],
125
+ "dependencyGraph": {
126
+ "src/foo.ts": ["src/bar.ts"]
127
+ }
128
+ }
129
+ ```
130
+
131
+ ## Incremental Updates
132
+
133
+ After the first ingest, a `.knowledge-ingest-cache.json` file is written to the target directory. On subsequent runs:
134
+
135
+ - **Unchanged files** (same SHA256): skipped entirely
136
+ - **Changed files**: re-extracted, corresponding knowledge entries updated
137
+ - **Deleted files**: their knowledge entries removed
138
+ - **New files**: processed normally
139
+
140
+ ## What Gets Created
141
+
142
+ A typical 200-file project produces:
143
+
144
+ | Category | Count | Content |
145
+ | ------------ | ----- | ------------------------------------------ |
146
+ | `projects/` | 1 | Project overview, tech stack, architecture |
147
+ | `notes/` | 10-25 | Subsystem summaries |
148
+ | `decisions/` | 2-10 | From rationale comments |
149
+ | `workflows/` | 1-5 | From CI/build configs |
150
+
151
+ Plus graph edges: `part_of`, `depends_on`, `builds_on`, and auto-linked `related_to`.
152
+
153
+ ## Customization
154
+
155
+ ### Exclude patterns
156
+
157
+ ```
158
+ /knowledge-ingest ./project --exclude "test,mock,generated"
159
+ ```
160
+
161
+ Or pass to the script directly:
162
+
163
+ ```bash
164
+ node scripts/tree-sitter-extract.mjs ./project --exclude "test,mock,generated,vendor"
165
+ ```
166
+
167
+ ### Large codebases
168
+
169
+ The `--max-files` flag (default 2000) caps how many files are processed, prioritizing the most recently modified. For very large repos, the agent also caps at 30 clusters and merges smaller ones.
170
+
171
+ ## Installation
172
+
173
+ ### Via setup script
174
+
175
+ ```bash
176
+ cd /path/to/agent-knowledge
177
+ node scripts/setup.js
178
+ ```
179
+
180
+ Installs to both `~/.claude/skills/knowledge-ingest/` and `~/.agents/skills/knowledge-ingest/`, covering Claude Code, OpenCode, Cursor, Codex CLI, and Gemini CLI.
181
+
182
+ ### Manual
183
+
184
+ Copy `skills/knowledge-ingest/SKILL.md` to either location:
185
+
186
+ ```bash
187
+ ~/.claude/skills/knowledge-ingest/SKILL.md # Claude Code, OpenCode, Cursor
188
+ ~/.agents/skills/knowledge-ingest/SKILL.md # Codex CLI, Gemini CLI, universal
189
+ ```
package/docs/SETUP.md CHANGED
@@ -61,7 +61,17 @@ agent-knowledge works with any MCP client (stdio) or HTTP client (REST API). Pic
61
61
 
62
62
  ### Claude Code
63
63
 
64
- #### Register the MCP server
64
+ #### Quick setup (recommended)
65
+
66
+ From the agent-knowledge clone:
67
+
68
+ ```bash
69
+ node scripts/setup.js
70
+ ```
71
+
72
+ This detects Claude Code, registers the MCP server in `~/.claude.json`, wires all 4 hooks into `~/.claude/settings.json`, and grants `mcp__agent-knowledge__*` permission in one shot. Restart Claude Code afterwards to pick up the new config.
73
+
74
+ #### Register the MCP server (manual)
65
75
 
66
76
  ```bash
67
77
  claude mcp add agent-knowledge -s user \
@@ -170,15 +180,18 @@ curl -X PUT http://localhost:3423/api/knowledge/notes/my-note \
170
180
 
171
181
  ## Hooks
172
182
 
173
- Hooks announce the dashboard URL on session start. Support varies by client.
174
-
175
- ### Claude Code Hooks
183
+ agent-knowledge ships **4 hook scripts** that wire into Claude Code's lifecycle events. `scripts/setup.js` installs them automatically; the block below shows the manual equivalent. Support varies by client.
176
184
 
177
- #### SessionStart + SubagentStart (`scripts/hooks/session-start.js`)
185
+ | Hook | Event | Purpose |
186
+ | ------------------------ | ------------ | ------------------------------------------------------------- |
187
+ | `session-start.js` | SessionStart | Announces the knowledge dashboard URL |
188
+ | `precompact-flush.mjs` | PreCompact | Rich session summary via the compiled agent-knowledge library |
189
+ | `precompact-distill.mjs` | PreCompact | Lightweight text snapshot of recent user prompts |
190
+ | `sessionend-distill.mjs` | SessionEnd | Final summary (turn counts, tool uses, first 20 prompts) |
178
191
 
179
- Announces the knowledge dashboard URL on session start. Also fires for subagents via `SubagentStart`, ensuring spawned agents know about the knowledge base.
192
+ See [`docs/hooks.md`](hooks.md) for the full hook reference including environment variables and test coverage.
180
193
 
181
- Add to `~/.claude/settings.json`:
194
+ ### Claude Code Hooks
182
195
 
183
196
  ```json
184
197
  {
@@ -194,13 +207,29 @@ Add to `~/.claude/settings.json`:
194
207
  ]
195
208
  }
196
209
  ],
197
- "SubagentStart": [
210
+ "PreCompact": [
198
211
  {
199
212
  "hooks": [
200
213
  {
201
214
  "type": "command",
202
- "command": "node \"/path/to/agent-knowledge/scripts/hooks/session-start.js\"",
203
- "timeout": 5
215
+ "command": "node \"/path/to/agent-knowledge/scripts/hooks/precompact-flush.mjs\"",
216
+ "timeout": 10
217
+ },
218
+ {
219
+ "type": "command",
220
+ "command": "node \"/path/to/agent-knowledge/scripts/hooks/precompact-distill.mjs\"",
221
+ "timeout": 10
222
+ }
223
+ ]
224
+ }
225
+ ],
226
+ "SessionEnd": [
227
+ {
228
+ "hooks": [
229
+ {
230
+ "type": "command",
231
+ "command": "node \"/path/to/agent-knowledge/scripts/hooks/sessionend-distill.mjs\"",
232
+ "timeout": 10
204
233
  }
205
234
  ]
206
235
  }
@@ -299,6 +328,38 @@ Add equivalent instructions to `.windsurfrules` in your project root.
299
328
 
300
329
  ---
301
330
 
331
+ ## Skills
332
+
333
+ agent-knowledge ships a **knowledge-ingest** skill for bootstrapping or updating the knowledge base from a codebase. The skill follows the [Agent Skills standard](https://agentskills.io) (`SKILL.md` format) and works across multiple platforms.
334
+
335
+ ### Automatic installation
336
+
337
+ `node scripts/setup.js` installs the skill to both `~/.claude/skills/` and `~/.agents/skills/`, covering all major platforms:
338
+
339
+ | Platform | Discovery path | Covered by setup.js |
340
+ | ----------- | ---------------------------------------- | ------------------- |
341
+ | Claude Code | `~/.claude/skills/` | Yes |
342
+ | OpenCode | `~/.claude/skills/`, `~/.agents/skills/` | Yes |
343
+ | Cursor | `~/.claude/skills/`, `~/.agents/skills/` | Yes |
344
+ | Codex CLI | `~/.agents/skills/` | Yes |
345
+ | Gemini CLI | `~/.agents/skills/` | Yes |
346
+
347
+ ### Manual installation
348
+
349
+ Copy `skills/knowledge-ingest/SKILL.md` to any of these locations:
350
+
351
+ ```bash
352
+ # Claude Code, OpenCode, Cursor
353
+ ~/.claude/skills/knowledge-ingest/SKILL.md
354
+
355
+ # Codex CLI, Gemini CLI, OpenCode (universal)
356
+ ~/.agents/skills/knowledge-ingest/SKILL.md
357
+ ```
358
+
359
+ See [Ingestion Guide](INGEST.md) for usage details, supported languages, and incremental updates.
360
+
361
+ ---
362
+
302
363
  ## Knowledge Base Setup
303
364
 
304
365
  The knowledge base is a git repository with categorized markdown files.
package/docs/hooks.md ADDED
@@ -0,0 +1,116 @@
1
+ # Hooks (Claude Code)
2
+
3
+ agent-knowledge ships four hook scripts that integrate with Claude Code's
4
+ lifecycle events. They live under `scripts/hooks/` and are installed by
5
+ `scripts/setup.js` into `~/.claude/settings.json`.
6
+
7
+ All hooks fail open — any internal error is logged to stderr and the hook
8
+ returns an empty JSON object so the user is never blocked.
9
+
10
+ | Script | Event | Purpose |
11
+ | ------------------------ | ------------ | -------------------------------------------------------- |
12
+ | `session-start.js` | SessionStart | Announces the knowledge dashboard URL |
13
+ | `precompact-flush.mjs` | PreCompact | Rich session summary via the library |
14
+ | `precompact-distill.mjs` | PreCompact | Lightweight text snapshot of recent user prompts |
15
+ | `sessionend-distill.mjs` | SessionEnd | Final summary (turn counts, tool uses, first 20 prompts) |
16
+
17
+ ## session-start.js
18
+
19
+ Prints the dashboard URL (`http://localhost:3423` by default, override via
20
+ `AGENT_KNOWLEDGE_PORT`) and injects it as session context.
21
+
22
+ ## precompact-flush.mjs
23
+
24
+ Called right before the host compacts the transcript. Imports
25
+ `dist/sessions/summary.js` from the built agent-knowledge library, pulls a
26
+ rich session summary via `getSessionSummary(sessionId)`, and writes it to
27
+ `~/agent-knowledge/sessions/<project-slug>/precompact-<timestamp>-<id>.md`.
28
+ Requires `npm run build` to have succeeded — if `dist/` is missing, the hook
29
+ logs the reason to stderr and emits `{}`.
30
+
31
+ ### Environment variables
32
+
33
+ | Variable | Default | Description |
34
+ | ---------------------- | ------------------- | -------------------------------- |
35
+ | `KNOWLEDGE_MEMORY_DIR` | `~/agent-knowledge` | Where rich summaries are written |
36
+
37
+ ## precompact-distill.mjs
38
+
39
+ A simpler heuristic companion to `precompact-flush.mjs`. Reads the last
40
+ ~200 KB of the transcript JSONL, extracts the last 10 user prompts, and
41
+ writes them to
42
+ `~/claude-memory/projects/precompact-<slug>-<session>.md`. This one does
43
+ not require the library to be built, so it still works when `dist/` is
44
+ missing. Running both is fine — they target different files.
45
+
46
+ ### Environment variables
47
+
48
+ | Variable | Default | Description |
49
+ | ---------------------- | ----------------- | ------------------------------------------------- |
50
+ | `KNOWLEDGE_MEMORY_DIR` | `~/claude-memory` | Where the snapshot is written (`<dir>/projects/`) |
51
+
52
+ ## sessionend-distill.mjs
53
+
54
+ On `SessionEnd`, walks the whole transcript JSONL, counts user / assistant
55
+ turns and `tool_use` blocks, collects the first 20 user prompts, and writes
56
+ a markdown summary to
57
+ `~/claude-memory/projects/session-<slug>-<session>.md`. Gives
58
+ agent-knowledge a breadcrumb for later session search/recall.
59
+
60
+ ## Manual configuration
61
+
62
+ `scripts/setup.js` writes the hooks into `~/.claude/settings.json`
63
+ automatically. If you need to configure them by hand, add entries like:
64
+
65
+ ```json
66
+ {
67
+ "hooks": {
68
+ "SessionStart": [
69
+ {
70
+ "hooks": [
71
+ {
72
+ "type": "command",
73
+ "command": "node \"/abs/path/to/agent-knowledge/scripts/hooks/session-start.js\"",
74
+ "timeout": 5
75
+ }
76
+ ]
77
+ }
78
+ ],
79
+ "PreCompact": [
80
+ {
81
+ "hooks": [
82
+ {
83
+ "type": "command",
84
+ "command": "node \"/abs/path/to/agent-knowledge/scripts/hooks/precompact-flush.mjs\"",
85
+ "timeout": 10
86
+ },
87
+ {
88
+ "type": "command",
89
+ "command": "node \"/abs/path/to/agent-knowledge/scripts/hooks/precompact-distill.mjs\"",
90
+ "timeout": 10
91
+ }
92
+ ]
93
+ }
94
+ ],
95
+ "SessionEnd": [
96
+ {
97
+ "hooks": [
98
+ {
99
+ "type": "command",
100
+ "command": "node \"/abs/path/to/agent-knowledge/scripts/hooks/sessionend-distill.mjs\"",
101
+ "timeout": 10
102
+ }
103
+ ]
104
+ }
105
+ ]
106
+ }
107
+ }
108
+ ```
109
+
110
+ Replace `/abs/path/to/agent-knowledge` with your clone path.
111
+
112
+ ## Testing the hooks
113
+
114
+ `tests/hooks/hooks.test.ts` covers all four scripts with shape-correct
115
+ fail-open assertions. Run `npm test -- tests/hooks/` to execute them in
116
+ isolation.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-knowledge",
3
- "version": "1.5.2",
3
+ "version": "1.6.0",
4
4
  "description": "Cross-session memory and recall for AI agents — git-synced knowledge base, knowledge graph, confidence scoring, hybrid semantic+TF-IDF search, auto-distillation with secrets scrubbing",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",
@@ -25,6 +25,7 @@
25
25
  "files": [
26
26
  "dist/",
27
27
  "scripts/",
28
+ "skills/",
28
29
  "agent-desk-plugin.json",
29
30
  "docs/*.md",
30
31
  "README.md",
@@ -79,6 +80,10 @@
79
80
  "better-sqlite3": "^12.8.0",
80
81
  "morphdom": "^2.7.8",
81
82
  "sqlite-vec": "^0.1.0",
83
+ "tree-sitter-wasms": "^0.1.13",
84
+ "vis-data": "^8.0.3",
85
+ "vis-network": "^10.0.2",
86
+ "web-tree-sitter": "^0.24.7",
82
87
  "ws": "^8.20.0"
83
88
  },
84
89
  "devDependencies": {
@@ -8,3 +8,7 @@ copyFileSync(
8
8
  join(root, 'node_modules', 'morphdom', 'dist', 'morphdom-umd.min.js'),
9
9
  join(root, 'dist', 'ui', 'morphdom.min.js'),
10
10
  );
11
+ copyFileSync(
12
+ join(root, 'node_modules', 'vis-network', 'standalone', 'umd', 'vis-network.min.js'),
13
+ join(root, 'dist', 'ui', 'vis-network.min.js'),
14
+ );
@@ -0,0 +1,105 @@
1
+ #!/usr/bin/env node
2
+
3
+ // =============================================================================
4
+ // agent-knowledge PreCompact distill hook
5
+ //
6
+ // Dumps the tail of the current session transcript into the knowledge base
7
+ // right before the host compacts the conversation, so cross-machine memory
8
+ // survives the summarization. This is a lightweight heuristic snapshot —
9
+ // the companion `precompact-flush.mjs` hook does a richer summary via the
10
+ // compiled agent-knowledge library when available. Running both is fine;
11
+ // they write to distinct files.
12
+ //
13
+ // Fail-open: any error logs to stderr and emits `{}` so it never blocks.
14
+ // =============================================================================
15
+
16
+ import { readFileSync, writeFileSync, existsSync, mkdirSync } from 'fs';
17
+ import { join } from 'path';
18
+ import { homedir } from 'os';
19
+
20
+ process.on('uncaughtException', (err) => {
21
+ process.stderr.write(`[precompact-distill] fatal: ${err.message}\n`);
22
+ console.log(JSON.stringify({}));
23
+ process.exit(0);
24
+ });
25
+
26
+ const MEMORY_DIR = process.env.KNOWLEDGE_MEMORY_DIR || join(homedir(), 'claude-memory');
27
+
28
+ async function main() {
29
+ let input = '';
30
+ for await (const chunk of process.stdin) input += chunk;
31
+
32
+ let hookData = {};
33
+ try {
34
+ hookData = JSON.parse(input);
35
+ } catch {
36
+ // ignore
37
+ }
38
+
39
+ const transcriptPath = hookData?.transcript_path;
40
+ const sessionId = hookData?.session_id || 'unknown';
41
+ const cwd = hookData?.workspace?.current_dir || process.cwd();
42
+
43
+ if (!transcriptPath || !existsSync(transcriptPath)) {
44
+ console.log(JSON.stringify({}));
45
+ return;
46
+ }
47
+
48
+ let recent = '';
49
+ try {
50
+ const buf = readFileSync(transcriptPath);
51
+ recent = buf.slice(Math.max(0, buf.length - 200_000)).toString('utf8');
52
+ } catch {
53
+ // ignore
54
+ }
55
+
56
+ const lines = recent.split(/\r?\n/).filter(Boolean);
57
+ const userMsgs = [];
58
+ for (const line of lines) {
59
+ try {
60
+ const obj = JSON.parse(line);
61
+ if (obj?.type === 'user' && typeof obj.message?.content === 'string') {
62
+ userMsgs.push(obj.message.content.slice(0, 500));
63
+ }
64
+ } catch {
65
+ // ignore non-JSON lines
66
+ }
67
+ }
68
+ const summary = userMsgs.slice(-10).join('\n---\n');
69
+
70
+ const memDir = join(MEMORY_DIR, 'projects');
71
+ try {
72
+ mkdirSync(memDir, { recursive: true });
73
+ } catch {
74
+ // ignore
75
+ }
76
+
77
+ const slug = cwd.replace(/[^a-zA-Z0-9]+/g, '-').slice(0, 64);
78
+ const file = join(memDir, `precompact-${slug}-${sessionId.slice(0, 8)}.md`);
79
+ const body = `---
80
+ type: precompact-snapshot
81
+ session: ${sessionId}
82
+ cwd: ${cwd}
83
+ created: ${new Date().toISOString()}
84
+ ---
85
+
86
+ # Pre-compact snapshot
87
+
88
+ Last ${userMsgs.length} user prompts (most recent ${Math.min(10, userMsgs.length)}):
89
+
90
+ ${summary}
91
+ `;
92
+
93
+ try {
94
+ writeFileSync(file, body, 'utf8');
95
+ } catch {
96
+ // ignore — don't block on write failure
97
+ }
98
+
99
+ console.log(JSON.stringify({}));
100
+ }
101
+
102
+ main().catch((err) => {
103
+ process.stderr.write(`[precompact-distill] ${err.message}\n`);
104
+ console.log(JSON.stringify({}));
105
+ });