@jacklika/dsh-tool-memory-graph 0.1.0-alpha.5 → 0.1.0-alpha.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -6
- package/README.zh.md +7 -6
- package/lib/index.js +269 -54
- package/lib/types/index.d.ts +9 -1
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)
|
|
|
9
9
|
|
|
10
10
|
## Summary
|
|
11
11
|
|
|
12
|
-
`dsh-tool-memory-graph` gives agents read-only graph queries over the Markdown vault used by [`dsh-tool-memory-filesystem`](../tool-memory-filesystem/README.md). The model sees one tool, `wiki_graph`, which returns note nodes and directed edges derived from Obsidian-style `[[link]]` references
|
|
12
|
+
`dsh-tool-memory-graph` gives agents read-only graph queries over the Markdown vault used by [`dsh-tool-memory-filesystem`](../tool-memory-filesystem/README.md). The model sees one tool, `wiki_graph`, which returns note nodes and directed edges derived from two sources: explicit Obsidian-style `[[link]]` references, and keyword co-occurrence between notes (frontmatter tags/title and body terms). It can therefore expose an implicit association such as "B and C both mention `word1`", and that association automatically disappears when one note is rewritten and the shared term is no longer present — or is replaced by a new shared term such as `word2`. The package reuses the filesystem plugin's vault resolution and parsing helpers, so path containment, frontmatter handling, and the `indexHiddenDirs` policy match without a second parser.
|
|
13
13
|
|
|
14
14
|
## Table of Contents
|
|
15
15
|
|
|
@@ -41,17 +41,18 @@ It resolves the same vault root per call as `tool-memory-filesystem`: an explici
|
|
|
41
41
|
| `indexHiddenDirs` | `boolean` | `false` | Index directories whose names start with `.` (`.git` and `node_modules` stay excluded). Enable when `vaultRoot` points at the workspace root so `.dsh/memory/` notes are included. |
|
|
42
42
|
| `maxDepth` | `number` | `1` | Maximum BFS depth when a center id is given. |
|
|
43
43
|
| `maxNodes` | `number` | `200` | Maximum nodes returned in one graph. |
|
|
44
|
+
| `maxEdges` | `number` | `1000` | Maximum edges returned in one graph. |
|
|
44
45
|
|
|
45
46
|
### Tools
|
|
46
47
|
|
|
47
|
-
- `wiki_graph(id?, depth?)` — without `id`, return every note and directed edge in the vault. With `id`, return the reachable subgraph within `depth` hops; edges from boundary nodes are included when both endpoints are selected.
|
|
48
|
+
- `wiki_graph(id?, depth?)` — without `id`, return every note and directed edge in the vault. With `id`, return the reachable subgraph within `depth` hops; edges from boundary nodes are included when both endpoints are selected. Edges carry `type` (`link` or `keyword`), `keywords` (for `keyword` edges), and `weight` (combined term frequency). Output is `{ nodes, edges, truncated, edgesTruncated }`: `truncated` reports the `maxNodes` cut and `edgesTruncated` the `maxEdges` cut, kept separate so a complete node set is not misread as a complete graph.
|
|
48
49
|
|
|
49
50
|
-----
|
|
50
51
|
|
|
51
52
|
<a id="understand-the-implementation"></a>
|
|
52
53
|
## Understand the implementation
|
|
53
54
|
|
|
54
|
-
The package is a single Cordis function plugin that registers `wiki_graph` on `ctx.tools`. Per call it resolves the vault root through the shared `resolveMemoryVaultRoot` helper, lists notes through the shared `listNotePaths` walker, and extracts `[[link]]` targets through `extractLinks`/`resolveLinkTarget`. Targets resolve with the same Obsidian-style tiers as `wiki_read` — exact vault-relative path, then path suffix, then basename anywhere in the vault, with deterministic ambiguity handling. A breadth-first walk
|
|
55
|
+
The package is a single Cordis function plugin that registers `wiki_graph` on `ctx.tools`. Per call it resolves the vault root through the shared `resolveMemoryVaultRoot` helper, lists notes through the shared `listNotePaths` walker, and extracts `[[link]]` targets through `extractLinks`/`resolveLinkTarget`. Targets resolve with the same Obsidian-style tiers as `wiki_read` — exact vault-relative path, then path suffix, then basename anywhere in the vault, with deterministic ambiguity handling. In addition to explicit links, the tool extracts significant terms from each note (frontmatter `tags` ×3, frontmatter `title` ×2, body terms ×1) and creates `keyword` edges between any two notes that share one or more terms. A breadth-first walk with a visited set (so cyclical link graphs terminate) follows both `link` and `keyword` edges to select the node set first; edges are then collected from every selected node so boundary-node outgoing edges are not dropped. When the node set exceeds `maxNodes`, a whole-vault graph keeps the lowest ids in code-unit order (the ordering `tool-memory-filesystem` tie-breaks with) and a subgraph keeps the notes nearest the center — BFS level first, then id — so which notes survive the cap never depends on directory-listing order and a subgraph never drops its own center. Because `wiki_graph` recomputes from disk on every call, rewriting a note removes stale `keyword` edges and creates new ones for the terms that survive or appear in the new text. Edges are capped at `maxEdges` after node selection: explicit `link` edges first, then `keyword` edges by descending weight, with a deterministic id tie-break, so the surviving set never depends on directory-listing order. Requesting a center id that is not an indexed note fails with `note not found`, and an unreadable note is skipped (logged as a warning) instead of failing the call. Output is `{ nodes: [{ id, title }], edges: [{ from, to, type, keywords?, weight? }], truncated: boolean, edgesTruncated: boolean }`, where `truncated` covers the node cap and `edgesTruncated` the edge cap.
|
|
55
56
|
|
|
56
57
|
-----
|
|
57
58
|
|
|
@@ -62,12 +63,12 @@ The package is a single Cordis function plugin that registers `wiki_graph` on `c
|
|
|
62
63
|
|
|
63
64
|
#### What the model sees
|
|
64
65
|
|
|
65
|
-
The model sees the generated `wiki_graph` schema. The description states that the graph is read-only and derived from `[[link]]` references.
|
|
66
|
+
The model sees the generated `wiki_graph` schema. The description states that the graph is read-only and derived from `[[link]]` references plus shared keywords.
|
|
66
67
|
|
|
67
68
|
##### Verbatim description for `wiki_graph`
|
|
68
69
|
|
|
69
70
|
```markdown
|
|
70
|
-
Return the
|
|
71
|
+
Return the link and keyword graph of the wiki vault: note ids, titles, and directed edges. Explicit `[[link]]` edges are type `link`; edges derived from shared significant terms are type `keyword` with `keywords` and `weight`. Without an id it returns the whole vault graph (node-capped); with an id it returns the subgraph reachable from that note within the given depth. Nodes are capped at maxNodes (`truncated`) and edges at maxEdges (`edgesTruncated`).
|
|
71
72
|
```
|
|
72
73
|
|
|
73
74
|
#### Token effect
|
|
@@ -83,7 +84,7 @@ Independent. The plugin only supplies tool results; it does not change the reque
|
|
|
83
84
|
<a id="known-limitations-and-deferred-work"></a>
|
|
84
85
|
|
|
85
86
|
- **No visualization** — the tool returns JSON nodes and edges; rendering (Obsidian graph view, Mermaid) is left to consumers.
|
|
86
|
-
- **Edges are scoped to selected nodes** — only links whose source and target both survive the `maxNodes` cap are returned, so boundary links outside the selected set are omitted.
|
|
87
|
+
- **Edges are scoped to selected nodes** — only links whose source and target both survive the `maxNodes` cap are returned, so boundary links outside the selected set are omitted. A dense vault can then also hit the `maxEdges` cap; `edgesTruncated` reports when the returned edges are only a deterministic slice.
|
|
87
88
|
|
|
88
89
|
<a id="dev-note"></a>
|
|
89
90
|
### Dev Note
|
package/README.zh.md
CHANGED
|
@@ -9,7 +9,7 @@ kind: "package-reference"
|
|
|
9
9
|
|
|
10
10
|
## 概述
|
|
11
11
|
|
|
12
|
-
`dsh-tool-memory-graph` 让 agent 对 [`dsh-tool-memory-filesystem`](../tool-memory-filesystem/README.zh.md) 使用的 Markdown 仓库做只读图查询。模型可见一个工具 `wiki_graph
|
|
12
|
+
`dsh-tool-memory-graph` 让 agent 对 [`dsh-tool-memory-filesystem`](../tool-memory-filesystem/README.zh.md) 使用的 Markdown 仓库做只读图查询。模型可见一个工具 `wiki_graph`,返回两类来源推导的笔记节点和有向边:显式 Obsidian 风格 `[[link]]` 引用,以及笔记之间的关键词共现(frontmatter tags/title 与正文中的显著词)。因此它能暴露类似“B 与 C 都提到 `word1`”的隐式关联,且当某条笔记被重写、共同词消失时该关联自动消失;若重写后双方出现新的共词(如 `word2`),则自动建立以 `word2` 为标签的新关联。该包复用 filesystem 插件的仓库根解析与解析辅助函数,路径约束、frontmatter 处理和 `indexHiddenDirs` 策略完全一致,不引入第二套解析器。
|
|
13
13
|
|
|
14
14
|
## 目录
|
|
15
15
|
|
|
@@ -41,17 +41,18 @@ kind: "package-reference"
|
|
|
41
41
|
| `indexHiddenDirs` | `boolean` | `false` | 是否索引以 `.` 开头的目录(`.git` 与 `node_modules` 始终排除)。当 `vaultRoot` 指向工作区根时开启,以纳入 `.dsh/memory/`。 |
|
|
42
42
|
| `maxDepth` | `number` | `1` | 指定中心笔记时 BFS 的最大深度。 |
|
|
43
43
|
| `maxNodes` | `number` | `200` | 单次图查询返回的最大节点数。 |
|
|
44
|
+
| `maxEdges` | `number` | `1000` | 单次图查询返回的最大边数。 |
|
|
44
45
|
|
|
45
46
|
### 工具
|
|
46
47
|
|
|
47
|
-
- `wiki_graph(id?, depth?)` —— 不传 `id` 时返回全仓库的节点与有向边;传 `id` 时返回 `depth`
|
|
48
|
+
- `wiki_graph(id?, depth?)` —— 不传 `id` 时返回全仓库的节点与有向边;传 `id` 时返回 `depth` 跳内的可达子图,两个端点都被选中时边界节点的出边也会包含。边携带 `type`(`link` 或 `keyword`)、`keywords`(仅 `keyword` 边)和 `weight`(合并词频)。输出为 `{ nodes, edges, truncated, edgesTruncated }`:`truncated` 对应 `maxNodes` 截断,`edgesTruncated` 对应 `maxEdges` 截断,两者分开报告,节点完整不会被误读为整图完整。
|
|
48
49
|
|
|
49
50
|
-----
|
|
50
51
|
|
|
51
52
|
<a id="understand-the-implementation"></a>
|
|
52
53
|
## 实现说明
|
|
53
54
|
|
|
54
|
-
本包是单个 Cordis 函数插件,在 `ctx.tools` 上注册 `wiki_graph`。每次调用通过共享的 `resolveMemoryVaultRoot` 解析仓库根,用 `listNotePaths` 遍历笔记,用 `extractLinks`/`resolveLinkTarget` 提取 `[[link]]` 目标。目标按与 `wiki_read` 相同的 Obsidian 语义解析——先精确仓库相对路径,再路径后缀,最后全仓库 basename
|
|
55
|
+
本包是单个 Cordis 函数插件,在 `ctx.tools` 上注册 `wiki_graph`。每次调用通过共享的 `resolveMemoryVaultRoot` 解析仓库根,用 `listNotePaths` 遍历笔记,用 `extractLinks`/`resolveLinkTarget` 提取 `[[link]]` 目标。目标按与 `wiki_read` 相同的 Obsidian 语义解析——先精确仓库相对路径,再路径后缀,最后全仓库 basename 匹配,多重命中按确定性规则选择。除显式链接外,工具还会从每条笔记提取显著词(frontmatter `tags` ×3、frontmatter `title` ×2、正文词 ×1),并为任意两篇共享一个或多个词的笔记创建 `keyword` 边。广度优先遍历带 visited 集合(循环链接图因此可终止),同时跟随 `link` 边与 `keyword` 边以确定节点集合,再从所有已选节点收集边,因此边界节点的出边不会丢失。节点集合超过 `maxNodes` 时,全仓库图保留码点序最小的若干 id(与 `tool-memory-filesystem` 的并列排序规则一致),子图则保留离中心最近的笔记——先按 BFS 层级、再按 id——因此保留哪些笔记不依赖目录列举顺序,子图也不会丢掉自己的中心节点。由于 `wiki_graph` 每次调用都会从磁盘重新计算,重写一条笔记会移除旧的 `keyword` 边,并为新正文中保留下来的或新出现的词建立新边。边在节点集合确定后按 `maxEdges` 截断:显式 `link` 边优先,其后是 `keyword` 边按 `weight` 降序,并以 id 作确定性并列裁决——保留哪些边同样不依赖目录列举顺序。以非已索引笔记作为中心 id 会以 `note not found` 失败;无法读取的笔记会被跳过(记入 logger 警告),不会让整次调用失败。输出为 `{ nodes: [{ id, title }], edges: [{ from, to, type, keywords?, weight? }], truncated: boolean, edgesTruncated: boolean }`,其中 `truncated` 对应节点上限、`edgesTruncated` 对应边上限。
|
|
55
56
|
|
|
56
57
|
-----
|
|
57
58
|
|
|
@@ -62,12 +63,12 @@ kind: "package-reference"
|
|
|
62
63
|
|
|
63
64
|
#### 模型可见内容
|
|
64
65
|
|
|
65
|
-
模型看到生成的 `wiki_graph` schema。描述声明该图是只读的,由 `[[link]]`
|
|
66
|
+
模型看到生成的 `wiki_graph` schema。描述声明该图是只读的,由 `[[link]]` 引用与共享关键词共同推导。
|
|
66
67
|
|
|
67
68
|
##### `wiki_graph` 的原文描述
|
|
68
69
|
|
|
69
70
|
```markdown
|
|
70
|
-
Return the
|
|
71
|
+
Return the link and keyword graph of the wiki vault: note ids, titles, and directed edges. Explicit `[[link]]` edges are type `link`; edges derived from shared significant terms are type `keyword` with `keywords` and `weight`. Without an id it returns the whole vault graph (node-capped); with an id it returns the subgraph reachable from that note within the given depth. Nodes are capped at maxNodes (`truncated`) and edges at maxEdges (`edgesTruncated`).
|
|
71
72
|
```
|
|
72
73
|
|
|
73
74
|
#### Token 影响
|
|
@@ -83,7 +84,7 @@ Return the Obsidian-style [[link]] graph of the wiki vault: note ids, titles, an
|
|
|
83
84
|
<a id="known-limitations-and-deferred-work"></a>
|
|
84
85
|
|
|
85
86
|
- **无可视化** —— 工具返回 JSON 节点与边;渲染(Obsidian 图谱、Mermaid)由消费方负责。
|
|
86
|
-
- **边限于已选节点** —— 只有两端都在 `maxNodes`
|
|
87
|
+
- **边限于已选节点** —— 只有两端都在 `maxNodes` 截断后保留下来的节点之间的链接才会出现,子图边界外的出边被省略;密集仓库随后可能触及 `maxEdges` 上限,`edgesTruncated` 会表明返回的边只是确定性切片。
|
|
87
88
|
|
|
88
89
|
<a id="dev-note"></a>
|
|
89
90
|
### 开发备注
|
package/lib/index.js
CHANGED
|
@@ -3,6 +3,12 @@
|
|
|
3
3
|
* registers `wiki_graph`, which returns the vault's note/link graph or the
|
|
4
4
|
* subgraph reachable from one note, so an agent can reason about how memory
|
|
5
5
|
* notes connect before reading them.
|
|
6
|
+
*
|
|
7
|
+
* In addition to explicit Obsidian-style `[[link]]` edges, `wiki_graph` now
|
|
8
|
+
* derives keyword edges: two notes share a directed keyword edge for every
|
|
9
|
+
* significant term that appears in both. Rewriting a note therefore
|
|
10
|
+
* automatically removes stale keyword edges and creates new ones for the terms
|
|
11
|
+
* that survive or appear in the new text.
|
|
6
12
|
* @module @jacklika/dsh-tool-memory-graph
|
|
7
13
|
*/
|
|
8
14
|
import { readFile } from 'node:fs/promises';
|
|
@@ -19,6 +25,7 @@ export const Config = z.object({
|
|
|
19
25
|
extensions: z.array(z.string()).default(['.md']),
|
|
20
26
|
maxDepth: z.number().default(1),
|
|
21
27
|
maxNodes: z.number().default(200),
|
|
28
|
+
maxEdges: z.number().default(1000),
|
|
22
29
|
indexHiddenDirs: z.boolean().default(false),
|
|
23
30
|
});
|
|
24
31
|
function assertPositiveInteger(name, value) {
|
|
@@ -26,8 +33,85 @@ function assertPositiveInteger(name, value) {
|
|
|
26
33
|
throw new Error(`tool-memory-graph: ${name} must be a positive integer`);
|
|
27
34
|
}
|
|
28
35
|
}
|
|
29
|
-
|
|
30
|
-
|
|
36
|
+
/**
|
|
37
|
+
* Minimal English stopwords. The list is intentionally short; the model-facing
|
|
38
|
+
* graph is most useful for distinctive project terms, not for hiding grammar.
|
|
39
|
+
*/
|
|
40
|
+
const STOPWORDS = new Set([
|
|
41
|
+
'the', 'a', 'an', 'and', 'or', 'but', 'is', 'are', 'was', 'were', 'be', 'been', 'being',
|
|
42
|
+
'have', 'has', 'had', 'do', 'does', 'did', 'will', 'would', 'shall', 'should', 'may', 'might',
|
|
43
|
+
'can', 'could', 'of', 'in', 'on', 'at', 'to', 'for', 'with', 'by', 'from', 'as', 'it', 'its',
|
|
44
|
+
'this', 'that', 'these', 'those', 'i', 'me', 'my', 'you', 'your', 'we', 'us', 'our', 'he', 'him',
|
|
45
|
+
'his', 'she', 'her', 'they', 'them', 'their', 'not', 'no', 'if', 'then', 'than', 'so', 'very',
|
|
46
|
+
'just', 'only', 'also', 'about', 'into', 'through', 'during', 'before', 'after', 'above', 'below',
|
|
47
|
+
'up', 'down', 'out', 'off', 'over', 'under', 'again', 'further', 'once', 'here', 'there', 'when',
|
|
48
|
+
'where', 'why', 'how', 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other', 'some',
|
|
49
|
+
'such', 'what', 'which', 'who', 'whom', 'whose', 'see', 'one', 'two', 'three', 'four', 'five',
|
|
50
|
+
'get', 'got', 'go', 'went', 'come', 'came', 'take', 'took', 'make', 'made', 'know', 'knew',
|
|
51
|
+
'think', 'thought', 'say', 'said', 'look', 'use', 'used', 'find', 'found', 'give', 'gave',
|
|
52
|
+
'tell', 'told', 'ask', 'asked', 'work', 'worked', 'seem', 'seemed', 'feel', 'felt', 'try',
|
|
53
|
+
'tried', 'leave', 'left', 'call', 'called', 'keep', 'kept', 'let', 'put', 'bring', 'brought',
|
|
54
|
+
'begin', 'began', 'help', 'helped', 'show', 'showed', 'hear', 'heard', 'play', 'played', 'run',
|
|
55
|
+
'ran', 'move', 'moved', 'live', 'lived', 'believe', 'believed', 'hold', 'held', 'must', 'need',
|
|
56
|
+
'needed', 'able', 'due', 'while', 'whereas', 'nevertheless', 'however', 'therefore', 'moreover',
|
|
57
|
+
'furthermore', 'nonetheless', 'otherwise', 'meanwhile', 'besides', 'thus', 'hence', 'accordingly',
|
|
58
|
+
'consequently', 'now', 'later', 'soon', 'early', 'late', 'long', 'much', 'many', 'little', 'less',
|
|
59
|
+
'least', 'well', 'better', 'best', 'bad', 'worse', 'worst', 'good', 'great', 'new', 'old',
|
|
60
|
+
'first', 'last', 'next', 'previous', 'same', 'different', 'own', 'another', 'none', 'either',
|
|
61
|
+
'neither', 'whether', 'nor', 'even', 'still', 'yet', 'already', 'always', 'never', 'often',
|
|
62
|
+
'sometimes', 'usually', 'rarely', 'seldom', 'back', 'already', 'quite', 'rather', 'really',
|
|
63
|
+
'actually', 'certainly', 'definitely', 'probably', 'possibly', 'perhaps', 'maybe', 'likely',
|
|
64
|
+
'sure', 'course', 'absolutely', 'completely', 'entirely', 'totally', 'almost', 'nearly',
|
|
65
|
+
'hardly', 'barely', 'scarcely', 'enough',
|
|
66
|
+
]);
|
|
67
|
+
/** Token pattern: starts with a letter, then letters or digits. Unicode-aware. */
|
|
68
|
+
const TOKEN_RE = /[\p{L}][\p{L}\p{N}]*/gu;
|
|
69
|
+
/**
|
|
70
|
+
* Extract normalised significant terms from Markdown text.
|
|
71
|
+
* - Frontmatter `tags` count 3x (they are explicit metadata).
|
|
72
|
+
* - Frontmatter `title` counts 2x.
|
|
73
|
+
* - Body terms count 1x.
|
|
74
|
+
* - Numbers-only tokens and short/stop tokens are dropped.
|
|
75
|
+
*/
|
|
76
|
+
function extractTerms(text) {
|
|
77
|
+
const { frontmatter, body } = splitFrontmatter(text);
|
|
78
|
+
const terms = new Map();
|
|
79
|
+
const addToken = (raw, multiplier = 1) => {
|
|
80
|
+
const normalized = raw.normalize('NFC').toLowerCase();
|
|
81
|
+
if (normalized.length < 2)
|
|
82
|
+
return;
|
|
83
|
+
if (/^\d+$/.test(normalized))
|
|
84
|
+
return;
|
|
85
|
+
if (STOPWORDS.has(normalized))
|
|
86
|
+
return;
|
|
87
|
+
terms.set(normalized, (terms.get(normalized) ?? 0) + multiplier);
|
|
88
|
+
};
|
|
89
|
+
const tokenize = (source) => {
|
|
90
|
+
const re = new RegExp(TOKEN_RE.source, TOKEN_RE.flags);
|
|
91
|
+
return source.match(re) ?? [];
|
|
92
|
+
};
|
|
93
|
+
if (Array.isArray(frontmatter.tags)) {
|
|
94
|
+
for (const tag of frontmatter.tags) {
|
|
95
|
+
if (typeof tag === 'string') {
|
|
96
|
+
for (const token of tokenize(tag))
|
|
97
|
+
addToken(token, 3);
|
|
98
|
+
}
|
|
99
|
+
}
|
|
100
|
+
}
|
|
101
|
+
if (typeof frontmatter.title === 'string') {
|
|
102
|
+
for (const token of tokenize(frontmatter.title))
|
|
103
|
+
addToken(token, 2);
|
|
104
|
+
}
|
|
105
|
+
// Strip Obsidian `[[link]]` syntax so explicit links do not also create
|
|
106
|
+
// redundant keyword edges; keep markdown link text, not the URL.
|
|
107
|
+
const bodyForTerms = body
|
|
108
|
+
.replace(/\[\[[^\]]*\]\]/g, ' ')
|
|
109
|
+
.replace(/\[([^\]]+)\]\([^)]+\)/g, '$1');
|
|
110
|
+
for (const token of tokenize(bodyForTerms))
|
|
111
|
+
addToken(token, 1);
|
|
112
|
+
return terms;
|
|
113
|
+
}
|
|
114
|
+
function noteTitleFromText(text, id) {
|
|
31
115
|
const { frontmatter, body } = splitFrontmatter(text);
|
|
32
116
|
const explicitTitle = typeof frontmatter.title === 'string' ? frontmatter.title.trim() : '';
|
|
33
117
|
if (explicitTitle !== '')
|
|
@@ -35,88 +119,213 @@ async function noteTitle(absolutePath, id) {
|
|
|
35
119
|
const heading = /^#\s+(.+)$/m.exec(body);
|
|
36
120
|
return heading?.[1]?.trim() ?? id.replace(/\.[^.]+$/, '');
|
|
37
121
|
}
|
|
122
|
+
/**
|
|
123
|
+
* Build undirected keyword edges for a set of notes. Each pair of notes that
|
|
124
|
+
* share one or more terms receives two directed edges (A→B and B→A) carrying
|
|
125
|
+
* the sorted shared keywords and a combined weight.
|
|
126
|
+
*/
|
|
127
|
+
function buildKeywordEdges(noteTerms) {
|
|
128
|
+
const inverted = new Map();
|
|
129
|
+
for (const [id, terms] of noteTerms) {
|
|
130
|
+
for (const term of terms.keys()) {
|
|
131
|
+
const list = inverted.get(term) ?? [];
|
|
132
|
+
list.push(id);
|
|
133
|
+
inverted.set(term, list);
|
|
134
|
+
}
|
|
135
|
+
}
|
|
136
|
+
const pairData = new Map();
|
|
137
|
+
for (const [term, ids] of inverted) {
|
|
138
|
+
if (ids.length < 2)
|
|
139
|
+
continue;
|
|
140
|
+
for (let i = 0; i < ids.length; i += 1) {
|
|
141
|
+
for (let j = i + 1; j < ids.length; j += 1) {
|
|
142
|
+
const a = ids[i];
|
|
143
|
+
const b = ids[j];
|
|
144
|
+
const [first, second] = codeUnitCompare(a, b) <= 0 ? [a, b] : [b, a];
|
|
145
|
+
const weight = (noteTerms.get(a)?.get(term) ?? 0) + (noteTerms.get(b)?.get(term) ?? 0);
|
|
146
|
+
const key = `${first}\x00${second}`;
|
|
147
|
+
const entry = pairData.get(key) ?? { keywords: [], weight: 0 };
|
|
148
|
+
entry.keywords.push(term);
|
|
149
|
+
entry.weight += weight;
|
|
150
|
+
pairData.set(key, entry);
|
|
151
|
+
}
|
|
152
|
+
}
|
|
153
|
+
}
|
|
154
|
+
const edges = [];
|
|
155
|
+
for (const [key, { keywords, weight }] of pairData) {
|
|
156
|
+
const parts = key.split('\x00');
|
|
157
|
+
const from = parts[0];
|
|
158
|
+
const to = parts[1];
|
|
159
|
+
const sortedKeywords = [...keywords].sort(codeUnitCompare);
|
|
160
|
+
edges.push({ from, to, type: 'keyword', keywords: sortedKeywords, weight });
|
|
161
|
+
edges.push({ from: to, to: from, type: 'keyword', keywords: sortedKeywords, weight });
|
|
162
|
+
}
|
|
163
|
+
return edges;
|
|
164
|
+
}
|
|
165
|
+
async function loadVaultNotes(root, resolved, onSkip) {
|
|
166
|
+
const paths = await listNotePaths(root, resolved.extensions, resolved.indexHiddenDirs, onSkip);
|
|
167
|
+
const ids = paths.map(path => vaultRelativeId(root, path));
|
|
168
|
+
const linkResolver = createLinkResolver(ids);
|
|
169
|
+
const parsed = [];
|
|
170
|
+
const parsedById = new Map();
|
|
171
|
+
const skipped = new Map();
|
|
172
|
+
for (const path of paths) {
|
|
173
|
+
const id = vaultRelativeId(root, path);
|
|
174
|
+
let text;
|
|
175
|
+
try {
|
|
176
|
+
text = await readFile(path, 'utf8');
|
|
177
|
+
}
|
|
178
|
+
catch (error) {
|
|
179
|
+
// One unreadable note must not fail the graph for the whole vault; the
|
|
180
|
+
// note is omitted and the loader logs the degradation.
|
|
181
|
+
skipped.set(id, error);
|
|
182
|
+
onSkip?.(path, error);
|
|
183
|
+
continue;
|
|
184
|
+
}
|
|
185
|
+
const note = { id, path, text, terms: extractTerms(text) };
|
|
186
|
+
parsed.push(note);
|
|
187
|
+
parsedById.set(id, note);
|
|
188
|
+
}
|
|
189
|
+
return { parsed, parsedById, linkResolver, skipped };
|
|
190
|
+
}
|
|
191
|
+
/** Build the skip callback that reports unreadable entries to the Loader logger. */
|
|
192
|
+
function skipLogger(ctx, root) {
|
|
193
|
+
return (path, error) => {
|
|
194
|
+
const code = error.code ?? 'unknown';
|
|
195
|
+
ctx.logger?.warn(`tool-memory-graph: skipped unreadable entry ${vaultRelativeId(root, path)} (${code})`);
|
|
196
|
+
};
|
|
197
|
+
}
|
|
198
|
+
/** Validate that at least one note extension is configured. */
|
|
199
|
+
function assertExtensions(extensions) {
|
|
200
|
+
if (extensions.length === 0) {
|
|
201
|
+
throw new Error('tool-memory-graph: extensions must list at least one file extension');
|
|
202
|
+
}
|
|
203
|
+
}
|
|
204
|
+
/**
|
|
205
|
+
* Apply the edge budget deterministically: explicit `link` edges are kept
|
|
206
|
+
* first, then `keyword` edges by descending weight and id order, so the same
|
|
207
|
+
* vault returns the same subset regardless of directory-listing order. Below
|
|
208
|
+
* the budget the edge list is returned untouched.
|
|
209
|
+
*/
|
|
210
|
+
function capEdges(edges, maxEdges) {
|
|
211
|
+
if (edges.length <= maxEdges)
|
|
212
|
+
return { edges, truncated: false };
|
|
213
|
+
const ordered = [...edges].sort((a, b) => {
|
|
214
|
+
const linkFirst = (a.type === 'keyword' ? 1 : 0) - (b.type === 'keyword' ? 1 : 0);
|
|
215
|
+
if (linkFirst !== 0)
|
|
216
|
+
return linkFirst;
|
|
217
|
+
const weightOrder = (b.weight ?? 0) - (a.weight ?? 0);
|
|
218
|
+
if (weightOrder !== 0)
|
|
219
|
+
return weightOrder;
|
|
220
|
+
return codeUnitCompare(a.from, b.from) || codeUnitCompare(a.to, b.to);
|
|
221
|
+
});
|
|
222
|
+
return { edges: ordered.slice(0, maxEdges), truncated: true };
|
|
223
|
+
}
|
|
38
224
|
/**
|
|
39
225
|
* Build the vault-wide link graph: every note is a node; every `[[link]]`
|
|
40
|
-
* that resolves to an existing note is an edge
|
|
226
|
+
* that resolves to an existing note is an edge; every shared significant term
|
|
227
|
+
* between two nodes becomes a directed `keyword` edge in both directions.
|
|
41
228
|
* @param root - vault root.
|
|
42
229
|
* @param resolved - applied plugin configuration.
|
|
43
230
|
* @returns the full graph, node-capped.
|
|
44
231
|
*/
|
|
45
|
-
async function buildGraph(root, resolved) {
|
|
46
|
-
const
|
|
47
|
-
const
|
|
48
|
-
const
|
|
49
|
-
const
|
|
50
|
-
|
|
51
|
-
// a filesystem-dependent subset of the vault.
|
|
52
|
-
const capped = [...entries].sort((a, b) => codeUnitCompare(a[0], b[0])).slice(0, resolved.maxNodes);
|
|
53
|
-
const nodes = [];
|
|
54
|
-
for (const [id, path] of capped)
|
|
55
|
-
nodes.push({ id, title: await noteTitle(path, id) });
|
|
56
|
-
const idSet = new Set(capped.map(([id]) => id));
|
|
232
|
+
async function buildGraph(root, resolved, onSkip) {
|
|
233
|
+
const { parsed, linkResolver } = await loadVaultNotes(root, resolved, onSkip);
|
|
234
|
+
const truncated = parsed.length > resolved.maxNodes;
|
|
235
|
+
const capped = [...parsed].sort((a, b) => codeUnitCompare(a.id, b.id)).slice(0, resolved.maxNodes);
|
|
236
|
+
const idSet = new Set(capped.map(p => p.id));
|
|
237
|
+
const nodes = capped.map(p => ({ id: p.id, title: noteTitleFromText(p.text, p.id) }));
|
|
57
238
|
const edges = [];
|
|
58
|
-
for (const
|
|
59
|
-
const
|
|
60
|
-
for (const link of extractLinks(text)) {
|
|
239
|
+
for (const p of capped) {
|
|
240
|
+
for (const link of extractLinks(p.text)) {
|
|
61
241
|
const targetPath = await resolveLinkTarget(root, resolved.extensions, link, linkResolver);
|
|
62
242
|
if (targetPath === undefined)
|
|
63
243
|
continue;
|
|
64
244
|
const targetId = vaultRelativeId(root, targetPath);
|
|
65
245
|
if (idSet.has(targetId))
|
|
66
|
-
edges.push({ from: id, to: targetId });
|
|
246
|
+
edges.push({ from: p.id, to: targetId });
|
|
67
247
|
}
|
|
68
248
|
}
|
|
69
|
-
|
|
249
|
+
const noteTerms = new Map(capped.map(p => [p.id, p.terms]));
|
|
250
|
+
edges.push(...buildKeywordEdges(noteTerms));
|
|
251
|
+
// Safety: every edge endpoint must be in the returned node set.
|
|
252
|
+
const safeEdges = edges.filter(e => idSet.has(e.from) && idSet.has(e.to));
|
|
253
|
+
const budgeted = capEdges(safeEdges, resolved.maxEdges);
|
|
254
|
+
return { nodes, edges: budgeted.edges, truncated, edgesTruncated: budgeted.truncated };
|
|
70
255
|
}
|
|
71
256
|
/**
|
|
72
257
|
* Collect the subgraph reachable from one note by following `[[link]]`
|
|
73
|
-
* references breadth-first up to `depth`.
|
|
258
|
+
* references and shared-keyword edges breadth-first up to `depth`.
|
|
74
259
|
* @param root - vault root.
|
|
75
260
|
* @param resolved - applied plugin configuration.
|
|
76
261
|
* @param startId - vault-relative id of the center note.
|
|
77
|
-
* @param depth - maximum link hops from the center.
|
|
262
|
+
* @param depth - maximum link/keyword hops from the center.
|
|
78
263
|
* @returns the neighborhood graph, node-capped.
|
|
79
264
|
*/
|
|
80
|
-
async function buildSubgraph(root, resolved, startId, depth) {
|
|
265
|
+
async function buildSubgraph(root, resolved, startId, depth, onSkip) {
|
|
266
|
+
const { parsed, parsedById, linkResolver, skipped } = await loadVaultNotes(root, resolved, onSkip);
|
|
81
267
|
const startPath = await containedPathReal(root, startId);
|
|
82
|
-
const
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
268
|
+
const startNoteId = vaultRelativeId(root, startPath);
|
|
269
|
+
// A missing path, a directory, or a file outside the configured extensions
|
|
270
|
+
// resolves fine but has no indexed note; seeding the BFS with it would crash
|
|
271
|
+
// on the first `parsedById.get(id)!` lookup.
|
|
272
|
+
if (!parsedById.has(startNoteId)) {
|
|
273
|
+
const skipError = skipped.get(startNoteId);
|
|
274
|
+
if (skipError !== undefined) {
|
|
275
|
+
const code = skipError.code ?? 'unknown error';
|
|
276
|
+
throw new Error(`tool-memory-graph: cannot read note ${JSON.stringify(startNoteId)}: ${code}`);
|
|
277
|
+
}
|
|
278
|
+
throw new Error(`tool-memory-graph: note not found: ${startNoteId}`);
|
|
279
|
+
}
|
|
280
|
+
const idSetAll = new Set(parsed.map(p => p.id));
|
|
281
|
+
const adjacency = new Map();
|
|
282
|
+
for (const p of parsed) {
|
|
283
|
+
const neighbors = [];
|
|
284
|
+
for (const link of extractLinks(p.text)) {
|
|
285
|
+
const targetPath = await resolveLinkTarget(root, resolved.extensions, link, linkResolver);
|
|
286
|
+
if (targetPath === undefined)
|
|
287
|
+
continue;
|
|
288
|
+
const targetId = vaultRelativeId(root, targetPath);
|
|
289
|
+
if (idSetAll.has(targetId) && !neighbors.includes(targetId))
|
|
290
|
+
neighbors.push(targetId);
|
|
291
|
+
}
|
|
292
|
+
adjacency.set(p.id, neighbors);
|
|
293
|
+
}
|
|
294
|
+
const noteTerms = new Map(parsed.map(p => [p.id, p.terms]));
|
|
295
|
+
const keywordEdges = buildKeywordEdges(noteTerms);
|
|
296
|
+
for (const { from, to } of keywordEdges) {
|
|
297
|
+
const list = adjacency.get(from) ?? [];
|
|
298
|
+
if (!list.includes(to))
|
|
299
|
+
list.push(to);
|
|
300
|
+
adjacency.set(from, list);
|
|
301
|
+
}
|
|
302
|
+
const visited = new Map([[startNoteId, 0]]);
|
|
303
|
+
let frontier = [startNoteId];
|
|
87
304
|
for (let level = 0; level < depth && frontier.length > 0; level += 1) {
|
|
88
305
|
const next = [];
|
|
89
|
-
for (const
|
|
90
|
-
const
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
continue;
|
|
95
|
-
const toId = vaultRelativeId(root, targetPath);
|
|
96
|
-
if (!visited.has(toId)) {
|
|
97
|
-
visited.set(toId, targetPath);
|
|
98
|
-
levels.set(toId, level + 1);
|
|
99
|
-
next.push(targetPath);
|
|
306
|
+
for (const id of frontier) {
|
|
307
|
+
for (const neighbor of adjacency.get(id) ?? []) {
|
|
308
|
+
if (!visited.has(neighbor)) {
|
|
309
|
+
visited.set(neighbor, level + 1);
|
|
310
|
+
next.push(neighbor);
|
|
100
311
|
}
|
|
101
312
|
}
|
|
102
313
|
}
|
|
103
314
|
frontier = next;
|
|
104
315
|
}
|
|
105
316
|
const truncated = visited.size > resolved.maxNodes;
|
|
106
|
-
// Truncate by distance from the center, then by id: the neighborhood closest
|
|
107
|
-
// to the requested note survives the cap, and which notes those are does not
|
|
108
|
-
// depend on readdir or link order.
|
|
109
317
|
const ids = [...visited.keys()]
|
|
110
|
-
.sort((a, b) => (
|
|
318
|
+
.sort((a, b) => (visited.get(a) ?? 0) - (visited.get(b) ?? 0) || codeUnitCompare(a, b))
|
|
111
319
|
.slice(0, resolved.maxNodes);
|
|
112
320
|
const idSet = new Set(ids);
|
|
321
|
+
const nodes = ids.map(id => {
|
|
322
|
+
const p = parsedById.get(id);
|
|
323
|
+
return { id, title: noteTitleFromText(p.text, id) };
|
|
324
|
+
});
|
|
113
325
|
const edges = [];
|
|
114
|
-
const nodes = [];
|
|
115
326
|
for (const id of ids) {
|
|
116
|
-
const
|
|
117
|
-
|
|
118
|
-
const text = await readFile(path, 'utf8');
|
|
119
|
-
for (const link of extractLinks(text)) {
|
|
327
|
+
const p = parsedById.get(id);
|
|
328
|
+
for (const link of extractLinks(p.text)) {
|
|
120
329
|
const targetPath = await resolveLinkTarget(root, resolved.extensions, link, linkResolver);
|
|
121
330
|
if (targetPath === undefined)
|
|
122
331
|
continue;
|
|
@@ -125,12 +334,15 @@ async function buildSubgraph(root, resolved, startId, depth) {
|
|
|
125
334
|
edges.push({ from: id, to: toId });
|
|
126
335
|
}
|
|
127
336
|
}
|
|
128
|
-
|
|
337
|
+
const filteredKeywordEdges = keywordEdges.filter(e => idSet.has(e.from) && idSet.has(e.to));
|
|
338
|
+
edges.push(...filteredKeywordEdges);
|
|
339
|
+
const budgeted = capEdges(edges, resolved.maxEdges);
|
|
340
|
+
return { nodes, edges: budgeted.edges, truncated, edgesTruncated: budgeted.truncated };
|
|
129
341
|
}
|
|
130
342
|
/**
|
|
131
343
|
* Register the `wiki_graph` tool on `ctx.tools`.
|
|
132
344
|
* @param ctx - registrant context carrying the tool registry.
|
|
133
|
-
* @param config -
|
|
345
|
+
* @param config - plugin configuration.
|
|
134
346
|
*/
|
|
135
347
|
export function apply(ctx, config) {
|
|
136
348
|
const tools = ctx.get('tools');
|
|
@@ -141,9 +353,11 @@ export function apply(ctx, config) {
|
|
|
141
353
|
const resolved = config;
|
|
142
354
|
assertPositiveInteger('maxDepth', resolved.maxDepth);
|
|
143
355
|
assertPositiveInteger('maxNodes', resolved.maxNodes);
|
|
356
|
+
assertPositiveInteger('maxEdges', resolved.maxEdges);
|
|
357
|
+
assertExtensions(resolved.extensions);
|
|
144
358
|
ctx.effect(() => ctx.tools.register(defineTool({
|
|
145
359
|
name: 'wiki_graph',
|
|
146
|
-
description: 'Return the
|
|
360
|
+
description: 'Return the link and keyword graph of the wiki vault: note ids, titles, and directed edges. Explicit `[[link]]` edges are type `link`; edges derived from shared significant terms are type `keyword` with `keywords` and `weight`. Without an id it returns the whole vault graph (node-capped); with an id it returns the subgraph reachable from that note within the given depth. Nodes are capped at maxNodes (`truncated`) and edges at maxEdges (`edgesTruncated`).',
|
|
147
361
|
parameters: {
|
|
148
362
|
id: {
|
|
149
363
|
type: 'string',
|
|
@@ -151,7 +365,7 @@ export function apply(ctx, config) {
|
|
|
151
365
|
},
|
|
152
366
|
depth: {
|
|
153
367
|
type: 'integer',
|
|
154
|
-
description: 'Maximum
|
|
368
|
+
description: 'Maximum link or keyword hops from the center note; defaults to the configured maxDepth.',
|
|
155
369
|
},
|
|
156
370
|
},
|
|
157
371
|
output: {
|
|
@@ -162,9 +376,10 @@ export function apply(ctx, config) {
|
|
|
162
376
|
const vaultRoot = resolveMemoryVaultRoot(resolved.vaultRoot, exec);
|
|
163
377
|
const depth = args.depth ?? resolved.maxDepth;
|
|
164
378
|
assertPositiveInteger('depth', depth);
|
|
379
|
+
const onSkip = skipLogger(ctx, vaultRoot);
|
|
165
380
|
const graph = args.id === undefined
|
|
166
|
-
? await buildGraph(vaultRoot, resolved)
|
|
167
|
-
: await buildSubgraph(vaultRoot, resolved, args.id, depth);
|
|
381
|
+
? await buildGraph(vaultRoot, resolved, onSkip)
|
|
382
|
+
: await buildSubgraph(vaultRoot, resolved, args.id, depth, onSkip);
|
|
168
383
|
return graph;
|
|
169
384
|
},
|
|
170
385
|
})));
|
package/lib/types/index.d.ts
CHANGED
|
@@ -3,6 +3,12 @@
|
|
|
3
3
|
* registers `wiki_graph`, which returns the vault's note/link graph or the
|
|
4
4
|
* subgraph reachable from one note, so an agent can reason about how memory
|
|
5
5
|
* notes connect before reading them.
|
|
6
|
+
*
|
|
7
|
+
* In addition to explicit Obsidian-style `[[link]]` edges, `wiki_graph` now
|
|
8
|
+
* derives keyword edges: two notes share a directed keyword edge for every
|
|
9
|
+
* significant term that appears in both. Rewriting a note therefore
|
|
10
|
+
* automatically removes stale keyword edges and creates new ones for the terms
|
|
11
|
+
* that survive or appear in the new text.
|
|
6
12
|
* @module @jacklika/dsh-tool-memory-graph
|
|
7
13
|
*/
|
|
8
14
|
import type { Context } from '@deepseek-ai/cordis';
|
|
@@ -25,6 +31,8 @@ export interface Config {
|
|
|
25
31
|
maxDepth?: number;
|
|
26
32
|
/** Maximum nodes returned in one graph. */
|
|
27
33
|
maxNodes?: number;
|
|
34
|
+
/** Maximum edges returned in one graph. */
|
|
35
|
+
maxEdges?: number;
|
|
28
36
|
/** Descend into dot-directories besides the fixed exclusions while indexing. */
|
|
29
37
|
indexHiddenDirs?: boolean;
|
|
30
38
|
}
|
|
@@ -33,7 +41,7 @@ export declare const Config: z<Config>;
|
|
|
33
41
|
/**
|
|
34
42
|
* Register the `wiki_graph` tool on `ctx.tools`.
|
|
35
43
|
* @param ctx - registrant context carrying the tool registry.
|
|
36
|
-
* @param config -
|
|
44
|
+
* @param config - plugin configuration.
|
|
37
45
|
*/
|
|
38
46
|
export declare function apply(ctx: Context, config: Config): void;
|
|
39
47
|
//# sourceMappingURL=index.d.ts.map
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@jacklika/dsh-tool-memory-graph",
|
|
3
3
|
"description": "Model-facing link-graph tool over a Markdown memory vault: returns note nodes and [[link]] edges, or the subgraph reachable from one note.",
|
|
4
|
-
"version": "0.1.0-alpha.
|
|
4
|
+
"version": "0.1.0-alpha.7",
|
|
5
5
|
"publishConfig": {
|
|
6
6
|
"access": "public"
|
|
7
7
|
},
|
|
@@ -32,7 +32,7 @@
|
|
|
32
32
|
"license": "MIT",
|
|
33
33
|
"dependencies": {
|
|
34
34
|
"@deepseek-ai/schemastery": "3.18.4",
|
|
35
|
-
"@jacklika/dsh-tool-memory-filesystem": "^0.1.3-alpha.
|
|
35
|
+
"@jacklika/dsh-tool-memory-filesystem": "^0.1.3-alpha.10"
|
|
36
36
|
},
|
|
37
37
|
"peerDependencies": {
|
|
38
38
|
"@deepseek-ai/cordis": "4.0.4",
|