@ngo-a/native-memory-citations 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 NGO-A
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,142 @@
1
+ # Native Memory Citations
2
+
3
+ Native OpenClaw plugin for cited local memory search and retrieval.
4
+ Native means OpenClaw-native plugin, not native system memory.
5
+
6
+ ## Tools
7
+
8
+ - `native_memory_search`: search approved memory roots and return snippets with source paths, line numbers, and file SHA-256 hashes.
9
+ - `native_memory_fetch`: fetch a cited source by `sourceId` or safe path, optionally checking an expected citation hash.
10
+ - `native_memory_answer`: build an extractive answer from cited memory snippets; says when no cited memory is found.
11
+
12
+ ## Default Scope
13
+
14
+ By default, the plugin searches:
15
+
16
+ - `memory/`
17
+ - `MEMORY.md`
18
+ - `USER.md`
19
+ - `IDENTITY.md`
20
+ - `TOOLS.md`
21
+
22
+ Set `sharedMode: true` in plugin config to exclude the private `MEMORY.md` from
23
+ the default root set. Setting `allowedRoots` explicitly overrides the default
24
+ set entirely and supersedes `sharedMode`.
25
+
26
+ Custom `allowedRoots` entries must be workspace-relative visible paths. Empty
27
+ entries, `.`, `..`, paths containing `..`, absolute paths, and hidden path
28
+ segments such as `memory/.dreams` are rejected.
29
+
30
+ ## Build, Generate, Validate
31
+
32
+ The manifest (`openclaw.plugin.json`) is generated from the `defineToolPlugin`
33
+ metadata in `src/index.ts`. Do not hand-edit it; regenerate after changing the
34
+ plugin id, name, description, `configSchema`, or any tool name:
35
+
36
+ ```bash
37
+ npm install
38
+ npm test
39
+ npm run plugin:build
40
+ npm run plugin:validate
41
+ ```
42
+
43
+ In CI, fail on stale generated metadata without rewriting files:
44
+
45
+ ```bash
46
+ npm run plugin:build:check
47
+ npm run plugin:validate
48
+ npm test
49
+ ```
50
+
51
+ ## Config
52
+
53
+ Plugin config is read from the plugin's entry in the OpenClaw Gateway config.
54
+ All keys are optional:
55
+
56
+ ```json
57
+ {
58
+ "workspace": "/home/ad/.openclaw/workspace",
59
+ "allowedRoots": ["memory", "USER.md", "TOOLS.md"],
60
+ "sharedMode": true,
61
+ "maxFileBytes": 1048576
62
+ }
63
+ ```
64
+
65
+ If `workspace` is omitted, the plugin uses `$OPENCLAW_WORKSPACE`, then
66
+ `~/.openclaw/workspace`. There is no hardcoded user-specific default.
67
+
68
+ ## Notes
69
+
70
+ This v1 is intentionally local-file based. It is portable and dependency-light.
71
+ A future version can add vector search while keeping the same public tool names.
72
+ Search is keyword/substring based with an mtime/size line cache, bounded scan
73
+ concurrency, and `AbortSignal` checks during scan.
74
+
75
+ Returned snippets, fetched content, match lines, and extractive answers are
76
+ redacted for common secret patterns such as bearer tokens, API keys, GitHub
77
+ tokens, password/secret/token assignment lines, credential URLs, JWTs, cloud
78
+ keys, and private key blocks. The named pattern list is best-effort labeling,
79
+ not the security boundary; returned text also masks sufficiently long
80
+ high-entropy tokens when no named pattern matches. Redaction is defense-in-depth,
81
+ not the access boundary; the access boundary is allowed roots, hidden-path
82
+ rejection, symlink/realpath checks, file-type filtering, and size limits.
83
+ Redaction does not mutate source files and does not affect citation hashes,
84
+ which are computed from original file text.
85
+
86
+ ## Citation Integrity
87
+
88
+ Search hits include `sha256`, computed from the full text file used for line
89
+ splitting and citation line numbers. Fetch results include the current `sha256`
90
+ for the same full-file content.
91
+
92
+ To detect stale citations, pass the hash from a prior search hit:
93
+
94
+ ```json
95
+ {
96
+ "sourceId": "memory/2026-06-17.md",
97
+ "lineStart": 12,
98
+ "lineEnd": 14,
99
+ "expectedSha256": "..."
100
+ }
101
+ ```
102
+
103
+ If the file changed, fetch still returns the current content for inspection, but
104
+ marks the result with `stale: true` and a `staleMessage` explaining the hash
105
+ mismatch. Because hashes cover the full file, appending to a daily journal marks
106
+ earlier citations stale even when the cited lines themselves are unchanged.
107
+
108
+ ## Install
109
+
110
+ ```bash
111
+ openclaw plugins install ./native-memory-citations # current/private local checkout
112
+ openclaw plugins install clawhub:ngo-a/native-memory-citations # future publish mode
113
+ openclaw plugins install @ngo-a/native-memory-citations # npm publish mode
114
+ ```
115
+
116
+ Reload the Gateway after installing so the plugin host exposes the tools.
117
+
118
+ ## Security Model
119
+
120
+ See [SECURITY.md](./SECURITY.md): `allowedRoots` is trusted operator config;
121
+ a symlink that escapes a root, and any caller-supplied fetch path, is untrusted
122
+ and re-checked with `realpath`; symlinks found while walking directories during
123
+ search are skipped. Fetch also rejects hidden path segments, non-text files, and
124
+ files larger than `maxFileBytes`. Citation hashes let callers detect when a
125
+ previous path-and-line citation may now point at changed content. Returned text
126
+ is redacted before it leaves the plugin; named secret patterns provide nicer
127
+ labels, while the high-entropy backstop handles unknown token formats. Redaction
128
+ is not an authorization or access-control boundary.
129
+
130
+ ## Publish
131
+
132
+ Deferred until distribution mode. When publishing: remove `private`, add the
133
+ `files` allowlist and `prepublishOnly` script, then
134
+ `clawhub package publish ngo-a/native-memory-citations` (or `npm publish`).
135
+
136
+ ## License
137
+
138
+ [MIT](./LICENSE)
139
+
140
+ ## Changelog
141
+
142
+ See [CHANGELOG.md](./CHANGELOG.md).
package/dist/core.d.ts ADDED
@@ -0,0 +1,65 @@
1
+ export type PluginConfig = {
2
+ workspace?: string;
3
+ allowedRoots?: string[];
4
+ sharedMode?: boolean;
5
+ maxFileBytes?: number;
6
+ };
7
+ export type SearchHit = {
8
+ sourceId: string;
9
+ path: string;
10
+ lineStart: number;
11
+ lineEnd: number;
12
+ score: number;
13
+ snippet: string;
14
+ matchLine: number;
15
+ matchText: string;
16
+ sha256: string;
17
+ };
18
+ export type FetchResult = {
19
+ sourceId: string;
20
+ path: string;
21
+ lineStart: number;
22
+ lineEnd: number;
23
+ citation: string;
24
+ content: string;
25
+ sha256: string;
26
+ stale?: boolean;
27
+ staleMessage?: string;
28
+ };
29
+ export type AnswerResult = {
30
+ answer: string;
31
+ citations: SearchHit[];
32
+ known: boolean;
33
+ };
34
+ export type MemoryLogger = {
35
+ debug?: (message: string) => void;
36
+ info?: (message: string) => void;
37
+ warn?: (message: string) => void;
38
+ error?: (message: string) => void;
39
+ };
40
+ export declare function workspaceFromConfig(config?: PluginConfig): string;
41
+ export declare function allowedRoots(config?: PluginConfig): string[];
42
+ export declare function toSafePath(config: PluginConfig, requested: string): Promise<string>;
43
+ export declare function sourceIdForPath(config: PluginConfig, absolutePath: string): string;
44
+ export declare function pathForSourceId(config: PluginConfig, sourceId: string): Promise<string>;
45
+ export declare function searchMemory(query: string, options?: {
46
+ limit?: number;
47
+ contextLines?: number;
48
+ config?: PluginConfig;
49
+ signal?: AbortSignal;
50
+ logger?: MemoryLogger;
51
+ }): Promise<SearchHit[]>;
52
+ export declare function fetchMemorySource(input: {
53
+ sourceId?: string;
54
+ filePath?: string;
55
+ lineStart?: number;
56
+ lineEnd?: number;
57
+ maxChars?: number;
58
+ expectedSha256?: string;
59
+ }, config?: PluginConfig): Promise<FetchResult>;
60
+ export declare function answerFromMemory(query: string, options?: {
61
+ limit?: number;
62
+ config?: PluginConfig;
63
+ signal?: AbortSignal;
64
+ logger?: MemoryLogger;
65
+ }): Promise<AnswerResult>;
package/dist/core.js ADDED
@@ -0,0 +1,545 @@
1
+ import { readdir, readFile, realpath, stat } from "node:fs/promises";
2
+ import { createHash } from "node:crypto";
3
+ import os from "node:os";
4
+ import path from "node:path";
5
+ const DEFAULT_PRIVATE_ROOTS = ["memory", "MEMORY.md", "USER.md", "IDENTITY.md", "TOOLS.md"];
6
+ const DEFAULT_SHARED_ROOTS = ["memory", "USER.md", "IDENTITY.md", "TOOLS.md"];
7
+ const TEXT_EXTENSIONS = new Set([".md", ".txt", ".json", ".jsonl", ".yaml", ".yml"]);
8
+ const ANSWER_MIN_SCORE = 3;
9
+ const ANSWER_MIN_TERM_RATIO = 0.5;
10
+ const MAX_REGION_LINES = 25;
11
+ const FILE_CACHE_MAX = 512;
12
+ const MAX_LINE_CHARS = 2000;
13
+ const MAX_SNIPPET_CHARS = 4000;
14
+ const DEFAULT_FETCH_CHARS = 8000;
15
+ const MAX_FETCH_CHARS = 20000;
16
+ const DEFAULT_MAX_FILE_BYTES = 1024 * 1024;
17
+ const SCAN_CONCURRENCY = 8;
18
+ const HIGH_ENTROPY_TOKEN_MIN_LENGTH = 24;
19
+ const HIGH_ENTROPY_TOKEN_MIN_BITS_PER_CHAR = 4;
20
+ const STOPWORDS = new Set([
21
+ "the",
22
+ "a",
23
+ "an",
24
+ "and",
25
+ "or",
26
+ "but",
27
+ "if",
28
+ "of",
29
+ "to",
30
+ "in",
31
+ "on",
32
+ "for",
33
+ "from",
34
+ "is",
35
+ "it",
36
+ "its",
37
+ "at",
38
+ "by",
39
+ "as",
40
+ "with",
41
+ "this",
42
+ "that",
43
+ "what",
44
+ "which",
45
+ "who",
46
+ "how",
47
+ "why",
48
+ "when",
49
+ "where",
50
+ "do",
51
+ "does",
52
+ "did",
53
+ "can",
54
+ "could",
55
+ "should",
56
+ "would",
57
+ "will",
58
+ "you",
59
+ "your",
60
+ "i",
61
+ "my",
62
+ "me",
63
+ "we",
64
+ "our",
65
+ "are",
66
+ "was",
67
+ "were",
68
+ "be",
69
+ "been",
70
+ ]);
71
+ const fileCache = new Map();
72
+ function clampInt(value, fallback, min, max) {
73
+ const n = Math.floor(value ?? fallback);
74
+ if (!Number.isFinite(n)) {
75
+ return fallback;
76
+ }
77
+ return Math.min(max, Math.max(min, n));
78
+ }
79
+ function maxFileBytes(config = {}) {
80
+ const n = Math.floor(config.maxFileBytes ?? DEFAULT_MAX_FILE_BYTES);
81
+ if (!Number.isFinite(n)) {
82
+ return DEFAULT_MAX_FILE_BYTES;
83
+ }
84
+ return Math.max(1024, n);
85
+ }
86
+ function defaultWorkspace() {
87
+ const fromEnv = process.env.OPENCLAW_WORKSPACE?.trim();
88
+ if (fromEnv) {
89
+ return fromEnv;
90
+ }
91
+ return path.join(os.homedir(), ".openclaw", "workspace");
92
+ }
93
+ export function workspaceFromConfig(config = {}) {
94
+ return path.resolve(config.workspace?.trim() || defaultWorkspace());
95
+ }
96
+ export function allowedRoots(config = {}) {
97
+ const roots = config.allowedRoots && config.allowedRoots.length > 0
98
+ ? config.allowedRoots
99
+ : config.sharedMode
100
+ ? DEFAULT_SHARED_ROOTS
101
+ : DEFAULT_PRIVATE_ROOTS;
102
+ const workspace = workspaceFromConfig(config);
103
+ return roots.map((root) => {
104
+ const trimmed = root.trim();
105
+ const segments = trimmed.split(/[\\/]+/g);
106
+ if (!trimmed
107
+ || trimmed === "."
108
+ || trimmed === ".."
109
+ || path.isAbsolute(trimmed)
110
+ || segments.some((segment) => segment === "..")
111
+ || segments.some((segment) => segment.startsWith("."))) {
112
+ throw new Error(`Invalid allowedRoots entry: ${root}`);
113
+ }
114
+ return path.resolve(workspace, trimmed);
115
+ });
116
+ }
117
+ function within(target, roots) {
118
+ return roots.some((root) => target === root || target.startsWith(`${root}${path.sep}`));
119
+ }
120
+ async function realpathOrSelf(p) {
121
+ return realpath(p).catch(() => p);
122
+ }
123
+ export async function toSafePath(config, requested) {
124
+ const workspace = workspaceFromConfig(config);
125
+ const resolved = path.resolve(workspace, requested);
126
+ const roots = allowedRoots(config);
127
+ if (!within(resolved, roots)) {
128
+ throw new Error(`Path is outside allowed memory roots: ${requested}`);
129
+ }
130
+ const realTarget = await realpathOrSelf(resolved);
131
+ const realRoots = await Promise.all(roots.map(realpathOrSelf));
132
+ if (!within(realTarget, realRoots)) {
133
+ throw new Error(`Path resolves via symlink outside allowed memory roots: ${requested}`);
134
+ }
135
+ return resolved;
136
+ }
137
+ export function sourceIdForPath(config, absolutePath) {
138
+ return path.relative(workspaceFromConfig(config), absolutePath).split(path.sep).join("/");
139
+ }
140
+ export async function pathForSourceId(config, sourceId) {
141
+ return toSafePath(config, sourceId);
142
+ }
143
+ function hasHiddenPathSegment(sourceId) {
144
+ return sourceId.split("/").some((segment) => segment.startsWith("."));
145
+ }
146
+ function isTextFile(file) {
147
+ return TEXT_EXTENSIONS.has(path.extname(file));
148
+ }
149
+ function sha256Text(text) {
150
+ return createHash("sha256").update(text, "utf8").digest("hex");
151
+ }
152
+ function isBeginMarker(line) {
153
+ return /^\s*-----BEGIN [A-Z0-9 ]*PRIVATE KEY(?: BLOCK)?-----\s*$/.test(line);
154
+ }
155
+ function isEndMarker(line) {
156
+ return /^\s*-----END [A-Z0-9 ]*PRIVATE KEY(?: BLOCK)?-----\s*$/.test(line);
157
+ }
158
+ function shannonEntropy(value) {
159
+ const counts = new Map();
160
+ for (const char of value) {
161
+ counts.set(char, (counts.get(char) ?? 0) + 1);
162
+ }
163
+ let entropy = 0;
164
+ for (const count of counts.values()) {
165
+ const p = count / value.length;
166
+ entropy -= p * Math.log2(p);
167
+ }
168
+ return entropy;
169
+ }
170
+ function redactHighEntropyTokens(line) {
171
+ return line.replace(/[A-Za-z0-9_+=-]{24,}/g, (token) => {
172
+ if (token.length >= HIGH_ENTROPY_TOKEN_MIN_LENGTH
173
+ && shannonEntropy(token) >= HIGH_ENTROPY_TOKEN_MIN_BITS_PER_CHAR) {
174
+ return "[REDACTED_HIGH_ENTROPY]";
175
+ }
176
+ return token;
177
+ });
178
+ }
179
+ function redactSingleLineSecrets(line) {
180
+ return redactHighEntropyTokens(line
181
+ .replace(/\bBearer\s+[A-Za-z0-9._~+/=-]{12,}/gi, "Bearer [REDACTED]")
182
+ .replace(/\b(?:sk-proj-|sk-)[A-Za-z0-9_-]{16,}\b/g, "[REDACTED_OPENAI_KEY]")
183
+ .replace(/\bgh[psuor]_[A-Za-z0-9_]{20,}\b/g, "[REDACTED_GITHUB_TOKEN]")
184
+ .replace(/\bgithub_pat_[A-Za-z0-9_]{20,}\b/g, "[REDACTED_GITHUB_TOKEN]")
185
+ .replace(/\b(AccountKey|SharedAccessKey)=[^;\s]+/gi, "$1=[REDACTED]")
186
+ .replace(/\bsig=[^&\s]+/gi, "sig=[REDACTED]")
187
+ .replace(/\b(?:AKIA|ASIA)[A-Z0-9]{16}\b/g, "[REDACTED_AWS_KEY_ID]")
188
+ .replace(/\bxox[baprs]-[A-Za-z0-9-]{10,}\b/g, "[REDACTED_SLACK_TOKEN]")
189
+ .replace(/\bAIza[0-9A-Za-z_-]{35}\b/g, "[REDACTED_GOOGLE_KEY]")
190
+ .replace(/\beyJ[A-Za-z0-9_-]+\.eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+/g, "[REDACTED_JWT]")
191
+ .replace(/\b([a-z][a-z0-9+.-]*:\/\/[^\s:@/]+):[^\s:@/]+@/gi, "$1:[REDACTED]@")
192
+ .replace(/\b([A-Za-z0-9_.-]*(?:API[_-]?KEY|SECRET|TOKEN|PASSWORD|CLIENT[_-]?SECRET)[A-Za-z0-9_.-]*\s*[:=]\s*)([^\s"'`]+|"[^"\n]+"|'[^'\n]+')/gi, "$1[REDACTED]"));
193
+ }
194
+ function buildRedactedLines(rawLines) {
195
+ const mask = new Array(rawLines.length).fill(false);
196
+ let open = -1;
197
+ for (let i = 0; i < rawLines.length; i += 1) {
198
+ if (isBeginMarker(rawLines[i] ?? "")) {
199
+ open = i;
200
+ mask[i] = true;
201
+ continue;
202
+ }
203
+ if (open >= 0) {
204
+ mask[i] = true;
205
+ if (isEndMarker(rawLines[i] ?? "")) {
206
+ open = -1;
207
+ }
208
+ }
209
+ }
210
+ for (let j = 0; j < rawLines.length; j += 1) {
211
+ if (isEndMarker(rawLines[j] ?? "") && !mask[j]) {
212
+ let k = j;
213
+ while (k >= 0 && (rawLines[k] ?? "").trim() !== "") {
214
+ mask[k] = true;
215
+ k -= 1;
216
+ }
217
+ }
218
+ }
219
+ return rawLines.map((line, index) => mask[index] ? "[REDACTED_PRIVATE_KEY]" : redactSingleLineSecrets(line));
220
+ }
221
+ function publicHit(hit) {
222
+ return {
223
+ sourceId: hit.sourceId,
224
+ path: hit.path,
225
+ lineStart: hit.lineStart,
226
+ lineEnd: hit.lineEnd,
227
+ score: hit.score,
228
+ snippet: hit.snippet,
229
+ matchLine: hit.matchLine,
230
+ matchText: hit.matchText,
231
+ sha256: hit.sha256,
232
+ };
233
+ }
234
+ async function collectFiles(root, logger) {
235
+ const info = await stat(root).catch(() => null);
236
+ if (!info) {
237
+ return [];
238
+ }
239
+ if (info.isFile()) {
240
+ return TEXT_EXTENSIONS.has(path.extname(root)) ? [root] : [];
241
+ }
242
+ if (!info.isDirectory()) {
243
+ return [];
244
+ }
245
+ const entries = await readdir(root, { withFileTypes: true });
246
+ const files = [];
247
+ for (const entry of entries) {
248
+ if (entry.name.startsWith(".") || entry.name === "node_modules") {
249
+ continue;
250
+ }
251
+ const child = path.join(root, entry.name);
252
+ if (entry.isDirectory()) {
253
+ files.push(...await collectFiles(child, logger));
254
+ }
255
+ else if (entry.isFile() && TEXT_EXTENSIONS.has(path.extname(entry.name))) {
256
+ files.push(child);
257
+ }
258
+ else if (entry.isSymbolicLink()) {
259
+ logger?.warn?.(`native-memory-citations: skipped symlink during search: ${child}`);
260
+ }
261
+ }
262
+ return files;
263
+ }
264
+ function terms(query) {
265
+ return Array.from(new Set(query
266
+ .toLowerCase()
267
+ .split(/[^a-z0-9_@.+-]+/g)
268
+ .map((term) => term.trim())
269
+ .filter((term) => term.length >= 2 && !STOPWORDS.has(term))));
270
+ }
271
+ function escapeRegExp(s) {
272
+ return s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
273
+ }
274
+ function buildMatchers(queryTerms) {
275
+ return queryTerms.map((term) => {
276
+ const weight = term.length >= 5 ? 3 : 1;
277
+ if (term.length >= 4) {
278
+ return { test: (lower) => lower.includes(term), weight };
279
+ }
280
+ const re = new RegExp(`(?:^|[^a-z0-9])${escapeRegExp(term)}(?:[^a-z0-9]|$)`);
281
+ return { test: (lower) => re.test(lower), weight };
282
+ });
283
+ }
284
+ function scoreLine(line, matchers) {
285
+ const lower = line.toLowerCase();
286
+ let score = 0;
287
+ for (const matcher of matchers) {
288
+ if (matcher.test(lower)) {
289
+ score += matcher.weight;
290
+ }
291
+ }
292
+ return score;
293
+ }
294
+ async function loadFile(file, maxFileBytes, logger) {
295
+ const info = await stat(file).catch(() => null);
296
+ if (!info || !info.isFile()) {
297
+ fileCache.delete(file);
298
+ return null;
299
+ }
300
+ if (info.size > maxFileBytes) {
301
+ fileCache.delete(file);
302
+ logger?.warn?.(`native-memory-citations: skipped oversized file: ${file}`);
303
+ return null;
304
+ }
305
+ const cached = fileCache.get(file);
306
+ if (cached && cached.mtimeMs === info.mtimeMs && cached.size === info.size) {
307
+ fileCache.delete(file);
308
+ fileCache.set(file, cached);
309
+ return cached;
310
+ }
311
+ const rawText = await readFile(file, "utf8").catch(() => "");
312
+ if (!rawText.trim()) {
313
+ fileCache.delete(file);
314
+ return null;
315
+ }
316
+ const rawLines = rawText.split(/\r?\n/g);
317
+ const loaded = {
318
+ mtimeMs: info.mtimeMs,
319
+ size: info.size,
320
+ rawText,
321
+ rawLines,
322
+ redactedLines: buildRedactedLines(rawLines),
323
+ sha256: sha256Text(rawText),
324
+ };
325
+ fileCache.set(file, loaded);
326
+ if (fileCache.size > FILE_CACHE_MAX) {
327
+ const oldest = fileCache.keys().next().value;
328
+ if (oldest !== undefined) {
329
+ fileCache.delete(oldest);
330
+ }
331
+ }
332
+ return loaded;
333
+ }
334
+ function mergeRegions(matches, contextLines, lineCount) {
335
+ const regions = [];
336
+ let current = null;
337
+ for (const match of matches) {
338
+ const start = Math.max(0, match.index - contextLines);
339
+ const end = Math.min(lineCount - 1, match.index + contextLines);
340
+ const currentLineCount = current ? current.end - current.start + 1 : 0;
341
+ if (current && start <= current.end + 1 && currentLineCount < MAX_REGION_LINES) {
342
+ current.end = Math.max(current.end, end);
343
+ current.score += match.score;
344
+ if (match.score > current.anchorScore) {
345
+ current.anchorScore = match.score;
346
+ current.anchorIndex = match.index;
347
+ }
348
+ }
349
+ else {
350
+ if (current) {
351
+ regions.push(current);
352
+ }
353
+ current = { start, end, score: match.score, anchorIndex: match.index, anchorScore: match.score };
354
+ }
355
+ }
356
+ if (current) {
357
+ regions.push(current);
358
+ }
359
+ return regions;
360
+ }
361
+ async function searchMemoryInternal(query, options = {}) {
362
+ const config = options.config ?? {};
363
+ const queryTerms = terms(query);
364
+ options.signal?.throwIfAborted();
365
+ const matchers = buildMatchers(queryTerms);
366
+ if (matchers.length === 0) {
367
+ return [];
368
+ }
369
+ const limit = clampInt(options.limit, 8, 1, 50);
370
+ const contextLines = clampInt(options.contextLines, 2, 0, 8);
371
+ const fileSizeLimit = maxFileBytes(config);
372
+ const files = (await Promise.all(allowedRoots(config).map((root) => collectFiles(root, options.logger)))).flat();
373
+ const hits = [];
374
+ for (let i = 0; i < files.length; i += SCAN_CONCURRENCY) {
375
+ options.signal?.throwIfAborted();
376
+ const batch = files.slice(i, i + SCAN_CONCURRENCY);
377
+ const batchHits = await Promise.all(batch.map(async (file) => {
378
+ options.signal?.throwIfAborted();
379
+ const loaded = await loadFile(file, fileSizeLimit, options.logger);
380
+ if (!loaded) {
381
+ return [];
382
+ }
383
+ const { rawLines, redactedLines, sha256 } = loaded;
384
+ const matches = [];
385
+ for (let index = 0; index < rawLines.length; index += 1) {
386
+ if (index % 512 === 0) {
387
+ options.signal?.throwIfAborted();
388
+ }
389
+ const score = scoreLine(rawLines[index] ?? "", matchers);
390
+ if (score > 0) {
391
+ matches.push({ index, score });
392
+ }
393
+ }
394
+ if (matches.length === 0) {
395
+ return [];
396
+ }
397
+ const sourceId = sourceIdForPath(config, file);
398
+ return mergeRegions(matches, contextLines, rawLines.length).map((region) => {
399
+ const rawSnippet = rawLines.slice(region.start, region.end + 1).join("\n").slice(0, MAX_SNIPPET_CHARS).trim();
400
+ const redactedSnippet = redactedLines
401
+ .slice(region.start, region.end + 1)
402
+ .join("\n")
403
+ .slice(0, MAX_SNIPPET_CHARS)
404
+ .trim();
405
+ const distinctTerms = matchedTermCount(rawSnippet, matchers);
406
+ const rawMatchText = (rawLines[region.anchorIndex] ?? "").slice(0, MAX_LINE_CHARS).trim();
407
+ return {
408
+ sourceId,
409
+ path: sourceId,
410
+ lineStart: region.start + 1,
411
+ lineEnd: region.end + 1,
412
+ score: distinctTerms * 100 + Math.min(region.score, 50),
413
+ snippet: redactedSnippet,
414
+ matchLine: region.anchorIndex + 1,
415
+ matchText: (redactedLines[region.anchorIndex] ?? "").slice(0, MAX_LINE_CHARS).trim(),
416
+ sha256,
417
+ rawSnippet,
418
+ rawMatchText,
419
+ };
420
+ });
421
+ }));
422
+ for (const fileHits of batchHits) {
423
+ hits.push(...fileHits);
424
+ }
425
+ }
426
+ const sortedHits = hits
427
+ .sort((a, b) => b.score - a.score || a.path.localeCompare(b.path) || a.lineStart - b.lineStart)
428
+ .slice(0, limit);
429
+ options.logger?.debug?.(`native-memory-citations: scanned ${files.length} files, returned ${sortedHits.length} hits`);
430
+ return sortedHits;
431
+ }
432
+ export async function searchMemory(query, options = {}) {
433
+ return (await searchMemoryInternal(query, options)).map(publicHit);
434
+ }
435
+ export async function fetchMemorySource(input, config = {}) {
436
+ const requested = input.sourceId || input.filePath;
437
+ if (!requested) {
438
+ throw new Error("sourceId or filePath is required");
439
+ }
440
+ const file = await toSafePath(config, requested);
441
+ const sourceId = sourceIdForPath(config, file);
442
+ if (hasHiddenPathSegment(sourceId)) {
443
+ throw new Error(`Path includes a hidden memory segment: ${sourceId}`);
444
+ }
445
+ if (!isTextFile(file)) {
446
+ throw new Error(`Path is not an approved text memory file: ${sourceId}`);
447
+ }
448
+ const info = await stat(file).catch(() => null);
449
+ const fileSizeLimit = maxFileBytes(config);
450
+ if (!info || !info.isFile()) {
451
+ throw new Error(`Path is not a readable memory file: ${sourceId}`);
452
+ }
453
+ if (info.size > fileSizeLimit) {
454
+ throw new Error(`Memory file exceeds maxFileBytes: ${sourceId}`);
455
+ }
456
+ const loaded = await loadFile(file, fileSizeLimit);
457
+ if (!loaded) {
458
+ throw new Error(`Path is not a readable memory file: ${sourceId}`);
459
+ }
460
+ const { rawLines, redactedLines, sha256 } = loaded;
461
+ const lineStart = clampInt(input.lineStart, 1, 1, rawLines.length);
462
+ const lineEnd = clampInt(input.lineEnd, rawLines.length, lineStart, rawLines.length);
463
+ const maxChars = clampInt(input.maxChars, DEFAULT_FETCH_CHARS, 256, MAX_FETCH_CHARS);
464
+ const content = redactedLines.slice(lineStart - 1, lineEnd).join("\n").slice(0, maxChars);
465
+ const expectedSha256 = input.expectedSha256?.trim().toLowerCase();
466
+ const stale = Boolean(expectedSha256 && expectedSha256 !== sha256);
467
+ return {
468
+ sourceId,
469
+ path: sourceId,
470
+ lineStart,
471
+ lineEnd,
472
+ citation: `${sourceId}:${lineStart}`,
473
+ content,
474
+ sha256,
475
+ ...(stale
476
+ ? {
477
+ stale: true,
478
+ staleMessage: `Citation hash mismatch for ${sourceId}: expected ${expectedSha256}, current ${sha256}`,
479
+ }
480
+ : {}),
481
+ };
482
+ }
483
+ function extractSentences(snippet, matchers) {
484
+ return snippet
485
+ .split(/(?<=[.!?])\s+|\n+/g)
486
+ .map((sentence) => sentence.trim())
487
+ .filter(Boolean)
488
+ .map((sentence, index) => ({ sentence, index, score: scoreLine(sentence, matchers) }))
489
+ .filter((item) => item.score > 0)
490
+ .sort((a, b) => b.score - a.score || a.index - b.index)
491
+ .map((item) => item.sentence)
492
+ .slice(0, 2);
493
+ }
494
+ function matchedTermCount(snippet, matchers) {
495
+ const lower = snippet.toLowerCase();
496
+ return matchers.filter((matcher) => matcher.test(lower)).length;
497
+ }
498
+ function requiredMatchedTerms(queryTerms) {
499
+ if (queryTerms.length <= 1) {
500
+ return queryTerms.length;
501
+ }
502
+ return Math.min(queryTerms.length, Math.max(2, Math.ceil(queryTerms.length * ANSWER_MIN_TERM_RATIO)));
503
+ }
504
+ export async function answerFromMemory(query, options = {}) {
505
+ const queryTerms = terms(query);
506
+ const matchers = buildMatchers(queryTerms);
507
+ const hits = await searchMemoryInternal(query, {
508
+ limit: options.limit ?? 6,
509
+ contextLines: 2,
510
+ config: options.config,
511
+ signal: options.signal,
512
+ logger: options.logger,
513
+ });
514
+ const distinctTermsMatched = matchers.filter((matcher) => hits.some((hit) => matcher.test(hit.rawSnippet.toLowerCase()))).length;
515
+ const requiredDistinctTerms = requiredMatchedTerms(queryTerms);
516
+ const topScore = hits[0]?.score ?? 0;
517
+ const supportingHits = hits.filter((hit) => matchedTermCount(hit.rawSnippet, matchers) >= requiredDistinctTerms);
518
+ const hasSupportingHit = supportingHits.length > 0;
519
+ const confident = hits.length > 0
520
+ && topScore >= ANSWER_MIN_SCORE
521
+ && distinctTermsMatched >= requiredDistinctTerms
522
+ && hasSupportingHit;
523
+ if (!confident) {
524
+ return {
525
+ answer: "I did not find a sufficiently specific cited memory source for that.",
526
+ citations: [],
527
+ known: false,
528
+ };
529
+ }
530
+ const points = [];
531
+ for (const hit of supportingHits.slice(0, 1)) {
532
+ const sentences = extractSentences(hit.snippet, matchers);
533
+ const text = hit.matchText || sentences[0] || hit.snippet.split(/\n+/g).find(Boolean) || "";
534
+ if (text.trim()) {
535
+ points.push(`- ${text.trim()} [${hit.path}:${hit.matchLine}]`);
536
+ }
537
+ }
538
+ return {
539
+ answer: points.length > 0
540
+ ? points.join("\n")
541
+ : `Found cited memory, but no concise extractive answer could be formed. Start with ${supportingHits[0]?.path}:${supportingHits[0]?.matchLine ?? supportingHits[0]?.lineStart}.`,
542
+ citations: supportingHits.map(publicHit),
543
+ known: true,
544
+ };
545
+ }
@@ -0,0 +1,2 @@
1
+ declare const _default: import("openclaw/plugin-sdk/tool-plugin").DefinedToolPluginEntry;
2
+ export default _default;
package/dist/index.js ADDED
@@ -0,0 +1,72 @@
1
+ import { Type } from "typebox";
2
+ import { defineToolPlugin } from "openclaw/plugin-sdk/tool-plugin";
3
+ import { answerFromMemory, fetchMemorySource, searchMemory } from "./core.js";
4
+ // Declared config schema. Without this, OpenClaw applies a strict empty-object
5
+ // schema and rejects every config key below. The generated manifest is produced
6
+ // from this by `openclaw plugins build`; do not hand-edit it.
7
+ const configSchema = Type.Object({
8
+ workspace: Type.Optional(Type.String({
9
+ description: "Absolute path to the OpenClaw workspace. Defaults to $OPENCLAW_WORKSPACE or ~/.openclaw/workspace.",
10
+ })),
11
+ allowedRoots: Type.Optional(Type.Array(Type.String(), {
12
+ description: "Workspace-relative memory roots to search. Overrides the built-in default set.",
13
+ })),
14
+ sharedMode: Type.Optional(Type.Boolean({ description: "When true, exclude the private MEMORY.md from the default root set." })),
15
+ maxFileBytes: Type.Optional(Type.Number({
16
+ description: "Per-file size cap in bytes. Files larger than this are skipped. Default 1048576.",
17
+ })),
18
+ }, { additionalProperties: false });
19
+ export default defineToolPlugin({
20
+ id: "native-memory-citations",
21
+ name: "Native Memory Citations",
22
+ description: "Search and fetch local OpenClaw memory with source citations and extractive cited answers.",
23
+ configSchema,
24
+ tools: (tool) => [
25
+ tool({
26
+ name: "native_memory_search",
27
+ label: "Memory Search (cited)",
28
+ description: "Search approved local memory roots and return snippets with source paths and line numbers.",
29
+ parameters: Type.Object({
30
+ query: Type.String({ description: "Search query." }),
31
+ limit: Type.Optional(Type.Number({ description: "Maximum hits to return. Default 8, max 50." })),
32
+ contextLines: Type.Optional(Type.Number({ description: "Context lines around each hit. Default 2, max 8." })),
33
+ }),
34
+ execute: async ({ query, limit, contextLines }, config, context) => {
35
+ context.signal?.throwIfAborted();
36
+ return searchMemory(query, { limit, contextLines, config, signal: context.signal, logger: context.api.logger });
37
+ },
38
+ }),
39
+ tool({
40
+ name: "native_memory_fetch",
41
+ label: "Memory Fetch (cited)",
42
+ description: "Fetch cited local memory content by sourceId/path and optional line range.",
43
+ parameters: Type.Object({
44
+ sourceId: Type.Optional(Type.String({ description: "Source id returned by native_memory_search." })),
45
+ filePath: Type.Optional(Type.String({ description: "Workspace-relative path inside an allowed memory root." })),
46
+ lineStart: Type.Optional(Type.Number({ description: "1-based starting line." })),
47
+ lineEnd: Type.Optional(Type.Number({ description: "1-based ending line." })),
48
+ maxChars: Type.Optional(Type.Number({ description: "Maximum characters returned. Default 8000." })),
49
+ expectedSha256: Type.Optional(Type.String({
50
+ description: "Optional SHA-256 from a prior citation. When it differs from the current file hash, the result is marked stale.",
51
+ })),
52
+ }),
53
+ execute: async (input, config, context) => {
54
+ context.signal?.throwIfAborted();
55
+ return fetchMemorySource(input, config);
56
+ },
57
+ }),
58
+ tool({
59
+ name: "native_memory_answer",
60
+ label: "Memory Answer (cited)",
61
+ description: "Answer from approved local memory using extractive snippets with citations. Says when no cited memory is found.",
62
+ parameters: Type.Object({
63
+ query: Type.String({ description: "Question to answer from local memory." }),
64
+ limit: Type.Optional(Type.Number({ description: "Maximum cited search hits to consider. Default 6." })),
65
+ }),
66
+ execute: async ({ query, limit }, config, context) => {
67
+ context.signal?.throwIfAborted();
68
+ return answerFromMemory(query, { limit, config, signal: context.signal, logger: context.api.logger });
69
+ },
70
+ }),
71
+ ],
72
+ });
@@ -0,0 +1,41 @@
1
+ {
2
+ "id": "native-memory-citations",
3
+ "activation": {
4
+ "onStartup": true
5
+ },
6
+ "name": "Native Memory Citations",
7
+ "description": "Search and fetch local OpenClaw memory with source citations and extractive cited answers.",
8
+ "configSchema": {
9
+ "type": "object",
10
+ "properties": {
11
+ "workspace": {
12
+ "type": "string",
13
+ "description": "Absolute path to the OpenClaw workspace. Defaults to $OPENCLAW_WORKSPACE or ~/.openclaw/workspace."
14
+ },
15
+ "allowedRoots": {
16
+ "type": "array",
17
+ "items": {
18
+ "type": "string"
19
+ },
20
+ "description": "Workspace-relative memory roots to search. Overrides the built-in default set."
21
+ },
22
+ "sharedMode": {
23
+ "type": "boolean",
24
+ "description": "When true, exclude the private MEMORY.md from the default root set."
25
+ },
26
+ "maxFileBytes": {
27
+ "type": "number",
28
+ "description": "Per-file size cap in bytes. Files larger than this are skipped. Default 1048576."
29
+ }
30
+ },
31
+ "additionalProperties": false
32
+ },
33
+ "version": "0.1.0",
34
+ "contracts": {
35
+ "tools": [
36
+ "native_memory_search",
37
+ "native_memory_fetch",
38
+ "native_memory_answer"
39
+ ]
40
+ }
41
+ }
package/package.json ADDED
@@ -0,0 +1,61 @@
1
+ {
2
+ "name": "@ngo-a/native-memory-citations",
3
+ "version": "0.1.0",
4
+ "type": "module",
5
+ "description": "Native OpenClaw plugin for cited local memory search and retrieval.",
6
+ "license": "MIT",
7
+ "publishConfig": {
8
+ "access": "public"
9
+ },
10
+ "author": "NGO-A <ngo-a@users.noreply.github.com>",
11
+ "homepage": "https://github.com/NGO-A/native-memory-citations#readme",
12
+ "repository": {
13
+ "type": "git",
14
+ "url": "git+https://github.com/NGO-A/native-memory-citations.git"
15
+ },
16
+ "bugs": {
17
+ "url": "https://github.com/NGO-A/native-memory-citations/issues"
18
+ },
19
+ "keywords": [
20
+ "openclaw",
21
+ "openclaw-plugin",
22
+ "memory",
23
+ "citations",
24
+ "search",
25
+ "retrieval"
26
+ ],
27
+ "openclaw": {
28
+ "extensions": [
29
+ "./dist/index.js"
30
+ ]
31
+ },
32
+ "files": [
33
+ "dist",
34
+ "openclaw.plugin.json",
35
+ "README.md"
36
+ ],
37
+ "engines": {
38
+ "node": ">=22.19.0"
39
+ },
40
+ "scripts": {
41
+ "clean": "node -e \"require('node:fs').rmSync('dist', { recursive: true, force: true })\"",
42
+ "build": "npm run clean && tsc -p tsconfig.json",
43
+ "prepublishOnly": "npm run build",
44
+ "plugin:build": "npm run build && openclaw plugins build --entry ./dist/index.js",
45
+ "plugin:build:check": "npm run build && openclaw plugins build --entry ./dist/index.js --check",
46
+ "plugin:validate": "npm run plugin:build && openclaw plugins validate --entry ./dist/index.js",
47
+ "test": "vitest run --dir src --no-file-parallelism"
48
+ },
49
+ "dependencies": {
50
+ "typebox": "1.1.39"
51
+ },
52
+ "devDependencies": {
53
+ "@types/node": "^22.19.0",
54
+ "openclaw": "2026.6.8",
55
+ "typescript": "^5.9.0",
56
+ "vitest": "^3.2.0"
57
+ },
58
+ "peerDependencies": {
59
+ "openclaw": ">=2026.5.17"
60
+ }
61
+ }