@agentmemory/agentmemory 0.8.11 → 0.8.12

package/AGENTS.md

@@ -111,8 +111,8 @@ Hook scripts in `src/hooks/` are standalone Node.js scripts (no iii-sdk import).
 
 ## Current Stats (v0.8.9)
 
-- 43 MCP tools (8 visible by default, `AGENTMEMORY_TOOLS=all` for all)
-- 103 REST endpoints
+- 44 MCP tools (8 visible by default, `AGENTMEMORY_TOOLS=all` for all)
+- 104 REST endpoints
 - 6 MCP resources, 3 MCP prompts
 - 12 hooks, 4 skills
 - 50+ iii functions

package/README.md

@@ -8,7 +8,7 @@
 </p>
 
 <p align="center">
-  <a href="https://gist.github.com/rohitg00/2067ab416f7bbe447c1977edaaa681e2"><img src="https://img.shields.io/badge/Viral%20GitHub%20Gist-669%20stars%20%2F%2091%20forks-FF6B35?style=for-the-badge&logo=github&logoColor=white&labelColor=1a1a1a" alt="Design doc: 686 stars / 94 forks on the gist" /></a>
+  <a href="https://gist.github.com/rohitg00/2067ab416f7bbe447c1977edaaa681e2"><img src="https://img.shields.io/badge/Viral%20GitHub%20Gist-719%20stars%20%2F%2097%20forks-FF6B35?style=for-the-badge&logo=github&logoColor=white&labelColor=1a1a1a" alt="Design doc: 719 stars / 97 forks on the gist" /></a>
 </p>
 
 <p align="center">
@@ -25,7 +25,7 @@
 <p align="center">
   <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-recall.svg"><img src="assets/tags/stat-recall.svg" alt="95.2% retrieval R@5" height="38" /></picture>
   <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tokens.svg"><img src="assets/tags/stat-tokens.svg" alt="92% fewer tokens" height="38" /></picture>
-  <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tools.svg"><img src="assets/tags/stat-tools.svg" alt="43 MCP tools" height="38" /></picture>
+  <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tools.svg"><img src="assets/tags/stat-tools.svg" alt="44 MCP tools" height="38" /></picture>
   <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-hooks.svg"><img src="assets/tags/stat-hooks.svg" alt="12 auto hooks" height="38" /></picture>
   <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-deps.svg"><img src="assets/tags/stat-deps.svg" alt="0 external DBs" height="38" /></picture>
   <picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/stat-tests.svg"><img src="assets/tags/stat-tests.svg" alt="654 tests passing" height="38" /></picture>
@@ -133,7 +133,7 @@ agentmemory works with any agent that supports hooks, MCP, or REST API. All agen
 <sub>AgentSDKProvider</sub>
 </td>
 <td align="center" width="12.5%">
-<img src="https://img.shields.io/badge/109-endpoints-1f6feb?style=flat-square" alt="REST API" width="48" /><br/>
+<img src="https://img.shields.io/badge/104-endpoints-1f6feb?style=flat-square" alt="REST API" width="48" /><br/>
 <strong>Any agent</strong><br/>
 <sub>REST API</sub>
 </td>
@@ -317,7 +317,7 @@ Implementation details live in `src/cli.ts` (see `runUpgrade` around the `src/cl
 ### Claude Code (one block, paste it)
 
 ```
-Install agentmemory: run `npx @agentmemory/agentmemory` in a separate terminal to start the memory server. Then run `/plugin marketplace add rohitg00/agentmemory` and `/plugin install agentmemory` — the plugin registers all 12 hooks, 4 skills, AND auto-wires the `@agentmemory/mcp` stdio server via its `.mcp.json`, so you get 43 MCP tools (memory_smart_search, memory_save, memory_sessions, memory_governance_delete, etc.) without any extra config step. Verify with `curl http://localhost:3111/agentmemory/health`. The real-time viewer is at http://localhost:3113.
+Install agentmemory: run `npx @agentmemory/agentmemory` in a separate terminal to start the memory server. Then run `/plugin marketplace add rohitg00/agentmemory` and `/plugin install agentmemory` — the plugin registers all 12 hooks, 4 skills, AND auto-wires the `@agentmemory/mcp` stdio server via its `.mcp.json`, so you get 44 MCP tools (memory_smart_search, memory_save, memory_sessions, memory_governance_delete, etc.) without any extra config step. Verify with `curl http://localhost:3111/agentmemory/health`. The real-time viewer is at http://localhost:3113.
 ```
 
 <details>
@@ -576,9 +576,9 @@ npm install @xenova/transformers
 
 <h2 id="mcp-server"><picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/section-mcp.svg"><img src="assets/tags/section-mcp.svg" alt="MCP Server" height="32" /></picture></h2>
 
-43 tools, 6 resources, 3 prompts, and 4 skills — the most comprehensive MCP memory toolkit for any agent.
+44 tools, 6 resources, 3 prompts, and 4 skills — the most comprehensive MCP memory toolkit for any agent.
 
-### 43 Tools
+### 44 Tools
 
 <details>
 <summary>Core tools (always available)</summary>
@@ -586,17 +586,21 @@ npm install @xenova/transformers
 | Tool | Description |
 |------|-------------|
 | `memory_recall` | Search past observations |
+| `memory_compress_file` | Compress markdown files while preserving structure |
 | `memory_save` | Save an insight, decision, or pattern |
+| `memory_patterns` | Detect recurring patterns |
 | `memory_smart_search` | Hybrid semantic + keyword search |
 | `memory_file_history` | Past observations about specific files |
 | `memory_sessions` | List recent sessions |
+| `memory_timeline` | Chronological observations |
 | `memory_profile` | Project profile (concepts, files, patterns) |
 | `memory_export` | Export all memory data |
+| `memory_relations` | Query relationship graph |
 
 </details>
 
 <details>
-<summary>Extended tools (43 total — set AGENTMEMORY_TOOLS=all)</summary>
+<summary>Extended tools (44 total — set AGENTMEMORY_TOOLS=all)</summary>
 
 | Tool | Description |
 |------|-------------|
@@ -772,7 +776,7 @@ Create `~/.agentmemory/.env`:
 # USER_ID=
 # TEAM_MODE=private
 
-# Tool visibility: "core" (7 tools) or "all" (43 tools)
+# Tool visibility: "core" (8 tools) or "all" (44 tools)
 # AGENTMEMORY_TOOLS=core
 ```
 
@@ -780,7 +784,7 @@ Create `~/.agentmemory/.env`:
 
 <h2 id="api"><picture><source media="(prefers-color-scheme: dark)" srcset="assets/tags/light/section-api.svg"><img src="assets/tags/section-api.svg" alt="API" height="32" /></picture></h2>
 
-109 endpoints on port `3111`. The REST API binds to `127.0.0.1` by default. Protected endpoints require `Authorization: Bearer <secret>` when `AGENTMEMORY_SECRET` is set, and mesh sync endpoints require `AGENTMEMORY_SECRET` on both peers.
+104 endpoints on port `3111`. The REST API binds to `127.0.0.1` by default. Protected endpoints require `Authorization: Bearer <secret>` when `AGENTMEMORY_SECRET` is set, and mesh sync endpoints require `AGENTMEMORY_SECRET` on both peers.
 
 <details>
 <summary>Key endpoints</summary>

package/dist/cli.mjs

@@ -286,12 +286,12 @@ async function main() {
 	p.intro("agentmemory");
 	if (skipEngine) {
 		p.log.info("Skipping engine check (--no-engine)");
-		await import("./src-M6V9yZW5.mjs");
+		await import("./src-68MXysnV.mjs");
 		return;
 	}
 	if (await isEngineRunning()) {
 		p.log.success("iii-engine is running");
-		await import("./src-M6V9yZW5.mjs");
+		await import("./src-68MXysnV.mjs");
 		return;
 	}
 	if (!await startEngine()) {
@@ -335,7 +335,7 @@ async function main() {
 		process.exit(1);
 	}
 	s.stop("iii-engine is ready");
-	await import("./src-M6V9yZW5.mjs");
+	await import("./src-68MXysnV.mjs");
 }
 async function runStatus() {
 	const port = getRestPort();
@@ -643,7 +643,7 @@ async function runUpgrade() {
 	].join("\n"), "agentmemory upgrade");
 }
 async function runMcp() {
-	await import("./standalone-XB9gPYmo.mjs");
+	await import("./standalone-c2xiEQJ9.mjs");
 }
 ({
 	status: runStatus,

package/dist/index.mjs

@@ -1,7 +1,7 @@
 #!/usr/bin/env node
 import { TriggerAction, registerWorker } from "iii-sdk";
 import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
-import { dirname, join, resolve, sep } from "node:path";
+import { basename, dirname, extname, join, resolve, sep } from "node:path";
 import { homedir } from "node:os";
 import Anthropic from "@anthropic-ai/sdk";
 import { createHash, createHmac, randomBytes, timingSafeEqual } from "node:crypto";
@@ -10,7 +10,7 @@ import { execFile } from "node:child_process";
 import { promisify } from "node:util";
 import { lookup } from "node:dns/promises";
 import { isIP } from "node:net";
-import { mkdir, writeFile } from "node:fs/promises";
+import { mkdir, readFile, writeFile } from "node:fs/promises";
 import { fileURLToPath } from "node:url";
 import { createServer } from "node:http";
 
@@ -2267,6 +2267,17 @@ function registerSearchFunction(sdk, kv) {
 		}
 		const projectFilter = typeof data.project === "string" && data.project.length > 0 ? data.project : void 0;
 		const cwdFilter = typeof data.cwd === "string" && data.cwd.length > 0 ? data.cwd : void 0;
+		const format = typeof data.format === "string" ? data.format : "full";
+		if (![
+			"full",
+			"compact",
+			"narrative"
+		].includes(format)) throw new Error("mem::search: format must be one of 'full', 'compact', or 'narrative'");
+		let tokenBudget;
+		if (data.token_budget !== void 0) {
+			if (!Number.isInteger(data.token_budget) || data.token_budget < 1) throw new Error("mem::search: token_budget must be a positive integer");
+			tokenBudget = data.token_budget;
+		}
 		if (idx.size === 0) {
 			const count = await rebuildIndex(kv);
 			logger.info("Search index rebuilt", { entries: count });
@@ -2303,13 +2314,81 @@ function registerSearchFunction(sdk, kv) {
 			});
 		}
 		recordAccessBatch(kv, enriched.map((r) => r.observation.id));
+		const estimateTokens = (value) => Math.max(1, Math.ceil(JSON.stringify(value).length / 3));
+		const applyTokenBudget = (items) => {
+			if (!tokenBudget) return {
+				items,
+				used: items.reduce((sum, item) => sum + estimateTokens(item), 0),
+				truncated: false
+			};
+			const selected = [];
+			let used = 0;
+			for (const item of items) {
+				const itemTokens = estimateTokens(item);
+				if (used + itemTokens > tokenBudget) return {
+					items: selected,
+					used,
+					truncated: selected.length < items.length
+				};
+				selected.push(item);
+				used += itemTokens;
+			}
+			return {
+				items: selected,
+				used,
+				truncated: false
+			};
+		};
+		if (format === "compact") {
+			const packed = applyTokenBudget(enriched.map((r) => ({
+				obsId: r.observation.id,
+				sessionId: r.sessionId,
+				title: r.observation.title,
+				type: r.observation.type,
+				score: r.score,
+				timestamp: r.observation.timestamp
+			})));
+			return {
+				format,
+				results: packed.items,
+				tokens_used: packed.used,
+				tokens_budget: tokenBudget,
+				truncated: packed.truncated
+			};
+		}
+		if (format === "narrative") {
+			const packed = applyTokenBudget(enriched.map((r) => ({
+				obsId: r.observation.id,
+				sessionId: r.sessionId,
+				title: r.observation.title,
+				narrative: r.observation.narrative,
+				score: r.score,
+				timestamp: r.observation.timestamp
+			})));
+			const text = packed.items.map((r, index) => `${index + 1}. ${r.title}\n${r.narrative}`).join("\n\n");
+			return {
+				format,
+				results: packed.items,
+				text,
+				tokens_used: packed.used,
+				tokens_budget: tokenBudget,
+				truncated: packed.truncated
+			};
+		}
+		const packed = applyTokenBudget(enriched);
 		logger.info("Search completed", {
 			query,
-			results: enriched.length,
+			results: packed.items.length,
 			hasProjectFilter: !!projectFilter,
 			hasCwdFilter: !!cwdFilter
 		});
-		return { results: enriched };
+		return {
+			format,
+			results: packed.items,
+			tokens_used: packed.used,
+			tokens_budget: tokenBudget,
+			truncated: packed.truncated
+		};
 	});
 }
 
@@ -4398,7 +4477,7 @@ function registerAutoForgetFunction(sdk, kv) {
 
 //#endregion
 //#region src/version.ts
-const VERSION = "0.8.11";
+const VERSION = "0.8.12";
 
 //#endregion
 //#region src/functions/export-import.ts
@@ -4513,7 +4592,8 @@ function registerExportImportFunction(sdk, kv) {
 			"0.8.8",
 			"0.8.9",
 			"0.8.10",
-			"0.8.11"
+			"0.8.11",
+			"0.8.12"
 		]).has(importData.version)) return {
 			success: false,
 			error: `Unsupported export version: ${importData.version}`
@@ -10932,6 +11012,121 @@ function registerRetentionFunctions(sdk, kv) {
 	});
 }
 
+//#endregion
+//#region src/functions/compress-file.ts
+const SENSITIVE_PATH_TERMS = [
+	"secret",
+	"credential",
+	"private_key",
+	".env",
+	"id_rsa",
+	"token"
+];
+const COMPRESS_FILE_SYSTEM_PROMPT = `You compress markdown while preserving structure.
+Rules:
+- Keep all headings exactly as-is.
+- Keep all URLs exactly as-is.
+- Keep all fenced code blocks exactly as-is.
+- Do not remove sections; shorten prose under each section.
+- Output only markdown, no wrappers or explanations.`;
+function stripMarkdownFence(text) {
+	const trimmed = text.trim();
+	const match = trimmed.match(/^```(?:markdown|md)?\s*([\s\S]*?)\s*```$/i);
+	return match ? match[1].trim() : trimmed;
+}
+function extractUrls(text) {
+	return Array.from(new Set(text.match(/https?:\/\/[^\s)]+/g) || []));
+}
+function extractHeadings(text) {
+	return text.split("\n").map((line) => line.trim()).filter((line) => /^#{1,6}\s+/.test(line));
+}
+function extractCodeBlocks(text) {
+	return text.match(/```[\s\S]*?```/g) || [];
+}
+function validateCompression(original, compressed) {
+	const errors = [];
+	const originalHeadings = extractHeadings(original);
+	const compressedHeadings = extractHeadings(compressed);
+	for (const heading of originalHeadings) if (!compressedHeadings.includes(heading)) errors.push(`missing heading: ${heading}`);
+	const originalUrls = extractUrls(original).sort();
+	const compressedUrls = extractUrls(compressed).sort();
+	if (originalUrls.length !== compressedUrls.length) errors.push("url count changed");
+	else for (let i = 0; i < originalUrls.length; i++) if (originalUrls[i] !== compressedUrls[i]) {
+		errors.push("url set changed");
+		break;
+	}
+	const originalBlocks = extractCodeBlocks(original);
+	const compressedBlocks = extractCodeBlocks(compressed);
+	if (originalBlocks.length !== compressedBlocks.length) errors.push("code block count changed");
+	else for (let i = 0; i < originalBlocks.length; i++) if (originalBlocks[i] !== compressedBlocks[i]) {
+		errors.push("code block content changed");
+		break;
+	}
+	return errors;
+}
+function resolveBackupPath(filePath) {
+	const base = basename(filePath, extname(filePath));
+	const name = base.endsWith(".original") ? base : `${base}.original`;
+	return join(dirname(filePath), `${name}.md`);
+}
+function registerCompressFileFunction(sdk, kv, provider) {
+	sdk.registerFunction("mem::compress-file", async (data) => {
+		if (!data?.filePath || typeof data.filePath !== "string") return {
+			success: false,
+			error: "filePath is required"
+		};
+		const absolutePath = resolve(data.filePath);
+		const lowerPath = absolutePath.toLowerCase();
+		if (extname(absolutePath).toLowerCase() !== ".md") return {
+			success: false,
+			error: "filePath must point to a .md file"
+		};
+		if (SENSITIVE_PATH_TERMS.some((term) => lowerPath.includes(term))) return {
+			success: false,
+			error: "refusing to process sensitive-looking path"
+		};
+		let original;
+		try {
+			original = await readFile(absolutePath, "utf-8");
+		} catch {
+			return {
+				success: false,
+				error: "failed to read file"
+			};
+		}
+		if (!original.trim()) return {
+			success: true,
+			skipped: true,
+			reason: "file is empty"
+		};
+		const compressed = stripMarkdownFence(await provider.summarize(COMPRESS_FILE_SYSTEM_PROMPT, `Compress this markdown file while preserving structure and code blocks:\n\n${original}`));
+		const validationErrors = validateCompression(original, compressed);
+		if (validationErrors.length > 0) return {
+			success: false,
+			error: "compression validation failed",
+			details: validationErrors
+		};
+		const backupPath = resolveBackupPath(absolutePath);
+		await writeFile(backupPath, original, "utf-8");
+		await writeFile(absolutePath, compressed, "utf-8");
+		try {
+			await recordAudit(kv, "compress", "mem::compress-file", [], {
+				filePath: absolutePath,
+				backupPath,
+				originalChars: original.length,
+				compressedChars: compressed.length
+			});
+		} catch {}
+		return {
+			success: true,
+			filePath: absolutePath,
+			backupPath,
+			originalChars: original.length,
+			compressedChars: compressed.length
+		};
+	});
+}
+
 //#endregion
 //#region src/health/thresholds.ts
 const DEFAULTS = {
@@ -11312,11 +11507,25 @@ function registerApiTriggers(sdk, kv, secret, metricsStore, provider) {
 			status_code: 400,
 			body: { error: "cwd must be a string" }
 		};
+		if (body.format !== void 0 && (typeof body.format !== "string" || ![
+			"full",
+			"compact",
+			"narrative"
+		].includes(body.format.trim().toLowerCase()))) return {
+			status_code: 400,
+			body: { error: "format must be one of: full, compact, narrative" }
+		};
+		if (body.token_budget !== void 0 && (!Number.isInteger(body.token_budget) || body.token_budget < 1)) return {
+			status_code: 400,
+			body: { error: "token_budget must be a positive integer" }
+		};
 		const payload = {
 			query: body.query.trim(),
 			limit: body.limit,
 			project: body.project,
-			cwd: body.cwd
+			cwd: body.cwd,
+			format: typeof body.format === "string" ? body.format.trim().toLowerCase() : void 0,
+			token_budget: body.token_budget
 		};
 		return {
 			status_code: 200,
@@ -11335,6 +11544,30 @@ function registerApiTriggers(sdk, kv, secret, metricsStore, provider) {
 			middleware_function_ids: ["middleware::api-auth"]
 		}
 	});
+	sdk.registerFunction("api::compress-file", async (req) => {
+		const authErr = checkAuth(req, secret);
+		if (authErr) return authErr;
+		const filePath = asNonEmptyString$1((req.body ?? {}).filePath);
+		if (!filePath) return {
+			status_code: 400,
+			body: { error: "filePath is required and must be a non-empty string" }
+		};
+		return {
+			status_code: 200,
+			body: await sdk.trigger({
+				function_id: "mem::compress-file",
+				payload: { filePath }
+			})
+		};
+	});
+	sdk.registerTrigger({
+		type: "http",
+		function_id: "api::compress-file",
+		config: {
+			api_path: "/agentmemory/compress-file",
+			http_method: "POST"
+		}
+	});
 	sdk.registerFunction("api::session::start", async (req) => {
 		const body = req.body ?? {};
 		const sessionId = asNonEmptyString$1(body.sessionId);
@@ -13799,11 +14032,31 @@ const CORE_TOOLS = [
 				limit: {
 					type: "number",
 					description: "Max results to return (default 10)"
+				},
+				format: {
+					type: "string",
+					description: "Result format: full, compact, or narrative (default full)"
+				},
+				token_budget: {
+					type: "number",
+					description: "Optional token budget to trim returned results"
 				}
 			},
 			required: ["query"]
 		}
 	},
+	{
+		name: "memory_compress_file",
+		description: "Compress a markdown file to reduce token usage while preserving headings, URLs, and code blocks. Creates a .original.md backup before writing.",
+		inputSchema: {
+			type: "object",
+			properties: { filePath: {
+				type: "string",
+				description: "Path to the markdown file to compress"
+			} },
+			required: ["filePath"]
+		}
+	},
 	{
 		name: "memory_save",
 		description: "Explicitly save an important insight, decision, or pattern to long-term memory.",
@@ -14758,13 +15011,46 @@ function registerMcpEndpoints(sdk, kv, secret) {
 						status_code: 400,
 						body: { error: "query is required for memory_recall" }
 					};
+					const format = typeof args.format === "string" ? args.format.trim().toLowerCase() : "full";
+					if (![
+						"full",
+						"compact",
+						"narrative"
+					].includes(format)) return {
+						status_code: 400,
+						body: { error: "format must be one of: full, compact, narrative" }
+					};
+					const tokenBudget = asNumber(args.token_budget);
+					if (args.token_budget !== void 0 && (!Number.isInteger(tokenBudget) || (tokenBudget ?? 0) < 1)) return {
+						status_code: 400,
+						body: { error: "token_budget must be a positive integer" }
+					};
 					const result = await sdk.trigger({
 						function_id: "mem::search",
 						payload: {
 							query: args.query,
-							limit: typeof args.limit === "number" ? args.limit : 10
+							limit: typeof args.limit === "number" ? args.limit : 10,
+							format,
+							token_budget: tokenBudget
 						}
 					});
+					return {
+						status_code: 200,
+						body: { content: [{
+							type: "text",
+							text: format === "narrative" && result && typeof result === "object" && "text" in result && typeof result.text === "string" ? result.text : JSON.stringify(result, null, 2)
+						}] }
+					};
+				}
+				case "memory_compress_file": {
+					if (typeof args.filePath !== "string" || !args.filePath.trim()) return {
+						status_code: 400,
+						body: { error: "filePath is required for memory_compress_file" }
+					};
+					const result = await sdk.trigger({
+						function_id: "mem::compress-file",
+						payload: { filePath: args.filePath.trim() }
+					});
 					return {
 						status_code: 200,
 						body: { content: [{
@@ -16472,6 +16758,7 @@ async function main() {
 	registerQueryExpansionFunction(sdk, provider);
 	registerTemporalGraphFunctions(sdk, kv, provider);
 	registerRetentionFunctions(sdk, kv);
+	registerCompressFileFunction(sdk, kv, provider);
 	console.log(`[agentmemory] v0.6 advanced retrieval: sliding-window, query-expansion, temporal-graph, retention-scoring`);
 	console.log(`[agentmemory] Orchestration layer: actions, frontier, leases, routines, signals, checkpoints, flow-compress, mesh, branch-aware, sentinels, sketches, crystallize, diagnostics, facets`);
 	const snapshotConfig = loadSnapshotConfig();
@@ -16511,7 +16798,7 @@ async function main() {
 		}
 	}
 	console.log(`[agentmemory] Ready. ${embeddingProvider ? "Triple-stream (BM25+Vector+Graph)" : "BM25+Graph"} search active.`);
-	console.log(`[agentmemory] Endpoints: 103 REST + 43 MCP tools + 6 MCP resources + 3 MCP prompts`);
+	console.log(`[agentmemory] Endpoints: 104 REST + 44 MCP tools + 6 MCP resources + 3 MCP prompts`);
 	const viewerServer = startViewerServer(config.restPort + 2, kv, sdk, secret, config.restPort);
 	const autoForgetIntervalMs = parseInt(process.env.AUTO_FORGET_INTERVAL_MS || "3600000", 10);
 	const consolidationIntervalMs = parseInt(process.env.CONSOLIDATION_INTERVAL_MS || "7200000", 10);