open-memex 0.8.2 → 0.9.0-alpha.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +21 -5
- package/CHANGELOG.md +10 -0
- package/README.md +46 -13
- package/README.zh-CN.md +25 -11
- package/dist/.build-stamp +1 -1
- package/dist/cli.js +103 -6
- package/dist/config.js +9 -0
- package/dist/doctor.js +4 -2
- package/dist/index.js +6 -1
- package/dist/init.js +13 -3
- package/dist/mcp.js +28 -5
- package/dist/opencode-v2.js +6 -2
- package/dist/plugin-core.js +28 -1
- package/dist/retrieve/inject.js +6 -0
- package/dist/retrieve/search.js +13 -2
- package/dist/review.js +7 -0
- package/dist/store/audit.js +14 -0
- package/dist/store/db.js +19 -0
- package/dist/store/lifecycle.js +8 -0
- package/dist/store/usage.js +151 -0
- package/dist/tools/memory.js +21 -2
- package/dist/tools/ops.js +123 -1
- package/dist/working-state.js +118 -0
- package/docs/CONCEPTS.md +16 -0
- package/docs/CONCEPTS.zh-CN.md +13 -0
- package/docs/V2-DESIGN.md +42 -1039
- package/docs/design/decisions.md +1146 -0
- package/docs/design/knowledge-graph.md +86 -0
- package/package.json +66 -66
- package/scripts/smoke-mcp.ts +67 -4
- package/scripts/smoke-pure.ts +70 -0
- package/scripts/test-full.ts +83 -0
- package/server.json +2 -2
- package/skills/open-memex/SKILL.md +19 -0
- package/src/cli.ts +111 -6
- package/src/config.ts +16 -0
- package/src/doctor.ts +4 -2
- package/src/index.ts +6 -0
- package/src/init.ts +13 -3
- package/src/mcp.ts +41 -4
- package/src/opencode-v2.ts +7 -2
- package/src/plugin-core.ts +32 -1
- package/src/retrieve/inject.ts +7 -0
- package/src/retrieve/search.ts +19 -3
- package/src/review.ts +6 -0
- package/src/store/audit.ts +20 -0
- package/src/store/db.ts +20 -0
- package/src/store/lifecycle.ts +9 -0
- package/src/store/usage.ts +178 -0
- package/src/tools/memory.ts +26 -1
- package/src/tools/ops.ts +158 -1
- package/src/working-state.ts +153 -0
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
## 20. Knowledge Graph & Compiled Wiki (v0.3 candidate)
|
|
2
|
+
|
|
3
|
+
*Status (2026-11-02): 未实现 — v0.3 candidate as a whole. §20.1–20.4, §20.6, §20.8 unbuilt; §20.5/§20.7 are no-change statements (confirmed, nothing to build).*
|
|
4
|
+
> v0.3 candidate (D88). Does not change the frozen v0.2 protocol; the graph/wiki is a derived layer (see §20.2). Sources: Karpathy's *LLM Wiki* (three layers / three operations) and Graphify (a working implementation of the same pattern on codebases, measured on muse-thinktank 2026-10-31: curated graph, 1029 nodes, edges tagged EXTRACTED/INFERRED with confidence). Both point to the same pattern: compile knowledge once, instead of re-deriving it on every query.
|
|
5
|
+
|
|
6
|
+
### 20.1 Why add a graph (motivation)
|
|
7
|
+
|
|
8
|
+
Today open-memex is RAG-style retrieval: a query finds relevant fragments from memory and assembles an answer. Every query re-discovers knowledge from scratch — nothing accumulates. A question that synthesizes five memories forces the system to re-find and re-assemble them every time.
|
|
9
|
+
|
|
10
|
+
The alternative: between memories (raw) and terminal documents (docs / ADRs / AGENTS.md), add a layer of **compiled knowledge** — a knowledge graph plus a set of LLM-maintained wiki pages. When a new memory arrives, the system does not just store it for later retrieval; it reads it, extracts the key information, and merges it into the existing graph and wiki — updating entity and concept pages, flagging where new content contradicts old conclusions, strengthening or challenging emerging syntheses. Knowledge is **compiled once, then kept current**, not re-derived on every query.
|
|
11
|
+
|
|
12
|
+
This follows Design Principle 7 ("Memory is the entrance of knowledge, not its final form"): memory is the entrance, the graph/wiki is the compiled form, terminal documents are the final form. The §3 frontmatter currently has no `related` field (not in v0.2); this section supplies the graph edges — edges live in the graph layer first, and whether `related` enters the frontmatter is a v0.3 protocol decision.
|
|
13
|
+
|
|
14
|
+
### 20.2 Three layers (Karpathy's raw / wiki / schema)
|
|
15
|
+
|
|
16
|
+
| Layer | In open-memex | Owned by | Mutable? |
|
|
17
|
+
|---|---|---|---|
|
|
18
|
+
| **Raw** | The memory files themselves (Markdown, one memory per file) + the docs they point at via `canonical_ref` | User / team | **Immutable** — the graph and wiki only read raw, never modify it; when something is wrong, trace back to the memory and the doc |
|
|
19
|
+
| **Wiki (compiled)** | Knowledge compiled from memories: entity pages, concept pages, comparisons, overviews, syntheses (Markdown files; the queryable graph is the same knowledge in another shape) | Maintained by the LLM, reviewed by humans | Mutable — updated when new memories arrive; it is a **derived layer**, rebuildable from raw if lost (echoes D1: Markdown is the source of truth, indexes are rebuildable) |
|
|
20
|
+
| **Schema (discipline)** | Capture rules + promotion rules + this section's conventions (what to record, how, what enters the graph, who may change it) | Co-written by human + LLM | Evolves — revised with practice, lives in the protocol doc and AGENTS.md |
|
|
21
|
+
|
|
22
|
+
Key point: **the wiki layer is derived, not a second source of truth.** Truth always lives in raw (the memory files). If the wiki is lost, wrong, or stale, it can be rebuilt or corrected from raw. This matches D1 (Markdown is the source of truth) and §12 ("Memory stores summaries and points at docs via `canonical_ref` — never copies them"): the wiki stores compiled summaries and links, not copies of the originals.
|
|
23
|
+
|
|
24
|
+
### 20.3 Graph edges: typed edges with provenance
|
|
25
|
+
|
|
26
|
+
The §3 frontmatter currently has no `related` field (not in v0.2, see the note at the end of §3). This section introduces **typed edges with provenance** in the graph layer (edges are derived and rebuildable; they do not enter the frontmatter — whether `related` enters the frontmatter is a v0.3 protocol decision):
|
|
27
|
+
|
|
28
|
+
- **Edge types**: `relates-to`, `supports`, `contradicts`, `derived-from`, `supersedes` (already exists, see the supersede chain in §3.3).
|
|
29
|
+
- **Every edge carries provenance** (borrowing Graphify's practice): `EXTRACTED` (explicit in the source — e.g. memory A explicitly says "see memory B", or two memories share a `canonical_ref`) or `INFERRED` (system-inferred — e.g. similar content, shared tag/path), with a confidence (`confidence: high | medium | low`, reusing the confidence semantics already in §3).
|
|
30
|
+
- **Inferred edges are always marked**: in retrieval results they must be distinguishable (e.g. the recall receipt names the lane + edge type); an inferred link must never look like a fact that was in the source.
|
|
31
|
+
|
|
32
|
+
Why split EXTRACTED/INFERRED: this extends the trust model in §11 — a link with unclear provenance must not quietly become a "fact". An inferred edge is a suggestion, not an assertion.
|
|
33
|
+
|
|
34
|
+
### 20.4 Three operations (Karpathy's ingest / query / lint)
|
|
35
|
+
|
|
36
|
+
#### Ingest: the graph updates when a memory is written / approved
|
|
37
|
+
|
|
38
|
+
- Trigger: a memory is created, updated, or its promotion state becomes approved (§5). Core triggers an incremental graph update on the write path (new nodes/edges, updates to the affected compiled pages) — not a separate background job.
|
|
39
|
+
- One ingest can touch several compiled pages (one memory can update an entity page, a concept page, and an overview). That is normal — the wiki's value is "one ingest, merged in many places".
|
|
40
|
+
- Ingest never changes the memory itself (raw is immutable); it only updates the derived graph and wiki.
|
|
41
|
+
|
|
42
|
+
#### Query: graph traversal becomes a third retrieval lane
|
|
43
|
+
|
|
44
|
+
- §7 today has two lanes: BM25/FTS5 and vector (optional). This section adds a third: **graph traversal** (a node's neighbours, the shortest path between two concepts, a community).
|
|
45
|
+
- Recall receipts (§7) gain a `graph` lane: each hit reports its lane (BM25 / vector / graph / recency); a graph hit reports which edge it travelled (type + EXTRACTED/INFERRED).
|
|
46
|
+
- A good answer from a graph query can be filed back into the wiki as a new page (a good comparison or analysis should not vanish into chat history). That is the compounding step.
|
|
47
|
+
|
|
48
|
+
#### Lint: a health check, run automatically before promotion
|
|
49
|
+
|
|
50
|
+
Lint runs **automatically before promotion** (decided 2026-10-31) and checks the graph/wiki for:
|
|
51
|
+
|
|
52
|
+
- Contradictions between memories (two memories asserting opposite facts)
|
|
53
|
+
- Stale claims superseded by newer memories (should have entered the supersede chain but hasn't)
|
|
54
|
+
- Orphan memories (written but never linked, in no compiled page)
|
|
55
|
+
- Important concepts mentioned but without their own page (missing pages)
|
|
56
|
+
- Missing cross-references
|
|
57
|
+
|
|
58
|
+
**Lint only discovers and reports; it never auto-resolves** (echoes Principle 5: Core never silently resolves semantic conflicts; and §9's conflict handling). The results go to a human (the curator / promotion reviewer) to decide. This is the input the §5 promotion flow needs: a list of "which memories contradict, which are stale, which are orphaned".
|
|
59
|
+
|
|
60
|
+
### 20.5 Local-first & privacy (no change to the existing iron rules)
|
|
61
|
+
|
|
62
|
+
- **The graph is derived and stored locally**: compiled from local Markdown, stored locally (SQLite index + Markdown compiled pages), on the same machine as the memory files. Lost, it can be rebuilt.
|
|
63
|
+
- **Deterministic extraction needs no LLM**: structural edges (shared tag, shared path, shared `canonical_ref`, explicit links) are extracted by deterministic rules, on-machine, without an LLM (borrowing Graphify's tree-sitter idea: structure is extracted, not inferred).
|
|
64
|
+
- **Semantic extraction of docs is confirmed at init, on or off** (decided 2026-10-31): understanding document content semantically (not just structurally) needs an explicit LLM backend. Init asks the user whether to turn it on or off (echoes Principle 4: nothing leaves the machine without explicit user action) — off means no semantic extraction. Once on, the default backend is the API (most machines have no local model); to use a local model, change it in config (Ollama / LM Studio).
|
|
65
|
+
- **The personal scope never syncs** (D3): personal memories stay out of the graph's sync scope; the graph's sync follows the same scope/visibility rules as memory sync (§4), no separate scheme.
|
|
66
|
+
- **Iron rules unchanged**: a graph edge can never automatically become an instruction (iron rule 1's role gate still sits at the instruction-context entry); a personal memory's instruction never enters AGENTS.md (iron rule 2). The graph is a knowledge layer, not an authorization layer.
|
|
67
|
+
|
|
68
|
+
### 20.6 Scope & phasing (v0.3 candidate, not in v0.2)
|
|
69
|
+
|
|
70
|
+
- **This section is a v0.3 candidate; it does not change the frozen v0.2 text.** v0.2 (Phases 0–2) proceeds as planned; the graph is a **parallel, benchmark-gated experiment** (same treatment as embeddings in Phase 2), not on the critical path.
|
|
71
|
+
- **Smallest closed loop first**: one domain (e.g. one project) → graph compiled from memories → neighbours and paths queryable → lint reporting contradictions and orphans. Expand only after that loop works.
|
|
72
|
+
- **Scale ceiling**: Karpathy's index pattern suffices at ~100 sources and a few hundred pages; before open-memex's memory count reaches that scale, the graph's index is "read the compiled pages first, then drill into the originals" — no embedding RAG infrastructure needed. Beyond that, add search/vector (reusing the §7 embeddings interface).
|
|
73
|
+
|
|
74
|
+
### 20.7 What this section does not change
|
|
75
|
+
|
|
76
|
+
- **Data model unchanged**: one memory per file, Markdown is the source of truth, ULID is the filename (§3). The graph is a derived layer; it adds no memory field (edges live in the graph layer; whether `related` enters the frontmatter is a v0.3 protocol decision).
|
|
77
|
+
- **Iron rules unchanged** (§3.2): the instruction gate and the personal-scope guard apply as before.
|
|
78
|
+
- **Promotion flow unchanged** (§5): promotion stays explicit and human-reviewed; lint only supplies input to the reviewer, it does not decide for them.
|
|
79
|
+
- **Retrieval baseline unchanged** (§7): BM25/FTS5 + CJK remain the default; the graph is an added lane, not a replacement.
|
|
80
|
+
|
|
81
|
+
### 20.8 Small amendments to §3 / §7 / §12 (land with this section, pending approval)
|
|
82
|
+
|
|
83
|
+
- **§3**: a v0.3 protocol revision will consider whether to add a `related` field (typed, with provenance); until then, edges live only in the graph layer, not in the frontmatter.
|
|
84
|
+
- **§7 Recall receipts**: add a `graph` lane; a graph hit reports its edge type and provenance.
|
|
85
|
+
- **§12 table**: add a sub-row under "OpenMemex" — "Compiled wiki/graph: knowledge compiled from memories (derived, rebuildable), between memories and terminal documents".
|
|
86
|
+
|
package/package.json
CHANGED
|
@@ -1,66 +1,66 @@
|
|
|
1
|
-
{
|
|
2
|
-
"name": "open-memex",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"mcpName": "io.github.stoneskin/open-memex",
|
|
5
|
-
"description": "Local-first memory layer and protocol for AI coding agents. Markdown source of truth, SQLite FTS5 index, zero cloud.",
|
|
6
|
-
"keywords": [
|
|
7
|
-
"mcp",
|
|
8
|
-
"model-context-protocol",
|
|
9
|
-
"mcp-server",
|
|
10
|
-
"opencode",
|
|
11
|
-
"ai-agents",
|
|
12
|
-
"agent-memory",
|
|
13
|
-
"persistent-memory",
|
|
14
|
-
"local-first",
|
|
15
|
-
"sqlite",
|
|
16
|
-
"fts5",
|
|
17
|
-
"markdown",
|
|
18
|
-
"memory",
|
|
19
|
-
"copilot",
|
|
20
|
-
"cursor",
|
|
21
|
-
"claude-code",
|
|
22
|
-
"vscode",
|
|
23
|
-
"knowledge-management",
|
|
24
|
-
"developer-tools",
|
|
25
|
-
"synonyms"
|
|
26
|
-
],
|
|
27
|
-
"type": "module",
|
|
28
|
-
"license": "Apache-2.0",
|
|
29
|
-
"repository": {
|
|
30
|
-
"type": "git",
|
|
31
|
-
"url": "git+https://github.com/stoneskin/open-memex.git"
|
|
32
|
-
},
|
|
33
|
-
"homepage": "https://github.com/stoneskin/open-memex#readme",
|
|
34
|
-
"bugs": {
|
|
35
|
-
"url": "https://github.com/stoneskin/open-memex/issues"
|
|
36
|
-
},
|
|
37
|
-
"main": "src/index.ts",
|
|
38
|
-
"scripts": {
|
|
39
|
-
"typecheck": "tsc --noEmit",
|
|
40
|
-
"cli": "node --experimental-strip-types src/cli.ts",
|
|
41
|
-
"mcp": "node --experimental-strip-types src/mcp.ts",
|
|
42
|
-
"build": "tsc -p tsconfig.build.json && node scripts/build-stamp.mjs",
|
|
43
|
-
"prepublishOnly": "npm run build",
|
|
44
|
-
"postinstall": "node scripts/postinstall.js",
|
|
45
|
-
"test-full": "node --experimental-strip-types scripts/test-full.ts"
|
|
46
|
-
},
|
|
47
|
-
"dependencies": {
|
|
48
|
-
"@modelcontextprotocol/sdk": "^1.30.1",
|
|
49
|
-
"better-sqlite3": "^13.0.3",
|
|
50
|
-
"js-yaml": "^4.1.0",
|
|
51
|
-
"zod": "^4.1.8"
|
|
52
|
-
},
|
|
53
|
-
"devDependencies": {
|
|
54
|
-
"@opencode-ai/plugin": "^1.18.30",
|
|
55
|
-
"@types/better-sqlite3": "^7.6.11",
|
|
56
|
-
"@types/js-yaml": "^4.0.9",
|
|
57
|
-
"@types/node": "^22.0.0",
|
|
58
|
-
"typescript": "^5.6.0"
|
|
59
|
-
},
|
|
60
|
-
"bin": {
|
|
61
|
-
"open-memex": "dist/cli.js"
|
|
62
|
-
},
|
|
63
|
-
"engines": {
|
|
64
|
-
"node": ">=22.14.0"
|
|
65
|
-
}
|
|
66
|
-
}
|
|
1
|
+
{
|
|
2
|
+
"name": "open-memex",
|
|
3
|
+
"version": "0.9.0-alpha.2",
|
|
4
|
+
"mcpName": "io.github.stoneskin/open-memex",
|
|
5
|
+
"description": "Local-first memory layer and protocol for AI coding agents. Markdown source of truth, SQLite FTS5 index, zero cloud.",
|
|
6
|
+
"keywords": [
|
|
7
|
+
"mcp",
|
|
8
|
+
"model-context-protocol",
|
|
9
|
+
"mcp-server",
|
|
10
|
+
"opencode",
|
|
11
|
+
"ai-agents",
|
|
12
|
+
"agent-memory",
|
|
13
|
+
"persistent-memory",
|
|
14
|
+
"local-first",
|
|
15
|
+
"sqlite",
|
|
16
|
+
"fts5",
|
|
17
|
+
"markdown",
|
|
18
|
+
"memory",
|
|
19
|
+
"copilot",
|
|
20
|
+
"cursor",
|
|
21
|
+
"claude-code",
|
|
22
|
+
"vscode",
|
|
23
|
+
"knowledge-management",
|
|
24
|
+
"developer-tools",
|
|
25
|
+
"synonyms"
|
|
26
|
+
],
|
|
27
|
+
"type": "module",
|
|
28
|
+
"license": "Apache-2.0",
|
|
29
|
+
"repository": {
|
|
30
|
+
"type": "git",
|
|
31
|
+
"url": "git+https://github.com/stoneskin/open-memex.git"
|
|
32
|
+
},
|
|
33
|
+
"homepage": "https://github.com/stoneskin/open-memex#readme",
|
|
34
|
+
"bugs": {
|
|
35
|
+
"url": "https://github.com/stoneskin/open-memex/issues"
|
|
36
|
+
},
|
|
37
|
+
"main": "src/index.ts",
|
|
38
|
+
"scripts": {
|
|
39
|
+
"typecheck": "tsc --noEmit",
|
|
40
|
+
"cli": "node --experimental-strip-types src/cli.ts",
|
|
41
|
+
"mcp": "node --experimental-strip-types src/mcp.ts",
|
|
42
|
+
"build": "tsc -p tsconfig.build.json && node scripts/build-stamp.mjs",
|
|
43
|
+
"prepublishOnly": "npm run build",
|
|
44
|
+
"postinstall": "node scripts/postinstall.js",
|
|
45
|
+
"test-full": "node --experimental-strip-types scripts/test-full.ts"
|
|
46
|
+
},
|
|
47
|
+
"dependencies": {
|
|
48
|
+
"@modelcontextprotocol/sdk": "^1.30.1",
|
|
49
|
+
"better-sqlite3": "^13.0.3",
|
|
50
|
+
"js-yaml": "^4.1.0",
|
|
51
|
+
"zod": "^4.1.8"
|
|
52
|
+
},
|
|
53
|
+
"devDependencies": {
|
|
54
|
+
"@opencode-ai/plugin": "^1.18.30",
|
|
55
|
+
"@types/better-sqlite3": "^7.6.11",
|
|
56
|
+
"@types/js-yaml": "^4.0.9",
|
|
57
|
+
"@types/node": "^22.0.0",
|
|
58
|
+
"typescript": "^5.6.0"
|
|
59
|
+
},
|
|
60
|
+
"bin": {
|
|
61
|
+
"open-memex": "dist/cli.js"
|
|
62
|
+
},
|
|
63
|
+
"engines": {
|
|
64
|
+
"node": ">=22.14.0"
|
|
65
|
+
}
|
|
66
|
+
}
|
package/scripts/smoke-mcp.ts
CHANGED
|
@@ -80,9 +80,9 @@ try {
|
|
|
80
80
|
const tools = await req("tools/list", {});
|
|
81
81
|
const names = (tools.result?.tools ?? []).map((t: any) => t.name).sort();
|
|
82
82
|
check(
|
|
83
|
-
"tools/list has
|
|
83
|
+
"tools/list has 13 memory tools",
|
|
84
84
|
JSON.stringify(names) ===
|
|
85
|
-
JSON.stringify(["memory_add", "memory_forget", "memory_list", "memory_pr_status", "memory_promote", "memory_propose", "memory_resolve", "memory_search", "memory_status", "memory_submit", "memory_supersede"]),
|
|
85
|
+
JSON.stringify(["memory_add", "memory_forget", "memory_list", "memory_pr_status", "memory_promote", "memory_propose", "memory_resolve", "memory_search", "memory_status", "memory_submit", "memory_supersede", "memory_useful", "memory_workingstate"]),
|
|
86
86
|
names.join(","),
|
|
87
87
|
);
|
|
88
88
|
// D76: annotations must tell the truth — pr_status mutates with apply=true.
|
|
@@ -124,8 +124,7 @@ try {
|
|
|
124
124
|
|
|
125
125
|
const list = await req("tools/call", { name: "memory_list", arguments: {} });
|
|
126
126
|
const listText = list.result?.content?.[0]?.text ?? "";
|
|
127
|
-
check("memory_list works", listText.includes(savedId));
|
|
128
|
-
// D68: the list is the user's inventory — numbered, plain language,
|
|
127
|
+
check("memory_list works", listText.includes(savedId)); // D68: the list is the user's inventory — numbered, plain language,
|
|
129
128
|
// with provenance and age, not the raw developer format.
|
|
130
129
|
check("memory_list is numbered structured lines (D68)", /^\d+\. \[fact\] id=/m.test(listText), listText.slice(0, 120));
|
|
131
130
|
check("memory_list keeps raw provenance field (D68)", listText.includes("source=tool"), listText.slice(0, 160));
|
|
@@ -188,6 +187,70 @@ try {
|
|
|
188
187
|
|
|
189
188
|
const search2 = await req("tools/call", { name: "memory_search", arguments: { query: "mcp e2e probe" } });
|
|
190
189
|
check("memory gone after forget", !(search2.result?.content?.[0]?.text ?? "").includes(savedId));
|
|
190
|
+
|
|
191
|
+
// D77-L2:reuse counters — explicit useful only; exposure counts retrieved.
|
|
192
|
+
const missing = await req("tools/call", {
|
|
193
|
+
name: "memory_useful",
|
|
194
|
+
arguments: { id: "01NEVEREXISTED0000000000" },
|
|
195
|
+
});
|
|
196
|
+
check(
|
|
197
|
+
"memory_useful refuses an unknown id (D77-L2)",
|
|
198
|
+
!missing.result?.isError && /No memory with id/.test(missing.result?.content?.[0]?.text ?? ""),
|
|
199
|
+
(missing.result?.content?.[0]?.text ?? "").slice(0, 120),
|
|
200
|
+
);
|
|
201
|
+
|
|
202
|
+
const add3 = await req("tools/call", {
|
|
203
|
+
name: "memory_add",
|
|
204
|
+
arguments: { content: "mcp e2e probe: useful counter memory for D77", type: "fact" },
|
|
205
|
+
});
|
|
206
|
+
const savedId3 = ((add3.result?.content?.[0]?.text ?? "").match(/id=([A-Za-z0-9_]+)/) ?? [])[1];
|
|
207
|
+
check("D77 probe memory added", !!savedId3, (add3.result?.content?.[0]?.text ?? "").slice(0, 120));
|
|
208
|
+
|
|
209
|
+
const useful1 = await req("tools/call", {
|
|
210
|
+
name: "memory_useful",
|
|
211
|
+
arguments: { id: savedId3 },
|
|
212
|
+
});
|
|
213
|
+
const useful1Text: string = useful1.result?.content?.[0]?.text ?? "";
|
|
214
|
+
check("memory_useful marks useful_self (auto split: project draft)", useful1Text.includes("Marked useful (self)") && useful1Text.includes("useful_self: 1"), useful1Text.slice(0, 160));
|
|
215
|
+
check("memory_useful reports unretrieved honestly (D77-L2)", useful1Text.includes("n/a"), useful1Text.slice(0, 160));
|
|
216
|
+
|
|
217
|
+
await req("tools/call", { name: "memory_search", arguments: { query: "useful counter memory for D77" } });
|
|
218
|
+
const useful2 = await req("tools/call", {
|
|
219
|
+
name: "memory_useful",
|
|
220
|
+
arguments: { id: savedId3, kind: "team" },
|
|
221
|
+
});
|
|
222
|
+
const useful2Text: string = useful2.result?.content?.[0]?.text ?? "";
|
|
223
|
+
check("exposure was recorded by memory_search (D77-L2)", useful2Text.includes("retrieved: 1"), useful2Text.slice(0, 160));
|
|
224
|
+
check("memory_useful kind override lands in useful_team (D77-L2)", useful2Text.includes("useful_team: 1") && useful2Text.includes("Marked useful (team)"), useful2Text.slice(0, 160));
|
|
225
|
+
const forgetUseful = await req("tools/call", { name: "memory_forget", arguments: { id: savedId3 } });
|
|
226
|
+
check("D77 probe memory cleaned up", (forgetUseful.result?.content?.[0]?.text ?? "").includes("Deleted"), (forgetUseful.result?.content?.[0]?.text ?? "").slice(0, 120));
|
|
227
|
+
|
|
228
|
+
// §3.6 (D89): the working-state note round-trips, expires honestly, and
|
|
229
|
+
// its file lives OUTSIDE memories/ so the index never picks it up.
|
|
230
|
+
const wsRead0 = await req("tools/call", { name: "memory_workingstate", arguments: {} });
|
|
231
|
+
const wsRead0Text = wsRead0.result?.content?.[0]?.text ?? "";
|
|
232
|
+
check("workingstate read reports none (D89)", wsRead0Text.includes("No working-state note"), wsRead0Text.slice(0, 120));
|
|
233
|
+
|
|
234
|
+
const wsWrite = await req("tools/call", {
|
|
235
|
+
name: "memory_workingstate",
|
|
236
|
+
arguments: { action: "write", content: "task: port the deploy script; next: run staging smoke" },
|
|
237
|
+
});
|
|
238
|
+
const wsWriteText = wsWrite.result?.content?.[0]?.text ?? "";
|
|
239
|
+
check("workingstate write works (D89)", wsWriteText.includes("written"), wsWriteText.slice(0, 160));
|
|
240
|
+
|
|
241
|
+
const wsRead = await req("tools/call", { name: "memory_workingstate", arguments: {} });
|
|
242
|
+
const wsReadText = wsRead.result?.content?.[0]?.text ?? "";
|
|
243
|
+
check("workingstate reads back (D89)", wsReadText.includes("run staging smoke"), wsReadText.slice(0, 160));
|
|
244
|
+
|
|
245
|
+
const wsFile = fs.readdirSync(path.join(HOME, "working-state")).find((f) => f.endsWith(".md"));
|
|
246
|
+
check("workingstate file outside memories/ (D89)", !!wsFile, `working-state/${wsFile}`);
|
|
247
|
+
|
|
248
|
+
const wsSearch = await req("tools/call", { name: "memory_search", arguments: { query: "staging smoke" } });
|
|
249
|
+
check("workingstate never enters the index (D89)", !(wsSearch.result?.content?.[0]?.text ?? "").includes("run staging smoke"), (wsSearch.result?.content?.[0]?.text ?? "").slice(0, 120));
|
|
250
|
+
|
|
251
|
+
const wsClear = await req("tools/call", { name: "memory_workingstate", arguments: { action: "clear" } });
|
|
252
|
+
check("workingstate clear works (D89)", (wsClear.result?.content?.[0]?.text ?? "").includes("deleted"), (wsClear.result?.content?.[0]?.text ?? "").slice(0, 120));
|
|
253
|
+
|
|
191
254
|
} finally {
|
|
192
255
|
child.kill();
|
|
193
256
|
}
|
package/scripts/smoke-pure.ts
CHANGED
|
@@ -965,5 +965,75 @@ console.log("== D83 expansions BM25 weight + matchedVia ==");
|
|
|
965
965
|
ok("body stays the strongest column", BM25_WEIGHTS[0] >= BM25_WEIGHTS[3], JSON.stringify(BM25_WEIGHTS));
|
|
966
966
|
}
|
|
967
967
|
|
|
968
|
+
console.log("== D89 working-state note (§3.6) ==");
|
|
969
|
+
{
|
|
970
|
+
const {
|
|
971
|
+
workingStatePath,
|
|
972
|
+
writeWorkingState,
|
|
973
|
+
readWorkingState,
|
|
974
|
+
clearWorkingState,
|
|
975
|
+
workingStateBlock,
|
|
976
|
+
} = await import("../src/working-state.ts");
|
|
977
|
+
const savedHome = process.env.OPEN_MEMEX_HOME;
|
|
978
|
+
const home = fs.mkdtempSync(path.join(os.tmpdir(), "om-wstate-"));
|
|
979
|
+
process.env.OPEN_MEMEX_HOME = home;
|
|
980
|
+
const KEY = "project__wstatetest__abcdef123456";
|
|
981
|
+
const ttlCfg = { ...DEFAULT_CONFIG, workingStateTtlDays: 7 };
|
|
982
|
+
try {
|
|
983
|
+
// Read on a store that was never touched: no polluting directory.
|
|
984
|
+
ok("missing note → null", readWorkingState(KEY, 7) === null);
|
|
985
|
+
ok("read never creates the dir", !fs.existsSync(path.join(home, "working-state")));
|
|
986
|
+
// Write: redact applies (a pasted command secret is masked).
|
|
987
|
+
const w = writeWorkingState(
|
|
988
|
+
KEY,
|
|
989
|
+
"resume deploy; saw key=sk-1234567890abcdefghij1234 in logs",
|
|
990
|
+
7,
|
|
991
|
+
DEFAULT_CONFIG.redactPatterns,
|
|
992
|
+
);
|
|
993
|
+
// NOTE: paths() caches the data root per process, and an earlier section
|
|
994
|
+
// in this file may already have pinned it to its own temp home. Assert
|
|
995
|
+
// the shape (…/working-state/<key>.md), not the absolute prefix.
|
|
996
|
+
ok(
|
|
997
|
+
"write landed under a working-state dir",
|
|
998
|
+
w.filePath.includes(path.join("working-state", `${KEY}.md`)),
|
|
999
|
+
w.filePath,
|
|
1000
|
+
);
|
|
1001
|
+
ok("secret masked", w.hadSecret && !fs.readFileSync(w.filePath, "utf8").includes("1234567890abcdefghij"));
|
|
1002
|
+
const note = readWorkingState(KEY, 7);
|
|
1003
|
+
ok("note reads back", note !== null && note.content.includes("resume deploy"));
|
|
1004
|
+
ok("ttl stamped from config", note?.ttlDays === 7);
|
|
1005
|
+
ok("fresh note not expired", note?.expired === false && note?.ageDays === 0);
|
|
1006
|
+
// Frontmatter edit override: the user's per-note ttl survives.
|
|
1007
|
+
const raw = fs.readFileSync(w.filePath, "utf8").replace("ttl_days: 7", "ttl_days: 3");
|
|
1008
|
+
fs.writeFileSync(w.filePath, raw, "utf8");
|
|
1009
|
+
const note2 = readWorkingState(KEY, 7);
|
|
1010
|
+
ok("per-note ttl override honored", note2?.ttlDays === 3);
|
|
1011
|
+
// Expiry: backdate updated_at past the ttl.
|
|
1012
|
+
const stale = fs.readFileSync(w.filePath, "utf8").replace(
|
|
1013
|
+
/^updated_at: .*$/m,
|
|
1014
|
+
`updated_at: ${new Date(Date.now() - 9 * 86_400_000).toISOString()}`,
|
|
1015
|
+
);
|
|
1016
|
+
fs.writeFileSync(w.filePath, stale, "utf8");
|
|
1017
|
+
const note3 = readWorkingState(KEY, 7);
|
|
1018
|
+
ok("aged note expired", note3?.expired === true && note3?.ttlDays === 3 && note3?.ageDays === 9, JSON.stringify(note3));
|
|
1019
|
+
// Expired notes never surface; a fresh one does, with the right framing.
|
|
1020
|
+
ok("expired note does not surface", workingStateBlock(KEY, "proj", ttlCfg) === null);
|
|
1021
|
+
writeWorkingState(KEY, "resume deploy; run smoke tests", 7, []);
|
|
1022
|
+
const block = workingStateBlock(KEY, "proj", ttlCfg);
|
|
1023
|
+
ok("fresh note surfaces", block !== null && block.includes("resume deploy"), String(block).slice(0, 40));
|
|
1024
|
+
ok("block frames it as scratch", block?.includes("not durable knowledge") === true);
|
|
1025
|
+
ok("block carries the wrap-up rule", block?.includes("memory_workingstate") === true);
|
|
1026
|
+
ok("disabled config never surfaces", workingStateBlock(KEY, "proj", { ...ttlCfg, workingStateEnabled: false }) === null);
|
|
1027
|
+
// Clear is honest: gone once, then reports gone.
|
|
1028
|
+
ok("clear deletes", clearWorkingState(KEY) === true);
|
|
1029
|
+
ok("clear on missing stays true", clearWorkingState(KEY) === true);
|
|
1030
|
+
ok("note gone", readWorkingState(KEY, 7) === null);
|
|
1031
|
+
} finally {
|
|
1032
|
+
if (savedHome === undefined) delete process.env.OPEN_MEMEX_HOME;
|
|
1033
|
+
else process.env.OPEN_MEMEX_HOME = savedHome;
|
|
1034
|
+
fs.rmSync(home, { recursive: true, force: true });
|
|
1035
|
+
}
|
|
1036
|
+
}
|
|
1037
|
+
|
|
968
1038
|
console.log(fails === 0 ? "\nALL PASS" : `\n${fails} FAILURES`);
|
|
969
1039
|
process.exit(fails === 0 ? 0 : 1);
|
package/scripts/test-full.ts
CHANGED
|
@@ -802,6 +802,89 @@ ${body}
|
|
|
802
802
|
r.out.slice(0, 200),
|
|
803
803
|
);
|
|
804
804
|
|
|
805
|
+
// ---------- D77-L2: reuse counters + useful ----------
|
|
806
|
+
// Counters are SQLite-local telemetry (memory_usage table), never written
|
|
807
|
+
// into frontmatter. Exposure counts retrieved_count from search/list/
|
|
808
|
+
// CLI reads; useful only via `useful`/memory_useful, supersede, or a
|
|
809
|
+
// human promote to approved.
|
|
810
|
+
console.log("== usage counters (D77-L2) ==");
|
|
811
|
+
const d77Proj = mkproj("d77-proj");
|
|
812
|
+
const uId = addMem(["fulltest D77 useful-signal memory"], d77Proj);
|
|
813
|
+
ok("D77 probe memory added", !!uId);
|
|
814
|
+
|
|
815
|
+
// No exposure yet: never searched/listed (add alone is not a retrieval).
|
|
816
|
+
r = cliRetry(["useful", uId], d77Proj);
|
|
817
|
+
ok(
|
|
818
|
+
"cli useful marks self (unreviewed project draft) with 0 retrieved",
|
|
819
|
+
/useful \(self\) /.test(r.out) && /retrieved: 0/.test(r.out) && /useful_self: 1/.test(r.out),
|
|
820
|
+
r.out.slice(0, 200),
|
|
821
|
+
);
|
|
822
|
+
|
|
823
|
+
// The retrieval surfaces count exposures; the audit view does not.
|
|
824
|
+
cliRetry(["search", "useful-signal memory"], d77Proj);
|
|
825
|
+
cliRetry(["list"], d77Proj);
|
|
826
|
+
r = cliRetry(["list", "--include", "all"], d77Proj);
|
|
827
|
+
r = cliRetry(["useful", uId], d77Proj);
|
|
828
|
+
ok(
|
|
829
|
+
"search + list recorded exposures, audit view did not",
|
|
830
|
+
/retrieved: 2/.test(r.out) && /useful_self: 2/.test(r.out) && /reuse_rate: 1\.00/.test(r.out),
|
|
831
|
+
r.out.slice(0, 220),
|
|
832
|
+
);
|
|
833
|
+
|
|
834
|
+
// Unknown id fails the batch.
|
|
835
|
+
r = cliRetry(["useful", "01MISSING000000000000000000"], d77Proj);
|
|
836
|
+
ok("cli useful reports not found", /not found/.test(r.out + r.err), (r.out + r.err).slice(0, 120));
|
|
837
|
+
|
|
838
|
+
// Behavioral: supersede records a useful tick on the OLD memory.
|
|
839
|
+
const oldId = addMem(["fulltest D77 supersede-then-count target"], d77Proj);
|
|
840
|
+
r = cliRetry(["supersede", oldId, "fulltest D77 supersede target replaced"], d77Proj);
|
|
841
|
+
ok("supersede works (setup)", !!grabArrowId(r.out), r.out.slice(0, 120));
|
|
842
|
+
r = cliRetry(["useful", oldId], d77Proj);
|
|
843
|
+
// supersede already ticked useful_self on the old memory; the probe adds
|
|
844
|
+
// one explicit tick, so the printed counter reads 2.
|
|
845
|
+
ok(
|
|
846
|
+
"supersede counted as useful on the old memory (D77)",
|
|
847
|
+
/useful_self: 2/.test(r.out),
|
|
848
|
+
r.out.slice(0, 200),
|
|
849
|
+
);
|
|
850
|
+
|
|
851
|
+
// Behavioral: human approve at review records useful_team (no memory_useful call).
|
|
852
|
+
// Approved is reachable only through the review ladder: personal → propose
|
|
853
|
+
// (outbox draft) → submit (repo, proposed) → promote (approved).
|
|
854
|
+
const tId = addMem(["fulltest D77 review-approved memory", "--scope", "personal"], d77Proj);
|
|
855
|
+
const proposed = proposeMem(tId, d77Proj);
|
|
856
|
+
const submitted = cliRetry(["submit", proposed], d77Proj);
|
|
857
|
+
ok("approved-path submit works (setup)", submitted.code === 0, (submitted.err || submitted.out).slice(0, 150));
|
|
858
|
+
r = cliRetry(["promote", proposed], d77Proj);
|
|
859
|
+
ok("promote to approved works (setup)", /\bapproved\b/.test(r.out), r.out.slice(0, 160));
|
|
860
|
+
r = cliRetry(["useful", proposed], d77Proj);
|
|
861
|
+
// promote itself already ticked useful_team; the useful probe below adds
|
|
862
|
+
// one explicit tick on top, so the printed counter reads 2.
|
|
863
|
+
ok(
|
|
864
|
+
"promote → approved counted useful_team (D77)",
|
|
865
|
+
/useful_team: 2/.test(r.out) && /useful \(team\)/.test(r.out) && /useful_self: 0/.test(r.out),
|
|
866
|
+
r.out.slice(0, 220),
|
|
867
|
+
);
|
|
868
|
+
|
|
869
|
+
// Hard forget prunes the telemetry row (no phantom counters).
|
|
870
|
+
const deadId = addMem(["fulltest D77 to-be-forgotten memory"], d77Proj);
|
|
871
|
+
cliRetry(["forget", deadId], d77Proj);
|
|
872
|
+
r = cliRetry(["useful", deadId], d77Proj);
|
|
873
|
+
ok("forget pruned the usage row (D77-L2)", r.code !== 0 && /not found/.test(r.out + r.err), (r.out + r.err).slice(0, 120));
|
|
874
|
+
|
|
875
|
+
// Counters survive a full schema rebuild (memory_usage sits outside the
|
|
876
|
+
// rebuilt index tables; D77-L2 store design).
|
|
877
|
+
cliRetry(["reindex"], d77Proj);
|
|
878
|
+
r = cliRetry(["useful", uId], d77Proj);
|
|
879
|
+
ok(
|
|
880
|
+
"counters survive reindex",
|
|
881
|
+
/retrieved: 2/.test(r.out) || /retrieved: 3/.test(r.out),
|
|
882
|
+
r.out.slice(0, 160),
|
|
883
|
+
);
|
|
884
|
+
// NOTE: the reindex section's useful tick bumps useful_self by 1 — that is
|
|
885
|
+
// the probe itself, not persistence; the retrieved floor above is what
|
|
886
|
+
// proves the table survived the rebuild.
|
|
887
|
+
|
|
805
888
|
// JSON: no absolute local paths, and the format id records the change.
|
|
806
889
|
r = cliRetry(["inventory", "--format", "json"], PROJ);
|
|
807
890
|
const invJson2 = JSON.parse(r.out.slice(r.out.indexOf("{")));
|
package/server.json
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
"name": "io.github.stoneskin/open-memex",
|
|
4
4
|
"title": "open-memex",
|
|
5
5
|
"description": "Local-first memory for AI coding agents — Markdown + SQLite FTS5, shared across editors via MCP.",
|
|
6
|
-
"version": "0.8.
|
|
6
|
+
"version": "0.8.2",
|
|
7
7
|
"repository": {
|
|
8
8
|
"url": "https://github.com/stoneskin/open-memex",
|
|
9
9
|
"source": "github"
|
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
{
|
|
14
14
|
"registryType": "npm",
|
|
15
15
|
"identifier": "open-memex",
|
|
16
|
-
"version": "0.8.
|
|
16
|
+
"version": "0.8.2",
|
|
17
17
|
"runtimeHint": "npx",
|
|
18
18
|
"packageArguments": [
|
|
19
19
|
{
|
|
@@ -57,6 +57,16 @@ with the `memory_add` tool pass `source: "inference"` so the capture is marked
|
|
|
57
57
|
as agent-proposed. Don't interrupt mid-task, and don't re-propose something
|
|
58
58
|
the user already declined.
|
|
59
59
|
|
|
60
|
+
## At wrap-up — refresh the working-state note
|
|
61
|
+
|
|
62
|
+
When the user signals the session or task is wrapping up (and the
|
|
63
|
+
`memory_workingstate` tool is available), write a short scratch note so an
|
|
64
|
+
interrupted task can resume in a fresh session: task summary (one paragraph),
|
|
65
|
+
open files, last commands, where the work stands, next step. It is **scratch
|
|
66
|
+
state, not knowledge** — never save it as a memory; anything here that
|
|
67
|
+
deserves to outlive the task goes through the propose flow above. The note
|
|
68
|
+
expires after 7 days; `open-memex wstate` shows it (`--clear` deletes it).
|
|
69
|
+
|
|
60
70
|
## Recall
|
|
61
71
|
|
|
62
72
|
```sh
|
|
@@ -70,6 +80,15 @@ try 2–3 phrasings per concept — synonyms, the user's other language, shorter
|
|
|
70
80
|
keyword forms — and check the other scope too, before concluding nothing is
|
|
71
81
|
stored.
|
|
72
82
|
|
|
83
|
+
## Record what paid off (D77)
|
|
84
|
+
|
|
85
|
+
When a person explicitly confirms a memory helped (or accepts work that
|
|
86
|
+
visibly relied on one), mark it: the `memory_useful` tool if available, else
|
|
87
|
+
`open-memex useful <id>`. Never mark a memory just because it appeared in
|
|
88
|
+
search results, a listing, or the session-start block — passive exposure is
|
|
89
|
+
counted automatically as `retrieved_count` and is not usefulness. Superseding
|
|
90
|
+
a memory or a human approving it at review already records a useful signal.
|
|
91
|
+
|
|
73
92
|
## Seeing and removing what is remembered
|
|
74
93
|
|
|
75
94
|
When the user asks "what do you remember about me?" (or wants to check or
|