@kenz1117/dsh-engram 0.7.7 → 0.7.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +26 -10
- package/README.md +26 -10
- package/lib/client.js +3 -1
- package/lib/client.js.map +1 -1
- package/lib/index.js +882 -60
- package/package.json +1 -1
package/README.en.md
CHANGED
|
@@ -58,7 +58,7 @@ Sensory and emotional dimensions (smell, temperature, emotional weight) are deli
|
|
|
58
58
|
|
|
59
59
|
## Features
|
|
60
60
|
|
|
61
|
-
- **Cross-session memory**: a user memory profile is injected at session start (entry cap plus token budget, both configurable), so the agent naturally knows who you are and what you are building; tools recall facts across sessions.
|
|
61
|
+
- **Cross-session memory**: a user memory profile is injected at session start (entry cap plus token budget, both configurable; a curated profile maintained via `engram_profile_edit` takes precedence over derived facts), so the agent naturally knows who you are and what you are building; tools recall facts across sessions.
|
|
62
62
|
- **Dual-scope stores**: `user.db` shared globally; `project-<hash>.db` isolated per git origin identity (falls back to a working-directory encoding without git; legacy stores migrate automatically) — personal preferences follow the person, project conventions follow the repo.
|
|
63
63
|
- **Hybrid retrieval**: FTS5 (unicode61 + Chinese 2-gram pre-tokenization) fused with local vectors (`Xenova/bge-small-zh-v1.5`, 512-dim, q8) via RRF, plus one-hop expansion over relation edges and a multiplicative recency/proof ranking boost; the embedding model runs offline, and a failed download degrades to keyword-only retrieval with an explicit marker.
|
|
64
64
|
- **Memory-palace information architecture** (v0.7.2+): all four principles live on the core path, not in display-layer paint — **location as index** (writes sort by kind into rooms and pin a `room#slot` coordinate; room capacity 9, overflow opens a new room, slots are never recycled); **fixed route as order** (`tour_routes` is append-only; `engram_tour mode=fixed` walks the whole palace in slot order); **skeleton reused long-term** (a topic always lands in the same room and the same index, so recall is sequential extraction rather than fresh search); **every marker unique** (placard rule scored 0-1: globally unique +0.4 / date anchor +0.3 / no 6-char prefix clash within the room +0.3; low scores enter the refurb list; a palace with fewer than 8 active memories is left unscanned to avoid small-store noise).
|
|
@@ -66,25 +66,26 @@ Sensory and emotional dimensions (smell, temperature, emotional weight) are deli
|
|
|
66
66
|
- **Retrieval-practice loop** (spaced repetition): `engram_review_queue` returns palace coordinates and placard cues but **never the body text**, forcing the model to recall first; `engram_review` reveals the entry and `engram_report grade` (0-5) self-rates it, advancing an SM-2 schedule (1 → 6 → round(prev × ease) days, reset on failure, ease floor 1.3). Entries under review scheduling **no longer take part in automatic decay** — their fate is decided by recall. Session-start injection reports how many memories are due.
|
|
67
67
|
- **History backfill** (v0.7.3+): distils dsh's persisted past sessions into the palace, turn by turn — each session is written into the project store matching **its own cwd** (no cross-project bleed), reusing the live-capture throttling, redaction, and echo suppression, with a `(session, turn)` idempotency key so an interrupted run resumes. Backfilled entries never enter the due-today queue. **You choose the import rules** (time window / turns per session / total turn budget / auxiliary model / include subagent·seeded·no-cwd sessions) and see a zero-cost estimate (no LLM calls) before running; staging happens in a dedicated **"History backfill" tab** with progress (including a **breakdown of skip reasons**) and pause. Auxiliary calls **reuse the model you are currently using** by default (a past session's log records the provider/model of that time, which may no longer be available here), or you can pick one explicitly from the host's registered providers/models in the tab's "Auxiliary model" dropdown.
|
|
68
68
|
- **Knowledge flywheel**: capture/save → contradiction candidates (high-similarity neighbors create `contradicts` edges on write, for the model/user to adjudicate) → hit reinforcement (confidence +0.05) → distillation (topic clusters merge into higher-level rules, supersedes chains, confidence inheritance) → decay (low-importance, long-unaccessed entries archive; restorable).
|
|
69
|
-
- **Automatic capture** (when `ingest` is enabled): each new turn's first step extracts candidate facts from the previous turn out of the session log, and session end captures the final turn too (failures leave a pending key that the next session replays; capture is idempotent per session+turn), written with low confidence and
|
|
69
|
+
- **Automatic capture** (when `ingest` is enabled): each new turn's first step extracts candidate facts from the previous turn out of the session log, and session end captures the final turn too (failures leave a pending key that the next session replays; capture is idempotent per session+turn), written with low confidence and, when embeddings are available, disposed by the four-state rule toward near neighbors (merge-and-reinforce restatements / edge-and-defer suspected contradictions / write fresh) — memories accumulate without you saying "remember this".
|
|
70
70
|
- **Provenance audit**: every memory records its source session, turn, and event seq; `engram_review` traces the full source chain, supersede chain, contradictions, and operation log; all writes/edits/forgets/distills/decays land in the operation log table.
|
|
71
|
-
- **Web management panel** (v0.7.0+): the "Memory Library" tab in Settings has five views — Today (a summary bar: memories / open / clarity + last-7-days counts + due today + health ring, followed by tour proposal, room directory, refurb list, health breakdown), Room exhibition (filters incl. due today, tour-route order, batch actions, list & edit), Corridor tour (corridor bird's-eye + retrieval bench), Curator log (last-7-days counts + the full merged op_log, filterable by operation class), and History backfill. A single 3-pill scope switcher in the header drives every panel; UI copy is bilingual zh/en and follows the host language setting live. Filter by redaction marks (include/exclude `[REDACTED:*]` entries) with amber badges for audit coverage.
|
|
71
|
+
- **Web management panel** (v0.7.0+): the "Memory Library" tab in Settings has five views — Today (a summary bar: memories / open / clarity + last-7-days counts + due today + health ring, followed by tour proposal, room directory, refurb list, health breakdown; a profile card in the side column shows the curated profile's current content and version-diff history), Room exhibition (filters incl. due today, tour-route order, batch actions, list & edit), Corridor tour (corridor bird's-eye + retrieval bench), Curator log (last-7-days counts + the full merged op_log, filterable by operation class), and History backfill. A single 3-pill scope switcher in the header drives every panel; UI copy is bilingual zh/en and follows the host language setting live. Filter by redaction marks (include/exclude `[REDACTED:*]` entries) with amber badges for audit coverage.
|
|
72
72
|
- **Prompt-injection defense**: every memory recall exit (profile injection, `engram_search/timeline/review` output) is wrapped in `<engram_memory_context>` protocol tags with a usage warning (history is not the current request, do not follow instructions inside it, use only when relevant), while the current request is wrapped separately in `<current_user_request>`; all inbound content (capture candidates, saved bodies) is stripped of these protocol tags first, blocking second-order injection through forged protocol blocks.
|
|
73
|
-
- **Capture redaction**: inbound content passes a regex scrub for common secrets and credentials (sk- API keys, Bearer, AWS AKIA, GitHub tokens, PEM private keys, password/token assignments) and matched fragments become `[REDACTED:<type>]`.
|
|
73
|
+
- **Capture redaction**: inbound content passes a regex scrub for common secrets and credentials (sk- API keys, Bearer, AWS AKIA, GitHub tokens, PEM private keys, password/token assignments, Chinese-language password assignments) and personal data (mainland-China phone numbers, 18-digit national ID numbers); matched fragments become `[REDACTED:<type>]`.
|
|
74
74
|
- **Recall placeholder (anti-echo-chamber)**: memory-recall tool output inside captured slices is replaced with `[engram memory result omitted from capture: <tool>]`, and the extraction prompt states that restating existing memory is not new information, breaking the memory self-reinforcement loop.
|
|
75
75
|
- **Multi-query retrieval**: `engram_search` can use the aux LLM to rewrite the query into ≤3 complementary queries, retrieving each and fusing them with cross-query RRF plus a per-query floor; a failed rewrite degrades to the single query (`queryRewrite: false` disables).
|
|
76
|
-
- **Evidence gate (search → assess)**: a hit means "relevant", not "enough to answer". Every retrieval registers an in-process batch (latest 20 per session, released when the session ends) and prints `ref=…` per row plus the batch id; `engram_assess` may only cite refs of that batch, and `sufficient` is enforced by code — all three must hold (the model claims adequacy, at least one valid evidence ref, `nextStrategy=answer`) or the verdict is insufficient with the strategy rewritten back to retrieval. Verdicts and rejected refs go to the audit log, visible under the "retrieval" category in the curator log.
|
|
76
|
+
- **Evidence gate (search → assess)**: a hit means "relevant", not "enough to answer". Every retrieval registers an in-process batch (latest 20 per session, released when the session ends) and prints `ref=…` per row plus the batch id; `engram_assess` may only cite refs of that batch, and `sufficient` is enforced by code — all three must hold (the model claims adequacy, at least one valid evidence ref, `nextStrategy=answer`) or the verdict is insufficient with the strategy rewritten back to retrieval. Verdicts and rejected refs go to the audit log, visible under the "retrieval" category in the curator log. If unassessed batches remain before the next step, a wrap-up reminder is injected (escalating to a change-strategy hint after ≥2 consecutive insufficient verdicts; `assessReminder: false` disables).
|
|
77
77
|
- **Portable data**: `engram_export` exports Markdown / JSON files in one step, with a redacted-view variant (secondary scrubbing + 40-char preview truncation, share-safe). `engram_mirror` writes an Obsidian / Logseq-friendly mirror directory (one Markdown per memory with YAML frontmatter + `[[id]]` backlinks), turning the palace into a human-readable private knowledge base.
|
|
78
78
|
- **AGI Architecture Exploration (dsh-market · AGI Architecture)**: listed in dsh-market's "AGI Architecture Exploration" category as a cognitive-science reframe of agent long-term memory — memory palace (imagery labels + room placards), corridor topology (force-directed graph), closure questions (the ingest prompt nudges the model to ask clarifying questions), consolidation merging (heuristic dedup + cosine similarity) — sitting alongside MemGPT / Letta in the "agent memory architecture" conversation.
|
|
79
79
|
|
|
80
|
-
## Tools (
|
|
80
|
+
## Tools (19, narrow parameters)
|
|
81
81
|
|
|
82
82
|
| Tool | Purpose |
|
|
83
83
|
|---|---|
|
|
84
|
-
| `engram_save` | Save (
|
|
84
|
+
| `engram_save` | Save (**four-state write disposition**: a restatement merges into and reinforces the existing entry (`merge`, no new node), a suspected contradiction is written with a `contradicts` edge pending adjudication (`defer`), fresh content is written (`accept`), low-entropy content is dropped (`drop`); without embeddings everything is `accept`); `items` array saves ≤10 in one call with shared scrubbing and in-batch dedup, reporting each entry's disposition (`disposition`/`mergedInto`), one failure not blocking the rest (`count`/`items`/`failed` summary); `placard` attaches a marker (scored unique · distinctive · dated; low scores get a rewrite hint) |
|
|
85
85
|
| `engram_search` | Hybrid semantic + keyword retrieval (hits reinforce confidence); `room` routes through the corridor — search inside one room only; the top 5 hits carry same-room neighbouring-slot cues. Output rows carry `id=` and `ref=`; the tail carries the batch id |
|
|
86
86
|
| `engram_assess` | Evidence gate: before answering, judge whether the retrieved content suffices. Submit `batchId` + ≤8 `evidenceRefs` (only `ref=` values from that batch's output) + `missing` + `nextStrategy`; the code requires all three for `sufficient` (claimed adequate, at least one valid in-batch evidence ref, `nextStrategy=answer`), otherwise it rules insufficient and rewrites the strategy back to retrieval; refs from other batches are rejected and listed, and the verdict is written to the audit log |
|
|
87
87
|
| `engram_timeline` | Timeline browsing: creation-time descending by default; `order: 'tour'` follows the fixed tour route's slot order instead (output carries palace coordinates, entries off-route sorted last) so the agent can re-walk the route |
|
|
88
|
+
| `engram_episode_timeline` | Dedicated timeline for episodic memories: browse experiences by date range and source session, grouped by session (newest group first, entries chronological within; `sessionId` drills into one session; group headers carry the one-sentence session summary generated during capture or at session end). With `around` set to a memory id, it switches to temporal-neighborhood expansion — listing episodes within ± a window (`windowMinutes`, default 60) of the anchor's creation time, answering "what else happened around then" |
|
|
88
89
|
| `engram_update` | Correct an entry (supersedes chain); can re-attach a `placard` marker too |
|
|
89
90
|
| `engram_forget` | Forget (soft-delete, restorable) |
|
|
90
91
|
| `engram_report` | Report an outcome (preferred for skill-kind entries): success +0.05 / failure -0.1, and persistently useless memories decay away naturally. With `grade` (0-5) it advances the SM-2 review schedule instead, acting as the self-rating entry point of retrieval practice |
|
|
@@ -95,9 +96,10 @@ Sensory and emotional dimensions (smell, temperature, emotional weight) are deli
|
|
|
95
96
|
| `engram_neighbors` | Corridor walk: from one room, follow 1-3 hops of relation edges and return a neighbour summary |
|
|
96
97
|
| `engram_tour` | Tour routing: `mode=fixed` walks the whole palace in fixed slot order (constant route, sequential extraction); `mode=thematic` plans 3-7 stops around a theme |
|
|
97
98
|
| `engram_audit_forgotten` | Closed-wing archaeology: list recent closed entries with their epitaphs to review whether past forgetting was sound |
|
|
98
|
-
| `engram_ingest_history` | History backfill: distil past dsh sessions into the palace turn by turn (per-cwd stores; already-captured turns skipped). `dryRun` defaults to true (estimate only); pass `dryRun=false` to run. Use the Settings "History backfill" tab for large batches |
|
|
99
|
+
| `engram_ingest_history` | History backfill: distil past dsh sessions into the palace turn by turn (per-cwd stores; already-captured turns skipped). After each session's turns are captured, an auxiliary LLM generates a one-sentence session summary stored for timeline group headers (live sessions get the same summary at dispose once the final turn is captured; failures stay silent and never block). `dryRun` defaults to true (estimate only); pass `dryRun=false` to run. Use the Settings "History backfill" tab for large batches |
|
|
99
100
|
| `engram_export` | Export Markdown / JSON files (data portability); `redactedView: true` emits a redacted view (secondary scrubbing + 40-char preview truncation, share-safe) |
|
|
100
101
|
| `engram_distill` | Distill: merge same-topic clusters into higher-level rules (LLM) |
|
|
102
|
+
| `engram_profile_edit` | Edit the curated profile block: `view` shows the current content and version history; `edit` writes a new version (optimistic lock via `expectedVersion`, conflicts fail loud); `rollback` restores a historical version (appended as a new version; history is never rewritten). The curated profile is injected ahead of the derived profile; every edit lands in the operation log, and the profile card on the panel's Today view shows version diffs |
|
|
101
103
|
|
|
102
104
|
## Configuration
|
|
103
105
|
|
|
@@ -110,7 +112,11 @@ Optional configuration (cordis.yml):
|
|
|
110
112
|
dbDir: '~/.dsh/engram' # store and model-cache root directory
|
|
111
113
|
injectProfile: true # inject the user memory profile at session start
|
|
112
114
|
profileTopN: 8 # injection entry cap (1-64)
|
|
113
|
-
injectTokenBudget: 1024 # injection token budget (128-8192; CJK text counted at 1.5 tokens/char, other text at 4 chars/token;
|
|
115
|
+
injectTokenBudget: 1024 # injection token budget (128-8192; CJK text counted at 1.5 tokens/char, other text at 4 chars/token; entries that still don't fit degrade to index lines)
|
|
116
|
+
injectItemBudgetStart: 160 # graduated budget: body chars for the first entry (40-2000, decayed per entry; truncate first before degrading)
|
|
117
|
+
injectItemBudgetDecay: 0.9 # graduated budget: per-entry decay factor (0.5-1)
|
|
118
|
+
injectItemBudgetFloor: 24 # graduated budget: per-entry body char floor (8-200, must not exceed start)
|
|
119
|
+
assessReminder: true # evidence-gate nudge: inject an engram_assess reminder before the next step when unassessed search batches exist
|
|
114
120
|
modelCacheDir: '~/.dsh/engram/models' # embedding model cache directory
|
|
115
121
|
hfEndpoint: 'https://huggingface.co' # model download endpoint; set a mirror behind restricted networks
|
|
116
122
|
ingest: 'off' # automatic capture: off | light (user messages only, ≤2/turn) | eager (assistant messages too, ≤5/turn)
|
|
@@ -169,6 +175,8 @@ Since v0.7.5: two underlying fixes plus one new capability. ① Profile-injectio
|
|
|
169
175
|
|
|
170
176
|
Since v0.7.6 scopes stop being hardcoded and the project palace follows the workspace. ① **Per-candidate palace on capture**: extraction now emits `scope` — content tied to the current project/repo (tech choices, project conventions, architecture decisions) goes to that project store, while cross-project material (personal preferences, habits, the user's own history) goes to the private store. Live capture (previous turn, final turn on dispose, pending replay) and history backfill share the same rule, and backfill files each session into **its own cwd's** store. Idempotency keys (`ingest-done` / `ingest-pending`) stay in the private store, decoupled from where memories land, so live and backfill share one `(session, turn)` key set and never re-ingest across paths. ② **Project palace follows the workspace**: `GET /api/engram/workspaces` lists the selectable project palaces (host workspace registry → cwds seen in session headers → plugin process directory, deduped by store name, so several worktrees of one repo share a palace) and every project-scoped endpoint takes `?project=<dbName>` (unknown selectors answer 404); `engram_save/search` and friends resolve their project store from the current session's cwd. In the panel, a new row under the three scope segments shows the **active workspace chip plus a workspace picker** (default "follow the current workspace", pinnable to one workspace) and switching workspaces switches every view. ③ Without git, the project store name changes from "first 12 chars of cwd" (which collided for sibling directories) to a **full cwd sha256**, with the old store renamed on startup; and unloading the plugin now closes every store connection (no more locked `.db` files on Windows).
|
|
171
177
|
|
|
178
|
+
Since v0.7.9 the memory-tiers phase begins. ① **Editable profile**: the new `engram_profile_edit` tool (19 tools) maintains the curated profile — the memory you explicitly vouch for — with `view` (current content + version history), `edit` (optimistic lock, conflicts fail loud), and `rollback` (restores a historical version as a new append; history is never rewritten); on injection the curated text runs ahead of the derived profile and claims the token budget first, every edit lands in the operation log, and the profile card on the panel's Today view shows version diffs. ② **Dedicated episode timeline**: the new `engram_episode_timeline` tool browses experiences by date range and source session, and with `around` set to an anchor id it switches to temporal-neighborhood expansion (episodes within ± a window of the anchor's moment), answering "what else happened around then". ③ **Panel "Past timeline" view**: the memory-library tab gains a sixth view listing episodes grouped by session, with a "Nearby" action per entry that expands the temporal-neighborhood card in one click. ④ **Session-summary group headers**: history backfill asks an auxiliary LLM for a one-sentence summary after each session's turns are captured, and live sessions do the same once the final-turn capture on dispose succeeds (reusing the active route; failures stay silent) into a `session_summaries` table; timeline group headers carry it (visible in tool output, the API, and the panel header alike), so reviewing the timeline no longer means expanding every entry.
|
|
179
|
+
|
|
172
180
|
## Development
|
|
173
181
|
|
|
174
182
|
```sh
|
|
@@ -188,7 +196,7 @@ Each turn's first step appends a plugin-source user snapshot: `User memory profi
|
|
|
188
196
|
|
|
189
197
|
#### Token effect
|
|
190
198
|
|
|
191
|
-
Profile injection is a conditional fixed cost (bounded by both the entry cap and the token budget); the tool schemas are a standing cost (
|
|
199
|
+
Profile injection is a conditional fixed cost (bounded by both the entry cap and the token budget); the tool schemas are a standing cost (19 narrow-parameter tools).
|
|
192
200
|
|
|
193
201
|
#### KV Cache effect
|
|
194
202
|
|
|
@@ -199,6 +207,14 @@ Profile text changes as the memory store changes — changes only land at turn b
|
|
|
199
207
|
- **No LLM adjudication of contradiction candidates** — writes only report candidates by vector similarity (≥0.88) and create edges; semantic-contradiction confirmation is left to model/user adjudication and distillation.
|
|
200
208
|
- **Memories written during embedder degradation have no vectors** — memories written before the model is ready do not participate in the semantic track; after semantics come online run `pnpm backfill` once to backfill existing vectors (after `pnpm build` has warmed the model cache; `HF_ENDPOINT` configurable).
|
|
201
209
|
|
|
210
|
+
## Roadmap
|
|
211
|
+
|
|
212
|
+
The palace IA (position-as-index / fixed routes / retrieval practice) is the indexing and audit skeleton; it evolves toward AI-native memory tiers and a temporal fact graph in three phases. Actual releases increment by 0.0.1; the phase labels are planning codenames only:
|
|
213
|
+
|
|
214
|
+
- **Phase 1 · Flywheel closure** (done, v0.7.8): four-state write disposition (ACCEPT new / MERGE reinforce-in-place / DROP low-entropy / DEFER contradiction-pending — `engram_save` and capture return it per item); graduated decaying injection budget (160 chars for the first entry, ×0.9 per subsequent entry, floor 24, all configurable); redaction extended to Chinese passwords, national ID numbers, and phone numbers; evidence-gate wrap-up reminder (inject a reminder when a turn ends with unassessed search batches).
|
|
215
|
+
- **Phase 2 · Memory tiers**: editable profile (**done, v0.7.9**: `engram_profile_edit` tool with version-chain rollback and panel diff audit, following Letta's core-memory model); a dedicated timeline index for episode memories (**done, v0.7.9**: `engram_episode_timeline` tool with date ranges, per-session grouping, temporal-neighborhood expansion, and a dedicated episode index; the panel "Past timeline" view plus capture-time and live-session summary group headers landed in the same batch); automated distillation (cluster size + similarity thresholds, suggest/auto modes); configurable room capacity and placard scoring (defaults unchanged).
|
|
216
|
+
- **Phase 3 · Temporal fact graph**: entity table with write-time extraction and resolution; fact-level triples carrying `valid_at` / `invalid_at` windows with soft-invalidated conflicts (Zep/Graphiti model); automatic supports / refines / related edge creation; a LoCoMo-zh evaluation subset in CI as a retrieval-quality regression gate (deterministic Recall@k / MRR metrics).
|
|
217
|
+
|
|
202
218
|
## Acknowledgements
|
|
203
219
|
|
|
204
220
|
Thanks to the community contributors who made this project better:
|
package/README.md
CHANGED
|
@@ -58,7 +58,7 @@ dsh plugin --profile web add @kenz1117/dsh-engram
|
|
|
58
58
|
|
|
59
59
|
## 特性
|
|
60
60
|
|
|
61
|
-
- **跨会话记忆**:会话开始注入用户画像摘要(条数 + token
|
|
61
|
+
- **跨会话记忆**:会话开始注入用户画像摘要(条数 + token 预算双重上限,可配;`engram_profile_edit` 维护的 curated 画像优先于派生事实),Agent 天然"记得"你是谁、在做什么;工具检索跨会话召回历史事实。
|
|
62
62
|
- **双层分库**:`user.db` 全局共享;`project-<hash>.db` 按 git origin 标识隔离(无 git 时回退 cwd 全量哈希,旧命名库自动迁移)——个人偏好跟人走,项目约定跟仓库走。**项目宫殿随工作区切换**:管理面板「项目」scope 默认跟随 GUI 当前选中的工作区(Header 显示工作区标题与路径),也可在下拉里固定到某个工作区;`engram_save/search` 等工具的 project 读写同样按当前会话 cwd 归属,与面板同一口径。
|
|
63
63
|
- **混合检索**:FTS5(unicode61 + 中文 2-gram 预切词)与本地向量(`Xenova/bge-small-zh-v1.5`,512 维,q8)RRF 融合 + 关系边一跳扩展 + 新鲜度/命中次数乘性排序 boost;嵌入模型离线运行,下载失败自动降级纯关键词并显式标记。
|
|
64
64
|
- **记忆宫殿信息架构**(v0.7.2+):四原则全部落进核心路径,而非展示层皮肤——**位置当索引**(写入按 kind 分房并钉「房间#桩位」坐标,房间容量 9,满员开新房,桩位只增不回收);**固定路线定顺序**(`tour_routes` append-only,`engram_tour mode=fixed` 按桩位顺序走全宫);**骨架长期复用**(同一 topic 永远落在同一房间同一序号,顺序提取而非重新检索);**标记独一无二**(门牌规则 0-1 评分:全库唯一 +0.4 / 带日期锚点 +0.3 / 同房前 6 字不重复 +0.3,低分进翻新清单;库内 active 记忆少于 8 条时不扫描,避免小库噪声)。
|
|
@@ -66,25 +66,26 @@ dsh plugin --profile web add @kenz1117/dsh-engram
|
|
|
66
66
|
- **检索练习闭环**(间隔重复):`engram_review_queue` 只给宫殿坐标与门牌线索、**不给正文**,迫使模型先主动回忆;`engram_review` 揭示核对,`engram_report grade`(0-5)自评推进 SM-2 调度(1 → 6 → round(prev × ease) 天,失败重置,ease 下限 1.3)。进入复习调度的条目**不再参与自动衰减**——命运由回忆结果决定。会话开始注入会提示今日待回忆条数。
|
|
67
67
|
- **历史会话回填**(v0.7.3+):把 dsh 已持久化的历史会话逐轮提炼进宫殿——默认按**每个会话自己的 cwd** 写进对应项目库(不串库),同样逐条判作用域(跨项目通用的个人偏好落私人库),复用实时摄取的节流/脱敏/防回声,靠 (会话, 轮次) 幂等键支持中断续跑(键固定在私人库,与实时路径共用);回填条目不进今日复习队列(避免一次性回填淹没「今日待回忆」)。**导入规则由你选**(时间窗 / 单会话轮数 / 总轮数上限 / 辅助模型 / 是否含子代理·种子·无 cwd 会话),先估算(零成本、不调 LLM)再执行;设置页有独立的**「历史回填」tab**,可看进度(含**跳过原因分布**)与暂停续做。辅助调用默认**复用你当前在用的模型**(历史日志里记的是当年的 provider/model,在当前环境可能已不可用),也可在面板「辅助模型」下拉里从宿主已注册的 provider/model 中直接指定。
|
|
68
68
|
- **知识飞轮**:摄取/保存 → 矛盾候选(写入时高相似近邻建 `contradicts` 边并报告,模型/用户裁决)→ 命中强化(confidence +0.05)→ 蒸馏(同主题簇合并为高层规律、supersedes 取代链、置信度继承)→ 衰减(低重要性且长期未访问归档,可恢复)。
|
|
69
|
-
- **自动摄取**(`ingest` 配置开启时):新一轮第一步从会话日志提取上一轮的候选事实,会话结束时补摄取最后一轮(失败留 pending 键,下次会话自动补做,幂等不重复),低 confidence
|
|
69
|
+
- **自动摄取**(`ingest` 配置开启时):新一轮第一步从会话日志提取上一轮的候选事实,会话结束时补摄取最后一轮(失败留 pending 键,下次会话自动补做,幂等不重复),低 confidence 写入,嵌入可用时按四态处置近邻(复述并入强化 / 疑似矛盾建边待裁决 / 全新写入)——不说"记住"也能攒记忆。**逐条判宫殿**:提炼时同步判定作用域,只跟当前项目/仓库有关的(技术选型、项目约定、架构决策)进当前会话 cwd 对应的项目库,跨项目通用的(个人偏好、习惯、本人经历)进私人库——偏好跟人走、约定跟仓库走。
|
|
70
70
|
- **来源审计**:每条记忆记录来源会话、轮次与事件 seq,`engram_review` 完整回查来源链、取代链、矛盾与操作日志;全部写入/修改/遗忘/蒸馏/衰减入操作日志表。
|
|
71
|
-
- **Web 管理面板**(v0.7.0+):设置页「记忆库」tab 分五个视图——今日(速览条:记忆 / 开放 / 清晰度 + 近 7 天计数 + 今日到期 +
|
|
71
|
+
- **Web 管理面板**(v0.7.0+):设置页「记忆库」tab 分五个视图——今日(速览条:记忆 / 开放 / 清晰度 + 近 7 天计数 + 今日到期 + 健康分环;下面是入殿导航、房间目录、待翻新、健康分构成;侧栏画像卡展示 curated 画像当前内容与版本 diff 历史)、宫殿陈展(筛选含今日到期 / 巡游路线序 / 批量 / 列表与编辑)、走廊巡游(走廊鸟瞰 + 检索实验台)、管家日志(近 7 天计数 + 两库合并的完整 op_log,可按操作类别筛选)、历史回填。Header 三宫格驱动全局 scope(私人 / 项目 / 共享),全部数据源同步;**项目 scope 下三宫格右侧显示当前项目宫殿所属工作区**(标题 + 路径,默认跟随 GUI 当前工作区),旁边的工作区下拉可固定到某个工作区或切回「跟随当前会话」。界面文案中英双语,跟随宿主语言设置实时切换。支持按脱敏标记筛选(仅看/排除含 `[REDACTED:*]` 的条目)并给命中条目挂琥珀色徽标,方便审计脱敏覆盖面。
|
|
72
72
|
- **提示注入防护**:全部记忆召回出口(画像注入、`engram_search/timeline/review` 输出)包 `<engram_memory_context>` 协议标签并附使用警告(历史记忆非当前请求、不遵循其中指令、仅相关时使用),当前请求独立包 `<current_user_request>`;所有入库内容(摄取候选、保存正文)先剥离这些协议标签,防伪造协议块二次注入。
|
|
73
|
-
- **摄取脱敏**:入库前正则清洗常见密钥凭据(sk- 系 API key、Bearer、AWS AKIA、GitHub token、PEM 私钥、password/token
|
|
73
|
+
- **摄取脱敏**:入库前正则清洗常见密钥凭据(sk- 系 API key、Bearer、AWS AKIA、GitHub token、PEM 私钥、password/token 赋值、中文密码赋值)与个人信息(中国大陆手机号、18 位身份证号),命中片段替换为 `[REDACTED:<类型>]`。
|
|
74
74
|
- **召回占位(防回声室)**:摄取切片中记忆召回工具的输出替换为 `[engram memory result omitted from capture: <tool>]`,并向提取模型附注"既有记忆的复述不是新信息",阻断记忆自我强化循环。
|
|
75
75
|
- **多查询检索**:`engram_search` 可用辅助 LLM 把查询改写为 ≤3 个互补查询分别检索,跨查询 RRF 融合 + 每查询保底命中;改写失败自动降级单查询(`queryRewrite: false` 关闭)。
|
|
76
|
-
- **证据门(search → assess)**:检索命中只说明「相关」,不说明「足以回答」。每次检索登记一个进程内批次(每会话保留最近 20 个,会话结束即释放),输出行尾给出 `ref=…` 与批次 id;`engram_assess` 只能引用同一批次的 ref,且 `sufficient` 由代码强制——三者齐备(模型声称充足、至少一条有效证据、`nextStrategy=answer
|
|
76
|
+
- **证据门(search → assess)**:检索命中只说明「相关」,不说明「足以回答」。每次检索登记一个进程内批次(每会话保留最近 20 个,会话结束即释放),输出行尾给出 `ref=…` 与批次 id;`engram_assess` 只能引用同一批次的 ref,且 `sufficient` 由代码强制——三者齐备(模型声称充足、至少一条有效证据、`nextStrategy=answer`)才算充足,否则判为不足并把策略改回继续检索。判定与拒绝明细写入审计日志,面板「管家日志」的「检索」类别可见。下一步开始前若仍有未判定批次,注入收尾提醒引导补判或说明不判(连续不足 ≥2 次时建议换检索方式或询问用户;`assessReminder: false` 关闭)。
|
|
77
77
|
- **数据可携带**:`engram_export` 一键导出 Markdown / JSON 文件,支持脱敏视图(内容二次清洗 + 预览截断,分享安全)。`engram_mirror` 导出可漫游的镜像目录(Obsidian / Logseq 友好:每条记忆一个 Markdown,正文 + YAML frontmatter + 双向链接 `[[id]]`),让「宫殿」也成为可人读的私人知识库。
|
|
78
78
|
- **认知架构探索(dsh-market · AGI 架构探索)**:本仓库是 dsh-market「AGI 架构探索」类目下,对 agent 长期记忆的认知科学方法论重构——记忆宫殿(意象标签 + 房间铭牌)、走廊拓扑(力导向图)、闭环提问(摄入时让模型主动追问用户细节)、巩固合并(启发式去重 + 余弦相似度),与 MemGPT/Letta 同层「agent 记忆架构」叙事。
|
|
79
79
|
|
|
80
|
-
## 工具(
|
|
80
|
+
## 工具(19 个,窄参数)
|
|
81
81
|
|
|
82
82
|
| 工具 | 作用 |
|
|
83
83
|
|---|---|
|
|
84
|
-
| `engram_save` |
|
|
84
|
+
| `engram_save` | 保存(**写入四态回报**:复述并入强化既有条目(merge,不新建)、疑似矛盾落库建边待裁决(defer)、全新写入(accept),低熵内容丢弃(drop);嵌入不可用时全部 accept);支持 `items` 数组单次批量保存 ≤10 条,统一清洗/批量内去重,逐条回报处置(`disposition`/`mergedInto`),单条失败不影响其余(`count`/`items`/`failed` 汇总返回);`placard` 挂门牌(按唯一·差异化·带日期评分,低分附改写建议)。`scope=project` 落当前会话 cwd 对应的项目宫殿 |
|
|
85
85
|
| `engram_search` | 语义 + 关键词混合检索(命中强化置信度);`room` 参数做走廊路由——只在指定房间内检索;命中 top5 附同房相邻桩位线索。输出行尾给 `id=` 与 `ref=`,末尾给批次 id。`scope=project` 查当前会话 cwd 对应的项目宫殿 |
|
|
86
86
|
| `engram_assess` | 证据门:作答前判定「检索到的内容是否足以回答」。提交 `batchId` + ≤8 条 `evidenceRefs`(只能取该批次输出里的 `ref=`)+ `missing` + `nextStrategy`;代码强制 `sufficient` 需同时满足「声称充足」「至少一条属于本批次的有效证据」「nextStrategy=answer」,否则判为不足并把策略改回继续检索;非本批次的 ref 会被拒绝并列出,判定写入审计日志 |
|
|
87
87
|
| `engram_timeline` | 时间线浏览:默认按创建时间倒序;`order: 'tour'` 改按固定巡游路线桩位顺序(输出附宫殿坐标,未上路线者排末尾),让 agent 也能沿固定路线复述 |
|
|
88
|
+
| `engram_episode_timeline` | episode 情景独立时间线:按日期范围与来源会话浏览经历,输出按会话分组(组间新→旧、组内时间升序,`sessionId` 可精确回查单会话,组头附摄取期或会话结束时生成的一句话会话摘要);`around` 传锚点记忆 id 时切换为时间邻近扩展——列出锚点时刻 ± 窗口(`windowMinutes`,默认 60)内的情景,回答「当时前后还发生了什么」 |
|
|
88
89
|
| `engram_update` | 修正(supersedes 取代链);可同时改挂 `placard` 门牌 |
|
|
89
90
|
| `engram_forget` | 遗忘(软删可恢复) |
|
|
90
91
|
| `engram_report` | 回报使用效果(skill 类首选):success 提权 +0.05 / failure 降权 -0.1,持续无效的记忆被衰减自然淘汰。传 `grade`(0-5)则按 SM-2 推进复习调度,作为检索练习的自评入口 |
|
|
@@ -95,9 +96,10 @@ dsh plugin --profile web add @kenz1117/dsh-engram
|
|
|
95
96
|
| `engram_neighbors` | 走廊漫步:从一间出发走 1-3 跳关系边,返回邻居简表 |
|
|
96
97
|
| `engram_tour` | 巡游路由:`mode=fixed` 按固定桩位路线走全宫(路线恒定,顺序提取);`mode=thematic` 按主题动态规划 3-7 站 |
|
|
97
98
|
| `engram_audit_forgotten` | 闭馆考古:列最近已闭馆条目与墓志铭,复核过去的遗忘是否得当 |
|
|
98
|
-
| `engram_ingest_history` | 历史会话回填:把 dsh 历史会话逐轮提炼进宫殿(按会话 cwd
|
|
99
|
+
| `engram_ingest_history` | 历史会话回填:把 dsh 历史会话逐轮提炼进宫殿(按会话 cwd 分库、已摄取轮次自动跳过)。每会话轮次摄取完成后用辅助 LLM 生成一句话会话摘要落库(失败静默不阻断),实时会话结束时同样收尾生成,供时间线组头回看。`dryRun` 缺省 true 只估算;`dryRun=false` 才执行。大批量建议用设置页「历史回填」tab |
|
|
99
100
|
| `engram_export` | 导出 Markdown / JSON 文件(数据可携带);`redactedView: true` 输出脱敏视图(二次清洗 + 40 字预览截断,可安全分享) |
|
|
100
101
|
| `engram_distill` | 蒸馏:同主题簇合并为高层规律(LLM) |
|
|
102
|
+
| `engram_profile_edit` | 画像 curated block 编辑:`view` 看当前内容与版本历史;`edit` 写入新版本(乐观锁 `expectedVersion`,冲突 loud 失败);`rollback` 回滚到历史版本(作为新版本追加,历史永不改写)。curated 画像优先于派生画像注入;编辑记录入操作日志,面板今日页画像卡可查版本 diff |
|
|
101
103
|
|
|
102
104
|
## 配置
|
|
103
105
|
|
|
@@ -110,7 +112,11 @@ dsh plugin --profile web add @kenz1117/dsh-engram
|
|
|
110
112
|
dbDir: '~/.dsh/engram' # 分库与模型缓存根目录
|
|
111
113
|
injectProfile: true # 会话开始注入用户画像摘要
|
|
112
114
|
profileTopN: 8 # 注入条数上限(1-64)
|
|
113
|
-
injectTokenBudget: 1024 # 注入 token 预算(128-8192,中文按 1.5 token/字、其余按 4 字符/token
|
|
115
|
+
injectTokenBudget: 1024 # 注入 token 预算(128-8192,中文按 1.5 token/字、其余按 4 字符/token 估算,装不下的条目降级为索引行)
|
|
116
|
+
injectItemBudgetStart: 160 # 分级递减预算:首条正文字符数(40-2000,逐条按 decay 递减、装不下先截断)
|
|
117
|
+
injectItemBudgetDecay: 0.9 # 分级递减预算:逐条递减系数(0.5-1)
|
|
118
|
+
injectItemBudgetFloor: 24 # 分级递减预算:单条正文字符下限(8-200,不得超过 start)
|
|
119
|
+
assessReminder: true # 证据门收尾提醒:存在未判定检索批次时在下一步开始前注入 engram_assess 提醒
|
|
114
120
|
modelCacheDir: '~/.dsh/engram/models' # 嵌入模型缓存目录
|
|
115
121
|
hfEndpoint: 'https://huggingface.co' # 模型下载端点,网络受限可配镜像
|
|
116
122
|
ingest: 'off' # 自动摄取:off | light(仅用户消息,每轮≤2条)| eager(含助手消息,每轮≤5条)
|
|
@@ -170,6 +176,8 @@ v0.7.5 起是两处底层修正加一层新能力。① 画像注入的 token
|
|
|
170
176
|
|
|
171
177
|
v0.7.6 起把「作用域」从写死改成逐条判、并让项目宫殿跟着工作区走。① **摄取逐条判宫殿**:提炼时同步判定 `scope`——只跟当前项目/仓库有关的(技术选型、项目约定、架构决策)进该项目库,跨项目通用的(个人偏好、习惯、本人经历)进私人库;实时摄取(上一轮 / 会话结束末轮 / 待补做重放)与历史回填同一口径,回填按**每条会话自己的 cwd** 落库。幂等键(`ingest-done` / `ingest-pending`)固定在私人库,与写入落点解耦,所以实时与回填共用一份 (会话, 轮次) 键、跨路径不重复摄取。② **项目宫殿随工作区切换**:新增 `GET /api/engram/workspaces`(来源 = 宿主工作区注册表 → 会话 header 里出现过的 cwd → 插件进程目录兜底,按分库名去重,同仓库多 worktree 共用一个宫殿),所有项目 scope 的接口接受 `?project=<dbName>` 选择器,未知选择器回 404;`engram_save/search` 等工具的 project 读写改按当前会话 cwd 归属。面板的「项目」作用域在作用域三宫格下方新增一行:**当前工作区 chip + 工作区下拉**(默认「跟随当前工作区」,可临时固定到某个工作区),切工作区即整体切换。③ 无 git 时的项目分库名从「cwd 前 12 字符编码」(同前缀目录会撞库)改为 **cwd 全量 sha256**,旧命名库启动时自动 rename 迁移;插件卸载时关闭所有分库连接(Windows 上不再锁住 .db)。
|
|
172
178
|
|
|
179
|
+
v0.7.9 起进入记忆分层阶段。① **画像可编辑**:新增 `engram_profile_edit` 工具(工具 19 个),curated 画像是你显式维护的最高优先记忆——`view` 看当前内容与版本历史、`edit` 写入新版本(乐观锁冲突 loud 失败)、`rollback` 回滚到历史版本(作为新版本追加,历史永不改写);注入时 curated 全文置于派生画像之前并优先占用 token 预算,编辑记录入操作日志,面板今日页画像卡可查版本 diff。② **episode 情景独立时间线**:新增 `engram_episode_timeline` 工具,按日期范围与来源会话浏览经历,`around` 传锚点 id 切换为时间邻近扩展(锚点时刻 ± 窗口内的情景),回答「当时前后还发生了什么」。③ **面板「往事时间线」视图**:设置页记忆库 tab 新增第六个视图,按会话分组浏览 episode,点条目「邻近」一键展开时间邻近卡。④ **会话摘要组头**:历史回填在每会话摄取完成后、实时会话在 disposed 末轮摄取成功后,用辅助 LLM 生成一句话摘要(复用当前路由,失败静默不阻断)存入 `session_summaries` 表;时间线组头携带摘要(工具输出、API、面板组头三处可见),回看时间线不必逐条展开。
|
|
180
|
+
|
|
173
181
|
## 开发
|
|
174
182
|
|
|
175
183
|
```sh
|
|
@@ -189,7 +197,7 @@ pnpm bundle
|
|
|
189
197
|
|
|
190
198
|
#### Token effect
|
|
191
199
|
|
|
192
|
-
画像注入为条件性固定成本(受条数上限与 token 预算双重约束);工具 schema 为常驻成本(
|
|
200
|
+
画像注入为条件性固定成本(受条数上限与 token 预算双重约束);工具 schema 为常驻成本(19 个窄参数工具)。
|
|
193
201
|
|
|
194
202
|
#### KV Cache effect
|
|
195
203
|
|
|
@@ -200,6 +208,14 @@ pnpm bundle
|
|
|
200
208
|
- **矛盾候选无 LLM 判定** —— 写入时仅按向量相似度(≥0.88)报告候选并建边,语义矛盾的确认留给模型/用户裁决与蒸馏。
|
|
201
209
|
- **嵌入器降级期间的记忆无向量** —— 模型未就绪时写入的记忆不参与语义道;语义上线后跑一次 `pnpm backfill` 补算存量向量(`pnpm build` 的模型缓存就绪后执行,可经 `HF_ENDPOINT` 配镜像)。
|
|
202
210
|
|
|
211
|
+
## 路线图
|
|
212
|
+
|
|
213
|
+
宫殿 IA(位置当索引 / 固定路线 / 检索练习)是索引与审计骨架,按三阶段向 AI 原生的记忆分层与时序事实图谱演进;实际发版按 0.0.1 递增,阶段代号仅为规划标签:
|
|
214
|
+
|
|
215
|
+
- **阶段一 · 飞轮闭环补强**(已完成,v0.7.8):写入四态回报(ACCEPT 新增 / MERGE 并入强化 / DROP 低熵 / DEFER 矛盾待裁决,`engram_save` 与摄取逐条返回处置);画像注入改分级递减预算(首条 160 字、逐条 ×0.9、下限 24,可配置);脱敏扩充中文密码 / 身份证 / 手机号三类;证据门收尾提醒(轮次结束存在未判定 assess 批次时注入提醒)。
|
|
216
|
+
- **阶段二 · 记忆分层**:画像可编辑(**已完成,v0.7.9**:`engram_profile_edit` 工具 + 版本链回滚 + 面板 diff 审计,Letta core-memory 路线);episode 情景记忆独立时间线索引(**已完成,v0.7.9**:`engram_episode_timeline` 工具,日期范围、按会话分组、时间邻近扩展,episode 专用索引;面板「往事时间线」视图 + 摄取期与实时会话摘要组头同批完成);蒸馏自动化(簇规模 + 相似度阈值,suggest/auto 两档);房间容量与门牌评分可配置化(默认与现状一致)。
|
|
217
|
+
- **阶段三 · 时序事实图谱**:实体表与写入期实体抽取消歧;事实级三元组携带 `valid_at` / `invalid_at` 时间窗,冲突事实软失效不删除(Zep/Graphiti 路线);supports / refines / related 自动建边;LoCoMo-zh 评测集进 CI 作检索质量回归门(确定性 Recall@k / MRR 指标)。
|
|
218
|
+
|
|
203
219
|
## 致谢
|
|
204
220
|
|
|
205
221
|
感谢社区贡献者让这个项目更好:
|