dsh-layered-memory 0.6.1 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +106 -81
- package/README.md +93 -80
- package/assets/img/Hero.png +0 -0
- package/assets/img/Layers.png +0 -0
- package/assets/img/Modes.png +0 -0
- package/assets/img/ui-dark.jpg +0 -0
- package/assets/img/ui-light.jpg +0 -0
- package/assets/readme/flow.svg +189 -0
- package/assets/readme/storage.svg +115 -0
- package/dist/client.js +1100 -402
- package/dist/config.d.ts +12 -0
- package/dist/config.js +20 -17
- package/dist/hooks/capture.d.ts +17 -1
- package/dist/hooks/capture.js +45 -11
- package/dist/hooks/recall.d.ts +7 -0
- package/dist/hooks/recall.js +20 -4
- package/dist/index.d.ts +8 -0
- package/dist/index.js +107 -44
- package/dist/pipeline/rebuild.d.ts +2 -0
- package/dist/pipeline/rebuild.js +7 -0
- package/dist/pipeline/runner.d.ts +7 -1
- package/dist/pipeline/runner.js +22 -7
- package/dist/runtime-package-lock.json +982 -0
- package/dist/settings.js +79 -20
- package/dist/stats.d.ts +7 -1
- package/dist/stats.js +131 -14
- package/dist/store/download-queue.d.ts +71 -0
- package/dist/store/download-queue.js +313 -0
- package/dist/store/embedding-source.d.ts +160 -0
- package/dist/store/embedding-source.js +421 -0
- package/dist/store/embedding.d.ts +3 -1
- package/dist/store/embedding.js +5 -0
- package/dist/store/l0.d.ts +13 -5
- package/dist/store/l0.js +40 -10
- package/dist/store/l1.d.ts +15 -5
- package/dist/store/l1.js +47 -19
- package/dist/store/local-embedding.d.ts +64 -0
- package/dist/store/local-embedding.js +120 -0
- package/dist/store/model-catalog.d.ts +45 -0
- package/dist/store/model-catalog.js +78 -0
- package/dist/store/runtime-installer.d.ts +65 -0
- package/dist/store/runtime-installer.js +220 -0
- package/dist/store/sqlite.d.ts +68 -7
- package/dist/store/sqlite.js +422 -73
- package/dist/tools/index.js +6 -4
- package/dist/util/filelog.d.ts +2 -0
- package/dist/util/filelog.js +20 -3
- package/package.json +1 -1
- package/assets/readme/hero.svg +0 -58
package/README.en.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
[简体中文](README.md) | **English**
|
|
2
2
|
|
|
3
3
|
<p align="center">
|
|
4
|
-
<img src="./assets/
|
|
5
|
-
alt="
|
|
4
|
+
<img src="./assets/img/Hero.png" width="100%"
|
|
5
|
+
alt="DeepSeek Harness hero banner: conversations distilled into layered memories and recalled before every model step — chat bubbles dissolve into three progressively brighter light layers flowing into a frosted-glass capsule with a glowing orb and gradient track (tick labels: 日常·工作·智能·关闭), with light threads looping back to suggest recall">
|
|
6
6
|
</p>
|
|
7
7
|
|
|
8
8
|
# dsh-layered-memory
|
|
@@ -11,57 +11,46 @@ A **layered distillation memory plugin** for DeepSeek Harness (persistent compos
|
|
|
11
11
|
plugin): conversations are processed in the background through L0 capture → L1 atomic
|
|
12
12
|
memories → L2 scene consolidation → L3 persona distillation, and relevant memories are
|
|
13
13
|
automatically injected into context before every model step — neither the user nor the
|
|
14
|
-
model needs to do anything.
|
|
15
|
-
[MemoryCore](https://github.com/TencentDB-Agent-Memory) (TencentDB Agent Memory):
|
|
16
|
-
prompts are kept as-is; only the L2/L3 "LLM manipulates files" flow is adapted to
|
|
17
|
-
"LLM outputs, engineering side executes".
|
|
14
|
+
model needs to do anything.
|
|
18
15
|
|
|
19
|
-
|
|
16
|
+
> The core memory capabilities of this plugin (the L0–L3 layered distillation pipeline,
|
|
17
|
+
> prompts, and dual-write storage design) are modeled after **MemoryCore** from
|
|
18
|
+
> [TencentDB-Agent-Memory](https://github.com/TencentCloud/TencentDB-Agent-Memory):
|
|
19
|
+
> prompts are kept as-is; only the L2/L3 "LLM manipulates files" flow is adapted to
|
|
20
|
+
> "LLM outputs, engineering side executes".
|
|
20
21
|
|
|
21
|
-
|
|
22
|
+
## Runtime Data Flow
|
|
22
23
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
| L2 | New memories consolidated into Markdown scene documents (META blocks, heat management, merge caps), consolidated per family | `scenes/chat/*.md`, `scenes/work/*.md` |
|
|
28
|
-
| L3 | Persona distilled from changed scenes (chat: user persona ≤2000 chars; work: Team Operating Doctrine ≤1200 chars), one per family | `persona-chat.md`, `persona-work.md` |
|
|
24
|
+
<p align="center">
|
|
25
|
+
<img src="./assets/readme/flow.svg" width="100%"
|
|
26
|
+
alt="Runtime data flow: session events from User and Assistant (left) flow into the plugin (L0 capture, L1–L3 distillation, retrieval, memory tools), which injects relevant memories into the DeepSeek Harness core (right) at agent/pre-step; distillation reuses the core's ctx.llm and data is dual-written to ~/.dsh/memory/">
|
|
27
|
+
</p>
|
|
29
28
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
29
|
+
The plugin attaches to DSH-native event seams (`session/event` for capture,
|
|
30
|
+
`agent/pre-step` for injection), reuses the host's `ctx.llm` for distillation, and stays
|
|
31
|
+
fully transparent to both user and model. It also registers three model-callable memory
|
|
32
|
+
tools: `memory_search` / `conversation_search` / `memory_read_scene`.
|
|
33
33
|
|
|
34
|
-
|
|
34
|
+
## Layered Memory (L0–L3)
|
|
35
|
+
|
|
36
|
+
<p align="center">
|
|
37
|
+
<img src="./assets/img/Layers.png" width="100%"
|
|
38
|
+
alt="Four memory layers refining from top-left to bottom-right: L0 raw conversation (chat bubbles) → L1 atomic memories (glowing fact particles) → L2 scene blocks (glass document slabs) → L3 core persona (radiant crystal core); stages connected by LLM extract/consolidate/distill light beams, shrinking width shows progressive refinement">
|
|
39
|
+
</p>
|
|
35
40
|
|
|
36
|
-
|
|
37
|
-
same mode**:
|
|
41
|
+
## Per-Session Memory Modes
|
|
38
42
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
| `work` | Narrow prompt with work 4 types only, work family only | work memories only + work persona/scene navigation |
|
|
44
|
-
| `Off` | No L0 writes, no distillation | No recall; the three memory tools return a "cloaked" notice |
|
|
43
|
+
<p align="center">
|
|
44
|
+
<img src="./assets/img/Modes.png" width="100%"
|
|
45
|
+
alt="Per-session memory modes: a glass capsule track with four stops (日常 · 工作 · 智能 · 关闭), the glowing orb resting on 智能 (default); a vignette above each — personal chat bubbles for 日常, code/document panes for 工作, two streams merging brightest for 智能, a dim dashed ghost bubble for 关闭">
|
|
46
|
+
</p>
|
|
45
47
|
|
|
46
48
|
- **Control**: the pill next to the mode selector in the input bar (`Memory · Auto`);
|
|
47
49
|
clicking opens a macOS-style sliding picker above — release to snap to the nearest
|
|
48
|
-
mode;
|
|
49
|
-
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
stay in their original family;
|
|
53
|
-
- Stacks with the global switches (global is the master gate); L2/L3 are fully
|
|
54
|
-
family-isolated — content never leaks across families.
|
|
55
|
-
|
|
56
|
-
### Memory Browser (Settings → Memory)
|
|
57
|
-
|
|
58
|
-
Multi-tab page with a mixed view of both families: **Overview** (per-layer counts +
|
|
59
|
-
memory mode switch panel, auto-refresh every 5s), **Memories** (L1 card list with
|
|
60
|
-
keyword/type/scene filters), **Scenes** (L2 full text), **Persona** (L3 full text),
|
|
61
|
-
**Log** (last 200 lines of `memory.log`). Switches go through the official settings
|
|
62
|
-
service (namespace `dsh-memory`, effective immediately, persisted across restarts);
|
|
63
|
-
effective rule = **static config (deployment ceiling) AND runtime switch**; data
|
|
64
|
-
channel is loopback RPC (`dsh-memory/*`).
|
|
50
|
+
mode; adapts to light/dark themes;
|
|
51
|
+
- Each session's choice is persisted by sessionId to `session-modes.json`, surviving
|
|
52
|
+
restarts/session restore; stacks with the global switches (global is the master gate);
|
|
53
|
+
L2/L3 are fully family-isolated — content never leaks across families.
|
|
65
54
|
|
|
66
55
|
## Getting Started
|
|
67
56
|
|
|
@@ -69,7 +58,7 @@ Requires Node ≥ 22.16. Two invocation styles — the `npx` prefix can replace
|
|
|
69
58
|
any command below:
|
|
70
59
|
|
|
71
60
|
```bash
|
|
72
|
-
# Option 1: run the official CLI directly via npx (no pre-installed dsh; version can be pinned, e.g. dsh-layered-memory@0.
|
|
61
|
+
# Option 1: run the official CLI directly via npx (no pre-installed dsh; version can be pinned, e.g. dsh-layered-memory@0.6.1)
|
|
73
62
|
npx -y @deepseek-ai/dsh plugin --profile web add dsh-layered-memory
|
|
74
63
|
|
|
75
64
|
# Option 2: with the dsh CLI installed (dsh is a pnpm forwarder; npm i -g pnpm first if missing)
|
|
@@ -106,6 +95,15 @@ npm run smoke # smoke test (rebuild first: see command b
|
|
|
106
95
|
npx tsc src/smoke.ts --outDir dist-smoke --module nodenext --moduleResolution nodenext --target es2022 --strict --skipLibCheck --esModuleInterop
|
|
107
96
|
```
|
|
108
97
|
|
|
98
|
+
## UI Preview
|
|
99
|
+
|
|
100
|
+
<p align="center">
|
|
101
|
+
<img src="./assets/img/ui-dark.jpg" width="49.5%"
|
|
102
|
+
alt="Settings memory browser overview in dark theme: status card (plugin version, capture/distill/recall switch states, FTS and vector capabilities, L1 memory count, distillation model) and stat tiles, glassy controls with a cold-blue accent">
|
|
103
|
+
<img src="./assets/img/ui-light.jpg" width="49.5%"
|
|
104
|
+
alt="The same settings memory browser overview in light theme: identical layout and information on light card backgrounds with the same accent family, theme switch without reload">
|
|
105
|
+
</p>
|
|
106
|
+
|
|
109
107
|
## Configuration
|
|
110
108
|
|
|
111
109
|
Override configs go into the profile's own `cordis.patch.yml` as a **top-level bare
|
|
@@ -141,6 +139,8 @@ the bundle layer appends and causes `duplicate loader entry id` startup failure)
|
|
|
141
139
|
| `l3.interval` | `20` | L3 distillation interval (new-memory count) |
|
|
142
140
|
| `recall.enabled` | `true` | Auto recall |
|
|
143
141
|
| `recall.maxResults` | `5` | L1 records injected per step |
|
|
142
|
+
| `recall.includePersona` | `true` | Inject persona context on recall (`<user-persona>`) |
|
|
143
|
+
| `recall.includeSceneNav` | `true` | Inject scene navigation on recall (`<scene-navigation>`) |
|
|
144
144
|
| `recall.strategy` | `hybrid` | Retrieval strategy: `keyword` / `embedding` / `hybrid` |
|
|
145
145
|
| `recall.scoreThreshold` | `0.3` | Recall score threshold (below is not injected; applies to keyword/embedding only, not pre-fusion hybrid; tool path unfiltered) |
|
|
146
146
|
| `embedding.enabled` | `false` | Vector retrieval switch; off = pure FTS |
|
|
@@ -148,56 +148,74 @@ the bundle layer appends and causes `duplicate loader entry id` startup failure)
|
|
|
148
148
|
| `embedding.apiKey` | empty | API key |
|
|
149
149
|
| `embedding.model` | empty | embedding model name |
|
|
150
150
|
| `embedding.dimensions` | `0` | Vector dimensions (required when enabled; must match model output) |
|
|
151
|
+
| `embedding.maxInputChars` | `5000` | Max characters per text (overlong inputs truncated) |
|
|
152
|
+
| `embedding.timeoutMs` | `10000` | Per-call embedding timeout (ms) |
|
|
153
|
+
| `embedding.allowLocalModels` | `true` | Allow the local embedding tier (deployment ceiling; when off, no model downloads and no local tier in settings) |
|
|
154
|
+
| `embedding.mirror` | `https://hf-mirror.com` | Download mirror root for local models (can be changed back to `https://huggingface.co`) |
|
|
151
155
|
| `llm.provider/model` | empty | Distillation model override (defaults to current selection) |
|
|
152
|
-
| `llm.maxTokens` | `
|
|
156
|
+
| `llm.maxTokens` | `256000` | Output token cap per distillation call (unified across stages; a reasoning model's reasoning shares this budget — too low gets fully consumed by thinking, leaving 0 chars of text) |
|
|
157
|
+
| `llm.reasoningEffort` | `off` | Distillation reasoning-effort tier (deployment default): `off` / `high` / `max`; empty string = don't send (follow model default). Distillation is structured extraction, so thinking is off by default — a reasoning model (e.g. v4-flash) at its default `high` tier can consume the entire output budget on thinking, leaving 0 chars of text; set to empty string for models that don't recognize the effort parameter. Switchable at runtime in Settings → Memory → Overview ("follow config" falls back to this value) |
|
|
153
158
|
| `llm.temperature` | `0.3` | Distillation temperature |
|
|
154
159
|
| `llm.maxInputChars` | `700000` | Input character budget per distillation call (over-budget L1 inputs are chunked automatically) |
|
|
160
|
+
| `llm.timeoutMs` | `120000` | Per-call distillation timeout (ms) |
|
|
155
161
|
| `tools` | `true` | Whether to register model-callable memory tools |
|
|
156
162
|
|
|
157
163
|
## Storage Layout
|
|
158
164
|
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
from dedup merges touch the retrieval DB only.
|
|
164
|
-
|
|
165
|
-
- **Three retrieval strategies** (`recall.strategy`): `keyword` (FTS5 BM25) /
|
|
166
|
-
`embedding` (vec0 cosine KNN) / `hybrid` (both lanes in parallel + RRF k=60
|
|
167
|
-
fusion, default); `conversation_search` (L0) uses the same fusion;
|
|
168
|
-
- **Vectors optional**: off by default (pure FTS). DSH's `ctx.llm` has no embeddings
|
|
169
|
-
endpoint; enabling requires any OpenAI-compatible `/embeddings` service (configure
|
|
170
|
-
`embedding.*`); config changes automatically drop the vector table and re-embed
|
|
171
|
-
everything in the background;
|
|
172
|
-
- **Degradation chain**: sqlite-vec load failure → pure FTS; embedding call failure
|
|
173
|
-
→ degrade to FTS for that call with a one-time warning; retrieval DB init failure
|
|
174
|
-
→ memory features disabled entirely but dsh itself starts normally;
|
|
175
|
-
- **Replacement seam**: `L1Store.search()` is the single retrieval entry point.
|
|
165
|
+
<p align="center">
|
|
166
|
+
<img src="./assets/readme/storage.svg" width="100%"
|
|
167
|
+
alt="Storage layout: dual-write architecture (append-only JSONL source of truth + memory.db retrieval engine); file forms include conversations/records/scenes/persona/state/pending/session-modes/embedding-source/model catalog/inference runtime/log and rebuild archives; three retrieval strategies keyword/embedding/hybrid (RRF k=60); a degradation chain never blocks the host">
|
|
168
|
+
</p>
|
|
176
169
|
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
170
|
+
Vectors are off by default (pure FTS). DSH's `ctx.llm` has no embeddings endpoint;
|
|
171
|
+
semantic retrieval is provided by a **three-state embedding source** (off / remote /
|
|
172
|
+
local), switchable at runtime in the settings page — see the next section.
|
|
173
|
+
|
|
174
|
+
## Semantic Retrieval (Embedding Source)
|
|
175
|
+
|
|
176
|
+
Pick the embedding source in Settings → Memory → Overview → Semantic Retrieval;
|
|
177
|
+
it takes effect immediately, no config edit or restart:
|
|
178
|
+
|
|
179
|
+
| Source | Description |
|
|
180
|
+
| --- | --- |
|
|
181
|
+
| **Off** (default) | No vector embedding at all; pure BM25 keyword retrieval |
|
|
182
|
+
| **Remote** | Bring any OpenAI-compatible `/embeddings` service (selectable only when the `embedding.*` quartet is configured) |
|
|
183
|
+
| **Local** | Pick from a built-in model catalog, ONNX-quantized **CPU inference** — no API key, data never leaves the machine |
|
|
184
|
+
|
|
185
|
+
The local catalog is a built-in allowlist (each model pinned to a revision with
|
|
186
|
+
per-file sha256; arbitrary repos cannot be downloaded):
|
|
187
|
+
|
|
188
|
+
| Model | Dims | Context | Size | Notes |
|
|
189
|
+
| --- | --- | --- | --- | --- |
|
|
190
|
+
| BGE small Chinese | 512 | 512 | ~25MB | Fastest on CPU; good first taste of semantic retrieval |
|
|
191
|
+
| EmbeddingGemma 300M | 768 | 2048 | ~330MB | 100+ languages incl. Chinese; balanced quality/cost (same as upstream MemoryCore) |
|
|
192
|
+
| BGE-M3 | 1024 | 8192 | ~590MB | Best Chinese quality; a single embedding can take seconds |
|
|
193
|
+
|
|
194
|
+
- **Download**: one click on the model card (default mirror `hf-mirror.com`, resumable
|
|
195
|
+
downloads + sha256 integrity checks); stored under `models/<id>/` in the data
|
|
196
|
+
directory, deletable from the settings page at any time;
|
|
197
|
+
- **On-demand runtime**: the inference runtime (transformers.js, ~100–200MB) is
|
|
198
|
+
installed only on first switch to the local tier, into `runtime/` in the data
|
|
199
|
+
directory — never in the plugin's dependency tree or install directory;
|
|
200
|
+
- **Live switching**: one click to swap sources — everything is re-embedded in the
|
|
201
|
+
background (visible progress, cancellable; retrieval silently degrades to keywords
|
|
202
|
+
in the meantime, conversations unaffected; a dimension change rebuilds the vector
|
|
203
|
+
table at the new size); a failed switch keeps the old source, which a restart
|
|
204
|
+
still uses;
|
|
205
|
+
- **Effective = deployment ceiling AND runtime choice**: `embedding.allowLocalModels=false`
|
|
206
|
+
disables the local tier entirely; without the `embedding.*` quartet the remote tier
|
|
207
|
+
is unavailable (enterprise deployments can lock this down). The choice persists in
|
|
208
|
+
`embedding-source.json`.
|
|
190
209
|
|
|
191
210
|
## Logging & Troubleshooting
|
|
192
211
|
|
|
193
212
|
The dsh host prints plugin logs to the console; the plugin mirrors info and above to
|
|
194
213
|
`memory.log` in its data directory. The typical log path of one conversation turn:
|
|
195
|
-
`L0
|
|
196
|
-
`
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
first stack frame.
|
|
214
|
+
`L0 capture` → `L0 flush` → `distillation pipeline start` → `LLM call (input/output
|
|
215
|
+
chars, duration)` → `L1 extraction done` → `pipeline end`; the next turn shows
|
|
216
|
+
`recall hit N L1 records`. Empty LLM output carries full diagnostics (finish reason /
|
|
217
|
+
token counts / reasoning excerpt); JSON parse failures include the first 400 characters
|
|
218
|
+
of the raw model output; all failure warns carry the first stack frame.
|
|
201
219
|
|
|
202
220
|
## Differences from MemoryCore
|
|
203
221
|
|
|
@@ -206,6 +224,13 @@ first stack frame.
|
|
|
206
224
|
- Recall injection happens at `agent/pre-step` + agent-scoped `systemPrompt.context` (DSH-native events/services);
|
|
207
225
|
- Storage/retrieval is a single-machine slimmed version of the official sqlite backend (drops multi-tenant isolation columns, TCVDB cloud backend, audit tables; tokenization uses a bundled CJK bigram instead of jieba, keeping zero native dependencies — sqlite-vec is the only native extension, auto-degrading on load failure).
|
|
208
226
|
|
|
227
|
+
## Credits
|
|
228
|
+
|
|
229
|
+
The core memory capabilities (layered distillation pipeline, prompt design, and the
|
|
230
|
+
dual-write storage architecture) are modeled after **MemoryCore** from
|
|
231
|
+
[TencentCloud/TencentDB-Agent-Memory](https://github.com/TencentCloud/TencentDB-Agent-Memory).
|
|
232
|
+
Thanks to the original project for open-sourcing its design and implementation.
|
|
233
|
+
|
|
209
234
|
## License
|
|
210
235
|
|
|
211
236
|
[MIT](LICENSE)
|
package/README.md
CHANGED
|
@@ -1,74 +1,57 @@
|
|
|
1
1
|
**简体中文** | [English](README.en.md)
|
|
2
2
|
|
|
3
3
|
<p align="center">
|
|
4
|
-
<img src="./assets/
|
|
5
|
-
alt="
|
|
4
|
+
<img src="./assets/img/Hero.png" width="100%"
|
|
5
|
+
alt="DeepSeek Harness hero 横幅:对话自动分层蒸馏成记忆,模型每步前自动召回注入——右侧对话气泡逐层溶解为三层渐亮光带,流入带发光圆球与渐变轨道的玻璃胶囊(下有 日常·工作·智能·关闭 四档刻度),光丝回流示意召回注入">
|
|
6
6
|
</p>
|
|
7
7
|
|
|
8
8
|
# dsh-layered-memory
|
|
9
9
|
|
|
10
10
|
DeepSeek Harness 的**分层蒸馏记忆插件**(持久组合插件):对话在后台自动完成
|
|
11
11
|
L0 捕获 → L1 原子记忆 → L2 场景整合 → L3 画像蒸馏,模型每一步前自动把相关记忆
|
|
12
|
-
|
|
13
|
-
[MemoryCore](https://github.com/TencentDB-Agent-Memory)(TencentDB Agent Memory)的
|
|
14
|
-
管线设计:Prompt 原样保留,仅把"LLM 操作文件"的 L2/L3 流程适配为"LLM 输出、工程侧执行"。
|
|
12
|
+
注入上下文——用户与模型都不需要做任何操作。
|
|
15
13
|
|
|
16
|
-
|
|
14
|
+
> 本插件的记忆核心能力(L0–L3 分层蒸馏管线、Prompt 与双写存储设计)参考自
|
|
15
|
+
> [TencentDB-Agent-Memory](https://github.com/TencentCloud/TencentDB-Agent-Memory)
|
|
16
|
+
> 中的 **MemoryCore**:Prompt 原样保留,仅把"LLM 操作文件"的 L2/L3 流程适配为
|
|
17
|
+
> "LLM 输出、工程侧执行"。
|
|
17
18
|
|
|
18
|
-
|
|
19
|
+
## 运行时数据流
|
|
19
20
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
| L2 | 把新记忆整合为 Markdown 场景文档(META 块、热度管理、合并上限),按族各自整合 | `scenes/chat/*.md`、`scenes/work/*.md` |
|
|
25
|
-
| L3 | 从变化场景蒸馏画像(chat: 用户画像 ≤2000 字;work: Team Operating Doctrine ≤1200 字),每族一份 | `persona-chat.md`、`persona-work.md` |
|
|
21
|
+
<p align="center">
|
|
22
|
+
<img src="./assets/readme/flow.svg" width="100%"
|
|
23
|
+
alt="dsh-layered-memory 运行时数据流:左侧 User 与 Assistant 的会话事件流入插件(L0 捕获、L1–L3 蒸馏、检索召回、记忆工具),插件经 agent/pre-step 把相关记忆注入右侧 DSH 核心;蒸馏复用核心的 ctx.llm,数据双写 ~/.dsh/memory/">
|
|
24
|
+
</p>
|
|
26
25
|
|
|
27
|
-
|
|
28
|
-
|
|
26
|
+
插件挂在 dsh 原生事件缝上(`session/event` 捕获、`agent/pre-step` 注入),
|
|
27
|
+
蒸馏调用复用宿主 `ctx.llm`,全程对用户与模型透明。另注册三个模型可主动调用的
|
|
28
|
+
记忆工具:`memory_search` / `conversation_search` / `memory_read_scene`。
|
|
29
29
|
|
|
30
|
-
|
|
31
|
-
缓冲里(`pending.json`,每次蒸馏尝试后原子落盘)——重启不丢,启动 20 秒后自动补跑一次,
|
|
32
|
-
失败则维持"等下一轮同档对话"的语义。
|
|
30
|
+
## 分层记忆(L0–L3)
|
|
33
31
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
L1
|
|
37
|
-
|
|
38
|
-
(会话数/消息数/预计调用数)、进度条与取消(已重建部分保留)。
|
|
32
|
+
<p align="center">
|
|
33
|
+
<img src="./assets/img/Layers.png" width="100%"
|
|
34
|
+
alt="分层记忆四层(自左上向右下逐层精炼):L0 原始对话(对话气泡)→ L1 原子记忆(发光事实粒子)→ L2 场景块(玻璃文档板)→ L3 核心画像(发光晶核);层间由 LLM 提取/整合/蒸馏光束相连,宽度递减表示数据逐层精炼">
|
|
35
|
+
</p>
|
|
39
36
|
|
|
40
|
-
|
|
37
|
+
## 会话级记忆档位
|
|
41
38
|
|
|
42
|
-
|
|
39
|
+
<p align="center">
|
|
40
|
+
<img src="./assets/img/Modes.png" width="100%"
|
|
41
|
+
alt="会话级记忆档位:一条玻璃胶囊滑轨四个停点(日常·工作·智能·关闭),发光圆球停在智能(默认)档;各档上方微场景——日常为个人聊天气泡、工作为代码文档窗格、智能为双流合流最亮、关闭为暗淡虚线幽灵泡">
|
|
42
|
+
</p>
|
|
43
43
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
| `work` | 窄 prompt 只提工作四类,只入 work 族 | 只查 work 记忆 + work 画像/场景导航 |
|
|
49
|
-
| `关闭` | 不写 L0、不蒸馏 | 不召回;三个记忆工具返回"已隐身"提示 |
|
|
50
|
-
|
|
51
|
-
- **控件**:输入栏内、模式选择器右侧的 pill(`记忆·自动`),点击在上方浮出 macOS 风格
|
|
52
|
-
滑动选择器——拖拽松手吸附最近档位;
|
|
53
|
-
- **默认档** = 配置 `family`(`auto|chat|work`,默认 `auto`);每会话的选择按 sessionId
|
|
54
|
-
持久化到 `session-modes.json`,重启/恢复会话不丢;中途切档下一轮生效,已提取记忆留在原族;
|
|
55
|
-
- 与全局开关叠加(全局是总闸);L2/L3 完全分族,间族内容不渗透。
|
|
56
|
-
|
|
57
|
-
### 记忆浏览器(设置页 → 记忆)
|
|
58
|
-
|
|
59
|
-
多 Tab 页面,两族混合视图:**概览**(各层计数 + 记忆模式开关面板 + 蒸馏思考档位选择器,5 秒自动刷新)、
|
|
60
|
-
**记忆**(L1 卡片列表,关键词/类型/情境筛选)、**场景**(L2 全文)、**画像**(L3 全文)、
|
|
61
|
-
**日志**(`memory.log` 尾部 200 行)。开关与思考档位走官方 settings 服务(命名空间 `dsh-memory`,
|
|
62
|
-
实时生效、重启保留);开关生效规则 = **静态 config(部署上限)AND 运行时开关**;
|
|
63
|
-
思考档位为运行时覆盖(选择器选"跟随配置"则用部署配置 `llm.reasoningEffort` 作默认),
|
|
64
|
-
数据通道为 loopback RPC(`dsh-memory/*`)。
|
|
44
|
+
- **控件**:输入栏内、模式选择器右侧的 pill(`记忆·自动`),点击在上方浮出 macOS
|
|
45
|
+
风格滑动选择器——拖拽松手吸附最近档位;深浅主题自适应;
|
|
46
|
+
- 每会话的选择按 sessionId 持久化到 `session-modes.json`,重启/恢复会话不丢;
|
|
47
|
+
与全局开关叠加(全局是总闸);L2/L3 完全分族,跨族内容不渗透。
|
|
65
48
|
|
|
66
49
|
## 快速开始
|
|
67
50
|
|
|
68
51
|
需要 Node ≥ 22.16。两种调用方式任选(`npx` 前缀可替换下面任何 `dsh` 命令):
|
|
69
52
|
|
|
70
53
|
```bash
|
|
71
|
-
# 方式一:npx 直接跑官方 CLI(无需预装 dsh;可 pin 版本,如 dsh-layered-memory@0.
|
|
54
|
+
# 方式一:npx 直接跑官方 CLI(无需预装 dsh;可 pin 版本,如 dsh-layered-memory@0.6.1)
|
|
72
55
|
npx -y @deepseek-ai/dsh plugin --profile web add dsh-layered-memory
|
|
73
56
|
|
|
74
57
|
# 方式二:已装 dsh CLI(dsh 是 pnpm 转发器,未装 pnpm 时先 npm i -g pnpm)
|
|
@@ -101,6 +84,15 @@ npm run smoke # 冒烟测试(先重编:见下方命
|
|
|
101
84
|
npx tsc src/smoke.ts --outDir dist-smoke --module nodenext --moduleResolution nodenext --target es2022 --strict --skipLibCheck --esModuleInterop
|
|
102
85
|
```
|
|
103
86
|
|
|
87
|
+
## 界面预览
|
|
88
|
+
|
|
89
|
+
<p align="center">
|
|
90
|
+
<img src="./assets/img/ui-dark.jpg" width="49.5%"
|
|
91
|
+
alt="深色主题下的设置页记忆浏览器概览:状态卡(插件版本、捕获/蒸馏/召回开关状态、FTS 与向量能力、L1 记忆计数、蒸馏模型)与统计瓦片,玻璃质感控件与冷蓝强调色">
|
|
92
|
+
<img src="./assets/img/ui-light.jpg" width="49.5%"
|
|
93
|
+
alt="浅色主题下的同一设置页记忆浏览器概览:同款布局与信息,浅色卡片底与同套强调色,主题切换无需重载">
|
|
94
|
+
</p>
|
|
95
|
+
|
|
104
96
|
## 配置
|
|
105
97
|
|
|
106
98
|
覆盖配置写在 profile 自己的 `cordis.patch.yml`,用**顶层裸 patch 条目**(直接 `id:`,
|
|
@@ -136,6 +128,8 @@ npx tsc src/smoke.ts --outDir dist-smoke --module nodenext --moduleResolution no
|
|
|
136
128
|
| `l3.interval` | `20` | L3 蒸馏间隔(新记忆条数) |
|
|
137
129
|
| `recall.enabled` | `true` | 自动召回 |
|
|
138
130
|
| `recall.maxResults` | `5` | 每步召回注入的 L1 条数 |
|
|
131
|
+
| `recall.includePersona` | `true` | 召回时注入画像上下文(`<user-persona>`) |
|
|
132
|
+
| `recall.includeSceneNav` | `true` | 召回时注入场景导航(`<scene-navigation>`) |
|
|
139
133
|
| `recall.strategy` | `hybrid` | 检索策略:`keyword` / `embedding` / `hybrid` |
|
|
140
134
|
| `recall.scoreThreshold` | `0.3` | 召回分数阈值(低于不注入;仅 keyword/embedding 策略生效,hybrid 融合前不过滤;工具路径不过滤) |
|
|
141
135
|
| `embedding.enabled` | `false` | 向量检索开关;关闭即纯 FTS 运行 |
|
|
@@ -143,52 +137,64 @@ npx tsc src/smoke.ts --outDir dist-smoke --module nodenext --moduleResolution no
|
|
|
143
137
|
| `embedding.apiKey` | 空 | API Key |
|
|
144
138
|
| `embedding.model` | 空 | embedding 模型名 |
|
|
145
139
|
| `embedding.dimensions` | `0` | 向量维度(启用时必填,须与模型输出一致) |
|
|
140
|
+
| `embedding.maxInputChars` | `5000` | 单条文本最大字符数(超长截断) |
|
|
141
|
+
| `embedding.timeoutMs` | `10000` | 单次 embedding 调用超时(ms) |
|
|
142
|
+
| `embedding.allowLocalModels` | `true` | 允许本地嵌入档(部署上限:关闭后设置页不能下载模型、不能切本地档) |
|
|
143
|
+
| `embedding.mirror` | `https://hf-mirror.com` | 本地模型下载镜像根地址(可改回官方 `https://huggingface.co`) |
|
|
146
144
|
| `llm.provider/model` | 空 | 蒸馏模型覆盖(默认用当前默认选择) |
|
|
147
145
|
| `llm.maxTokens` | `256000` | 单次蒸馏输出 token 上限(全阶段统一;推理模型的 reasoning 与正文共享该预算,过低会被思考吃光导致正文 0 字符) |
|
|
148
146
|
| `llm.reasoningEffort` | `off` | 蒸馏思考档位(部署默认):`off` / `high` / `max`,空串不传(跟随模型默认)。蒸馏是结构化抽取任务,默认关思考——推理模型(如 v4-flash)默认 high 档的思考可把任意输出预算全部吃光导致正文 0 字符;非推理模型不认识 effort 时需设为空串。运行时可在设置页 → 记忆 → 概览临时切换(选"跟随配置"即回退本值) |
|
|
149
147
|
| `llm.temperature` | `0.3` | 蒸馏温度 |
|
|
150
148
|
| `llm.maxInputChars` | `700000` | 单次蒸馏输入字符预算(超限的 L1 输入自动分块抽取) |
|
|
149
|
+
| `llm.timeoutMs` | `120000` | 单次蒸馏调用超时(ms) |
|
|
151
150
|
| `tools` | `true` | 是否注册模型可调用的记忆工具 |
|
|
152
151
|
|
|
153
152
|
## 存储布局
|
|
154
153
|
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
154
|
+
<p align="center">
|
|
155
|
+
<img src="./assets/readme/storage.svg" width="100%"
|
|
156
|
+
alt="存储布局:双写架构(JSONL 事实源只增不改 + memory.db 主检索库);文件形态含 conversations/records/scenes/persona/state/pending/session-modes/embedding-source/模型目录/推理运行时/日志与重建归档;检索三策略 keyword/embedding/hybrid(RRF k=60);降级链保证永不阻塞宿主">
|
|
157
|
+
</p>
|
|
158
158
|
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
- **向量能力可选**:默认关闭(纯 FTS)。DSH 的 `ctx.llm` 无 embeddings 端点,启用需自备
|
|
162
|
-
任意 OpenAI 兼容 `/embeddings` 服务(配置 `embedding.*`);配置变化自动 drop 向量表并
|
|
163
|
-
后台全量重嵌入;
|
|
164
|
-
- **降级链**:sqlite-vec 加载失败 → 纯 FTS;embedding 调用失败 → 该次降级 FTS 并告警一次;
|
|
165
|
-
检索库初始化失败 → 记忆功能整体停用但 dsh 本体照常启动;
|
|
166
|
-
- **替换缝**:`L1Store.search()` 是唯一检索入口。
|
|
159
|
+
向量能力默认关闭(纯 FTS)。DSH 的 `ctx.llm` 无 embeddings 端点,语义检索由
|
|
160
|
+
**三态嵌入源**提供(关闭 / 远程 / 本地),设置页可运行时切换——见下节。
|
|
167
161
|
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
162
|
+
## 语义检索(嵌入源)
|
|
163
|
+
|
|
164
|
+
设置页(记忆 → 概览 → 语义检索)选择嵌入源,即时生效、无需改配置重启:
|
|
165
|
+
|
|
166
|
+
| 嵌入源 | 说明 |
|
|
167
|
+
| --- | --- |
|
|
168
|
+
| **关闭**(默认) | 不做任何向量嵌入,纯 BM25 关键词检索 |
|
|
169
|
+
| **远程** | 自备任意 OpenAI 兼容 `/embeddings` 服务(`embedding.*` 四件套配齐才可选) |
|
|
170
|
+
| **本地** | 内置模型目录选一款,ONNX 量化 **CPU 推理**——无需 API Key,数据不出本机 |
|
|
171
|
+
|
|
172
|
+
本地模型目录是插件内置白名单(每款锁定 revision + 每文件 sha256,不可下载任意仓库):
|
|
173
|
+
|
|
174
|
+
| 模型 | 维度 | 上下文 | 体积 | 特点 |
|
|
175
|
+
| --- | --- | --- | --- | --- |
|
|
176
|
+
| BGE small 中文 | 512 | 512 | ~25MB | CPU 嵌入最快,适合先体验语义检索 |
|
|
177
|
+
| EmbeddingGemma 300M | 768 | 2048 | ~330MB | 100+ 语言含中文,质量与开销均衡(上游 MemoryCore 同款) |
|
|
178
|
+
| BGE-M3 | 1024 | 8192 | ~590MB | 中文质量最强,单条嵌入可达秒级 |
|
|
179
|
+
|
|
180
|
+
- **下载**:模型卡一键下载(默认镜像 `hf-mirror.com`,断点续传 + sha256 完整性
|
|
181
|
+
校验),落盘数据目录 `models/<id>/`,不用了随时在设置页删除;
|
|
182
|
+
- **按需运行时**:首次切换本地档才安装推理运行时(transformers.js,约 100~200MB,
|
|
183
|
+
装进数据目录 `runtime/`——不进插件依赖树,不碰插件安装目录);
|
|
184
|
+
- **活切换**:一键换源——自动后台全量重嵌(进度可见、可取消,期间检索自动降级
|
|
185
|
+
关键词,不影响对话;维度变化时向量表按新维度重建);切换失败保持旧源,重启仍
|
|
186
|
+
按原源运行;
|
|
187
|
+
- **生效规则 = 部署上限 AND 运行时选择**:`embedding.allowLocalModels=false` 可整体
|
|
188
|
+
禁用本地档、未配 `embedding.*` 四件套则远程档不可选(企业部署可收口),状态持久
|
|
189
|
+
化在 `embedding-source.json`。
|
|
183
190
|
|
|
184
191
|
## 日志与排查
|
|
185
192
|
|
|
186
|
-
dsh 宿主把插件日志打到控制台,插件将 info 及以上镜像到数据目录 `memory.log
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
携带错误堆栈首帧。
|
|
193
|
+
dsh 宿主把插件日志打到控制台,插件将 info 及以上镜像到数据目录 `memory.log`。一轮
|
|
194
|
+
对话的典型日志路径:`L0 捕获` → `L0 落盘` → `蒸馏管线开始` → `LLM 调用(输入/输出
|
|
195
|
+
字符数、耗时)` → `L1 抽取完成` → `蒸馏管线结束`;下一轮有 `召回命中 N 条 L1`。
|
|
196
|
+
LLM 空输出带完整诊断(finish 原因 / token 计数 / reasoning 摘录),JSON 解析失败
|
|
197
|
+
附模型原始输出前 400 字符,失败 warn 均带堆栈首帧。
|
|
192
198
|
|
|
193
199
|
## 与 MemoryCore 的差异
|
|
194
200
|
|
|
@@ -196,7 +202,14 @@ token 计数 / reasoning 摘录);JSON 解析失败附带模型原始输出
|
|
|
196
202
|
- L2/L3 由"LLM 操作文件工具"改为"LLM 输出操作 JSON / 完整文档,工程侧执行";
|
|
197
203
|
- 召回注入点在 `agent/pre-step` + agent 作用域 `systemPrompt.context`(DSH 原生事件/服务);
|
|
198
204
|
- 存储/检索即官方 sqlite 后端的单机裁剪版(裁掉多租户隔离列、TCVDB 云后端、审计表;
|
|
199
|
-
分词用自带 CJK 二元组替代 jieba,保持零原生依赖——仅 sqlite-vec
|
|
205
|
+
分词用自带 CJK 二元组替代 jieba,保持零原生依赖——仅 sqlite-vec 一个原生扩展,
|
|
206
|
+
加载失败自动降级)。
|
|
207
|
+
|
|
208
|
+
## 致谢
|
|
209
|
+
|
|
210
|
+
记忆核心能力(分层蒸馏管线、Prompt 设计、双写存储架构)参考自
|
|
211
|
+
[TencentCloud/TencentDB-Agent-Memory](https://github.com/TencentCloud/TencentDB-Agent-Memory)
|
|
212
|
+
项目中的 **MemoryCore**,感谢原项目开放的设计与实现。
|
|
200
213
|
|
|
201
214
|
## License
|
|
202
215
|
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|