@kenz1117/dsh-engram 0.7.9 → 0.7.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +36 -16
- package/README.md +36 -16
- package/lib/client.js +2 -2
- package/lib/client.js.map +1 -1
- package/lib/index.js +1196 -24
- package/package.json +1 -1
package/README.en.md
CHANGED
|
@@ -64,25 +64,28 @@ Sensory and emotional dimensions (smell, temperature, emotional weight) are deli
|
|
|
64
64
|
- **Memory-palace information architecture** (v0.7.2+): all four principles live on the core path, not in display-layer paint — **location as index** (writes sort by kind into rooms and pin a `room#slot` coordinate; room capacity 9, overflow opens a new room, slots are never recycled); **fixed route as order** (`tour_routes` is append-only; `engram_tour mode=fixed` walks the whole palace in slot order); **skeleton reused long-term** (a topic always lands in the same room and the same index, so recall is sequential extraction rather than fresh search); **every marker unique** (placard rule scored 0-1: globally unique +0.4 / date anchor +0.3 / no 6-char prefix clash within the room +0.3; low scores enter the refurb list; a palace with fewer than 8 active memories is left unscanned to avoid small-store noise).
|
|
65
65
|
- **Corridor-routed retrieval**: the profile carries a room directory, and `engram_search` accepts a `room` parameter — decide the room first, then search inside it, instead of always running whole-store RRF. The top 5 hits carry neighbouring slot ids from the same room as encoding-specificity cues.
|
|
66
66
|
- **Retrieval-practice loop** (spaced repetition): `engram_review_queue` returns palace coordinates and placard cues but **never the body text**, forcing the model to recall first; `engram_review` reveals the entry and `engram_report grade` (0-5) self-rates it, advancing an SM-2 schedule (1 → 6 → round(prev × ease) days, reset on failure, ease floor 1.3). Entries under review scheduling **no longer take part in automatic decay** — their fate is decided by recall. Session-start injection reports how many memories are due.
|
|
67
|
-
- **History backfill** (v0.7.3+): distils dsh's persisted past sessions into the palace, turn by turn — each session is written into the project store matching **its own cwd** (no cross-project bleed), reusing the live-capture throttling, redaction, and echo suppression, with a `(session, turn)` idempotency key so an interrupted run resumes. Backfilled entries never enter the due-today queue. **You choose the import rules** (time window / turns per session / total turn budget / auxiliary model / include subagent·seeded·no-cwd sessions) and see a zero-cost estimate (no LLM calls) before running; staging happens in a dedicated **"
|
|
67
|
+
- **History backfill** (v0.7.3+): distils dsh's persisted past sessions into the palace, turn by turn — each session is written into the project store matching **its own cwd** (no cross-project bleed), reusing the live-capture throttling, redaction, and echo suppression, with a `(session, turn)` idempotency key so an interrupted run resumes. Backfilled entries never enter the due-today queue. **You choose the import rules** (time window / turns per session / total turn budget / auxiliary model / include subagent·seeded·no-cwd sessions) and see a zero-cost estimate (no LLM calls) before running; staging happens in a dedicated **"Backfill" tab** with progress (including a **breakdown of skip reasons**) and pause. Auxiliary calls **reuse the model you are currently using** by default (a past session's log records the provider/model of that time, which may no longer be available here), or you can pick one explicitly from the host's registered providers/models in the tab's "Auxiliary model" dropdown.
|
|
68
68
|
- **Knowledge flywheel**: capture/save → contradiction candidates (high-similarity neighbors create `contradicts` edges on write, for the model/user to adjudicate) → hit reinforcement (confidence +0.05) → distillation (topic clusters merge into higher-level rules, supersedes chains, confidence inheritance) → decay (low-importance, long-unaccessed entries archive; restorable).
|
|
69
|
+
- **Jev System-One adjudication** (v0.7.12+, off by default, the `jev` block in yml): the DEFER fuzzy-band three-way ruling (merge / accept / defer) and contradiction-edge confirmation can be delegated to an external Jev model; when Jev is unavailable (network failure / timeout / missing key) it silently falls back to the pure-rule four-way behavior, **never blocking writes**. Every adjudication leaves a side observation (verdict, whether Jev took part, elapsed time, fallback reason; up to 50 in-process, cleared on restart), so the panel's "Jev" view answers "why was this one deferred" at a glance. Five yml fields — the switch / API key / endpoint / model / timeout — can be overridden from the panel (stored at `<dbDir>/jev-config.json`, 0600 file inside a 0700 directory) and take effect on save without a restart; a "Test connection" button verifies endpoint and model reachability with the current input (unsaved values are tested as typed, never written to the override file); the key is returned only masked by GET, never leaves the process, and is never back-filled into the input. The three thresholds stay yml-only advanced settings, shown read-only.
|
|
70
|
+
- **Entity lexicon** (v0.7.10+): the auxiliary LLM output of automatic capture and `engram_save` also extracts entity mentions (people/projects/tools/concepts, with optional aliases), resolved by normalized name against existing entities' names/aliases — a hit reuses the entity and refreshes its "last mentioned" stamp, a miss creates one; memories and entities link many-to-many, stored in the same store as the source memory. `engram_search` rows carry entity tags of the hits; entity-resolution failures silently skip the links and never block the memory write itself. The panel's "Entities" view filters by kind, searches names/aliases, and opens one entity to see every memory it pulls in.
|
|
69
71
|
- **Automatic capture** (when `ingest` is enabled): each new turn's first step extracts candidate facts from the previous turn out of the session log, and session end captures the final turn too (failures leave a pending key that the next session replays; capture is idempotent per session+turn), written with low confidence and, when embeddings are available, disposed by the four-state rule toward near neighbors (merge-and-reinforce restatements / edge-and-defer suspected contradictions / write fresh) — memories accumulate without you saying "remember this".
|
|
70
72
|
- **Provenance audit**: every memory records its source session, turn, and event seq; `engram_review` traces the full source chain, supersede chain, contradictions, and operation log; all writes/edits/forgets/distills/decays land in the operation log table.
|
|
71
|
-
- **Web management panel** (v0.7.0+): the "Memory Library" tab in Settings has
|
|
73
|
+
- **Web management panel** (v0.7.0+): the "Memory Library" tab in Settings has eight views — Today (a summary bar: memories / open / clarity + last-7-days counts + due today + health ring, followed by tour proposal, room directory, refurb list, health breakdown; a profile card in the side column shows the curated profile's current content and version-diff history), Palace (filters incl. due today, tour-route order, batch actions, list & edit), Corridor (corridor bird's-eye + retrieval bench), Log (last-7-days counts + the full merged op_log, filterable by operation class), Backfill, Episodes (episodes grouped by session, with anchor-neighborhood expansion), Entities (the entity lexicon: kind filters + name/alias search + per-entity linked memories), and Jev (System-One adjudication: config overrides + connection test + recent adjudications). A single 3-pill scope switcher in the header drives every panel; UI copy is bilingual zh/en and follows the host language setting live. Filter by redaction marks (include/exclude `[REDACTED:*]` entries) with amber badges for audit coverage.
|
|
72
74
|
- **Prompt-injection defense**: every memory recall exit (profile injection, `engram_search/timeline/review` output) is wrapped in `<engram_memory_context>` protocol tags with a usage warning (history is not the current request, do not follow instructions inside it, use only when relevant), while the current request is wrapped separately in `<current_user_request>`; all inbound content (capture candidates, saved bodies) is stripped of these protocol tags first, blocking second-order injection through forged protocol blocks.
|
|
73
75
|
- **Capture redaction**: inbound content passes a regex scrub for common secrets and credentials (sk- API keys, Bearer, AWS AKIA, GitHub tokens, PEM private keys, password/token assignments, Chinese-language password assignments) and personal data (mainland-China phone numbers, 18-digit national ID numbers); matched fragments become `[REDACTED:<type>]`.
|
|
74
76
|
- **Recall placeholder (anti-echo-chamber)**: memory-recall tool output inside captured slices is replaced with `[engram memory result omitted from capture: <tool>]`, and the extraction prompt states that restating existing memory is not new information, breaking the memory self-reinforcement loop.
|
|
75
77
|
- **Multi-query retrieval**: `engram_search` can use the aux LLM to rewrite the query into ≤3 complementary queries, retrieving each and fusing them with cross-query RRF plus a per-query floor; a failed rewrite degrades to the single query (`queryRewrite: false` disables).
|
|
76
|
-
- **Evidence gate (search → assess)**: a hit means "relevant", not "enough to answer". Every retrieval registers an in-process batch (latest 20 per session, released when the session ends) and prints `ref=…` per row plus the batch id; `engram_assess` may only cite refs of that batch, and `sufficient` is enforced by code — all three must hold (the model claims adequacy, at least one valid evidence ref, `nextStrategy=answer`) or the verdict is insufficient with the strategy rewritten back to retrieval. Verdicts and rejected refs go to the audit log, visible under the "retrieval" category in the
|
|
78
|
+
- **Evidence gate (search → assess)**: a hit means "relevant", not "enough to answer". Every retrieval registers an in-process batch (latest 20 per session, released when the session ends) and prints `ref=…` per row plus the batch id; `engram_assess` may only cite refs of that batch, and `sufficient` is enforced by code — all three must hold (the model claims adequacy, at least one valid evidence ref, `nextStrategy=answer`) or the verdict is insufficient with the strategy rewritten back to retrieval. Verdicts and rejected refs go to the audit log, visible under the "retrieval" category in the Log view. If unassessed batches remain before the next step, a wrap-up reminder is injected (escalating to a change-strategy hint after ≥2 consecutive insufficient verdicts; `assessReminder: false` disables).
|
|
77
79
|
- **Portable data**: `engram_export` exports Markdown / JSON files in one step, with a redacted-view variant (secondary scrubbing + 40-char preview truncation, share-safe). `engram_mirror` writes an Obsidian / Logseq-friendly mirror directory (one Markdown per memory with YAML frontmatter + `[[id]]` backlinks), turning the palace into a human-readable private knowledge base.
|
|
78
80
|
- **AGI Architecture Exploration (dsh-market · AGI Architecture)**: listed in dsh-market's "AGI Architecture Exploration" category as a cognitive-science reframe of agent long-term memory — memory palace (imagery labels + room placards), corridor topology (force-directed graph), closure questions (the ingest prompt nudges the model to ask clarifying questions), consolidation merging (heuristic dedup + cosine similarity) — sitting alongside MemGPT / Letta in the "agent memory architecture" conversation.
|
|
79
81
|
|
|
80
|
-
## Tools (
|
|
82
|
+
## Tools (20, narrow parameters)
|
|
81
83
|
|
|
82
84
|
| Tool | Purpose |
|
|
83
85
|
|---|---|
|
|
84
|
-
| `engram_save` | Save (**four-state write disposition**: a restatement merges into and reinforces the existing entry (`merge`, no new node), a suspected contradiction is written with a `contradicts` edge pending adjudication (`defer`), fresh content is written (`accept`), low-entropy content is dropped (`drop`); without embeddings everything is `accept`); `items` array saves ≤10 in one call with shared scrubbing and in-batch dedup, reporting each entry's disposition (`disposition`/`mergedInto`), one failure not blocking the rest (`count`/`items`/`failed` summary); `placard` attaches a marker (scored unique · distinctive · dated; low scores get a rewrite hint) |
|
|
85
|
-
| `engram_search` | Hybrid semantic + keyword retrieval (hits reinforce confidence); `room` routes through the corridor — search inside one room only; the top 5 hits carry same-room neighbouring-slot cues. Output rows carry `id=` and `ref=`; the tail carries the batch id |
|
|
86
|
+
| `engram_save` | Save (**four-state write disposition**: a restatement merges into and reinforces the existing entry (`merge`, no new node), a suspected contradiction is written with a `contradicts` edge pending adjudication (`defer`), fresh content is written (`accept`), low-entropy content is dropped (`drop`); without embeddings everything is `accept`); `items` array saves ≤10 in one call with shared scrubbing and in-batch dedup, reporting each entry's disposition (`disposition`/`mergedInto`), one failure not blocking the rest (`count`/`items`/`failed` summary); `placard` attaches a marker (scored unique · distinctive · dated; low scores get a rewrite hint); `entities` carries entity mentions (resolved and linked into the entity lexicon with the memory) |
|
|
87
|
+
| `engram_search` | Hybrid semantic + keyword retrieval (hits reinforce confidence); `room` routes through the corridor — search inside one room only; the top 5 hits carry same-room neighbouring-slot cues, and hit rows carry linked-entity tags; `asOf` does point-in-time lookback — only entity facts valid at that moment count, and hit rows carry point-in-time facts of linked entities at the row tail. Output rows carry `id=` and `ref=`; the tail carries the batch id |
|
|
88
|
+
| `engram_facts` | Entity fact-chain read/write. Write mode takes a `facts` array (≤50 entries, each `entity`/`entityId` + `content`); `replaces` supersedes an old fact (soft-invalidated, history chain kept); bad entries land in `failed` without interrupting the rest. Query mode takes `entity` (by name; no entity is created on miss, ambiguity lists candidates, near misses suggest alternatives) or `entityId` (exact); `includeInvalid` returns the full chain including invalidated facts, `asOf` filters to facts valid at that point in time |
|
|
86
89
|
| `engram_assess` | Evidence gate: before answering, judge whether the retrieved content suffices. Submit `batchId` + ≤8 `evidenceRefs` (only `ref=` values from that batch's output) + `missing` + `nextStrategy`; the code requires all three for `sufficient` (claimed adequate, at least one valid in-batch evidence ref, `nextStrategy=answer`), otherwise it rules insufficient and rewrites the strategy back to retrieval; refs from other batches are rejected and listed, and the verdict is written to the audit log |
|
|
87
90
|
| `engram_timeline` | Timeline browsing: creation-time descending by default; `order: 'tour'` follows the fixed tour route's slot order instead (output carries palace coordinates, entries off-route sorted last) so the agent can re-walk the route |
|
|
88
91
|
| `engram_episode_timeline` | Dedicated timeline for episodic memories: browse experiences by date range and source session, grouped by session (newest group first, entries chronological within; `sessionId` drills into one session; group headers carry the one-sentence session summary generated during capture or at session end). With `around` set to a memory id, it switches to temporal-neighborhood expansion — listing episodes within ± a window (`windowMinutes`, default 60) of the anchor's creation time, answering "what else happened around then" |
|
|
@@ -96,7 +99,7 @@ Sensory and emotional dimensions (smell, temperature, emotional weight) are deli
|
|
|
96
99
|
| `engram_neighbors` | Corridor walk: from one room, follow 1-3 hops of relation edges and return a neighbour summary |
|
|
97
100
|
| `engram_tour` | Tour routing: `mode=fixed` walks the whole palace in fixed slot order (constant route, sequential extraction); `mode=thematic` plans 3-7 stops around a theme |
|
|
98
101
|
| `engram_audit_forgotten` | Closed-wing archaeology: list recent closed entries with their epitaphs to review whether past forgetting was sound |
|
|
99
|
-
| `engram_ingest_history` | History backfill: distil past dsh sessions into the palace turn by turn (per-cwd stores; already-captured turns skipped). After each session's turns are captured, an auxiliary LLM generates a one-sentence session summary stored for timeline group headers (live sessions get the same summary at dispose once the final turn is captured; failures stay silent and never block). `dryRun` defaults to true (estimate only); pass `dryRun=false` to run. Use the Settings "
|
|
102
|
+
| `engram_ingest_history` | History backfill: distil past dsh sessions into the palace turn by turn (per-cwd stores; already-captured turns skipped). After each session's turns are captured, an auxiliary LLM generates a one-sentence session summary stored for timeline group headers (live sessions get the same summary at dispose once the final turn is captured; failures stay silent and never block). `dryRun` defaults to true (estimate only); pass `dryRun=false` to run. Use the Settings "Backfill" tab for large batches |
|
|
100
103
|
| `engram_export` | Export Markdown / JSON files (data portability); `redactedView: true` emits a redacted view (secondary scrubbing + 40-char preview truncation, share-safe) |
|
|
101
104
|
| `engram_distill` | Distill: merge same-topic clusters into higher-level rules (LLM) |
|
|
102
105
|
| `engram_profile_edit` | Edit the curated profile block: `view` shows the current content and version history; `edit` writes a new version (optimistic lock via `expectedVersion`, conflicts fail loud); `rollback` restores a historical version (appended as a new version; history is never rewritten). The curated profile is injected ahead of the derived profile; every edit lands in the operation log, and the profile card on the panel's Today view shows version diffs |
|
|
@@ -137,6 +140,19 @@ Optional configuration (cordis.yml):
|
|
|
137
140
|
historyBackfillIncludeSubagents: false # exclude subagent sessions by default
|
|
138
141
|
historyBackfillIncludeSeeded: false # exclude seeded sessions by default
|
|
139
142
|
historyBackfillIncludeNoCwd: false # exclude sessions without cwd by default (they can only go to the user store)
|
|
143
|
+
# Jev System-One adjudication (off by default; enabling requires an apiKey; failures silently degrade to the pure-rule four-way behavior)
|
|
144
|
+
# DEFER fuzzy-band three-way ruling: probability ≥ deferMergeAbove → same memory (merge), ≤ deferAcceptBelow → different (accept), in between stays defer;
|
|
145
|
+
# before a defer lands, contradiction edges are confirmed by Jev first (an edge is created only at probability ≥ contradictMinProbability).
|
|
146
|
+
jev:
|
|
147
|
+
enabled: false # switch
|
|
148
|
+
# apiKey: '' # Jev API key (Bearer auth)
|
|
149
|
+
# baseUrl: 'https://api.typesafe.ai' # official endpoint, POST /v1/systemone
|
|
150
|
+
# model: 'jev-latest' # adjudication model name
|
|
151
|
+
# timeoutMs: 3000 # per-request timeout in ms (1000-60000)
|
|
152
|
+
# deferMergeAbove: 0.85 # fuzzy-band probability ≥ this → same memory (merge)
|
|
153
|
+
# deferAcceptBelow: 0.15 # fuzzy-band probability ≤ this → different (accept)
|
|
154
|
+
# contradictMinProbability: 0.8 # contradiction confirmation probability ≥ this → create the contradicts edge
|
|
155
|
+
# The five fields above (enabled/apiKey/baseUrl/model/timeoutMs) can be overridden from the panel's "Jev" tab (stored at <dbDir>/jev-config.json, field-level override, effective immediately on save); the three thresholds are configurable only here
|
|
140
156
|
```
|
|
141
157
|
|
|
142
158
|
## How It Works
|
|
@@ -165,17 +181,21 @@ Session agent Host half (Node)
|
|
|
165
181
|
|
|
166
182
|
On profiles with a webServer (web, etc.), a "Memory Library" tab appears in **Settings** automatically (registered through the `settings.section` slot; the client half is a React component loaded from `lib/client.js` through the host module table): stat cards, filter by status/kind/content, inline detail and edit (through the supersede chain), forget/restore, Markdown/JSON export. Data flows through the loopback API `/api/engram/*` (writes verify the loopback Origin). Profiles without a webServer (headless, etc.) skip the panel; every other capability is unaffected.
|
|
167
183
|
|
|
168
|
-
Since v0.7.2 the Today view opens with a **"Due today" card**: each due memory is listed by cue only (`room#slot` · placard · overdue days), the body appears after pressing "Reveal placard", and you then self-rate it as Remembered / Hazy / Forgot (mapped to SM-2 grades 5/3/1) to advance the schedule. When reviews are due, a **red header badge** shows the count and jumps straight to that card; answering decrements it. The
|
|
184
|
+
Since v0.7.2 the Today view opens with a **"Due today" card**: each due memory is listed by cue only (`room#slot` · placard · overdue days), the body appears after pressing "Reveal placard", and you then self-rate it as Remembered / Hazy / Forgot (mapped to SM-2 grades 5/3/1) to advance the schedule. When reviews are due, a **red header badge** shows the count and jumps straight to that card; answering decrements it. The Palace list gains a **"By tour route"** sort toggle for walking memories in fixed slot order, and each row shows its palace coordinate.
|
|
169
185
|
|
|
170
|
-
Since v0.7.3 there is a dedicated **"
|
|
186
|
+
Since v0.7.3 there is a dedicated **"Backfill" tab**: you choose every import rule (time window / turns per session / total turn budget / include subagent·seeded·no-cwd sessions), press "Re-estimate" to see candidate sessions and pending turns at zero cost (no LLM calls), then start. While running it shows progress (sessions / turns / written / skipped / failed) and can be paused at any time — finished turns are skipped by idempotency key, so starting again resumes. The same release rebuilds the panel as five views (Today / Palace / Corridor / Log / Backfill): the always-on nine-cell curator bar folds into a summary card at the top of Today (three headline metrics + last-7-days counts + health ring), and the front page keeps only what to do (due today, refurb) and what to reference (room directory, tour proposal); the corridor bird's-eye and the retrieval bench move to Corridor; the Log becomes a full page filterable by writes / capture / retrieval / organize; and each of the five rooms gets its own hue (fact blue / preference purple / decision teal / episode orange / skill magenta) across tags, the room directory, and corridor nodes. The exhibition list becomes compact rows: hairlines instead of cards, two-line body, action buttons revealed on hover (or keyboard focus) and wrapped below the body on narrow screens — roughly twice as many rows per screen. The refurb list stops scanning a palace with fewer than 8 active memories, avoiding small-store noise.
|
|
171
187
|
|
|
172
|
-
Since v0.7.4 the panel gets a second pass of polish: Today is rearranged to "tour proposal on the left, room directory + due today + refurb list on the right", with roomier rows in the tour proposal and the room directory; the
|
|
188
|
+
Since v0.7.4 the panel gets a second pass of polish: Today is rearranged to "tour proposal on the left, room directory + due today + refurb list on the right", with roomier rows in the tour proposal and the room directory; the Palace toolbar becomes one row of search + status + sort with the room filter as its own wrapping chip row; the Log merges its counts and category filter into a single toolbar and marks every row with a category dot (writes / capture / retrieval / organize); Backfill's rules, estimate, and run blocks are separated by hairlines instead of a tinted estimate box; and section spacing now comes solely from the container gap, removing the asymmetry where a heading hugged the card above but sat far from the one below.
|
|
173
189
|
|
|
174
|
-
Since v0.7.5: two underlying fixes plus one new capability. ① Profile-injection token estimation is now CJK-aware (CJK at 1.5 tokens/char, other text at 4 chars/token); the old "length / 4" undercounted Chinese by more than 4×, so Chinese users' injections routinely exceeded `injectTokenBudget` by 17%–50% — the trailing `+N more` counter line now counts against the budget too, so the injection never exceeds its promise. ② A new **evidence gate** `engram_assess` (17 tools): a hit means "relevant", not "enough to answer". `engram_search` registers an in-process evidence batch per call (latest 20 per session, released when the session ends) and prints `ref=` per row plus the batch id; `engram_assess` may only cite refs of that batch and `sufficient` is enforced by code — the model's claim, at least one valid evidence ref, and `nextStrategy=answer` must all hold, otherwise the verdict is insufficient with the strategy rewritten back to retrieval; refs from other batches are rejected and listed, and the verdict goes to the audit log (visible under the "retrieval" category of the
|
|
190
|
+
Since v0.7.5: two underlying fixes plus one new capability. ① Profile-injection token estimation is now CJK-aware (CJK at 1.5 tokens/char, other text at 4 chars/token); the old "length / 4" undercounted Chinese by more than 4×, so Chinese users' injections routinely exceeded `injectTokenBudget` by 17%–50% — the trailing `+N more` counter line now counts against the budget too, so the injection never exceeds its promise. ② A new **evidence gate** `engram_assess` (17 tools): a hit means "relevant", not "enough to answer". `engram_search` registers an in-process evidence batch per call (latest 20 per session, released when the session ends) and prints `ref=` per row plus the batch id; `engram_assess` may only cite refs of that batch and `sufficient` is enforced by code — the model's claim, at least one valid evidence ref, and `nextStrategy=answer` must all hold, otherwise the verdict is insufficient with the strategy rewritten back to retrieval; refs from other batches are rejected and listed, and the verdict goes to the audit log (visible under the "retrieval" category of the Log). ③ The Log gains the missing op labels and detail formatting (consolidation / review answer / slot assigned / slots backfilled / room opened) instead of raw op names and JSON.
|
|
175
191
|
|
|
176
192
|
Since v0.7.6 scopes stop being hardcoded and the project palace follows the workspace. ① **Per-candidate palace on capture**: extraction now emits `scope` — content tied to the current project/repo (tech choices, project conventions, architecture decisions) goes to that project store, while cross-project material (personal preferences, habits, the user's own history) goes to the private store. Live capture (previous turn, final turn on dispose, pending replay) and history backfill share the same rule, and backfill files each session into **its own cwd's** store. Idempotency keys (`ingest-done` / `ingest-pending`) stay in the private store, decoupled from where memories land, so live and backfill share one `(session, turn)` key set and never re-ingest across paths. ② **Project palace follows the workspace**: `GET /api/engram/workspaces` lists the selectable project palaces (host workspace registry → cwds seen in session headers → plugin process directory, deduped by store name, so several worktrees of one repo share a palace) and every project-scoped endpoint takes `?project=<dbName>` (unknown selectors answer 404); `engram_save/search` and friends resolve their project store from the current session's cwd. In the panel, a new row under the three scope segments shows the **active workspace chip plus a workspace picker** (default "follow the current workspace", pinnable to one workspace) and switching workspaces switches every view. ③ Without git, the project store name changes from "first 12 chars of cwd" (which collided for sibling directories) to a **full cwd sha256**, with the old store renamed on startup; and unloading the plugin now closes every store connection (no more locked `.db` files on Windows).
|
|
177
193
|
|
|
178
|
-
Since v0.7.9 the memory-tiers phase begins. ① **Editable profile**: the new `engram_profile_edit` tool (19 tools) maintains the curated profile — the memory you explicitly vouch for — with `view` (current content + version history), `edit` (optimistic lock, conflicts fail loud), and `rollback` (restores a historical version as a new append; history is never rewritten); on injection the curated text runs ahead of the derived profile and claims the token budget first, every edit lands in the operation log, and the profile card on the panel's Today view shows version diffs. ② **Dedicated episode timeline**: the new `engram_episode_timeline` tool browses experiences by date range and source session, and with `around` set to an anchor id it switches to temporal-neighborhood expansion (episodes within ± a window of the anchor's moment), answering "what else happened around then". ③ **Panel "
|
|
194
|
+
Since v0.7.9 the memory-tiers phase begins. ① **Editable profile**: the new `engram_profile_edit` tool (19 tools) maintains the curated profile — the memory you explicitly vouch for — with `view` (current content + version history), `edit` (optimistic lock, conflicts fail loud), and `rollback` (restores a historical version as a new append; history is never rewritten); on injection the curated text runs ahead of the derived profile and claims the token budget first, every edit lands in the operation log, and the profile card on the panel's Today view shows version diffs. ② **Dedicated episode timeline**: the new `engram_episode_timeline` tool browses experiences by date range and source session, and with `around` set to an anchor id it switches to temporal-neighborhood expansion (episodes within ± a window of the anchor's moment), answering "what else happened around then". ③ **Panel "Episodes" view**: the memory-library tab gains a sixth view listing episodes grouped by session, with a "Nearby" action per entry that expands the temporal-neighborhood card in one click. ④ **Session-summary group headers**: history backfill asks an auxiliary LLM for a one-sentence summary after each session's turns are captured, and live sessions do the same once the final-turn capture on dispose succeeds (reusing the active route; failures stay silent) into a `session_summaries` table; timeline group headers carry it (visible in tool output, the API, and the panel header alike), so reviewing the timeline no longer means expanding every entry.
|
|
195
|
+
|
|
196
|
+
Since v0.7.10 the entity layer arrives (first batch of the temporal fact graph). ① **Entity extraction folded into the auxiliary call**: the auxiliary LLM output of automatic capture and `engram_save` carries `entities` mentions (name required, ≤80 chars; kind person/project/tool/concept/other; aliases optional), resolved by normalized name (trim + whitespace collapse + lowercase) against existing entities' names/aliases — a hit reuses the entity and refreshes its "last mentioned" stamp (the lexicon lists by it, descending), a miss creates one; memory↔entity links are written idempotently, stored in the same store as the source memory; entity-resolution failures silently skip the links and memories still land. ② **Seventh panel view "Entities"**: a paged lexicon (last-mentioned descending) with each entity's active linked-memory count, kind filters, substring search over names/aliases, and a "View" expansion per entity (entity record + recent linked memories); new `GET /api/engram/entities` and `GET /api/engram/entity` loopback routes. ③ **Retrieval and export carry entities**: `engram_search` hit entries carry linked entities (id + canonical name); the `engram_export` JSON view includes the entity lexicon. Schema v9 → v10 (the entities + node_entities tables, migrated automatically on startup; existing data preserved).
|
|
197
|
+
|
|
198
|
+
Since v0.7.12 the plugin integrates **Jev System-One adjudication** (off by default, the `jev` block in yml) and adds an eighth panel view, **"Jev"**: the DEFER fuzzy-band three-way ruling and contradiction-edge confirmation can be delegated to Jev, and when Jev is unavailable (network failure, timeout, or a missing key) it silently falls back to the pure-rule four-way behavior without blocking writes. The panel overrides five yml fields — the switch, API key, endpoint, model, and timeout — stored at `<dbDir>/jev-config.json` (0600 file inside a 0700 directory) and effective immediately on save: the judge re-reads the override at every assembly point, no restart needed; the three thresholds stay yml-only advanced settings, shown read-only in the panel. Key safety: GET responses return only a mask (last 4 characters when longer than 8), the plaintext never leaves the process, and the input field is never back-filled. Turning the switch on without any key degrades to off automatically; a corrupted override file heals as an empty override, fixed by the next panel save. The Jev tab also gains a "Test connection" action (verify endpoint and model reachability right after filling in the key — unsaved input is tested as typed and never written to the override file) and a "Recent adjudications" card (the last 50 adjudications in this process: verdict, whether Jev took part, elapsed time, and the fallback reason; cleared on restart), so "why was this one deferred" no longer needs guesswork.
|
|
179
199
|
|
|
180
200
|
## Development
|
|
181
201
|
|
|
@@ -196,7 +216,7 @@ Each turn's first step appends a plugin-source user snapshot: `User memory profi
|
|
|
196
216
|
|
|
197
217
|
#### Token effect
|
|
198
218
|
|
|
199
|
-
Profile injection is a conditional fixed cost (bounded by both the entry cap and the token budget); the tool schemas are a standing cost (
|
|
219
|
+
Profile injection is a conditional fixed cost (bounded by both the entry cap and the token budget); the tool schemas are a standing cost (20 narrow-parameter tools).
|
|
200
220
|
|
|
201
221
|
#### KV Cache effect
|
|
202
222
|
|
|
@@ -204,7 +224,7 @@ Profile text changes as the memory store changes — changes only land at turn b
|
|
|
204
224
|
|
|
205
225
|
## Known Limitations and Deferred Work
|
|
206
226
|
|
|
207
|
-
- **
|
|
227
|
+
- **Contradiction checks are LLM-free by default** — writes report candidates by vector similarity (≥0.88) and create edges; semantic-contradiction confirmation is left to model/user adjudication and distillation. Optional Jev adjudication (the panel's "Jev" tab or the yml `jev` block) routes the fuzzy band and contradiction edges to an external judge, falling back to this default whenever Jev is unavailable.
|
|
208
228
|
- **Memories written during embedder degradation have no vectors** — memories written before the model is ready do not participate in the semantic track; after semantics come online run `pnpm backfill` once to backfill existing vectors (after `pnpm build` has warmed the model cache; `HF_ENDPOINT` configurable).
|
|
209
229
|
|
|
210
230
|
## Roadmap
|
|
@@ -212,8 +232,8 @@ Profile text changes as the memory store changes — changes only land at turn b
|
|
|
212
232
|
The palace IA (position-as-index / fixed routes / retrieval practice) is the indexing and audit skeleton; it evolves toward AI-native memory tiers and a temporal fact graph in three phases. Actual releases increment by 0.0.1; the phase labels are planning codenames only:
|
|
213
233
|
|
|
214
234
|
- **Phase 1 · Flywheel closure** (done, v0.7.8): four-state write disposition (ACCEPT new / MERGE reinforce-in-place / DROP low-entropy / DEFER contradiction-pending — `engram_save` and capture return it per item); graduated decaying injection budget (160 chars for the first entry, ×0.9 per subsequent entry, floor 24, all configurable); redaction extended to Chinese passwords, national ID numbers, and phone numbers; evidence-gate wrap-up reminder (inject a reminder when a turn ends with unassessed search batches).
|
|
215
|
-
- **Phase 2 · Memory tiers**: editable profile (**done, v0.7.9**: `engram_profile_edit` tool with version-chain rollback and panel diff audit, following Letta's core-memory model); a dedicated timeline index for episode memories (**done, v0.7.9**: `engram_episode_timeline` tool with date ranges, per-session grouping, temporal-neighborhood expansion, and a dedicated episode index; the panel "
|
|
216
|
-
- **Phase 3 · Temporal fact graph**: entity table with write-time extraction and resolution; fact-level triples carrying `valid_at` / `invalid_at` windows with soft-invalidated conflicts (
|
|
235
|
+
- **Phase 2 · Memory tiers**: editable profile (**done, v0.7.9**: `engram_profile_edit` tool with version-chain rollback and panel diff audit, following Letta's core-memory model); a dedicated timeline index for episode memories (**done, v0.7.9**: `engram_episode_timeline` tool with date ranges, per-session grouping, temporal-neighborhood expansion, and a dedicated episode index; the panel "Episodes" view plus capture-time and live-session summary group headers landed in the same batch); automated distillation (cluster size + similarity thresholds, suggest/auto modes); configurable room capacity and placard scoring (defaults unchanged).
|
|
236
|
+
- **Phase 3 · Temporal fact graph**: entity table with write-time extraction and resolution (**done, v0.7.10**: the entities + node_entities tables; capture/save auxiliary output carries entity mentions resolved on write; `engram_search` hits carry entity tags; panel "Entities" view + entity loopback routes); fact-level triples carrying `valid_at` / `invalid_at` windows with soft-invalidated conflicts (**done, v0.7.11**: the facts table (entity + time window + `replaced_by` + source memory), capture auxiliary output carries facts matched into the table by normalized entity name, `replaces` declares the soft-invalidation chain, `engram_search` gains `asOf` point-in-time lookback, the `engram_facts` tool reads and writes fact chains, and the panel entity page shows fact-chain details); automatic supports / refines / related edge creation; a LoCoMo-zh evaluation subset in CI as a retrieval-quality regression gate (deterministic Recall@k / MRR metrics).
|
|
217
237
|
|
|
218
238
|
## Acknowledgements
|
|
219
239
|
|
package/README.md
CHANGED
|
@@ -64,25 +64,28 @@ dsh plugin --profile web add @kenz1117/dsh-engram
|
|
|
64
64
|
- **记忆宫殿信息架构**(v0.7.2+):四原则全部落进核心路径,而非展示层皮肤——**位置当索引**(写入按 kind 分房并钉「房间#桩位」坐标,房间容量 9,满员开新房,桩位只增不回收);**固定路线定顺序**(`tour_routes` append-only,`engram_tour mode=fixed` 按桩位顺序走全宫);**骨架长期复用**(同一 topic 永远落在同一房间同一序号,顺序提取而非重新检索);**标记独一无二**(门牌规则 0-1 评分:全库唯一 +0.4 / 带日期锚点 +0.3 / 同房前 6 字不重复 +0.3,低分进翻新清单;库内 active 记忆少于 8 条时不扫描,避免小库噪声)。
|
|
65
65
|
- **走廊路由检索**:画像里附房间目录,`engram_search` 支持 `room` 参数——先决定进哪个房间,再在房内检索,而非一上来全库 RRF。检索命中 top5 附同房间相邻桩位 id 作为编码特异性线索。
|
|
66
66
|
- **检索练习闭环**(间隔重复):`engram_review_queue` 只给宫殿坐标与门牌线索、**不给正文**,迫使模型先主动回忆;`engram_review` 揭示核对,`engram_report grade`(0-5)自评推进 SM-2 调度(1 → 6 → round(prev × ease) 天,失败重置,ease 下限 1.3)。进入复习调度的条目**不再参与自动衰减**——命运由回忆结果决定。会话开始注入会提示今日待回忆条数。
|
|
67
|
-
- **历史会话回填**(v0.7.3+):把 dsh 已持久化的历史会话逐轮提炼进宫殿——默认按**每个会话自己的 cwd** 写进对应项目库(不串库),同样逐条判作用域(跨项目通用的个人偏好落私人库),复用实时摄取的节流/脱敏/防回声,靠 (会话, 轮次) 幂等键支持中断续跑(键固定在私人库,与实时路径共用);回填条目不进今日复习队列(避免一次性回填淹没「今日待回忆」)。**导入规则由你选**(时间窗 / 单会话轮数 / 总轮数上限 / 辅助模型 / 是否含子代理·种子·无 cwd 会话),先估算(零成本、不调 LLM
|
|
67
|
+
- **历史会话回填**(v0.7.3+):把 dsh 已持久化的历史会话逐轮提炼进宫殿——默认按**每个会话自己的 cwd** 写进对应项目库(不串库),同样逐条判作用域(跨项目通用的个人偏好落私人库),复用实时摄取的节流/脱敏/防回声,靠 (会话, 轮次) 幂等键支持中断续跑(键固定在私人库,与实时路径共用);回填条目不进今日复习队列(避免一次性回填淹没「今日待回忆」)。**导入规则由你选**(时间窗 / 单会话轮数 / 总轮数上限 / 辅助模型 / 是否含子代理·种子·无 cwd 会话),先估算(零成本、不调 LLM)再执行;设置页有独立的**「回填」tab**,可看进度(含**跳过原因分布**)与暂停续做。辅助调用默认**复用你当前在用的模型**(历史日志里记的是当年的 provider/model,在当前环境可能已不可用),也可在面板「辅助模型」下拉里从宿主已注册的 provider/model 中直接指定。
|
|
68
68
|
- **知识飞轮**:摄取/保存 → 矛盾候选(写入时高相似近邻建 `contradicts` 边并报告,模型/用户裁决)→ 命中强化(confidence +0.05)→ 蒸馏(同主题簇合并为高层规律、supersedes 取代链、置信度继承)→ 衰减(低重要性且长期未访问归档,可恢复)。
|
|
69
|
+
- **Jev 系统一裁决**(v0.7.12+,默认关闭,yml `jev` 子配置):DEFER 模糊带的三路判定(并入 / 放行 / 搁置)与矛盾边确认可交外部 Jev 模型判决;Jev 不可用(网络失败 / 超时 / 未配密钥)时静默回落纯规则四态,**不阻断写入**。每次裁决旁路留观测(判定结果、Jev 是否参与、耗时、回落原因,进程内最多 50 条、重启清空),面板「裁决」视图一眼排查「为什么这条被搁置」。yml 的开关 / 密钥 / 端点 / 模型 / 超时五项可在面板覆盖(存 `<dbDir>/jev-config.json`,0600 文件 + 0700 目录),保存即时生效无需重启;「测试连接」按钮用当前输入即时验证端点与模型可达性(未保存也按当前值测、不落覆盖文件);密钥 GET 只回掩码、明文不出进程、输入框不回填。三阈值保持 yml 高级配置,面板只读展示。
|
|
70
|
+
- **实体词典**(v0.7.10+):自动摄取与 `engram_save` 的辅助 LLM 输出同时抽取实体提及(人物/项目/工具/概念,可带别名),按归一化名与既有实体的 name/aliases 精确匹配消解——命中复用并刷新「最近提及」,未命中新建;记忆与实体多对多关联,随来源记忆所在 scope 分库。`engram_search` 命中行附关联实体标签;实体消解故障只静默跳过关联,不影响记忆落库。面板「实体」视图按类别筛选、按名称/别名搜索,点开单实体看它牵出的所有记忆。
|
|
69
71
|
- **自动摄取**(`ingest` 配置开启时):新一轮第一步从会话日志提取上一轮的候选事实,会话结束时补摄取最后一轮(失败留 pending 键,下次会话自动补做,幂等不重复),低 confidence 写入,嵌入可用时按四态处置近邻(复述并入强化 / 疑似矛盾建边待裁决 / 全新写入)——不说"记住"也能攒记忆。**逐条判宫殿**:提炼时同步判定作用域,只跟当前项目/仓库有关的(技术选型、项目约定、架构决策)进当前会话 cwd 对应的项目库,跨项目通用的(个人偏好、习惯、本人经历)进私人库——偏好跟人走、约定跟仓库走。
|
|
70
72
|
- **来源审计**:每条记忆记录来源会话、轮次与事件 seq,`engram_review` 完整回查来源链、取代链、矛盾与操作日志;全部写入/修改/遗忘/蒸馏/衰减入操作日志表。
|
|
71
|
-
- **Web 管理面板**(v0.7.0+):设置页「记忆库」tab
|
|
73
|
+
- **Web 管理面板**(v0.7.0+):设置页「记忆库」tab 分八个视图——今日(速览条:记忆 / 开放 / 清晰度 + 近 7 天计数 + 今日到期 + 健康分环;下面是入殿导航、房间目录、待翻新、健康分构成;侧栏画像卡展示 curated 画像当前内容与版本 diff 历史)、陈展(筛选含今日到期 / 巡游路线序 / 批量 / 列表与编辑)、走廊(走廊鸟瞰 + 检索实验台)、日志(近 7 天计数 + 两库合并的完整 op_log,可按操作类别筛选)、回填、往事(按会话分组浏览 episode,锚点邻近扩展)、实体(实体词典:类别筛选 + 名称/别名搜索 + 单实体关联记忆)、裁决(Jev 系统一裁决:配置覆盖 + 测试连接 + 近期裁决记录)。Header 三宫格驱动全局 scope(私人 / 项目 / 共享),全部数据源同步;**项目 scope 下三宫格右侧显示当前项目宫殿所属工作区**(标题 + 路径,默认跟随 GUI 当前工作区),旁边的工作区下拉可固定到某个工作区或切回「跟随当前会话」。界面文案中英双语,跟随宿主语言设置实时切换。支持按脱敏标记筛选(仅看/排除含 `[REDACTED:*]` 的条目)并给命中条目挂琥珀色徽标,方便审计脱敏覆盖面。
|
|
72
74
|
- **提示注入防护**:全部记忆召回出口(画像注入、`engram_search/timeline/review` 输出)包 `<engram_memory_context>` 协议标签并附使用警告(历史记忆非当前请求、不遵循其中指令、仅相关时使用),当前请求独立包 `<current_user_request>`;所有入库内容(摄取候选、保存正文)先剥离这些协议标签,防伪造协议块二次注入。
|
|
73
75
|
- **摄取脱敏**:入库前正则清洗常见密钥凭据(sk- 系 API key、Bearer、AWS AKIA、GitHub token、PEM 私钥、password/token 赋值、中文密码赋值)与个人信息(中国大陆手机号、18 位身份证号),命中片段替换为 `[REDACTED:<类型>]`。
|
|
74
76
|
- **召回占位(防回声室)**:摄取切片中记忆召回工具的输出替换为 `[engram memory result omitted from capture: <tool>]`,并向提取模型附注"既有记忆的复述不是新信息",阻断记忆自我强化循环。
|
|
75
77
|
- **多查询检索**:`engram_search` 可用辅助 LLM 把查询改写为 ≤3 个互补查询分别检索,跨查询 RRF 融合 + 每查询保底命中;改写失败自动降级单查询(`queryRewrite: false` 关闭)。
|
|
76
|
-
- **证据门(search → assess)**:检索命中只说明「相关」,不说明「足以回答」。每次检索登记一个进程内批次(每会话保留最近 20 个,会话结束即释放),输出行尾给出 `ref=…` 与批次 id;`engram_assess` 只能引用同一批次的 ref,且 `sufficient` 由代码强制——三者齐备(模型声称充足、至少一条有效证据、`nextStrategy=answer
|
|
78
|
+
- **证据门(search → assess)**:检索命中只说明「相关」,不说明「足以回答」。每次检索登记一个进程内批次(每会话保留最近 20 个,会话结束即释放),输出行尾给出 `ref=…` 与批次 id;`engram_assess` 只能引用同一批次的 ref,且 `sufficient` 由代码强制——三者齐备(模型声称充足、至少一条有效证据、`nextStrategy=answer`)才算充足,否则判为不足并把策略改回继续检索。判定与拒绝明细写入审计日志,面板「日志」的「检索」类别可见。下一步开始前若仍有未判定批次,注入收尾提醒引导补判或说明不判(连续不足 ≥2 次时建议换检索方式或询问用户;`assessReminder: false` 关闭)。
|
|
77
79
|
- **数据可携带**:`engram_export` 一键导出 Markdown / JSON 文件,支持脱敏视图(内容二次清洗 + 预览截断,分享安全)。`engram_mirror` 导出可漫游的镜像目录(Obsidian / Logseq 友好:每条记忆一个 Markdown,正文 + YAML frontmatter + 双向链接 `[[id]]`),让「宫殿」也成为可人读的私人知识库。
|
|
78
80
|
- **认知架构探索(dsh-market · AGI 架构探索)**:本仓库是 dsh-market「AGI 架构探索」类目下,对 agent 长期记忆的认知科学方法论重构——记忆宫殿(意象标签 + 房间铭牌)、走廊拓扑(力导向图)、闭环提问(摄入时让模型主动追问用户细节)、巩固合并(启发式去重 + 余弦相似度),与 MemGPT/Letta 同层「agent 记忆架构」叙事。
|
|
79
81
|
|
|
80
|
-
## 工具(
|
|
82
|
+
## 工具(20 个,窄参数)
|
|
81
83
|
|
|
82
84
|
| 工具 | 作用 |
|
|
83
85
|
|---|---|
|
|
84
|
-
| `engram_save` | 保存(**写入四态回报**:复述并入强化既有条目(merge,不新建)、疑似矛盾落库建边待裁决(defer)、全新写入(accept),低熵内容丢弃(drop);嵌入不可用时全部 accept);支持 `items` 数组单次批量保存 ≤10 条,统一清洗/批量内去重,逐条回报处置(`disposition`/`mergedInto`),单条失败不影响其余(`count`/`items`/`failed` 汇总返回);`placard`
|
|
85
|
-
| `engram_search` | 语义 + 关键词混合检索(命中强化置信度);`room` 参数做走廊路由——只在指定房间内检索;命中 top5
|
|
86
|
+
| `engram_save` | 保存(**写入四态回报**:复述并入强化既有条目(merge,不新建)、疑似矛盾落库建边待裁决(defer)、全新写入(accept),低熵内容丢弃(drop);嵌入不可用时全部 accept);支持 `items` 数组单次批量保存 ≤10 条,统一清洗/批量内去重,逐条回报处置(`disposition`/`mergedInto`),单条失败不影响其余(`count`/`items`/`failed` 汇总返回);`placard` 挂门牌(按唯一·差异化·带日期评分,低分附改写建议);`entities` 附实体提及(与记忆消解关联进实体词典)。`scope=project` 落当前会话 cwd 对应的项目宫殿 |
|
|
87
|
+
| `engram_search` | 语义 + 关键词混合检索(命中强化置信度);`room` 参数做走廊路由——只在指定房间内检索;命中 top5 附同房相邻桩位线索,命中行附关联实体标签;`asOf` 做时点回看——只认该时刻有效的实体事实,命中行尾附关联实体的时点事实。输出行尾给 `id=` 与 `ref=`,末尾给批次 id。`scope=project` 查当前会话 cwd 对应的项目宫殿 |
|
|
88
|
+
| `engram_facts` | 实体事实链读写。写入模式传 `facts` 数组(≤50 条,每条 `entity`/`entityId` + `content`),`replaces` 声明取代旧事实(软失效,历史链保留),坏条目进 `failed` 不中断其余;查询模式传 `entity`(按名查询,未命中不新建、歧义列候选、近邻给建议)或 `entityId`(精确),`includeInvalid` 看全链含已失效,`asOf` 只看该时点有效的事实 |
|
|
86
89
|
| `engram_assess` | 证据门:作答前判定「检索到的内容是否足以回答」。提交 `batchId` + ≤8 条 `evidenceRefs`(只能取该批次输出里的 `ref=`)+ `missing` + `nextStrategy`;代码强制 `sufficient` 需同时满足「声称充足」「至少一条属于本批次的有效证据」「nextStrategy=answer」,否则判为不足并把策略改回继续检索;非本批次的 ref 会被拒绝并列出,判定写入审计日志 |
|
|
87
90
|
| `engram_timeline` | 时间线浏览:默认按创建时间倒序;`order: 'tour'` 改按固定巡游路线桩位顺序(输出附宫殿坐标,未上路线者排末尾),让 agent 也能沿固定路线复述 |
|
|
88
91
|
| `engram_episode_timeline` | episode 情景独立时间线:按日期范围与来源会话浏览经历,输出按会话分组(组间新→旧、组内时间升序,`sessionId` 可精确回查单会话,组头附摄取期或会话结束时生成的一句话会话摘要);`around` 传锚点记忆 id 时切换为时间邻近扩展——列出锚点时刻 ± 窗口(`windowMinutes`,默认 60)内的情景,回答「当时前后还发生了什么」 |
|
|
@@ -96,7 +99,7 @@ dsh plugin --profile web add @kenz1117/dsh-engram
|
|
|
96
99
|
| `engram_neighbors` | 走廊漫步:从一间出发走 1-3 跳关系边,返回邻居简表 |
|
|
97
100
|
| `engram_tour` | 巡游路由:`mode=fixed` 按固定桩位路线走全宫(路线恒定,顺序提取);`mode=thematic` 按主题动态规划 3-7 站 |
|
|
98
101
|
| `engram_audit_forgotten` | 闭馆考古:列最近已闭馆条目与墓志铭,复核过去的遗忘是否得当 |
|
|
99
|
-
| `engram_ingest_history` | 历史会话回填:把 dsh 历史会话逐轮提炼进宫殿(按会话 cwd 分库、已摄取轮次自动跳过)。每会话轮次摄取完成后用辅助 LLM 生成一句话会话摘要落库(失败静默不阻断),实时会话结束时同样收尾生成,供时间线组头回看。`dryRun` 缺省 true 只估算;`dryRun=false`
|
|
102
|
+
| `engram_ingest_history` | 历史会话回填:把 dsh 历史会话逐轮提炼进宫殿(按会话 cwd 分库、已摄取轮次自动跳过)。每会话轮次摄取完成后用辅助 LLM 生成一句话会话摘要落库(失败静默不阻断),实时会话结束时同样收尾生成,供时间线组头回看。`dryRun` 缺省 true 只估算;`dryRun=false` 才执行。大批量建议用设置页「回填」tab |
|
|
100
103
|
| `engram_export` | 导出 Markdown / JSON 文件(数据可携带);`redactedView: true` 输出脱敏视图(二次清洗 + 40 字预览截断,可安全分享) |
|
|
101
104
|
| `engram_distill` | 蒸馏:同主题簇合并为高层规律(LLM) |
|
|
102
105
|
| `engram_profile_edit` | 画像 curated block 编辑:`view` 看当前内容与版本历史;`edit` 写入新版本(乐观锁 `expectedVersion`,冲突 loud 失败);`rollback` 回滚到历史版本(作为新版本追加,历史永不改写)。curated 画像优先于派生画像注入;编辑记录入操作日志,面板今日页画像卡可查版本 diff |
|
|
@@ -137,6 +140,19 @@ dsh plugin --profile web add @kenz1117/dsh-engram
|
|
|
137
140
|
historyBackfillIncludeSubagents: false # 默认排除子代理会话
|
|
138
141
|
historyBackfillIncludeSeeded: false # 默认排除种子会话
|
|
139
142
|
historyBackfillIncludeNoCwd: false # 默认排除无 cwd 会话(这类只能进 user 库)
|
|
143
|
+
# Jev 系统一裁决(默认关闭;enabled=true 必须提供 apiKey;Jev 不可用时静默降级回纯规则四态)
|
|
144
|
+
# DEFER 模糊带三路裁决:概率≥deferMergeAbove 判同一条(merge),概率≤deferAcceptBelow 判不同条(accept),中间保持 defer;
|
|
145
|
+
# defer 落库前矛盾边先经 Jev 确认(概率≥contradictMinProbability 才建 contradicts 边)。
|
|
146
|
+
jev:
|
|
147
|
+
enabled: false # 开关
|
|
148
|
+
# apiKey: '' # Jev API 密钥(Bearer 认证)
|
|
149
|
+
# baseUrl: 'https://api.typesafe.ai' # 官方端点,POST /v1/systemone
|
|
150
|
+
# model: 'jev-latest' # 判决模型名
|
|
151
|
+
# timeoutMs: 3000 # 单次请求超时(毫秒,1000-60000)
|
|
152
|
+
# deferMergeAbove: 0.85 # 模糊带概率≥此值判同一条(merge)
|
|
153
|
+
# deferAcceptBelow: 0.15 # 模糊带概率≤此值判不同条(accept)
|
|
154
|
+
# contradictMinProbability: 0.8 # 矛盾确认概率≥此值才建 contradicts 边
|
|
155
|
+
# 以上 enabled/apiKey/baseUrl/model/timeoutMs 五项可在管理面板「裁决」tab 覆盖(存 <dbDir>/jev-config.json,字段级覆盖,保存即时生效);三阈值仅此处可配
|
|
140
156
|
```
|
|
141
157
|
|
|
142
158
|
## 工作原理
|
|
@@ -166,17 +182,21 @@ dsh plugin --profile web add @kenz1117/dsh-engram
|
|
|
166
182
|
|
|
167
183
|
宿主带 webServer 的 profile(web 等)会在**设置页**自动出现「记忆库」tab(经 `settings.section` 槽位注册,client 半为 React 组件、随 `lib/client.js` 由宿主模块表装载):统计卡片、按状态/种类/内容过滤、行内详情与编辑(走取代链)、遗忘/恢复、导出 Markdown/JSON 下载。数据经回环 API `/api/engram/*`(写操作校验回环 Origin)。headless 等无 webServer 的组合不挂载,其余能力不受影响。
|
|
168
184
|
|
|
169
|
-
v0.7.2 起「今日」视图首屏新增**「今日待回忆」卡**:按线索(房间#桩位 · 门牌 · 逾期天数)逐条列出待回忆记忆,点「揭示铭牌」才显示正文,随后以「记得 / 模糊 / 忘了」三档自评(映射 SM-2 grade 5/3/1)推进调度。有待回忆时 Header
|
|
185
|
+
v0.7.2 起「今日」视图首屏新增**「今日待回忆」卡**:按线索(房间#桩位 · 门牌 · 逾期天数)逐条列出待回忆记忆,点「揭示铭牌」才显示正文,随后以「记得 / 模糊 / 忘了」三档自评(映射 SM-2 grade 5/3/1)推进调度。有待回忆时 Header 出现**红色角标**(显示条数),点击直达该卡;答题后角标自动递减。陈展列表新增**「按巡游路线」排序**开关,可切到固定桩位顺序浏览,列表行同时显示每条记忆的宫殿坐标。
|
|
170
186
|
|
|
171
|
-
v0.7.3
|
|
187
|
+
v0.7.3 起新增独立的**「回填」tab**:导入规则全部由你选择(时间窗 / 单会话轮数 / 总轮数上限 / 是否包含子代理·种子·无 cwd 会话),点「重新估算」先看候选会话数与待处理轮数(零成本、不调 LLM),确认后「开始回填」;运行中显示进度(会话 / 轮次 / 写入条数 / 跳过 / 失败)并可随时暂停——已完成的轮次按幂等键跳过,再点开始即续做。同一版把面板重做成五个视图(今日 / 陈展 / 走廊 / 日志 / 回填):原常驻的九格「管家日报」条收进「今日」视图的速览卡(三个大指标 + 近 7 天计数 + 健康分环),首页只留在办与参考两块(待回忆、待翻新 / 房间目录、入殿导航);走廊鸟瞰与检索实验台移入「走廊」;日志独立成整页,可按落成类 / 发掘 / 检索 / 整理筛选;五间房各一色(事实蓝 / 偏好紫 / 决策青 / 往事橙 / 技法品红)贯穿标签、房间目录与走廊节点。陈展列表改为紧凑行式:分隔线取代卡片描边、正文两行截断、操作按钮 hover(或键盘聚焦)才显现、窄屏折到正文下方,一屏可读条目约翻一倍。翻新清单在库内 active 少于 8 条时不再扫描,避免小库噪声。
|
|
172
188
|
|
|
173
|
-
v0.7.4 起继续打磨面板细节:今日视图重排为「左入殿导航 ·
|
|
189
|
+
v0.7.4 起继续打磨面板细节:今日视图重排为「左入殿导航 · 右房间目录、今日待回忆、翻新清单」,入殿导航与房间目录加大行间距;陈展工具栏改「搜索 + 状态 + 排序」一行、房间筛选独立成可换行 chips;日志把计数与类别筛选合成一条工具条,并给每行加类别色点(落成 / 发掘 / 检索 / 整理);回填的规则、估算、执行三段改用分隔线切块;区域间距统一由容器间距给出,消除「标题贴住上方卡片、下方却过松」的不对称。
|
|
174
190
|
|
|
175
|
-
v0.7.5 起是两处底层修正加一层新能力。① 画像注入的 token 估算改为 CJK 感知(中文按 1.5 token/字、其余按 4 字符/token):此前按长度除以 4 会把中文低估四倍以上,中文用户的实际注入长期超出 `injectTokenBudget` 约 17%–50%;末尾 `+N more` 计数行也纳入预算,注入总量不再超承诺。② 新增**证据门** `engram_assess`(工具 17 个):检索命中只说明「相关」,不说明「足以回答」;`engram_search` 每次登记一个进程内证据批次(每会话保留最近 20 个,会话结束即释放),输出每行带 `ref=`、末尾带批次 id;`engram_assess` 只能引用同一批次的 ref,且 `sufficient` 由代码强制——声称充足、至少一条有效证据、`nextStrategy=answer` 三者齐备才算充足,否则判为不足并把策略改回继续检索;不属于该批次的 ref
|
|
191
|
+
v0.7.5 起是两处底层修正加一层新能力。① 画像注入的 token 估算改为 CJK 感知(中文按 1.5 token/字、其余按 4 字符/token):此前按长度除以 4 会把中文低估四倍以上,中文用户的实际注入长期超出 `injectTokenBudget` 约 17%–50%;末尾 `+N more` 计数行也纳入预算,注入总量不再超承诺。② 新增**证据门** `engram_assess`(工具 17 个):检索命中只说明「相关」,不说明「足以回答」;`engram_search` 每次登记一个进程内证据批次(每会话保留最近 20 个,会话结束即释放),输出每行带 `ref=`、末尾带批次 id;`engram_assess` 只能引用同一批次的 ref,且 `sufficient` 由代码强制——声称充足、至少一条有效证据、`nextStrategy=answer` 三者齐备才算充足,否则判为不足并把策略改回继续检索;不属于该批次的 ref 会被拒绝并列出,判定写入审计日志(日志「检索」类别可见)。③ 日志补齐 op 词典与明细格式化(闭馆整理 / 复习答题 / 排桩 / 批量排桩 / 开新房),不再显示英文原名与原始 JSON。
|
|
176
192
|
|
|
177
193
|
v0.7.6 起把「作用域」从写死改成逐条判、并让项目宫殿跟着工作区走。① **摄取逐条判宫殿**:提炼时同步判定 `scope`——只跟当前项目/仓库有关的(技术选型、项目约定、架构决策)进该项目库,跨项目通用的(个人偏好、习惯、本人经历)进私人库;实时摄取(上一轮 / 会话结束末轮 / 待补做重放)与历史回填同一口径,回填按**每条会话自己的 cwd** 落库。幂等键(`ingest-done` / `ingest-pending`)固定在私人库,与写入落点解耦,所以实时与回填共用一份 (会话, 轮次) 键、跨路径不重复摄取。② **项目宫殿随工作区切换**:新增 `GET /api/engram/workspaces`(来源 = 宿主工作区注册表 → 会话 header 里出现过的 cwd → 插件进程目录兜底,按分库名去重,同仓库多 worktree 共用一个宫殿),所有项目 scope 的接口接受 `?project=<dbName>` 选择器,未知选择器回 404;`engram_save/search` 等工具的 project 读写改按当前会话 cwd 归属。面板的「项目」作用域在作用域三宫格下方新增一行:**当前工作区 chip + 工作区下拉**(默认「跟随当前工作区」,可临时固定到某个工作区),切工作区即整体切换。③ 无 git 时的项目分库名从「cwd 前 12 字符编码」(同前缀目录会撞库)改为 **cwd 全量 sha256**,旧命名库启动时自动 rename 迁移;插件卸载时关闭所有分库连接(Windows 上不再锁住 .db)。
|
|
178
194
|
|
|
179
|
-
v0.7.9 起进入记忆分层阶段。① **画像可编辑**:新增 `engram_profile_edit` 工具(工具 19 个),curated 画像是你显式维护的最高优先记忆——`view` 看当前内容与版本历史、`edit` 写入新版本(乐观锁冲突 loud 失败)、`rollback` 回滚到历史版本(作为新版本追加,历史永不改写);注入时 curated 全文置于派生画像之前并优先占用 token 预算,编辑记录入操作日志,面板今日页画像卡可查版本 diff。② **episode 情景独立时间线**:新增 `engram_episode_timeline` 工具,按日期范围与来源会话浏览经历,`around` 传锚点 id 切换为时间邻近扩展(锚点时刻 ± 窗口内的情景),回答「当时前后还发生了什么」。③
|
|
195
|
+
v0.7.9 起进入记忆分层阶段。① **画像可编辑**:新增 `engram_profile_edit` 工具(工具 19 个),curated 画像是你显式维护的最高优先记忆——`view` 看当前内容与版本历史、`edit` 写入新版本(乐观锁冲突 loud 失败)、`rollback` 回滚到历史版本(作为新版本追加,历史永不改写);注入时 curated 全文置于派生画像之前并优先占用 token 预算,编辑记录入操作日志,面板今日页画像卡可查版本 diff。② **episode 情景独立时间线**:新增 `engram_episode_timeline` 工具,按日期范围与来源会话浏览经历,`around` 传锚点 id 切换为时间邻近扩展(锚点时刻 ± 窗口内的情景),回答「当时前后还发生了什么」。③ **面板「往事」视图**:设置页记忆库 tab 新增第六个视图,按会话分组浏览 episode,点条目「邻近」一键展开时间邻近卡。④ **会话摘要组头**:历史回填在每会话摄取完成后、实时会话在 disposed 末轮摄取成功后,用辅助 LLM 生成一句话摘要(复用当前路由,失败静默不阻断)存入 `session_summaries` 表;时间线组头携带摘要(工具输出、API、面板组头三处可见),回看时间线不必逐条展开。
|
|
196
|
+
|
|
197
|
+
v0.7.10 起建实体层(时序事实图谱第一批)。① **实体抽取并入辅助调用**:自动摄取与 `engram_save` 的辅助 LLM 输出带 `entities` 提及(name 必填 ≤80 字,kind 为人物/项目/工具/概念/其他,可带别名),按归一化名(trim + 压缩空白 + 小写)与既有实体的 name/aliases 精确匹配消解——命中复用并刷新「最近提及」(词典列表按此倒序),未命中新建;记忆↔实体多对多关联幂等写入,随来源记忆所在 scope 分库;实体消解故障只静默跳过关联,记忆照常落库。② **面板第七个视图「实体」**:实体词典分页列表(最近提及倒序)带每实体关联 active 记忆数,类别筛选与名称/别名子串搜索,点「查看」展开单实体详情(实体记录 + 最近关联记忆);新增 `GET /api/engram/entities` 与 `GET /api/engram/entity` 回环路由。③ **检索与导出带实体**:`engram_search` 命中条目附关联实体(id + 规范名);`engram_export` JSON 视图带实体词典。schema v9 → v10(entities + node_entities 两表,启动时自动迁移,旧库数据保留)。
|
|
198
|
+
|
|
199
|
+
v0.7.12 起接入 **Jev 系统一裁决**(默认关闭,yml `jev` 子配置)并给面板加第八个视图**「裁决」**:DEFER 模糊带的三路判定与矛盾边确认可交 Jev 判决,Jev 不可用(网络失败/超时/未配密钥)时静默回落纯规则四态,不阻断写入。面板可覆盖 yml 的开关、密钥、端点、模型、超时五项(存 `<dbDir>/jev-config.json`,0600 文件 + 0700 目录双保护),保存即时生效——judge 每次装配即时读覆盖,无需重启;三阈值保持 yml 高级配置、面板只读展示。密钥安全:GET 响应只回掩码(长度 > 8 显尾 4 位),明文不出进程,输入框不回填。合并后开了开关但没有密钥时自动降级为关闭;覆盖文件损坏按空覆盖自愈,面板重新保存即修复。裁决 tab 另配「测试连接」按钮(填完密钥即时验证端点与模型可达性,未保存的输入按当前输入测、不落覆盖文件)与「近期裁决」卡(本进程内最近 50 条裁决:判定结果、Jev 是否参与、耗时与回落原因,重启清空),排查「为什么这条被搁置」不必再猜。
|
|
180
200
|
|
|
181
201
|
## 开发
|
|
182
202
|
|
|
@@ -197,7 +217,7 @@ pnpm bundle
|
|
|
197
217
|
|
|
198
218
|
#### Token effect
|
|
199
219
|
|
|
200
|
-
画像注入为条件性固定成本(受条数上限与 token 预算双重约束);工具 schema 为常驻成本(
|
|
220
|
+
画像注入为条件性固定成本(受条数上限与 token 预算双重约束);工具 schema 为常驻成本(20 个窄参数工具)。
|
|
201
221
|
|
|
202
222
|
#### KV Cache effect
|
|
203
223
|
|
|
@@ -205,7 +225,7 @@ pnpm bundle
|
|
|
205
225
|
|
|
206
226
|
## Known Limitations and Deferred Work
|
|
207
227
|
|
|
208
|
-
-
|
|
228
|
+
- **矛盾判定默认无 LLM** —— 写入时默认仅按向量相似度(≥0.88)报告候选并建边,语义矛盾的确认留给模型/用户裁决与蒸馏;可配 Jev 裁决(面板「裁决」tab 或 yml `jev`)让模糊带与矛盾边走外部判决,Jev 不可用时回落本默认。
|
|
209
229
|
- **嵌入器降级期间的记忆无向量** —— 模型未就绪时写入的记忆不参与语义道;语义上线后跑一次 `pnpm backfill` 补算存量向量(`pnpm build` 的模型缓存就绪后执行,可经 `HF_ENDPOINT` 配镜像)。
|
|
210
230
|
|
|
211
231
|
## 路线图
|
|
@@ -213,8 +233,8 @@ pnpm bundle
|
|
|
213
233
|
宫殿 IA(位置当索引 / 固定路线 / 检索练习)是索引与审计骨架,按三阶段向 AI 原生的记忆分层与时序事实图谱演进;实际发版按 0.0.1 递增,阶段代号仅为规划标签:
|
|
214
234
|
|
|
215
235
|
- **阶段一 · 飞轮闭环补强**(已完成,v0.7.8):写入四态回报(ACCEPT 新增 / MERGE 并入强化 / DROP 低熵 / DEFER 矛盾待裁决,`engram_save` 与摄取逐条返回处置);画像注入改分级递减预算(首条 160 字、逐条 ×0.9、下限 24,可配置);脱敏扩充中文密码 / 身份证 / 手机号三类;证据门收尾提醒(轮次结束存在未判定 assess 批次时注入提醒)。
|
|
216
|
-
- **阶段二 · 记忆分层**:画像可编辑(**已完成,v0.7.9**:`engram_profile_edit` 工具 + 版本链回滚 + 面板 diff 审计,Letta core-memory 路线);episode 情景记忆独立时间线索引(**已完成,v0.7.9**:`engram_episode_timeline` 工具,日期范围、按会话分组、时间邻近扩展,episode
|
|
217
|
-
- **阶段三 ·
|
|
236
|
+
- **阶段二 · 记忆分层**:画像可编辑(**已完成,v0.7.9**:`engram_profile_edit` 工具 + 版本链回滚 + 面板 diff 审计,Letta core-memory 路线);episode 情景记忆独立时间线索引(**已完成,v0.7.9**:`engram_episode_timeline` 工具,日期范围、按会话分组、时间邻近扩展,episode 专用索引;面板「往事」视图 + 摄取期与实时会话摘要组头同批完成);蒸馏自动化(簇规模 + 相似度阈值,suggest/auto 两档);房间容量与门牌评分可配置化(默认与现状一致)。
|
|
237
|
+
- **阶段三 · 时序事实图谱**:实体表与写入期实体抽取消歧(**已完成,v0.7.10**:entities + node_entities 两表,摄取/保存的辅助 LLM 输出带 entities 提及并消解落库,`engram_search` 命中附实体标签,面板「实体」视图 + 实体回环路由);事实级三元组携带 `valid_at` / `invalid_at` 时间窗,冲突事实软失效不删除(**已完成,v0.7.11**:facts 表(实体 + 时间窗 + `replaced_by` + 来源记忆),摄取辅助 LLM 输出带 facts 并按实体归一化匹配落表,`replaces` 声明软失效链,`engram_search` 加 `asOf` 时点回看,`engram_facts` 工具读写事实链,面板实体页事实链详情);supports / refines / related 自动建边;LoCoMo-zh 评测集进 CI 作检索质量回归门(确定性 Recall@k / MRR 指标)。
|
|
218
238
|
|
|
219
239
|
## 致谢
|
|
220
240
|
|