memory-eternal 0.9.0 → 0.9.1

package/README.en.md

@@ -37,6 +37,14 @@ dsh plugin --profile web add memory-eternal
 
 # or update directly in the profile (pnpm workspace). Use pnpm — `npm install` throws EUNSUPPORTEDPROTOCOL
 cd ~/.dsh/profiles/web && pnpm add memory-eternal@latest
+
+# right after a release, @latest can be held back by pnpm's minimum-release-age gate or a cached
+# packument (it silently stays on the old version): pin the version instead — pnpm adds it to the
+# workspace's minimumReleaseAgeExclude automatically
+cd ~/.dsh/profiles/web && pnpm add memory-eternal@0.9.0
+
+# verify the version, then restart dsh web
+node -e "console.log(require('memory-eternal/package.json').version)"
 ```
 
 After **restarting dsh web**, three things are live immediately:
@@ -182,6 +190,8 @@ Auto-capture is a silent background pipeline: when it broke, the only symptom us
 
 ## 🛡️ Audit Center & Recycle Bin
 
+> **Agents cannot approve on your behalf (since v0.9.1).** Whenever audit is active, the plugin injects a red line into the agent context: new cards stay `pending`, and calling the `approve`/`reject` endpoints (or any equivalent bypass) is forbidden. The decision is always yours.
+
 New cards go to the **Audit Center** (`pending`) by default and enter the main vault only after you approve them; rejected ones move to "Rejected", where you can restore or delete to the recycle bin. Cards matching an exemption (audit mode = skip all / exempt agents / exempt kinds) go straight in; recycle-bin cards are recoverable within 30 days, then auto-purged.
 
 <p align="center">
@@ -236,6 +246,7 @@ npm run build   # builds lib/client.js (DSH embed) + web/app.js (standalone web
 
 | Version | Date | Highlights |
 |---|---|---|
+| **v0.9.1** | 2026-09-12 | **Performance & UX batch**: memory page loads lazily per view (opening Settings no longer drags in 250KB of cards + 890KB of graph; first paint needs ~0.5KB); graph generation **3232ms → 43ms** (bigram sets computed once + lossless mathematical-bound pruning + tag inverted index) plus a server-side cache with **fingerprint invalidation** (write/audit/delete rebuilds immediately, no stale reads); site-wide **brotli q5** (graph 877KB → **54KB**, 34% smaller than gzip; cards 246KB → 83KB → 20KB with br) and large responses compressed with both br and gzip keeping the smaller, so no response is ever worse than gzip; `/cards` gained **real pagination** (offset/limit/total + server-side title sort, globally ordered across pages) loading 100 cards first and fetching more on scroll; the "Agent cards" panel in Usage/Today filters by agent server-side (30 rows); **audit red line**: the plugin injects "never call approve/reject on the user's behalf" into the agent context; the watchdog now **reaps redundant instances on fallback ports** (a duplicate web process holding 76MB was reaped in testing) |
 | **v0.9.0** | 2026-09-12 | **Fixed silent auto-capture death (6 days without a single card)**: after a DSH upgrade the session event API moved from `events` to `ownEvents()/snapshotEvents()`; the listener read nothing, returned silently and left no trace — now it tries `ownEvents()` → `snapshotEvents()` → `events` and logs a per-session attach entry. Daily quota now counts **real writes only** (previously every "not worth saving" verdict burned quota, exhausting the 24h budget after 40 attempts); `captureCooldownMs` actually applies; a single capture input is tail-capped at 20k chars. New **capture log** panel and **active failure alerts** (red strip + agent-context injection + turn-stopping stall sweep). Removed the automatic "Daily review" digest (timer / `/todayBrief` route / UI button). |
 | **v0.8.0** | 2026-09-06 | **Full UI i18n**: follows the DSH system language live (zh/en) across cards, graph, audit center, config panel and new-card templates; SQLite EBUSY fix for the Windows test suite |
 | **v0.3.1** | 2026-09-03 | Knowledge graph loads on demand (fetch on open, unregister on close) |

package/README.md

@@ -37,6 +37,13 @@ dsh plugin --profile web add memory-eternal
 
 # 或直接在 profile（pnpm workspace）里更新。注意用 pnpm，npm install 会报 EUNSUPPORTEDPROTOCOL
 cd ~/.dsh/profiles/web && pnpm add memory-eternal@latest
+
+# 刚发新版时 @latest 可能被 pnpm 的最小发布年龄门槛/元数据缓存挡住（静默停在旧版）：
+# 直接指定版本号最快，pnpm 会自动把它加进 workspace 的 minimumReleaseAgeExclude
+cd ~/.dsh/profiles/web && pnpm add memory-eternal@0.9.0
+
+# 验版本（应等于上面装的号），然后重启 dsh web 生效
+node -e "console.log(require('memory-eternal/package.json').version)"
 ```
 
 **重启 dsh web** 后，三样东西立即生效：
@@ -204,6 +211,7 @@ dsh-memory watchdog [--port 7799]    # 看门狗保活 web（独立进程）
 
 - **待审核 / 已驳回** 双页签，按类型 / 日期 / 智能体筛选，全选后一键「批准 / 驳回 / 删除进回收站」。
 - 审核规则在「记忆配置 → 自动审核配置」：`审核模式`（全部要审 / 全部免审）+ `免审智能体` + `免审类型` + `回收保留天数`。
+- **Agent 不能代你审批（v0.9.1 起）**：只要审核生效，插件就往 Agent 的上下文里注入红线——新卡只能停在「待审核」，并明确禁止调用 `approve`/`reject` 接口或任何等价方式绕过；判断权始终在你手里。
 
 ### 🥇 为什么审核系统更靠谱（对比其它记忆产品）
 
@@ -251,6 +259,7 @@ npm run build   # 构建 lib/client.js（DSH 内嵌）+ web/app.js（独立 web
 
 | 版本 | 日期 | 关键改动 |
 |---|---|---|
+| **v0.9.1** | 2026-09-12 | **性能与体验批次**：记忆页按视图**懒加载**（打开「设置」不再顺带拉 250KB 卡片 + 890KB 图谱，首屏只需约 0.5KB）；知识图谱生成 **3232ms → 43ms**（bigram 集合只算一次 + 数学上界剪枝无损跳过 + 标签倒排），并加**指纹精确失效**的服务端缓存（写卡/审核/删卡立即重建，无脏读）；全站 **brotli q5**（图谱 877KB → **54KB**，比 gzip 再小 34%；卡片 246KB → 83KB → br 20KB），大响应体同时压 br + gzip 取更小者，保证任何响应都不比 gzip 差；`/cards` 支持**真分页**（offset/limit/total + 服务端标题排序，跨页全局有序），首屏只拉 100 条、滚到底续拉；「用量/今日」的 Agent 卡清单改为服务端署名过滤（30 条）；**审核红线**：插件在 Agent 上下文注入「禁止调用 approve/reject 代用户审批」；watchdog **回收备用端口的多余实例**（实测清掉一个常驻 76MB 的重复 web 进程） |
 | **v0.9.0** | 2026-09-12 | **修复自动沉淀静默失效（一连 6 天不写卡）**：DSH 升级后会话事件接口由 `events` 改为 `ownEvents()/snapshotEvents()`，监听器读不到事件后静默返回、不留任何痕迹 —— 现改为三级自适应取法（`ownEvents()` → `snapshotEvents()` → `events`）+ 每会话接入留痕；日配额改为**只统计真实写卡**（此前「不值得保存」的判定也照样扣配额，40 次即耗尽 24h 配额）、`captureCooldownMs` 冷却真正生效、单次沉淀输入截尾 20000 字；新增**自动沉淀日志**面板与**异常主动提示**（页面红条 + 注入 Agent 上下文 + 轮次收尾停滞巡检）；移除「每日回顾」自动简报（定时器 / `/todayBrief` 接口 / UI 按钮） |
 | **v0.8.0** | 2026-09-06 | **全套 UI i18n**：跟随 DSH 系统语言实时切换（中/英），覆盖知识卡 / 知识图谱 / 审核中心 / 配置面板 / 新建卡模板；Windows 测试收尾 SQLite 连接 EBUSY 修复 |
 | **v0.3.1** | 2026-09-03 | 知识图谱按需加载（点击时 fetch，关闭时注销） |

package/index.js

@@ -28,7 +28,7 @@ import { defineTool } from '@deepseek-ai/dsh-tools'
 import { ensureVault, search } from './lib/vault.js'
 import { migrateFromMarkdown, setAuditConfig, backupDb } from './lib/db.js'
 import { summarizeTurn, extractLastTurn, sliceNewEvents, sessionEvents, sessionEventApi, createCaptureHealth, resolveRoute, captureCard, captureUpdate, pickNeighbors } from './lib/capture.js'
-import { createApi, json } from './lib/api.js'
+import { createApi, json, encodeBody } from './lib/api.js'
 
 export const name = 'memory-eternal'
 export const inject = ['systemPrompt', 'settings']
@@ -381,6 +381,10 @@ export function apply(ctx, config) {
         '3. 若检索结果为空，就诚实说明当前记忆库没有相关内容，不要编造。',
         '4. 知识卡存储在 SQLite 数据库中（memory-eternal.db），**禁止**用文件工具直接读写 vault 目录下的任何文件。沉淀记忆必须通过 memory_recall 工具或 /memory-eternal/api/write API。',
         '5. 若下方出现「自动沉淀异常」，必须在本次回复的第一句用中文转述该异常并提醒用户处理，不要自行猜测或尝试修复。',
+        // 审核红线（主上 2026-09-12 强制要求）：只在审核真正生效时注入。
+        ...((cfg.auditMode ?? 'all') === 'none' ? [] : [
+          '6. **审核红线（强制）**：写卡一律停在 pending，由用户在「审核中心」审批。**禁止**调用 /memory-eternal/api/audit/approve 或 /audit/reject 代替用户审批，也不得用任何等价方式绕过审核；用户说「记录一下 / 增加记忆」不等于允许免审入库。需要立刻可用时，写卡后明确告知用户「已进审核中心，待批准」。',
+        ]),
       ].join('\n')
       // 异常提示：沉淀管线坏了，靠这一句把消息送到用户面前（不依赖用户去翻页面）。
       const h = health.snapshot()
@@ -600,8 +604,13 @@ export function apply(ctx, config) {
             const webRoot = path.join(PACKAGE_ROOT, 'web')
             if (pathname.endsWith('app.js')) {
               const buf = await readFile(path.join(webRoot, 'app.js'))
-              res.writeHead(200, { 'Content-Type': 'application/javascript; charset=utf-8', 'Cache-Control': 'no-store' })
-              return res.end(buf)
+              // client bundle 270KB，每次打开配置页都要重下 —— 支持 gzip 的客户端走压缩
+              const headers = { 'Content-Type': 'application/javascript; charset=utf-8', 'Cache-Control': 'no-store', 'Vary': 'Accept-Encoding' }
+              const { body: out, encoding } = encodeBody(res, buf)
+              if (encoding) headers['Content-Encoding'] = encoding
+              headers['Content-Length'] = out.length
+              res.writeHead(200, headers)
+              return res.end(out)
             }
             const buf = await readFile(path.join(webRoot, 'index.html'))
             res.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8', 'Cache-Control': 'no-store' })

package/lib/api.js

@@ -11,7 +11,8 @@
 // - getSettings() → 插件设置对象（/budget /compress 读取；可选，缺省 {}）
 // - getCaptureLog() → 自动沉淀运行轨迹（DSH host 注入；独立 web server 无此数据）
 
-import { listCards, readCard, search, searchAll, graph, graphAll, overview, exportCards, deleteCard, writeCard, parseCard, stats, optimizeCandidates, optimizeApply, addFeedback, dailyCounts, mergeCards, ensureVault, auditQueue, recycleList, purgeExpired, setCardStatus, restoreCard } from './vault.js'
+import { gzipSync, brotliCompressSync, constants as zlibConstants } from 'node:zlib'
+import { listCards, countCards, readCard, search, searchAll, graph, graphAll, overview, exportCards, deleteCard, writeCard, parseCard, stats, optimizeCandidates, optimizeApply, addFeedback, dailyCounts, mergeCards, ensureVault, auditQueue, recycleList, purgeExpired, setCardStatus, restoreCard } from './vault.js'
 import { compressExcerpt } from './capture.js'
 import { getSetupStatus, runSetup, mcpAgentAction } from './setup.js'
 import { backupDb } from './db.js'
@@ -44,17 +45,25 @@ export function createApi(deps = {}) {
         const kind = query.get('kind') || ''
         const q = query.get('q') || ''
         const status = (query.get('status') || 'approved').toLowerCase()
-        const limit = Math.min(Number(query.get('limit')) || 200, 500)
+        // 真分页：offset/limit + total（客户端据此判断还有没有下一页）
+        const limit = Math.min(Math.max(Number(query.get('limit')) || 200, 1), 500)
+        const offset = Math.max(Number(query.get('offset')) || 0, 0)
+        const sort = query.get('sort') === 'title' ? 'title' : 'recent'
+        const agent = query.get('agent') || '' // 服务端署名过滤（如 agent=deepseek-harness）
         // all = 全部（含 pending/rejected，排除回收站）；默认/approved = 已审核。
         const want = status === 'all' ? ['approved', 'pending', 'rejected'] : status === 'pending' || status === 'rejected' ? [status] : ['approved']
-        let cards = await listCards(vaultRoot, { status: want })
-        if (kind) cards = cards.filter((c) => c.kind === kind)
         if (q.trim()) {
-          const hits = await search(vaultRoot, q, { limit: 200 })
+          // 检索是相关性排序：先把命中集合取齐，再在内存里分页（保证 total 与顺序一致）
+          const hits = await search(vaultRoot, q, { limit: 500 })
           const hitPaths = new Set(hits.map((h) => h.path))
-          cards = cards.filter((c) => hitPaths.has(c.path))
+          let cards = (await listCards(vaultRoot, { status: want, kind, agent })).filter((c) => hitPaths.has(c.path))
+          const total = cards.length
+          json(res, 200, { ok: true, cards: cards.slice(offset, offset + limit), total, offset, limit })
+          return
         }
-        json(res, 200, { ok: true, cards: cards.slice(0, limit) })
+        const total = await countCards(vaultRoot, { status: want, kind, agent })
+        const cards = await listCards(vaultRoot, { status: want, kind, agent, limit, offset, sort })
+        json(res, 200, { ok: true, cards, total, offset, limit })
         return
       }
       case '/card': {
@@ -317,12 +326,69 @@ export function createApi(deps = {}) {
 
 export function json(res, status, body) {
   const payload = JSON.stringify(body)
-  res.writeHead(status, {
+  const headers = {
     'Content-Type': 'application/json; charset=utf-8',
-    'Content-Length': Buffer.byteLength(payload),
     'Cache-Control': 'no-store',
-  })
-  res.end(payload)
+    'Vary': 'Accept-Encoding',
+  }
+  const { body: out, encoding } = encodeBody(res, Buffer.from(payload, 'utf8'))
+  if (encoding) headers['Content-Encoding'] = encoding
+  headers['Content-Length'] = out.length
+  res.writeHead(status, headers)
+  res.end(out)
+}
+
+/** 请求方的 Accept-Encoding（`res.req` 由 Node 的 http server 挂上；取不到当不支持）。 */
+export function acceptEncoding(res) {
+  try {
+    const req = res && (res.req || res.__meReq)
+    return String((req && req.headers && req.headers['accept-encoding']) || '').toLowerCase()
+  } catch { return '' }
+}
+
+/** 请求方是否接受 gzip。 */
+export function acceptsGzip(res) {
+  return acceptEncoding(res).includes('gzip')
+}
+
+/**
+ * 协商压缩大响应体：
+ * - 支持 brotli 就优先 br（质量 4 = 在线压缩的速度/体积平衡点，比 gzip 再小约 30%）
+ * - 否则退 gzip
+ * - 都不支持、体积太小、或压完反而更大 → 明文
+ */
+export function encodeBody(res, buf) {
+  if (buf.length <= 2048) return { body: buf, encoding: '' }
+  const ae = acceptEncoding(res)
+  const wantBr = /(^|[,\s])br($|[,\s;])/.test(ae)
+  const wantGzip = ae.includes('gzip')
+  // 大响应体同时压 br + gzip 取更小的那个：br q5 在图谱这种高重复 JSON 上比 gzip 小 35%，
+  // 但在高熵的 JS/小 JSON 上偶尔略大 —— 取小者保证永不劣化（成本仅几毫秒）。
+  if (buf.length >= 16 * 1024) {
+    let br = null
+    if (wantBr) {
+      try { br = brotliCompressSync(buf, { params: { [zlibConstants.BROTLI_PARAM_QUALITY]: 5, [zlibConstants.BROTLI_PARAM_SIZE_HINT]: buf.length } }) } catch { br = null }
+    }
+    let gz = null
+    if (wantGzip) { try { gz = gzipSync(buf) } catch { gz = null } }
+    const best = [br && { body: br, encoding: 'br' }, gz && { body: gz, encoding: 'gzip' }]
+      .filter(Boolean)
+      .filter((x) => x.body.length < buf.length)
+      .sort((a, b) => a.body.length - b.body.length)[0]
+    if (best) return best
+    return { body: buf, encoding: '' }
+  }
+  if (wantBr) {
+    try {
+      const out = brotliCompressSync(buf, { params: { [zlibConstants.BROTLI_PARAM_QUALITY]: 5, [zlibConstants.BROTLI_PARAM_SIZE_HINT]: buf.length } })
+      if (out.length < buf.length) return { body: out, encoding: 'br' }
+    } catch { /* 落到 gzip */ }
+  }
+  if (wantGzip) {
+    const out = gzipSync(buf)
+    if (out.length < buf.length) return { body: out, encoding: 'gzip' }
+  }
+  return { body: buf, encoding: '' }
 }
 
 /** 读请求体（POST body），返回解析后的 JSON 对象或空。 */