@modusensus/dsh-mneme 0.14.0-beta.1 → 0.14.0-beta.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (543) hide show
  1. package/LICENSE +21 -21
  2. package/README.md +830 -830
  3. package/assets/icon.svg +3 -3
  4. package/bin/cli.mjs +964 -964
  5. package/bin/dsh-mneme-choro.mjs +262 -262
  6. package/bin/dsh-mneme-mcp.mjs +510 -510
  7. package/bin/dsh-mneme-serve.mjs +151 -151
  8. package/cordis.patch.yml +21 -21
  9. package/docs/CONFIGURATION.md +407 -404
  10. package/docs/CONTINUITY.md +139 -139
  11. package/docs/DAEMON.md +57 -57
  12. package/docs/DIRECTORIES.md +114 -114
  13. package/docs/DISTILL.md +397 -371
  14. package/docs/DSH-GO.md +97 -97
  15. package/docs/ENTITIES.md +247 -247
  16. package/docs/FORGET.md +77 -77
  17. package/docs/KB.md +110 -110
  18. package/docs/LOCAL_MODEL.md +222 -222
  19. package/docs/SEMANTIC.md +256 -256
  20. package/docs/SHARE.md +552 -552
  21. package/docs/SLEEP.md +202 -202
  22. package/docs/STORAGE.md +122 -122
  23. package/docs/WORKFLOWS.md +366 -366
  24. package/lib/api-standalone.js +1007 -1007
  25. package/lib/api.js +3600 -3600
  26. package/lib/attachment-text.js +122 -122
  27. package/lib/bootstrap-http.js +105 -105
  28. package/lib/bootstrap.js +215 -215
  29. package/lib/candidates.js +454 -454
  30. package/lib/capabilities.js +819 -819
  31. package/lib/choro-command.js +111 -111
  32. package/lib/client.js +14420 -14322
  33. package/lib/commands.js +95 -95
  34. package/lib/config-keys.js +144 -144
  35. package/lib/config.js +1361 -1344
  36. package/lib/content-hash.js +54 -54
  37. package/lib/continuity.js +258 -258
  38. package/lib/date-hint.js +32 -0
  39. package/lib/diary/auto.js +190 -190
  40. package/lib/diary/compose.js +440 -440
  41. package/lib/diary/dir.js +59 -59
  42. package/lib/diary/export.js +97 -97
  43. package/lib/diary/index.js +26 -26
  44. package/lib/diary/library.js +189 -189
  45. package/lib/diary/qq-inbox.js +142 -142
  46. package/lib/diary/rollover.js +61 -61
  47. package/lib/diary/session-provider.js +122 -122
  48. package/lib/document-index.js +94 -94
  49. package/lib/document.js +322 -322
  50. package/lib/dream/clustering.js +123 -123
  51. package/lib/dream/decisions.js +945 -945
  52. package/lib/dream/narratives.js +44 -44
  53. package/lib/dream/sleep.js +2027 -2027
  54. package/lib/dream/triage.js +139 -139
  55. package/lib/dream.js +1879 -1879
  56. package/lib/dsh-go/host.js +162 -162
  57. package/lib/dsh-go/panel.js +2091 -2091
  58. package/lib/dsh-go/text.js +342 -342
  59. package/lib/dsh-go/views/common.js +68 -68
  60. package/lib/dsh-go/views/entity.js +48 -48
  61. package/lib/dsh-go/views/memory.js +198 -198
  62. package/lib/dsh-go/views/root.js +30 -30
  63. package/lib/dsh-go/views/settings.js +398 -398
  64. package/lib/dsh-go/views/status.js +51 -51
  65. package/lib/embedding.js +342 -342
  66. package/lib/entities/extractor.js +285 -285
  67. package/lib/fence.js +24 -24
  68. package/lib/graph/anchoring.js +184 -184
  69. package/lib/guide.js +74 -74
  70. package/lib/heat.js +144 -144
  71. package/lib/hot-memory.js +119 -119
  72. package/lib/image-describe.js +158 -0
  73. package/lib/index.js +852 -852
  74. package/lib/inject.js +656 -651
  75. package/lib/kb/index.js +309 -309
  76. package/lib/kb/routes.js +219 -219
  77. package/lib/kb/tools.js +148 -148
  78. package/lib/lang.js +607 -595
  79. package/lib/local-embedder.js +358 -358
  80. package/lib/maintenance.js +162 -162
  81. package/lib/memory-source.js +9 -9
  82. package/lib/mirror.js +330 -330
  83. package/lib/organize.js +456 -456
  84. package/lib/peak-hours.js +172 -172
  85. package/lib/pipeline-activities.js +178 -178
  86. package/lib/pipeline-attribution.js +165 -165
  87. package/lib/pipeline-command.js +279 -279
  88. package/lib/pipeline-confirm.js +142 -142
  89. package/lib/pipeline-exec.js +174 -174
  90. package/lib/pipeline-nudge-auto.js +129 -129
  91. package/lib/pipeline-nudge.js +1623 -1623
  92. package/lib/pipeline-observations.js +181 -181
  93. package/lib/pipeline-observe.js +133 -133
  94. package/lib/pipeline-runner.js +361 -361
  95. package/lib/pipeline-runs.js +442 -442
  96. package/lib/pipeline.js +716 -716
  97. package/lib/placement.js +13 -13
  98. package/lib/pre-inject-gate.js +354 -354
  99. package/lib/profile-extract.js +164 -164
  100. package/lib/profile-facts.js +356 -356
  101. package/lib/provenance-http.js +251 -251
  102. package/lib/quality-filter.js +162 -162
  103. package/lib/recall-stats.js +190 -190
  104. package/lib/reclaim-http.js +150 -150
  105. package/lib/reranker.js +269 -269
  106. package/lib/runtime/adopt.js +176 -176
  107. package/lib/runtime/closure.js +172 -172
  108. package/lib/runtime/download.js +247 -247
  109. package/lib/runtime/layout.js +265 -265
  110. package/lib/runtime/loader.js +252 -252
  111. package/lib/runtime/model-fetch.js +361 -361
  112. package/lib/runtime/provision.js +237 -237
  113. package/lib/runtime/tarball.js +130 -130
  114. package/lib/runtime/verify.js +223 -223
  115. package/lib/safe-slice.js +18 -18
  116. package/lib/scope-state.js +277 -277
  117. package/lib/scope.js +187 -187
  118. package/lib/search/adaptive.js +60 -60
  119. package/lib/search/bm25.js +96 -96
  120. package/lib/semantic.js +572 -572
  121. package/lib/sensitive-scan.js +209 -209
  122. package/lib/serve.js +228 -228
  123. package/lib/service.js +3268 -3268
  124. package/lib/session-gate.js +91 -91
  125. package/lib/session-usage.js +350 -350
  126. package/lib/settings.js +827 -817
  127. package/lib/share-http.js +281 -281
  128. package/lib/share-import.js +234 -234
  129. package/lib/share.js +335 -335
  130. package/lib/store.js +5895 -5895
  131. package/lib/summarize.js +1388 -1343
  132. package/lib/tools.js +1370 -1356
  133. package/lib/user-settings.js +230 -230
  134. package/lib/vector-index.js +180 -180
  135. package/lib/version-check.js +122 -122
  136. package/lib/wiki/claims.js +267 -267
  137. package/lib/wiki/cli-hint.js +90 -90
  138. package/lib/wiki/inject.js +164 -164
  139. package/lib/wiki/partition.js +200 -200
  140. package/lib/wiki/plan.js +214 -214
  141. package/lib/wiki/queries.js +640 -640
  142. package/lib/wiki/render.js +261 -261
  143. package/lib/wiki/scan.js +371 -371
  144. package/lib/wiki/t0.js +500 -500
  145. package/lib/wiki/t1-jobs.js +232 -232
  146. package/lib/wiki/t1.js +609 -609
  147. package/lib/write-admission.js +404 -404
  148. package/locale/en.json +6 -6
  149. package/locale/zh.json +6 -6
  150. package/package.json +1 -1
  151. package/runtime-manifest.json +526 -526
  152. package/scripts/benchmark-embed.js +201 -201
  153. package/scripts/benchmark-recall.js +249 -249
  154. package/scripts/benchmark-rerank.js +166 -166
  155. package/scripts/build-runtime-manifest.mjs +224 -224
  156. package/scripts/changelog-prep.mjs +91 -91
  157. package/scripts/check-sync.js +44 -44
  158. package/scripts/e2e-dsh.js +220 -220
  159. package/scripts/mneme-runtime.mjs +277 -277
  160. package/scripts/publish-gate.mjs +197 -197
  161. package/scripts/release-docs.mjs +210 -210
  162. package/scripts/release-notes.mjs +164 -164
  163. package/scripts/stress-dsh.js +255 -255
  164. package/scripts/sync-lib.js +47 -47
  165. package/scripts/sync-test-badge.mjs +69 -69
  166. package/scripts/test-count-sync.mjs +48 -48
  167. package/src/api-standalone.js +1007 -1007
  168. package/src/api.js +3600 -3600
  169. package/src/attachment-text.js +122 -122
  170. package/src/bootstrap-http.js +105 -105
  171. package/src/bootstrap.js +215 -215
  172. package/src/candidates.js +454 -454
  173. package/src/capabilities.js +819 -819
  174. package/src/choro-command.js +111 -111
  175. package/src/commands.js +95 -95
  176. package/src/config-keys.js +144 -144
  177. package/src/config.js +1361 -1344
  178. package/src/content-hash.js +54 -54
  179. package/src/continuity.js +258 -258
  180. package/src/date-hint.js +32 -0
  181. package/src/diary/auto.js +190 -190
  182. package/src/diary/compose.js +440 -440
  183. package/src/diary/dir.js +59 -59
  184. package/src/diary/export.js +97 -97
  185. package/src/diary/index.js +26 -26
  186. package/src/diary/library.js +189 -189
  187. package/src/diary/qq-inbox.js +142 -142
  188. package/src/diary/rollover.js +61 -61
  189. package/src/diary/session-provider.js +122 -122
  190. package/src/document-index.js +94 -94
  191. package/src/document.js +322 -322
  192. package/src/dream/clustering.js +123 -123
  193. package/src/dream/decisions.js +945 -945
  194. package/src/dream/narratives.js +44 -44
  195. package/src/dream/sleep.js +2027 -2027
  196. package/src/dream/triage.js +139 -139
  197. package/src/dream.js +1879 -1879
  198. package/src/dsh-go/host.js +162 -162
  199. package/src/dsh-go/panel.js +2091 -2091
  200. package/src/dsh-go/text.js +342 -342
  201. package/src/dsh-go/views/common.js +68 -68
  202. package/src/dsh-go/views/entity.js +48 -48
  203. package/src/dsh-go/views/memory.js +198 -198
  204. package/src/dsh-go/views/root.js +30 -30
  205. package/src/dsh-go/views/settings.js +398 -398
  206. package/src/dsh-go/views/status.js +51 -51
  207. package/src/embedding.js +342 -342
  208. package/src/entities/extractor.js +285 -285
  209. package/src/fence.js +24 -24
  210. package/src/graph/anchoring.js +184 -184
  211. package/src/guide.js +74 -74
  212. package/src/heat.js +144 -144
  213. package/src/hot-memory.js +119 -119
  214. package/src/image-describe.js +158 -0
  215. package/src/index.js +852 -852
  216. package/src/inject.js +656 -651
  217. package/src/kb/index.js +309 -309
  218. package/src/kb/routes.js +219 -219
  219. package/src/kb/tools.js +148 -148
  220. package/src/lang.js +607 -595
  221. package/src/local-embedder.js +358 -358
  222. package/src/maintenance.js +162 -162
  223. package/src/memory-source.js +9 -9
  224. package/src/mirror.js +330 -330
  225. package/src/organize.js +456 -456
  226. package/src/peak-hours.js +172 -172
  227. package/src/pipeline-activities.js +178 -178
  228. package/src/pipeline-attribution.js +165 -165
  229. package/src/pipeline-command.js +279 -279
  230. package/src/pipeline-confirm.js +142 -142
  231. package/src/pipeline-exec.js +174 -174
  232. package/src/pipeline-nudge-auto.js +129 -129
  233. package/src/pipeline-nudge.js +1623 -1623
  234. package/src/pipeline-observations.js +181 -181
  235. package/src/pipeline-observe.js +133 -133
  236. package/src/pipeline-runner.js +361 -361
  237. package/src/pipeline-runs.js +442 -442
  238. package/src/pipeline.js +716 -716
  239. package/src/placement.js +13 -13
  240. package/src/pre-inject-gate.js +354 -354
  241. package/src/profile-extract.js +164 -164
  242. package/src/profile-facts.js +356 -356
  243. package/src/provenance-http.js +251 -251
  244. package/src/quality-filter.js +162 -162
  245. package/src/recall-stats.js +190 -190
  246. package/src/reclaim-http.js +150 -150
  247. package/src/reranker.js +269 -269
  248. package/src/runtime/adopt.js +176 -176
  249. package/src/runtime/closure.js +172 -172
  250. package/src/runtime/download.js +247 -247
  251. package/src/runtime/layout.js +265 -265
  252. package/src/runtime/loader.js +252 -252
  253. package/src/runtime/model-fetch.js +361 -361
  254. package/src/runtime/provision.js +237 -237
  255. package/src/runtime/tarball.js +130 -130
  256. package/src/runtime/verify.js +223 -223
  257. package/src/safe-slice.js +18 -18
  258. package/src/scope-state.js +277 -277
  259. package/src/scope.js +187 -187
  260. package/src/search/adaptive.js +60 -60
  261. package/src/search/bm25.js +96 -96
  262. package/src/semantic.js +572 -572
  263. package/src/sensitive-scan.js +209 -209
  264. package/src/serve.js +228 -228
  265. package/src/service.js +3268 -3268
  266. package/src/session-gate.js +91 -91
  267. package/src/session-usage.js +350 -350
  268. package/src/settings.js +827 -817
  269. package/src/share-http.js +281 -281
  270. package/src/share-import.js +234 -234
  271. package/src/share.js +335 -335
  272. package/src/store.js +5895 -5895
  273. package/src/summarize.js +1388 -1343
  274. package/src/tools.js +1370 -1356
  275. package/src/user-settings.js +230 -230
  276. package/src/vector-index.js +180 -180
  277. package/src/version-check.js +122 -122
  278. package/src/wiki/claims.js +267 -267
  279. package/src/wiki/cli-hint.js +90 -90
  280. package/src/wiki/inject.js +164 -164
  281. package/src/wiki/partition.js +200 -200
  282. package/src/wiki/plan.js +214 -214
  283. package/src/wiki/queries.js +640 -640
  284. package/src/wiki/render.js +261 -261
  285. package/src/wiki/scan.js +371 -371
  286. package/src/wiki/t0.js +500 -500
  287. package/src/wiki/t1-jobs.js +232 -232
  288. package/src/wiki/t1.js +609 -609
  289. package/src/write-admission.js +404 -404
  290. package/test/adaptive-inject-budget.test.js +85 -85
  291. package/test/api-hardening.test.js +173 -173
  292. package/test/api-nudge-route.test.js +259 -259
  293. package/test/api-routes-gaps.test.js +1092 -1092
  294. package/test/api.test.js +3007 -2998
  295. package/test/attachment-text.test.js +139 -139
  296. package/test/audit.test.js +448 -448
  297. package/test/baseline-persist.test.js +248 -248
  298. package/test/benchmark.test.js +94 -94
  299. package/test/bm25-corpus-cache.test.js +145 -145
  300. package/test/bootstrap.test.js +228 -228
  301. package/test/candidates.test.js +345 -345
  302. package/test/capabilities.test.js +577 -577
  303. package/test/capacity-warning.test.js +262 -262
  304. package/test/changelog-prep.test.js +125 -125
  305. package/test/choro-command.test.js +218 -218
  306. package/test/cli-enums.test.js +37 -37
  307. package/test/cli-flag-value.test.js +86 -86
  308. package/test/cli-receive.test.js +390 -390
  309. package/test/cli-share.test.js +437 -437
  310. package/test/client-share.test.js +225 -225
  311. package/test/client.test.js +4595 -4425
  312. package/test/clustering.test.js +100 -100
  313. package/test/commands.test.js +106 -106
  314. package/test/config-keys-routes.test.js +233 -233
  315. package/test/config.test.js +153 -153
  316. package/test/conflict-freeze.test.js +344 -344
  317. package/test/conflict-ledger-enforce.test.js +452 -452
  318. package/test/conflict-ledger.test.js +242 -242
  319. package/test/conflict-reviewed.test.js +79 -79
  320. package/test/content-hash.test.js +204 -204
  321. package/test/continuity.test.js +452 -452
  322. package/test/contrast.test.js +432 -432
  323. package/test/date-hint.test.js +156 -0
  324. package/test/dedupe-candidate-window.test.js +180 -180
  325. package/test/dependency-overrides.test.js +78 -78
  326. package/test/diary-library.test.js +187 -187
  327. package/test/diary.test.js +576 -576
  328. package/test/differentiate-feedback.test.js +248 -248
  329. package/test/distill-tx-embed-gap.test.js +110 -110
  330. package/test/docs-config-keys.test.js +65 -65
  331. package/test/document-dir.test.js +233 -233
  332. package/test/document.test.js +703 -703
  333. package/test/dream-archive-guardrail.test.js +116 -116
  334. package/test/dream-baseline-clamp.test.js +174 -174
  335. package/test/dream-candidate-hybrid.test.js +281 -281
  336. package/test/dream-coverage-degrade.test.js +86 -86
  337. package/test/dream-decision-evidence.test.js +314 -314
  338. package/test/dream-failure-backoff.test.js +173 -173
  339. package/test/dream-fence.test.js +207 -207
  340. package/test/dream-maxtokens-fallback.test.js +132 -132
  341. package/test/dream-merge-guardrail.test.js +219 -219
  342. package/test/dream-peak-hours.test.js +194 -194
  343. package/test/dream.test.js +1528 -1528
  344. package/test/dsh-go-host.test.js +134 -134
  345. package/test/dsh-go-panel-actions.test.js +854 -854
  346. package/test/dsh-go-panel.test.js +251 -251
  347. package/test/embed-ready-poll.test.js +94 -94
  348. package/test/embed-text-reasons.test.js +112 -112
  349. package/test/embedding-cache-f32.test.js +276 -276
  350. package/test/embedding-ready.test.js +226 -226
  351. package/test/entities.test.js +711 -711
  352. package/test/entity-recall.test.js +179 -179
  353. package/test/epistemic.test.js +325 -325
  354. package/test/failure-query.test.js +147 -147
  355. package/test/failure-retention.test.js +105 -105
  356. package/test/fnew-0112.test.js +311 -311
  357. package/test/fnew-03.test.js +422 -422
  358. package/test/forget-corrections.test.js +70 -70
  359. package/test/graph-anchoring.test.js +184 -184
  360. package/test/graph-api.test.js +333 -333
  361. package/test/graph-boosted-relations.test.js +189 -189
  362. package/test/graph-inject-budget.test.js +103 -103
  363. package/test/graph-multipath.test.js +269 -269
  364. package/test/graph-passive-confirm.test.js +91 -91
  365. package/test/graph-weight-evolution.test.js +104 -104
  366. package/test/heat.test.js +175 -175
  367. package/test/helpers/dream-mock.js +86 -86
  368. package/test/helpers/peer-worker.mjs +25 -25
  369. package/test/helpers/runtime-source.js +60 -60
  370. package/test/helpers/tar-builder.js +80 -80
  371. package/test/helpers/write-admission-samples.js +312 -312
  372. package/test/hot-memory.test.js +259 -259
  373. package/test/image-description.test.js +271 -0
  374. package/test/image-memory.test.js +170 -170
  375. package/test/inject-content-cap.test.js +91 -91
  376. package/test/inject-first-round-bm25.test.js +89 -89
  377. package/test/inject-heat.test.js +77 -77
  378. package/test/inject-parent-gate.test.js +134 -109
  379. package/test/inject-pin-explicit.test.js +300 -300
  380. package/test/inject-pin-vector-crowd.test.js +225 -225
  381. package/test/inject-pools.test.js +195 -195
  382. package/test/inject-recall.test.js +172 -172
  383. package/test/inject-rotation.test.js +161 -161
  384. package/test/inject.test.js +553 -553
  385. package/test/issue-617-step1-query-and-fusion.test.js +361 -361
  386. package/test/kb-api-wiring.test.js +128 -128
  387. package/test/kb-collections.test.js +272 -272
  388. package/test/kb-routes.test.js +379 -379
  389. package/test/lang.test.js +98 -98
  390. package/test/lastrunat-persist.test.js +101 -101
  391. package/test/lib-smoke.test.js +116 -116
  392. package/test/llm-audit.test.js +815 -815
  393. package/test/llm-message-source.test.js +207 -207
  394. package/test/local-embedder.test.js +292 -292
  395. package/test/maintenance.test.js +501 -501
  396. package/test/mcp-stdio.test.js +385 -385
  397. package/test/mint-source.test.js +311 -311
  398. package/test/mirror-dirty.test.js +424 -424
  399. package/test/mirror-edit-digest.test.js +187 -187
  400. package/test/mirror-generation.test.js +499 -499
  401. package/test/mirror-types.test.js +208 -208
  402. package/test/mirror.test.js +249 -249
  403. package/test/narrative-bars.test.js +162 -162
  404. package/test/organize-tool.test.js +232 -232
  405. package/test/organize.test.js +418 -418
  406. package/test/panel-chart-scale.test.js +90 -90
  407. package/test/panel-tokens.test.js +245 -245
  408. package/test/peak-hours-observability.test.js +70 -70
  409. package/test/peer-blockers.test.js +193 -193
  410. package/test/persona-addressing.test.js +83 -83
  411. package/test/persona-api.test.js +172 -172
  412. package/test/persona-by-session.test.js +568 -568
  413. package/test/persona-flag.test.js +36 -36
  414. package/test/persona-inject.test.js +145 -145
  415. package/test/persona-settings.test.js +198 -198
  416. package/test/pipeline-activities.test.js +155 -155
  417. package/test/pipeline-attribution.test.js +175 -175
  418. package/test/pipeline-command.test.js +243 -243
  419. package/test/pipeline-confirm.test.js +148 -148
  420. package/test/pipeline-exec.test.js +172 -172
  421. package/test/pipeline-nudge-auto.test.js +248 -248
  422. package/test/pipeline-nudge.test.js +1278 -1278
  423. package/test/pipeline-observations.test.js +157 -157
  424. package/test/pipeline-observe.test.js +127 -127
  425. package/test/pipeline-r1r2.test.js +278 -278
  426. package/test/pipeline-runner.test.js +468 -468
  427. package/test/pipeline-runs.test.js +767 -767
  428. package/test/pipeline.test.js +636 -636
  429. package/test/placement.test.js +36 -36
  430. package/test/policy-epoch.test.js +259 -259
  431. package/test/pre-inject-gate.test.js +444 -444
  432. package/test/profile-extract.test.js +269 -269
  433. package/test/profile-facts.test.js +492 -492
  434. package/test/prompt-stance.test.js +128 -128
  435. package/test/provenance-http.test.js +367 -367
  436. package/test/publish-gate.test.js +272 -272
  437. package/test/quality-backfill.test.js +290 -290
  438. package/test/quality-filter.test.js +295 -295
  439. package/test/readme-claims.test.js +113 -113
  440. package/test/reasoning-effort.test.js +752 -752
  441. package/test/recall-anti-update.test.js +82 -82
  442. package/test/recall-evals.test.js +253 -253
  443. package/test/recall-layer.test.js +326 -326
  444. package/test/recall-retention.test.js +128 -128
  445. package/test/recall-stats.test.js +274 -274
  446. package/test/receipt-chain.test.js +451 -451
  447. package/test/reclaim-http.test.js +209 -209
  448. package/test/reflection-update-gate.test.js +129 -129
  449. package/test/reflection.test.js +226 -226
  450. package/test/reindex-backfill.test.js +475 -475
  451. package/test/reindex-scheduler.test.js +369 -369
  452. package/test/release-docs.test.js +156 -156
  453. package/test/release-notes.test.js +265 -265
  454. package/test/release-workflow.test.js +93 -93
  455. package/test/reranker-single-logit.test.js +73 -73
  456. package/test/reranker.test.js +197 -197
  457. package/test/resident-status.test.js +83 -83
  458. package/test/route-policy.test.js +113 -113
  459. package/test/runtime-adopt.test.js +206 -206
  460. package/test/runtime-cli.test.js +89 -89
  461. package/test/runtime-closure.test.js +193 -193
  462. package/test/runtime-download.test.js +302 -302
  463. package/test/runtime-layout.test.js +163 -163
  464. package/test/runtime-loader.test.js +255 -255
  465. package/test/runtime-manifest.test.js +159 -159
  466. package/test/runtime-model-fetch.test.js +348 -348
  467. package/test/runtime-provision.test.js +101 -101
  468. package/test/runtime-tarball.test.js +81 -81
  469. package/test/runtime-verify.test.js +319 -319
  470. package/test/scope-candidates.test.js +491 -491
  471. package/test/scope-distill-stamp.test.js +134 -134
  472. package/test/scope-explicit.test.js +452 -452
  473. package/test/scope-inject-soft.test.js +106 -106
  474. package/test/scope-optional-service.test.js +110 -110
  475. package/test/scope-retrieval.test.js +197 -197
  476. package/test/scope-state.test.js +610 -610
  477. package/test/scope-storage.test.js +255 -255
  478. package/test/scope-strict.test.js +367 -367
  479. package/test/search-fusion.test.js +90 -90
  480. package/test/security-doc.test.js +188 -188
  481. package/test/semantic.test.js +124 -124
  482. package/test/sensitive-scan.test.js +251 -251
  483. package/test/serve-bin.test.js +121 -121
  484. package/test/serve.test.js +258 -258
  485. package/test/service-search.test.js +174 -174
  486. package/test/service.test.js +410 -410
  487. package/test/session-gate.test.js +264 -264
  488. package/test/session-usage.test.js +317 -317
  489. package/test/settings.test.js +273 -273
  490. package/test/share-foreign-rows.test.js +144 -144
  491. package/test/share-http.test.js +144 -144
  492. package/test/share-import.test.js +577 -577
  493. package/test/share-imported-guard.test.js +311 -311
  494. package/test/share-imported-identity.test.js +111 -111
  495. package/test/share-pack.test.js +511 -511
  496. package/test/share-receive-http.test.js +756 -756
  497. package/test/share-routes.test.js +337 -337
  498. package/test/share-store.test.js +133 -133
  499. package/test/share-validate.test.js +101 -101
  500. package/test/skip-invalid-restore.test.js +262 -262
  501. package/test/sleep-action-set.test.js +441 -441
  502. package/test/sleep-catchup.test.js +315 -315
  503. package/test/sleep-failure-backoff.test.js +156 -156
  504. package/test/sleep-heat.test.js +174 -174
  505. package/test/sleep-scheduler-rearm.test.js +89 -89
  506. package/test/sleep.test.js +1455 -1455
  507. package/test/standalone-api.test.js +1344 -1344
  508. package/test/store-vector-cache.test.js +98 -98
  509. package/test/store-write-path-columns.test.js +191 -191
  510. package/test/store.test.js +744 -744
  511. package/test/stress.test.js +209 -209
  512. package/test/summarize-occurred-at.test.js +127 -127
  513. package/test/summarize-parse-diagnostics.test.js +75 -75
  514. package/test/summarize-reasoning-effort.test.js +189 -189
  515. package/test/summarize.test.js +2251 -2251
  516. package/test/supersede-state-tracking.test.js +394 -394
  517. package/test/test-count-sync.test.js +71 -71
  518. package/test/tool-state-ignore.test.js +65 -65
  519. package/test/tools-dto-schema-matrix.test.js +398 -398
  520. package/test/tools-memory-get-schema.test.js +65 -65
  521. package/test/tools.test.js +585 -585
  522. package/test/updated-at-semantics.test.js +113 -113
  523. package/test/user-settings-preview.test.js +127 -127
  524. package/test/user-settings-segments.test.js +527 -527
  525. package/test/vector-index.test.js +334 -334
  526. package/test/version-check.test.js +152 -152
  527. package/test/webServer-optional.test.js +82 -82
  528. package/test/wiki-api.test.js +734 -734
  529. package/test/wiki-claims.test.js +194 -194
  530. package/test/wiki-cli-hint.test.js +118 -118
  531. package/test/wiki-inject.test.js +265 -265
  532. package/test/wiki-partition.test.js +112 -112
  533. package/test/wiki-plan.test.js +80 -80
  534. package/test/wiki-provenance.test.js +202 -202
  535. package/test/wiki-queries.test.js +398 -398
  536. package/test/wiki-scan.test.js +208 -208
  537. package/test/wiki-store.test.js +190 -190
  538. package/test/wiki-t0.test.js +456 -456
  539. package/test/wiki-t1-jobs.test.js +125 -125
  540. package/test/wiki-t1.test.js +460 -460
  541. package/test/wiki-tools.test.js +176 -176
  542. package/test/workflow-guards.test.js +155 -155
  543. package/test/write-admission.test.js +668 -668
package/docs/SEMANTIC.md CHANGED
@@ -1,256 +1,256 @@
1
- # dsh-mneme 语义增强架构 — 设计文档
2
-
3
- - **日期**:2026-08-15
4
- - **状态**:✅ 已实现(本地 Embedding / Rerank / 聚类随 v0.2+ 落地;本文档转为历史设计存档,当前能力以 [README](../README.md) 正文为准)
5
- - **范围**:为 dsh-mneme 增加"完全离线的语义记忆引擎",覆盖本地 Embedding、Rerank 精排、autoDream 向量聚类三个阶段
6
- - **相关文档**:[本地模型部署指南](LOCAL_MODEL.md) · [全键配置参考](CONFIGURATION.md)
7
-
8
- ## 1. 背景与目标
9
-
10
- dsh-mneme v0.1 已具备**可选的向量搜索**:通过 `/api/dsh-mneme/vector-config` 配置 OpenAI 兼容 `/embeddings` 端点,命中"字面不同但语义相近"的记忆;未配置或失败时降级为 LIKE 子串搜索。
11
-
12
- v0.2 的目标是把这条**可选、依赖外部 API** 的向量链路升级为**默认可用、完全离线**的语义记忆引擎:
13
-
14
- 1. **本地 Embedding**:ONNX(`Xenova/bge-small-zh-v1.5`)或 Ollama,记忆向量不再依赖外网 API
15
- 2. **Rerank 精排**:向量初筛后交叉编码精排,把 Top-K 准确率再抬一档
16
- 3. **autoDream 语义增强**:对记忆向量做聚类,让自动巩固(去重/合并/冲突裁决)从"主题相近"升级为"语义相近"
17
-
18
- ## 2. 总体架构
19
-
20
- ```
21
- ┌──────────────────────────────────────────────────────────────┐
22
- │ 语义引擎(v0.2,完全离线可选) │
23
- ├──────────────────────────────────────────────────────────────┤
24
- │ ① Embedder 层(三选一,统一接口,可自动降级) │
25
- │ LocalEmbedder(ONNX) │ OllamaEmbedder │ OpenAIEmbedder │
26
- │ init() / embed(texts) / embedSingle() / dimension / hash │
27
- ├──────────────────────────────────────────────────────────────┤
28
- │ ② VectorIndex 层 │
29
- │ SQLite memories.embedding 列(JSON 向量) │
30
- │ + 余弦相似度检索(score 0..1)+ 缺失向量增量补建 reindex │
31
- ├──────────────────────────────────────────────────────────────┤
32
- │ ③ Rerank 层(可选) │
33
- │ LocalReranker(Xenova/bge-reranker-base,交叉编码) │
34
- │ 召回候选 → 精排 → Top-K │
35
- ├──────────────────────────────────────────────────────────────┤
36
- │ ④ autoDream 聚类(可选) │
37
- │ clusterMemories(向量 K-Means) / findPotentialConflicts │
38
- │ 语义分组 + 疑似矛盾检测 → 巩固决策更精准 │
39
- └──────────────────────────────────────────────────────────────┘
40
- ```
41
-
42
- - **① Embedder**:`src/local-embedder.js`,三后端统一接口,`createEmbedderByProvider(provider, opts)` 按名创建
43
- - **② VectorIndex**:复用 v0.1 的 `memories.embedding` 列与 `store.js` 的余弦检索,无新存储层
44
- - **③ Rerank**:`src/reranker.js`(Phase 2),可选开启
45
- - **④ 聚类**:`src/dream/clustering.js`(Phase 3),autoDream 调度器集成
46
-
47
- ### 目录结构(新增/变化部分)
48
-
49
- ```
50
- src/
51
- ├── local-embedder.js # [新增] Local/Ollama/OpenAI 三 Embedder + createEmbedderByProvider
52
- ├── reranker.js # [新增·Phase2] LocalReranker 交叉编码精排
53
- ├── dream/
54
- │ └── clustering.js # [新增·Phase3] clusterMemories / findPotentialConflicts / kMeans
55
- └── embedding.js # [保留] v0.1 OpenAI 兼容客户端(作为降级路径之一复用)
56
- scripts/
57
- ├── benchmark-embed.js # [新增] Embedding 吞吐量基准
58
- └── benchmark-rerank.js # [新增] Rerank 延迟基准
59
- docs/
60
- ├── SEMANTIC.md # 本文档
61
- ├── LOCAL_MODEL.md # 本地模型部署指南
62
- └── CONFIGURATION.md # 全键配置参考
63
- ```
64
-
65
- ## 3. 搜索流水线
66
-
67
- `memory_search` 在 v0.2 的完整链路(对应 `src/tools.js` 的 `mode: auto | keyword | vector | hybrid`):
68
-
69
- ```
70
- query
71
- │
72
- ┌───────────┴───────────┐
73
- ▼ ▼
74
- keyword 召回 vector 召回
75
- (LIKE 子串 title/ (Embedder.embed(query)
76
- content/tags) → 余弦相似度 top-N
77
- → 缺失向量自动补建)
78
- │ │
79
- └───── 混合融合 ─────────┘
80
- (auto: 关键词命中优先 + 向量补足;分数归一化合并)
81
- │
82
- ▼
83
- [Rerank 开启?]
84
- ├─ 是 → LocalReranker.rerank(query, candidates, topK)
85
- │ 交叉编码精排,取 topK
86
- └─ 否 → 按 importance/score 直接截取 topK
87
- │
88
- ▼
89
- memory_search 结果
90
- ```
91
-
92
- - **召回阶段(候选)**:混合召回,宁多勿漏,候选集通常取向量 top-N(默认远超 Top-K,供精排筛选)
93
- - **精排阶段(Rerank)**:对候选逐条打分(query 与候选的交互编码),取 Top-K
94
- - **结果阶段**:返回条目 + `score`(召回分/精排分)+ `source` 时间戳,与 v0.1 返回结构保持一致
95
-
96
- ## 4. 模块接口
97
-
98
- ### 4.1 Embedder 统一接口(`src/local-embedder.js`)
99
-
100
- 三个后端共享同一接口,方法失败**抛错**,由上层编排降级链(与 v0.1 `embedding.js` 的"失败返回 null"不同)。
101
-
102
- | 成员 | 说明 |
103
- |------|------|
104
- | `init()` | 加载模型 / 探测服务,失败抛错 |
105
- | `embed(texts: string[])` | 批量嵌入,返回 `number[][]`(本地模型 mean pooling + L2 归一化) |
106
- | `embedSingle(text)` | 单条嵌入 → `number[]` |
107
- | `dimension` | 向量维度(getter) |
108
- | `modelHash` | 模型指纹(`<model>#<hash>`),用于向量索引一致性校验,模型换名/换维度时提示重建索引 |
109
- | `dispose()` | 释放模型资源 |
110
-
111
- **LocalEmbedder(ONNX)**
112
-
113
- ```js
114
- new LocalEmbedder({
115
- model: "Xenova/bge-small-zh-v1.5", // 默认;中文优化,全离线
116
- dimension: 512, // 默认与模型匹配
117
- device: "cpu", // cpu | wasm | gpu(onnxruntime 后端)
118
- batchSize: 8, // 分批嵌入,控制峰值内存
119
- cacheDir: "", // 自定义模型缓存目录,空则用 HF 默认缓存
120
- useDtype: "q8", // 量化精度 q8/fp32/fp16
121
- logger: null
122
- })
123
- ```
124
-
125
- **OllamaEmbedder**
126
-
127
- ```js
128
- new OllamaEmbedder({
129
- baseUrl: "http://localhost:11434", // 默认
130
- model: "nomic-embed-text", // 默认;维度从首次响应自动推断
131
- logger: null
132
- })
133
- ```
134
-
135
- **OpenAIEmbedder(兼容 v0.1 行为)**
136
-
137
- ```js
138
- new OpenAIEmbedder({
139
- baseUrl: "", // 例如 https://api.openai.com/v1,也支持 SiliconFlow/智谱/Ollama 代理
140
- apiKey: "",
141
- model: "",
142
- timeoutMs: 15000
143
- })
144
- ```
145
-
146
- **工厂**
147
-
148
- ```js
149
- import { createEmbedderByProvider } from "../src/local-embedder.js";
150
- const embedder = createEmbedderByProvider("local", { device: "cpu" }); // local | ollama | openai
151
- await embedder.init();
152
- const vecs = await embedder.embed(["你好", "记忆库"]);
153
- ```
154
-
155
- ### 4.2 Reranker(`src/reranker.js`,Phase 2)
156
-
157
- | 成员 | 说明 |
158
- |------|------|
159
- | `constructor({ model, device, batchSize, cacheDir, logger })` | `model` 默认 `Xenova/bge-reranker-base` |
160
- | `init()` | 加载交叉编码模型 |
161
- | `rerank(query, candidates, topK?)` | 对候选逐条打分,返回按相关性降序的候选(附 `score`),`topK` 缺省返回全量排序 |
162
-
163
- 示例:
164
-
165
- ```js
166
- import { LocalReranker } from "../src/reranker.js";
167
- const reranker = new LocalReranker({ model: "Xenova/bge-reranker-base" });
168
- await reranker.init();
169
- const top = await reranker.rerank("怎么部署本地模型", candidates, 5);
170
- ```
171
-
172
- ### 4.3 autoDream 聚类(`src/dream/clustering.js`,Phase 3,已实现)
173
-
174
- | 成员 | 说明 |
175
- |------|------|
176
- | `cosineSimilarity(a, b)` | 余弦相似度,零向量/长度不齐返回 0,数值稳定 |
177
- | `kMeans(vectors, k, opts)` | K-Means(k-means++ 播种 + 空簇修复),返回每簇索引数组 |
178
- | `clusterMemories(memories, vectors, k)` | 按向量把记忆分组,返回簇(保留原对象引用) |
179
- | `findPotentialConflicts(memories, vectors, threshold = 0.85)` | 同类型、高相似(可能矛盾)的记忆对 |
180
-
181
- autoDream 调度器在触发巩固时:对候选记忆做 `clusterMemories` 分组 → 每组送 LLM 决策(merge/archive/conflict),并用 `findPotentialConflicts` 预先标出疑似矛盾对,减少漏判。
182
-
183
- ## 5. 降级策略
184
-
185
- ```
186
- 首选 回退 1 回退 2 兜底
187
- LocalEmbedder(ONNX) ──失败──► OllamaEmbedder ──失败──► OpenAIEmbedder ──失败/未配置──► 关键词 LIKE
188
- ```
189
-
190
- - Embedder 方法**抛错**,由上层 `createEmbedderByProvider` 编排的降级链逐级捕获切换;全链路失败则回退到 v0.1 的 LIKE 子串关键词搜索(本库无 FTS5,关键词召回即 LIKE 子串扫描)
191
- - **原则**:语义链路任何一环失败都**静默降级、不阻断记忆读写**——写入照常落库,只是当次不补向量
192
- - **降级是每请求动态的**:同一会话内 ONNX 偶发失败,下一次请求自动重试首选路径,不固化到降级档
193
- - **索引一致性**:Embedder 的 `modelHash` 变化(换模型/换维度)时,旧向量与新向量不可混算余弦,需触发「重建索引」(复用 v0.1 的 `reindexMissing` 补建缺失向量)
194
-
195
- ## 6. 模型清单
196
-
197
- | 用途 | 模型 | 后端 | 维度 | 说明 |
198
- |------|------|------|------|------|
199
- | Embedding | `Xenova/bge-small-zh-v1.5` | ONNX(默认) | 512 | 中文优化,全离线,体积小 |
200
- | Embedding | `bge-m3` | Ollama | 1024 | 多语言(中/英/日等),质量高 |
201
- | Embedding | `nomic-embed-text` | Ollama | 768 | 英文为主,通用 |
202
- | Embedding | `text-embedding-3-small` | OpenAI 兼容 | 1536 | v0.1 云端默认,升级后仍可用 |
203
- | Rerank | `Xenova/bge-reranker-base` | ONNX(Phase 2) | — | 交叉编码,首候选精排 |
204
- | Rerank | `bge-reranker-v2-m3` | 云/本地 | — | 更强精排,多语言 |
205
-
206
- > 维度随模型而定:换用不同维度的模型后,已存向量与模型指纹不匹配,需重建索引。
207
-
208
- ## 7. 配置新增项
209
-
210
- 见 [CONFIGURATION.md](CONFIGURATION.md) 的检索与运行时键表。核心:`embedProvider`(`local` / `ollama` / `openai`,默认 `openai` 保持 v0.1 行为)、`localEmbedModel`、`localEmbedDimension`、`localEmbedDevice`、`localEmbedBatchSize`、`ollamaBaseUrl`、`ollamaModel`、`embedModelCacheDir`、`rerankEnabled`、`rerankModel`。
211
-
212
- ## 8. 测试与基准
213
-
214
- - 单元:`src/local-embedder.js` 三后端接口一致性、`clustering.js` 聚类正确性(node:test)
215
- - 降级链:mock 各后端抛错,断言逐级回退到 LIKE
216
- - 基准:`scripts/benchmark-embed.js`(嵌入延迟/吞吐)、`scripts/benchmark-rerank.js`(Rerank 延迟 vs 候选规模)
217
-
218
- ## 9. 反思更新(v0.2.1 `update` 决策 + 失败追踪)
219
-
220
- ### 9.1 `update` 决策类型
221
-
222
- autoDream 整理时,LLM 可输出第 5 种决策 `update`,直接修正单条记忆的过时/错误内容:
223
-
224
- - **约束**:`ids` 只能含一个 id;必须产生实际字段变化;不能更新 `summary`;新建 < 24h 的记忆不可 update
225
- - **频率限制**:每次 autoDream 最多 `reflectionUpdateMaxPerRun`(默认 2)个 update,超限整单拒绝
226
- - **幂等**:字段已与目标一致时跳过(可重放无副作用)
227
- - **审计**:`dream_runs` 记录 update 的 `_before` 快照,可回溯变更前内容
228
- - **向量同步**:update 后删除旧向量、按新内容重嵌入,保持索引一致
229
-
230
- ### 9.2 失败追踪(failure_memories)
231
-
232
- `failure_memories` 表记录记忆纠正/失败事件,为后续反思进化积累数据:
233
-
234
- ```sql
235
- CREATE TABLE failure_memories (
236
- id TEXT PRIMARY KEY,
237
- query TEXT, -- 用户原始查询/意图(可选)
238
- expected TEXT, -- 期望结果
239
- actual TEXT, -- 实际结果
240
- failure_type TEXT, -- outdated / miss / wrong / user_correction
241
- memory_id TEXT, -- 关联的记忆 id
242
- created_at TEXT NOT NULL
243
- );
244
- ```
245
-
246
- - **触发**:用户调用 `memory_update` 且内容变化时,若 `reflectionFailureTracking` 开启则记录 `user_correction` 行(actual=旧值、expected=新值)
247
- - **查询**:`store.listFailures`(过滤/分页)、`store.getFailureStats`(按类型计数)
248
-
249
- ### 9.3 配置项
250
-
251
- ```javascript
252
- reflectionUpdateEnabled: true, // update 决策总开关
253
- reflectionFailureTracking: true, // 失败追踪总开关
254
- reflectionUpdateMaxPerRun: 2, // 每次整理最多 update 数
255
- reflectionUpdateMinAgeHours: 24 // 新建记忆保护期(小时)
256
- ```
1
+ # dsh-mneme 语义增强架构 — 设计文档
2
+
3
+ - **日期**:2026-08-15
4
+ - **状态**:✅ 已实现(本地 Embedding / Rerank / 聚类随 v0.2+ 落地;本文档转为历史设计存档,当前能力以 [README](../README.md) 正文为准)
5
+ - **范围**:为 dsh-mneme 增加"完全离线的语义记忆引擎",覆盖本地 Embedding、Rerank 精排、autoDream 向量聚类三个阶段
6
+ - **相关文档**:[本地模型部署指南](LOCAL_MODEL.md) · [全键配置参考](CONFIGURATION.md)
7
+
8
+ ## 1. 背景与目标
9
+
10
+ dsh-mneme v0.1 已具备**可选的向量搜索**:通过 `/api/dsh-mneme/vector-config` 配置 OpenAI 兼容 `/embeddings` 端点,命中"字面不同但语义相近"的记忆;未配置或失败时降级为 LIKE 子串搜索。
11
+
12
+ v0.2 的目标是把这条**可选、依赖外部 API** 的向量链路升级为**默认可用、完全离线**的语义记忆引擎:
13
+
14
+ 1. **本地 Embedding**:ONNX(`Xenova/bge-small-zh-v1.5`)或 Ollama,记忆向量不再依赖外网 API
15
+ 2. **Rerank 精排**:向量初筛后交叉编码精排,把 Top-K 准确率再抬一档
16
+ 3. **autoDream 语义增强**:对记忆向量做聚类,让自动巩固(去重/合并/冲突裁决)从"主题相近"升级为"语义相近"
17
+
18
+ ## 2. 总体架构
19
+
20
+ ```
21
+ ┌──────────────────────────────────────────────────────────────┐
22
+ │ 语义引擎(v0.2,完全离线可选) │
23
+ ├──────────────────────────────────────────────────────────────┤
24
+ │ ① Embedder 层(三选一,统一接口,可自动降级) │
25
+ │ LocalEmbedder(ONNX) │ OllamaEmbedder │ OpenAIEmbedder │
26
+ │ init() / embed(texts) / embedSingle() / dimension / hash │
27
+ ├──────────────────────────────────────────────────────────────┤
28
+ │ ② VectorIndex 层 │
29
+ │ SQLite memories.embedding 列(JSON 向量) │
30
+ │ + 余弦相似度检索(score 0..1)+ 缺失向量增量补建 reindex │
31
+ ├──────────────────────────────────────────────────────────────┤
32
+ │ ③ Rerank 层(可选) │
33
+ │ LocalReranker(Xenova/bge-reranker-base,交叉编码) │
34
+ │ 召回候选 → 精排 → Top-K │
35
+ ├──────────────────────────────────────────────────────────────┤
36
+ │ ④ autoDream 聚类(可选) │
37
+ │ clusterMemories(向量 K-Means) / findPotentialConflicts │
38
+ │ 语义分组 + 疑似矛盾检测 → 巩固决策更精准 │
39
+ └──────────────────────────────────────────────────────────────┘
40
+ ```
41
+
42
+ - **① Embedder**:`src/local-embedder.js`,三后端统一接口,`createEmbedderByProvider(provider, opts)` 按名创建
43
+ - **② VectorIndex**:复用 v0.1 的 `memories.embedding` 列与 `store.js` 的余弦检索,无新存储层
44
+ - **③ Rerank**:`src/reranker.js`(Phase 2),可选开启
45
+ - **④ 聚类**:`src/dream/clustering.js`(Phase 3),autoDream 调度器集成
46
+
47
+ ### 目录结构(新增/变化部分)
48
+
49
+ ```
50
+ src/
51
+ ├── local-embedder.js # [新增] Local/Ollama/OpenAI 三 Embedder + createEmbedderByProvider
52
+ ├── reranker.js # [新增·Phase2] LocalReranker 交叉编码精排
53
+ ├── dream/
54
+ │ └── clustering.js # [新增·Phase3] clusterMemories / findPotentialConflicts / kMeans
55
+ └── embedding.js # [保留] v0.1 OpenAI 兼容客户端(作为降级路径之一复用)
56
+ scripts/
57
+ ├── benchmark-embed.js # [新增] Embedding 吞吐量基准
58
+ └── benchmark-rerank.js # [新增] Rerank 延迟基准
59
+ docs/
60
+ ├── SEMANTIC.md # 本文档
61
+ ├── LOCAL_MODEL.md # 本地模型部署指南
62
+ └── CONFIGURATION.md # 全键配置参考
63
+ ```
64
+
65
+ ## 3. 搜索流水线
66
+
67
+ `memory_search` 在 v0.2 的完整链路(对应 `src/tools.js` 的 `mode: auto | keyword | vector | hybrid`):
68
+
69
+ ```
70
+ query
71
+ │
72
+ ┌───────────┴───────────┐
73
+ ▼ ▼
74
+ keyword 召回 vector 召回
75
+ (LIKE 子串 title/ (Embedder.embed(query)
76
+ content/tags) → 余弦相似度 top-N
77
+ → 缺失向量自动补建)
78
+ │ │
79
+ └───── 混合融合 ─────────┘
80
+ (auto: 关键词命中优先 + 向量补足;分数归一化合并)
81
+ │
82
+ ▼
83
+ [Rerank 开启?]
84
+ ├─ 是 → LocalReranker.rerank(query, candidates, topK)
85
+ │ 交叉编码精排,取 topK
86
+ └─ 否 → 按 importance/score 直接截取 topK
87
+ │
88
+ ▼
89
+ memory_search 结果
90
+ ```
91
+
92
+ - **召回阶段(候选)**:混合召回,宁多勿漏,候选集通常取向量 top-N(默认远超 Top-K,供精排筛选)
93
+ - **精排阶段(Rerank)**:对候选逐条打分(query 与候选的交互编码),取 Top-K
94
+ - **结果阶段**:返回条目 + `score`(召回分/精排分)+ `source` 时间戳,与 v0.1 返回结构保持一致
95
+
96
+ ## 4. 模块接口
97
+
98
+ ### 4.1 Embedder 统一接口(`src/local-embedder.js`)
99
+
100
+ 三个后端共享同一接口,方法失败**抛错**,由上层编排降级链(与 v0.1 `embedding.js` 的"失败返回 null"不同)。
101
+
102
+ | 成员 | 说明 |
103
+ |------|------|
104
+ | `init()` | 加载模型 / 探测服务,失败抛错 |
105
+ | `embed(texts: string[])` | 批量嵌入,返回 `number[][]`(本地模型 mean pooling + L2 归一化) |
106
+ | `embedSingle(text)` | 单条嵌入 → `number[]` |
107
+ | `dimension` | 向量维度(getter) |
108
+ | `modelHash` | 模型指纹(`<model>#<hash>`),用于向量索引一致性校验,模型换名/换维度时提示重建索引 |
109
+ | `dispose()` | 释放模型资源 |
110
+
111
+ **LocalEmbedder(ONNX)**
112
+
113
+ ```js
114
+ new LocalEmbedder({
115
+ model: "Xenova/bge-small-zh-v1.5", // 默认;中文优化,全离线
116
+ dimension: 512, // 默认与模型匹配
117
+ device: "cpu", // cpu | wasm | gpu(onnxruntime 后端)
118
+ batchSize: 8, // 分批嵌入,控制峰值内存
119
+ cacheDir: "", // 自定义模型缓存目录,空则用 HF 默认缓存
120
+ useDtype: "q8", // 量化精度 q8/fp32/fp16
121
+ logger: null
122
+ })
123
+ ```
124
+
125
+ **OllamaEmbedder**
126
+
127
+ ```js
128
+ new OllamaEmbedder({
129
+ baseUrl: "http://localhost:11434", // 默认
130
+ model: "nomic-embed-text", // 默认;维度从首次响应自动推断
131
+ logger: null
132
+ })
133
+ ```
134
+
135
+ **OpenAIEmbedder(兼容 v0.1 行为)**
136
+
137
+ ```js
138
+ new OpenAIEmbedder({
139
+ baseUrl: "", // 例如 https://api.openai.com/v1,也支持 SiliconFlow/智谱/Ollama 代理
140
+ apiKey: "",
141
+ model: "",
142
+ timeoutMs: 15000
143
+ })
144
+ ```
145
+
146
+ **工厂**
147
+
148
+ ```js
149
+ import { createEmbedderByProvider } from "../src/local-embedder.js";
150
+ const embedder = createEmbedderByProvider("local", { device: "cpu" }); // local | ollama | openai
151
+ await embedder.init();
152
+ const vecs = await embedder.embed(["你好", "记忆库"]);
153
+ ```
154
+
155
+ ### 4.2 Reranker(`src/reranker.js`,Phase 2)
156
+
157
+ | 成员 | 说明 |
158
+ |------|------|
159
+ | `constructor({ model, device, batchSize, cacheDir, logger })` | `model` 默认 `Xenova/bge-reranker-base` |
160
+ | `init()` | 加载交叉编码模型 |
161
+ | `rerank(query, candidates, topK?)` | 对候选逐条打分,返回按相关性降序的候选(附 `score`),`topK` 缺省返回全量排序 |
162
+
163
+ 示例:
164
+
165
+ ```js
166
+ import { LocalReranker } from "../src/reranker.js";
167
+ const reranker = new LocalReranker({ model: "Xenova/bge-reranker-base" });
168
+ await reranker.init();
169
+ const top = await reranker.rerank("怎么部署本地模型", candidates, 5);
170
+ ```
171
+
172
+ ### 4.3 autoDream 聚类(`src/dream/clustering.js`,Phase 3,已实现)
173
+
174
+ | 成员 | 说明 |
175
+ |------|------|
176
+ | `cosineSimilarity(a, b)` | 余弦相似度,零向量/长度不齐返回 0,数值稳定 |
177
+ | `kMeans(vectors, k, opts)` | K-Means(k-means++ 播种 + 空簇修复),返回每簇索引数组 |
178
+ | `clusterMemories(memories, vectors, k)` | 按向量把记忆分组,返回簇(保留原对象引用) |
179
+ | `findPotentialConflicts(memories, vectors, threshold = 0.85)` | 同类型、高相似(可能矛盾)的记忆对 |
180
+
181
+ autoDream 调度器在触发巩固时:对候选记忆做 `clusterMemories` 分组 → 每组送 LLM 决策(merge/archive/conflict),并用 `findPotentialConflicts` 预先标出疑似矛盾对,减少漏判。
182
+
183
+ ## 5. 降级策略
184
+
185
+ ```
186
+ 首选 回退 1 回退 2 兜底
187
+ LocalEmbedder(ONNX) ──失败──► OllamaEmbedder ──失败──► OpenAIEmbedder ──失败/未配置──► 关键词 LIKE
188
+ ```
189
+
190
+ - Embedder 方法**抛错**,由上层 `createEmbedderByProvider` 编排的降级链逐级捕获切换;全链路失败则回退到 v0.1 的 LIKE 子串关键词搜索(本库无 FTS5,关键词召回即 LIKE 子串扫描)
191
+ - **原则**:语义链路任何一环失败都**静默降级、不阻断记忆读写**——写入照常落库,只是当次不补向量
192
+ - **降级是每请求动态的**:同一会话内 ONNX 偶发失败,下一次请求自动重试首选路径,不固化到降级档
193
+ - **索引一致性**:Embedder 的 `modelHash` 变化(换模型/换维度)时,旧向量与新向量不可混算余弦,需触发「重建索引」(复用 v0.1 的 `reindexMissing` 补建缺失向量)
194
+
195
+ ## 6. 模型清单
196
+
197
+ | 用途 | 模型 | 后端 | 维度 | 说明 |
198
+ |------|------|------|------|------|
199
+ | Embedding | `Xenova/bge-small-zh-v1.5` | ONNX(默认) | 512 | 中文优化,全离线,体积小 |
200
+ | Embedding | `bge-m3` | Ollama | 1024 | 多语言(中/英/日等),质量高 |
201
+ | Embedding | `nomic-embed-text` | Ollama | 768 | 英文为主,通用 |
202
+ | Embedding | `text-embedding-3-small` | OpenAI 兼容 | 1536 | v0.1 云端默认,升级后仍可用 |
203
+ | Rerank | `Xenova/bge-reranker-base` | ONNX(Phase 2) | — | 交叉编码,首候选精排 |
204
+ | Rerank | `bge-reranker-v2-m3` | 云/本地 | — | 更强精排,多语言 |
205
+
206
+ > 维度随模型而定:换用不同维度的模型后,已存向量与模型指纹不匹配,需重建索引。
207
+
208
+ ## 7. 配置新增项
209
+
210
+ 见 [CONFIGURATION.md](CONFIGURATION.md) 的检索与运行时键表。核心:`embedProvider`(`local` / `ollama` / `openai`,默认 `openai` 保持 v0.1 行为)、`localEmbedModel`、`localEmbedDimension`、`localEmbedDevice`、`localEmbedBatchSize`、`ollamaBaseUrl`、`ollamaModel`、`embedModelCacheDir`、`rerankEnabled`、`rerankModel`。
211
+
212
+ ## 8. 测试与基准
213
+
214
+ - 单元:`src/local-embedder.js` 三后端接口一致性、`clustering.js` 聚类正确性(node:test)
215
+ - 降级链:mock 各后端抛错,断言逐级回退到 LIKE
216
+ - 基准:`scripts/benchmark-embed.js`(嵌入延迟/吞吐)、`scripts/benchmark-rerank.js`(Rerank 延迟 vs 候选规模)
217
+
218
+ ## 9. 反思更新(v0.2.1 `update` 决策 + 失败追踪)
219
+
220
+ ### 9.1 `update` 决策类型
221
+
222
+ autoDream 整理时,LLM 可输出第 5 种决策 `update`,直接修正单条记忆的过时/错误内容:
223
+
224
+ - **约束**:`ids` 只能含一个 id;必须产生实际字段变化;不能更新 `summary`;新建 < 24h 的记忆不可 update
225
+ - **频率限制**:每次 autoDream 最多 `reflectionUpdateMaxPerRun`(默认 2)个 update,超限整单拒绝
226
+ - **幂等**:字段已与目标一致时跳过(可重放无副作用)
227
+ - **审计**:`dream_runs` 记录 update 的 `_before` 快照,可回溯变更前内容
228
+ - **向量同步**:update 后删除旧向量、按新内容重嵌入,保持索引一致
229
+
230
+ ### 9.2 失败追踪(failure_memories)
231
+
232
+ `failure_memories` 表记录记忆纠正/失败事件,为后续反思进化积累数据:
233
+
234
+ ```sql
235
+ CREATE TABLE failure_memories (
236
+ id TEXT PRIMARY KEY,
237
+ query TEXT, -- 用户原始查询/意图(可选)
238
+ expected TEXT, -- 期望结果
239
+ actual TEXT, -- 实际结果
240
+ failure_type TEXT, -- outdated / miss / wrong / user_correction
241
+ memory_id TEXT, -- 关联的记忆 id
242
+ created_at TEXT NOT NULL
243
+ );
244
+ ```
245
+
246
+ - **触发**:用户调用 `memory_update` 且内容变化时,若 `reflectionFailureTracking` 开启则记录 `user_correction` 行(actual=旧值、expected=新值)
247
+ - **查询**:`store.listFailures`(过滤/分页)、`store.getFailureStats`(按类型计数)
248
+
249
+ ### 9.3 配置项
250
+
251
+ ```javascript
252
+ reflectionUpdateEnabled: true, // update 决策总开关
253
+ reflectionFailureTracking: true, // 失败追踪总开关
254
+ reflectionUpdateMaxPerRun: 2, // 每次整理最多 update 数
255
+ reflectionUpdateMinAgeHours: 24 // 新建记忆保护期(小时)
256
+ ```