@hyperspell/openclaw-hyperspell 0.20.0 → 0.21.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -149,6 +149,7 @@ files back to Hyperspell.
149
149
  | `knowledgeGraph.enabled` | boolean | `false` | Memory Network: extract entities (people, projects, organizations, topics) from memories into `memory/` markdown files. See [Memory Network](#memory-network). |
150
150
  | `knowledgeGraph.scanIntervalMinutes` | number | `60` | Extraction cadence the setup wizard bakes into the cron job it creates. The cron job is the runtime source of truth — to change cadence after setup, edit the cron job (and keep this field in sync). |
151
151
  | `knowledgeGraph.batchSize` | number | `20` | Memories per extraction scan batch |
152
+ | `coverageLog` | boolean | `false` | **Opt-in.** Append a local-only JSONL event whenever auto-context finds no relevant memories, so capture gaps can be told apart from ranking misses. Events include prompt text — see [Coverage log](#coverage-log). |
152
153
  | `debug` | boolean | `false` | Enable diagnostic logging. One-line diagnostics (auto-context ranked/cut/injection summaries, orientation and emotional-context injection counts) are emitted at **info** level, so they appear in `gateway.log` at default host log levels — no host `logging.level` change needed. Verbose output (per-request/response dumps, per-candidate score lines) stays at debug level. |
153
154
  | `dreaming.enabled` | boolean | `false` | Allow `memory-core` to sidecar-load so Dreaming can consolidate local session transcripts into `workspace/MEMORY.md`. See [Running alongside Dreaming](#running-alongside-dreaming). |
154
155
 
@@ -271,6 +272,61 @@ composite = relevance
271
272
  Chatter is additionally capped at `chatterQuota` results per injection,
272
273
  regardless of score.
273
274
 
275
+ Provenance is a signal too: an optional `sourceWeights` map multiplies a
276
+ result's **base relevance** (before the kind boosts/penalties are added) by a
277
+ per-source factor, so "a journaled Notion page is more intentional memory than
278
+ a titled Slack aside on the same topic" is expressible. Any source not listed
279
+ — including sources that don't exist yet — is neutral (`1.0`); the shipped
280
+ default is `{}`, a strict no-op. It weights sources already in the result set;
281
+ to *exclude* a source, use the `sources` filter instead (weights of 0 are
282
+ rejected at load with exactly that pointer). Suggested starting points, priors
283
+ to seed your own tuning rather than measured truth: authored systems slightly
284
+ up (`notion` 1.15, drive/box/dropbox 1.1), conversational exhaust slightly
285
+ down (`slack`/`microsoft_teams` 0.85, `trace` 0.8) — magnitudes small enough
286
+ to break ties, never to override a real relevance gap. Typos in weight keys
287
+ are not schema errors (new backend sources must not fail validation); with
288
+ `debug: true` unknown keys are flagged once at startup in `gateway.log`.
289
+
290
+ Selected results are also checked against each other for **near-duplicates**:
291
+ when the same memory exists in several forms (a re-synced doc section, a
292
+ `remember` note quoting it, a curated copy of a hot-buffer row), every copy
293
+ clears the threshold, none is chatter, and pre-dedup they filled multiple
294
+ injection slots with one piece of information. Each candidate's lead text is
295
+ compared against already-selected results by token overlap; above
296
+ `dedupThreshold` (0.8) the copy is skipped and its slot passes to the
297
+ next-ranked *different* memory. Skipped copies never consume the chatter
298
+ quota. Set `dedupThreshold: 0` to disable. Paraphrased duplicates (same fact,
299
+ different words) are beyond a string measure and out of scope.
300
+
301
+ Within each selected memory, the second excerpt earns its place: a result
302
+ renders up to two highlight bullets, and the runner-up rides along only when
303
+ its score is within 0.15 of the top one's — a .95/.40 pair renders one bullet
304
+ (the distant second is usually a weaker paraphrase), a .95/.85 pair renders
305
+ both. The top highlight is always kept, so a selected memory never disappears
306
+ from the injection because of this rule.
307
+
308
+ An optional **elbow cutoff** (`ranking.elbow`, **off by default**) stops
309
+ injecting at a natural score cliff instead of always filling `maxResults`: a
310
+ narrow query backed by 3 genuinely relevant memories no longer pads the
311
+ context with a plateau of marginal ones that happen to clear the threshold.
312
+ It is strictly conservative — it only ever cuts *earlier*, never later, never
313
+ below `minResults` (default 3), and when no clear cliff exists (gradual
314
+ decline, flat plateau) selection is identical to today. A cliff means the
315
+ drop from the last accepted result is both large against the decline seen so
316
+ far (`gapRatio` × mean gap, default 2.5×) *and* material in absolute terms
317
+ (`minGap`, default 0.05). Before enabling, run
318
+ `node --experimental-strip-types docs/elbow-scan.mjs` against your real data
319
+ to check firing rate and cut depth (with `debug: true` the live verdict shows
320
+ in `gateway.log` as `auto-context: elbow stopped at k (ceiling N)`).
321
+
322
+ Age matters too, gently: results accrue a small **recency penalty** that grows
323
+ with age (exponential decay, 90-day half-life by default) and is hard-capped at
324
+ `recencyMaxPenalty` (0.1) so it can break near-ties toward current information
325
+ but never bury a strong old match. Deliberately-kept memory (curated/story)
326
+ ages at half rate (`recencyCuratedFactor`) — an old truth you chose to keep
327
+ still beats a fresh shallow echo. Results without a timestamp are never
328
+ penalized. Set `recencyHalfLifeDays: 0` to disable.
329
+
274
330
  **`storyBoost` does nothing until you set `storyTerms`.** The default is `[]`,
275
331
  so no result ever classifies as "story". Set it to the distinctive proper nouns
276
332
  of your active work — the title, character names, a project codename, invented
@@ -313,7 +369,7 @@ Tips:
313
369
  The `hyperspell_search` tool and `/getcontext` return raw relevance order —
314
370
  don't test `storyTerms` there and conclude it's broken.
315
371
  - To verify it's working, enable `debug: true` and watch `gateway.log` for the
316
- per-search tally (`auto-context: ranked {...} candidates → selected {...}`) —
372
+ per-search tally (`auto-context: ranked {...} candidates → selected {...}`) and the cut tally (threshold / max-results / elbow / near-duplicate / chatter-quota)
317
373
  it appears at default host log levels. The per-candidate lines
318
374
  (`[story] 0.47→0.82 The Lighthouse Keeper — Chapter 3`), which show story
319
375
  matches even when they lose to the threshold, are debug-level and also need
@@ -329,10 +385,34 @@ Full knobs and defaults:
329
385
  "storyBoost": 0.15, // extra lift for storyTerms matches (stacks with curationBoost)
330
386
  "storyTerms": [], // REQUIRED for storyBoost to do anything
331
387
  "candidateMultiplier": 3, // fetch this × maxResults candidates before re-ranking
332
- "chatterQuota": 2 // hard cap on chatter results per injection
388
+ "chatterQuota": 2, // hard cap on chatter results per injection
389
+ "recencyHalfLifeDays": 90, // age at which half the recency penalty has accrued (0 = off)
390
+ "recencyMaxPenalty": 0.1, // ceiling on the recency penalty — a tiebreaker, never a burial
391
+ "recencyCuratedFactor": 0.5, // kept memory (curated/story) ages at this fraction of the rate
392
+ "sourceWeights": {}, // per-source multiplier on base relevance; unlisted = 1.0
393
+ "dedupThreshold": 0.8, // token-overlap ratio that skips a near-duplicate (0 = off)
394
+ "elbow": { // stop early at a natural score cliff (opt-in)
395
+ "enabled": false,
396
+ "minResults": 3, // never cut below this many accepted results
397
+ "gapRatio": 2.5, // cliff = drop >= this multiple of the mean gap so far...
398
+ "minGap": 0.05 // ...AND at least this big on the raw composite scale
399
+ }
333
400
  }
334
401
  ```
335
402
 
403
+ ### Coverage log
404
+
405
+ No ranking tweak can surface a memory that was never captured — and by default nothing distinguishes "ranking failed" from "it was never stored." With `coverageLog: true`, every auto-context turn that injects **no** memory sections appends one line to `<workspaceDir>/.hyperspell-coverage.jsonl`:
406
+
407
+ - `outcome: "empty"` — the search succeeded but returned zero candidates (a capture question: was this ever stored?)
408
+ - `outcome: "below_threshold"` — candidates existed but none cleared `relevanceThreshold`; `topScore` vs `threshold` says how near the miss was (a ranking question)
409
+
410
+ Failed searches never produce events — backend-unavailable is not "no memories." In multi-user mode there is one event per turn with per-lane detail, and a lane whose search failed is recorded as `status: "error"`, never as zero candidates.
411
+
412
+ **Local-only guarantee:** the log is written only to the workspace directory and is never sent to Hyperspell or anywhere remote. Because each event carries the triggering prompt (truncated to 500 chars), the feature is **off by default** — prompt text reaches disk only if you explicitly opt in. The file is capped at 5 MB with one `.old` rotation generation (~10 MB total), so content ages out instead of accumulating.
413
+
414
+ Review with `jq`, e.g.: `jq -r '[.ts, .outcome, .topScore, .prompt] | @tsv' ~/.openclaw/workspace/.hyperspell-coverage.jsonl` — after a week or two of labeling (capture gap / ranking near-miss / correct absence), the tallies say whether the next investment belongs in capture or ranking. Delete the file when done.
415
+
336
416
  ### `excludeChannels` is forward-only
337
417
 
338
418
  Adding a channel to `excludeChannels` stops all future injection/writes/tools for that conversation, but does **not** remove content that was synced before the channel was quarantined.
@@ -3,7 +3,7 @@ import { gatherOrientation, personalUserId } from "../hooks/startup-orientation.
3
3
  import { isExcludedChannel } from "../lib/exclude-channels.js";
4
4
  import { log } from "../logger.js";
5
5
  /**
6
- * Assemble a read-only report of what before_agent_start WOULD inject at the
6
+ * Assemble a read-only report of what the injection hook WOULD inject at the
7
7
  * start of the next session. Calls only the pure fetch/format functions the
8
8
  * real hooks are composed from — never the hook handlers themselves — so it
9
9
  * cannot touch the inject-once session caches, and it never calls rollMood
@@ -12,7 +12,7 @@ import { log } from "../logger.js";
12
12
  */
13
13
  export async function buildPreviewReport(client, cfg, ctx) {
14
14
  // Quarantined channels get no injected memory of any kind (index.ts guards
15
- // before_agent_start with the same check) — report that instead of a bundle.
15
+ // the injection hook with the same check) — report that instead of a bundle.
16
16
  if (isExcludedChannel({ channelId: ctx.channel }, cfg)) {
17
17
  return "This channel is quarantined (excludeChannels): no context would be injected here and no memory would be written.";
18
18
  }
@@ -293,6 +293,7 @@ async function runSetup() {
293
293
  maxResults: 10,
294
294
  relevanceThreshold: 0.6,
295
295
  ranking: DEFAULT_RANKING,
296
+ coverageLog: false,
296
297
  debug: false,
297
298
  knowledgeGraph: { enabled: false, scanIntervalMinutes: 60, batchSize: 20 },
298
299
  startupOrientation: {
package/dist/config.js CHANGED
@@ -1,7 +1,7 @@
1
1
  import * as fs from "node:fs";
2
2
  import * as os from "node:os";
3
3
  import * as path from "node:path";
4
- import { DEFAULT_RANKING } from "./lib/ranking.js";
4
+ import { DEFAULT_ELBOW, DEFAULT_RANKING, } from "./lib/ranking.js";
5
5
  /**
6
6
  * Convert user-facing scope names (which may contain hyphens) to SDK-safe
7
7
  * metadata values (alphanumeric + underscore only). Must be applied at every
@@ -27,12 +27,13 @@ const ALLOWED_KEYS = [
27
27
  "maxResults",
28
28
  "relevanceThreshold",
29
29
  "ranking",
30
+ "coverageLog",
30
31
  "debug",
31
32
  "knowledgeGraph",
32
33
  "multiUser",
33
34
  "dreaming",
34
35
  ];
35
- const VALID_SOURCES = [
36
+ export const VALID_SOURCES = [
36
37
  "reddit",
37
38
  "notion",
38
39
  "slack",
@@ -95,6 +96,27 @@ function parseWatchPaths(raw) {
95
96
  };
96
97
  });
97
98
  }
99
+ /** Per-source multipliers: non-numeric values silently skip (num()'s
100
+ * convention); explicit non-positive values throw (a 0 weight is a filtering
101
+ * decision wearing a weighting costume — the sources filter is the right
102
+ * tool, and the error says so). Keys lowercase but are NOT validated against
103
+ * VALID_SOURCES: the backend adds sources independently of plugin releases,
104
+ * and an unmatched key is a harmless neutral no-op (startup diag logs it). */
105
+ function parseSourceWeights(raw) {
106
+ if (!raw || typeof raw !== "object" || Array.isArray(raw))
107
+ return {};
108
+ const out = {};
109
+ for (const [k, v] of Object.entries(raw)) {
110
+ if (typeof v !== "number")
111
+ continue;
112
+ if (!Number.isFinite(v) || v <= 0) {
113
+ throw new Error(`ranking.sourceWeights.${k} must be a positive number (got ${v}); ` +
114
+ `to exclude a source entirely, use the "sources" filter instead`);
115
+ }
116
+ out[k.trim().toLowerCase()] = v;
117
+ }
118
+ return out;
119
+ }
98
120
  function parseRanking(raw) {
99
121
  const r = (raw ?? {});
100
122
  const num = (v, d) => (typeof v === "number" ? v : d);
@@ -116,6 +138,24 @@ function parseRanking(raw) {
116
138
  : DEFAULT_RANKING.storyTerms,
117
139
  candidateMultiplier: Math.max(1, num(r.candidateMultiplier, DEFAULT_RANKING.candidateMultiplier)),
118
140
  chatterQuota: Math.max(0, num(r.chatterQuota, DEFAULT_RANKING.chatterQuota)),
141
+ recencyHalfLifeDays: Math.max(0, num(r.recencyHalfLifeDays, DEFAULT_RANKING.recencyHalfLifeDays)),
142
+ recencyMaxPenalty: Math.max(0, num(r.recencyMaxPenalty, DEFAULT_RANKING.recencyMaxPenalty)),
143
+ recencyCuratedFactor: Math.min(1, Math.max(0, num(r.recencyCuratedFactor, DEFAULT_RANKING.recencyCuratedFactor))),
144
+ sourceWeights: parseSourceWeights(r.sourceWeights),
145
+ dedupThreshold: Math.min(1, Math.max(0, num(r.dedupThreshold, DEFAULT_RANKING.dedupThreshold))),
146
+ elbow: parseElbow(r.elbow),
147
+ };
148
+ }
149
+ function parseElbow(raw) {
150
+ const e = (raw ?? {});
151
+ const num = (v, d) => (typeof v === "number" ? v : d);
152
+ return {
153
+ enabled: e.enabled ?? DEFAULT_ELBOW.enabled,
154
+ // >= 2: at least one gap must exist before meanGap is defined (the
155
+ // first accepted result never records a gap).
156
+ minResults: Math.max(2, num(e.minResults, DEFAULT_ELBOW.minResults)),
157
+ gapRatio: Math.max(1, num(e.gapRatio, DEFAULT_ELBOW.gapRatio)),
158
+ minGap: Math.max(0, num(e.minGap, DEFAULT_ELBOW.minGap)),
119
159
  };
120
160
  }
121
161
  function parseSources(raw) {
@@ -376,6 +416,10 @@ export function parseConfig(raw) {
376
416
  maxResults: cfg.maxResults ?? 10,
377
417
  relevanceThreshold: cfg.relevanceThreshold ?? 0.6,
378
418
  ranking: parseRanking(cfg.ranking),
419
+ // Default OFF (deviates from proposal 15 §6 by maintainer decision):
420
+ // coverage events carry prompt text, so nothing is written to disk
421
+ // unless the operator explicitly opts in.
422
+ coverageLog: cfg.coverageLog ?? false,
379
423
  debug: cfg.debug ?? false,
380
424
  knowledgeGraph: {
381
425
  enabled: kgRaw.enabled ?? false,
@@ -2,6 +2,7 @@ import { appendFileSync } from "node:fs";
2
2
  import { buildScopeFilter, getCanReadScopes, resolveUser, } from "../lib/sender.js";
3
3
  import { excludeFilterFor, mergeWithExclude } from "../lib/filters.js";
4
4
  import { explainSelection, kindTally, rerank, } from "../lib/ranking.js";
5
+ import { recordCoverageEvent, } from "../lib/coverage-log.js";
5
6
  import { classifySearchError, logSearchError } from "../lib/search-error.js";
6
7
  import { resolveCurrentSessionId } from "../lib/session.js";
7
8
  import { recordSender, senderIdFromCtx } from "../lib/speaker-tracker.js";
@@ -54,22 +55,38 @@ function formatHighlightBullets(results, maxResults, threshold) {
54
55
  return null;
55
56
  return sections.join("\n\n");
56
57
  }
58
+ // A second highlight rides along only when it is within this much of the top
59
+ // one's score — a distant second is dilution inside a correct pick (proposal
60
+ // 12). Absolute difference (not a ratio) to match every other score rule in
61
+ // this pipeline; 0.15 is "one storyBoost's worth" on the composite scale.
62
+ // Deliberately a constant, not config: a formatting heuristic two levels
63
+ // below anything a user reasons about — promote to ranking.highlightGap only
64
+ // if live tuning proves deployments actually differ.
65
+ const HIGHLIGHT_GAP = 0.15;
57
66
  /**
58
67
  * Format already-SELECTED composite-ranked results (threshold + chatter quota
59
68
  * applied upstream by selectRanked). Highlights are floored at the lower of
60
69
  * (threshold, the result's own base relevance), so we don't hide the very lines
61
- * that define a boosted-but-quiet memory.
70
+ * that define a boosted-but-quiet memory. Exported for direct testing.
62
71
  */
63
- function formatSelected(selected, threshold) {
72
+ export function formatSelected(selected, threshold) {
64
73
  const sections = [];
65
74
  for (const r of selected) {
66
75
  const hiFloor = Math.min(threshold, r._base);
67
- const chosen = [...r.highlights]
76
+ const passing = [...r.highlights]
68
77
  .sort((a, b) => (b.score ?? 0) - (a.score ?? 0))
69
- .filter((h) => (h.score ?? 0) >= hiFloor)
70
- .slice(0, 2);
71
- if (chosen.length === 0)
78
+ .filter((h) => (h.score ?? 0) >= hiFloor);
79
+ if (passing.length === 0)
72
80
  continue;
81
+ // Floor first (is this highlight relevant at all?), gap second (is the
82
+ // runner-up close enough to the winner to be worth its tokens?). The TOP
83
+ // highlight is always kept — the lowered hiFloor exists to protect
84
+ // boosted-but-quiet memories, and the gap rule must never re-hide them; a
85
+ // selected result can never format to fewer sections than today.
86
+ const [top, second] = passing;
87
+ const chosen = second && (top.score ?? 0) - (second.score ?? 0) <= HIGHLIGHT_GAP
88
+ ? [top, second]
89
+ : [top];
73
90
  const title = r.title ?? `[${r.source}]`;
74
91
  const bullets = chosen
75
92
  .map((h) => `- ${h.text.replace(/\n/g, " ")} [${Math.round((h.score ?? 0) * 100)}%]`)
@@ -168,7 +185,12 @@ export function dropCurrentSession(results, currentSessionId) {
168
185
  }
169
186
  return kept;
170
187
  }
171
- export function buildAutoContextHandler(client, cfg) {
188
+ /** Highest raw relevance among candidates — `topScore 0.54` vs `threshold 0.6`
189
+ * is a ranking near-miss; `0.12` means the vault has nothing close (capture). */
190
+ function topScoreOf(results) {
191
+ return results.length ? Math.max(...results.map((r) => r.score ?? 0)) : null;
192
+ }
193
+ export function buildAutoContextHandler(client, cfg, opts) {
172
194
  return async (event, ctx) => {
173
195
  const prompt = event.prompt;
174
196
  if (!prompt || prompt.length < 5)
@@ -184,7 +206,7 @@ export function buildAutoContextHandler(client, cfg) {
184
206
  // Multi-user path
185
207
  if (cfg.multiUser) {
186
208
  const resolved = resolveUser(ctx, cfg);
187
- return multiUserSearch(client, cfg, prompt, resolved, currentSessionId);
209
+ return multiUserSearch(client, cfg, prompt, resolved, currentSessionId, opts?.stateRoot);
188
210
  }
189
211
  // Single-user path — preserves main's highlights + threshold behavior
190
212
  log.debug(`auto-context: searching for "${prompt.slice(0, 50)}..."`);
@@ -195,13 +217,19 @@ export function buildAutoContextHandler(client, cfg) {
195
217
  const limit = ranking.enabled
196
218
  ? cfg.maxResults * ranking.candidateMultiplier
197
219
  : cfg.maxResults;
198
- const results = dropCurrentSession(await client.search(prompt, { limit, filter: excludeFilterFor(cfg) }), currentSessionId);
220
+ // Kept as two steps so the coverage event below can report the pre-drop
221
+ // fetch count (a session-echo-dominated pool must be visible as such).
222
+ const rawResults = await client.search(prompt, {
223
+ limit,
224
+ filter: excludeFilterFor(cfg),
225
+ });
226
+ const results = dropCurrentSession(rawResults, currentSessionId);
199
227
  let formatted;
200
228
  if (ranking.enabled) {
201
229
  const ranked = rerank(results, ranking);
202
230
  // Threshold + chatter quota applied here, so a high-similarity echo can
203
231
  // inform but never flood (the quota bounds count; the penalty bounds rank).
204
- const explained = explainSelection(ranked, cfg.maxResults, cfg.relevanceThreshold, ranking.chatterQuota);
232
+ const explained = explainSelection(ranked, cfg.maxResults, cfg.relevanceThreshold, ranking.chatterQuota, ranking.dedupThreshold, ranking.elbow);
205
233
  logScoreSamples(prompt, currentSessionId, "single", explained, cfg.relevanceThreshold);
206
234
  const selected = explained.filter((e) => e.selected).map((e) => e.result);
207
235
  // Candidates → selected kind tally, logged UNCONDITIONALLY (unlike the
@@ -217,6 +245,11 @@ export function buildAutoContextHandler(client, cfg) {
217
245
  // nothing is injected (e.g. chatterQuota 0 with only chatter clearing
218
246
  // the threshold). Stable "auto-context: cut" prefix for log greps.
219
247
  // flatMap (not filter) so TS narrows to the cut member of the union.
248
+ // Elbow verdict — the live-validation instrument for proposal 13's
249
+ // rollout: firing rate + cut depth come straight from this line.
250
+ if (ranking.elbow.enabled && explained.some((e) => e.cut === "elbow")) {
251
+ log.diag(`auto-context: elbow stopped at ${selected.length} (ceiling ${cfg.maxResults})`);
252
+ }
220
253
  const cuts = explained.flatMap((e) => (e.selected ? [] : [e]));
221
254
  if (cuts.length > 0) {
222
255
  const cutTally = cuts.reduce((acc, e) => ((acc[e.cut] = (acc[e.cut] ?? 0) + 1), acc), {});
@@ -241,6 +274,23 @@ export function buildAutoContextHandler(client, cfg) {
241
274
  // every-turn banner (which reads as framing and trains search-as-ritual).
242
275
  if (!formatted) {
243
276
  log.debug("auto-context: no relevant memories found");
277
+ // Coverage signal (proposal 15): search SUCCEEDED but injected nothing —
278
+ // durably distinguish "never captured" (empty) from "captured but ranked
279
+ // out" (below_threshold). Thrown searches never reach here (the catch
280
+ // below owns them), so availability blips can't pollute the log.
281
+ if (cfg.coverageLog) {
282
+ recordCoverageEvent({
283
+ outcome: results.length > 0 ? "below_threshold" : "empty",
284
+ prompt,
285
+ fetched: rawResults.length,
286
+ candidates: results.length,
287
+ droppedCurrentSession: rawResults.length - results.length,
288
+ topScore: topScoreOf(results),
289
+ threshold: cfg.relevanceThreshold,
290
+ ranking: ranking.enabled,
291
+ sessionId: currentSessionId,
292
+ }, opts?.stateRoot);
293
+ }
244
294
  return;
245
295
  }
246
296
  return { prependContext: wrapContext(formatted) };
@@ -259,7 +309,7 @@ export function buildAutoContextHandler(client, cfg) {
259
309
  // single-user path only: this path never runs rerank/explainSelection, so
260
310
  // there is no composite score to attribute. If ranking ever lands here, call
261
311
  // logScoreSamples with scope "personal" / "shared" per search.
262
- async function multiUserSearch(client, cfg, prompt, resolved, currentSessionId) {
312
+ async function multiUserSearch(client, cfg, prompt, resolved, currentSessionId, stateRoot) {
263
313
  const multiUser = cfg.multiUser;
264
314
  const isKnownSender = !!resolved?.resolved;
265
315
  const includeShared = multiUser.includeSharedInSearch;
@@ -297,9 +347,18 @@ async function multiUserSearch(client, cfg, prompt, resolved, currentSessionId)
297
347
  let idx = 0;
298
348
  let personalResults = [];
299
349
  let sharedResults = [];
350
+ // Per-lane fulfillment + pre-drop counts feed the coverage event below: a
351
+ // lane that REJECTED must never be recorded as "zero candidates" (#39's
352
+ // unavailable-is-not-empty distinction, kept intact in the log schema).
353
+ let personalOk = false;
354
+ let sharedOk = false;
355
+ let rawPersonalCount = 0;
356
+ let rawSharedCount = 0;
300
357
  if (personalSearch) {
301
358
  const r = settled[idx++];
302
359
  if (r.status === "fulfilled") {
360
+ personalOk = true;
361
+ rawPersonalCount = r.value.length;
303
362
  personalResults = dropCurrentSession(r.value, currentSessionId);
304
363
  }
305
364
  else {
@@ -309,6 +368,8 @@ async function multiUserSearch(client, cfg, prompt, resolved, currentSessionId)
309
368
  if (sharedSearch) {
310
369
  const r = settled[idx++];
311
370
  if (r.status === "fulfilled") {
371
+ sharedOk = true;
372
+ rawSharedCount = r.value.length;
312
373
  sharedResults = dropCurrentSession(r.value, currentSessionId);
313
374
  }
314
375
  else {
@@ -342,6 +403,49 @@ async function multiUserSearch(client, cfg, prompt, resolved, currentSessionId)
342
403
  const haveMemorySections = sections.some((s) => s.startsWith("<personal-context>") || s.startsWith("<shared-context>"));
343
404
  if (!haveMemorySections) {
344
405
  log.debug("auto-context: no relevant memories found");
406
+ // Coverage signal (proposal 15): one event per turn (not per lane), with
407
+ // per-lane detail so "personal lane was down" is never misread as
408
+ // "personal memory is empty". Written only when at least one lane
409
+ // fulfilled — every-lane-failed is an availability event (#39), already
410
+ // logged by logSearchError above, and "unknown" is not "zero".
411
+ if (cfg.coverageLog) {
412
+ const lanes = [];
413
+ if (personalSearch)
414
+ lanes.push(personalOk
415
+ ? {
416
+ lane: "personal",
417
+ status: "ok",
418
+ candidates: personalResults.length,
419
+ topScore: topScoreOf(personalResults),
420
+ }
421
+ : { lane: "personal", status: "error" });
422
+ if (sharedSearch)
423
+ lanes.push(sharedOk
424
+ ? {
425
+ lane: "shared",
426
+ status: "ok",
427
+ candidates: sharedResults.length,
428
+ topScore: topScoreOf(sharedResults),
429
+ }
430
+ : { lane: "shared", status: "error" });
431
+ if (lanes.some((l) => l.status === "ok")) {
432
+ const candidates = personalResults.length + sharedResults.length;
433
+ const fetched = rawPersonalCount + rawSharedCount;
434
+ recordCoverageEvent({
435
+ outcome: candidates > 0 ? "below_threshold" : "empty",
436
+ prompt,
437
+ fetched,
438
+ candidates,
439
+ droppedCurrentSession: fetched - candidates,
440
+ topScore: topScoreOf([...personalResults, ...sharedResults]),
441
+ threshold: cfg.relevanceThreshold,
442
+ ranking: cfg.ranking.enabled,
443
+ sessionId: currentSessionId,
444
+ userId: resolved?.userId,
445
+ lanes,
446
+ }, stateRoot);
447
+ }
448
+ }
345
449
  if (isKnownSender && resolved) {
346
450
  const contextLine = resolved.context ? ` ${resolved.context}` : "";
347
451
  return {
@@ -58,7 +58,7 @@ const sessionMoods = new Map();
58
58
  * hundred tokens of repeated wrapper per turn.
59
59
  *
60
60
  * Lifecycle:
61
- * - first before_agent_start in a session: fetch, inject, mark.
61
+ * - first injection-hook fire in a session: fetch, inject, mark.
62
62
  * - subsequent turns in same session: skip (return undefined).
63
63
  * - after_compaction: clear the mark so the next turn re-injects (the
64
64
  * initial injection may have been compacted out of history).
@@ -185,7 +185,7 @@ export function buildEmotionalContext(states) {
185
185
  * context. On later turns of the same session, return undefined — the
186
186
  * injection from the first turn is already in the conversation history.
187
187
  *
188
- * Runs on `before_agent_start` (which fires every turn).
188
+ * Runs on the session injection hook (fires every turn).
189
189
  */
190
190
  export function buildEmotionalStateFetchHandler(client, cfg, deps = {}) {
191
191
  const now = deps.now ?? Date.now;
@@ -1,3 +1,4 @@
1
+ import * as fs from "node:fs";
1
2
  import * as path from "node:path";
2
3
  import { getWorkspaceDir } from "../config.js";
3
4
  import { log } from "../logger.js";
@@ -128,6 +129,64 @@ export function buildFileSyncHandler(client, cfg) {
128
129
  }
129
130
  };
130
131
  }
132
+ /**
133
+ * Plugin-owned file watcher feeding the file-sync handler.
134
+ *
135
+ * OpenClaw 2026.7.2 removed the host `file_changed` hook with no replacement,
136
+ * so live sync must come from the plugin itself. Used on EVERY host version
137
+ * (old hosts still expose `file_changed`, but registering both would
138
+ * double-sync) — one canonical path. Events are forwarded in the old hook's
139
+ * `{ file_path }` shape; the handler already owns syncability filtering and
140
+ * debounce, so the watcher stays dumb.
141
+ */
142
+ export function buildMemorySyncWatcher(cfg, onFileChanged) {
143
+ const workspaceDir = getWorkspaceDir();
144
+ const roots = [
145
+ path.join(workspaceDir, "memory"),
146
+ ...(cfg.syncMemoriesConfig.watchPaths ?? []).map((wp) => resolveWatchPath(workspaceDir, wp.path)),
147
+ ];
148
+ const watchers = [];
149
+ const start = () => {
150
+ let watched = 0;
151
+ for (const root of roots) {
152
+ let isDirectory;
153
+ try {
154
+ isDirectory = fs.statSync(root).isDirectory();
155
+ }
156
+ catch {
157
+ log.debug(`memory-sync watcher: root missing, skipping: ${root}`);
158
+ continue;
159
+ }
160
+ // Watching a file directly breaks on editor rename-replace cycles;
161
+ // watch the parent and filter to the file's own events instead.
162
+ const dir = isDirectory ? root : path.dirname(root);
163
+ try {
164
+ const watcher = fs.watch(dir, { recursive: isDirectory }, (_type, filename) => {
165
+ if (!filename)
166
+ return;
167
+ const filePath = path.join(dir, filename.toString());
168
+ if (!isDirectory && filePath !== root)
169
+ return;
170
+ onFileChanged({ file_path: filePath });
171
+ });
172
+ watchers.push(watcher);
173
+ watched += 1;
174
+ }
175
+ catch (err) {
176
+ log.error(`memory-sync watcher failed for ${dir}`, err);
177
+ }
178
+ }
179
+ if (watched > 0) {
180
+ log.info(`memory-sync: watching ${watched} root(s) with plugin-owned watchers`);
181
+ }
182
+ };
183
+ const stop = () => {
184
+ for (const watcher of watchers)
185
+ watcher.close();
186
+ watchers.length = 0;
187
+ };
188
+ return { start, stop };
189
+ }
131
190
  /**
132
191
  * Sync all existing memory files on startup.
133
192
  * Uses section-level sync when sectionize is enabled.
@@ -194,7 +194,7 @@ async function fetchRecentConversations(client, limit, userId) {
194
194
  }
195
195
  /**
196
196
  * The fetch+format core of startup orientation, shared by the real
197
- * before_agent_start handler and the read-only /previewcontext command.
197
+ * session-start injection handler and the read-only /previewcontext command.
198
198
  * Pure with respect to session lifecycle: no inject-once cache, no retry
199
199
  * counting — callers own that policy. Keeping one formatter here keeps the
200
200
  * preview byte-identical to real injection.
@@ -207,7 +207,7 @@ export async function gatherOrientation(client, cfg, userId) {
207
207
  // off agents). Fall back to agent_end traces only when there's no hot
208
208
  // buffer but auto-trace is on. Otherwise skip: there's nothing to fetch,
209
209
  // and the trace-source list is expensive (observed ~12s + failing/turn),
210
- // blocking before_agent_start and slowing the reply.
210
+ // blocking the injection hook and slowing the reply.
211
211
  const recentSource = cfg.hotBuffer.enabled
212
212
  ? "hotBuffer"
213
213
  : cfg.autoTrace.enabled
package/dist/index.js CHANGED
@@ -1,11 +1,11 @@
1
1
  import { HyperspellClient } from "./client.js";
2
2
  import { registerCommands } from "./commands/slash.js";
3
3
  import { registerCliCommands } from "./commands/setup.js";
4
- import { parseConfig, hyperspellConfigSchema, getWorkspaceDir } from "./config.js";
4
+ import { parseConfig, hyperspellConfigSchema, getWorkspaceDir, VALID_SOURCES } from "./config.js";
5
5
  import { buildAutoContextHandler } from "./hooks/auto-context.js";
6
6
  import { buildAutoTraceHandler } from "./hooks/auto-trace.js";
7
7
  import { buildEmotionalStateCompactionHandler, buildEmotionalStateFetchHandler, buildEmotionalStateSessionCleanupHandler, buildEmotionalStateStoreHandler, } from "./hooks/emotional-state.js";
8
- import { buildFileSyncHandler, syncMemoriesOnStartup } from "./hooks/memory-sync.js";
8
+ import { buildFileSyncHandler, buildMemorySyncWatcher, syncMemoriesOnStartup, } from "./hooks/memory-sync.js";
9
9
  import { buildHotBufferHandler, buildHotBufferSessionCleanupHandler, } from "./hooks/hot-buffer.js";
10
10
  import { buildStartupOrientationCompactionHandler, buildStartupOrientationHandler, buildStartupOrientationSessionCleanupHandler, } from "./hooks/startup-orientation.js";
11
11
  import { isExcludedChannel } from "./lib/exclude-channels.js";
@@ -79,10 +79,17 @@ export default {
79
79
  }
80
80
  const cfg = parseConfig(api.pluginConfig);
81
81
  initLogger(api.logger, cfg.debug);
82
+ // sourceWeights keys are deliberately not schema-validated (new backend
83
+ // sources must not fail manifest validation), so a typo is a silent
84
+ // neutral no-op — surface it once at startup instead.
85
+ const unknownWeightKeys = Object.keys(cfg.ranking.sourceWeights).filter((k) => !VALID_SOURCES.includes(k));
86
+ if (unknownWeightKeys.length > 0) {
87
+ log.diag(`ranking.sourceWeights keys not in the known source list (typo? they weight nothing): ${unknownWeightKeys.join(", ")}`);
88
+ }
82
89
  const client = new HyperspellClient(cfg);
83
90
  // Channel quarantine (cfg.excludeChannels): excluded conversations get no
84
91
  // memory surface in either direction. Guard the shared choke points here —
85
- // before_agent_start (all injection), agent_end (all writes), and the tool
92
+ // the injection hook (all injection), agent_end (all writes), and the tool
86
93
  // factories — so individual hooks stay quarantine-unaware.
87
94
  const quarantined = (ctx) => {
88
95
  if (!isExcludedChannel(ctx, cfg))
@@ -130,8 +137,19 @@ export default {
130
137
  api.on("after_compaction", buildStartupOrientationCompactionHandler());
131
138
  api.on("session_end", buildStartupOrientationSessionCleanupHandler());
132
139
  }
140
+ // Injection hook selection: OpenClaw 2026.7.2 removed the plugin-facing
141
+ // `before_agent_start`; its replacement `agent_turn_prepare` shipped in the
142
+ // same core commit as `enqueueNextTurnInjection` (openclaw#72287, 2026-04),
143
+ // so probing that api member tells us exactly which hook this host knows.
144
+ // Register exactly ONE of the two: cores between 2026-04 and 2026-07
145
+ // accept both and would double-inject. Both hooks share the same
146
+ // { prependContext } result contract and provide event.prompt.
147
+ const injectionHook = typeof api
148
+ .enqueueNextTurnInjection === "function"
149
+ ? "agent_turn_prepare"
150
+ : "before_agent_start";
133
151
  if (startHandlers.length > 0) {
134
- api.on("before_agent_start", async (event, ctx) => {
152
+ api.on(injectionHook, async (event, ctx) => {
135
153
  // Quarantined channels get no injected memory of any kind.
136
154
  if (quarantined(ctx))
137
155
  return undefined;
@@ -160,10 +178,16 @@ export default {
160
178
  api.on("agent_end", unlessQuarantined(buildHotBufferHandler(client, cfg)));
161
179
  api.on("session_end", buildHotBufferSessionCleanupHandler());
162
180
  }
163
- // Register memory sync hook
181
+ // Memory sync live watcher. Plugin-owned on every host version:
182
+ // 2026.7.2 removed the host `file_changed` hook, and on older hosts
183
+ // registering both paths would double-sync each edit. Started/stopped
184
+ // by the lifecycle service below.
185
+ let memorySyncWatcher;
164
186
  if (cfg.syncMemories) {
165
187
  const fileSyncHandler = buildFileSyncHandler(client, cfg);
166
- api.on("file_changed", fileSyncHandler);
188
+ memorySyncWatcher = buildMemorySyncWatcher(cfg, (event) => {
189
+ void fileSyncHandler(event);
190
+ });
167
191
  api.logger.info(`hyperspell: memory sync enabled (sectionize=${cfg.syncMemoriesConfig.sectionize}, watchPaths=${cfg.syncMemoriesConfig.watchPaths.length})`);
168
192
  }
169
193
  // Register memory network tools
@@ -212,8 +236,10 @@ export default {
212
236
  api.logger.error("hyperspell: background memory sync failed", err);
213
237
  });
214
238
  }
239
+ memorySyncWatcher?.start();
215
240
  },
216
241
  stop: () => {
242
+ memorySyncWatcher?.stop();
217
243
  api.logger.info("hyperspell: stopped");
218
244
  },
219
245
  });