@unblocklabs/unblock-memory 0.2.5 → 0.2.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -53,14 +53,9 @@ directories, or globs into named corpora:
53
53
  chatTypes: ["channel", "group"],
54
54
  },
55
55
  {
56
- name: "canon",
56
+ name: "knowledge",
57
57
  kind: "files",
58
- paths: ["knowledge/canon/**/*.md"],
59
- },
60
- {
61
- name: "reflections",
62
- kind: "files",
63
- paths: ["knowledge/reflections/**/*.md"],
58
+ paths: ["knowledge/**/*.md"],
64
59
  },
65
60
  ],
66
61
  // Optional: omit unless the local analysis worker is installed.
@@ -85,7 +80,7 @@ context resident after first use. Set it to `false` to restore QMD's five-minute
85
80
  idle unload behavior.
86
81
 
87
82
  `memory_search` searches every configured corpus by default. Pass
88
- `corpora: ["canon"]` to search selected corpora or `corpora: ["all"]` to
83
+ `corpora: ["knowledge"]` to search selected corpora or `corpora: ["all"]` to
89
84
  request all of them explicitly. Search results include their corpus name and
90
85
  remain readable by passing the returned `qmd://` path to `memory_get`.
91
86
 
@@ -115,18 +110,22 @@ The optional `sessions` corpus reads the current agent's normal OpenClaw SQLite
115
110
  store and indexes its active user/assistant transcript branch. It defaults to
116
111
  channel and group conversations; add `direct` explicitly to include DMs. Run
117
112
  `memory_sync_sessions` to start a refresh, then use `memory_sync_status` to
118
- check its progress or result. Projections are private derived Markdown
119
- under the agent's `unblock-memory/sessions` state directory and can be rebuilt
120
- from OpenClaw at any time. Session results include provider, chat type,
121
- conversation identity, and start time as an ISO 8601 timestamp. They participate
122
- in the same search and clustering index as file memory. This phase does not sync
123
- sessions at startup or on a schedule; refreshes are manual through
113
+ check its progress or result. Projections are private derived Markdown under the
114
+ agent's `unblock-memory/sessions` state directory and can be rebuilt from
115
+ OpenClaw at any time. Their embedded text contains only `# Transcript` and
116
+ timestamped speaker messages; filtering metadata remains in the session
117
+ manifest. The projected file modification time matches the session start time
118
+ for meaningful chronological cluster reads. Session results include provider,
119
+ chat type, conversation identity, and start time as an ISO 8601 timestamp. They
120
+ participate in the same search and clustering index as file memory. This phase
121
+ does not sync sessions at startup or on a schedule; refreshes are manual through
124
122
  `memory_sync_sessions`.
125
123
 
126
124
  Indexes live at `~/.openclaw/agents/<agentId>/unblock-memory/index.sqlite` (or the
127
- equivalent configured OpenClaw state directory). The first lookup builds the
128
- index; Markdown filesystem changes queue a debounced, serialized background
129
- refresh.
125
+ equivalent configured OpenClaw state directory). Durable agent-supplied event
126
+ dates and maintenance proposals live separately in `curation.sqlite`, so a QMD
127
+ index rebuild does not discard them. The first lookup builds the index;
128
+ Markdown filesystem changes queue a debounced, serialized background refresh.
130
129
 
131
130
  ## Memory analysis
132
131
 
@@ -166,17 +165,38 @@ Unblock Memory exposes:
166
165
  - `memory_list_clusters` to cheaply list current clusters and report whether the
167
166
  retained analysis is stale
168
167
  - `memory_recluster` to explicitly rebuild clusters when the list is missing or stale
169
- - `memory_fetch_cluster` to return up to `topK` representative QMD chunks for a
170
- short `clusterId` returned by `memory_list_clusters`
168
+ - `memory_fetch_cluster` to return a sorted, paginated selection of QMD chunks
169
+ for a short `clusterId` returned by `memory_list_clusters`
171
170
 
172
171
  `memory_recluster` optionally accepts UMAP controls (`method`, components,
173
172
  neighbors, and minimum distance), HDBSCAN controls (minimum cluster size,
174
173
  minimum samples, selection method and epsilon, and single-cluster behavior),
175
174
  and a deterministic seed. Omitting them uses the worker's defaults.
176
175
 
177
- Cluster reads return at most 50 members. Member excerpts are capped at 2 KB
178
- each and 12 KB across a response; source aliases are capped at five per member
179
- and 50 across a response.
176
+ `memory_fetch_cluster` accepts `topK` (1–50), a zero-based `offset`, and
177
+ `sort`: `representative` (the default), `score_desc`, `score_asc`, `date_desc`,
178
+ or `date_asc`. Score is cluster membership probability for normal clusters and
179
+ outlier score for noise. Each member reports raw `sourceModifiedAt` separately
180
+ from `eventTime` and `eventTimeBasis`. Session start times and dated memory paths
181
+ resolve programmatically; reviewed annotations resolve otherwise ambiguous
182
+ chunks or whole documents. Date sorting uses resolved event time when available
183
+ and the clearly labeled source modification time only as a fallback. Responses
184
+ include page totals and the next offset when more members remain.
185
+
186
+ A chronological cluster read creates a coalesced maintenance proposal only for
187
+ returned documents whose event time remains ambiguous; it does not scan the
188
+ whole corpus for chores. Persisted exact-duplicate analysis can likewise create
189
+ review proposals for non-session Markdown. `memory_list_maintenance_tasks`
190
+ returns at most ten tasks, while `memory_update_maintenance_task` can resolve,
191
+ defer, or mark one irrelevant and optionally attach a supported event date.
192
+ These tools never edit or delete source Markdown. Duplicate cleanup remains a
193
+ reviewed source change outside the maintenance tool, and generated session
194
+ projections must never be edited directly.
195
+
196
+ Member excerpts are capped at 2 KB each and 12 KB across a response; source
197
+ aliases are capped at five per member and 50 across a response. These budgets
198
+ are shared across the page so every returned member receives a useful excerpt
199
+ and at least one source path, including a full 50-member page.
180
200
 
181
201
  If indexing changes content or vectors, the previous derived analysis is kept
182
202
  and marked stale. Cluster reads include the analysis timestamp, stale timestamp,
@@ -186,33 +206,37 @@ A failed rebuild leaves the stale result intact, while a successful rebuild
186
206
  atomically replaces it. Analysis is never scheduled automatically. If the worker
187
207
  is absent or fails, `memory_search` and `memory_get` continue to work.
188
208
 
189
- ## Curating canon and reflections
209
+ ## Curating knowledge
190
210
 
191
211
  The plugin bundles the `memory-curator` skill for turning useful clusters into
192
212
  durable knowledge. It becomes available when the plugin is enabled. If the
193
213
  agent has an explicit skill allowlist, include `memory-curator`.
194
214
 
195
- Keep curated files outside `memory/**` so each file belongs to only one corpus:
215
+ Keep maintained knowledge outside `memory/**` so each file belongs to only one
216
+ corpus. Use stable topic files updated in place:
196
217
 
197
218
  ```text
198
219
  knowledge/
199
- ├── canon/
200
- │ └── gateway-restarts.md
201
- └── reflections/
202
- └── 2026-08-26.md
220
+ ├── fleet.md
221
+ ├── people/
222
+ └── rico.md
223
+ └── projects/
224
+ └── unblock-memory.md
203
225
  ```
204
226
 
205
- Canon files are stable topic files updated in place. Each one contains only the
206
- current affirmative rule or understanding, its update time, current rationale,
207
- and `qmd://` evidence links. Do not include old procedures, changelogs, or a
208
- `Supersedes` section: semantic chunking may retrieve those passages without the
209
- surrounding warning that they are obsolete.
227
+ Knowledge is the agent's maintained, current understanding of its unique world:
228
+ facts such as fleet membership, local decisions and preferences, assessments,
229
+ and explicit uncertainty that would be expensive to reconstruct from scattered
230
+ history. Each claim should carry its own epistemic qualification so it remains
231
+ honest when semantic chunking retrieves it alone. Remove stale conclusions
232
+ instead of preserving history, changelogs, or `Supersedes` passages in the same
233
+ file; raw memory and sessions retain the evidence history.
210
234
 
211
- Reflection files are daily or timestamped and append-oriented. They hold useful
212
- patterns, hypotheses, contradictions, and uncertainty, but are not
213
- authoritative. Both corpora participate in later search and clustering, so a
214
- later run can reconsider prior reasoning. Repeated derived text is not
215
- independent corroboration; durable canon still needs underlying source evidence.
235
+ Public or vendor-owned facts, generic command syntax, and behavior likely to
236
+ change with third-party releases should normally be looked up from the current
237
+ authoritative source. A local policy or deliberate divergence may belong in
238
+ knowledge, but the local decision—not copied generic documentation—is the
239
+ durable content.
216
240
 
217
241
  For a manual run, ask the agent:
218
242
 
@@ -226,10 +250,12 @@ For recurring curation, use an OpenClaw automation with the same thin message:
226
250
  Use $memory-curator to run the scheduled memory curation cycle.
227
251
  ```
228
252
 
229
- The skill lists current clusters, reclusters only when analysis is missing or
230
- stale, follows representative sources with `memory_get`, and may correctly
231
- write nothing. Its own writes are indexed for the next cycle; it does not
232
- recluster recursively in the same run.
253
+ The skill treats a cluster as an incomplete attention signal. It frames the
254
+ question raised, uses representative, score, and chronological views as useful,
255
+ searches existing knowledge and adjacent corpora, and investigates live systems,
256
+ files, documentation, or the web when those are better evidence. It then updates
257
+ a stable knowledge topic or correctly writes nothing. Its own writes are indexed
258
+ for the next cycle; it does not recluster recursively in the same run.
233
259
 
234
260
  Existing `unblock-qmd` indexes are derived caches and may be left in place;
235
261
  Unblock Memory rebuilds its own index from configured corpora.
@@ -1,5 +1,8 @@
1
1
  import type { QMDStore } from "@unblocklabs/qmd";
2
2
  type AnalysisDatabase = QMDStore["internal"]["db"];
3
+ export type TemporalReadOptions = {
4
+ sessionCollection?: string;
5
+ };
3
6
  export type MemoryReclusterOptions = {
4
7
  space?: {
5
8
  method?: "umap" | "none";
@@ -30,9 +33,15 @@ type MemoryAnalysisMember = {
30
33
  x: number;
31
34
  y: number;
32
35
  representativeRank: number | null;
36
+ sourceModifiedAt: string;
37
+ eventTime: string | null;
38
+ eventTimeBasis: "path" | "frontmatter" | "session" | "agent_verified" | null;
39
+ eventTimeSource: string;
40
+ contentFingerprint: string;
33
41
  text: string;
34
42
  sourcePaths: string[];
35
43
  };
44
+ export type MemoryClusterSort = "representative" | "score_desc" | "score_asc" | "date_desc" | "date_asc";
36
45
  export type MemoryAnalysisSummary = {
37
46
  status: "ok";
38
47
  runId: string;
@@ -72,6 +81,13 @@ export type MemoryClusterDetail = AnalysisReadMetadata & {
72
81
  runId?: string;
73
82
  cluster?: Omit<MemoryClusterSummary, "preview">;
74
83
  members?: MemoryAnalysisMember[];
84
+ page?: {
85
+ offset: number;
86
+ returned: number;
87
+ total: number;
88
+ hasMore: boolean;
89
+ nextOffset?: number;
90
+ };
75
91
  };
76
92
  export declare function ensureMemoryAnalysisSchema(db: AnalysisDatabase): void;
77
93
  export declare function markMemoryAnalysisStale(db: AnalysisDatabase): void;
@@ -85,5 +101,5 @@ export declare function runAnalysisWorker(params: {
85
101
  export declare function latestAnalysisRunId(db: AnalysisDatabase): string | undefined;
86
102
  export declare function readAnalysisSummary(db: AnalysisDatabase): MemoryAnalysisSummary | undefined;
87
103
  export declare function readClusters(db: AnalysisDatabase, requestedLimit?: number): MemoryClusterList;
88
- export declare function readCluster(db: AnalysisDatabase, clusterReferenceId: string, requestedLimit?: number): MemoryClusterDetail;
104
+ export declare function readCluster(db: AnalysisDatabase, clusterReferenceId: string, requestedLimit?: number, requestedOffset?: number, sort?: MemoryClusterSort, temporal?: TemporalReadOptions): MemoryClusterDetail;
89
105
  export {};
@@ -48,6 +48,17 @@ export function ensureMemoryAnalysisSchema(db) {
48
48
  CREATE INDEX IF NOT EXISTS idx_memory_analysis_memberships_cluster
49
49
  ON memory_analysis_memberships(run_id, cluster_id, representative_rank);
50
50
 
51
+ CREATE TABLE IF NOT EXISTS memory_analysis_duplicate_occurrences (
52
+ run_id TEXT NOT NULL,
53
+ content_fingerprint TEXT NOT NULL,
54
+ canonical_hash TEXT NOT NULL,
55
+ canonical_seq INTEGER NOT NULL,
56
+ duplicate_hash TEXT NOT NULL,
57
+ duplicate_seq INTEGER NOT NULL,
58
+ PRIMARY KEY (run_id, duplicate_hash, duplicate_seq),
59
+ FOREIGN KEY (run_id) REFERENCES memory_analysis_runs(id) ON DELETE CASCADE
60
+ );
61
+
51
62
  CREATE VIEW IF NOT EXISTS memory_analysis_available_memberships AS
52
63
  SELECT
53
64
  m.run_id, m.hash, m.seq, m.cluster_id, m.probability, m.outlier_score,
@@ -61,6 +72,18 @@ export function ensureMemoryAnalysisSchema(db) {
61
72
  WHERE d.hash = m.hash AND d.active = 1
62
73
  );
63
74
  `);
75
+ db.exec(`
76
+ CREATE TEMP TABLE IF NOT EXISTS memory_temporal_annotations (
77
+ collection TEXT NOT NULL,
78
+ path TEXT NOT NULL,
79
+ qmd_hash TEXT,
80
+ qmd_seq INTEGER,
81
+ event_time TEXT NOT NULL,
82
+ basis TEXT NOT NULL,
83
+ document_wide INTEGER NOT NULL,
84
+ PRIMARY KEY (collection, path, qmd_hash, qmd_seq, document_wide)
85
+ );
86
+ `);
64
87
  }
65
88
  export function markMemoryAnalysisStale(db) {
66
89
  db.prepare(`
@@ -208,8 +231,8 @@ function byteSlice(text, maxBytes) {
208
231
  return "";
209
232
  return bytes.subarray(0, maxBytes - 3).toString("utf8").replace(/\uFFFD$/u, "") + "…";
210
233
  }
211
- function members(db, runId, clusterId, limit, maxExcerptBytes = MAX_EXCERPT_BYTES, maxTotalBytes = MAX_TOTAL_EXCERPT_BYTES, maxTotalAliases = MAX_TOTAL_ALIASES) {
212
- const noiseOrder = clusterId === -1
234
+ function members(db, runId, clusterId, limit, offset = 0, sort = "representative", maxExcerptBytes = MAX_EXCERPT_BYTES, maxTotalBytes = MAX_TOTAL_EXCERPT_BYTES, maxTotalAliases = MAX_TOTAL_ALIASES, temporal = {}) {
235
+ const representativeOrder = clusterId === -1
213
236
  ? "m.outlier_score DESC, m.hash, m.seq"
214
237
  : `CASE WHEN m.representative_rank IS NULL THEN 1 ELSE 0 END,
215
238
  m.representative_rank,
@@ -217,23 +240,87 @@ function members(db, runId, clusterId, limit, maxExcerptBytes = MAX_EXCERPT_BYTE
217
240
  m.outlier_score,
218
241
  m.hash,
219
242
  m.seq`;
243
+ const score = clusterId === -1 ? "m.outlier_score" : "m.probability";
244
+ const order = {
245
+ representative: representativeOrder,
246
+ score_desc: `${score} DESC, m.hash, m.seq`,
247
+ score_asc: `${score} ASC, m.hash, m.seq`,
248
+ date_desc: "julianday(COALESCE(m.event_time, m.source_modified_at)) DESC, m.hash, m.seq",
249
+ date_asc: "julianday(COALESCE(m.event_time, m.source_modified_at)) ASC, m.hash, m.seq",
250
+ }[sort];
220
251
  const rows = db.prepare(`
221
- SELECT
222
- m.hash, m.seq, m.probability, m.outlier_score, m.x, m.y,
223
- m.representative_rank, m.pos, m.chunk_len, m.doc
224
- FROM memory_analysis_available_memberships m
225
- WHERE m.run_id = ? AND m.cluster_id = ?
226
- ORDER BY ${noiseOrder}
227
- LIMIT ?
228
- `).all(runId, clusterId, limit);
252
+ WITH candidate_times AS (
253
+ SELECT
254
+ m.hash,
255
+ m.seq,
256
+ d.collection,
257
+ d.path,
258
+ d.modified_at AS source_modified_at,
259
+ CASE
260
+ WHEN d.collection = ? THEN d.modified_at
261
+ WHEN d.path GLOB '*[12][0-9][0-9][0-9]-[01][0-9]-[0-3][0-9].md'
262
+ THEN substr(d.path, length(d.path) - 12, 10) || 'T00:00:00.000Z'
263
+ ELSE annotation.event_time
264
+ END AS event_time,
265
+ CASE
266
+ WHEN d.collection = ? THEN 'session'
267
+ WHEN d.path GLOB '*[12][0-9][0-9][0-9]-[01][0-9]-[0-3][0-9].md' THEN 'path'
268
+ ELSE annotation.basis
269
+ END AS event_time_basis,
270
+ CASE
271
+ WHEN d.collection = ? THEN 1
272
+ WHEN d.path GLOB '*[12][0-9][0-9][0-9]-[01][0-9]-[0-3][0-9].md' THEN 2
273
+ WHEN annotation.event_time IS NOT NULL THEN 3
274
+ ELSE 4
275
+ END AS priority
276
+ FROM memory_analysis_available_memberships m
277
+ JOIN documents d ON d.hash = m.hash AND d.active = 1
278
+ LEFT JOIN memory_temporal_annotations annotation
279
+ ON annotation.collection = d.collection
280
+ AND annotation.path = d.path
281
+ AND (annotation.document_wide = 1 OR
282
+ (annotation.qmd_hash = m.hash AND annotation.qmd_seq = m.seq))
283
+ WHERE m.run_id = ? AND m.cluster_id = ?
284
+ ), ranked_times AS (
285
+ SELECT *, ROW_NUMBER() OVER (
286
+ PARTITION BY hash, seq
287
+ ORDER BY priority, julianday(COALESCE(event_time, source_modified_at)) DESC, collection, path
288
+ ) AS rank
289
+ FROM candidate_times
290
+ ), member_rows AS (
291
+ SELECT
292
+ m.hash, m.seq, m.probability, m.outlier_score, m.x, m.y,
293
+ m.representative_rank, m.pos, m.chunk_len, m.doc,
294
+ (
295
+ SELECT d.modified_at
296
+ FROM documents d
297
+ WHERE d.hash = m.hash AND d.active = 1
298
+ ORDER BY julianday(d.modified_at) DESC, d.collection, d.path
299
+ LIMIT 1
300
+ ) AS source_modified_at,
301
+ temporal.event_time,
302
+ temporal.event_time_basis,
303
+ temporal.collection AS event_collection,
304
+ temporal.path AS event_path
305
+ FROM memory_analysis_available_memberships m
306
+ JOIN ranked_times temporal
307
+ ON temporal.hash = m.hash AND temporal.seq = m.seq AND temporal.rank = 1
308
+ WHERE m.run_id = ? AND m.cluster_id = ?
309
+ )
310
+ SELECT * FROM member_rows m
311
+ ORDER BY ${order}
312
+ LIMIT ? OFFSET ?
313
+ `).all(temporal.sessionCollection ?? "", temporal.sessionCollection ?? "", temporal.sessionCollection ?? "", runId, clusterId, runId, clusterId, limit, offset);
229
314
  let remaining = maxTotalBytes;
230
315
  let remainingAliases = maxTotalAliases;
231
- return rows.map((row) => {
232
- const text = remaining <= 0
233
- ? ""
234
- : byteSlice(row.doc.slice(row.pos, row.pos + row.chunk_len), Math.min(maxExcerptBytes, remaining));
316
+ return rows.map((row, index) => {
317
+ const remainingRows = rows.length - index;
318
+ const excerptBudget = Math.min(maxExcerptBytes, Math.floor(remaining / remainingRows));
319
+ const fullText = row.doc.slice(row.pos, row.pos + row.chunk_len);
320
+ const text = byteSlice(fullText, excerptBudget);
235
321
  remaining -= Buffer.byteLength(text);
236
- const aliases = sourcePaths(db, row.hash, Math.min(MAX_ALIASES_PER_MEMBER, remainingAliases));
322
+ const aliasBudget = Math.min(MAX_ALIASES_PER_MEMBER, Math.floor(remainingAliases / remainingRows));
323
+ const aliases = sourcePaths(db, row.hash, aliasBudget);
237
324
  remainingAliases -= aliases.length;
238
325
  return {
239
326
  hash: row.hash,
@@ -243,6 +330,11 @@ function members(db, runId, clusterId, limit, maxExcerptBytes = MAX_EXCERPT_BYTE
243
330
  x: row.x,
244
331
  y: row.y,
245
332
  representativeRank: row.representative_rank,
333
+ sourceModifiedAt: row.source_modified_at,
334
+ eventTime: row.event_time,
335
+ eventTimeBasis: row.event_time_basis,
336
+ eventTimeSource: `qmd://${row.event_collection}/${row.event_path}`,
337
+ contentFingerprint: createHash("sha256").update(fullText).digest("hex"),
246
338
  text,
247
339
  sourcePaths: aliases,
248
340
  };
@@ -276,7 +368,7 @@ function readMetadata(run) {
276
368
  }
277
369
  function toSummary(db, run, row, includePreview, previewBytes = 600, aliasLimit = MAX_TOTAL_ALIASES) {
278
370
  const preview = includePreview
279
- ? members(db, run.id, row.cluster_id, 1, previewBytes, previewBytes, aliasLimit)[0]
371
+ ? members(db, run.id, row.cluster_id, 1, 0, "representative", previewBytes, previewBytes, aliasLimit)[0]
280
372
  : undefined;
281
373
  return {
282
374
  clusterId: clusterReference(run.id, row.cluster_id),
@@ -362,7 +454,7 @@ function resolveClusterId(db, runId, reference) {
362
454
  `).all(runId, runId);
363
455
  return clusterIds.find((row) => clusterReference(runId, row.cluster_id) === reference)?.cluster_id;
364
456
  }
365
- export function readCluster(db, clusterReferenceId, requestedLimit = DEFAULT_MEMBER_LIMIT) {
457
+ export function readCluster(db, clusterReferenceId, requestedLimit = DEFAULT_MEMBER_LIMIT, requestedOffset = 0, sort = "representative", temporal = {}) {
366
458
  const run = latestValidRun(db);
367
459
  if (!run) {
368
460
  return { status: "not_analyzed", ...readMetadata() };
@@ -388,6 +480,11 @@ export function readCluster(db, clusterReferenceId, requestedLimit = DEFAULT_MEM
388
480
  return { status: "not_found", runId: run.id, ...metadata };
389
481
  }
390
482
  const limit = Math.max(1, Math.min(MAX_MEMBER_LIMIT, Math.floor(requestedLimit)));
483
+ const offset = Math.max(0, Math.floor(requestedOffset));
484
+ const total = availableSize(db, run.id, clusterId);
485
+ const pageMembers = members(db, run.id, clusterId, limit, offset, sort, MAX_EXCERPT_BYTES, MAX_TOTAL_EXCERPT_BYTES, MAX_TOTAL_ALIASES, temporal);
486
+ const nextOffset = offset + pageMembers.length;
487
+ const hasMore = nextOffset < total;
391
488
  return {
392
489
  status: "ok",
393
490
  runId: run.id,
@@ -395,9 +492,16 @@ export function readCluster(db, clusterReferenceId, requestedLimit = DEFAULT_MEM
395
492
  cluster: {
396
493
  clusterId: clusterReferenceId,
397
494
  size: row.size,
398
- availableSize: availableSize(db, run.id, clusterId),
495
+ availableSize: total,
399
496
  meanProbability: row.mean_probability,
400
497
  },
401
- members: members(db, run.id, clusterId, limit),
498
+ members: pageMembers,
499
+ page: {
500
+ offset,
501
+ returned: pageMembers.length,
502
+ total,
503
+ hasMore,
504
+ ...(hasMore ? { nextOffset } : {}),
505
+ },
402
506
  };
403
507
  }
@@ -0,0 +1,70 @@
1
+ declare const TEMPORAL_BASES: readonly ["path", "frontmatter", "session", "agent_verified"];
2
+ export type TemporalBasis = typeof TEMPORAL_BASES[number];
3
+ declare const MAINTENANCE_TASK_TYPES: readonly ["ambiguous_event_time", "exact_duplicate"];
4
+ export type MaintenanceTaskType = typeof MAINTENANCE_TASK_TYPES[number];
5
+ declare const MAINTENANCE_STATUSES: readonly ["pending", "resolved", "deferred", "irrelevant"];
6
+ export type MaintenanceStatus = typeof MAINTENANCE_STATUSES[number];
7
+ export type TemporalAnnotation = {
8
+ corpus: string;
9
+ collection: string;
10
+ path: string;
11
+ contentFingerprint: string;
12
+ eventTime: string;
13
+ basis: TemporalBasis;
14
+ evidence: string;
15
+ qmdHash: string | null;
16
+ qmdSeq: number | null;
17
+ createdAt: string;
18
+ updatedAt: string;
19
+ };
20
+ export type MaintenanceTask = {
21
+ id: string;
22
+ type: MaintenanceTaskType;
23
+ corpus: string;
24
+ collection: string;
25
+ path: string;
26
+ reason: string;
27
+ contentFingerprint: string;
28
+ detail: string | null;
29
+ resolutionNote: string | null;
30
+ status: MaintenanceStatus;
31
+ createdAt: string;
32
+ updatedAt: string;
33
+ };
34
+ export declare function chunkFingerprint(text: string): string;
35
+ export declare class CurationStore {
36
+ #private;
37
+ constructor(path: string);
38
+ close(): void;
39
+ annotations(): TemporalAnnotation[];
40
+ addTask(candidate: {
41
+ type: MaintenanceTaskType;
42
+ corpus: string;
43
+ collection: string;
44
+ path: string;
45
+ reason: string;
46
+ contentFingerprint?: string;
47
+ detail?: string;
48
+ }): void;
49
+ listTasks(params?: {
50
+ status?: MaintenanceStatus;
51
+ limit?: number;
52
+ }): MaintenanceTask[];
53
+ updateTask(params: {
54
+ id: string;
55
+ status: Exclude<MaintenanceStatus, "pending">;
56
+ note?: string;
57
+ annotation?: {
58
+ scope: "chunk" | "document";
59
+ eventTime: string;
60
+ basis: TemporalBasis;
61
+ evidence: string;
62
+ };
63
+ }): MaintenanceTask | undefined;
64
+ updateAnnotationLocation(params: {
65
+ annotation: TemporalAnnotation;
66
+ qmdHash: string | null;
67
+ qmdSeq: number | null;
68
+ }): void;
69
+ }
70
+ export {};
@@ -0,0 +1,191 @@
1
+ import { createHash, randomUUID } from "node:crypto";
2
+ import { chmodSync, mkdirSync } from "node:fs";
3
+ import { dirname } from "node:path";
4
+ import { DatabaseSync } from "node:sqlite";
5
+ const TEMPORAL_BASES = ["path", "frontmatter", "session", "agent_verified"];
6
+ const MAINTENANCE_TASK_TYPES = ["ambiguous_event_time", "exact_duplicate"];
7
+ const MAINTENANCE_STATUSES = ["pending", "resolved", "deferred", "irrelevant"];
8
+ function annotation(row) {
9
+ return {
10
+ corpus: row.corpus,
11
+ collection: row.collection,
12
+ path: row.path,
13
+ contentFingerprint: row.content_fingerprint,
14
+ eventTime: row.event_time,
15
+ basis: row.basis,
16
+ evidence: row.evidence,
17
+ qmdHash: row.qmd_hash,
18
+ qmdSeq: row.qmd_seq,
19
+ createdAt: row.created_at,
20
+ updatedAt: row.updated_at,
21
+ };
22
+ }
23
+ function task(row) {
24
+ return {
25
+ id: row.id,
26
+ type: row.type,
27
+ corpus: row.corpus,
28
+ collection: row.collection,
29
+ path: row.path,
30
+ reason: row.reason,
31
+ contentFingerprint: row.content_fingerprint,
32
+ detail: row.detail,
33
+ resolutionNote: row.resolution_note,
34
+ status: row.status,
35
+ createdAt: row.created_at,
36
+ updatedAt: row.updated_at,
37
+ };
38
+ }
39
+ export function chunkFingerprint(text) {
40
+ return createHash("sha256").update(text).digest("hex");
41
+ }
42
+ export class CurationStore {
43
+ #db;
44
+ constructor(path) {
45
+ mkdirSync(dirname(path), { recursive: true, mode: 0o700 });
46
+ this.#db = new DatabaseSync(path);
47
+ chmodSync(path, 0o600);
48
+ this.#db.exec(`
49
+ PRAGMA journal_mode = WAL;
50
+ PRAGMA busy_timeout = 5000;
51
+
52
+ CREATE TABLE IF NOT EXISTS temporal_annotations (
53
+ corpus TEXT NOT NULL,
54
+ collection TEXT NOT NULL,
55
+ path TEXT NOT NULL,
56
+ content_fingerprint TEXT NOT NULL DEFAULT '',
57
+ event_time TEXT NOT NULL,
58
+ basis TEXT NOT NULL CHECK (basis IN ('path', 'frontmatter', 'session', 'agent_verified')),
59
+ evidence TEXT NOT NULL,
60
+ qmd_hash TEXT,
61
+ qmd_seq INTEGER,
62
+ created_at TEXT NOT NULL,
63
+ updated_at TEXT NOT NULL,
64
+ PRIMARY KEY (corpus, collection, path, content_fingerprint)
65
+ );
66
+
67
+ `);
68
+ this.#ensureMaintenanceSchema();
69
+ }
70
+ #ensureMaintenanceSchema() {
71
+ this.#db.exec(`
72
+ CREATE TABLE IF NOT EXISTS maintenance_tasks (
73
+ id TEXT PRIMARY KEY,
74
+ type TEXT NOT NULL CHECK (type IN ('ambiguous_event_time', 'exact_duplicate')),
75
+ corpus TEXT NOT NULL,
76
+ collection TEXT NOT NULL,
77
+ path TEXT NOT NULL,
78
+ reason TEXT NOT NULL,
79
+ content_fingerprint TEXT NOT NULL,
80
+ detail TEXT,
81
+ resolution_note TEXT,
82
+ status TEXT NOT NULL CHECK (status IN ('pending', 'resolved', 'deferred', 'irrelevant')),
83
+ created_at TEXT NOT NULL,
84
+ updated_at TEXT NOT NULL,
85
+ UNIQUE (type, corpus, collection, path, reason, content_fingerprint)
86
+ );
87
+ `);
88
+ this.#db.exec(`
89
+ CREATE INDEX IF NOT EXISTS maintenance_tasks_status_created
90
+ ON maintenance_tasks(status, created_at);
91
+ `);
92
+ }
93
+ close() {
94
+ this.#db.close();
95
+ }
96
+ annotations() {
97
+ return this.#db.prepare(`
98
+ SELECT * FROM temporal_annotations
99
+ ORDER BY collection, path, content_fingerprint
100
+ `).all().map((row) => annotation(row));
101
+ }
102
+ addTask(candidate) {
103
+ const now = new Date().toISOString();
104
+ this.#db.prepare(`
105
+ INSERT INTO maintenance_tasks
106
+ (id, type, corpus, collection, path, reason, content_fingerprint, detail, status, created_at, updated_at)
107
+ VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?)
108
+ ON CONFLICT(type, corpus, collection, path, reason, content_fingerprint) DO UPDATE SET
109
+ detail = CASE
110
+ WHEN maintenance_tasks.status = 'pending' THEN excluded.detail
111
+ ELSE maintenance_tasks.detail
112
+ END,
113
+ updated_at = CASE
114
+ WHEN maintenance_tasks.status = 'pending' THEN excluded.updated_at
115
+ ELSE maintenance_tasks.updated_at
116
+ END
117
+ `).run(randomUUID(), candidate.type, candidate.corpus, candidate.collection, candidate.path, candidate.reason, candidate.contentFingerprint ?? "", candidate.detail ?? null, now, now);
118
+ }
119
+ listTasks(params = {}) {
120
+ const status = params.status ?? "pending";
121
+ const limit = Math.max(1, Math.min(10, Math.floor(params.limit ?? 5)));
122
+ return this.#db.prepare(`
123
+ SELECT * FROM maintenance_tasks
124
+ WHERE status = ?
125
+ ORDER BY created_at, id
126
+ LIMIT ?
127
+ `).all(status, limit).map((row) => task(row));
128
+ }
129
+ updateTask(params) {
130
+ this.#db.exec("BEGIN IMMEDIATE");
131
+ try {
132
+ const row = this.#db.prepare("SELECT * FROM maintenance_tasks WHERE id = ?")
133
+ .get(params.id);
134
+ if (!row) {
135
+ this.#db.exec("COMMIT");
136
+ return undefined;
137
+ }
138
+ const now = new Date().toISOString();
139
+ if (row.type === "ambiguous_event_time" && params.status === "resolved" && !params.annotation) {
140
+ throw new Error("resolving an ambiguous event-time task requires a date annotation");
141
+ }
142
+ if (params.annotation) {
143
+ if (row.type !== "ambiguous_event_time") {
144
+ throw new Error("date annotations can only resolve ambiguous event-time tasks");
145
+ }
146
+ if (params.status !== "resolved") {
147
+ throw new Error("date annotations require resolved status");
148
+ }
149
+ if (!Number.isFinite(Date.parse(params.annotation.eventTime))) {
150
+ throw new Error("date annotation eventTime must be an ISO 8601 timestamp");
151
+ }
152
+ const fingerprint = params.annotation.scope === "document" ? "" : row.content_fingerprint;
153
+ if (params.annotation.scope === "chunk" && !fingerprint) {
154
+ throw new Error("chunk annotation requires a content fingerprint");
155
+ }
156
+ this.#db.prepare(`
157
+ INSERT INTO temporal_annotations
158
+ (corpus, collection, path, content_fingerprint, event_time, basis, evidence,
159
+ qmd_hash, qmd_seq, created_at, updated_at)
160
+ VALUES (?, ?, ?, ?, ?, ?, ?, NULL, NULL, ?, ?)
161
+ ON CONFLICT(corpus, collection, path, content_fingerprint) DO UPDATE SET
162
+ event_time = excluded.event_time,
163
+ basis = excluded.basis,
164
+ evidence = excluded.evidence,
165
+ qmd_hash = NULL,
166
+ qmd_seq = NULL,
167
+ updated_at = excluded.updated_at
168
+ `).run(row.corpus, row.collection, row.path, fingerprint, params.annotation.eventTime, params.annotation.basis, params.annotation.evidence, now, now);
169
+ }
170
+ this.#db.prepare(`
171
+ UPDATE maintenance_tasks
172
+ SET status = ?, resolution_note = ?, updated_at = ?
173
+ WHERE id = ?
174
+ `).run(params.status, params.note ?? null, now, params.id);
175
+ const updated = task(this.#db.prepare("SELECT * FROM maintenance_tasks WHERE id = ?").get(params.id));
176
+ this.#db.exec("COMMIT");
177
+ return updated;
178
+ }
179
+ catch (error) {
180
+ this.#db.exec("ROLLBACK");
181
+ throw error;
182
+ }
183
+ }
184
+ updateAnnotationLocation(params) {
185
+ this.#db.prepare(`
186
+ UPDATE temporal_annotations
187
+ SET qmd_hash = ?, qmd_seq = ?
188
+ WHERE corpus = ? AND collection = ? AND path = ? AND content_fingerprint = ?
189
+ `).run(params.qmdHash, params.qmdSeq, params.annotation.corpus, params.annotation.collection, params.annotation.path, params.annotation.contentFingerprint);
190
+ }
191
+ }
@@ -1,7 +1,8 @@
1
1
  import type { QMDStore } from "@unblocklabs/qmd";
2
- import { type AnalysisRunner, type MemoryAnalysisSummary, type MemoryClusterDetail, type MemoryClusterList, type MemoryReclusterOptions } from "./analysis.js";
2
+ import { type AnalysisRunner, type MemoryAnalysisSummary, type MemoryClusterDetail, type MemoryClusterList, type MemoryClusterSort, type MemoryReclusterOptions } from "./analysis.js";
3
3
  import type { CorpusMemorySearchResult, CorpusSearchOptions, MemoryEmbeddingProbeResult, MemoryProviderStatus, MemoryReadResult, MemorySearchManagerContract, MemorySyncParams } from "./contracts.js";
4
4
  import type { ChatType } from "./config.js";
5
+ import { type MaintenanceStatus, type TemporalBasis } from "./curation.js";
5
6
  import { type SessionSyncResult } from "./session-sync.js";
6
7
  import { type ResolvedSource } from "./sources.js";
7
8
  export type ManagerStore = Pick<QMDStore, "update" | "embed" | "getStatus" | "listCollections" | "searchLex" | "vsearch" | "get" | "getDocumentBody" | "close">;
@@ -28,6 +29,7 @@ export declare class QmdMemoryManager implements MemorySearchManagerContract {
28
29
  #private;
29
30
  constructor(params: {
30
31
  dbPath: string;
32
+ curationPath?: string;
31
33
  workspaceDir: string;
32
34
  sources: readonly ResolvedSource[];
33
35
  storeFactory?: () => Promise<ManagerStore>;
@@ -44,7 +46,24 @@ export declare class QmdMemoryManager implements MemorySearchManagerContract {
44
46
  fetchCluster(params: {
45
47
  clusterId: string;
46
48
  topK?: number;
49
+ offset?: number;
50
+ sort?: MemoryClusterSort;
47
51
  }): Promise<MemoryClusterDetail>;
52
+ listMaintenanceTasks(params?: {
53
+ status?: MaintenanceStatus;
54
+ limit?: number;
55
+ }): import("./curation.js").MaintenanceTask[];
56
+ updateMaintenanceTask(params: {
57
+ id: string;
58
+ status: Exclude<MaintenanceStatus, "pending">;
59
+ note?: string;
60
+ annotation?: {
61
+ scope: "chunk" | "document";
62
+ eventTime: string;
63
+ basis: TemporalBasis;
64
+ evidence: string;
65
+ };
66
+ }): import("./curation.js").MaintenanceTask | undefined;
48
67
  search(query: string, opts?: CorpusSearchOptions): Promise<CorpusMemorySearchResult[]>;
49
68
  readFile(params: {
50
69
  relPath: string;
@@ -2,6 +2,7 @@ import { mkdir, stat } from "node:fs/promises";
2
2
  import { dirname } from "node:path";
3
3
  import chokidar from "chokidar";
4
4
  import { ensureMemoryAnalysisSchema, latestAnalysisRunId, markMemoryAnalysisStale, readAnalysisSummary, readCluster, readClusters, runAnalysisWorker, } from "./analysis.js";
5
+ import { CurationStore, chunkFingerprint, } from "./curation.js";
5
6
  import { readSessionManifest, sessionMetadataByPath, syncSessionProjections, } from "./session-sync.js";
6
7
  import { parseSafeVirtualPath } from "./sources.js";
7
8
  const DEFAULT_READ_LINES = 120;
@@ -135,6 +136,7 @@ function sessionAllowedPaths(metadataByPath, collection, filter) {
135
136
  export class QmdMemoryManager {
136
137
  #dbPath;
137
138
  #workspaceDir;
139
+ #curationPath;
138
140
  #sources;
139
141
  #storeFactory;
140
142
  #keepModelsWarm;
@@ -142,6 +144,7 @@ export class QmdMemoryManager {
142
144
  #analysisRunner;
143
145
  #sessions;
144
146
  #store;
147
+ #curation;
145
148
  #cleanupRemovedDocuments;
146
149
  #operationChain;
147
150
  #watcher;
@@ -155,6 +158,7 @@ export class QmdMemoryManager {
155
158
  #sessionManifestMtimeNs;
156
159
  constructor(params) {
157
160
  this.#dbPath = params.dbPath;
161
+ this.#curationPath = params.curationPath ?? `${params.dbPath}.curation.sqlite`;
158
162
  this.#workspaceDir = params.workspaceDir;
159
163
  this.#sources = new Map(params.sources.map((source) => [source.collection, source]));
160
164
  this.#storeFactory = params.storeFactory;
@@ -391,7 +395,126 @@ export class QmdMemoryManager {
391
395
  return this.#enqueue(async () => readClusters((await this.#getAnalysisStore()).internal.db, limit));
392
396
  }
393
397
  fetchCluster(params) {
394
- return this.#enqueue(async () => readCluster((await this.#getAnalysisStore()).internal.db, params.clusterId, params.topK));
398
+ return this.#enqueue(async () => {
399
+ const db = (await this.#getAnalysisStore()).internal.db;
400
+ this.#loadTemporalAnnotations(db);
401
+ const detail = readCluster(db, params.clusterId, params.topK, params.offset, params.sort, { sessionCollection: this.#sessions?.collection });
402
+ if (params.sort === "date_asc" || params.sort === "date_desc") {
403
+ for (const member of detail.members ?? []) {
404
+ if (member.eventTime !== null)
405
+ continue;
406
+ const safe = parseSafeVirtualPath(member.eventTimeSource, this.#sources);
407
+ if (!safe)
408
+ continue;
409
+ this.#getCuration().addTask({
410
+ type: "ambiguous_event_time",
411
+ corpus: safe.source.corpus,
412
+ collection: safe.source.collection,
413
+ path: safe.relativePath,
414
+ reason: "cluster chronology has no reliable event time",
415
+ contentFingerprint: member.contentFingerprint,
416
+ detail: "Inspect the document and relevant evidence; annotate a date only when one can be supported.",
417
+ });
418
+ }
419
+ }
420
+ if (detail.runId && detail.members) {
421
+ this.#addDuplicateTasks(db, detail.runId, detail.members);
422
+ }
423
+ return detail;
424
+ });
425
+ }
426
+ listMaintenanceTasks(params = {}) {
427
+ return this.#getCuration().listTasks(params);
428
+ }
429
+ updateMaintenanceTask(params) {
430
+ return this.#getCuration().updateTask(params);
431
+ }
432
+ #getCuration() {
433
+ this.#curation ??= new CurationStore(this.#curationPath);
434
+ return this.#curation;
435
+ }
436
+ #loadTemporalAnnotations(db) {
437
+ db.exec("DELETE FROM memory_temporal_annotations");
438
+ const findChunks = db.prepare(`
439
+ SELECT d.hash, vectors.seq, vectors.pos, vectors.chunk_len, content.doc
440
+ FROM documents d
441
+ JOIN content ON content.hash = d.hash
442
+ JOIN content_vectors vectors ON vectors.hash = d.hash
443
+ WHERE d.collection = ? AND d.path = ? AND d.active = 1
444
+ ORDER BY vectors.seq
445
+ `);
446
+ const insert = db.prepare(`
447
+ INSERT OR REPLACE INTO memory_temporal_annotations
448
+ (collection, path, qmd_hash, qmd_seq, event_time, basis, document_wide)
449
+ VALUES (?, ?, ?, ?, ?, ?, ?)
450
+ `);
451
+ const curation = this.#getCuration();
452
+ for (const annotation of curation.annotations()) {
453
+ if (!annotation.contentFingerprint) {
454
+ insert.run(annotation.collection, annotation.path, null, null, annotation.eventTime, annotation.basis, 1);
455
+ continue;
456
+ }
457
+ const rows = findChunks.all(annotation.collection, annotation.path);
458
+ const matched = rows.find((row) => chunkFingerprint(row.doc.slice(row.pos, row.pos + row.chunk_len)) === annotation.contentFingerprint);
459
+ curation.updateAnnotationLocation({
460
+ annotation,
461
+ qmdHash: matched?.hash ?? null,
462
+ qmdSeq: matched?.seq ?? null,
463
+ });
464
+ if (matched) {
465
+ insert.run(annotation.collection, annotation.path, matched.hash, matched.seq, annotation.eventTime, annotation.basis, 0);
466
+ }
467
+ }
468
+ }
469
+ #addDuplicateTasks(db, runId, members) {
470
+ if (members.length === 0)
471
+ return;
472
+ const pageMatch = members.map(() => "(duplicates.canonical_hash = ? AND duplicates.canonical_seq = ?) OR " +
473
+ "(duplicates.duplicate_hash = ? AND duplicates.duplicate_seq = ?)").join(" OR ");
474
+ const pageParams = members.flatMap((member) => [member.hash, member.seq, member.hash, member.seq]);
475
+ const sessionCollections = [...this.#sources.values()]
476
+ .filter((source) => source.kind === "sessions")
477
+ .map((source) => source.collection);
478
+ const excludeSessions = sessionCollections.length > 0
479
+ ? `duplicate_document.collection NOT IN (${sessionCollections.map(() => "?").join(", ")})`
480
+ : "1 = 1";
481
+ const rows = db.prepare(`
482
+ SELECT
483
+ duplicate_document.collection,
484
+ duplicate_document.path,
485
+ duplicates.content_fingerprint,
486
+ COUNT(*) AS occurrence_count
487
+ FROM memory_analysis_duplicate_occurrences duplicates
488
+ JOIN (SELECT DISTINCT hash FROM documents WHERE active = 1) canonical_document
489
+ ON canonical_document.hash = duplicates.canonical_hash
490
+ JOIN documents duplicate_document
491
+ ON duplicate_document.hash = duplicates.duplicate_hash
492
+ AND duplicate_document.active = 1
493
+ WHERE duplicates.run_id = ?
494
+ AND (${pageMatch})
495
+ AND ${excludeSessions}
496
+ GROUP BY duplicate_document.collection, duplicate_document.path,
497
+ duplicates.content_fingerprint
498
+ ORDER BY duplicate_document.collection, duplicate_document.path,
499
+ duplicates.content_fingerprint
500
+ LIMIT 10
501
+ `).all(runId, ...pageParams, ...sessionCollections);
502
+ const curation = this.#getCuration();
503
+ for (const row of rows) {
504
+ const source = this.#sources.get(row.collection);
505
+ if (!source || source.kind === "sessions")
506
+ continue;
507
+ curation.addTask({
508
+ type: "exact_duplicate",
509
+ corpus: source.corpus,
510
+ collection: row.collection,
511
+ path: row.path,
512
+ reason: "exact chunk content repeats in this source document",
513
+ contentFingerprint: row.content_fingerprint,
514
+ detail: `${row.occurrence_count} exact duplicate occurrence${row.occurrence_count === 1 ? "" : "s"}. ` +
515
+ "Review the source and propose cleanup only if repetition is accidental.",
516
+ });
517
+ }
395
518
  }
396
519
  async #getAnalysisStore() {
397
520
  const store = await this.#getStore();
@@ -529,5 +652,7 @@ export class QmdMemoryManager {
529
652
  await this.#operationChain?.catch(() => undefined);
530
653
  await this.#store?.close();
531
654
  this.#store = undefined;
655
+ this.#curation?.close();
656
+ this.#curation = undefined;
532
657
  }
533
658
  }
@@ -192,6 +192,14 @@ function createListClustersTool(runtime, ctx) {
192
192
  const fetchClusterParameters = Type.Object({
193
193
  clusterId: Type.String({ pattern: "^[0-9a-f]{10}$" }),
194
194
  topK: Type.Optional(Type.Integer({ minimum: 1, maximum: 50 })),
195
+ offset: Type.Optional(Type.Integer({ minimum: 0 })),
196
+ sort: Type.Optional(Type.Union([
197
+ Type.Literal("representative"),
198
+ Type.Literal("score_desc"),
199
+ Type.Literal("score_asc"),
200
+ Type.Literal("date_desc"),
201
+ Type.Literal("date_asc"),
202
+ ])),
195
203
  }, { additionalProperties: false });
196
204
  function createFetchClusterTool(runtime, ctx) {
197
205
  const active = getContext(ctx);
@@ -200,14 +208,89 @@ function createFetchClusterTool(runtime, ctx) {
200
208
  return {
201
209
  name: "memory_fetch_cluster",
202
210
  label: "Fetch Memory Cluster",
203
- description: "Fetch the top representative QMD chunks for a clusterId returned by memory_list_clusters.",
211
+ description: "Fetch a sorted page of QMD chunks for a clusterId returned by memory_list_clusters.",
204
212
  parameters: fetchClusterParameters,
205
213
  async execute(_toolCallId, params) {
206
- const { clusterId, topK } = Value.Parse(fetchClusterParameters, params);
214
+ const { clusterId, topK, offset, sort } = Value.Parse(fetchClusterParameters, params);
207
215
  const { manager, error } = await runtime.getMemorySearchManager(active);
208
216
  if (!manager)
209
217
  return jsonResult({ status: "unavailable", error: error ?? "memory unavailable" });
210
- return jsonResult(await manager.fetchCluster({ clusterId, topK }));
218
+ return jsonResult(await manager.fetchCluster({ clusterId, topK, offset, sort }));
219
+ },
220
+ };
221
+ }
222
+ const maintenanceStatus = Type.Union([
223
+ Type.Literal("pending"),
224
+ Type.Literal("resolved"),
225
+ Type.Literal("deferred"),
226
+ Type.Literal("irrelevant"),
227
+ ]);
228
+ const listMaintenanceParameters = Type.Object({
229
+ status: Type.Optional(maintenanceStatus),
230
+ limit: Type.Optional(Type.Integer({ minimum: 1, maximum: 10 })),
231
+ }, { additionalProperties: false });
232
+ function createListMaintenanceTool(runtime, ctx) {
233
+ const active = getContext(ctx);
234
+ if (!active)
235
+ return null;
236
+ return {
237
+ name: "memory_list_maintenance_tasks",
238
+ label: "List Memory Maintenance Tasks",
239
+ description: "List a bounded curation inbox of memory chronology and duplicate-review proposals.",
240
+ parameters: listMaintenanceParameters,
241
+ async execute(_toolCallId, params) {
242
+ const options = Value.Parse(listMaintenanceParameters, params);
243
+ const { manager, error } = await runtime.getMemorySearchManager(active);
244
+ if (!manager)
245
+ return jsonResult({ status: "unavailable", error: error ?? "memory unavailable" });
246
+ return jsonResult({ status: "ok", tasks: manager.listMaintenanceTasks(options) });
247
+ },
248
+ };
249
+ }
250
+ const isoTimestamp = Type.String({
251
+ pattern: "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(?:\\.\\d{1,9})?(?:Z|[+-]\\d{2}:\\d{2})$",
252
+ });
253
+ const updateMaintenanceParameters = Type.Object({
254
+ taskId: Type.String({ pattern: "\\S" }),
255
+ action: Type.Union([
256
+ Type.Literal("resolve"),
257
+ Type.Literal("defer"),
258
+ Type.Literal("irrelevant"),
259
+ ]),
260
+ note: Type.Optional(Type.String({ minLength: 1, maxLength: 500 })),
261
+ annotation: Type.Optional(Type.Object({
262
+ scope: Type.Optional(Type.Union([Type.Literal("chunk"), Type.Literal("document")])),
263
+ eventTime: isoTimestamp,
264
+ basis: Type.Union([
265
+ Type.Literal("path"),
266
+ Type.Literal("frontmatter"),
267
+ Type.Literal("session"),
268
+ Type.Literal("agent_verified"),
269
+ ]),
270
+ evidence: Type.String({ minLength: 1, maxLength: 500 }),
271
+ }, { additionalProperties: false })),
272
+ }, { additionalProperties: false });
273
+ function createUpdateMaintenanceTool(runtime, ctx) {
274
+ const active = getContext(ctx);
275
+ if (!active)
276
+ return null;
277
+ return {
278
+ name: "memory_update_maintenance_task",
279
+ label: "Update Memory Maintenance Task",
280
+ description: "Resolve, defer, or dismiss a memory-maintenance proposal. This tool never edits source Markdown.",
281
+ parameters: updateMaintenanceParameters,
282
+ async execute(_toolCallId, params) {
283
+ const { taskId, action, note, annotation } = Value.Parse(updateMaintenanceParameters, params);
284
+ const { manager, error } = await runtime.getMemorySearchManager(active);
285
+ if (!manager)
286
+ return jsonResult({ status: "unavailable", error: error ?? "memory unavailable" });
287
+ const updated = manager.updateMaintenanceTask({
288
+ id: taskId,
289
+ status: action === "resolve" ? "resolved" : action === "defer" ? "deferred" : "irrelevant",
290
+ note,
291
+ ...(annotation ? { annotation: { ...annotation, scope: annotation.scope ?? "chunk" } } : {}),
292
+ });
293
+ return jsonResult(updated ? { status: "ok", task: updated } : { status: "not_found" });
211
294
  },
212
295
  };
213
296
  }
@@ -304,4 +387,6 @@ export function registerUnblockMemory(api) {
304
387
  api.registerTool((ctx) => createReclusterTool(runtime, ctx), { names: ["memory_recluster"] });
305
388
  api.registerTool((ctx) => createListClustersTool(runtime, ctx), { names: ["memory_list_clusters"] });
306
389
  api.registerTool((ctx) => createFetchClusterTool(runtime, ctx), { names: ["memory_fetch_cluster"] });
390
+ api.registerTool((ctx) => createListMaintenanceTool(runtime, ctx), { names: ["memory_list_maintenance_tasks"] });
391
+ api.registerTool((ctx) => createUpdateMaintenanceTool(runtime, ctx), { names: ["memory_update_maintenance_task"] });
307
392
  }
@@ -204,6 +204,7 @@ export class QmdMemoryRuntime {
204
204
  const manager = new QmdMemoryManager({
205
205
  workspaceDir,
206
206
  dbPath: join(stateDir, "index.sqlite"),
207
+ curationPath: join(stateDir, "curation.sqlite"),
207
208
  sources,
208
209
  keepModelsWarm: this.#keepEmbeddingModelWarm,
209
210
  analysisExecutable: this.#analysisExecutable,
@@ -100,9 +100,6 @@ function formatTimestamp(value, timezone) {
100
100
  return `${part("year")}-${part("month")}-${part("day")} ` +
101
101
  `${part("hour")}:${part("minute")}:${part("second")} ${part("timeZoneName")}`.trim();
102
102
  }
103
- function inline(value) {
104
- return value.replace(/[\r\n]+/gu, " ").replaceAll("`", "\\`");
105
- }
106
103
  export function projectSession(input) {
107
104
  const messages = input.events.flatMap((event) => {
108
105
  const projected = projectMessage(event, input);
@@ -110,22 +107,8 @@ export function projectSession(input) {
110
107
  });
111
108
  if (messages.length === 0)
112
109
  return undefined;
113
- const provider = input.provider ?? "unknown";
114
- const header = [
115
- "# Session",
116
- "",
117
- `- Session ID: \`${inline(input.sessionId)}\``,
118
- `- Provider: ${inline(provider)}`,
119
- `- Chat type: ${input.chatType}`,
120
- ...(input.label ? [`- Conversation: ${inline(input.label)}`] : []),
121
- ...(input.conversationId ? [`- Conversation ID: \`${inline(input.conversationId)}\``] : []),
122
- `- Started: ${new Date(input.startedAt).toISOString()}`,
123
- "",
124
- "## Transcript",
125
- "",
126
- ];
127
110
  const transcript = messages.map((message) => `${formatTimestamp(message.timestamp, input.timezone)} — ${message.speaker}: ${message.text}`);
128
- return `${[...header, ...transcript].join("\n\n")}\n`;
111
+ return `# Transcript\n\n${transcript.join("\n\n")}\n`;
129
112
  }
130
113
  function hash(value) {
131
114
  return createHash("sha256").update(value).digest("hex").slice(0, 16);
@@ -1,11 +1,11 @@
1
1
  import { createHash, randomUUID } from "node:crypto";
2
2
  import { existsSync, lstatSync } from "node:fs";
3
- import { chmod, mkdir, readFile, rename, unlink, writeFile } from "node:fs/promises";
3
+ import { chmod, mkdir, readFile, rename, unlink, utimes, writeFile } from "node:fs/promises";
4
4
  import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
5
5
  import { DatabaseSync } from "node:sqlite";
6
6
  import { projectSession, sessionDocumentPath, } from "./session-projector.js";
7
7
  const MANIFEST_VERSION = 1;
8
- const PROJECTOR_VERSION = 1;
8
+ const PROJECTOR_VERSION = 2;
9
9
  const SUPPORTED_SCHEMA_VERSION = 17;
10
10
  const REQUIRED_COLUMNS = {
11
11
  schema_meta: ["meta_key", "role", "schema_version", "agent_id", "app_version"],
@@ -261,6 +261,7 @@ export async function syncSessionProjections(params) {
261
261
  }
262
262
  const target = projectionPath(params.outputDir, documentPath);
263
263
  await atomicWrite(target, content, 0o600);
264
+ await utimes(target, new Date(), new Date(metadata.startedAt));
264
265
  if (previous?.documentPath && previous.documentPath !== documentPath) {
265
266
  await remove(projectionPath(params.outputDir, previous.documentPath));
266
267
  }
@@ -1,18 +1,20 @@
1
1
  {
2
2
  "id": "unblock-memory",
3
3
  "name": "Unblock Memory",
4
- "version": "0.2.5",
4
+ "version": "0.2.7",
5
5
  "description": "Indexes, retrieves, and analyzes configured workspace memory with existing QMD vectors.",
6
6
  "kind": "memory",
7
7
  "activation": { "onStartup": false },
8
8
  "skills": ["./skills"],
9
- "contracts": { "tools": ["memory_search", "memory_get", "memory_sync_sessions", "memory_sync_status", "memory_recluster", "memory_list_clusters", "memory_fetch_cluster"] },
9
+ "contracts": { "tools": ["memory_search", "memory_get", "memory_sync_sessions", "memory_sync_status", "memory_recluster", "memory_list_clusters", "memory_fetch_cluster", "memory_list_maintenance_tasks", "memory_update_maintenance_task"] },
10
10
  "toolMetadata": {
11
11
  "memory_sync_sessions": { "sideEffecting": true },
12
12
  "memory_sync_status": { "replaySafe": true },
13
13
  "memory_recluster": { "sideEffecting": true },
14
14
  "memory_list_clusters": { "replaySafe": true },
15
- "memory_fetch_cluster": { "replaySafe": true }
15
+ "memory_fetch_cluster": { "replaySafe": true },
16
+ "memory_list_maintenance_tasks": { "replaySafe": true },
17
+ "memory_update_maintenance_task": { "sideEffecting": true }
16
18
  },
17
19
  "uiHints": {
18
20
  "keepEmbeddingModelWarm": {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@unblocklabs/unblock-memory",
3
- "version": "0.2.5",
3
+ "version": "0.2.7",
4
4
  "description": "Workspace-native memory for OpenClaw, powered by QMD",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -30,7 +30,7 @@
30
30
  "preflight": "npm run knip && npm run build && npm run typecheck && npm test && npm run plugin:inspect && npm run plugin:inspect:runtime && npm pack --dry-run"
31
31
  },
32
32
  "dependencies": {
33
- "@unblocklabs/qmd": "https://github.com/unblocklabs-ai/qmd/releases/download/v2.9.2/unblocklabs-qmd-2.9.2.tgz",
33
+ "@unblocklabs/qmd": "https://github.com/unblocklabs-ai/qmd/releases/download/v2.9.3/unblocklabs-qmd-2.9.3.tgz",
34
34
  "chokidar": "5.0.0",
35
35
  "picomatch": "^4.0.5",
36
36
  "typebox": "1.3.6"
@@ -1,78 +1,84 @@
1
1
  ---
2
2
  name: memory-curator
3
- description: Review Unblock Memory clusters and turn supported current knowledge into canon or exploratory reasoning into reflections.
3
+ description: Investigate Unblock Memory clusters and maintain supported, agent-specific knowledge that would otherwise be difficult to reconstruct.
4
4
  ---
5
5
 
6
6
  # Memory Curator
7
7
 
8
- Use Unblock Memory's semantic clusters to maintain durable workspace knowledge.
9
- Clusters show similarity, not truth or consensus. Prefer no write over a weak or
10
- duplicative artifact.
11
-
12
- ## Review clusters
13
-
14
- 1. Call `memory_list_clusters`.
15
- 2. If analysis is missing or stale, call `memory_recluster`, then list again.
16
- 3. Fetch useful clusters with `memory_fetch_cluster`. Treat noise as optional
17
- review material, not automatically important content.
18
- 4. Follow representative `qmd://` source paths with `memory_get` whenever the
19
- excerpt lacks context or a conclusion could change durable knowledge.
20
- 5. Distinguish underlying memory or session evidence from earlier canon and
21
- reflections. Derived artifacts may help locate, challenge, or revise an
22
- understanding, but repetition does not make them independent evidence.
23
- 6. For each reviewed cluster, choose canon, reflection, or no write.
24
-
25
- ## Write canon
26
-
27
- Use `knowledge/canon/<stable-topic>.md` for a supported, durable rule or current
28
- understanding. Update the topic file in place instead of creating dated copies.
29
-
30
- Every semantic chunk in canon must remain correct if retrieved alone:
31
-
32
- - State only the current affirmative truth.
33
- - Remove obsolete instructions rather than preserving them for comparison.
34
- - Never add `Supersedes`, history, changelog, old-process, or migration sections.
35
- - Include a human-readable `Updated` timestamp, the current rationale, and
36
- `qmd://` evidence citations.
37
- - Preserve uncertainty in the claim itself. If the evidence does not support a
38
- stable current claim, write a reflection or nothing.
39
-
40
- A concise shape is sufficient:
41
-
42
- ```markdown
43
- # Topic
44
-
45
- Updated: 2026-08-26 14:30 EDT
46
-
47
- ## Current understanding
48
-
49
- Present-tense rule or facts.
50
-
51
- ## Rationale
52
-
53
- Why this is the current understanding.
54
-
55
- ## Evidence
56
-
57
- - qmd://memory/...
58
- - qmd://sessions/...
59
- ```
60
-
61
- ## Write reflections
62
-
63
- Use `knowledge/reflections/YYYY-MM-DD.md` for patterns, hypotheses,
64
- contradictions, open questions, or reasoning worth revisiting. Append a
65
- timestamped section when the day's file already exists. Label uncertainty
66
- plainly and include the `qmd://` evidence examined. Reflections are not
67
- authoritative instructions.
8
+ Use semantic clusters as attention signals for maintaining the agent's current
9
+ understanding of its world. A cluster shows similarity, not a complete timeline,
10
+ truth, or consensus. Do not write from a cluster alone, and prefer no write over
11
+ weak, duplicative, or easily looked-up knowledge.
12
+
13
+ ## Investigate
14
+
15
+ 1. Call `memory_list_clusters`. If analysis is missing or stale, call
16
+ `memory_recluster`, then list again.
17
+ 2. Fetch a useful cluster with `memory_fetch_cluster`. Start with
18
+ `sort: "representative"`; use `score_desc`, `date_asc`, or `date_desc` and
19
+ pagination when relevance, evolution, or recent state matters. Treat
20
+ `eventTime` as event chronology; `sourceModifiedAt` is only a labeled
21
+ fallback when `eventTime` is unresolved.
22
+ 3. State the question the cluster raises: what may be repeated, contradictory,
23
+ changing, or worth understanding?
24
+ 4. Search existing knowledge with `memory_search`, using
25
+ `corpora: ["knowledge"]`. If that corpus is not configured, report that and
26
+ do not create an unindexed file.
27
+ 5. Investigate the evidence needed to answer the question. Follow important
28
+ `qmd://` sources with `memory_get`, search adjacent memory or sessions, inspect
29
+ other clusters, and check live systems, local files, authoritative docs, or
30
+ the web when they are the right source. The selected cluster is not assumed
31
+ to contain the whole timeline or the current truth.
32
+ 6. Distinguish underlying evidence from earlier agent-created knowledge.
33
+ Derived repetition can locate or challenge a conclusion, but is not
34
+ independent corroboration.
35
+
36
+ ## Decide what belongs in knowledge
37
+
38
+ Write knowledge when the result is agent-specific, evolving, and expensive to
39
+ reconstruct, such as current fleet composition, relationships, preferences,
40
+ project decisions, local conventions, or an assessment synthesized across
41
+ time.
42
+
43
+ Prefer a lookup for public or vendor-owned facts, generic command syntax, and
44
+ behavior likely to change with a third-party release. Those sources may verify
45
+ a local conclusion, but do not copy ordinary documentation into knowledge. A
46
+ local policy or deliberate divergence can belong when its local meaning and
47
+ rationale are the durable part.
48
+
49
+ Choose one outcome: update knowledge or no write. No write is successful when
50
+ the evidence is insufficient, the information is already accurate, or lookup
51
+ is better than memory.
52
+
53
+ ## Maintain knowledge
54
+
55
+ Use a stable `knowledge/<topic>.md` file and update it in place. Facts, agent
56
+ assessments, and uncertainty may coexist. Qualify each claim where it appears so
57
+ it remains honest if semantic chunking retrieves it alone:
58
+
59
+ - identify what is verified and when or against which current source;
60
+ - label an interpretation as the agent's assessment rather than a settled fact;
61
+ - state uncertainty or an open question directly with the affected claim.
62
+
63
+ Keep the file focused on current understanding. Remove stale conclusions rather
64
+ than retaining `Supersedes`, history, changelog, migration, or old-process
65
+ sections. Raw memory and sessions preserve the evidence history. Include a
66
+ human-readable update time and useful evidence citations, but do not force a
67
+ rigid document template.
68
68
 
69
69
  ## Finish the cycle
70
70
 
71
71
  - Do not rewrite raw memory or session projections.
72
- - Avoid restating knowledge already captured accurately.
73
- - Verify new or updated knowledge with `memory_search`, selecting `canon` or
74
- `reflections` when useful.
75
- - Report clusters reviewed, files changed, evidence used, uncertainties, and
76
- intentional skips.
72
+ - Review a small page from `memory_list_maintenance_tasks`. For ambiguous dates,
73
+ investigate supporting evidence and use `memory_update_maintenance_task` to
74
+ attach a chunk or document date only when supported; otherwise defer or mark
75
+ it irrelevant. For exact-duplicate proposals, decide whether cleanup should
76
+ be proposed, but do not treat repetition across historical files as an error.
77
+ The maintenance tools never change source Markdown, and generated session
78
+ projections must never be manually cleaned.
79
+ - Verify an updated file with `memory_search`, using
80
+ `corpora: ["knowledge"]`, and check all-corpora ranking when useful.
81
+ - Report the questions investigated, evidence consulted beyond each cluster,
82
+ files changed, current uncertainties, and intentional no-write decisions.
77
83
  - Do not recluster again after this cycle's writes. Let them enter the next
78
84
  scheduled cycle so the run cannot recursively react to its own output.