@gamaze/hicortex 0.16.0 → 0.16.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. package/README.md +9 -0
  2. package/dist/capture.d.ts +18 -1
  3. package/dist/capture.js +3 -2
  4. package/dist/classify-domains.d.ts +1 -1
  5. package/dist/classify-domains.js +5 -7
  6. package/dist/cli.js +10 -2
  7. package/dist/cluster.d.ts +5 -4
  8. package/dist/cluster.js +2 -3
  9. package/dist/consolidate.js +6 -5
  10. package/dist/db.js +23 -0
  11. package/dist/dedup.js +1 -1
  12. package/dist/distiller.js +19 -12
  13. package/dist/domain-classify.d.ts +1 -1
  14. package/dist/domain-classify.js +1 -5
  15. package/dist/eval/relevance-eval.d.ts +64 -0
  16. package/dist/eval/relevance-eval.js +1954 -0
  17. package/dist/eval/run-eval.js +0 -1
  18. package/dist/index.js +3 -3
  19. package/dist/init.d.ts +165 -0
  20. package/dist/init.js +283 -57
  21. package/dist/lessons-context.js +3 -2
  22. package/dist/mcp-server.js +72 -25
  23. package/dist/nightly.js +35 -3
  24. package/dist/nofit.d.ts +1 -1
  25. package/dist/nofit.js +1 -2
  26. package/dist/prompts.js +22 -13
  27. package/dist/recall-index.d.ts +56 -21
  28. package/dist/recall-index.js +51 -29
  29. package/dist/retrieval.d.ts +7 -7
  30. package/dist/retrieval.js +20 -24
  31. package/dist/schema-prototypes.d.ts +8 -13
  32. package/dist/schema-prototypes.js +13 -22
  33. package/dist/seed-lesson.d.ts +1 -1
  34. package/dist/seed-lesson.js +1 -2
  35. package/dist/storage.d.ts +9 -12
  36. package/dist/storage.js +19 -21
  37. package/dist/types.d.ts +47 -23
  38. package/domains.example.json +2 -3
  39. package/hermes-plugin/hicortex/README.md +3 -1
  40. package/hermes-plugin/hicortex/config.py +33 -2
  41. package/hermes-plugin/hicortex/plugin.yaml +1 -1
  42. package/hermes-plugin/hicortex/provider.py +5 -0
  43. package/package.json +2 -1
package/README.md CHANGED
@@ -164,6 +164,7 @@ npx @gamaze/hicortex context edit <name> # Edit a context section in $EDIT
164
164
  npx @gamaze/hicortex context show --agent <id> # Show a specific agent's resolved context (0.13)
165
165
  npx @gamaze/hicortex init --agent-name <name> # Opt in to a per-agent context id (default: unset — shared global context)
166
166
  npx @gamaze/hicortex init --agent-name "" # Clear it back to global context
167
+ npx @gamaze/hicortex init --repair-config # Recover from a malformed config.json (see below)
167
168
  npx @gamaze/hicortex telemetry # Show exactly what anonymous telemetry sends
168
169
  npx @gamaze/hicortex status # Show config, DB stats
169
170
  npx @gamaze/hicortex uninstall # Remove CC integration (keeps DB)
@@ -226,6 +227,7 @@ Config at `~/.hicortex/config.json`. Created by `init`. Key options:
226
227
  | `recallMinSimilarity` | Relevance floor for index entries (default: 0.55; text-search matches always pass) |
227
228
  | `recallReshowTurns` | Turns before an already-shown memory may reappear in the same session (default: 30) |
228
229
  | `recallMinPromptChars` | Prompts shorter than this skip the recall index (default: 20) |
230
+ | `recallTitleChars` | Chars of each memory's first line shown in an index entry (default: 150, range 40–400). Raised from 100 on 2026-08-02: with topic-first memory titles, 150 chars carries the subject *and* its claim, where 100 cut the claim mid-sentence. Costs roughly +74 tokens per 6-line block |
229
231
  | `sessionIntentWeight` | Blend weight of the session-intent rolling centroid in the recall search vector: `query = (1-w)·prompt + w·centroid` (default: 0.33; set 0 to disable — pure-prompt recall, the kill-switch). The first turn of a session searches with pure prompt and seeds the centroid; subsequent turns blend so recall follows the session's intent instead of being query-literal. The EMA rate (0.4) is a shipped constant, not configurable |
230
232
  | `dedupMergeThreshold` | Minimum cosine similarity for `hicortex dedup` to cluster memories as near-duplicates (default: 0.92) |
231
233
  | `supersessionMinSimilarity` | Minimum cosine similarity for a nightly supersession candidate pair (default: 0.80) |
@@ -329,6 +331,13 @@ npm test
329
331
 
330
332
  ## Troubleshooting
331
333
 
334
+ **`init` fails with "Refusing to write ~/.hicortex/config.json":** the file exists but is not valid JSON — usually a hand-edit slip (a trailing comma, a truncated write). `init` refuses rather than overwriting it, because overwriting would lose `authToken`, `licenseKey`, `distillApiKey`, and your `domains` list. Two ways out:
335
+
336
+ 1. **Preferred — fix the JSON.** The error names the parse failure and its position. Correct it and re-run `init`. Nothing is lost.
337
+ 2. **`npx @gamaze/hicortex init --repair-config`.** Moves the broken file to `config.json.corrupt-<timestamp>` and rebuilds from scratch. Nothing is deleted, and it prints the top-level key names it found (names only — never secret values) so you know what to copy back. **This mints a new `authToken`**, so every thin client pointing at this server must be updated or its recall will silently 401 (recall is fail-soft — you will see no error, just no memories).
338
+
339
+ The nightly and the server behave differently on purpose: a malformed config makes them log a warning and run degraded rather than refuse to start, so a broken config never takes recall offline.
340
+
332
341
  **Tools not visible to agent (OC):** The plugin auto-adds tools to `tools.allow` on startup. Restart the gateway after install.
333
342
 
334
343
  **OC plugin: "Server unreachable":** The plugin requires a running Hicortex server. Run `npx @gamaze/hicortex init` on the same machine, or set `serverUrl` in the plugin config to point at a remote server.
package/dist/capture.d.ts CHANGED
@@ -52,11 +52,17 @@ export interface Segment {
52
52
  export interface DistillBody {
53
53
  text: string;
54
54
  source_agent: string;
55
+ /** Stable client UUID (config.json `agentId`). Attribution only. */
56
+ source_agent_id?: string | null;
57
+ /** Client-declared topic/domain of the capturing agent. Provenance only. */
58
+ source_domain?: string | null;
55
59
  project: string;
56
60
  session_id: string;
57
61
  segment_id: string;
58
62
  session_date: string;
59
- privacy: string;
63
+ /** 0.16.x: optional/vestigial. The distiller no longer sets it; a legacy
64
+ * client may. Honored if present, else the memory stores NULL. */
65
+ privacy?: string;
60
66
  }
61
67
  /** Normalized POST result the caller's transport returns. */
62
68
  export interface PostResult {
@@ -73,6 +79,17 @@ export interface CaptureOptions {
73
79
  dryRun?: boolean;
74
80
  /** Segment size cap; defaults to SEGMENT_MAX_CHARS. Lowered in tests. */
75
81
  segmentMaxChars?: number;
82
+ /**
83
+ * Per-client attribution UUID (config.json `agentId`). Sent on every
84
+ * segment as `source_agent_id`. Null when the client has no `agentId`
85
+ * (e.g. a pre-0.16.x config that has not re-run init).
86
+ */
87
+ sourceAgentId?: string | null;
88
+ /**
89
+ * Per-client declared topic/domain (config.json `sourceDomain`). Sent as
90
+ * `source_domain` provenance. Null when undeclared.
91
+ */
92
+ sourceDomain?: string | null;
76
93
  }
77
94
  export interface CaptureResult {
78
95
  memoriesIngested: number;
package/dist/capture.js CHANGED
@@ -134,7 +134,7 @@ function packSegments(entries, startCursor, entryCursors, maxChars = exports.SEG
134
134
  * boundary) while other sessions continue. A 429/401 stops the whole loop.
135
135
  */
136
136
  async function captureBatches(batches, opts) {
137
- const { post, cursorStore, dryRun = false, segmentMaxChars = exports.SEGMENT_MAX_CHARS } = opts;
137
+ const { post, cursorStore, dryRun = false, segmentMaxChars = exports.SEGMENT_MAX_CHARS, sourceAgentId, sourceDomain } = opts;
138
138
  let memoriesIngested = 0;
139
139
  let sessionsSent = 0;
140
140
  let hadTransientFailure = false;
@@ -186,11 +186,12 @@ async function captureBatches(batches, opts) {
186
186
  const body = {
187
187
  text: seg.text,
188
188
  source_agent: batch.sourceAgent ?? `claude-code/${batch.projectName}`,
189
+ source_agent_id: sourceAgentId ?? null,
190
+ source_domain: sourceDomain ?? null,
189
191
  project: batch.projectName,
190
192
  session_id: batch.sessionId,
191
193
  segment_id: `${genPrefix}${seg.segStart}-${seg.segEnd}${seg.idSuffix}`,
192
194
  session_date: batch.date,
193
- privacy: "WORK",
194
195
  };
195
196
  let result;
196
197
  try {
@@ -16,7 +16,7 @@
16
16
  * reflect tier), same as the nightly. The LLM emits ONLY the ordered tag
17
17
  * set; per-tag weights come from the domain prototypes (computed once at
18
18
  * run start) and the PRIMARY (memories.domain) is derived (argmax weight,
19
- * compartment override, LLM order breaking ties) inside
19
+ * LLM order breaking ties) inside
20
20
  * storage.setMemoryTags. After a completed (non-aborted) run the
21
21
  * prototypes, all weights, and all primaries are recomputed from the
22
22
  * final tag sets — same reconsolidation pass as the nightly.
@@ -17,7 +17,7 @@
17
17
  * reflect tier), same as the nightly. The LLM emits ONLY the ordered tag
18
18
  * set; per-tag weights come from the domain prototypes (computed once at
19
19
  * run start) and the PRIMARY (memories.domain) is derived (argmax weight,
20
- * compartment override, LLM order breaking ties) inside
20
+ * LLM order breaking ties) inside
21
21
  * storage.setMemoryTags. After a completed (non-aborted) run the
22
22
  * prototypes, all weights, and all primaries are recomputed from the
23
23
  * final tag sets — same reconsolidation pass as the nightly.
@@ -195,7 +195,6 @@ async function runClassifyDomains(options = {}) {
195
195
  // Prototypes once at run start — newly classified memories get their
196
196
  // weights from these; the post-run reconsolidation pass refreshes
197
197
  // everything from the final tag sets.
198
- const compartments = (0, schema_prototypes_js_1.compartmentSet)(domains);
199
198
  const { prototypes } = await (0, schema_prototypes_js_1.computeDomainPrototypes)(db, domains, getEmbedFn);
200
199
  // Scope filter: default = NULL / not-in-set / no tags yet; --all = everything.
201
200
  const placeholders = domains.map(() => "?").join(", ");
@@ -255,10 +254,9 @@ async function runClassifyDomains(options = {}) {
255
254
  continue;
256
255
  }
257
256
  // Derived primary (argmax weight from the run-start prototypes,
258
- // compartment override, LLM order breaking ties) — the same value
259
- // setMemoryTags will write below.
257
+ // LLM order breaking ties) — the same value setMemoryTags writes below.
260
258
  const weights = (0, schema_prototypes_js_1.computeTagWeights)(db, row.id, result.tags, prototypes);
261
- const derived = (0, schema_prototypes_js_1.derivePrimary)(result.tags.map((tag) => ({ tag, weight: weights[tag] ?? null })), compartments);
259
+ const derived = (0, schema_prototypes_js_1.derivePrimary)(result.tags.map((tag) => ({ tag, weight: weights[tag] ?? null })));
262
260
  if (derived === row.domain) {
263
261
  batchUnchanged++;
264
262
  }
@@ -273,10 +271,10 @@ async function runClassifyDomains(options = {}) {
273
271
  const tx = db.transaction(() => {
274
272
  for (const w of writes) {
275
273
  if (w.kind === "tags") {
276
- storage.setMemoryTags(db, w.id, w.tags, { weights: w.weights, compartments });
274
+ storage.setMemoryTags(db, w.id, w.tags, { weights: w.weights });
277
275
  }
278
276
  else if (w.resolution.kind === "weak_primary") {
279
- (0, nofit_js_1.applyWeakPrimary)(db, w.id, w.resolution.domain, w.resolution.weight, compartments);
277
+ (0, nofit_js_1.applyWeakPrimary)(db, w.id, w.resolution.domain, w.resolution.weight);
280
278
  }
281
279
  else {
282
280
  (0, nofit_js_1.applyNoAssociationDecay)(db, w.id);
package/dist/cli.js CHANGED
@@ -43,9 +43,14 @@ switch (command) {
43
43
  console.error("[hicortex] init: --agent-name requires a value, e.g. --agent-name lenovo");
44
44
  process.exit(1);
45
45
  }
46
+ const repairConfig = process.argv.includes("--repair-config");
46
47
  import("./init.js").then(({ runInit }) => {
47
- runInit({ serverUrl, agentName }).catch((err) => {
48
- console.error("[hicortex] Init failed:", err);
48
+ runInit({ serverUrl, agentName, repairConfig }).catch((err) => {
49
+ // Operator-fixable failures (a malformed config.json) carry a complete,
50
+ // actionable message — print that alone. A stack trace would bury it.
51
+ // Anything else is a real bug and gets the full error object.
52
+ const operatorFixable = err instanceof Error && /^Refusing to (read|write) /.test(err.message);
53
+ console.error("[hicortex] Init failed:", operatorFixable ? err.message : err);
49
54
  process.exit(1);
50
55
  });
51
56
  });
@@ -250,6 +255,9 @@ Commands:
250
255
  init --server <url> Set up as client (remote server)
251
256
  init --agent-name <name> Opt in to a per-agent context id (default: unset — shared global context)
252
257
  Pass --agent-name "" to clear it back to global
258
+ init --repair-config Recover from a malformed ~/.hicortex/config.json: move it to
259
+ config.json.corrupt-<timestamp> and rebuild. Nothing is deleted.
260
+ Mints a NEW authToken — every thin client must be updated.
253
261
  nightly Run nightly denoise + capture + consolidate
254
262
  relink Resumable link-discovery pass over the ENTIRE corpus (server mode)
255
263
  dedup Cluster + merge near-duplicate memories (server mode; dry run by default)
package/dist/cluster.d.ts CHANGED
@@ -25,18 +25,19 @@ export declare class UnionFind {
25
25
  export declare function clusterEdges(edges: Edge[], threshold: number): string[][];
26
26
  /** Excess = sum(cluster size − 1) — rows that would disappear if every cluster merged to one. */
27
27
  export declare function clusterExcess(clusters: string[][]): number;
28
- /** The three metadata fields a merge candidate cluster must agree on. */
28
+ /** Metadata fields a merge candidate cluster must agree on. `privacy` is on the row (the
29
+ * column still exists) but is NOT a merge-safety field — vestigial since 0.16.2. */
29
30
  export interface ClusterMetaRow {
30
31
  project: string | null;
31
- privacy: string;
32
+ privacy: string | null;
32
33
  source_agent: string;
33
34
  }
34
35
  export interface ClusterMetadataMismatch {
35
36
  projectMismatch: boolean;
36
- privacyMismatch: boolean;
37
37
  sourceAgentMismatch: boolean;
38
38
  }
39
- /** Do cluster members disagree on project/privacy/source_agent? (merge-safety input for #100). */
39
+ /** Do cluster members disagree on project / source_agent? (merge-safety input for #100).
40
+ * Privacy is intentionally NOT checked — it is vestigial since 0.16.2. */
40
41
  export declare function clusterMetadataMismatch(members: ClusterMetaRow[]): ClusterMetadataMismatch;
41
42
  /**
42
43
  * Build the max-cosine edge set via top-K KNN on `memory_vectors`, keeping
package/dist/cluster.js CHANGED
@@ -69,14 +69,13 @@ function clusterEdges(edges, threshold) {
69
69
  function clusterExcess(clusters) {
70
70
  return clusters.reduce((sum, c) => sum + (c.length - 1), 0);
71
71
  }
72
- /** Do cluster members disagree on project/privacy/source_agent? (merge-safety input for #100). */
72
+ /** Do cluster members disagree on project / source_agent? (merge-safety input for #100).
73
+ * Privacy is intentionally NOT checked — it is vestigial since 0.16.2. */
73
74
  function clusterMetadataMismatch(members) {
74
75
  const projects = new Set(members.map((m) => m.project ?? "\u0000null"));
75
- const privacies = new Set(members.map((m) => m.privacy));
76
76
  const agents = new Set(members.map((m) => m.source_agent));
77
77
  return {
78
78
  projectMismatch: projects.size > 1,
79
- privacyMismatch: privacies.size > 1,
80
79
  sourceAgentMismatch: agents.size > 1,
81
80
  };
82
81
  }
@@ -307,7 +307,10 @@ async function stageReflection(db, memories, llm, budget, embedFn, dryRun) {
307
307
  const severity = String(lo.severity ?? "important");
308
308
  const confidence = String(lo.confidence ?? "medium");
309
309
  const sourcePattern = String(lo.source_pattern ?? "");
310
- let content = `## Lesson: ${lessonText}\n\n`;
310
+ // No `## Lesson:` prefix: memory_type='lesson' carries the type, and the
311
+ // text is the topic-first first line (display reads the first line, not a
312
+ // header parse — see lessons-context.ts / index.ts).
313
+ let content = `${lessonText}\n\n`;
311
314
  content += `**Type:** ${lessonType}\n`;
312
315
  content += `**Severity:** ${severity}\n`;
313
316
  content += `**Confidence:** ${confidence}\n`;
@@ -353,7 +356,6 @@ async function stageReflection(db, memories, llm, budget, embedFn, dryRun) {
353
356
  project,
354
357
  memoryType: "lesson",
355
358
  baseStrength: baseStrength[severity] ?? 0.8,
356
- privacy: "WORK",
357
359
  });
358
360
  generated++;
359
361
  }
@@ -433,7 +435,6 @@ async function stageContentDomains(db, domains, llm, budget, embedFn, dryRun, st
433
435
  return { curated: false, domains: domains.length, classified: 0, reason: `dry_run (${rows.length} would classify)` };
434
436
  }
435
437
  const getEmbedFn = async () => embedFn;
436
- const compartments = (0, schema_prototypes_js_1.compartmentSet)(domains);
437
438
  let classified = 0;
438
439
  let weakPrimary = 0;
439
440
  let noAssociationDecayed = 0;
@@ -462,7 +463,7 @@ async function stageContentDomains(db, domains, llm, budget, embedFn, dryRun, st
462
463
  // double-halves.
463
464
  const resolution = (0, nofit_js_1.resolveNoFit)(db, row.id, domains, startPrototypes, weakPrimaryFloor);
464
465
  if (resolution.kind === "weak_primary") {
465
- (0, nofit_js_1.applyWeakPrimary)(db, row.id, resolution.domain, resolution.weight, compartments);
466
+ (0, nofit_js_1.applyWeakPrimary)(db, row.id, resolution.domain, resolution.weight);
466
467
  weakPrimary++;
467
468
  }
468
469
  else {
@@ -472,7 +473,7 @@ async function stageContentDomains(db, domains, llm, budget, embedFn, dryRun, st
472
473
  continue;
473
474
  }
474
475
  const weights = (0, schema_prototypes_js_1.computeTagWeights)(db, row.id, result.tags, startPrototypes);
475
- storage.setMemoryTags(db, row.id, result.tags, { weights, compartments });
476
+ storage.setMemoryTags(db, row.id, result.tags, { weights });
476
477
  classified++;
477
478
  }
478
479
  }
package/dist/db.js CHANGED
@@ -441,6 +441,29 @@ const MIGRATIONS = [
441
441
  `);
442
442
  },
443
443
  },
444
+ {
445
+ version: 11,
446
+ name: "add_source_attribution",
447
+ up: (db) => {
448
+ // 0.16.x attribution + provenance. Two nullable columns on `memories`,
449
+ // both populated ONLY by capture (/distill) from client-declared values;
450
+ // nothing filters, scopes, or scores on either (attribution + echo).
451
+ //
452
+ // `source_agent_id`: the capturing client's stable UUID (config.json
453
+ // `agentId`, generated once by init). Survives agent/machine renames
454
+ // — unlike `source_agent`, a readable name. NULL on legacy rows.
455
+ // `source_domain`: the client-declared topic/domain of the capturing
456
+ // agent (config.json `domain`). Distinct from the content-classified
457
+ // `domain` column (which stays the LLM/prototype-derived primary).
458
+ // Guarded with hasColumn for idempotency across partially-migrated DBs.
459
+ if (!hasColumn(db, "memories", "source_agent_id")) {
460
+ db.exec("ALTER TABLE memories ADD COLUMN source_agent_id TEXT");
461
+ }
462
+ if (!hasColumn(db, "memories", "source_domain")) {
463
+ db.exec("ALTER TABLE memories ADD COLUMN source_domain TEXT");
464
+ }
465
+ },
466
+ },
444
467
  ];
445
468
  /**
446
469
  * Run all pending migrations against the database.
package/dist/dedup.js CHANGED
@@ -287,7 +287,7 @@ async function runDedup(options = {}) {
287
287
  if (members.length < 2)
288
288
  continue; // defensive — a member vanished between KNN and load
289
289
  const mismatch = (0, cluster_js_1.clusterMetadataMismatch)(members);
290
- if (mismatch.projectMismatch || mismatch.privacyMismatch || mismatch.sourceAgentMismatch) {
290
+ if (mismatch.projectMismatch || mismatch.sourceAgentMismatch) {
291
291
  mismatchSkipped.push({ size: members.length, memberIds: members.map((m) => m.id), mismatch });
292
292
  continue;
293
293
  }
package/dist/distiller.js CHANGED
@@ -314,6 +314,14 @@ async function distillChunk(llm, transcript, projectName, date) {
314
314
  return { entries: [], dropped: [] };
315
315
  }
316
316
  const parsed = parseDistilledEntries(result);
317
+ // Smoke alarm (PR #218 review): the prompt enforces topic-first, but models
318
+ // sometimes ignore constraints (cf. the prior max-15-bullet failure). Count
319
+ // entries that still look actor-led or bracket-led so a format regression
320
+ // shows in nightly logs, not months later in the next eval. Non-blocking.
321
+ const offTopic = parsed.filter((e) => /^\s*(user|ai|the user|assistant)\b/i.test(e) || /^\s*\[/.test(e)).length;
322
+ if (parsed.length > 0 && offTopic > 0) {
323
+ console.log(`[hicortex] topic-first check: ${offTopic}/${parsed.length} entries look actor/bracket-led (prompt may be ignored)`);
324
+ }
317
325
  const entries = [];
318
326
  const dropped = [];
319
327
  for (const entry of parsed) {
@@ -409,23 +417,22 @@ function hasMinimalSubstance(entry) {
409
417
  function parseDistilledEntries(markdown) {
410
418
  const entries = [];
411
419
  const lines = markdown.split("\n");
412
- let currentSection = "";
413
420
  for (const line of lines) {
414
421
  const trimmed = line.trim();
415
- // Section headers
416
- if (trimmed.startsWith("### ")) {
417
- currentSection = trimmed.slice(4).trim();
422
+ // Skip all markdown headers (session title, section headings). Sections
423
+ // are NOT prefixed onto entries: each bullet already
424
+ // starts with its [SUBJECT] (topic-first, enforced by prompts.ts), and
425
+ // prepending "[Section]" re-introduced the category-first prefix the
426
+ // 2026-08-02 corpus rewrite removed. The section label is unused
427
+ // downstream (distilled memories all store memory_type='episode').
428
+ if (trimmed.startsWith("# ") ||
429
+ trimmed.startsWith("## ") ||
430
+ trimmed.startsWith("### ")) {
418
431
  continue;
419
432
  }
420
- // Skip top-level headers and classification
421
- if (trimmed.startsWith("# ") || trimmed.startsWith("## "))
422
- continue;
423
- // Bullet items are individual memories
433
+ // Bullet items are individual, already topic-first memories.
424
434
  if (trimmed.startsWith("- ") && trimmed.length > 5) {
425
- const entry = currentSection
426
- ? `[${currentSection}] ${trimmed.slice(2)}`
427
- : trimmed.slice(2);
428
- entries.push(entry);
435
+ entries.push(trimmed.slice(2));
429
436
  }
430
437
  }
431
438
  return entries;
@@ -18,7 +18,7 @@
18
18
  * (the order is used solely as an exact-weight tiebreak downstream).
19
19
  * The PRIMARY (memories.domain) is NO LONGER requested from the LLM — audits
20
20
  * proved LLM primaries a coin-flip on overlapping spheres. It is DERIVED
21
- * deterministically (argmax association weight + compartment override) in
21
+ * deterministically (argmax association weight, LLM tag order breaking ties) in
22
22
  * schema-prototypes.ts / storage.setMemoryTags.
23
23
  *
24
24
  * NO-FIT = EMPTY TAG SET (owner amendment 07.07): "Unsorted" is a non-tag —
@@ -19,7 +19,7 @@
19
19
  * (the order is used solely as an exact-weight tiebreak downstream).
20
20
  * The PRIMARY (memories.domain) is NO LONGER requested from the LLM — audits
21
21
  * proved LLM primaries a coin-flip on overlapping spheres. It is DERIVED
22
- * deterministically (argmax association weight + compartment override) in
22
+ * deterministically (argmax association weight, LLM tag order breaking ties) in
23
23
  * schema-prototypes.ts / storage.setMemoryTags.
24
24
  *
25
25
  * NO-FIT = EMPTY TAG SET (owner amendment 07.07): "Unsorted" is a non-tag —
@@ -84,10 +84,6 @@ function parseConfigDomains(config) {
84
84
  if (!name)
85
85
  continue;
86
86
  const def = { name, description };
87
- // Compartment policy passthrough (graded-schema spec): a domain flagged
88
- // `compartment: true` becomes the primary whenever tagged.
89
- if (d.compartment === true)
90
- def.compartment = true;
91
87
  out.push(def);
92
88
  }
93
89
  return out.length > 0 ? out : null;
@@ -0,0 +1,64 @@
1
+ #!/usr/bin/env node
2
+ /**
3
+ * Real-query relevance + SNIPPET eval — recall QUALITY on real agent prompts.
4
+ *
5
+ * v2 (spec `specs/2026-08-02-relevance-eval.md`) extends the v1 selection-only
6
+ * eval with the SNIPPET layer: v1 asked "did retrieve() surface the right
7
+ * memories?" (judge sees up to 2000 chars). Production shows the agent only a
8
+ * ~100-char one-liner (`recall-index.ts#memoryTitle`), so a memory can be
9
+ * genuinely relevant while its rendered line is useless — v1 scored that as a
10
+ * win. v2 grades BOTH: `full_verdict` (selection quality, judge sees full
11
+ * content) and `line_verdict` (snippet quality, judge sees ONLY the rendered
12
+ * production one-liner — imported from `recall-index.ts`, never reimplemented).
13
+ *
14
+ * v2 additions (spec §4, §5, §6, §6b) layered onto the v1 base (prompt
15
+ * sampling, readonly snapshot handling, embed-once + neverCalledEmbed,
16
+ * lenient JSON parse, distribution/CI reporting):
17
+ * 1. Dual verdict per surfaced memory — TWO separate, blind judge calls.
18
+ * 2. Snippet-length sweep (100/200/300/title+1st-sentence) on a fixed
19
+ * 40-prompt subset (8 per source).
20
+ * 3. Similarity-floor + retrieval-source analysis (near-free — logged, not
21
+ * re-judged).
22
+ * 4. Token-cost estimate (char/4) per K and per snippet-length variant.
23
+ * 5. ~20 rendered ACTUAL production blocks dumped into the report.
24
+ * 6. Redundancy — one set-level judge call per (prompt × mode) over the
25
+ * production 6.
26
+ * 7. Rate limiting + resumability (§6b, MANDATORY): serial calls,
27
+ * `--judge-delay-ms` (default 2000), exponential backoff with jitter on
28
+ * 429/5xx/timeout (5→10→20→40→80s, max 5 retries, respects
29
+ * `Retry-After`), checkpoint-per-call to a `.jsonl` sidecar, `--resume`,
30
+ * progress logging, 10%-error-rate abort, `--max-calls` budget guard
31
+ * (default 900).
32
+ *
33
+ * Prompt corpus (spec §2, owner decision §11.1): EVEN split, 20 prompts per
34
+ * source × 5 sources — Hermes (lenny, raider, nano) + CC (the DevOps
35
+ * `infrastructure` project, the `aironic-marine` project). Saved to
36
+ * `data/prompts.json`, stable/reused verbatim once a valid v2 set exists.
37
+ *
38
+ * Judge: GLM-5.2 via z.ai — the INSTRUMENT only. It never picks candidates;
39
+ * retrieve() (LLM-free) does. A dedicated raw HTTP caller (NOT `LlmClient`) is
40
+ * used here on purpose: `LlmClient.completeReflect` bakes in a
41
+ * nightly-tolerant retry policy (30s/60s/120s, unlimited rate-limit patience)
42
+ * that conflicts with §6b's specific real-time batch policy (5/10/20/40/80s +
43
+ * jitter, 5 retries, a hard call budget). Implemented directly here rather
44
+ * than adding a second retry mode to `llm.ts` (out of scope for this eval,
45
+ * and another agent is concurrently working elsewhere in this repo).
46
+ *
47
+ * Honesty invariants (non-negotiable — mirror recall-sweep.ts + spec §7):
48
+ * - Snapshot opened READONLY via openSnapshot — never initDb.
49
+ * - noStrengthen: true on every retrieve() call.
50
+ * - Real bge-small-en-v1.5 embedder, embed-once + queryEmbedding reuse.
51
+ * - neverCalledEmbed self-check ABORTS the run if retrieve() ignores
52
+ * queryEmbedding (would invalidate every measured number).
53
+ * - GLM-5.2 is the JUDGE only, real prompts, no synthetic queries.
54
+ * - Production renderer (`formatIndexLine`/`memoryTitle`) imported from
55
+ * `recall-index.ts`, never reimplemented.
56
+ * - judge_error batches/units excluded from every denominator, reported
57
+ * separately.
58
+ *
59
+ * Run:
60
+ * npm run eval:relevance -- <snapshot.db> [prompts.json] [report.md] \
61
+ * [--judge-delay-ms=2000] [--max-calls=900] [--resume] \
62
+ * [--verdicts-json=path] [--verdicts-jsonl=path]
63
+ */
64
+ export {};