@gamaze/hicortex 0.16.0 → 0.16.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -0
- package/dist/capture.d.ts +18 -1
- package/dist/capture.js +3 -2
- package/dist/classify-domains.d.ts +1 -1
- package/dist/classify-domains.js +5 -7
- package/dist/cli.js +10 -2
- package/dist/cluster.d.ts +5 -4
- package/dist/cluster.js +2 -3
- package/dist/consolidate.js +6 -5
- package/dist/db.js +23 -0
- package/dist/dedup.js +1 -1
- package/dist/distiller.js +19 -12
- package/dist/domain-classify.d.ts +1 -1
- package/dist/domain-classify.js +1 -5
- package/dist/eval/relevance-eval.d.ts +64 -0
- package/dist/eval/relevance-eval.js +1954 -0
- package/dist/eval/run-eval.js +0 -1
- package/dist/index.js +3 -3
- package/dist/init.d.ts +165 -0
- package/dist/init.js +283 -57
- package/dist/lessons-context.js +3 -2
- package/dist/mcp-server.js +72 -25
- package/dist/nightly.js +35 -3
- package/dist/nofit.d.ts +1 -1
- package/dist/nofit.js +1 -2
- package/dist/prompts.js +22 -13
- package/dist/recall-index.d.ts +56 -21
- package/dist/recall-index.js +51 -29
- package/dist/retrieval.d.ts +7 -7
- package/dist/retrieval.js +20 -24
- package/dist/schema-prototypes.d.ts +8 -13
- package/dist/schema-prototypes.js +13 -22
- package/dist/seed-lesson.d.ts +1 -1
- package/dist/seed-lesson.js +1 -2
- package/dist/storage.d.ts +9 -12
- package/dist/storage.js +19 -21
- package/dist/types.d.ts +47 -23
- package/domains.example.json +2 -3
- package/hermes-plugin/hicortex/README.md +3 -1
- package/hermes-plugin/hicortex/config.py +33 -2
- package/hermes-plugin/hicortex/plugin.yaml +1 -1
- package/hermes-plugin/hicortex/provider.py +5 -0
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -164,6 +164,7 @@ npx @gamaze/hicortex context edit <name> # Edit a context section in $EDIT
|
|
|
164
164
|
npx @gamaze/hicortex context show --agent <id> # Show a specific agent's resolved context (0.13)
|
|
165
165
|
npx @gamaze/hicortex init --agent-name <name> # Opt in to a per-agent context id (default: unset — shared global context)
|
|
166
166
|
npx @gamaze/hicortex init --agent-name "" # Clear it back to global context
|
|
167
|
+
npx @gamaze/hicortex init --repair-config # Recover from a malformed config.json (see below)
|
|
167
168
|
npx @gamaze/hicortex telemetry # Show exactly what anonymous telemetry sends
|
|
168
169
|
npx @gamaze/hicortex status # Show config, DB stats
|
|
169
170
|
npx @gamaze/hicortex uninstall # Remove CC integration (keeps DB)
|
|
@@ -226,6 +227,7 @@ Config at `~/.hicortex/config.json`. Created by `init`. Key options:
|
|
|
226
227
|
| `recallMinSimilarity` | Relevance floor for index entries (default: 0.55; text-search matches always pass) |
|
|
227
228
|
| `recallReshowTurns` | Turns before an already-shown memory may reappear in the same session (default: 30) |
|
|
228
229
|
| `recallMinPromptChars` | Prompts shorter than this skip the recall index (default: 20) |
|
|
230
|
+
| `recallTitleChars` | Chars of each memory's first line shown in an index entry (default: 150, range 40–400). Raised from 100 on 2026-08-02: with topic-first memory titles, 150 chars carries the subject *and* its claim, where 100 cut the claim mid-sentence. Costs roughly +74 tokens per 6-line block |
|
|
229
231
|
| `sessionIntentWeight` | Blend weight of the session-intent rolling centroid in the recall search vector: `query = (1-w)·prompt + w·centroid` (default: 0.33; set 0 to disable — pure-prompt recall, the kill-switch). The first turn of a session searches with pure prompt and seeds the centroid; subsequent turns blend so recall follows the session's intent instead of being query-literal. The EMA rate (0.4) is a shipped constant, not configurable |
|
|
230
232
|
| `dedupMergeThreshold` | Minimum cosine similarity for `hicortex dedup` to cluster memories as near-duplicates (default: 0.92) |
|
|
231
233
|
| `supersessionMinSimilarity` | Minimum cosine similarity for a nightly supersession candidate pair (default: 0.80) |
|
|
@@ -329,6 +331,13 @@ npm test
|
|
|
329
331
|
|
|
330
332
|
## Troubleshooting
|
|
331
333
|
|
|
334
|
+
**`init` fails with "Refusing to write ~/.hicortex/config.json":** the file exists but is not valid JSON — usually a hand-edit slip (a trailing comma, a truncated write). `init` refuses rather than overwriting it, because overwriting would lose `authToken`, `licenseKey`, `distillApiKey`, and your `domains` list. Two ways out:
|
|
335
|
+
|
|
336
|
+
1. **Preferred — fix the JSON.** The error names the parse failure and its position. Correct it and re-run `init`. Nothing is lost.
|
|
337
|
+
2. **`npx @gamaze/hicortex init --repair-config`.** Moves the broken file to `config.json.corrupt-<timestamp>` and rebuilds from scratch. Nothing is deleted, and it prints the top-level key names it found (names only — never secret values) so you know what to copy back. **This mints a new `authToken`**, so every thin client pointing at this server must be updated or its recall will silently 401 (recall is fail-soft — you will see no error, just no memories).
|
|
338
|
+
|
|
339
|
+
The nightly and the server behave differently on purpose: a malformed config makes them log a warning and run degraded rather than refuse to start, so a broken config never takes recall offline.
|
|
340
|
+
|
|
332
341
|
**Tools not visible to agent (OC):** The plugin auto-adds tools to `tools.allow` on startup. Restart the gateway after install.
|
|
333
342
|
|
|
334
343
|
**OC plugin: "Server unreachable":** The plugin requires a running Hicortex server. Run `npx @gamaze/hicortex init` on the same machine, or set `serverUrl` in the plugin config to point at a remote server.
|
package/dist/capture.d.ts
CHANGED
|
@@ -52,11 +52,17 @@ export interface Segment {
|
|
|
52
52
|
export interface DistillBody {
|
|
53
53
|
text: string;
|
|
54
54
|
source_agent: string;
|
|
55
|
+
/** Stable client UUID (config.json `agentId`). Attribution only. */
|
|
56
|
+
source_agent_id?: string | null;
|
|
57
|
+
/** Client-declared topic/domain of the capturing agent. Provenance only. */
|
|
58
|
+
source_domain?: string | null;
|
|
55
59
|
project: string;
|
|
56
60
|
session_id: string;
|
|
57
61
|
segment_id: string;
|
|
58
62
|
session_date: string;
|
|
59
|
-
|
|
63
|
+
/** 0.16.x: optional/vestigial. The distiller no longer sets it; a legacy
|
|
64
|
+
* client may. Honored if present, else the memory stores NULL. */
|
|
65
|
+
privacy?: string;
|
|
60
66
|
}
|
|
61
67
|
/** Normalized POST result the caller's transport returns. */
|
|
62
68
|
export interface PostResult {
|
|
@@ -73,6 +79,17 @@ export interface CaptureOptions {
|
|
|
73
79
|
dryRun?: boolean;
|
|
74
80
|
/** Segment size cap; defaults to SEGMENT_MAX_CHARS. Lowered in tests. */
|
|
75
81
|
segmentMaxChars?: number;
|
|
82
|
+
/**
|
|
83
|
+
* Per-client attribution UUID (config.json `agentId`). Sent on every
|
|
84
|
+
* segment as `source_agent_id`. Null when the client has no `agentId`
|
|
85
|
+
* (e.g. a pre-0.16.x config that has not re-run init).
|
|
86
|
+
*/
|
|
87
|
+
sourceAgentId?: string | null;
|
|
88
|
+
/**
|
|
89
|
+
* Per-client declared topic/domain (config.json `sourceDomain`). Sent as
|
|
90
|
+
* `source_domain` provenance. Null when undeclared.
|
|
91
|
+
*/
|
|
92
|
+
sourceDomain?: string | null;
|
|
76
93
|
}
|
|
77
94
|
export interface CaptureResult {
|
|
78
95
|
memoriesIngested: number;
|
package/dist/capture.js
CHANGED
|
@@ -134,7 +134,7 @@ function packSegments(entries, startCursor, entryCursors, maxChars = exports.SEG
|
|
|
134
134
|
* boundary) while other sessions continue. A 429/401 stops the whole loop.
|
|
135
135
|
*/
|
|
136
136
|
async function captureBatches(batches, opts) {
|
|
137
|
-
const { post, cursorStore, dryRun = false, segmentMaxChars = exports.SEGMENT_MAX_CHARS } = opts;
|
|
137
|
+
const { post, cursorStore, dryRun = false, segmentMaxChars = exports.SEGMENT_MAX_CHARS, sourceAgentId, sourceDomain } = opts;
|
|
138
138
|
let memoriesIngested = 0;
|
|
139
139
|
let sessionsSent = 0;
|
|
140
140
|
let hadTransientFailure = false;
|
|
@@ -186,11 +186,12 @@ async function captureBatches(batches, opts) {
|
|
|
186
186
|
const body = {
|
|
187
187
|
text: seg.text,
|
|
188
188
|
source_agent: batch.sourceAgent ?? `claude-code/${batch.projectName}`,
|
|
189
|
+
source_agent_id: sourceAgentId ?? null,
|
|
190
|
+
source_domain: sourceDomain ?? null,
|
|
189
191
|
project: batch.projectName,
|
|
190
192
|
session_id: batch.sessionId,
|
|
191
193
|
segment_id: `${genPrefix}${seg.segStart}-${seg.segEnd}${seg.idSuffix}`,
|
|
192
194
|
session_date: batch.date,
|
|
193
|
-
privacy: "WORK",
|
|
194
195
|
};
|
|
195
196
|
let result;
|
|
196
197
|
try {
|
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
* reflect tier), same as the nightly. The LLM emits ONLY the ordered tag
|
|
17
17
|
* set; per-tag weights come from the domain prototypes (computed once at
|
|
18
18
|
* run start) and the PRIMARY (memories.domain) is derived (argmax weight,
|
|
19
|
-
*
|
|
19
|
+
* LLM order breaking ties) inside
|
|
20
20
|
* storage.setMemoryTags. After a completed (non-aborted) run the
|
|
21
21
|
* prototypes, all weights, and all primaries are recomputed from the
|
|
22
22
|
* final tag sets — same reconsolidation pass as the nightly.
|
package/dist/classify-domains.js
CHANGED
|
@@ -17,7 +17,7 @@
|
|
|
17
17
|
* reflect tier), same as the nightly. The LLM emits ONLY the ordered tag
|
|
18
18
|
* set; per-tag weights come from the domain prototypes (computed once at
|
|
19
19
|
* run start) and the PRIMARY (memories.domain) is derived (argmax weight,
|
|
20
|
-
*
|
|
20
|
+
* LLM order breaking ties) inside
|
|
21
21
|
* storage.setMemoryTags. After a completed (non-aborted) run the
|
|
22
22
|
* prototypes, all weights, and all primaries are recomputed from the
|
|
23
23
|
* final tag sets — same reconsolidation pass as the nightly.
|
|
@@ -195,7 +195,6 @@ async function runClassifyDomains(options = {}) {
|
|
|
195
195
|
// Prototypes once at run start — newly classified memories get their
|
|
196
196
|
// weights from these; the post-run reconsolidation pass refreshes
|
|
197
197
|
// everything from the final tag sets.
|
|
198
|
-
const compartments = (0, schema_prototypes_js_1.compartmentSet)(domains);
|
|
199
198
|
const { prototypes } = await (0, schema_prototypes_js_1.computeDomainPrototypes)(db, domains, getEmbedFn);
|
|
200
199
|
// Scope filter: default = NULL / not-in-set / no tags yet; --all = everything.
|
|
201
200
|
const placeholders = domains.map(() => "?").join(", ");
|
|
@@ -255,10 +254,9 @@ async function runClassifyDomains(options = {}) {
|
|
|
255
254
|
continue;
|
|
256
255
|
}
|
|
257
256
|
// Derived primary (argmax weight from the run-start prototypes,
|
|
258
|
-
//
|
|
259
|
-
// setMemoryTags will write below.
|
|
257
|
+
// LLM order breaking ties) — the same value setMemoryTags writes below.
|
|
260
258
|
const weights = (0, schema_prototypes_js_1.computeTagWeights)(db, row.id, result.tags, prototypes);
|
|
261
|
-
const derived = (0, schema_prototypes_js_1.derivePrimary)(result.tags.map((tag) => ({ tag, weight: weights[tag] ?? null }))
|
|
259
|
+
const derived = (0, schema_prototypes_js_1.derivePrimary)(result.tags.map((tag) => ({ tag, weight: weights[tag] ?? null })));
|
|
262
260
|
if (derived === row.domain) {
|
|
263
261
|
batchUnchanged++;
|
|
264
262
|
}
|
|
@@ -273,10 +271,10 @@ async function runClassifyDomains(options = {}) {
|
|
|
273
271
|
const tx = db.transaction(() => {
|
|
274
272
|
for (const w of writes) {
|
|
275
273
|
if (w.kind === "tags") {
|
|
276
|
-
storage.setMemoryTags(db, w.id, w.tags, { weights: w.weights
|
|
274
|
+
storage.setMemoryTags(db, w.id, w.tags, { weights: w.weights });
|
|
277
275
|
}
|
|
278
276
|
else if (w.resolution.kind === "weak_primary") {
|
|
279
|
-
(0, nofit_js_1.applyWeakPrimary)(db, w.id, w.resolution.domain, w.resolution.weight
|
|
277
|
+
(0, nofit_js_1.applyWeakPrimary)(db, w.id, w.resolution.domain, w.resolution.weight);
|
|
280
278
|
}
|
|
281
279
|
else {
|
|
282
280
|
(0, nofit_js_1.applyNoAssociationDecay)(db, w.id);
|
package/dist/cli.js
CHANGED
|
@@ -43,9 +43,14 @@ switch (command) {
|
|
|
43
43
|
console.error("[hicortex] init: --agent-name requires a value, e.g. --agent-name lenovo");
|
|
44
44
|
process.exit(1);
|
|
45
45
|
}
|
|
46
|
+
const repairConfig = process.argv.includes("--repair-config");
|
|
46
47
|
import("./init.js").then(({ runInit }) => {
|
|
47
|
-
runInit({ serverUrl, agentName }).catch((err) => {
|
|
48
|
-
|
|
48
|
+
runInit({ serverUrl, agentName, repairConfig }).catch((err) => {
|
|
49
|
+
// Operator-fixable failures (a malformed config.json) carry a complete,
|
|
50
|
+
// actionable message — print that alone. A stack trace would bury it.
|
|
51
|
+
// Anything else is a real bug and gets the full error object.
|
|
52
|
+
const operatorFixable = err instanceof Error && /^Refusing to (read|write) /.test(err.message);
|
|
53
|
+
console.error("[hicortex] Init failed:", operatorFixable ? err.message : err);
|
|
49
54
|
process.exit(1);
|
|
50
55
|
});
|
|
51
56
|
});
|
|
@@ -250,6 +255,9 @@ Commands:
|
|
|
250
255
|
init --server <url> Set up as client (remote server)
|
|
251
256
|
init --agent-name <name> Opt in to a per-agent context id (default: unset — shared global context)
|
|
252
257
|
Pass --agent-name "" to clear it back to global
|
|
258
|
+
init --repair-config Recover from a malformed ~/.hicortex/config.json: move it to
|
|
259
|
+
config.json.corrupt-<timestamp> and rebuild. Nothing is deleted.
|
|
260
|
+
Mints a NEW authToken — every thin client must be updated.
|
|
253
261
|
nightly Run nightly denoise + capture + consolidate
|
|
254
262
|
relink Resumable link-discovery pass over the ENTIRE corpus (server mode)
|
|
255
263
|
dedup Cluster + merge near-duplicate memories (server mode; dry run by default)
|
package/dist/cluster.d.ts
CHANGED
|
@@ -25,18 +25,19 @@ export declare class UnionFind {
|
|
|
25
25
|
export declare function clusterEdges(edges: Edge[], threshold: number): string[][];
|
|
26
26
|
/** Excess = sum(cluster size − 1) — rows that would disappear if every cluster merged to one. */
|
|
27
27
|
export declare function clusterExcess(clusters: string[][]): number;
|
|
28
|
-
/**
|
|
28
|
+
/** Metadata fields a merge candidate cluster must agree on. `privacy` is on the row (the
|
|
29
|
+
* column still exists) but is NOT a merge-safety field — vestigial since 0.16.2. */
|
|
29
30
|
export interface ClusterMetaRow {
|
|
30
31
|
project: string | null;
|
|
31
|
-
privacy: string;
|
|
32
|
+
privacy: string | null;
|
|
32
33
|
source_agent: string;
|
|
33
34
|
}
|
|
34
35
|
export interface ClusterMetadataMismatch {
|
|
35
36
|
projectMismatch: boolean;
|
|
36
|
-
privacyMismatch: boolean;
|
|
37
37
|
sourceAgentMismatch: boolean;
|
|
38
38
|
}
|
|
39
|
-
/** Do cluster members disagree on project/
|
|
39
|
+
/** Do cluster members disagree on project / source_agent? (merge-safety input for #100).
|
|
40
|
+
* Privacy is intentionally NOT checked — it is vestigial since 0.16.2. */
|
|
40
41
|
export declare function clusterMetadataMismatch(members: ClusterMetaRow[]): ClusterMetadataMismatch;
|
|
41
42
|
/**
|
|
42
43
|
* Build the max-cosine edge set via top-K KNN on `memory_vectors`, keeping
|
package/dist/cluster.js
CHANGED
|
@@ -69,14 +69,13 @@ function clusterEdges(edges, threshold) {
|
|
|
69
69
|
function clusterExcess(clusters) {
|
|
70
70
|
return clusters.reduce((sum, c) => sum + (c.length - 1), 0);
|
|
71
71
|
}
|
|
72
|
-
/** Do cluster members disagree on project/
|
|
72
|
+
/** Do cluster members disagree on project / source_agent? (merge-safety input for #100).
|
|
73
|
+
* Privacy is intentionally NOT checked — it is vestigial since 0.16.2. */
|
|
73
74
|
function clusterMetadataMismatch(members) {
|
|
74
75
|
const projects = new Set(members.map((m) => m.project ?? "\u0000null"));
|
|
75
|
-
const privacies = new Set(members.map((m) => m.privacy));
|
|
76
76
|
const agents = new Set(members.map((m) => m.source_agent));
|
|
77
77
|
return {
|
|
78
78
|
projectMismatch: projects.size > 1,
|
|
79
|
-
privacyMismatch: privacies.size > 1,
|
|
80
79
|
sourceAgentMismatch: agents.size > 1,
|
|
81
80
|
};
|
|
82
81
|
}
|
package/dist/consolidate.js
CHANGED
|
@@ -307,7 +307,10 @@ async function stageReflection(db, memories, llm, budget, embedFn, dryRun) {
|
|
|
307
307
|
const severity = String(lo.severity ?? "important");
|
|
308
308
|
const confidence = String(lo.confidence ?? "medium");
|
|
309
309
|
const sourcePattern = String(lo.source_pattern ?? "");
|
|
310
|
-
|
|
310
|
+
// No `## Lesson:` prefix: memory_type='lesson' carries the type, and the
|
|
311
|
+
// text is the topic-first first line (display reads the first line, not a
|
|
312
|
+
// header parse — see lessons-context.ts / index.ts).
|
|
313
|
+
let content = `${lessonText}\n\n`;
|
|
311
314
|
content += `**Type:** ${lessonType}\n`;
|
|
312
315
|
content += `**Severity:** ${severity}\n`;
|
|
313
316
|
content += `**Confidence:** ${confidence}\n`;
|
|
@@ -353,7 +356,6 @@ async function stageReflection(db, memories, llm, budget, embedFn, dryRun) {
|
|
|
353
356
|
project,
|
|
354
357
|
memoryType: "lesson",
|
|
355
358
|
baseStrength: baseStrength[severity] ?? 0.8,
|
|
356
|
-
privacy: "WORK",
|
|
357
359
|
});
|
|
358
360
|
generated++;
|
|
359
361
|
}
|
|
@@ -433,7 +435,6 @@ async function stageContentDomains(db, domains, llm, budget, embedFn, dryRun, st
|
|
|
433
435
|
return { curated: false, domains: domains.length, classified: 0, reason: `dry_run (${rows.length} would classify)` };
|
|
434
436
|
}
|
|
435
437
|
const getEmbedFn = async () => embedFn;
|
|
436
|
-
const compartments = (0, schema_prototypes_js_1.compartmentSet)(domains);
|
|
437
438
|
let classified = 0;
|
|
438
439
|
let weakPrimary = 0;
|
|
439
440
|
let noAssociationDecayed = 0;
|
|
@@ -462,7 +463,7 @@ async function stageContentDomains(db, domains, llm, budget, embedFn, dryRun, st
|
|
|
462
463
|
// double-halves.
|
|
463
464
|
const resolution = (0, nofit_js_1.resolveNoFit)(db, row.id, domains, startPrototypes, weakPrimaryFloor);
|
|
464
465
|
if (resolution.kind === "weak_primary") {
|
|
465
|
-
(0, nofit_js_1.applyWeakPrimary)(db, row.id, resolution.domain, resolution.weight
|
|
466
|
+
(0, nofit_js_1.applyWeakPrimary)(db, row.id, resolution.domain, resolution.weight);
|
|
466
467
|
weakPrimary++;
|
|
467
468
|
}
|
|
468
469
|
else {
|
|
@@ -472,7 +473,7 @@ async function stageContentDomains(db, domains, llm, budget, embedFn, dryRun, st
|
|
|
472
473
|
continue;
|
|
473
474
|
}
|
|
474
475
|
const weights = (0, schema_prototypes_js_1.computeTagWeights)(db, row.id, result.tags, startPrototypes);
|
|
475
|
-
storage.setMemoryTags(db, row.id, result.tags, { weights
|
|
476
|
+
storage.setMemoryTags(db, row.id, result.tags, { weights });
|
|
476
477
|
classified++;
|
|
477
478
|
}
|
|
478
479
|
}
|
package/dist/db.js
CHANGED
|
@@ -441,6 +441,29 @@ const MIGRATIONS = [
|
|
|
441
441
|
`);
|
|
442
442
|
},
|
|
443
443
|
},
|
|
444
|
+
{
|
|
445
|
+
version: 11,
|
|
446
|
+
name: "add_source_attribution",
|
|
447
|
+
up: (db) => {
|
|
448
|
+
// 0.16.x attribution + provenance. Two nullable columns on `memories`,
|
|
449
|
+
// both populated ONLY by capture (/distill) from client-declared values;
|
|
450
|
+
// nothing filters, scopes, or scores on either (attribution + echo).
|
|
451
|
+
//
|
|
452
|
+
// `source_agent_id`: the capturing client's stable UUID (config.json
|
|
453
|
+
// `agentId`, generated once by init). Survives agent/machine renames
|
|
454
|
+
// — unlike `source_agent`, a readable name. NULL on legacy rows.
|
|
455
|
+
// `source_domain`: the client-declared topic/domain of the capturing
|
|
456
|
+
// agent (config.json `domain`). Distinct from the content-classified
|
|
457
|
+
// `domain` column (which stays the LLM/prototype-derived primary).
|
|
458
|
+
// Guarded with hasColumn for idempotency across partially-migrated DBs.
|
|
459
|
+
if (!hasColumn(db, "memories", "source_agent_id")) {
|
|
460
|
+
db.exec("ALTER TABLE memories ADD COLUMN source_agent_id TEXT");
|
|
461
|
+
}
|
|
462
|
+
if (!hasColumn(db, "memories", "source_domain")) {
|
|
463
|
+
db.exec("ALTER TABLE memories ADD COLUMN source_domain TEXT");
|
|
464
|
+
}
|
|
465
|
+
},
|
|
466
|
+
},
|
|
444
467
|
];
|
|
445
468
|
/**
|
|
446
469
|
* Run all pending migrations against the database.
|
package/dist/dedup.js
CHANGED
|
@@ -287,7 +287,7 @@ async function runDedup(options = {}) {
|
|
|
287
287
|
if (members.length < 2)
|
|
288
288
|
continue; // defensive — a member vanished between KNN and load
|
|
289
289
|
const mismatch = (0, cluster_js_1.clusterMetadataMismatch)(members);
|
|
290
|
-
if (mismatch.projectMismatch || mismatch.
|
|
290
|
+
if (mismatch.projectMismatch || mismatch.sourceAgentMismatch) {
|
|
291
291
|
mismatchSkipped.push({ size: members.length, memberIds: members.map((m) => m.id), mismatch });
|
|
292
292
|
continue;
|
|
293
293
|
}
|
package/dist/distiller.js
CHANGED
|
@@ -314,6 +314,14 @@ async function distillChunk(llm, transcript, projectName, date) {
|
|
|
314
314
|
return { entries: [], dropped: [] };
|
|
315
315
|
}
|
|
316
316
|
const parsed = parseDistilledEntries(result);
|
|
317
|
+
// Smoke alarm (PR #218 review): the prompt enforces topic-first, but models
|
|
318
|
+
// sometimes ignore constraints (cf. the prior max-15-bullet failure). Count
|
|
319
|
+
// entries that still look actor-led or bracket-led so a format regression
|
|
320
|
+
// shows in nightly logs, not months later in the next eval. Non-blocking.
|
|
321
|
+
const offTopic = parsed.filter((e) => /^\s*(user|ai|the user|assistant)\b/i.test(e) || /^\s*\[/.test(e)).length;
|
|
322
|
+
if (parsed.length > 0 && offTopic > 0) {
|
|
323
|
+
console.log(`[hicortex] topic-first check: ${offTopic}/${parsed.length} entries look actor/bracket-led (prompt may be ignored)`);
|
|
324
|
+
}
|
|
317
325
|
const entries = [];
|
|
318
326
|
const dropped = [];
|
|
319
327
|
for (const entry of parsed) {
|
|
@@ -409,23 +417,22 @@ function hasMinimalSubstance(entry) {
|
|
|
409
417
|
function parseDistilledEntries(markdown) {
|
|
410
418
|
const entries = [];
|
|
411
419
|
const lines = markdown.split("\n");
|
|
412
|
-
let currentSection = "";
|
|
413
420
|
for (const line of lines) {
|
|
414
421
|
const trimmed = line.trim();
|
|
415
|
-
//
|
|
416
|
-
|
|
417
|
-
|
|
422
|
+
// Skip all markdown headers (session title, section headings). Sections
|
|
423
|
+
// are NOT prefixed onto entries: each bullet already
|
|
424
|
+
// starts with its [SUBJECT] (topic-first, enforced by prompts.ts), and
|
|
425
|
+
// prepending "[Section]" re-introduced the category-first prefix the
|
|
426
|
+
// 2026-08-02 corpus rewrite removed. The section label is unused
|
|
427
|
+
// downstream (distilled memories all store memory_type='episode').
|
|
428
|
+
if (trimmed.startsWith("# ") ||
|
|
429
|
+
trimmed.startsWith("## ") ||
|
|
430
|
+
trimmed.startsWith("### ")) {
|
|
418
431
|
continue;
|
|
419
432
|
}
|
|
420
|
-
//
|
|
421
|
-
if (trimmed.startsWith("# ") || trimmed.startsWith("## "))
|
|
422
|
-
continue;
|
|
423
|
-
// Bullet items are individual memories
|
|
433
|
+
// Bullet items are individual, already topic-first memories.
|
|
424
434
|
if (trimmed.startsWith("- ") && trimmed.length > 5) {
|
|
425
|
-
|
|
426
|
-
? `[${currentSection}] ${trimmed.slice(2)}`
|
|
427
|
-
: trimmed.slice(2);
|
|
428
|
-
entries.push(entry);
|
|
435
|
+
entries.push(trimmed.slice(2));
|
|
429
436
|
}
|
|
430
437
|
}
|
|
431
438
|
return entries;
|
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
* (the order is used solely as an exact-weight tiebreak downstream).
|
|
19
19
|
* The PRIMARY (memories.domain) is NO LONGER requested from the LLM — audits
|
|
20
20
|
* proved LLM primaries a coin-flip on overlapping spheres. It is DERIVED
|
|
21
|
-
* deterministically (argmax association weight
|
|
21
|
+
* deterministically (argmax association weight, LLM tag order breaking ties) in
|
|
22
22
|
* schema-prototypes.ts / storage.setMemoryTags.
|
|
23
23
|
*
|
|
24
24
|
* NO-FIT = EMPTY TAG SET (owner amendment 07.07): "Unsorted" is a non-tag —
|
package/dist/domain-classify.js
CHANGED
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
* (the order is used solely as an exact-weight tiebreak downstream).
|
|
20
20
|
* The PRIMARY (memories.domain) is NO LONGER requested from the LLM — audits
|
|
21
21
|
* proved LLM primaries a coin-flip on overlapping spheres. It is DERIVED
|
|
22
|
-
* deterministically (argmax association weight
|
|
22
|
+
* deterministically (argmax association weight, LLM tag order breaking ties) in
|
|
23
23
|
* schema-prototypes.ts / storage.setMemoryTags.
|
|
24
24
|
*
|
|
25
25
|
* NO-FIT = EMPTY TAG SET (owner amendment 07.07): "Unsorted" is a non-tag —
|
|
@@ -84,10 +84,6 @@ function parseConfigDomains(config) {
|
|
|
84
84
|
if (!name)
|
|
85
85
|
continue;
|
|
86
86
|
const def = { name, description };
|
|
87
|
-
// Compartment policy passthrough (graded-schema spec): a domain flagged
|
|
88
|
-
// `compartment: true` becomes the primary whenever tagged.
|
|
89
|
-
if (d.compartment === true)
|
|
90
|
-
def.compartment = true;
|
|
91
87
|
out.push(def);
|
|
92
88
|
}
|
|
93
89
|
return out.length > 0 ? out : null;
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Real-query relevance + SNIPPET eval — recall QUALITY on real agent prompts.
|
|
4
|
+
*
|
|
5
|
+
* v2 (spec `specs/2026-08-02-relevance-eval.md`) extends the v1 selection-only
|
|
6
|
+
* eval with the SNIPPET layer: v1 asked "did retrieve() surface the right
|
|
7
|
+
* memories?" (judge sees up to 2000 chars). Production shows the agent only a
|
|
8
|
+
* ~100-char one-liner (`recall-index.ts#memoryTitle`), so a memory can be
|
|
9
|
+
* genuinely relevant while its rendered line is useless — v1 scored that as a
|
|
10
|
+
* win. v2 grades BOTH: `full_verdict` (selection quality, judge sees full
|
|
11
|
+
* content) and `line_verdict` (snippet quality, judge sees ONLY the rendered
|
|
12
|
+
* production one-liner — imported from `recall-index.ts`, never reimplemented).
|
|
13
|
+
*
|
|
14
|
+
* v2 additions (spec §4, §5, §6, §6b) layered onto the v1 base (prompt
|
|
15
|
+
* sampling, readonly snapshot handling, embed-once + neverCalledEmbed,
|
|
16
|
+
* lenient JSON parse, distribution/CI reporting):
|
|
17
|
+
* 1. Dual verdict per surfaced memory — TWO separate, blind judge calls.
|
|
18
|
+
* 2. Snippet-length sweep (100/200/300/title+1st-sentence) on a fixed
|
|
19
|
+
* 40-prompt subset (8 per source).
|
|
20
|
+
* 3. Similarity-floor + retrieval-source analysis (near-free — logged, not
|
|
21
|
+
* re-judged).
|
|
22
|
+
* 4. Token-cost estimate (char/4) per K and per snippet-length variant.
|
|
23
|
+
* 5. ~20 rendered ACTUAL production blocks dumped into the report.
|
|
24
|
+
* 6. Redundancy — one set-level judge call per (prompt × mode) over the
|
|
25
|
+
* production 6.
|
|
26
|
+
* 7. Rate limiting + resumability (§6b, MANDATORY): serial calls,
|
|
27
|
+
* `--judge-delay-ms` (default 2000), exponential backoff with jitter on
|
|
28
|
+
* 429/5xx/timeout (5→10→20→40→80s, max 5 retries, respects
|
|
29
|
+
* `Retry-After`), checkpoint-per-call to a `.jsonl` sidecar, `--resume`,
|
|
30
|
+
* progress logging, 10%-error-rate abort, `--max-calls` budget guard
|
|
31
|
+
* (default 900).
|
|
32
|
+
*
|
|
33
|
+
* Prompt corpus (spec §2, owner decision §11.1): EVEN split, 20 prompts per
|
|
34
|
+
* source × 5 sources — Hermes (lenny, raider, nano) + CC (the DevOps
|
|
35
|
+
* `infrastructure` project, the `aironic-marine` project). Saved to
|
|
36
|
+
* `data/prompts.json`, stable/reused verbatim once a valid v2 set exists.
|
|
37
|
+
*
|
|
38
|
+
* Judge: GLM-5.2 via z.ai — the INSTRUMENT only. It never picks candidates;
|
|
39
|
+
* retrieve() (LLM-free) does. A dedicated raw HTTP caller (NOT `LlmClient`) is
|
|
40
|
+
* used here on purpose: `LlmClient.completeReflect` bakes in a
|
|
41
|
+
* nightly-tolerant retry policy (30s/60s/120s, unlimited rate-limit patience)
|
|
42
|
+
* that conflicts with §6b's specific real-time batch policy (5/10/20/40/80s +
|
|
43
|
+
* jitter, 5 retries, a hard call budget). Implemented directly here rather
|
|
44
|
+
* than adding a second retry mode to `llm.ts` (out of scope for this eval,
|
|
45
|
+
* and another agent is concurrently working elsewhere in this repo).
|
|
46
|
+
*
|
|
47
|
+
* Honesty invariants (non-negotiable — mirror recall-sweep.ts + spec §7):
|
|
48
|
+
* - Snapshot opened READONLY via openSnapshot — never initDb.
|
|
49
|
+
* - noStrengthen: true on every retrieve() call.
|
|
50
|
+
* - Real bge-small-en-v1.5 embedder, embed-once + queryEmbedding reuse.
|
|
51
|
+
* - neverCalledEmbed self-check ABORTS the run if retrieve() ignores
|
|
52
|
+
* queryEmbedding (would invalidate every measured number).
|
|
53
|
+
* - GLM-5.2 is the JUDGE only, real prompts, no synthetic queries.
|
|
54
|
+
* - Production renderer (`formatIndexLine`/`memoryTitle`) imported from
|
|
55
|
+
* `recall-index.ts`, never reimplemented.
|
|
56
|
+
* - judge_error batches/units excluded from every denominator, reported
|
|
57
|
+
* separately.
|
|
58
|
+
*
|
|
59
|
+
* Run:
|
|
60
|
+
* npm run eval:relevance -- <snapshot.db> [prompts.json] [report.md] \
|
|
61
|
+
* [--judge-delay-ms=2000] [--max-calls=900] [--resume] \
|
|
62
|
+
* [--verdicts-json=path] [--verdicts-jsonl=path]
|
|
63
|
+
*/
|
|
64
|
+
export {};
|