memtrails 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -3,6 +3,22 @@
3
3
  All notable changes are listed here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and
4
4
  versions follow [Semantic Versioning](https://semver.org/).
5
5
 
6
+ ## [0.3.0] — 2026-11-02
7
+
8
+ - `gather(q, { notes: true })`: the same model call also writes what the kept memories say that answers the question,
9
+ every instance with its date and only what they state, reading all 80 candidates at once. Shown before the
10
+ memories, LoCoMo 79.5 → 80.9 (Mem0 76.5);
11
+ `bench/locomo.ts --gather 40 --notes` reproduces it.
12
+
13
+ ## [0.2.0] — 2026-11-01
14
+
15
+ - Consolidation reads "last week" as the seven days before it was said, and the calendar line names the last weekend
16
+ and each of the six days before, so "on Friday" is dated without counting. Dates had landed a week early.
17
+ - Observations carry 2 to 4 cues (questions they answer, in the asker's words); search reads cues as their own channel
18
+ at a tenth of the dense weight. LoCoMo, all 1,307 held-out questions: multi-hop 50.6 → 55.2, all 77.2 → 77.4.
19
+ - `gather` reads as many as `limit` asks for (it always read 200). At 40, for any question, the rooms it builds score
20
+ 79.5 on LoCoMo against Mem0's 76.5 (+110/−71 per question); `bench/locomo.ts --gather 40` reproduces it.
21
+
6
22
  ## [0.1.0] — 2026-10-31
7
23
 
8
24
  First public release.
package/README.md CHANGED
@@ -40,7 +40,8 @@ await memory.close();
40
40
  stored at one bit a number): on 2,011 benchmark questions the answer reached the reader in 53.1% of top-5 lists
41
41
  against 50.3% for plain search, and most of a cross-encoder reranker's gain, in about 3 ms on one CPU thread.
42
42
  - **Measured, honestly.** On all 1,307 held-out LoCoMo questions with 6,000 characters to read, memtrails alone on a
43
- CPU scores 77.2 (strict judge) against Mem0's 76.5 — a tie, reported as one. [Benchmarks](docs/BENCHMARKS.md).
43
+ CPU scores 77.4 (strict judge) against Mem0's 76.5 — a tie, reported as one; with `gather` reading what search
44
+ found and noting what it says, 80.9 — ahead beyond chance. [Benchmarks](docs/BENCHMARKS.md).
44
45
  - **Explorable.** A concept graph links what records share — people, places, systems, names — so an agent can walk
45
46
  from what it found to what a search for the words does not find.
46
47
  - **Built for agents.** `memory.tools()` gives `memory_search`, `memory_open` and `memory_explore` with zod and JSON
@@ -188,13 +189,18 @@ the [API reference](docs/API.md), and [what was tried and why](docs/RESEARCH.md)
188
189
 
189
190
  | | memtrails | Mem0 | Graphiti |
190
191
  |---|---|---|---|
191
- | LoCoMo, all 1,307 held-out questions, 6,000 characters, strict judge | **77.2** (no server) | 76.5 | — |
192
- | the same, Mem0's own lenient judge | **89.3** | 87.8 | — |
192
+ | LoCoMo, all 1,307 held-out questions, 6,000 characters, strict judge, `gather` at 40 with notes | **80.9** | 76.5 | — |
193
+ | the same, Mem0's own lenient judge | **91.9** | 87.8 | — |
194
+ | the same, `gather` without notes, strict / lenient | 79.5 / 91.4 | 76.5 / 87.8 | — |
195
+ | the same with `search` alone, no model when searching, strict / lenient | 77.4 / 89.3 | 76.5 / 87.8 | — |
193
196
  | LoCoMo, 1,307 held-out questions, 28,000 characters, one search¹ | 79.9 | 80.6 | 42.1 |
194
197
  | LoCoMo, the same with an agent using each system's own tools¹ | **79.0** | 77.1 | 47.0 |
195
198
  | LongMemEval-S retrieval, strict all sessions @5 (467)² | **96.15** | — | — |
196
199
 
197
- The same reader and judges for every system. The small-room and one-search comparisons are ties; the agent lead is
200
+ The same reader and judges for every system. With `gather` (a model reads what search found, keeps what the answer
201
+ needs and notes what it says) memtrails leads Mem0 beyond chance: +113/−55 questions strict, +78/−25 lenient (both
202
+ p < 0.001). With `search` alone,
203
+ the small-room and one-search comparisons are ties; the agent lead is
198
204
  small and was judged once. ¹ An earlier build (2026-10-27). ² gbrain-evals publishes 95.96 on the same count: a tie. [Full results, protocol and limitations](docs/BENCHMARKS.md).
199
205
 
200
206
  ## Limitations
@@ -146,7 +146,7 @@ function observationItem(o, src) {
146
146
  engine: src.engine, session: src.session, cwd: src.cwd, where: src.title, ts: src.ts, firstTs: src.ts,
147
147
  title: cut(o.text, 100), summary: o.text, body: '', files: [], appliesWhen: '', confidence: 'stated', importance: 2,
148
148
  status: 'active', provenance: o.provenance, seen: 1, fingerprint: fingerprint('observation', o.text, o.eventAt),
149
- cues: [], concepts: o.concepts, ...(o.eventAt ? { eventAt: o.eventAt } : {}),
149
+ cues: o.cues ?? [], concepts: o.concepts, ...(o.eventAt ? { eventAt: o.eventAt } : {}),
150
150
  ...(o.startAt ? { startAt: o.startAt } : {}), ...(o.endAt ? { endAt: o.endAt } : {}), phase: o.phase, ...(o.about ? { about: o.about } : {}),
151
151
  ...(o.origin ? { origin: o.origin } : {}),
152
152
  };
@@ -23,8 +23,10 @@ export declare const isConversation: (e: {
23
23
  }) => boolean;
24
24
  /** A record's body: what its key summarises, its chunks are cut from, and opening it shows — whole, never shortened. */
25
25
  export declare function episodeBody(e: Pick<ReadableRequest, 'request' | 'reply' | 'files'> & Partial<Pick<ReadableRequest, 'said' | 'steps' | 'engine'>>): string;
26
- /** For each day a request was made, the days a time said relatively that day means — yesterday, this week, last week,
27
- * last month, last year (weeks start on Monday, UTC as the requests' times) — so the model reads "last week" off a line
28
- * instead of counting: measured on LoCoMo, "last week" said on Thursday 6 July was written as the week of 3 July. */
26
+ /** For each day a request was made, the days a time said relatively that day means — yesterday, the week before, the
27
+ * last weekend, each of the six days before by name, this week, last month, last year (UTC as the requests' times) —
28
+ * so the model reads "last week" or "on Friday" off a line instead of counting. "Last week" is the seven days before,
29
+ * as people mean it, not the Monday-to-Sunday week before this one: measured on LoCoMo, the calendar week put events
30
+ * said on a Sunday thirteen days back, a week early. */
29
31
  export declare function calendarOf(tss: number[]): string;
30
32
  export declare function renderForExtraction(e: ReadableRequest, n: number): string;
@@ -17,19 +17,23 @@ export function episodeBody(e) {
17
17
  parts.push(`Files: ${e.files.join(', ')}`);
18
18
  return parts.join('\n');
19
19
  }
20
- /** For each day a request was made, the days a time said relatively that day means — yesterday, this week, last week,
21
- * last month, last year (weeks start on Monday, UTC as the requests' times) — so the model reads "last week" off a line
22
- * instead of counting: measured on LoCoMo, "last week" said on Thursday 6 July was written as the week of 3 July. */
20
+ /** For each day a request was made, the days a time said relatively that day means — yesterday, the week before, the
21
+ * last weekend, each of the six days before by name, this week, last month, last year (UTC as the requests' times) —
22
+ * so the model reads "last week" or "on Friday" off a line instead of counting. "Last week" is the seven days before,
23
+ * as people mean it, not the Monday-to-Sunday week before this one: measured on LoCoMo, the calendar week put events
24
+ * said on a Sunday thirteen days back, a week early. */
23
25
  export function calendarOf(tss) {
24
26
  const DAY = 86_400_000;
25
27
  const iso = (t) => new Date(t).toISOString().slice(0, 10);
26
28
  const wd = (t) => new Date(t).toLocaleDateString('en-GB', { weekday: 'short', timeZone: 'UTC' });
27
29
  const days = [...new Set(tss.filter((t) => t > 0).map((t) => Math.floor(t / DAY) * DAY))].sort((a, b) => a - b);
28
30
  return days.map((d) => {
29
- const monday = d - ((new Date(d).getUTCDay() + 6) % 7) * DAY;
30
31
  const dt = new Date(d);
32
+ const monday = d - ((dt.getUTCDay() + 6) % 7) * DAY;
33
+ const sunday = d - (dt.getUTCDay() === 0 ? 7 : dt.getUTCDay()) * DAY;
31
34
  const lastMonth = new Date(Date.UTC(dt.getUTCFullYear(), dt.getUTCMonth() - 1, 1)).toISOString().slice(0, 7);
32
- return `- ${wd(d)} ${iso(d)}: yesterday ${wd(d - DAY)} ${iso(d - DAY)} · this week ${iso(monday)} to ${iso(monday + 6 * DAY)} · last week ${iso(monday - 7 * DAY)} to ${iso(monday - DAY)} · last month ${lastMonth} · last year ${dt.getUTCFullYear() - 1}`;
35
+ const before = [2, 3, 4, 5, 6, 7].map((n) => `${wd(d - n * DAY)} ${iso(d - n * DAY)}`).join(', ');
36
+ return `- ${wd(d)} ${iso(d)}: yesterday ${wd(d - DAY)} ${iso(d - DAY)} · the days before ${before} · last week (the week before) ${iso(d - 7 * DAY)} to ${iso(d - DAY)} · last weekend ${iso(sunday - DAY)} to ${iso(sunday)} · this week ${iso(monday)} to ${iso(monday + 6 * DAY)} · last month ${lastMonth} · last year ${dt.getUTCFullYear() - 1}`;
33
37
  }).join('\n');
34
38
  }
35
39
  export function renderForExtraction(e, n) {
@@ -41,6 +41,8 @@ export interface Observation {
41
41
  /** Episode uids it was told in. */
42
42
  provenance: string[];
43
43
  concepts: string[];
44
+ /** Questions it answers, in the asker's words (searched beside its text, quieter). */
45
+ cues: string[];
44
46
  origin?: MemoryOrigin;
45
47
  }
46
48
  export interface KeptEpisode {
@@ -72,6 +74,7 @@ type RawObservation = {
72
74
  phase?: string;
73
75
  requests?: number[];
74
76
  concepts?: unknown;
77
+ cues?: unknown;
75
78
  };
76
79
  /** The observations as memory keeps them: with text, from a request of this part, scrubbed; a date only when it is one. */
77
80
  export declare function cleanObservations(raw: RawObservation[], uidOf: (n: number) => string | undefined): Observation[];
@@ -42,9 +42,9 @@ what they tell about themselves counts as much. For each:
42
42
  - text: 10 to 60 words that stand alone — the event or fact itself, as told ("Melanie took her family camping in the
43
43
  mountains"), never "On <date>, X said that …" (the date has its own field). Name who and what (names, never "he",
44
44
  "she", "it", "they"); keep every proper noun, title, number and quoted phrase exactly. A time said relatively stays
45
- as said, with the date it means, read off the Calendar line of the day it was said: "last week (the week of 19 June
46
- 2023)", "yesterday (7 May 2023)". When the part names something only by description ("the book you recommended", "my
47
- home country", "the kids") and KNOWN FROM OTHER CONVERSATIONS names it, write the name too ("Becoming Nicole",
45
+ as said, with the date it means, read off the Calendar line of the day it was said: "last week (the week before 26 June
46
+ 2023)", "yesterday (7 May 2023)", "on Friday (12 April 2024)". When the part names something only by description
47
+ ("the book you recommended", "my home country", "the kids") and KNOWN FROM OTHER CONVERSATIONS names it, write the name too ("Becoming Nicole",
48
48
  "Sweden", "her three children") — never a fact from KNOWN that this part does not tell. What the person or a speaker
49
49
  says about their own life is true as told: no commentary on what the conversation did or did not confirm. A plan is
50
50
  written as a plan ("Melanie planned to go swimming with the kids"). When it replaces something said before, say both
@@ -52,7 +52,8 @@ what they tell about themselves counts as much. For each:
52
52
  - about: who or what it is about, named as the conversation names them ("the person" for the one whose requests these
53
53
  are, unless the conversation names them).
54
54
  - event_date: when it happened or applies, resolved with the Calendar line of the day its request was made ("yesterday"
55
- is that line's yesterday; "last week" its last week's first day; "last month" its last month; "in June" is 2023-06)
55
+ is that line's yesterday; "last week" the first day of its week before; "last weekend" its last weekend's Saturday;
56
+ "on Friday" the Friday among its days before; "last month" its last month; "in June" is 2023-06)
56
57
  — YYYY-MM-DD, YYYY-MM or
57
58
  YYYY, as precise as the conversation makes it; empty when no time is given or implied. A plan's event_date is when it
58
59
  is planned for.
@@ -63,7 +64,10 @@ what they tell about themselves counts as much. For each:
63
64
  implies no such time, and for a one-off event (its event_date says it).
64
65
  - phase: done (it happened), plan (intended, not done yet), ongoing, or state (how things are).
65
66
  - requests: the request numbers it comes from.
66
- - concepts: up to 6 specific things it names — people, places, organisations, products, titles, files, services.`;
67
+ - concepts: up to 6 specific things it names — people, places, organisations, products, titles, files, services.
68
+ - cues: 2 to 4 questions someone may ask later that this answers, in the words they would ask in rather than its own —
69
+ from the broad ("What instruments does Priya play?", "Who are Priya's siblings?") to the specific ("When did Priya
70
+ start the cello?"), in the language the conversation is in.`;
67
71
  const SYSTEM = `You consolidate the long-term memory of a person who works with AI agents (coding, research, writing,
68
72
  operations). You get part of one of their conversations: the person's requests (numbered, each with its date), the
69
73
  agent's steps (tool calls, with errors) and what the agent said. Ask of everything you write: will a future agent act
@@ -184,12 +188,13 @@ const SCHEMA = {
184
188
  observations: {
185
189
  type: 'array',
186
190
  items: {
187
- type: 'object', additionalProperties: false, required: ['text', 'about', 'event_date', 'start_date', 'end_date', 'phase', 'requests', 'concepts'],
191
+ type: 'object', additionalProperties: false, required: ['text', 'about', 'event_date', 'start_date', 'end_date', 'phase', 'requests', 'concepts', 'cues'],
188
192
  properties: {
189
193
  text: { type: 'string' }, about: { type: 'string' }, event_date: { type: 'string' },
190
194
  start_date: { type: 'string' }, end_date: { type: 'string' },
191
195
  phase: { type: 'string', enum: [...PHASES] },
192
196
  requests: { type: 'array', items: { type: 'integer' } }, concepts: { type: 'array', items: { type: 'string' } },
197
+ cues: { type: 'array', items: { type: 'string' } },
193
198
  },
194
199
  },
195
200
  },
@@ -244,7 +249,7 @@ export function cleanObservations(raw, uidOf) {
244
249
  text, about: redactString((x.about ?? '').trim()), eventAt: dateOf(x.event_date),
245
250
  ...(dateOf(x.start_date) ? { startAt: dateOf(x.start_date) } : {}), ...(dateOf(x.end_date) ? { endAt: dateOf(x.end_date) } : {}),
246
251
  phase: PHASES.includes(String(x.phase)) ? x.phase : 'done',
247
- provenance, concepts: cleanConcepts(x.concepts),
252
+ provenance, concepts: cleanConcepts(x.concepts), cues: cleanCues(x.cues).slice(0, 4),
248
253
  });
249
254
  }
250
255
  return out;
@@ -7,6 +7,9 @@ export interface Gathered {
7
7
  kept: string[];
8
8
  /** The searches the model asked for. */
9
9
  searches: string[];
10
+ /** With `notes`: what the kept memories say that answers the question, a few short lines (empty otherwise). Show them
11
+ * before the memories, as notes on them: on LoCoMo this took 79.5 to 80.9 (docs/BENCHMARKS.md). */
12
+ notes: string[];
10
13
  }
11
14
  /** `from`/`to`: the time the question asks about ("on 10 April 2022", "in October 2023"). What happened then is searched
12
15
  * for besides everything else and read first — not instead: a question also names dates it only counts from ("as of
@@ -14,4 +17,5 @@ export interface Gathered {
14
17
  export declare function memoryGather(d: RecallDeps, q: SearchQuery, o: {
15
18
  ask?: Ask;
16
19
  limit?: number;
20
+ notes?: boolean;
17
21
  }): Promise<Gathered>;
@@ -20,6 +20,9 @@ import { memorySearch, MAX_SEARCH } from './recall.js';
20
20
  import { cut } from './text.js';
21
21
  /** Candidates the model reads in one ask. */
22
22
  const PART = 40;
23
+ /** With notes, all of them in one ask, so a question's notes are written once: on LoCoMo the same accuracy as two parts
24
+ * of 40 (80.9 against 80.6), at 58% of the cost and 2.9 s against 7.5 s median. */
25
+ const NOTES_PART = 80;
23
26
  /** Searches it may ask for. */
24
27
  const MORE = 6;
25
28
  const SYSTEM = `You pick, from someone's memories, every one needed to answer their question.
@@ -40,6 +43,19 @@ const SCHEMA = {
40
43
  searches: { type: 'array', items: { type: 'string' } },
41
44
  },
42
45
  };
46
+ /** With notes: the same reading also writes down what the kept memories say that answers the question, so a list spread
47
+ * over many memories reaches the reader as one line. */
48
+ const NOTES_RULE = `
49
+
50
+ In "notes", write what the memories you kept say that answers the question, in a few short lines: every instance with
51
+ its date as the memories give it ("Books Tim has read: Harry Potter (2023-08-02), The Name of the Wind (2023-08-11)"),
52
+ names, numbers and words as written. Only what they state, as plain statements: never write that something is not said,
53
+ not given, unclear or unknown — leave it out instead, since other memories may hold it — and never a guess. Leave notes
54
+ empty when the memories you kept say nothing about it. In the language of the question.`;
55
+ const NOTES_SCHEMA = {
56
+ type: 'object', additionalProperties: false, required: ['keep', 'searches', 'notes'],
57
+ properties: { ...SCHEMA.properties, notes: { type: 'array', items: { type: 'string' } } },
58
+ };
43
59
  const day = (ts) => (ts ? new Date(ts).toISOString().slice(0, 10) : '');
44
60
  /** One candidate as the model reads it: when it happened (from when to when, for what lasted) and when it was told, its
45
61
  * text, and the passage that matched. */
@@ -49,25 +65,29 @@ function line(h, n) {
49
65
  const text = h.kind === 'observation' ? h.summary : `${h.title} — ${cut(h.summary, h.kind === 'profile' ? PROFILE_SHOWN : 600)}`;
50
66
  return `[m${n}] (${when}) ${text}${h.match ? `\n … ${cut(h.match, 600)}` : ''}`;
51
67
  }
52
- async function pick(ask, question, list) {
68
+ async function pick(ask, question, list, notes = false) {
53
69
  const parts = [];
54
- for (let i = 0; i < list.length; i += PART)
55
- parts.push(list.slice(i, i + PART));
70
+ const size = notes ? NOTES_PART : PART;
71
+ for (let i = 0; i < list.length; i += size)
72
+ parts.push(list.slice(i, i + size));
56
73
  const answers = await Promise.all(parts.map(async (part) => {
57
74
  const prompt = `Question: ${question}\n\nMemories:\n${part.map((h, n) => line(h, n + 1)).join('\n')}`;
58
75
  const a = await ask({
59
- purpose: 'memory', system: SYSTEM, prompt: redactString(prompt), schema: SCHEMA,
60
- name: 'memory_gather', maxTokens: 40 * part.length + 600, timeoutMs: 120_000, consequence: 'so these memories are kept unread',
76
+ purpose: 'memory', system: notes ? SYSTEM + NOTES_RULE : SYSTEM, prompt: redactString(prompt),
77
+ schema: (notes ? NOTES_SCHEMA : SCHEMA),
78
+ name: 'memory_gather', maxTokens: 40 * part.length + (notes ? 1400 : 600), timeoutMs: 120_000, consequence: 'so these memories are kept unread',
61
79
  });
62
80
  if (!('answer' in a))
63
- return { keep: part.map((h) => h.id), searches: [] };
81
+ return { keep: part.map((h) => h.id), searches: [], notes: [] };
64
82
  const got = a.answer;
65
83
  const ids = Array.isArray(got.keep) ? got.keep.filter((x) => typeof x === 'string') : [];
66
84
  const keep = ids.map((x) => part[Number(x.replace(/^m/, '')) - 1]?.id).filter((x) => !!x);
67
85
  const searches = Array.isArray(got.searches) ? got.searches.filter((x) => typeof x === 'string' && !!x.trim()) : [];
68
- return { keep, searches };
86
+ const said = notes && Array.isArray(got.notes) ? got.notes.filter((x) => typeof x === 'string' && !!x.trim()).map((x) => redactString(x.trim())) : [];
87
+ return { keep, searches, notes: said };
69
88
  }));
70
- return { keep: new Set(answers.flatMap((a) => a.keep)), searches: [...new Set(answers.flatMap((a) => a.searches.map((s) => s.trim())))] };
89
+ return { keep: new Set(answers.flatMap((a) => a.keep)), searches: [...new Set(answers.flatMap((a) => a.searches.map((s) => s.trim())))],
90
+ notes: answers.flatMap((a) => a.notes) };
71
91
  }
72
92
  /** `from`/`to`: the time the question asks about ("on 10 April 2022", "in October 2023"). What happened then is searched
73
93
  * for besides everything else and read first — not instead: a question also names dates it only counts from ("as of
@@ -79,7 +99,7 @@ export async function memoryGather(d, q, o) {
79
99
  const [inDates, wide] = await Promise.all([from || to ? memorySearch(d, { ...q, limit }) : [], memorySearch(d, { ...open, limit })]);
80
100
  const dated = new Set(inDates.map((h) => h.id));
81
101
  const first = [...inDates, ...wide.filter((h) => !dated.has(h.id))];
82
- const one = await pick(ask, q.query, first);
102
+ const one = await pick(ask, q.query, first, o.notes);
83
103
  const searches = one.searches.slice(0, MORE);
84
104
  const have = new Set(first.map((h) => h.id));
85
105
  const more = [];
@@ -90,9 +110,9 @@ export async function memoryGather(d, q, o) {
90
110
  more.push(h);
91
111
  }
92
112
  }
93
- const two = more.length ? await pick(ask, q.query, more) : { keep: new Set(), searches: [] };
113
+ const two = more.length ? await pick(ask, q.query, more, o.notes) : { keep: new Set(), searches: [], notes: [] };
94
114
  const all = [...first, ...more];
95
115
  const kept = all.filter((h) => one.keep.has(h.id) || two.keep.has(h.id));
96
116
  const keptIds = new Set(kept.map((h) => h.id));
97
- return { found: [...kept, ...all.filter((h) => !keptIds.has(h.id))], kept: kept.map((h) => h.id), searches };
117
+ return { found: [...kept, ...all.filter((h) => !keptIds.has(h.id))], kept: kept.map((h) => h.id), searches, notes: [...one.notes, ...two.notes] };
98
118
  }
@@ -46,6 +46,10 @@ const SMALL = { optimizers_config: { default_segment_number: 2 } };
46
46
  * was the best share on both benchmarks, and CombSUM beat rank fusion at the top of both — LoCoMo requests any@3
47
47
  * 71.6 → 74.5, nDCG@10 64.6 → 65.7; LongMemEval-S any@1 63.4 → 64.5, sessions any@1 92.6 → 93.2. */
48
48
  const DENSE_SHARE = 0.7;
49
+ /** Cues — the questions an item answers, in the asker's words — search as their own channel at this share of the dense
50
+ * weight rather than among the item's points. Measured on LoCoMo's 1,307 held-out questions, together with cues on
51
+ * every observation and the seven-days-before calendar: multi-hop 50.6 → 55.2, overall 77.2 → 77.4 (docs/RESEARCH.md). */
52
+ const CUE_SHARE = 0.1;
49
53
  /** A point's id from its item and its place (0 = the key, n = the n-th child): the same item writes the same ids. */
50
54
  export function pointId(item, n) {
51
55
  const h = createHash('sha1').update(`${item}#${n}`).digest('hex');
@@ -278,15 +282,19 @@ export class QdrantClient {
278
282
  const channels = [];
279
283
  const sparse = q.sparse?.indices.length ? q.sparse : null;
280
284
  const denseWeight = (sparse ? DENSE_SHARE : 1) / Math.max(1, q.dense.length);
281
- for (const v of q.dense)
282
- channels.push({ query: v, using: 'dense', params: RESCORE, weight: denseWeight });
285
+ const cueOff = { ...(filter ?? {}), must_not: [...(filter?.must_not ?? []), { key: 'part', match: { value: 'cue' } }] };
286
+ const cueOnly = { ...(filter ?? {}), must: [...(filter?.must ?? []), { key: 'part', match: { value: 'cue' } }] };
287
+ for (const v of q.dense) {
288
+ channels.push({ query: v, using: 'dense', params: RESCORE, weight: denseWeight * (1 - CUE_SHARE), filter: cueOff });
289
+ channels.push({ query: v, using: 'dense', params: RESCORE, weight: denseWeight * CUE_SHARE, filter: cueOnly });
290
+ }
283
291
  if (sparse)
284
292
  channels.push({ query: sparse, using: 'bm25', weight: 1 - DENSE_SHARE });
285
293
  if (!channels.length)
286
294
  return [];
287
295
  const lists = await Promise.all(channels.map((c) => this.call('POST', `/collections/${this.collection}/points/query/groups`, {
288
296
  query: c.query, using: c.using, ...(c.params ? { params: c.params } : {}),
289
- ...(filter ? { filter } : {}),
297
+ ...(c.filter ?? filter ? { filter: c.filter ?? filter } : {}),
290
298
  group_by: 'item',
291
299
  group_size: 1,
292
300
  limit: per,
package/dist/memory.d.ts CHANGED
@@ -84,7 +84,9 @@ export interface Memory {
84
84
  limit?: number;
85
85
  }): Promise<Explored>;
86
86
  /** A set for a question that counts, lists or totals: a wide search, then the language model keeps every instance. */
87
- gather(q: SearchQuery): Promise<Gathered>;
87
+ gather(q: SearchQuery, o?: {
88
+ notes?: boolean;
89
+ }): Promise<Gathered>;
88
90
  /** Read what was added since last time and keep typed knowledge, dated observations and kept episodes. Needs a
89
91
  * language model. Returns how many windows were read. */
90
92
  consolidate(): Promise<ConsolidationReport>;
package/dist/memory.js CHANGED
@@ -201,7 +201,7 @@ export async function openMemory(o) {
201
201
  return sources.length ? { ...p, sources } : p;
202
202
  },
203
203
  explore: (s) => memoryExplore(deps, s),
204
- gather: (s) => memoryGather(deps, s, { ask }),
204
+ gather: (s, g) => memoryGather(deps, s, { ask, ...(s.limit ? { limit: s.limit } : {}), ...(g?.notes ? { notes: true } : {}) }),
205
205
  async consolidate() {
206
206
  const worker = new Consolidator({ ...base, concurrency: o.concurrency ?? 4 });
207
207
  const total = { windows: 0, added: 0, merged: 0, updated: 0, superseded: 0, linked: 0, dropped: 0, episodes: 0, observed: 0, cued: 0 };
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "memtrails",
3
- "version": "0.1.0",
3
+ "version": "0.3.0",
4
4
  "description": "Retrieval and exploration for any text — e-mail, chat, notes, documents, news, commits, conversations. Searchable at once, on your own machine, in any language; optional consolidation into typed knowledge with a language model.",
5
5
  "keywords": [
6
6
  "retrieval",