@staix/agent-mem 0.9.1 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-mem",
3
- "version": "0.9.1",
3
+ "version": "0.11.0",
4
4
  "description": "Bounded project memory tools and seven deterministic capture hooks, without a resident worker.",
5
5
  "mcpServers": {
6
6
  "agent-mem": {
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  Curated memory with append-only JSONL logs, a derived SQLite index, short- and long-term tiers, a typed graph, goals with leases, and git sync between machines. Extracted from [agent-hub](https://github.com/STAIxBWLB/agent-hub), with compatible storage and entry IDs.
4
4
 
5
- Version: 0.9.1. Requires Bun 1.4.2 or later; no runtime dependencies.
5
+ Version: 0.11.0. Requires Bun 1.4.2 or later; no runtime dependencies.
6
6
 
7
7
  ## Quick start
8
8
 
@@ -15,7 +15,7 @@ amem setup all
15
15
  amem setup claude --apply
16
16
  ```
17
17
 
18
- Replace `claude` with `codex`, `kimi` or `pi` for that client. Setup shows its changes and asks for confirmation. Complete the client's own hook-trust review when prompted.
18
+ Replace `claude` with `codex`, `kimi`, `pi`, `qwen` or `jcode` for that client. Setup shows its changes and asks for confirmation. Complete the client's own hook-trust review when prompted.
19
19
 
20
20
  Agents submit pending memory candidates through MCP. Inspect them after a session and accept or reject each one:
21
21
 
@@ -26,12 +26,18 @@ amem review accept CANDIDATE_ID # or: amem review reject CANDIDATE_ID
26
26
 
27
27
  Replace `CANDIDATE_ID` with an id from the list. Acceptance is person-only and applies the configured privacy policy.
28
28
 
29
- Claude's registered hooks capture events during the session. The configured Codex/Kimi SessionEnd hooks and Pi shutdown extension ingest transcript tails incrementally at session end. If those integrations are unavailable, run `amem ingest codex` (or `kimi` / `pi`) after each session, then repeat the review loop. The [setup section](#setup--plugin) describes the registration and native trust steps.
29
+ Claude's registered hooks capture events during the session. The configured Codex/Kimi/Qwen SessionEnd hooks and Pi shutdown extension ingest transcript tails incrementally at session end. If those integrations are unavailable, run `amem ingest codex` (or `kimi` / `pi` / `qwen`) after each session, then repeat the review loop. The [setup section](#setup--plugin) describes the registration and native trust steps.
30
30
 
31
31
  From a checkout, install development dependencies with `bun install` and run `bun src/cli.ts --help`.
32
32
 
33
33
  ## Setup / plugin
34
34
 
35
+ `amem setup agy` registers MCP only in the current git repository's `.agents/mcp_config.json`, with an explicit absolute repository cwd and a private receipt per repository. `setup all` excludes agy; run it separately in each repository. Setup refuses tracked config files and JSONC, and warns when the file is not ignored. Restart Antigravity after applying, use one workspace per session, and ensure no global, plugin or running server is also named `amem`. No global configuration, ingest command or hook is installed. Undo removes `.agents` only when setup created it and it is still empty; pre-existing or nonempty directories remain. Older receipts still restore their configuration without directory cleanup. See the [Antigravity contract and capture limitation](docs/antigravity.md).
36
+
37
+ `amem setup jcode` registers MCP only, using Jcode v0.99.1's global `JCODE_HOME/mcp.json` (default `~/.jcode/mcp.json`). An explicitly set `JCODE_HOME` must be absolute for amem setup: empty, relative and literal `~` values are refused. Unset uses the default, and absolute values retain their literal spelling without trimming. It owns only `servers.amem`, sets `shared: false` for a workspace-bound session, and preserves unrelated configuration bytes. Jcode has no native MCP add/remove command in this release, so setup edits the owned entry directly. No Jcode ingest command or capture hook is installed: saved messages lack per-message origin while the session cwd can change. Launch Jcode with `JCODE_DISABLE_CLAUDE_MCP=1` and without a same-name project override, which could otherwise replace the private binding. Use a new workspace-bound session when changing projects. See the [Jcode contract and capture limitation](docs/jcode.md).
38
+
39
+ `amem setup qwen` previews a user-scope MCP registration and a bounded SessionEnd ingest hook in the same `QWEN_HOME/settings.json` (default `~/.qwen/settings.json`). `--apply` and `--undo` follow the same confirmed, private-receipt contract as the other targets. Unrelated settings, servers and hooks are preserved. Qwen runs this hook on normal interactive exit; use `amem ingest qwen` when the session ends without that hook. Native-client operation is source-verified; the installed-package tests use stub registration CLIs.
40
+
35
41
  Preview the exact configuration files, native registration commands and private undo receipts before changing anything:
36
42
 
37
43
  ```sh
@@ -203,7 +209,11 @@ Project identity (#15 D2) keeps `memoryProjectKey`: a worktree, a submodule and
203
209
 
204
210
  `store.search(query, filter)` searches the curated tier and the current project's capture shard with SQLite FTS5 and BM25 plus tier/recency weighting: natural multi-word queries match partial term sets (the substring filter required every word), and a curated row outranks the same text in the capture tier. Prefix terms match inflections and Korean particles in either direction by retaining the original query term and adding a common-particle-stripped alternative. The FTS copy of the curated rows lives in its own derived `search.db` (`index.db`'s table set is a frozen contract and gains nothing); it is written from the same apply path as the index, and a prune, crash or rebuild is detected and resynced, so the incremental FTS copy equals a full resync. Where FTS5 is unavailable the engine falls back to the substring filter and reports `engine: "substring"`; capture recall needs FTS5 and is skipped in the fallback. The library entry exports `CaptureStore`, `captureSearch`, `readCaptureConfig`, `writeCaptureConfig`, `SearchIndex`, `fts5Available`, the shard helpers (`resolveAlias`, `appendAlias`, `listAliases`, `shardKeyHash`), the catalog (`rebuildCatalog`, `readCatalog`, `scanCatalog`) and the capacity advisor (`capacityReport`).
205
211
 
206
- `amem eval [--set ko|<file>] [--k 1,5,10] [--json]` measures recall with no model and no network (agent-mem#40 F13): it replays a question set and reports recall@k, hit@k and MRR. It scores the ranked `store.search` (BM25 with tier and recency weights), which is what the MCP `amem_search` tool and hub recall rank with and what #13 AC4 measured. It is not the `amem search` command: that one is a substring filter (every word must appear, newest first), so `amem search lighthouse beacon` can print nothing for a note that `amem eval` finds. Whether `amem search` should use the ranked search is a separate question. Without `--set` it scores the built-in synthetic set (26 invented harbour entries, long and short tier, written a day apart, and 32 questions, shipped in `src/memory/eval-set.json`), which is seeded into a throwaway store, so it never reads or writes your memory home and gives the same numbers on every machine. A set of your own either carries its own `entries` (self-contained, named by `key`) or, without entries, names real entry ids (or a prefix of 8 or more hex characters) and is scored against this project's store; each name is resolved like `amem show` does (a promoted short-term id finds its long-term copy, and a superseded id stands for the entry that replaced it, since search returns only live entries), and a prefix that matches more than one live entry is refused. `mem:<id>` as a brief prints it is accepted. Questions whose expected entry is not in the store score 0 in `all` and are left out of `answerable`, and `answerable` is scored over the entries the store holds, so a ranking problem and a coverage problem stay apart (`n/a` when no question is answerable). `--set ko` scores the second built-in set instead: 37 invented Korean entries and 38 questions (particle variants, spaced and unspaced compounds, mixed Korean-English terms, Latin abbreviations, short and long tier), with its baseline recorded in `docs/contracts.md` (agent-mem#62). Set format: `docs/contracts.md#recall-evaluation-f13`.
212
+ `amem search <words...>` ranks the current project's current curated notes with `store.search` (FTS5/BM25 plus tier and recency weights). Natural multi-word queries can match partial term sets: `amem search lighthouse beacon` can return a note containing only one of those terms. Add `--filter` to retain the previous behavior for current-project curated notes: every query word must appear as a substring, and results are newest first. `--history`, `--as-of` and an empty query retain the previous filter/newest-first path. Repository-scope notes remain a separately screened append to these results. `--all-projects` keeps its existing ranked path and cannot be combined with `--filter`. JSON output retains the existing note fields.
213
+
214
+ `amem eval [--set ko|<file>] [--k 1,5,10] [--json]` measures recall with no model and no network (agent-mem#40 F13): it replays a question set against ranked `store.search` and reports recall@k, hit@k and MRR. The ordinary curated project `amem search` path now uses that ranked search; `--filter` selects the legacy substring behavior. Without `--set` it scores the built-in synthetic set (26 invented harbour entries, long and short tier, written a day apart, and 32 questions, shipped in `src/memory/eval-set.json`), which is seeded into a throwaway store, so it never reads or writes your memory home and gives the same numbers on every machine. A set of your own either carries its own `entries` (self-contained, named by `key`) or, without entries, names real entry ids (or a prefix of 8 or more hex characters) and is scored against this project's store; each name is resolved like `amem show` does (a promoted short-term id finds its long-term copy, and a superseded id stands for the entry that replaced it, since search returns only live entries), and a prefix that matches more than one live entry is refused. `mem:<id>` as a brief prints it is accepted. Questions whose expected entry is not in the store score 0 in `all` and are left out of `answerable`, and `answerable` is scored over the entries the store holds, so a ranking problem and a coverage problem stay apart (`n/a` when no question is answerable). `--set ko` scores the second built-in set instead: 37 invented Korean entries and 38 questions (particle variants, spaced and unspaced compounds, mixed Korean-English terms, Latin abbreviations, short and long tier), with its baseline recorded in `docs/contracts.md` (agent-mem#62). Set format: `docs/contracts.md#recall-evaluation-f13`.
215
+
216
+ External retrieval measurements for MCP, library search and a plain BM25 baseline are recorded in [LongMemEval S/M](docs/bench/longmemeval.md). The full datasets stay outside the repository; CI runs only synthetic harness fixtures.
207
217
 
208
218
  The R7 capacity advisor reports this machine's load and proposes capture bounds: `amem capacity [--json]` shows capture size and growth, disk, memory, the rolling `metrics.jsonl` readings and the proposal with its binding constraint; `amem capacity apply [--cap-mb N] [--retention-days N]` saves a preview, and only a matching later `--confirm` writes the home default config. Applying is person-only (agent markers refused, a terminal required); an agent may read the proposal through the library, never apply it. Under the 2026-10-17 owner decision (item 2) the proposal is the largest cap and retention that fit, and it may be above the current values: the cap is the disk-share bound itself (default 2% of free space, `--disk-share`), and retention is the share divided by the measured daily growth, up to a 365-day ceiling (a `ponytail:` constant, a person setting is the upgrade path). A hook p95 over 150 ms or hook RSS over 100 MB halves the proposed cap, and search RSS over 200 MB halves the retention. The reason names every binding constraint: `disk-share retention` or `retention ceiling`, `disk-share cap`, `R3 search RSS`, `R2 hook cost`. A proposal is never written by the advisor, and `apply` still takes `--cap-mb` and `--retention-days` to override it. Benchmark samples stay in the measuring script's synthetic home; only real hook and search invocations feed the advisor's rolling metrics.
209
219
 
@@ -282,7 +292,7 @@ Successful native project identities have a private, bounded derived cache (128
282
292
 
283
293
  `amem doctor` prints one read-only report of this project's memory: duplicate and stale notes, pending candidates, capture freshness and hook registration for each CLI (a CLI with no hook is `manual`), the SessionStart brief's token cost, whether the index equals a rebuild of the logs, reader leases and the policy in effect. `amem doctor --json` is a stable machine-readable form (`amem.doctor/v1`). It writes nothing into the memory home (every check runs on a temporary copy of the logs) and creates nothing in a home you never used.
284
294
 
285
- Intake runs at the end of each Codex, Kimi and Pi session once `amem setup <cli> --apply` has registered it. Sync is off unless you ask for it: amem registers no sync hook and runs nothing between sessions. To sync on your own schedule, call `amem sync --if-due` from your shell (for example an EXIT trap in `~/.zshrc`; it needs a terminal). It syncs at most hourly, backs off 15 minutes doubling to a day after failures, stops after 30 seconds, and does nothing without a remote set by `amem sync --remote <private git url>`. `--force` skips the schedule and the backoff.
295
+ Intake runs at the end of each Codex, Kimi, Pi and Qwen session once `amem setup <cli> --apply` has registered it. Sync is off unless you ask for it: amem registers no sync hook and runs nothing between sessions. To sync on your own schedule, call `amem sync --if-due` from your shell (for example an EXIT trap in `~/.zshrc`; it needs a terminal). It syncs at most hourly, backs off 15 minutes doubling to a day after failures, stops after 30 seconds, and does nothing without a remote set by `amem sync --remote <private git url>`. `--force` skips the schedule and the backoff.
286
296
 
287
297
  Split-home sync durably holds verified foreign-generation prunes with a "held, not applied" warning while unrelated sync continues; targets can remain visible, and full cross-generation removal remains a separate ceiling (see [the sync contract](docs/contracts.md#per-shard-private-sync-15-t3)).
288
298
 
@@ -298,6 +308,8 @@ amem ingest pi --since 2026-10-01T00:00:00Z --json
298
308
 
299
309
  Discovery uses `CODEX_HOME` or `~/.codex` (`sessions/` and `archived_sessions/`), `KIMI_CODE_HOME` or `~/.kimi-code` (`sessions/**/wire.jsonl`), and `PI_CODING_AGENT_DIR` or `~/.pi/agent` (`sessions/`). It does not follow directory symlinks. A native record must carry a recognized schema/time and an observed absolute working-directory binding that resolves to this canonical project, including reviewed aliases. Unknown working directories, unsupported Kimi wire versions, missing timestamps and unavailable source roots are reported rather than guessed as an empty successful history. Foreign/private/secret rows are not imported. Full native envelopes and extracted capture inputs both pass current policy.
300
310
 
311
+ Qwen Code uses `QWEN_RUNTIME_DIR`, then `QWEN_HOME`, then `~/.qwen`, with bounded discovery of `projects/*/chats/*.jsonl`. Each row proves its own project through `cwd`; a sanitized directory name is not an identity. The reader supports the verified 0.26.1 native envelope, an internal format rather than a public schema, and excludes thought/media content. For `advanced.runtimeOutputDir`, export the matching `QWEN_RUNTIME_DIR`; amem does not reproduce merged Qwen settings or read `.env`. See the [Qwen capture contract](docs/qwen-code.md) for source pins, cursor upgrade behavior and limitations.
312
+
301
313
  Per-source cursor databases and a fair rotation cursor live in the capture shard's private `ingest/` directory. Filenames and source identities are hashes; cursor contents retain offsets, anchor hashes, scope/protocol/cutoff and ownership metadata, not raw transcript paths or text. Capture receipts make replay idempotent across a crash between append and cursor acknowledgement. Repeating the command resumes its bounded tour instead of repeatedly favoring the first files. Broadening `--since` rechecks earlier history while existing capture receipts prevent duplicates. The JSON receipt reports `files`, `read`, `imported`, `skipped`, `spooled`, `remaining`, `errors` and an optional bounded `reason`; unavailable coverage is not silently represented by zero imported rows.
302
314
 
303
315
  The default invocation reads at most 4 MiB and attempts 32 sources; trusted library options can raise those limits to 16 MiB and 128 sources. It examines at most 4,096 records, bounds a record at 256 KiB, and caps discovery at 4,096 directory entries/depth 8. Re-run a receipt with `remaining: true` after inspecting its reason; ingest creates no polling timer or watcher. Oversized records, including an unbroken record longer than the 4 MiB default window, are counted once and discarded through their next newline using a durable byte offset/skip flag. Later valid records remain reachable without retaining oversized content; an unfinished oversized tail stays pending until its newline arrives. The cursor assumes append-only native files and verifies file identity plus the last 1,024 acknowledged bytes. Historical rewrites outside that anchor are a declared ceiling, not full-file verification. These readers do not establish account-backed native interoperability, P5 cutover or P6 live-week acceptance.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@staix/agent-mem",
3
- "version": "0.9.1",
3
+ "version": "0.11.0",
4
4
  "description": "Curated, shared memory for coding agents: append-only logs, a derived index, tiers and a typed graph.",
5
5
  "license": "MIT",
6
6
  "type": "module",
package/src/cli.ts CHANGED
@@ -43,6 +43,7 @@ import { appendGitattributes, appendRepoLine, boundRepoRoot, planGitattributes,
43
43
  import type { Visibility } from "./memory/sync.ts";
44
44
  import { applySplit, previewSplit } from "./memory/capacity-split.ts";
45
45
  import { evalText, parseKs } from "./memory/eval.ts";
46
+ import { unrankedScoreDetails } from "./memory/search-score.ts";
46
47
  import { builtinSet, evalSeeded, evalStore, FIXTURE_PATH, readEvalSet } from "./memory/eval-fixture.ts";
47
48
  import { markShown, pendingSummary } from "./memory/candidate-pending.ts";
48
49
  import { buildNote, FAIL_TEMPLATE, hasFailLabel, noteAdd, parseFailTemplate, type FailFields } from "./memory/note-add.ts";
@@ -168,9 +169,9 @@ export async function runCli(argv: readonly string[], context: CliContext = {}):
168
169
  const parsed = parseCommand(argv);
169
170
  if (parsed.options["--help"] === true) return commandHelp();
170
171
  const kind = parsed.positionals[0];
171
- if (parsed.positionals.length !== 1 || !["codex", "kimi", "pi"].includes(kind ?? "")) throw new Error("usage: amem ingest codex|kimi|pi [--since ISO] [--json]");
172
+ if (parsed.positionals.length !== 1 || !["codex", "kimi", "pi", "qwen"].includes(kind ?? "")) throw new Error("usage: amem ingest codex|kimi|pi|qwen [--since ISO] [--json]");
172
173
  const { ingestTranscript } = await import("./ingest.ts");
173
- return JSON.stringify(await ingestTranscript(kind as "codex" | "kimi" | "pi", {
174
+ return JSON.stringify(await ingestTranscript(kind as "codex" | "kimi" | "pi" | "qwen", {
174
175
  cwd: context.cwd ?? process.cwd(), base: memoryHome(context.env ?? process.env),
175
176
  ...(typeof parsed.options["--since"] === "string" ? { since: parsed.options["--since"] as string } : {}),
176
177
  }));
@@ -195,10 +196,10 @@ export async function runCli(argv: readonly string[], context: CliContext = {}):
195
196
  }
196
197
  const { definition, options, rawArgs: args } = parsed;
197
198
  if (parsed.path[0] === "setup") {
198
- if (parsed.positionals.length !== 1 || options["--apply"] === true && options["--undo"] === true) throw new Error("usage: amem setup claude|codex|kimi|pi|all [--apply|--undo]");
199
+ if (parsed.positionals.length !== 1 || options["--apply"] === true && options["--undo"] === true) throw new Error("usage: amem setup claude|codex|kimi|pi|qwen|jcode|agy|all [--apply|--undo]");
199
200
  const { runSetup } = await import("./setup.ts");
200
201
  return runSetup(parsed.positionals[0]!, options["--apply"] === true ? "apply" : options["--undo"] === true ? "undo" : "preview", {
201
- env: context.env ?? process.env, stdinTTY: context.stdinTTY ?? process.stdin.isTTY,
202
+ env: context.env ?? process.env, stdinTTY: context.stdinTTY ?? process.stdin.isTTY, cwd: context.cwd ?? process.cwd(),
202
203
  ...(context.setupConfirm ? { confirm: context.setupConfirm } : {}),
203
204
  });
204
205
  }
@@ -573,7 +574,7 @@ export async function runCli(argv: readonly string[], context: CliContext = {}):
573
574
  try {
574
575
  const result = await importProject(store, { from: options["--from"], project: key, confirm: options["--confirm"] === true,
575
576
  allowShared: options["--allow-shared"] === true, policy: note => policy.screenStandalone(note) });
576
- const counts = `sources ${JSON.stringify(result.provenance.source)}, peers ${JSON.stringify(result.provenance.peer)}, scopes ${JSON.stringify(result.provenance.scope)}`;
577
+ const counts = `sources ${JSON.stringify(result.provenance.source)}, peers ${JSON.stringify(result.provenance.peer)}, scopes ${JSON.stringify(result.provenance.scope)}, machines ${JSON.stringify(result.provenance.machine)}`;
577
578
  const summary = result.applied ? `imported ${result.added} of ${result.read} curated notes into ${result.project} with their original ids`
578
579
  : `preview: ${result.wouldAdd} of ${result.read} curated notes can be imported into ${result.project} with their original ids (re-run with --confirm to write)`;
579
580
  return options["--json"] ? JSON.stringify(result, null, 2)
@@ -589,6 +590,8 @@ export async function runCli(argv: readonly string[], context: CliContext = {}):
589
590
  return options["--json"] ? JSON.stringify(written, null, 2) : `${op === "forget" ? "forgot" : op === "archive" ? "archived" : "unarchived"} ${written.key}${requested !== written.key ? ` (alias ${requested})` : ""}`;
590
591
  }
591
592
  if (sub === "search" && options["--all-projects"] === true) {
593
+ if (options["--explain"] === true) throw new Error("--explain applies to current-project ranked search, not --all-projects");
594
+ if (options["--filter"] === true) throw new Error("--filter applies to project-local search, not --all-projects");
592
595
  const words = parsed.positionals.join(" ");
593
596
  if (!words.trim()) throw new Error("usage: amem search --all-projects <words> [--limit N] [--include-archived] [--json]");
594
597
  const result = searchAllProjects(memoryHome(), words, { ...(options["--limit"] !== undefined ? { limit: Number(options["--limit"]) } : {}), includeArchived: options["--include-archived"] === true });
@@ -597,6 +600,11 @@ export async function runCli(argv: readonly string[], context: CliContext = {}):
597
600
  return [`crossed projects: searched ${result.projects.length} (${result.projects.join(", ")})`, ...(result.hits.length ? result.hits.map(one) : ["(nothing)"])].join("\n");
598
601
  }
599
602
  if (sub === "search" && options["--include-archived"] === true) throw new Error("--include-archived only applies with --all-projects");
603
+ if (sub === "search" && options["--explain"] === true) {
604
+ if (options["--filter"] === true) throw new Error("--explain cannot be combined with --filter");
605
+ if (options["--history"] === true || options["--as-of"] !== undefined) throw new Error("--explain cannot be combined with --history or --as-of");
606
+ if (!parsed.positionals.join(" ").trim()) throw new Error("--explain requires a non-empty search query");
607
+ }
600
608
  if (sub === "alias") {
601
609
  if (parsed.path[1] === "add") {
602
610
  const [from, to] = parsed.positionals;
@@ -685,10 +693,12 @@ export async function runCli(argv: readonly string[], context: CliContext = {}):
685
693
  const kind = options["--kind"], since = options["--since"], limit = options["--limit"] !== undefined ? Number(options["--limit"]) : 50;
686
694
  const rows = read.views.filter(row => options["--tier"] !== "short" && (kind === undefined || row.kind === kind) && (typeof since !== "string" || Date.parse(row.at) >= Date.parse(since))
687
695
  && words.every(word => `${row.title ?? ""}\n${row.text}\n${row.about.join(" ")}`.toLowerCase().includes(word))).slice(0, limit);
688
- if (options["--json"]) { try { const base = JSON.parse(output) as unknown[]; return JSON.stringify([...base, ...rows], null, 2); } catch { return output; } }
696
+ const repoDetails = explainSearch ? rows.map(row => ({ ...row, scoreDetails: unrankedScoreDetails("repo", null) })) : rows;
697
+ if (options["--json"]) { try { const base = JSON.parse(output) as unknown[]; return JSON.stringify([...base, ...repoDetails], null, 2); } catch { return output; } }
689
698
  const line = (row: RepoView) => `repo:${row.id.slice(0, 8)} ${row.at.slice(0, 16).replace("T", " ")} repo ${row.kind.padEnd(8)} ${`${row.title ? `${row.title}: ` : ""}${row.text}`.replace(/\s+/g, " ").slice(0, 100)}`;
690
699
  const note = read.state === "unsafe" ? [`(repo scope unavailable: ${read.reason})`] : read.skipped.malformed + read.skipped.mismatched + read.skipped.withheld ? [`(${read.skipped.malformed + read.skipped.mismatched} repo lines skipped as invalid, ${read.skipped.withheld} withheld by the screen)`] : [];
691
- return [output, ...(rows.length ? ["repo scope (.amem/log.jsonl):", ...rows.map(line)] : []), ...note].join("\n");
700
+ const lines = explainSearch ? rows.flatMap(row => [line(row), `score: ${JSON.stringify(unrankedScoreDetails("repo", null))}`]) : rows.map(line);
701
+ return [output, ...(rows.length ? ["repo scope (.amem/log.jsonl):", ...lines] : []), ...note].join("\n");
692
702
  };
693
703
  const crosses = definition.policyFree === true;
694
704
  const cwd = context.cwd ?? process.cwd();
@@ -731,11 +741,14 @@ export async function runCli(argv: readonly string[], context: CliContext = {}):
731
741
  if (args[index] === "--capture-days") { captureArgs.add(index); captureArgs.add(index + 1); }
732
742
  else if (args[index]?.startsWith("--capture-days=")) captureArgs.add(index);
733
743
  }
734
- const syncArgs = args.filter((arg, index) => arg !== "--if-due" && arg !== "--force" && !captureArgs.has(index));
744
+ // --filter and --explain select standalone CLI behavior; hosts keep their existing raw command arguments.
745
+ const legacySearch = options["--filter"] === true;
746
+ const explainSearch = sub === "search" && options["--explain"] === true;
747
+ const syncArgs = args.filter((arg, index) => arg !== "--if-due" && arg !== "--force" && !captureArgs.has(index) && (!legacySearch || arg !== "--filter") && (!explainSearch || arg !== "--explain"));
735
748
  let output: string;
736
749
  const stop = ifDue ? stopSignal(context.syncDeadlineMs ?? SYNC_IF_DUE_DEADLINE_MS, process) : undefined;
737
750
  try {
738
- output = await withRepoRows(await rawMemoryCommand(syncArgs, cwd, { person: true, project: boundProject(), open, readable: notes => notes, stale: createRecallStalenessProbe({ cwd: recallRoot ?? cwd, rootConfirmed, policy: () => configured }), outputReady: notes => outputReady(notes.map(view => view.id)), importPolicy: () => text => configured.declinesPersonal(text),
751
+ output = await withRepoRows(await rawMemoryCommand(syncArgs, cwd, { person: true, rankedSearch: !legacySearch, ...(explainSearch ? { explainSearch: true } : {}), project: boundProject(), open, readable: notes => notes, stale: createRecallStalenessProbe({ cwd: recallRoot ?? cwd, rootConfirmed, policy: () => configured }), outputReady: notes => outputReady(notes.map(view => view.id)), importPolicy: () => text => configured.declinesPersonal(text),
739
752
  sync: { screen: row => configured.screenStandalone(row), ...(stop ? { signal: stop.signal } : {}) }, // the repository-narrowed policy binds what sync publishes too
740
753
  onBridgeStatus: status => { receipt.bridgeStatus = status; } }));
741
754
  } catch (error) {
@@ -25,20 +25,21 @@ export const GLOBAL_OPTIONS = { "--help": flag("Show help") };
25
25
  export const ROOT_OPTIONS = { "--version": flag("Show version"), ...json };
26
26
  export const COMMANDS: Record<string, CommandDefinition> = {
27
27
  consolidate: { description: "Queue deterministic project-local consolidation candidates; curated notes change only after person review", mutates: true, options: { ...json, "--project": text("The current project's key; another project is refused (run from its directory)") } },
28
- setup: { description: "Preview reversible MCP and hook setup for native coding CLIs; apply and undo are person-only", options: { "--apply": flag("Apply after terminal confirmation"), "--undo": flag("Restore original configuration after confirmation") }, args: [{ name: "target", values: ["claude", "codex", "kimi", "pi", "all"] }] },
28
+ setup: { description: "Preview reversible MCP setup and supported capture hooks for native coding CLIs; apply and undo are person-only", options: { "--apply": flag("Apply after terminal confirmation"), "--undo": flag("Restore original configuration after confirmation") }, args: [{ name: "target", values: ["claude", "codex", "kimi", "pi", "qwen", "jcode", "agy", "all"] }] },
29
29
  mcp: { description: "Run the project-bound MCP stdio server for this session" },
30
30
  hook: { description: "Run a fail-open, context-only Claude Code hook", args: [{ name: "event", values: [...HOOK_EVENTS] }] },
31
- ingest: { description: "Incrementally capture native session transcripts without a watcher", options: { ...json, "--since": text("ISO date or time") }, args: [{ name: "kind", values: ["codex", "kimi", "pi"] }] },
31
+ ingest: { description: "Incrementally capture native session transcripts without a watcher", options: { ...json, "--since": text("ISO date or time") }, args: [{ name: "kind", values: ["codex", "kimi", "pi", "qwen"] }] },
32
32
  "vault-candidates": { description: "List screened durable curated vault candidates (per project, or across projects)", options: { ...json, "--limit": text("Maximum candidates per project (1..100)"),
33
33
  "--all-projects": flag("List every active project's candidates; the reply says it crossed projects"), "--project": text("List one named project's candidates") } },
34
34
  list: { description: "List memory", options: { ...filters, ...history } },
35
- search: { description: "Search memory", options: { ...filters, ...history, "--all-projects": flag("Search every active project; the reply says it crossed projects"),
35
+ search: { description: "Rank current curated project notes; history/as-of and empty queries retain newest-first filtering", options: { ...filters, ...history,
36
+ "--filter": flag("Current-project curated notes: match every word as a substring, newest first; not with --all-projects"), "--explain": flag("Include the score factors for ranked project-local results"), "--all-projects": flag("Search every active project; the reply says it crossed projects"),
36
37
  "--include-archived": flag("With --all-projects, also read archived projects") }, args: [{ name: "words", variadic: true }] },
37
38
  brief: { description: "Read intent/path memory: fails first, then decisions and contracts; untrusted data within a UTF-8 byte cap", options: { ...json,
38
39
  "--for": text("Explicit intent"), "--paths": text("Repeatable named path"), "--symbols": text("Repeatable named symbol"), "--bytes": text("Complete brief-body byte cap (1..8192, default 2048)") } },
39
40
  projects: { description: "List the project shards of this memory home (#15 D3)", options: { ...json, "--all": flag("Include forgotten projects (archived ones are always listed)") } },
40
41
  usage: { description: "Totals and the largest project shards", options: { ...json, "--limit": text("Largest shards to list (1..100)"), "--all": flag("Include forgotten projects") } },
41
- eval: { description: "Replay a local question set against the ranked store.search (the ranking MCP and hub recall use, not the `search` command's substring filter) and report recall@k and MRR (no model, no network; the built-in synthetic set without --set, `--set ko` for the built-in Korean one)", options: { ...json,
42
+ eval: { description: "Replay a local question set against ranked store.search and report recall@k and MRR (no model, no network; the built-in synthetic set without --set, `--set ko` for the built-in Korean one)", options: { ...json,
42
43
  "--set": { description: "`ko` (the built-in Korean set) or a question set file (amem.eval/v1); with entries it is self-contained, without them its ids name entries of this project", value: "path" }, "--k": text("Cutoffs, comma separated (default 1,5,10; each 1..100)") } },
43
44
  gc: { description: "Apply each shard's retention and cap across shards in bounded batches (preview without --confirm)", mutates: "confirm", options: {
44
45
  ...json, "--confirm": flag("Evict now; without it, preview only"), "--batches": text("evict() windows per shard in this run (1..1000, default 1)") } },
package/src/index.ts CHANGED
@@ -1,4 +1,5 @@
1
1
  export * from "./memory/store.ts";
2
+ export type { ScoreFactors } from "./memory/search-score.ts";
2
3
  export * from "./memory/capture.ts";
3
4
  export { fts5Available, ftsTerms, SearchIndex, captureSearch } from "./memory/search-fts.ts";
4
5
  export type { CuratedDoc, RankedRef } from "./memory/search-fts.ts";
package/src/ingest.ts CHANGED
@@ -2,7 +2,7 @@ import { Database } from "bun:sqlite";
2
2
  import { createHash, randomUUID } from "node:crypto";
3
3
  import { closeSync, chmodSync, constants, fchmodSync, fstatSync, lstatSync, mkdirSync, openSync, opendirSync, readSync, type Stats } from "node:fs";
4
4
  import { homedir } from "node:os";
5
- import { dirname, isAbsolute, join } from "node:path";
5
+ import { dirname, isAbsolute, join, resolve } from "node:path";
6
6
  import { CaptureStore, type CaptureInput } from "./memory/capture.ts";
7
7
  import { memoryHome } from "./memory/home.ts";
8
8
  import { canonicalPath, processLiveness, processSignature, realPath } from "./memory/platform.ts";
@@ -11,7 +11,7 @@ import { configuredPolicyOptions } from "./memory/config.ts";
11
11
  import { memoryProjectKey } from "./memory/project.ts";
12
12
  import { effectiveAliases } from "./memory/shard.ts";
13
13
 
14
- export type IngestKind = "codex" | "kimi" | "pi";
14
+ export type IngestKind = "codex" | "kimi" | "pi" | "qwen";
15
15
  export interface IngestOptions {
16
16
  cwd: string; base?: string;
17
17
  /** Trusted host/test seam. The ordinary CLI discovers native roots instead. */
@@ -28,6 +28,8 @@ interface Position { identity: string; offset: number; anchor: string; scope: Sc
28
28
  interface Cursor extends Position { token: string | null; pid: number | null; signature: string | null }
29
29
  const MAX_BYTES = 16 * 1024 * 1024, DEFAULT_BYTES = 4 * 1024 * 1024, MAX_FILES = 128, DEFAULT_FILES = 32;
30
30
  const MAX_CURSOR_BYTES = 128 * 1024;
31
+ // Bump after extending the verified Qwen decoder: replay skipped versions with existing capture receipts deduplicating accepted rows.
32
+ const QWEN_CONTRACT = "qwen-0.26.1-v1";
31
33
  const utf8 = new TextDecoder("utf-8", { fatal: true });
32
34
  const MAX_RECORD = 256 * 1024, MAX_RECORDS = 4096, MAX_DISCOVERY = 4096, MAX_DEPTH = 8, ANCHOR_BYTES = 1024;
33
35
  const hash = (value: string | Uint8Array) => createHash("sha256").update(value).digest("hex");
@@ -115,6 +117,7 @@ function lease(file: string): { position: Position; save(value: Position): void;
115
117
 
116
118
  /** Native roots only. Discovery never follows a directory symlink or enters an unbounded tree. */
117
119
  function discover(kind: IngestKind): { files: string[]; bounded: boolean; unavailable: boolean } {
120
+ if (kind === "qwen") return discoverQwen();
118
121
  const root = kind === "codex" ? process.env.CODEX_HOME?.trim() || join(homedir(), ".codex")
119
122
  : kind === "kimi" ? process.env.KIMI_CODE_HOME?.trim() || join(homedir(), ".kimi-code")
120
123
  : process.env.PI_CODING_AGENT_DIR?.trim() || join(homedir(), ".pi", "agent");
@@ -139,6 +142,38 @@ function discover(kind: IngestKind): { files: string[]; bounded: boolean; unavai
139
142
  return { files: files.sort(), bounded, unavailable: rootsRead === 0 };
140
143
  }
141
144
 
145
+ /** Only the native projects/<one directory>/chats/*.jsonl layout, never settings or account files. */
146
+ function discoverQwen(): { files: string[]; bounded: boolean; unavailable: boolean } {
147
+ const given = process.env.QWEN_RUNTIME_DIR || process.env.QWEN_HOME || join(homedir(), ".qwen");
148
+ const expanded = given === "~" ? homedir() : given.startsWith("~/") || given.startsWith("~\\") ? join(homedir(), ...given.slice(2).split(/[/\\]+/)) : given;
149
+ const root = resolve(expanded);
150
+ const files: string[] = []; let visited = 0, bounded = false;
151
+ const open = (folder: string) => {
152
+ const stat = lstatSync(folder);
153
+ if (!stat.isDirectory() || stat.isSymbolicLink()) throw Error("native directory unavailable");
154
+ return opendirSync(folder);
155
+ };
156
+ let projects: ReturnType<typeof opendirSync>;
157
+ try { if (lstatSync(root).isSymbolicLink()) throw Error("native directory unavailable"); projects = open(join(root, "projects")); }
158
+ catch { return { files, bounded, unavailable: true }; }
159
+ try {
160
+ for (let project = projects.readSync(); project; project = projects.readSync()) {
161
+ if (++visited > MAX_DISCOVERY) { bounded = true; break; }
162
+ if (!project.isDirectory() || project.isSymbolicLink()) continue;
163
+ let chats: ReturnType<typeof opendirSync>;
164
+ try { chats = open(join(root, "projects", project.name, "chats")); } catch { continue; }
165
+ try {
166
+ for (let item = chats.readSync(); item; item = chats.readSync()) {
167
+ if (++visited > MAX_DISCOVERY) { bounded = true; break; }
168
+ if (item.isFile() && !item.isSymbolicLink() && item.name.endsWith(".jsonl")) files.push(join(root, "projects", project.name, "chats", item.name));
169
+ }
170
+ } finally { chats.closeSync(); }
171
+ if (bounded) break;
172
+ }
173
+ } finally { projects.closeSync(); }
174
+ return { files: files.sort(), bounded, unavailable: false };
175
+ }
176
+
142
177
  function timeOf(record: Record<string, any>, kind: IngestKind): string | undefined {
143
178
  const value = kind === "kimi" ? record.time : record.timestamp;
144
179
  const millis = typeof value === "number" && kind === "kimi" && Number.isFinite(value) && value >= 100_000_000_000 ? value
@@ -151,7 +186,55 @@ function contentText(content: unknown): string {
151
186
  if (!Array.isArray(content)) return "";
152
187
  return content.filter(item => object(item) && ["text", "input_text", "output_text"].includes(item.type) && typeof item.text === "string").map(item => item.text).join("\n");
153
188
  }
189
+ const qwenUuid = (value: unknown): boolean => typeof value === "string" && /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(value);
190
+ const qwenSession = (value: unknown): boolean => typeof value === "string" && /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}(?:-agent-[a-zA-Z0-9_.-]+)?$/i.test(value);
191
+ function qwenPart(part: unknown): boolean {
192
+ if (!object(part)) return false;
193
+ if (part.thought === true) return true;
194
+ if (Object.keys(part).some(key => !["text", "thought", "thoughtSignature", "functionCall", "functionResponse"].includes(key))) return false;
195
+ if (part.thought !== undefined && typeof part.thought !== "boolean" || part.thoughtSignature !== undefined && typeof part.thoughtSignature !== "string") return false;
196
+ const forms = ["text", "functionCall", "functionResponse"].filter(key => part[key] !== undefined);
197
+ if (forms.length !== 1) return false;
198
+ if (forms[0] === "text") return typeof part.text === "string";
199
+ const value = part[forms[0]!];
200
+ return object(value) && typeof value.name === "string" && value.name.length > 0
201
+ && (value.id === undefined || typeof value.id === "string")
202
+ && (forms[0] === "functionCall" ? value.args === undefined || object(value.args)
203
+ : object(value.response) && (value.parts === undefined || Array.isArray(value.parts)));
204
+ }
205
+ function qwenRecord(raw: Record<string, any>): boolean {
206
+ // ponytail: CLI version is not a schema tag. Support only the 0.26.1 writer shape verified at
207
+ // QwenLM/qwen-code 1ce173117196ab9a9f77b5d19ac148cbabe25dfd; extend with native fixtures, not version guessing.
208
+ if (raw.version !== "0.26.1" || !qwenUuid(raw.uuid) || !qwenSession(raw.sessionId)
209
+ || raw.parentUuid !== null && !qwenUuid(raw.parentUuid) || !["user", "assistant", "tool_result", "system"].includes(raw.type)
210
+ || typeof raw.cwd !== "string" || !isAbsolute(raw.cwd) || !timeOf(raw, "qwen")) return false;
211
+ if (raw.type === "system" || raw.message === undefined) return true;
212
+ return object(raw.message) && raw.message.role === (raw.type === "assistant" ? "model" : "user")
213
+ && Array.isArray(raw.message.parts) && raw.message.parts.every(qwenPart);
214
+ }
215
+ function extractQwen(raw: Record<string, any>, project: string): CaptureInput | undefined {
216
+ if (raw.type === "system" || !object(raw.message)) return undefined;
217
+ const excerpts: string[] = [], tools: string[] = [], events: string[] = [];
218
+ for (const part of raw.message.parts) {
219
+ if (part.thought === true) continue;
220
+ if (typeof part.text === "string") { if (part.text) { excerpts.push(part.text); events.push("message"); } }
221
+ else if (object(part.functionCall)) {
222
+ const call = part.functionCall;
223
+ excerpts.push(JSON.stringify({ functionCall: { name: call.name, ...(call.id !== undefined ? { id: call.id } : {}), ...(call.args !== undefined ? { args: call.args } : {}) } }));
224
+ tools.push(call.name); events.push("function_call");
225
+ } else if (object(part.functionResponse)) {
226
+ const response = part.functionResponse;
227
+ // FunctionResponse.parts contains media, not textual tool output. Never stringify it into capture.
228
+ excerpts.push(JSON.stringify({ functionResponse: { name: response.name, ...(response.id !== undefined ? { id: response.id } : {}), response: response.response } }));
229
+ tools.push(response.name); events.push("function_call_output");
230
+ }
231
+ }
232
+ if (!excerpts.length) return undefined;
233
+ return { project, agent: "qwen", event: events.every(event => event === events[0]) ? events[0]! : "message", excerpt: excerpts.join("\n").slice(0, 2000),
234
+ ...(tools.length && tools.every(tool => tool === tools[0]) ? { tool: tools[0]!.slice(0, 64) } : {}) };
235
+ }
154
236
  function extract(kind: IngestKind, raw: Record<string, any>, project: string): CaptureInput | undefined {
237
+ if (kind === "qwen") return extractQwen(raw, project);
155
238
  const message = kind === "codex" && raw.type === "response_item" ? raw.payload
156
239
  : kind === "pi" && raw.type === "message" ? raw.message
157
240
  : kind === "kimi" && raw.type === "context.append_message" ? raw.message : undefined;
@@ -171,7 +254,10 @@ function extract(kind: IngestKind, raw: Record<string, any>, project: string): C
171
254
  /**
172
255
  * Incremental deterministic native-message ingestion; no watcher, account, model or network operation.
173
256
  * Native source contracts: Codex rust-v0.163.0 protocol RolloutItem; Pi session-manager v3;
174
- * Kimi b5e28fd6 human/persist/v2/wire.ts (protocol 1.5) and sessionIndexSource.recoverCwd.
257
+ * Kimi b5e28fd6 human/persist/v2/wire.ts (protocol 1.5) and sessionIndexSource.recoverCwd;
258
+ * Qwen 1ce173117196ab9a9f77b5d19ac148cbabe25dfd ChatRecord and config/storage.ts.
259
+ * Qwen discovery uses exported environment roots, not merged settings or .env files. For a configured
260
+ * advanced.runtimeOutputDir, export its resolved path as QWEN_RUNTIME_DIR or pass a trusted native file.
175
261
  * ponytail: discovery covers at most 4096 directory entries and depth 8. A larger native tree reports
176
262
  * remaining/source_bound; a future native index cursor can extend coverage without a resident watcher.
177
263
  * Within that bound a durable hash-only rotation admits at most maxFiles source attempts per call, including
@@ -185,7 +271,7 @@ export async function ingestTranscript(kind: IngestKind, options: IngestOptions)
185
271
  const result: IngestReceipt = { files: 0, read: 0, imported: 0, skipped: 0, spooled: 0, remaining: false, errors: 0 };
186
272
  const fail = (reason: NonNullable<IngestReceipt["reason"]>, remaining = false) => { result.errors++; result.reason ??= reason; result.remaining ||= remaining; };
187
273
  const bytes = options.maxBytes ?? DEFAULT_BYTES, count = options.maxFiles ?? DEFAULT_FILES;
188
- if (!["codex", "kimi", "pi"].includes(kind) || !Number.isSafeInteger(bytes) || bytes < 1 || bytes > MAX_BYTES || !Number.isSafeInteger(count) || count < 1 || count > MAX_FILES
274
+ if (!["codex", "kimi", "pi", "qwen"].includes(kind) || !Number.isSafeInteger(bytes) || bytes < 1 || bytes > MAX_BYTES || !Number.isSafeInteger(count) || count < 1 || count > MAX_FILES
189
275
  || options.files !== undefined && (!Array.isArray(options.files) || options.files.length > MAX_FILES || options.files.some(file => typeof file !== "string" || !isAbsolute(file) || file.includes("\0")))
190
276
  || options.since !== undefined && !Number.isFinite(Date.parse(options.since))) { fail("source_bound"); return result; }
191
277
  const since = options.since === undefined ? undefined : new Date(options.since).toISOString();
@@ -238,7 +324,9 @@ export async function ingestTranscript(kind: IngestKind, options: IngestOptions)
238
324
  if (!cursor) { fail("cursor_busy", true); continue; }
239
325
  let position = cursor.position;
240
326
  const cutoff = since ?? "", broader = position.cutoff !== "" && (cutoff === "" || cutoff < position.cutoff);
241
- if (position.identity !== identity || position.offset > initial.size || position.anchor !== anchor(fd, position.offset) || broader) position = { ...blank(), identity };
327
+ if (position.identity !== identity || position.offset > initial.size || position.anchor !== anchor(fd, position.offset) || broader
328
+ || kind === "qwen" && position.protocol !== QWEN_CONTRACT) position = { ...blank(), identity };
329
+ if (kind === "qwen") position.protocol = QWEN_CONTRACT;
242
330
  position.cutoff = cutoff;
243
331
  if (!position.anchor) position.anchor = anchor(fd, position.offset);
244
332
  cursor.save(position); result.files++;
@@ -299,12 +387,15 @@ export async function ingestTranscript(kind: IngestKind, options: IngestOptions)
299
387
  : kind === "pi" && raw.type === "session" ? raw : undefined;
300
388
  const permitted = await policy.screen(raw);
301
389
  guard(); metadataGuard();
302
- if (context !== undefined) position.scope = permitted && object(context) ? scopeOf(context.cwd) : "unknown";
390
+ const known = kind !== "qwen" || qwenRecord(raw);
391
+ if (kind === "qwen") position.scope = permitted && known ? scopeOf(raw.cwd) : "unknown";
392
+ else if (context !== undefined) position.scope = permitted && object(context) ? scopeOf(context.cwd) : "unknown";
303
393
  else if (raw.cwd !== undefined) position.scope = permitted ? scopeOf(raw.cwd) : "unknown";
304
394
  if (kind === "kimi" && raw.type === "metadata") {
305
395
  position.protocol = typeof raw.protocol_version === "string" && ["1.0", "1.1", "1.2", "1.3", "1.4", "1.5"].includes(raw.protocol_version) ? raw.protocol_version : "unknown";
306
396
  }
307
- const input = extract(kind, raw, project);
397
+ if (!known) fail("schema_or_time");
398
+ const input = known ? extract(kind, raw, project) : undefined;
308
399
  if (input) {
309
400
  const at = timeOf(raw, kind);
310
401
  if (!at || position.scope === "unknown" || kind === "kimi" && !/^1\.[0-5]$/.test(position.protocol)) { fail("schema_or_time"); result.skipped++; }
package/src/mcp-memory.ts CHANGED
@@ -10,7 +10,7 @@ import { join } from "node:path";
10
10
  import type { McpHandlers, McpOptions } from "./mcp.ts";
11
11
  import { CandidateStore, acceptedCorrectionText, candidateTargetRevision, type CandidateInput } from "./memory/candidates.ts";
12
12
  import { capacityReport } from "./memory/capacity.ts";
13
- import { CaptureStore } from "./memory/capture.ts";
13
+ import { CaptureStore, type CaptureRow } from "./memory/capture.ts";
14
14
  import { hashJSON, readSelector } from "./memory/generation.ts";
15
15
  import { memoryHome } from "./memory/home.ts";
16
16
  import { canonicalPath, realPath } from "./memory/platform.ts";
@@ -18,7 +18,7 @@ import { createStandalonePolicy, type StandalonePolicy, type StandalonePolicyOpt
18
18
  import { configuredPolicyOptions } from "./memory/config.ts";
19
19
  import { memoryProjectKey } from "./memory/project.ts";
20
20
  import { registerReaderLease } from "./memory/reader-lease.ts";
21
- import { type ReadRef, type ReadWindow, SearchIndex } from "./memory/search-fts.ts";
21
+ import { type ReadRef, type ReadWindow, SearchIndex, ScreenedSearchIndex } from "./memory/search-fts.ts";
22
22
  import { aliasedKeys, readShardedSelection, resolveAlias } from "./memory/shard.ts";
23
23
  import { projectState } from "./memory/lifecycle.ts";
24
24
  import { boundRepoRoot, readRepoScope, repoReadPolicy, type RepoView } from "./memory/repo-scope.ts";
@@ -28,9 +28,10 @@ import { WorkListStore, type WorkListFields } from "./memory/work-lists.ts";
28
28
 
29
29
  // Graph retains its complete-corpus ceiling; search/timeline/show page matching bodies instead.
30
30
  export const MCP_SCAN_LIMIT = 5000;
31
- // ponytail: rank the newest 5000 permitted matching candidates per tier; use since to narrow or upgrade to a policy-bound persistent index for global ranking.
31
+ // ponytail: evidence still ranks its newest 5000 permitted candidates; upgrade to the screened search stream if evidence needs global ranking.
32
32
  const SEARCH_CANDIDATES = 5000;
33
33
  const READ_PAGE = 256;
34
+ type SearchValue = { source: "curated"; evidence: true; view: View } | { source: "repo"; evidence: false; view: RepoView } | { source: "capture"; evidence: false; capture: CaptureRow };
34
35
  export interface MemoryMcpOptions { cwd: string; base?: string; project?: string; policy?: StandalonePolicyOptions; allowed?: () => boolean; policyVersion?: () => string; assertCurrent?: () => void }
35
36
  export interface MemoryMcp { tools: McpHandlers; close(): void; onEmitted: NonNullable<McpOptions["onEmitted"]> }
36
37
  const unavailable = (): never => { throw new Error("memory operation unavailable"); };
@@ -109,19 +110,25 @@ export function createMemoryMcp(options: MemoryMcpOptions): MemoryMcp {
109
110
  if (rows.length > MCP_SCAN_LIMIT) unavailable();
110
111
  return screen(rows.filter(row => row.project === project && recallViewIsLive(row, store.readNow())));
111
112
  };
112
- const readPermitted = async <T>(read: (window: ReadWindow) => { rows: T[]; next?: ReadRef; engine?: "fts5" | "substring" },
113
- options: Omit<ReadWindow, "project" | "limit" | "after">, cap: number, matches: (row: T, engine?: "fts5" | "substring") => boolean = () => true): Promise<T[]> => {
114
- const result: T[] = []; let after: ReadWindow["after"];
115
- while (result.length < cap) {
113
+ const permittedRows = async function* <T>(read: (window: ReadWindow) => { rows: T[]; next?: ReadRef; engine?: "fts5" | "substring" },
114
+ options: Omit<ReadWindow, "project" | "limit" | "after">, matches: (row: T, engine?: "fts5" | "substring") => boolean = () => true): AsyncGenerator<T> {
115
+ let after: ReadWindow["after"];
116
+ for (;;) {
116
117
  guard();
117
118
  const page = read({ ...options, project, limit: READ_PAGE, after });
118
- for (const row of page.rows) if (await currentPolicy().screen(row) && matches(row, page.engine)) {
119
- result.push(row); if (result.length === cap) break;
119
+ for (const row of page.rows) {
120
+ const allowed = await currentPolicy().screen(row); guard();
121
+ if (allowed && matches(row, page.engine)) yield row;
120
122
  }
121
123
  guard();
122
124
  if (!page.next) break;
123
125
  after = { position: page.next.position, id: page.next.id };
124
126
  }
127
+ };
128
+ const readPermitted = async <T>(read: (window: ReadWindow) => { rows: T[]; next?: ReadRef; engine?: "fts5" | "substring" },
129
+ options: Omit<ReadWindow, "project" | "limit" | "after">, cap: number, matches: (row: T, engine?: "fts5" | "substring") => boolean = () => true): Promise<T[]> => {
130
+ const result: T[] = [];
131
+ for await (const row of permittedRows(read, options, matches)) { result.push(row); if (result.length === cap) break; }
125
132
  return result;
126
133
  };
127
134
  const anchorAt = async (id: string): Promise<string | undefined> => {
@@ -180,41 +187,46 @@ export function createMemoryMcp(options: MemoryMcpOptions): MemoryMcp {
180
187
  } finally { index.close(); }
181
188
  },
182
189
  async amem_search(args) {
183
- bound(args, ["query", "limit", "since", "includeCapture"]); const query = textOf(args, "query")!, limit = limitOf(args), since = sinceOf(args);
190
+ bound(args, ["query", "limit", "since", "includeCapture", "explain"]); const query = textOf(args, "query")!, limit = limitOf(args), since = sinceOf(args);
184
191
  if (args.includeCapture !== undefined && typeof args.includeCapture !== "boolean") unavailable();
192
+ if (args.explain !== undefined && typeof args.explain !== "boolean") unavailable();
185
193
  if (!await currentPolicy().screen({ query })) unavailable();
186
- const index = new SearchIndex(":memory:");
194
+ // ponytail: scan all matches into private disposable disk state, keeping one body page in JS.
195
+ // Work scales with matches; a policy-bound cache is the upgrade if this reaches the unchanged watchdog.
196
+ const index = new ScreenedSearchIndex<SearchValue>();
197
+ const releaseIndex = () => index.close(); closers.push(releaseIndex);
187
198
  try {
188
199
  const words = query.toLowerCase().split(/\s+/).filter(Boolean);
189
200
  const captureState: { engine?: "fts5" | "substring" } = {};
190
- const recalls = args.includeCapture === false ? [] : await readPermitted(window => {
191
- const page = capture.readPage({ ...window, query, ...(captureState.engine === "substring" ? { match: "substring" as const } : {}) });
192
- captureState.engine ??= page.engine;
193
- return page;
194
- }, { since }, SEARCH_CANDIDATES,
195
- (row, engine) => index.available && engine === "fts5" || words.every(word => `${row.excerpt}\n${row.paths.join(" ")}\n${row.event}`.toLowerCase().includes(word)));
201
+ let batch: { body: string; ref: string; tier: string; at: string; value: SearchValue; expiresAt?: string | null }[] = [];
202
+ const append = (value: SearchValue) => {
203
+ const row = value.source === "capture" ? value.capture : value.view;
204
+ batch.push(value.source === "capture"
205
+ ? { value, ref: `capture:${value.capture.id}`, at: row.at, tier: "capture", body: `${value.capture.excerpt}\n${value.capture.paths.join(" ")}\n${value.capture.event}` }
206
+ : { value, ref: value.source === "repo" ? `repo:${value.view.id}` : value.view.id, at: row.at, tier: value.source === "repo" ? "long" : value.view.tier, body: `${value.view.title ?? ""}\n${value.view.text}\n${value.view.about.join(" ")}`, expiresAt: value.source === "curated" ? value.view.expiresAt : null });
207
+ if (batch.length === READ_PAGE) { index.append(batch); batch = []; }
208
+ };
209
+ if (args.includeCapture !== false) {
210
+ for await (const row of permittedRows(window => {
211
+ const page = capture.readPage({ ...window, query, ...(captureState.engine === "substring" ? { match: "substring" as const } : {}) });
212
+ captureState.engine ??= page.engine; return page;
213
+ }, { since, order: "id" }, (row, engine) => index.available && engine === "fts5" || words.every(word => `${row.excerpt}\n${row.paths.join(" ")}\n${row.event}`.toLowerCase().includes(word)))) {
214
+ append({ source: "capture", evidence: false, capture: row });
215
+ }
216
+ }
196
217
  const useFts = index.available && captureState.engine !== "substring";
197
- const notes = await readPermitted(window => store.readPage({ ...window, ...(useFts ? { query } : {}) }), { since }, SEARCH_CANDIDATES,
198
- view => recallViewIsLive(view, store.readNow()) && (useFts || words.every(word => `${view.title ?? ""}\n${view.text}\n${view.about.join(" ")}`.toLowerCase().includes(word))));
218
+ for await (const view of permittedRows(window => store.readPage({ ...window, ...(useFts ? { query } : {}) }), { since, order: "id" },
219
+ view => recallViewIsLive(view, store.readNow()) && (useFts || words.every(word => `${view.title ?? ""}\n${view.text}\n${view.about.join(" ")}`.toLowerCase().includes(word))))) append({ source: "curated", evidence: true, view });
199
220
  const repoNotes = (await repoRows()).filter(view => (!since || Date.parse(view.at) >= Date.parse(since)) && (useFts || words.every(word => `${view.title ?? ""}\n${view.text}\n${view.about.join(" ")}`.toLowerCase().includes(word))));
200
- const hits = [...notes.map(view => ({ source: "curated" as const, evidence: true, view, ref: view.id, at: view.at, tier: view.tier, body: `${view.title ?? ""}\n${view.text}\n${view.about.join(" ")}` })),
201
- ...repoNotes.map(view => ({ source: "repo" as const, evidence: false, view, ref: `repo:${view.id}`, at: view.at, tier: "long", body: `${view.title ?? ""}\n${view.text}\n${view.about.join(" ")}` })),
202
- ...recalls.map(row => ({ source: "capture" as const, evidence: false, capture: row, ref: `capture:${row.id}`, at: row.at, tier: "capture", body: `${row.excerpt}\n${row.paths.join(" ")}\n${row.event}` }))].filter(hit => (!since || Date.parse(hit.at) >= Date.parse(since)) && (hit.source !== "curated" || recallViewIsLive(hit.view, store.readNow())));
203
- if (useFts) {
204
- index.resync(() => hits.map(hit => ({ id: hit.ref, at: hit.at, project, tier: hit.tier, scope: "project", body: hit.body })));
205
- if (index.needsResync(hits.length)) unavailable();
206
- }
207
- const ranks = new Map((useFts ? index.query(query, MCP_SCAN_LIMIT * 2, project) : []).map(row => [row.ref, row.rank]));
208
- const selected = hits.flatMap(hit => {
209
- const rank = ranks.get(hit.ref);
210
- if (useFts ? rank === undefined : !words.every(word => hit.body.toLowerCase().includes(word))) return [];
211
- const weight = hit.tier === "long" ? 1 : hit.tier === "capture" ? 0.5 : 0.9;
212
- const score = -(rank ?? -1) * weight / (1 + Math.max(0, Date.now() - Date.parse(hit.at)) / (30 * 86400_000));
213
- return [{ source: hit.source, evidence: hit.evidence, ...(hit.source === "capture" ? { capture: hit.capture } : { view: hit.view }), score }];
214
- }).sort((a, b) => b.score - a.score).slice(0, limit);
221
+ for (const view of repoNotes) append({ source: "repo", evidence: false, view });
222
+ if (batch.length) index.append(batch);
223
+ const selected = index.best(query, limit, store.readNow(), useFts, args.explain === true).map(({ value, score, scoreDetails }) => {
224
+ const hit = { ...value, score };
225
+ return scoreDetails ? { ...hit, scoreDetails: { source: value.source, ...scoreDetails } } : hit;
226
+ });
215
227
  const probe = stale(), result = { engine: useFts ? "fts5" : "substring", hits: selected.map(hit => hit.source === "curated" && "view" in hit && hit.view ? { ...hit, view: decorate(hit.view as View, probe) } : hit) };
216
228
  guard(); return observed(result, selected.flatMap(hit => hit.source === "curated" && "view" in hit && hit.view ? [hit.view.id] : []), selected.map(hit => "view" in hit ? hit.view : hit.capture));
217
- } finally { index.close(); }
229
+ } finally { const position = closers.indexOf(releaseIndex); if (position >= 0) closers.splice(position, 1); releaseIndex(); }
218
230
  },
219
231
  async amem_show(args) {
220
232
  bound(args, ["id"]); guard(); const id = textOf(args, "id")!;
package/src/mcp.ts CHANGED
@@ -9,7 +9,7 @@ const tool = <Name extends string>(name: Name, description: string, inputSchema:
9
9
 
10
10
  /** Storage scope comes from startup, never tool arguments; candidate proposals remain person-reviewed. */
11
11
  export const MCP_TOOLS = [
12
- tool("amem_search", "Search screened memory in this project. Capture results are recall, not curated evidence.", object({ query: string(1000), limit, since: string(40), includeCapture: { type: "boolean" } }, ["query"])),
12
+ tool("amem_search", "Search screened memory in this project. Capture results are recall, not curated evidence.", object({ query: string(1000), limit, since: string(40), includeCapture: { type: "boolean" }, explain: { type: "boolean" } }, ["query"])),
13
13
  tool("amem_evidence", "Return a bounded bundle of cited, live curated entries for a question. Untrusted memory data only: no answer or model call. Capture, repo-only notes, pending candidates and model answers are not evidence.", object({ question: string(1000), limit: { type: "integer", minimum: 1, maximum: 20 }, maxBytes: { type: "integer", minimum: 256, maximum: 8192 } }, ["question"])),
14
14
  tool("amem_show", "Show a screened memory entry in this project by its exact id or unique prefix.", object({ id: string(80) }, ["id"])),
15
15
  tool("amem_timeline", "Read screened memory near an entry or time in this project.", object({ id: string(80), since: string(40), limit })),