claude-db 0.5.2 → 0.5.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,93 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.5.4
4
+
5
+ ### Fixed
6
+
7
+ - **`reembed` could exhaust memory on a large database.** ([#7](https://github.com/Avijit07x/claude-db/issues/7))
8
+ Embedding walked the store 500 rows at a time and handed all 500 to a single
9
+ `embedder.embed()` call, which became one forward pass over 500 sequences in
10
+ the local model. Peak memory was set by that batch, so a database whose rows
11
+ were long enough went to swap and never came back. Embedding is now chunked by
12
+ `embeddings.batchSize`, so peak memory is a property of the chunk rather than
13
+ of the database: re-embedding 1,400 observations holds ~0.8 GB and takes the
14
+ same time per row as re-embedding 128. The chunking lives in
15
+ `embedObservations`, the one call every path routes through, so `flush`,
16
+ `seed` and `manual` capture are bounded by the same change — `seed` had the
17
+ identical ceiling and would have hit it on a large enough repo.
18
+
19
+ ### Added
20
+
21
+ - **`embeddings.batchSize`, default 8.** Measured on `all-MiniLM-L6-v2` with
22
+ worst-case 512-token observations, peak RSS runs 0.8 GB at 8, 1.3 GB at 16,
23
+ 2.4 GB at 32 and 4.5 GB at 64 — while wall-clock stays flat from 1 to 8,
24
+ because CPU inference is compute-bound and the token count is the same however
25
+ it is grouped. Batching buys no speed here and only costs memory, so the
26
+ default is deliberately small; lower it further if you are tight on RAM.
27
+
28
+ - **`reembed --project`.** `reembed` re-embedded every project on the machine
29
+ with no way to narrow it. On a database holding several projects that is the
30
+ difference between minutes and seconds, and the work skipped is work that
31
+ never has to be bounded. Without the flag the behaviour is unchanged.
32
+
33
+ ## 0.5.3
34
+
35
+ ### Fixed
36
+
37
+ - **`timeline` could not accept the ids it was shown.** Every id that reaches a
38
+ caller is shortened to 13 characters for display, and `get_observations`
39
+ resolves those by prefix — but `timeline` matched exactly, in all three
40
+ adapters, and its miss surfaced as the plausible "No matching observations."
41
+ rather than an error. The tool had never worked over MCP. It now resolves the
42
+ anchor the same way `get_observations` does; the smoke test that missed this
43
+ fed it a full id straight from the store, the one input shape no real caller
44
+ produces, and now feeds a short one.
45
+ - **`Class.method()` did not count as a reference to `Class`.** A member call
46
+ produced edges to the method and to the qualified name, never to the object,
47
+ so a class only ever built through a static factory looked unreferenced —
48
+ including every store backend in this project, where `SqliteStore.create(uri)`
49
+ is the sole construction site. A blast-radius check would have reported one
50
+ caller (itself) and a rename on that basis breaks the store factory.
51
+ - **Callers inside anonymous functions were named after a nearby variable.**
52
+ `const ctx = await createContext()` attributed the call to `ctx`, and
53
+ `const result = await flushSession(...)` to `result` — assignment targets, not
54
+ callers, and every hook entry point in this codebase is an anonymous callback.
55
+ A variable declared on the line making the call is no longer treated as the
56
+ enclosing scope; those sites now name the file, which is at least true.
57
+ - **One call was reported as two edges, and destructuring patterns as symbols.**
58
+ `ctx.store.insertObservations()` appeared once qualified and once bare, and
59
+ `const { turns, nextOffset } = ...` stored the pattern text as a symbol name.
60
+ The two edges are now merged into one, keeping the qualified name for reading
61
+ and the resolved target for traversal — the qualified edge resolves to nothing
62
+ on its own, so dropping the bare one instead would have severed the only
63
+ working link.
64
+
65
+ ### Changed
66
+
67
+ - **Stopped pre-expanding a full observation body on every prompt.** The
68
+ per-prompt block pasted in the top hit's entire body, up to 900 characters,
69
+ before anything had asked for it — layer 3 of a three-layer disclosure design,
70
+ paid unconditionally for a roughly 1-in-9 chance of being used. `expandTop`
71
+ now defaults to `0`, and the index line carries the matched snippet instead,
72
+ which is capped at 110 characters and shows *why* a result matched. Measured
73
+ across four real prompts, the block goes from ~1,560 to ~520 characters, a 67%
74
+ cut, and still names the id to expand on demand. `promptMaxChars` rises 500 ->
75
+ 700 so the budget fits the four results `promptResults` asks for.
76
+
77
+ Existing installs materialise every config value to disk, so a schema default
78
+ alone would have reached nobody. A saved `inject` block still holding both old
79
+ defaults exactly is now treated as untouched and picks up the new ones; anyone
80
+ who tuned either value keeps what they set.
81
+ - **Stopped re-injecting what a session had already been shown.** Every prompt
82
+ got a `<recalled-memory>` block from a fresh search, and because the best
83
+ match for one prompt is usually the best match for the next, the same
84
+ observations were pasted in over and over. Measured across 25 real sessions,
85
+ **56% of everything injected was already in the conversation** — roughly 950
86
+ wasted tokens per session. Each session now tracks which observations it has
87
+ surfaced and skips them; when every match is already in context it injects
88
+ nothing at all, which happened on 28% of prompts in the replay. Nothing is
89
+ lost: the content was in the transcript either way.
90
+
3
91
  ## 0.5.2
4
92
 
5
93
  ### Fixed
package/README.md CHANGED
@@ -6,18 +6,32 @@
6
6
  [![CI](https://github.com/Avijit07x/claude-db/actions/workflows/ci.yml/badge.svg)](https://github.com/Avijit07x/claude-db/actions/workflows/ci.yml)
7
7
  [![license](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](./LICENSE)
8
8
 
9
+ [Documentation](https://claude-db.vercel.app/docs) ·
10
+ [Quick start](https://claude-db.vercel.app/docs/quick-start) ·
11
+ [CLI reference](https://claude-db.vercel.app/docs/cli)
12
+
9
13
  ---
10
14
 
11
- ## What it is
15
+ ## Every session starts from zero
16
+
17
+ You spent an hour yesterday explaining why the store uses three adapters, which
18
+ approach you already tried and abandoned, and why that one function must not be
19
+ touched. Today Claude knows none of it. So you explain it again.
12
20
 
13
- Claude Code forgets everything when a session ends. Next time you open it, you
14
- re-explain the architecture, why you chose X over Y, and what already failed.
21
+ claude-db fixes that, and the other half too. Claude re-derives your codebase
22
+ every time, grepping a name and opening files to work out what calls what.
15
23
 
16
- claude-db records those decisions as you work and gives them back
17
- automatically. Local by default, no account, no subscription.
24
+ ```console
25
+ $ claude-db search "why does capture read the transcript"
18
26
 
19
- It also maps your code: `claude-db scan` builds a graph of every symbol and how
20
- they connect, so "what breaks if I change this" is a query instead of a grep.
27
+ 83cf1246-eb14 decision 2026-08-19 Tested and written up.
28
+ …the old defaults prompt was now cut why does capture read the transcript…
29
+ 88fab24f-5e59 decision 2026-08-19 Finding 4's other half needs checking…
30
+ …cannot work: 0.029483 "why does capture read…" <- highly relevant…
31
+ ```
32
+
33
+ You never run that command. Claude gets the same context injected on every
34
+ prompt, so it walks in already knowing. Nothing leaves your machine.
21
35
 
22
36
  ## Install
23
37
 
@@ -28,27 +42,26 @@ cd your-project
28
42
  claude-db install --project
29
43
  ```
30
44
 
31
- Restart Claude Code. That's it — capture and recall are hooks, so they run
45
+ Restart Claude Code. That's it: capture and recall are hooks, so they run
32
46
  without being asked.
33
47
 
34
48
  `--project` scopes to one repo instead of every project on the machine. It
35
- writes `.claude/settings.local.json` and `.mcp.json` — add `.mcp.json` to your
49
+ writes `.claude/settings.local.json` and `.mcp.json`. Add `.mcp.json` to your
36
50
  `.gitignore`, since it holds a machine-specific path.
37
51
 
38
52
  ## Map your code
39
53
 
54
+ Memory fills itself as you work. The code graph needs one command:
55
+
40
56
  ```bash
41
57
  claude-db scan
42
-
43
- claude-db usages --mode explain resolveProject
44
- claude-db usages --mode path cmdScan observationId
45
58
  ```
46
59
 
47
- Claude gets the same four modes through MCP: `text` (a live grep, no scan
48
- needed), `usages`, `explain` and `path`. Answers re-check the working tree
49
- before replying, so they never report a line the source has moved past.
60
+ Claude then gets four modes through MCP: `text` (a live grep, works with no
61
+ scan at all), `usages`, `explain` and `path`. Re-running `scan` is cheap: it
62
+ hashes files and re-parses only what changed.
50
63
 
51
- Works out of the box on TypeScript, TSX, JavaScript, Python, Go and Rust — the
64
+ Works out of the box on TypeScript, TSX, JavaScript, Python, Go and Rust. The
52
65
  parser ships with the package, so there is nothing else to install.
53
66
 
54
67
  ## Use another database
@@ -64,33 +77,42 @@ claude-db use "postgres://user:pass@host:5432/memory"
64
77
  Install the driver you need (`npm install mongodb` or `pg`); neither ships by
65
78
  default.
66
79
 
80
+ ## With it, and without it
81
+
82
+ Asking _who defines and calls `closeObservations`_ by hand means a grep, then
83
+ opening three store adapters to see which hits are definitions. One `explain`
84
+ call answers it already classified: **2.1x cheaper**, measured across eight
85
+ real symbols in this repo.
86
+
87
+ Recall is the standing cost of ~180 tokens a prompt, skipped entirely on 28% of
88
+ prompts because the answer is already in context:
89
+
90
+ ```
91
+ every prompt costs 180 tokens of recall
92
+ every lookup refunds 597 tokens (1,115 by hand - 518 with)
93
+
94
+ so one lookup pays for 3.3 prompts of recall
95
+ ```
96
+
97
+ Check it on your own repo with `npm run bench:ab` and `npm run bench:tokens`
98
+ from a clone. [With and without](./docs/with-and-without.md) has every number,
99
+ including the symbols where plain grep wins.
100
+
67
101
  ## Commands
68
102
 
69
- `cdb` is a shorter alias for all of them.
70
-
71
- | Command | What it does |
72
- | ------------------------------------------------- | ------------------------------------------------------ |
73
- | `claude-db install [--project]` | Register hooks and the MCP server |
74
- | `claude-db uninstall [--project]` | Remove them, keeping your memory |
75
- | `claude-db status` | Is it wired up, and when did it last record anything |
76
- | `claude-db doctor [--deep]` | Resolved config; `--deep` proves a full round trip |
77
- | `claude-db search [--all] [--tag <name>] <query>` | Search this project's memory, or every project |
78
- | `claude-db remember [--key <name>] <text>` | Record a rule outright, e.g. "always use pnpm here" |
79
- | `claude-db forget <id>` | Delete specific observations by id |
80
- | `claude-db seed --from-git` | Fill a cold memory from this repo's history |
81
- | `claude-db scan [--force]` | Build the code graph for this repo |
82
- | `claude-db usages [--mode <m>] <symbol>` | What uses a symbol: live `git grep`, or the code graph |
83
- | `claude-db stats` | What this project's memory is made of |
84
- | `claude-db projects` | Every project with memory stored |
85
- | `claude-db merge [<path>]` | Move memory from an old project path onto this one |
86
- | `claude-db use <url>` | Switch database and verify it |
87
- | `claude-db sync <url>` | Two-way merge with another database |
88
- | `claude-db flush` | Re-ingest every transcript for this project |
89
- | `claude-db export [--all]` | Dump memory as JSONL, for backup or migration |
90
- | `claude-db import <file>` | Load a dump back in; safe to repeat |
91
- | `claude-db reembed` | Re-embed everything with the current model |
92
- | `claude-db prune --older-than <days>` | Delete old memory (dry run without `--yes`) |
93
- | `claude-db reset [--project] --yes` | Delete memory (dry run without `--yes`) |
103
+ `cdb` is a shorter alias for all of them. These are the ones you actually type:
104
+
105
+ | Command | What it does |
106
+ | ---------------------------------------- | ------------------------------------------------------ |
107
+ | `claude-db install [--project]` | Register hooks and the MCP server |
108
+ | `claude-db status` | Is it wired up, and when did it last record anything |
109
+ | `claude-db doctor [--deep]` | Resolved config; `--deep` proves a full round trip |
110
+ | `claude-db scan [--force]` | Build the code graph for this repo |
111
+ | `claude-db usages [--mode <m>] <symbol>` | What uses a symbol: live `git grep`, or the code graph |
112
+ | `claude-db use <url>` | Switch database and verify it |
113
+
114
+ Every other command, including `search`, `remember`, `seed`, `sync`, `export`,
115
+ `prune` and `reset`, is in the [CLI reference](https://claude-db.vercel.app/docs/cli).
94
116
 
95
117
  Install also adds a `/cdb-scan` skill: run it once on an existing project and
96
118
  Claude maps the codebase into memory, so search has something to find before
@@ -113,16 +135,23 @@ before anything is written.
113
135
 
114
136
  ## Documentation
115
137
 
116
- - [How it works](./docs/how-it-works.md) — what gets captured and injected, how
117
- the code graph is built and kept current, and how the database backends differ
118
- - [Changelog](./CHANGELOG.md)
138
+ **[claude-db.vercel.app/docs](https://claude-db.vercel.app/docs)** has the rest:
139
+ the [CLI reference](https://claude-db.vercel.app/docs/cli),
140
+ [MCP tools](https://claude-db.vercel.app/docs/mcp-tools),
141
+ [how it works](https://claude-db.vercel.app/docs/how-it-works),
142
+ [databases](https://claude-db.vercel.app/docs/databases) and
143
+ [troubleshooting](https://claude-db.vercel.app/docs/troubleshooting).
144
+
145
+ [Benchmarks](https://claude-db.vercel.app/docs/benchmarks) has every number,
146
+ including the symbols where plain grep wins. Releases are in the
147
+ [changelog](./CHANGELOG.md).
119
148
 
120
149
  ## Development
121
150
 
122
151
  ```bash
123
152
  npm install
124
153
  npm run build
125
- npm test # 263 checks
154
+ npm test # 326 checks
126
155
  npm run try # simulate a session, touches nothing
127
156
  npm run format
128
157
  ```
@@ -36,14 +36,22 @@ export async function embedObservations(ctx, observations) {
36
36
  const embedder = await ctx.embedder();
37
37
  if (embedder.dimensions === 0)
38
38
  return;
39
- const vectors = await embedder.embed(observations.map((obs) => `${obs.title}\n${obs.body}`));
40
- observations.forEach((obs, index) => {
41
- const vector = vectors[index];
42
- if (vector && vector.length > 0) {
43
- obs.embedding = vector;
44
- obs.embedder = embedder.id;
45
- }
46
- });
39
+ // ponytail: chunked because peak memory scales with the batch handed to one
40
+ // forward pass, not with the database. Measured on all-MiniLM-L6-v2, 512-token
41
+ // bodies: 8 -> 0.8GB, 16 -> 1.3GB, 32 -> 2.4GB, 64 -> 4.5GB, all at identical
42
+ // wall-clock. Batching buys nothing on CPU, so keep it small.
43
+ const size = ctx.config.embeddings.batchSize;
44
+ for (let start = 0; start < observations.length; start += size) {
45
+ const chunk = observations.slice(start, start + size);
46
+ const vectors = await embedder.embed(chunk.map((obs) => `${obs.title}\n${obs.body}`));
47
+ chunk.forEach((obs, index) => {
48
+ const vector = vectors[index];
49
+ if (vector && vector.length > 0) {
50
+ obs.embedding = vector;
51
+ obs.embedder = embedder.id;
52
+ }
53
+ });
54
+ }
47
55
  }
48
56
  catch (error) {
49
57
  process.stderr.write(`[claude-db] embedding skipped: ` +
@@ -96,6 +104,7 @@ export function resetCursor(sessionId) {
96
104
  }
97
105
  export function clearCursor(sessionId) {
98
106
  rmSync(cursorPath(sessionId), { force: true });
107
+ rmSync(cursorPath(sessionId).replace(/\.offset$/, '.shown'), { force: true });
99
108
  }
100
109
  export function sweepCursors() {
101
110
  const dir = join(CONFIG_DIR, 'cursors');
@@ -106,10 +115,11 @@ export function sweepCursors() {
106
115
  catch {
107
116
  return 0;
108
117
  }
109
- const live = new Set(sessionIdsOnDisk().map(cursorName));
118
+ const live = new Set(sessionIdsOnDisk().map((id) => id.replace(/[^\w-]/g, '_')));
110
119
  let removed = 0;
111
120
  for (const file of files) {
112
- if (!file.endsWith('.offset') || live.has(file))
121
+ const match = /^(.+)\.(offset|shown)$/.exec(file);
122
+ if (!match?.[1] || live.has(match[1]))
113
123
  continue;
114
124
  rmSync(join(dir, file), { force: true });
115
125
  removed += 1;
@@ -6,4 +6,4 @@ export declare function eachObservation(ctx: RecallContext, filter: {
6
6
  export declare function cmdExport(argv: (string | undefined)[]): Promise<void>;
7
7
  export declare function cmdImport(path: string | undefined): Promise<void>;
8
8
  export declare function cmdPrune(argv: (string | undefined)[]): Promise<void>;
9
- export declare function cmdReembed(): Promise<void>;
9
+ export declare function cmdReembed(argv?: (string | undefined)[]): Promise<void>;
@@ -98,7 +98,8 @@ export async function cmdPrune(argv) {
98
98
  await ctx.close();
99
99
  }
100
100
  }
101
- export async function cmdReembed() {
101
+ export async function cmdReembed(argv = []) {
102
+ const scoped = argv.includes('--project') || argv.includes('-p');
102
103
  const base = loadConfig();
103
104
  const ctx = await createContext({ embeddings: { ...base.embeddings, timeoutMs: 0 } });
104
105
  try {
@@ -109,7 +110,8 @@ export async function cmdReembed() {
109
110
  }
110
111
  let updated = 0;
111
112
  let skipped = 0;
112
- const scanned = await eachObservation(ctx, {}, async (batch) => {
113
+ const filter = scoped ? { project: resolveProject(undefined) } : {};
114
+ const scanned = await eachObservation(ctx, filter, async (batch) => {
113
115
  const stale = batch.filter((obs) => obs.embedder !== embedder.id);
114
116
  skipped += batch.length - stale.length;
115
117
  if (stale.length === 0)
package/dist/cli/index.js CHANGED
@@ -57,7 +57,7 @@ async function run() {
57
57
  await cmdPrune(args);
58
58
  break;
59
59
  case 'reembed':
60
- await cmdReembed();
60
+ await cmdReembed(args);
61
61
  break;
62
62
  case 'stats':
63
63
  await cmdStats();
package/dist/cli/usage.js CHANGED
@@ -25,7 +25,7 @@ export function usage() {
25
25
  import <file.jsonl> Load a dump back in (safe to repeat)
26
26
  sync <url> [--yes] Two-way merge with another database
27
27
  update Install a newer compatible release now
28
- reembed Re-embed everything with the current model
28
+ reembed [--project] Re-embed with the current model
29
29
  prune --older-than <days> Delete old memory (dry run without --yes)
30
30
  reset [--project] --yes Delete stored memory (dry run without --yes)
31
31
 
@@ -5,13 +5,27 @@ import { ConfigSchema } from './schema.js';
5
5
  export const CONFIG_DIR = join(homedir(), '.claude-memory');
6
6
  export const CONFIG_PATH = join(CONFIG_DIR, 'config.json');
7
7
  export const DEFAULT_DB_PATH = join(CONFIG_DIR, 'memory.db');
8
+ const SUPERSEDED_INJECT = { expandTop: 1, promptMaxChars: 500 };
9
+ function dropSupersededDefaults(raw) {
10
+ const file = raw;
11
+ const inject = file?.inject;
12
+ if (!inject)
13
+ return raw;
14
+ const untouched = Object.entries(SUPERSEDED_INJECT).every(([key, value]) => inject[key] === value);
15
+ if (!untouched)
16
+ return raw;
17
+ const trimmed = { ...inject };
18
+ for (const key of Object.keys(SUPERSEDED_INJECT))
19
+ delete trimmed[key];
20
+ return { ...file, inject: trimmed };
21
+ }
8
22
  export function loadConfig() {
9
23
  let fileConfig = {};
10
24
  try {
11
25
  fileConfig = JSON.parse(readFileSync(CONFIG_PATH, 'utf8'));
12
26
  }
13
27
  catch { }
14
- const config = ConfigSchema.parse(fileConfig);
28
+ const config = ConfigSchema.parse(dropSupersededDefaults(fileConfig));
15
29
  const envUrl = process.env['CLAUDE_DB_URL']?.trim();
16
30
  return {
17
31
  ...config,
@@ -1,7 +1,7 @@
1
1
  import { z } from 'zod';
2
2
  export declare const ConfigSchema: z.ZodObject<{
3
3
  database: z.ZodDefault<z.ZodString>;
4
- embeddings: z.ZodDefault<z.ZodObject<{
4
+ embeddings: z.ZodPrefault<z.ZodObject<{
5
5
  provider: z.ZodDefault<z.ZodEnum<{
6
6
  auto: "auto";
7
7
  local: "local";
@@ -10,6 +10,7 @@ export declare const ConfigSchema: z.ZodObject<{
10
10
  }>>;
11
11
  maxScanCandidates: z.ZodDefault<z.ZodNumber>;
12
12
  timeoutMs: z.ZodDefault<z.ZodNumber>;
13
+ batchSize: z.ZodDefault<z.ZodNumber>;
13
14
  }, z.core.$strip>>;
14
15
  updates: z.ZodDefault<z.ZodEnum<{
15
16
  auto: "auto";
@@ -6,8 +6,9 @@ export const ConfigSchema = z.object({
6
6
  provider: z.enum(['auto', 'local', 'builtin', 'none']).default('auto'),
7
7
  maxScanCandidates: z.number().int().positive().default(25000),
8
8
  timeoutMs: z.number().int().nonnegative().default(3000),
9
+ batchSize: z.number().int().positive().default(8),
9
10
  })
10
- .default({ provider: 'auto', maxScanCandidates: 25000, timeoutMs: 3000 }),
11
+ .prefault({}),
11
12
  updates: z.enum(['auto', 'notify', 'off']).default('notify'),
12
13
  capture: z
13
14
  .object({
@@ -22,8 +23,8 @@ export const ConfigSchema = z.object({
22
23
  maxChars: z.number().int().positive().default(6000),
23
24
  perPrompt: z.boolean().default(true),
24
25
  promptResults: z.number().int().positive().max(10).default(4),
25
- promptMaxChars: z.number().int().positive().default(500),
26
- expandTop: z.number().int().min(0).max(3).default(1),
26
+ promptMaxChars: z.number().int().positive().default(700),
27
+ expandTop: z.number().int().min(0).max(3).default(0),
27
28
  expandMaxChars: z.number().int().positive().default(900),
28
29
  })
29
30
  .prefault({}),
@@ -7,6 +7,7 @@ const SHARED_DEFINITIONS = [
7
7
  const SHARED_REFERENCES = [
8
8
  { kind: 'call_expression', field: ['function'], relation: 'calls' },
9
9
  { kind: 'call_expression', field: ['function', 'property'], relation: 'calls' },
10
+ { kind: 'call_expression', field: ['function', 'object'], relation: 'references' },
10
11
  { kind: 'new_expression', field: ['constructor'], relation: 'calls' },
11
12
  { kind: 'import_statement', field: ['source'], relation: 'imports' },
12
13
  { kind: 'export_statement', field: ['source'], relation: 'imports' },
@@ -2,6 +2,7 @@ import { observationId } from '../../capture/identity.js';
2
2
  import { redact } from '../../capture/turn-extractor.js';
3
3
  import { languageHandle, loadParser } from '../parser.js';
4
4
  const CALLABLE = new Set(['function', 'method', 'class']);
5
+ const IDENTIFIER = /^[\w$]+$/;
5
6
  export function symbolId(project, file, name, kind) {
6
7
  return observationId('graph', 0, `${project}\0${file}\0${name}\0${kind}`);
7
8
  }
@@ -27,10 +28,12 @@ function enclosing(line, spans) {
27
28
  if (CALLABLE.has(span.symbol.kind)) {
28
29
  if (!best || width < best.end - best.start)
29
30
  best = span;
31
+ continue;
30
32
  }
31
- else if (!fallback || width < fallback.end - fallback.start) {
33
+ if (span.start === line)
34
+ continue;
35
+ if (!fallback || width < fallback.end - fallback.start)
32
36
  fallback = span;
33
- }
34
37
  }
35
38
  return (best ?? fallback)?.symbol ?? null;
36
39
  }
@@ -44,7 +47,7 @@ export function extractFile(file, project) {
44
47
  for (const node of root.findAll({ rule: { kind: rule.kind } })) {
45
48
  const named = resolveField(node, rule.field);
46
49
  const name = named?.text();
47
- if (!named || !name)
50
+ if (!named || !name || !IDENTIFIER.test(name))
48
51
  continue;
49
52
  const line = named.range().start.line + 1;
50
53
  const symbol = {
@@ -87,6 +90,8 @@ export function extractFile(file, project) {
87
90
  const name = unquote(raw.trim());
88
91
  if (!name || /\s/.test(name))
89
92
  continue;
93
+ if (rule.relation === 'references' && name.includes('.'))
94
+ continue;
90
95
  const line = target.range().start.line + 1;
91
96
  references.push({
92
97
  file: file.path,
@@ -44,5 +44,38 @@ export function resolveEdges(project, symbols, references) {
44
44
  line: reference.line,
45
45
  });
46
46
  }
47
- return edges;
47
+ return mergeMemberCalls(edges);
48
+ }
49
+ function mergeMemberCalls(edges) {
50
+ const byLine = new Map();
51
+ for (const edge of edges) {
52
+ const key = `${edge.file}\0${edge.line}`;
53
+ const bucket = byLine.get(key);
54
+ if (bucket)
55
+ bucket.push(edge);
56
+ else
57
+ byLine.set(key, [edge]);
58
+ }
59
+ const merged = new Set();
60
+ for (const bucket of byLine.values()) {
61
+ for (const qualified of bucket) {
62
+ const dot = qualified.dstName.lastIndexOf('.');
63
+ if (dot < 0)
64
+ continue;
65
+ const tail = qualified.dstName.slice(dot + 1);
66
+ for (const bare of bucket) {
67
+ if (bare === qualified)
68
+ continue;
69
+ if (bare.dstName !== tail || bare.relation !== qualified.relation)
70
+ continue;
71
+ if (!qualified.dstId && bare.dstId) {
72
+ qualified.dstId = bare.dstId;
73
+ qualified.confidence = bare.confidence;
74
+ qualified.score = bare.score;
75
+ }
76
+ merged.add(bare.id);
77
+ }
78
+ }
79
+ }
80
+ return edges.filter((edge) => !merged.has(edge.id));
48
81
  }
@@ -14,7 +14,8 @@ export function renderPromptContext(entries, maxChars, expanded = [], expandMaxC
14
14
  if (expandedIds.has(entry.id))
15
15
  continue;
16
16
  const date = new Date(entry.createdAt).toISOString().slice(0, 7);
17
- const line = `${toShortId(entry.id)} ${entry.kind} ${date} ${entry.title}`;
17
+ const head = `${toShortId(entry.id)} ${entry.kind} ${date} ${entry.title}`;
18
+ const line = entry.snippet ? `${head}\n ${entry.snippet}` : head;
18
19
  if (line.length > budget)
19
20
  break;
20
21
  budget -= line.length + 1;
@@ -0,0 +1,2 @@
1
+ export declare function readShown(sessionId: string): Set<string>;
2
+ export declare function markShown(sessionId: string, ids: string[]): void;
@@ -0,0 +1,26 @@
1
+ import { mkdirSync, readFileSync, writeFileSync } from 'node:fs';
2
+ import { dirname, join } from 'node:path';
3
+ import { CONFIG_DIR } from '../config/index.js';
4
+ const LIMIT = 300;
5
+ function shownPath(sessionId) {
6
+ return join(CONFIG_DIR, 'cursors', `${sessionId.replace(/[^\w-]/g, '_')}.shown`);
7
+ }
8
+ export function readShown(sessionId) {
9
+ try {
10
+ return new Set(readFileSync(shownPath(sessionId), 'utf8').split('\n').filter(Boolean));
11
+ }
12
+ catch {
13
+ return new Set();
14
+ }
15
+ }
16
+ export function markShown(sessionId, ids) {
17
+ if (ids.length === 0)
18
+ return;
19
+ const path = shownPath(sessionId);
20
+ const kept = [...readShown(sessionId), ...ids].slice(-LIMIT);
21
+ try {
22
+ mkdirSync(dirname(path), { recursive: true });
23
+ writeFileSync(path, `${[...new Set(kept)].join('\n')}\n`, 'utf8');
24
+ }
25
+ catch { }
26
+ }
@@ -2,6 +2,7 @@
2
2
  import { flushSession } from '../capture/index.js';
3
3
  import { createContext } from '../context.js';
4
4
  import { emitContext, readPayload, runHook } from './payload.js';
5
+ import { markShown, readShown } from './shown.js';
5
6
  import { renderPromptContext } from './relevance.js';
6
7
  import { isSearchable } from '../util/prompt.js';
7
8
  import { resolveProject } from '../util/project.js';
@@ -20,18 +21,24 @@ await runHook(async () => {
20
21
  const prompt = payload.prompt ?? '';
21
22
  if (!ctx.config.inject.perPrompt || !isSearchable(prompt))
22
23
  return;
23
- const entries = await ctx.search.search({
24
+ const shown = readShown(sessionId);
25
+ const found = await ctx.search.search({
24
26
  text: prompt,
25
27
  project,
26
28
  limit: ctx.config.inject.promptResults,
27
29
  });
30
+ const entries = found.filter((entry) => !shown.has(entry.id));
31
+ if (entries.length === 0)
32
+ return;
28
33
  const toExpand = entries.slice(0, ctx.config.inject.expandTop);
29
34
  const expanded = toExpand.length > 0
30
35
  ? await ctx.search.getObservations(toExpand.map((entry) => entry.id))
31
36
  : [];
32
37
  const block = renderPromptContext(entries, ctx.config.inject.promptMaxChars, expanded, ctx.config.inject.expandMaxChars);
33
- if (block)
34
- emitContext(`${block}\n`);
38
+ if (!block)
39
+ return;
40
+ markShown(sessionId, entries.map((entry) => entry.id));
41
+ emitContext(`${block}\n`);
35
42
  }
36
43
  finally {
37
44
  await ctx.close();
@@ -38,7 +38,10 @@ export class SearchService {
38
38
  }
39
39
  }
40
40
  async timeline(query) {
41
- return this.store.timeline(query);
41
+ const [anchor] = await this.store.getObservations([query.observationId]);
42
+ if (!anchor)
43
+ return [];
44
+ return this.store.timeline({ ...query, observationId: anchor.id });
42
45
  }
43
46
  async getObservations(ids) {
44
47
  return this.store.getObservations(ids);
@@ -6,7 +6,7 @@ graph stays honest.
6
6
 
7
7
  ## Capture
8
8
 
9
- Claude Code already writes every session to disk as JSONL — your prompts, its
9
+ Claude Code already writes every session to disk as JSONL: your prompts, its
10
10
  replies, every tool call. claude-db reads that file, so it captures **why** you
11
11
  did something, not just which files changed.
12
12
 
@@ -41,7 +41,7 @@ best match in full on each prompt. Roughly 350 tokens when something relevant
41
41
  exists, zero when it doesn't.
42
42
 
43
43
  **What search returns.** An id, kind, date, title and one line of the matching
44
- body — enough to tell two similarly-titled rows apart without expanding either.
44
+ body, enough to tell two similarly-titled rows apart without expanding either.
45
45
  Full bodies come only from `get_observations`, for ids you picked.
46
46
 
47
47
  One stored memory looks like this:
@@ -71,8 +71,8 @@ already landed, or a finished task would flicker back into the list every time
71
71
  a neighbouring line changed.
72
72
 
73
73
  `claude-db status` lists what is still open, and SessionStart injects the
74
- newest few. That is what lets a brand-new chat answer "what was I doing" —
75
- memory search ranks by relevance, and a prompt like "do the last task" contains
74
+ newest few. That is what lets a brand-new chat answer "what was I doing".
75
+ Memory search ranks by relevance, and a prompt like "do the last task" contains
76
76
  no words worth matching, so recency has to come from somewhere else.
77
77
 
78
78
  Rows captured before this existed default to `done`. Marking a whole existing
@@ -82,7 +82,7 @@ database unfinished would be worse than saying nothing.
82
82
 
83
83
  `claude-db install` copies two things to disk: the `/cdb-scan` skill, and a
84
84
  standing instruction block in `CLAUDE.local.md`. Claude Code only reads skills
85
- from `~/.claude/skills/`, so a copy has to exist there — but a copy drifts as
85
+ from `~/.claude/skills/`, so a copy has to exist there, but a copy drifts as
86
86
  soon as the package updates.
87
87
 
88
88
  The hooks are registered by absolute path into the installed package, so they
@@ -97,7 +97,7 @@ back, and anything `uninstall` removed stays removed.
97
97
  ## The code graph
98
98
 
99
99
  `claude-db scan` parses every supported source file and stores what it finds:
100
- each symbol, and each relationship between symbols — what calls what, what
100
+ each symbol, and each relationship between symbols: what calls what, what
101
101
  imports what, what extends what. Parsing is local and deterministic, costs no
102
102
  tokens, and takes about a second on a mid-size repository. A rescan only
103
103
  re-parses files whose contents changed.
@@ -117,7 +117,7 @@ Shortest path (4 hops):
117
117
  ```
118
118
 
119
119
  **Every edge carries its confidence.** `EXTRACTED` means the relationship was
120
- read literally out of the syntax tree — an import names its own target, so
120
+ read literally out of the syntax tree. An import names its own target, so
121
121
  nothing is guessed. `INFERRED` means the target was matched by name across
122
122
  files, which is wrong when two files export the same identifier, and carries a
123
123
  score saying how sure the match was. Name matching is the one guess this makes,
@@ -131,7 +131,7 @@ have been scanned, but correctness does not depend on it having run.
131
131
 
132
132
  Traversal is keyed on symbol id rather than name. Keying by name would merge
133
133
  every same-named symbol into one node, and a repository with a `run` in each
134
- test file would then grow shortcuts between unrelated code — making the
134
+ test file would then grow shortcuts between unrelated code, making the
135
135
  shortest path a route nothing can actually take.
136
136
 
137
137
  Languages: TypeScript, TSX, JavaScript, Python, Go and Rust. The parser is
@@ -141,8 +141,8 @@ nothing compiles at install and nothing else has to be installed.
141
141
  ## The `/cdb-scan` skill
142
142
 
143
143
  Installed alongside the hooks, it runs two passes: `claude-db scan` for the
144
- graph, then five written notes — stack, layout, conventions, workflows,
145
- architecture — stored under stable keys so a re-run updates them in place.
144
+ graph, then five written notes (stack, layout, conventions, workflows,
145
+ architecture) stored under stable keys so a re-run updates them in place.
146
146
 
147
147
  The graph records what the code _is_; the notes record why it is built that
148
148
  way. Those notes are tagged `inferred`, because they are Claude's reading of
@@ -154,7 +154,7 @@ database is testimony, and a search result must not blur the two.
154
154
  SQLite by default, at `~/.claude-memory/memory.db`. The connection string picks
155
155
  the backend, and `CLAUDE_DB_URL` overrides the config file.
156
156
 
157
- Search is hybrid — keyword plus vector — on all three backends. Ranking, fusion
157
+ Search is hybrid, keyword plus vector, on all three backends. Ranking, fusion
158
158
  and token budgeting live in `src/search` rather than in the adapters, so recall
159
159
  behaves identically no matter which database is plugged in; only retrieval cost
160
160
  changes. Memory is partitioned by project path, so one database serves every
@@ -179,5 +179,5 @@ Atlas indexes are not created by this tool. Add one by hand on the
179
179
  ```
180
180
 
181
181
  `numDimensions` is 256 for the builtin embedder, 384 once
182
- `@xenova/transformers` is installed. Without this index Mongo still works —
182
+ `@xenova/transformers` is installed. Without this index Mongo still works,
183
183
  vector search falls back to scoring candidates in process.
@@ -0,0 +1,123 @@
1
+ # With claude-db and without it
2
+
3
+ Does it actually save tokens? Here is the same work measured both ways, so you
4
+ can decide before you install.
5
+
6
+ Run `npm run bench:ab` from a clone to reproduce all of it on your own
7
+ codebase. Tokens are approximated at 4 characters each.
8
+
9
+ ## The short answer
10
+
11
+ **One lookup pays for about three prompts of recall.**
12
+
13
+ Recall costs ~180 tokens on every prompt. A symbol lookup saves ~600 tokens
14
+ against doing the same work by hand. Ask your codebase something once every few
15
+ prompts and you are ahead. Never ask it anything and you are paying for context
16
+ you did not use.
17
+
18
+ ## Looking up a symbol
19
+
20
+ The question is _who uses this, and how?_ Without claude-db that means a grep,
21
+ then opening the file it points into to see which hits are definitions, which
22
+ are calls, and which are noise. With it, one command returns the same answer
23
+ already classified.
24
+
25
+ Measured on eight real symbols in this repository:
26
+
27
+ ```
28
+ symbol WITH grep +read WITHOUT
29
+ ---------------------------------------------------------
30
+ isSearchable 1734 1228 1153 2381
31
+ observationsFromTurns 2042 2362 7529 9891
32
+ closeObservations 3209 1354 6597 7951
33
+ flushSession 1786 818 4442 5260
34
+ refreshGraph 2640 1143 961 2104
35
+ observationId 2020 2418 572 2990
36
+ toSnippet 1994 1789 704 2493
37
+ openWork 1166 1294 1305 2599
38
+ ---------------------------------------------------------
39
+ TOTAL chars 16591 35669
40
+ TOTAL tokens 4148 8917
41
+ 2.1x
42
+ ```
43
+
44
+ **518 tokens per lookup instead of 1,115.**
45
+
46
+ The average hides something useful. `refreshGraph` and `observationId` are
47
+ cheaper _without_ claude-db, because their defining files are small and grep
48
+ answers them fine. `observationsFromTurns` and `closeObservations` are four to
49
+ five times cheaper with it, because answering those by hand means reading a
50
+ 200-line file, or three of them spread across different modules.
51
+
52
+ The win concentrates on questions that span files, which is also where a wrong
53
+ answer costs you the most.
54
+
55
+ ## Asking why something is the way it is
56
+
57
+ Some questions have no grep equivalent at all. _Why does capture read the
58
+ transcript instead of hooking the tools?_ leaves no trace in the code. The
59
+ closest substitute is `git log -S` plus reading diffs, and that tells you what
60
+ changed, never why.
61
+
62
+ Measured against that substitute the two come out roughly level, but they are
63
+ not answering the same question. The real cost without stored memory is
64
+ explaining it to Claude again yourself, which no benchmark can price.
65
+
66
+ ## What it costs
67
+
68
+ Recall arrives automatically on every prompt, so it has a standing cost. What
69
+ arrives is index lines, never full bodies: an id, a kind, a date, a title, and
70
+ the line that matched. Claude pulls a full body only when a title earns it.
71
+
72
+ ```
73
+ every prompt costs 180 tokens of recall
74
+ every lookup refunds 597 tokens (1,115 by hand, 518 with)
75
+
76
+ so one lookup pays for 3.3 prompts of recall
77
+ ```
78
+
79
+ On 28% of prompts nothing is injected at all, because everything matching is
80
+ already in the conversation.
81
+
82
+ ```
83
+ a session of recall costs lookups to break even
84
+ ------------------------------------------------------
85
+ 1 prompt 180 0.3
86
+ 5 prompts 900 1.5
87
+ 10 prompts 1,800 3.0
88
+ 20 prompts 3,600 6.0
89
+ 60 prompts 10,800 18.1
90
+ ```
91
+
92
+ ## When it does not pay off
93
+
94
+ Worth saying plainly, because it is a real case: a session where you never ask
95
+ anything about the codebase pays for recall and gets nothing back for it.
96
+
97
+ Three things push you the other way. Refactoring, where every rename wants a
98
+ blast-radius check. Returning to a project after a break, where the stored
99
+ reasoning is the point. And any codebase large enough that "who calls this"
100
+ means opening several files rather than one.
101
+
102
+ If your sessions are short and mostly conversational, turn the per-prompt
103
+ recall off and keep the code graph:
104
+
105
+ ```json
106
+ { "inject": { "perPrompt": false } }
107
+ ```
108
+
109
+ Session-start context still arrives, lookups still work, and the standing cost
110
+ goes to zero.
111
+
112
+ ## Reproducing this
113
+
114
+ ```bash
115
+ git clone https://github.com/Avijit07x/claude-db && cd claude-db
116
+ npm install && npm run build
117
+
118
+ npm run bench:ab # this document, on your own repo
119
+ npm run bench:tokens # where the injected tokens go
120
+ ```
121
+
122
+ The grep side excludes markdown, so both sides see the same code rather than
123
+ counting prose that happens to mention a symbol name.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "claude-db",
3
- "version": "0.5.2",
3
+ "version": "0.5.4",
4
4
  "description": "Persistent memory for Claude Code. Captures why decisions were made, not just which files changed. Bring your own database: SQLite, MongoDB or Postgres. No cloud, no subscription.",
5
5
  "license": "Apache-2.0",
6
6
  "type": "module",
@@ -35,6 +35,8 @@
35
35
  "local": "node scripts/local.mjs",
36
36
  "try": "node scripts/try.mjs",
37
37
  "bench": "node scripts/bench.mjs",
38
+ "bench:tokens": "node scripts/bench-tokens.mjs",
39
+ "bench:ab": "node scripts/bench-ab.mjs",
38
40
  "postinstall": "node -e \"const v=process.versions.node.split('.').map(Number); if(v[0]<22||(v[0]===22&&v[1]<5)){console.error('\\n claude-db needs Node >= 22.5 (node:sqlite is builtin from there). You have '+process.versions.node+'.\\n');process.exit(1)}\"",
39
41
  "prepublishOnly": "npm run build && npm test",
40
42
  "format": "prettier --write .",