nexusmem 0.1.0 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,337 +1,254 @@
1
- # NexusMem
2
-
3
- [![License: MIT](https://img.shields.io/badge/license-MIT-informational)](LICENSE)
4
- ![Node](https://img.shields.io/badge/node-%3E%3D22-brightgreen)
5
-
6
- A local-first persistent memory engine for AI coding agents (Claude Code, Cursor, MCP-based agents).
7
-
8
- AI coding assistants forget context once a session ends, and re-uploading the entire repository as
9
- context on every request is slow and expensive. NexusMem records local machine events — git history,
10
- shell commands, docs, and conversation transcripts into an on-disk SQLite database, returning only
11
- the relevant context slice within a strict token budget.
12
-
13
- All data remains local on your machine. No cloud dependencies, accounts, or telemetry.
14
-
15
- ---
16
-
17
- ## Design Principles
18
-
19
- - **100% Local-First**: SQLite database stored in `.nexusmem/` inside your repository using
20
- `sqlite-vec` and `FTS5`. Works fully offline.
21
- - **Kind-Agnostic Core**: Every source normalizes to a single `MemoryNode` schema, allowing git
22
- commits, shell commands, and documentation to be scored and ranked on an equal basis.
23
- - **Hybrid Search (BM25 + Vector)**: Combines exact keyword matching via SQLite FTS5 (BM25) with
24
- semantic vector search (`sqlite-vec` via a local Ollama model) using Reciprocal Rank Fusion (RRF).
25
- RRF fuses on rank position only, never on raw scores, which is what makes it safe to combine a BM25
26
- cost with a vector distance on an unrelated scale. Degrades gracefully to BM25-only if Ollama is
27
- offline.
28
- - **Ranked, Budgeted Retrieval**: Scores candidates using
29
- `score = relevance × signal^a × recency^b`, then packs nodes into a caller-specified token budget.
30
- Each factor is floored into `[floor, 1]` rather than `[0, 1]`, so no single low factor can zero out
31
- a strong match. The exponents `a` and `b` are derived, not tuned: `relevance` is the only
32
- query-derived factor, so each query-independent prior is raised to the power that caps its entire
33
- range at overturning a relevance gap (`span^exponent = 2`, giving `a ≈ 0.431`, `b ≈ 0.576`).
34
- - **MCP Server Native**: Exposes `search_memory`, `sync_project`, and `get_status` as Model Context
35
- Protocol (MCP) tools over stdio for Claude Desktop, Cursor, and Windsurf.
36
-
37
- ---
38
-
39
- ## Architecture
40
-
41
- ```
42
- git / shell / docs / transcripts
43
-
44
- ▼ collectors/ normalize to one MemoryNode shape
45
-
46
- ▼ store/ SQLite (FTS5 + sqlite-vec)
47
-
48
- ▼ retrieval/ RRF fuse -> rank -> pack to token budget
49
- ```
50
-
51
- ### Key Subsystems
52
-
53
- 1. **Git Collector**: Ingests commits, diff statistics, renames, and conventional commit signals
54
- incrementally via stream iterators. Sync cursors are validated as ancestors of `HEAD` before being
55
- trusted, so a rebase or amend widens the walk instead of silently skipping commits.
56
- 2. **Shell Collector**: Scrapes default history files (`PSReadLine`, `.bash_history`,
57
- `.zsh_history`). An optional PowerShell profile hook upgrades capture to include exact timestamps,
58
- working directories, and exit codes (where failed commands receive a higher structural signal).
59
- 3. **Docs Collector**: Indexes Markdown documentation (`.md` files) tracked by git. Line endings are
60
- normalized to LF before chunking to prevent CRLF splitting failures on Windows. Scoped pruning
61
- removes orphaned sections when headings are renamed or deleted, scoped by project and exact source
62
- so it cannot affect git, shell, or conversation nodes.
63
- 4. **Conversation Collector** (opt-in): Indexes AI assistant transcripts, redacting secrets before
64
- writing to disk. Replies are chunked at heading and bold-lead boundaries rather than stored as
65
- whole exchanges.
66
-
67
- ### Storage Model
68
-
69
- Node ids are content-addressed (`sha256(projectId + kind + naturalKey)`), so running `sync` twice
70
- cannot produce duplicates and ingestion stays correct even if a cursor is lost. Project identity is
71
- derived from the normalized origin URL when one exists, falling back to the absolute path, so two
72
- clones of the same repository share one memory namespace.
73
-
74
- `nodes_fts` is trigger-populated and stays consistent automatically. `nodes_vec` is not computing
75
- an embedding requires an async call to Ollama, which a synchronous SQL trigger cannot make — so it is
76
- filled by an explicit pass after `sync` writes nodes, and a node whose content changes has its stale
77
- embedding dropped for re-embedding.
78
-
79
- ---
80
-
81
- ## Quickstart
82
-
83
- ### Prerequisites
84
-
85
- - Node.js 22 `better-sqlite3` publishes no prebuilt binary below this, so Node 20 would
86
- have to compile the native addon from source (and on Windows that means a full Visual Studio
87
- C++ toolchain). Node 20 also reached end of life in April 2026.
88
- - Git
89
- - Local Ollama instance with an embedding model (optional, for vector search)
90
-
91
- ### Installation
92
-
93
- ```bash
94
- git clone https://github.com/yaminbakoh4-dot/NexusMem.git
95
- cd NexusMem
96
- npm install
97
- npm run build
98
- npm link
99
- ```
100
-
101
- `npm link` puts `nexusmem` on your `PATH`, so it runs against any repository on your machine.
102
-
103
- ### Basic Usage
104
-
105
- Run from any git repository:
106
-
107
- ```bash
108
- nexusmem init
109
- nexusmem sync
110
- nexusmem query "why does the retry logic exist"
111
- ```
112
-
113
- ### Optional: High-Precision Shell Hook
114
-
115
- To capture exact working directory and exit status for shell history:
116
-
117
- ```bash
118
- nexusmem hook install
119
- ```
120
-
121
- This wraps your existing PowerShell prompt rather than replacing it, is idempotent, and is undone
122
- cleanly by `nexusmem hook remove`.
123
-
124
- ---
125
-
126
- ## MCP Server Configuration
127
-
128
- Add the following to your MCP client configuration file:
129
-
130
- ```json
131
- {
132
- "mcpServers": {
133
- "nexusmem": {
134
- "command": "nexusmem",
135
- "args": ["mcp"]
136
- }
137
- }
138
- }
139
- ```
140
-
141
- Available tools:
142
-
143
- | Tool | Description |
144
- | --- | --- |
145
- | `search_memory` | Searches and ranks memory for a given prompt within a token budget. |
146
- | `sync_project` | Runs ingestion and updates embeddings for the specified repository root. |
147
- | `get_status` | Returns current ingestion counts and database state per source. |
148
-
149
- Each tool takes an explicit `projectRoot`, because MCP tool calls carry no implicit shell working
150
- directory. `sync_project` runs `init` first automatically if the repository has not been set up yet.
151
-
152
- ---
153
-
154
- ## Benchmarks & Evaluation
155
-
156
- NexusMem distinguishes between **packer efficiency** (internal packing performance against candidate
157
- sets) and **end-to-end token savings** (real-world savings on the context bill). The two are not
158
- interchangeable, and quoting the first as if it were the second is the specific overclaim this
159
- section exists to prevent.
160
-
161
- ### Packer Efficiency
162
-
163
- Measures how effectively the ranking packer drops low-scoring candidate nodes relative to the raw
164
- candidate body sum within a strict token budget:
165
-
166
- | Scenario | Candidate Corpus | Result |
167
- | --- | --- | --- |
168
- | Fixture repo (tight budget, 3 matches, 1 dropped) | 23 commits | 25% |
169
- | Fixture repo (generous budget, 6 matches, all kept) | 23 commits | -15% (overhead exceeds trim) |
170
- | Core repo design evaluation | 515 nodes | 81% 84% |
171
-
172
- Efficiency is derived from excluding irrelevant low-scoring candidates entirely, not from text
173
- summarization. It increases with corpus size and goes negative on a tiny one, where fixed per-node
174
- formatting overhead outweighs the little there is to trim.
175
-
176
- The baseline it divides by is hypothetical: without NexusMem those candidate bodies would never have
177
- entered the context window at all. This figure is useful for tuning the ranker, not as a claim about
178
- a session's token bill.
179
-
180
- ### End-to-End Token Savings
181
-
182
- Measures packed context size against reading the equivalent full source files into context.
183
-
184
- **Measured result: ~40%** on design queries evaluated against this codebase (reading `README.md` +
185
- `docs/phase-2-spec.md` in full, ~32k chars 8–9k tokens, versus retrieving relevant packed context).
186
- Hand-tallied from one real session, not instrumented treat it as an order-of-magnitude figure.
187
-
188
- **The long-term >70% target is not met at this scale, and this repository cannot demonstrate it.**
189
- The target describes large repositories (thousands of commits) where the win comes from omitting
190
- hundreds of unrelated history items rather than shaving a handful. A benchmark against a repository
191
- of that size is still outstanding.
192
-
193
- One caveat in NexusMem's favour is not a percentage at all: the conversation turns and shell commands
194
- in memory have no cheap `grep` equivalent. Without a collector recording them they are gone, not
195
- merely more expensive to retrieve.
196
-
197
- ### Search Latency
198
-
199
- Measured on this repository's corpus (~530 nodes), warm, p50 over 10 runs:
200
-
201
- | Operation | Latency |
202
- | --- | --- |
203
- | BM25-only retrieval pipeline (FTS5) | ~1.1 ms |
204
- | Vector search (`sqlite-vec` KNN) | ~3.2 ms |
205
- | RRF fuse + rank + pack | ~0.6 ms |
206
- | Query embedding (local Ollama call) | ~55–77 ms |
207
- | End-to-end hybrid retrieval | ~56 ms |
208
-
209
- All SQLite-side work totals roughly 5 ms. The end-to-end figure is dominated by the local embedding
210
- call, which is the only meaningful latency target on this path.
211
-
212
- ---
213
-
214
- ## Command Reference
215
-
216
- | Command | Description |
217
- | --- | --- |
218
- | `nexusmem init` | Initializes `.nexusmem/` directory and SQLite schema. |
219
- | `nexusmem sync` | Ingests new events (git, shell, docs; `--conversation` for transcripts). |
220
- | `nexusmem status` | Prints memory counts per source and database status. |
221
- | `nexusmem query <text>` | Executes hybrid search, ranks, and packs context to stdout. |
222
- | `nexusmem scan-git` | Dry-run preview of git nodes and signal scores without writing to DB. |
223
- | `nexusmem scan-shell` | Dry-run preview of shell history nodes without writing to DB. |
224
- | `nexusmem scan-docs` | Dry-run preview of doc section nodes without writing to DB. |
225
- | `nexusmem scan-conversation` | Dry-run preview of conversation nodes without writing to DB. |
226
- | `nexusmem hook install` | Installs PowerShell profile wrapper for high-precision shell logs. |
227
- | `nexusmem hook remove` | Removes the PowerShell profile wrapper. |
228
- | `nexusmem hook status` | Reports whether the hook is installed. |
229
- | `nexusmem mcp` | Starts the MCP stdio server. |
230
-
231
- Every command accepts `-C, --cwd <path>` to target a repository other than the current directory.
232
- Useful `sync` flags: `--conversation` opts the conversation source in for one run without persisting
233
- it to config; `--no-embed` skips the vector-embedding pass; `--rebuild` drops the project's nodes and
234
- re-ingests from scratch.
235
-
236
- ---
237
-
238
- ## On-Disk Layout
239
-
240
- ```
241
- <repo>/.nexusmem/
242
- .gitignore '*' — the workspace ignores itself, so init never edits a file it does not own
243
- config.json validated on read; a corrupt config fails loudly, never silently
244
- memory.db SQLite (WAL): nodes, node_files, nodes_fts, nodes_vec, sync_state
245
- ```
246
-
247
- Deleting `.nexusmem/` loses nothing that `sync` cannot rebuild.
248
-
249
- ---
250
-
251
- ## Technical Limitations & Edge Cases
252
-
253
- - **Windows Line Endings**: Markdown files are normalized from CRLF to LF prior to chunking.
254
- Un-normalized CRLF causes the paragraph splitter (`\n{2,}`) to never fire — `\r\n\r\n` contains no
255
- two consecutive `\n` — collapsing an entire file into a few coarse, heading-less blocks.
256
- - **Git Rebase / Amend**: Rewriting git history leaves orphaned nodes for unreachable commits. These
257
- are real events, so they are not wrong, but a targeted prune does not exist yet;
258
- `sync --rebuild` forces a clean re-scan if required.
259
- - **Non-Segmented Languages**: FTS5 `unicode61` tokenization splits on whitespace. Languages without
260
- space boundaries (Thai, Japanese, Chinese) rely on the vector search pass for recall.
261
- - **Unscoped Shell History**: Scraped shell history files without the PowerShell hook lack directory
262
- context and are attributed to whichever repository `sync` was executed from. Bounded to the tail
263
- window, and an approximation rather than a guarantee.
264
- - **PSReadLine Multi-Line Entries**: A function typed across several lines at the prompt is read as
265
- separate single-line commands, not reconstructed.
266
- - **Scrape-Fallback Id Drift**: Position-based ids for the scrape fallbacks can drift if the
267
- underlying history file is trimmed from the front between syncs. Installing the hook fixes this.
268
- - **Conversation Retrieval Precision**: Chunking replies at heading boundaries improved precision on
269
- long replies but has not been evaluated systematically.
270
- - **Embedding Batch Size**: The embedding pass processes a bounded batch per `sync`; a large corpus
271
- needs several runs to embed fully.
272
-
273
- ---
274
-
275
- ## Roadmap
276
-
277
- Phases 1 and 2 are shipped. Phase 3 is in progress.
278
-
279
- ### Phase 1 — Core ingestion and retrieval
280
-
281
- - [x] `init` / `sync` / `query` command surface
282
- - [x] Git collector (commits, diff stats, renames, conventional-commit signal)
283
- - [x] Shell collector (PSReadLine, bash, zsh) with optional PowerShell hook
284
- - [x] SQLite storage with FTS5/BM25
285
- - [x] Token-budgeted context packing
286
-
287
- ### Phase 2 — Hybrid retrieval and MCP
288
-
289
- - [x] `sqlite-vec` embeddings via a local Ollama model
290
- - [x] Reciprocal Rank Fusion over BM25 + vector results
291
- - [x] MCP server (stdio): `search_memory`, `sync_project`, `get_status`
292
- - [x] Conversation collector (opt-in), chunked below whole-exchange granularity
293
-
294
- ### Phase 3 — In progress
295
-
296
- - [x] Docs collector for tracked Markdown files
297
- - [x] Scoped pruning of orphaned doc sections on re-sync
298
- - [ ] Diff-level nodes (currently commit-level only)
299
- - [ ] Session summarization via a local SLM
300
- - [ ] Cross-project recall (queries are scoped to one project today)
301
- - [ ] Batch the embedding pass (capped at 200 nodes per `sync`)
302
-
303
- ### Before a tagged release
304
-
305
- - [ ] CI
306
- - [ ] Retry on transient process-spawn failures on Windows
307
- - [ ] Benchmark against a large repository — the >70% end-to-end target is
308
- unproven at this corpus size, where ~40% is what was measured
309
-
310
- ---
311
-
312
- ## Development & Testing
313
-
314
- ```bash
315
- npm install
316
- npm run typecheck
317
- npm test
318
- npm run build
319
- ```
320
-
321
- `scan-git`, `scan-shell`, `scan-docs` and `scan-conversation` write nothing — they print the
322
- `MemoryNode`s ingestion would create, with their signal scores, which is the intended way to tune
323
- scoring against a real repository before committing to a schema change. Add `--json` to pipe the
324
- output elsewhere.
325
-
326
- There is no CI configured yet.
327
-
328
- ## Development Note
329
- This project was initially prototyped and built using **Claude Code** to test the viability of local context memory engines for AI agents.
330
-
331
- While the codebase was generated through AI-assisted workflows, the architecture, system design, and product specifications were directed by human requirements. Contributions, code audits, and refactoring from the community are extremely welcome!
332
-
333
- ---
334
-
335
- ## License
336
-
337
- MIT
1
+ # NexusMem
2
+
3
+ [![npm](https://img.shields.io/npm/v/nexusmem)](https://www.npmjs.com/package/nexusmem)
4
+ [![License: MIT](https://img.shields.io/badge/license-MIT-informational)](LICENSE)
5
+ ![Node](https://img.shields.io/badge/node-%3E%3D22-brightgreen)
6
+
7
+ Your coding agent can read `git log`. It cannot read the four things you tried last Tuesday that
8
+ didn't work.
9
+
10
+ NexusMem records what actually happened on your machine (shell commands and their exit codes, git
11
+ history, project docs, optionally your assistant transcripts) into a local SQLite database, and
12
+ serves back a ranked, token-budgeted slice of it on demand. Everything stays on disk. No account, no
13
+ cloud, no telemetry.
14
+
15
+ The shell history is the part worth caring about. Git tells an agent what shipped. Shell history
16
+ tells it what was attempted, in what order, and which commands exited non-zero. That information
17
+ exists nowhere else, and it disappears when your terminal scrollback rolls over.
18
+
19
+ ## Try it
20
+
21
+ From inside any git repository:
22
+
23
+ ```
24
+ npx nexusmem init
25
+ npx nexusmem sync
26
+ ```
27
+
28
+ Then ask it something. Real output from this repository, top 2 of 5 hits:
29
+
30
+ ```
31
+ $ nexusmem query "windows spawn failure"
32
+
33
+ Relevant history for: windows spawn failure
34
+
35
+ - 2026-08-09 fix: distinguish a failed git spawn from "not a git repository"
36
+ readRepoInfo collapsed three unrelated failures into one error: git running and reporting
37
+ the path is not a work tree, git not being installed, and the process failing to spawn at
38
+ all. Dogfooding hit the third case in two separate sessions...
39
+ - 2026-08-09 README.md — Before a tagged release
40
+ - [ ] Retry on transient process-spawn failures on Windows
41
+ ```
42
+
43
+ A commit and a docs section, ranked against each other, inside whatever token budget you gave it.
44
+ Nothing was summarized by a model on the way out; the ranker just decided what not to send.
45
+
46
+ For a sense of what actually accumulates, here is `nexusmem status` on this repo after two days:
47
+
48
+ ```
49
+ 527 node(s) 2026-08-08 .. 2026-08-09
50
+ 321 shell_command
51
+ 130 conversation_turn
52
+ 60 doc_section
53
+ 16 git_commit
54
+ ```
55
+
56
+ Sixteen commits. Three hundred and twenty-one shell commands. The commits were already retrievable
57
+ by any agent with a terminal. The rest was not.
58
+
59
+ That `conversation_turn` row only appears because this corpus was synced with `--conversation`.
60
+ Assistant transcripts are the one source that is off by default and stays off until you opt in, since
61
+ they are the likeliest place for a pasted credential to be sitting. A default install indexes git,
62
+ shell and docs.
63
+
64
+ Requirements: Node 22 or newer, and git. Node 20 will not work, because `better-sqlite3` ships no
65
+ prebuilt binary for it and Node 20 went end-of-life in April 2026. Ollama is optional and only
66
+ affects semantic search (see below).
67
+
68
+ ## How retrieval works
69
+
70
+ Every source normalizes to the same `MemoryNode` shape, so a commit, a shell command and a docs
71
+ section compete on equal terms. Retrieval runs BM25 over FTS5 and, if an embedding model is
72
+ reachable, a vector search over `sqlite-vec`, then fuses the two with Reciprocal Rank Fusion.
73
+
74
+ RRF fuses on rank *position* only, never on raw scores. That is the entire reason it is safe here: a
75
+ BM25 cost and a vector distance live on unrelated, unbounded scales, and position is the only thing
76
+ they agree on. No hand-tuned normalization constant sits between them.
77
+
78
+ Ranking then multiplies three factors:
79
+
80
+ ```
81
+ score = relevance × signal^0.431 × recency^0.576
82
+ ```
83
+
84
+ `relevance` comes from the query. `signal` (a `fix:` commit outranks a `chore:`; a command that
85
+ exited non-zero outranks one that succeeded) and `recency` are priors that hold before any query
86
+ exists. Each factor is floored into `[floor, 1]` rather than `[0, 1]`, so one weak dimension cannot
87
+ zero out a strong match.
88
+
89
+ Those exponents are derived, not tuned. Priors kept overturning the query: on one real query a `fix:`
90
+ commit took rank 1 from a better-matching docs section on a 44% signal edge against a 15% relevance
91
+ deficit. So each prior is raised to the power that caps its entire range at overturning a 2× relevance
92
+ gap, by solving `span^exponent = 2`. Priors still order equally-relevant hits exactly as before, since
93
+ the transform is monotonic. They just cannot outvote the question anymore.
94
+
95
+ Without Ollama, vector search is skipped and you get BM25 only. That path is fully supported, not a
96
+ degraded error state; `sync` and `query` both succeed and simply do less.
97
+
98
+ ## Use it from an agent
99
+
100
+ ```json
101
+ {
102
+ "mcpServers": {
103
+ "nexusmem": {
104
+ "command": "npx",
105
+ "args": ["-y", "nexusmem", "mcp"]
106
+ }
107
+ }
108
+ }
109
+ ```
110
+
111
+ Three tools over stdio: `search_memory` returns the packed context block, `sync_project` ingests, and
112
+ `get_status` reports what is currently remembered. Each takes an explicit `projectRoot`, because an
113
+ MCP tool call carries no shell working directory. `sync_project` runs `init` for you if the
114
+ repository has not been set up.
115
+
116
+ ## Optional: exact shell capture
117
+
118
+ Scraped history files (PSReadLine, `.bash_history`, `.zsh_history`) give you command text and not
119
+ much else. The hook gives you working directory, exit code and a real timestamp:
120
+
121
+ ```bash
122
+ nexusmem hook install
123
+ ```
124
+
125
+ It wraps your existing PowerShell prompt rather than replacing it, is idempotent, and
126
+ `nexusmem hook remove` undoes it cleanly.
127
+
128
+ Exit codes are what make this worth installing. A failed command is a stronger signal than a
129
+ successful one, and without the hook there is no way to tell them apart.
130
+
131
+ ## What it costs you
132
+
133
+ Two numbers get conflated in tools like this, so they are kept apart here.
134
+
135
+ **Packer efficiency** is how much the ranker trims from its own candidate set. On this repository's
136
+ corpus it runs 81–84%. It is useful for tuning the ranker and useless as a claim about your bill,
137
+ because the baseline is hypothetical: without NexusMem those candidates were never going into your
138
+ context window in the first place.
139
+
140
+ **End-to-end saving** compares packed context against reading the equivalent files in full. Measured
141
+ at **~40%** on design queries against this codebase, hand-tallied from one real session rather than
142
+ instrumented. Treat it as an order of magnitude.
143
+
144
+ The long-term target is >70%, and this repository cannot demonstrate it. That figure describes repos
145
+ with thousands of commits, where the win comes from omitting hundreds of unrelated items rather than
146
+ shaving a handful. A benchmark at that size is still outstanding, and until it exists the honest
147
+ number is 40%.
148
+
149
+ One thing that is not a percentage: shell commands and conversation turns have no cheap `grep`
150
+ equivalent. Without something recording them, they are gone, not merely more expensive to find.
151
+
152
+ Latency on a ~530-node corpus, warm, p50 over 10 runs:
153
+
154
+ | Operation | |
155
+ | --- | --- |
156
+ | BM25 retrieval (FTS5) | ~1.1 ms |
157
+ | Vector KNN (`sqlite-vec`) | ~3.2 ms |
158
+ | Fuse, rank, pack | ~0.6 ms |
159
+ | Query embedding (local Ollama) | ~55–77 ms |
160
+ | **End-to-end hybrid** | **~56 ms** |
161
+
162
+ All the SQLite work totals about 5 ms. The embedding call is the only thing on this path worth
163
+ optimizing, and it is somebody else's process.
164
+
165
+ ## Where it breaks
166
+
167
+ - **Shell history without the hook is unscoped.** Scraped history has no directory context, so it is
168
+ attributed to whichever repository you ran `sync` from. Bounded to a tail window, and an
169
+ approximation rather than a guarantee.
170
+ - **Thai, Japanese and Chinese depend on the vector pass.** FTS5's `unicode61` tokenizer splits on
171
+ whitespace, so languages without space boundaries get no useful BM25 recall.
172
+ - **Rebasing strands nodes.** Rewritten history leaves nodes for unreachable commits. They describe
173
+ real events so they are not wrong, but a targeted prune does not exist yet. `sync --rebuild`
174
+ forces a clean re-scan.
175
+ - **Multi-line PowerShell input is read as separate commands.** A function typed across several lines
176
+ at the prompt is not reconstructed.
177
+ - **Scrape-fallback ids drift** if the history file is trimmed from the front between syncs.
178
+ Installing the hook fixes this.
179
+ - **The embedding pass is capped per `sync`**, so a large corpus needs a few runs to embed fully.
180
+ - **Conversation chunking is unevaluated.** Splitting long replies at heading boundaries measurably
181
+ helped, but it has never been tested systematically.
182
+ - **A burst of recent, high-signal commits crowds unrelated queries.** Each prior is individually
183
+ capped at overturning a 2× relevance gap, but the caps are per-prior, not joint, so a node that is
184
+ both very fresh and highly scored can overturn roughly 4×. Found by dogfooding: a query about the
185
+ PowerShell hook returned two same-day `fix:` commits with nothing to do with it at ranks 3 and 4,
186
+ while the section that actually answered the question sat at rank 6. Gets worse on days with a lot
187
+ of commits, which are exactly the days you have most to remember.
188
+
189
+ ## Commands
190
+
191
+ `init`, `sync`, `query <text>`, `status`, `mcp`, and `hook install|remove|status`.
192
+
193
+ There are also four dry-run previews (`scan-git`, `scan-shell`, `scan-docs`, `scan-conversation`)
194
+ that write nothing and print the nodes ingestion *would* create along with their signal scores. That
195
+ is the intended way to tune scoring against a real repository before committing to a change. Add
196
+ `--json` to pipe them somewhere.
197
+
198
+ Every command takes `-C <path>` to target another repository. On `sync`, `--conversation` opts the
199
+ transcript source in for one run without persisting it, `--no-embed` skips the vector pass, and
200
+ `--rebuild` drops the project's nodes and re-ingests from scratch.
201
+
202
+ ## On disk
203
+
204
+ ```
205
+ <repo>/.nexusmem/
206
+ .gitignore '*' the workspace ignores itself, so init never edits a file it doesn't own
207
+ config.json validated on read; a corrupt config fails loudly rather than silently
208
+ memory.db SQLite in WAL mode
209
+ ```
210
+
211
+ Node ids are content-addressed from `sha256(projectId + kind + naturalKey)`, so running `sync` twice
212
+ cannot produce duplicates and ingestion stays correct even if a cursor is lost. Project identity
213
+ comes from the normalized origin URL when there is one, falling back to the absolute path, so two
214
+ clones of the same repo share one memory namespace.
215
+
216
+ Deleting `.nexusmem/` loses nothing that `sync` cannot rebuild.
217
+
218
+ ## Status
219
+
220
+ Ingestion, hybrid retrieval, budgeted packing and the MCP server all work and are covered by 210
221
+ tests running on Linux and Windows across Node 22 and 24.
222
+
223
+ Not done yet: diff bodies are not indexed (commits stop at metadata and diff stats), queries are
224
+ scoped to a single project, there is no local-model summarization pass, and the conversation
225
+ collector has never been audited for the stale-node bug that was found and fixed in the docs
226
+ collector.
227
+
228
+ ## Development
229
+
230
+ ```bash
231
+ npm install
232
+ npm run typecheck
233
+ npm test
234
+ npm run build
235
+ ```
236
+
237
+ Tests are behavioral rather than snapshot-based, and several are regressions tied to specific
238
+ observed failures. `tests/git-errors.test.ts` injects a fake `spawn` to exercise the Windows
239
+ process-spawn faults, which cannot be provoked on demand.
240
+
241
+ ## On how this was built
242
+
243
+ This started as an experiment in whether a local context-memory engine for coding agents was viable,
244
+ prototyped with Claude Code. The code was written through AI-assisted workflows; the architecture,
245
+ the design decisions and the specifications were human-directed.
246
+
247
+ That is worth stating plainly because it should change how you read the code, not whether you trust
248
+ it. Audits, corrections and PRs are genuinely welcome, and the commit history is deliberately
249
+ detailed about *why* things are the way they are, including the times an earlier assumption turned
250
+ out to be wrong.
251
+
252
+ ## License
253
+
254
+ MIT