peon-mem 1.0.0 → 1.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,10 +1,12 @@
1
1
  # 🧠 Peon — a memory brain for your AI coding agents
2
2
 
3
+ [![npm](https://img.shields.io/npm/v/peon-mem)](https://www.npmjs.com/package/peon-mem) [![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE) [![tests](https://img.shields.io/badge/tests-passing-brightgreen)](test/)
4
+
3
5
  **Local-first, hierarchical, self-improving memory for Claude Code, Codex, and any MCP client.**
4
6
 
5
- Your AI forgets everything between sessions. Peon doesn't. It records your sessions, consolidates
6
- them into typed *beliefs* with an LLM, and injects the relevant ones back into every prompt —
7
- automatically, from a daemon that never leaves your machine.
7
+ Your AI forgets everything between sessions. Peon doesn't. It records your sessions, uses an
8
+ LLM to consolidate them into typed *beliefs*, and injects the relevant ones back into every
9
+ prompt. It runs as a daemon on your machine, and nothing leaves it.
8
10
 
9
11
  ```
10
12
  PEON GLOBAL BRAIN ← user-level facts & preferences, inherited everywhere
@@ -14,25 +16,30 @@ automatically, from a daemon that never leaves your machine.
14
16
 
15
17
  ## Why Peon
16
18
 
17
- - **Hierarchical brains** one global parent brain (who you are, your rules, your tools) plus an
18
- isolated child brain per project. Every injection = project memory + inherited global memory.
19
- - **Two memory layers, honestly measured** consolidated *beliefs* (decisions, preferences, facts,
20
- artifacts) for gist, plus an *episodic* verbatim layer that recovers exact details lossy
21
- summaries drop (measured on LongMemEval: raw-episodic recall 61% vs belief-only 17%).
22
- - **Auto capture + auto injection** — Claude Code hooks record messages/events and inject a
23
- query-ranked memory block (with an `⚠ MOST RELEVANT` headline) into every prompt. Zero effort.
24
- - **Cost-gated consolidation** an LLM distills sessions into beliefs only when enough new
25
- memory accumulates. Supersede / merge / conflict-detect; nothing is destructively deleted.
26
- - **Hybrid retrieval** lexical + semantic RRF fusion, MMR diversity, reinforcement, recency;
27
- query-embedding cache (persisted) so repeat prompts cost nothing.
28
- - **The Neural Universe** — a live monitor at `localhost:3737/monitor` that renders every belief
29
- as a star: projects are galaxies, search makes matches flare, autonomous curation pulses.
30
- - **A daily self-improvement loop (STL)** Peon audits itself every day: what it recorded,
31
- injected, what failed, what consolidation did and files a report with a verdict.
32
- - **Eval-gated development** a committed results ledger (git SHA + qrels + brain fingerprint per
33
- row) so retrieval changes are *proven*, not asserted. Negative results stay documented.
34
- - **Local-first & locked down** plain JSONL you can read, loopback-only daemon with
35
- DNS-rebinding protection, secret redaction at the injection boundary, path-traversal guards.
19
+ - **Hierarchical brains.** One global parent brain holds who you are, your rules, and your
20
+ tools. Each project gets an isolated child brain. Every injection carries both.
21
+ - **Two memory layers, honestly measured.** Consolidated *beliefs* (decisions, preferences,
22
+ facts, artifacts) give you the gist. An *episodic* verbatim layer recovers the exact details
23
+ that lossy summaries drop. On LongMemEval, raw-episodic recall scored 61% where belief-only
24
+ scored 17%.
25
+ - **Automatic capture and injection.** Claude Code hooks record messages and events, then
26
+ inject a query-ranked memory block (with an `⚠ MOST RELEVANT` headline) into every prompt.
27
+ You never have to remember to save anything.
28
+ - **Cost-gated consolidation.** An LLM distills sessions into beliefs only once enough new
29
+ memory accumulates. It can supersede, merge, and flag conflicts. Nothing is destructively
30
+ deleted.
31
+ - **Hybrid retrieval.** Lexical and semantic RRF fusion, MMR diversity, reinforcement, and
32
+ recency. Query embeddings are cached to disk, so repeat prompts cost nothing.
33
+ - **The Neural Universe.** A live monitor at `localhost:3737/monitor` renders every belief as
34
+ a star. Projects are galaxies, search makes matches flare, and autonomous curation pulses.
35
+ - **A daily self-audit (STL).** Peon checks itself every day: what it recorded, what it
36
+ injected, what failed, what consolidation did. Then it files a report with a verdict.
37
+ - **Eval-gated development.** A committed results ledger (git SHA, qrels, and brain
38
+ fingerprint per row) means retrieval changes are proven, not asserted. Negative results stay
39
+ documented.
40
+ - **Local-first and locked down.** Plain JSONL you can read, a loopback-only daemon with
41
+ DNS-rebinding protection, secret redaction at the injection boundary, and path-traversal
42
+ guards.
36
43
 
37
44
 
38
45
 
@@ -40,17 +47,20 @@ automatically, from a daemon that never leaves your machine.
40
47
  ![The Neural Universe — every belief is a star, projects are galaxies](docs/assets/neural-universe.png)
41
48
  *The live monitor: 18k real beliefs rendered as stars. Type to make matching beliefs flare; click one to inspect it.*
42
49
 
50
+ ![Search flare — type a query and matching beliefs light up across every galaxy](docs/assets/search-flare.gif)
51
+ *Ask the field: typing "wulver cluster" makes 400+ matching beliefs flare while the rest dim, and the camera flies to them.*
52
+
43
53
  ## Why "Peon"?
44
54
 
45
- The name comes from Indian offices. Every office had a **peon** the person who walked desk to
55
+ The name comes from Indian offices. Every office had a **peon**, the person who walked desk to
46
56
  desk all day: collect a file from this table, note who needs what, carry it to the next table,
47
- remember where everything is. Not the boss, not the star but the one person the whole office
48
- quietly ran on. Nothing moved without him, and he never forgot where anything was.
57
+ remember where everything is. He wasn't the boss or the star, but the whole office quietly ran
58
+ on him. Nothing moved without him, and he never forgot where anything was.
49
59
 
50
- That's exactly this framework, with AI. Peon walks between your sessions and your projects
51
- collects what happened at one desk (a session), files it in the right cabinet (a project brain),
52
- carries the relevant papers to the next desk before you ask (injection), and keeps the master
53
- ledger upstairs (the global brain). Quiet clerk. Perfect memory. The office runs on him.
60
+ That's this framework, with AI. Peon walks between your sessions and your projects. It collects
61
+ what happened at one desk (a session), files it in the right cabinet (a project brain), carries
62
+ the relevant papers to the next desk before you ask (injection), and keeps the master ledger
63
+ upstairs (the global brain). Quiet clerk. Perfect memory. The office runs on him.
54
64
 
55
65
  ## How Peon differs from existing memory tools
56
66
 
@@ -66,9 +76,9 @@ ledger upstairs (the global brain). Quiet clerk. Perfect memory. The office runs
66
76
  | Observability | **live Neural Universe monitor + daily self-audit (STL) + serve telemetry** | dashboard | — | — | — |
67
77
  | Verification | **committed eval ledger; negative results kept** | vendor benchmarks | — | vendor benchmarks | — |
68
78
 
69
- Positioning in one line: mem0/Zep are memory **platforms for products you build**; Peon is a
70
- memory **brain for the coding agents you already use** plug into Claude Code/Codex in five
71
- minutes, watch it think, audit every number.
79
+ The short version: mem0 and Zep are memory platforms for products you build. Peon is memory
80
+ for the coding agents you already use. It plugs into Claude Code or Codex in about five
81
+ minutes, and you can watch it think and audit every number it claims.
72
82
 
73
83
  ## Quickstart
74
84
 
@@ -78,20 +88,21 @@ recommended (consolidation + semantic embeddings); without one Peon still works
78
88
  One line:
79
89
 
80
90
  ```bash
81
- curl -fsSL https://raw.githubusercontent.com/VineetV2/peon-mem/main/install.sh | bash
91
+ npm install -g peon-mem && peon-mem install
82
92
  ```
83
93
 
84
- (or, once you have the CLI: `npm install -g peon-mem && peon-mem install`)
94
+ (no Node? `curl -fsSL https://raw.githubusercontent.com/VineetV2/peon-mem/main/install.sh | bash`)
85
95
 
86
96
  The guided setup asks four things:
87
97
 
88
98
  1. **Where your global brain lives** (default: `~/Library/Application Support/Peon`)
89
- 2. **Which LLM** OpenRouter (one key, any model) · OpenAI · Anthropic · **Ollama (100% local & free)** · or skip
90
- 3. Installs the **daemon** as an auto-start service
91
- 4. **Detects your AI apps** and wires the MCP server (+ hooks for Claude Code) into the ones you
92
- pick auto-configured: Claude Code, Claude Desktop, Codex, Gemini CLI, Cursor, Windsurf,
93
- VS Code (Copilot MCP), Zed, LM Studio; detected with in-app instructions: ChatGPT Desktop,
94
- Perplexity Desktop. Every touched config gets a `.peon-backup`.
99
+ 2. **Which LLM**: OpenRouter (one key, any model) · OpenAI · Anthropic · **Ollama (100% local
100
+ & free)** · or skip
101
+ 3. Installs the **daemon** as an auto-start service (launchd on macOS, systemd user unit on Linux)
102
+ 4. **Detects your AI apps** and wires the MCP server (plus hooks for Claude Code) into the
103
+ ones you pick. Auto-configured: Claude Code, Claude Desktop, Codex, Gemini CLI, Cursor,
104
+ Windsurf, VS Code (Copilot MCP), Zed, LM Studio. Detected with in-app instructions: ChatGPT
105
+ Desktop, Perplexity Desktop. Every touched config gets a `.peon-backup`.
95
106
 
96
107
  That builds the package, starts the daemon as a service, wires your Claude Code hooks + MCP
97
108
  server (with a backup of your settings), and writes a config template. Then add your key to
@@ -122,25 +133,28 @@ node dist/daemon-cli.js # serves 127.0.0.1:3737
122
133
 
123
134
  Then wire your agent (the installer prints these filled in for your paths):
124
135
 
125
- - **Claude Code** add the hook to `~/.claude/settings.json` (SessionStart / UserPromptSubmit /
136
+ - **Claude Code**: add the hook to `~/.claude/settings.json` (SessionStart / UserPromptSubmit /
126
137
  SessionEnd → `scripts/claude-peon-hook.mjs`) and the MCP server (`dist/index.js`).
127
- - **Codex / any MCP client** register `dist/index.js` as a stdio MCP server; 16 tools
138
+ - **Codex / any MCP client**: register `dist/index.js` as a stdio MCP server; 16 tools
128
139
  (`start_session`, `get_context`, `search_memory`, `record_message`, `process_memory`, …).
129
140
 
130
141
  Open `http://127.0.0.1:3737/monitor` and watch your brain grow.
131
142
 
132
143
  ## How it works
133
144
 
134
- 1. **Record** hooks stream messages/events/tool-calls into `<project>/.peon/raw/` (append-only).
135
- 2. **Consolidate** — past a size gate, an LLM turns the session delta into typed belief records in
136
- `.peon/brain/memories.jsonl` (importance/confidence scores, entities, provenance pointers),
137
- reconciling against existing beliefs: supersede, merge, conflict-flag. Recoverable, never deleted.
138
- 3. **Retrieve + inject** on every prompt, beliefs are ranked (RRF lexical+semantic, MMR,
139
- reinforcement) and injected alongside episodic verbatim matches and inherited global beliefs.
140
- 4. **Self-curate** a background brain pass reinforces recalled beliefs, compresses stale
141
- clusters, resolves duplicates every action logged and undoable.
142
- 5. **Self-audit (STL)** — a daily job reports: recorded / injected / went-wrong / consolidation
143
- correctness, with serve-latency telemetry and a health verdict.
145
+ 1. **Record.** Hooks stream messages, events, and tool calls into `<project>/.peon/raw/`
146
+ (append-only).
147
+ 2. **Consolidate.** Past a size gate, an LLM turns the session delta into typed belief records
148
+ in `.peon/brain/memories.jsonl` (importance/confidence scores, entities, provenance
149
+ pointers), reconciling against existing beliefs: supersede, merge, conflict-flag.
150
+ Recoverable, never deleted.
151
+ 3. **Retrieve + inject.** On every prompt, beliefs are ranked (RRF lexical+semantic, MMR,
152
+ reinforcement) and injected alongside episodic verbatim matches and inherited global
153
+ beliefs.
154
+ 4. **Self-curate.** A background brain pass reinforces recalled beliefs, compresses stale
155
+ clusters, and resolves duplicates. Every action is logged and undoable.
156
+ 5. **Self-audit (STL).** A daily job reports what was recorded, what was injected, what went
157
+ wrong, and whether consolidation behaved, with serve-latency telemetry and a health verdict.
144
158
 
145
159
 
146
160
  ## Full install (copy-paste)
@@ -172,7 +186,9 @@ launchctl load ~/Library/LaunchAgents/com.peon.daemon.plist
172
186
  curl http://127.0.0.1:3737/health # → {"ok":true}
173
187
  ```
174
188
 
175
- Linux: run `node dist/daemon-cli.js` under systemd (`Restart=always`).
189
+ Linux: the installer writes `~/.config/systemd/user/peon-mem.service` and enables it. Manual
190
+ equivalent: `systemctl --user enable --now peon-mem.service`, plus `loginctl enable-linger $USER`
191
+ so it survives logout. (Linux support is young — [issue #1](https://github.com/VineetV2/peon-mem/issues/1) tracks real-machine reports.)
176
192
 
177
193
  ### 2. Claude Code — hooks (auto capture + injection)
178
194
 
@@ -209,7 +225,7 @@ args = ["/ABSOLUTE/PATH/TO/peon/dist/index.js"]
209
225
  PEON_DAEMON_URL = "http://127.0.0.1:3737"
210
226
  ```
211
227
 
212
- Codex has no hooks add usage rules to `~/.codex/AGENTS.md` telling it to call
228
+ Codex has no hooks, so add usage rules to `~/.codex/AGENTS.md` telling it to call
213
229
  `start_session` + `get_context` at session start and `record_message` for durable facts
214
230
  (example block in [docs/](docs/)).
215
231
 
@@ -220,8 +236,8 @@ curl "http://127.0.0.1:3737/context?projectPath=$PWD&query=test" # JSON contex
220
236
  open http://127.0.0.1:3737/monitor # the Neural Universe
221
237
  ```
222
238
 
223
- Start a Claude Code session in any project, say something decision-shaped, end the session
224
- within a minute the monitor shows the belief. Next session injects it.
239
+ Start a Claude Code session in any project, say something decision-shaped, and end the session.
240
+ Within a minute the monitor shows the belief. The next session injects it.
225
241
 
226
242
  ### MCP tools exposed
227
243
 
@@ -240,17 +256,17 @@ launchctl unload ~/Library/LaunchAgents/com.peon.daemon.plist
240
256
 
241
257
  ## Troubleshooting / FAQ
242
258
 
243
- - **No injection appearing?** `curl http://127.0.0.1:3737/health`; check hook is registered
244
- (`claude` run any prompt monitor Systems page shows the request).
245
- - **431 errors on huge prompts?** Handled the hook caps the retrieval query at 2k chars.
259
+ - **No injection appearing?** `curl http://127.0.0.1:3737/health`; check the hook is registered
260
+ (run any prompt in `claude`, then look at the monitor's Systems page for the request).
261
+ - **431 errors on huge prompts?** Handled. The hook caps the retrieval query at 2k chars.
246
262
  - **No OpenRouter key?** Everything still runs; retrieval is lexical + episodic only
247
263
  (semantic ranking and consolidation need a model). `PEON_EMBEDDING_MODE=ollama` works too.
248
- - **Cost?** Consolidation is gated (default: fires per ~6k new chars, ~cents/day with
249
- flash-lite). Query embeddings are cached to disk repeats are free.
250
- - **Multiple machines?** Brains are plain files in your repos commit `.peon/` if you want
251
- memory to travel (redact first: raw layer contains session text).
264
+ - **Cost?** Consolidation is gated (default: fires per ~6k new chars, around cents/day with
265
+ flash-lite). Query embeddings are cached to disk, so repeats are free.
266
+ - **Multiple machines?** Brains are plain files in your repos. Commit `.peon/` if you want
267
+ memory to travel (redact first: the raw layer contains session text).
252
268
  - **Is my data sent anywhere?** Only consolidation/embedding calls to your configured model
253
- provider. No telemetry, no cloud store. Daemon rejects non-loopback callers.
269
+ provider. No telemetry, no cloud store. The daemon rejects non-loopback callers.
254
270
 
255
271
  ## Configuration (env)
256
272
 
@@ -270,32 +286,129 @@ launchctl unload ~/Library/LaunchAgents/com.peon.daemon.plist
270
286
 
271
287
  ## Project brains
272
288
 
273
- - A brain lives in `<project>/.peon/` human-readable JSONL + markdown. Commit it or ignore it;
274
- your choice (`.gitignore` ships ignoring it).
275
- - `.peon/root` marks a brain boundary. New brains are born rooted; a parent directory can never
276
- swallow a project's memory.
289
+ - A brain lives in `<project>/.peon/` as human-readable JSONL and markdown. Commit it or
290
+ ignore it; your choice (`.gitignore` ships ignoring it).
291
+ - `.peon/root` marks a brain boundary. New brains are born rooted, so a parent directory can
292
+ never swallow a project's memory.
277
293
  - The global brain lives in `~/Library/Application Support/Peon/global/` (macOS).
278
294
 
295
+ ## Using Peon with NO AI at all
296
+
297
+ Some people want a memory system that never calls a model: no API keys, no local LLM, no
298
+ embeddings. Peon supports that. Pick **skip** in the install wizard, or set two env vars in
299
+ `<memory-home>/.env`:
300
+
301
+ ```
302
+ PEON_AI_MODE=off
303
+ PEON_EMBEDDING_MODE=off
304
+ ```
305
+
306
+ The pipeline looks like this without a model. Hooks still capture everything you do into
307
+ `<project>/.peon/raw/` as plain JSONL. As each event arrives, rule-based extraction files it
308
+ into readable brain files: a line that starts with "we decided" or "always use" lands in
309
+ `decisions.md`, preferences land in `preferences.md`, file paths land in `artifacts.md`. No
310
+ model reads it; it's pattern matching, and you can open the files to see exactly what it
311
+ caught. At your next session start, the hook asks the daemon for context, the daemon ranks
312
+ those records by keyword match, recency, and importance, and injects the best ones into your
313
+ prompt. The only step that disappears entirely is consolidation, the nightly LLM pass that
314
+ compresses raw history into deduplicated beliefs. Without it your memory is a well-indexed
315
+ journal instead of a distilled one. Search still finds things; nothing summarizes them.
316
+
317
+ What still works (all of it deterministic code, no model anywhere):
318
+
319
+ - **Capture.** Hooks record every prompt, tool call, and session event to plain JSONL in
320
+ `<project>/.peon/raw/`.
321
+ - **Real-time brain files.** Decisions, preferences, open questions, and artifacts are written
322
+ live to readable `.md` files by rule-based extraction as events arrive.
323
+ - **Injection.** Session-start context comes from those real-time files, query-focused and
324
+ budgeted, same as always.
325
+ - **Search.** Lexical retrieval (RRF over keyword rank, recency, importance, and type priors).
326
+ No embeddings needed. This is the same degrade path the semantic stack falls back to, so it
327
+ gets exercised constantly.
328
+ - **Episodic recall.** Verbatim what-was-said lookup is lexical by design, so it is unaffected.
329
+ - **Monitor UI, token tracking, cross-project search, backups.** All model-free.
330
+
331
+ What you give up: consolidation (raw events are never distilled into deduplicated beliefs, so
332
+ memory grows as an append-only journal), semantic search (paraphrased queries need shared
333
+ keywords), automatic entity extraction, and stale-shadow demotion (it compares embeddings).
334
+
335
+ Two escape hatches if you want curation without external AI:
336
+
337
+ 1. `process_memory` accepts a pre-built `aiResult`. The coding agent you already run (Claude
338
+ Code, Codex) can do the distillation itself in-session and hand Peon the structured result.
339
+ Memory stays curated, and Peon itself never spends a token.
340
+ 2. Everything is plain JSONL/Markdown on disk. You can edit beliefs by hand or through the
341
+ monitor's memory endpoints, and Peon backs up before every mutation.
342
+
343
+ ## Measured: does memory actually save tokens?
344
+
345
+ A/B test with real `claude -p` sessions: one question per session, same repo, same model.
346
+ ON means Peon hooks active (memory injected at session start). OFF means `PEON_DISABLED=1`,
347
+ so the agent falls back to reading files. 20 questions across procedures, past results,
348
+ decisions, and current-state facts; 15 clean ON/OFF pairs survived tooling issues. Token
349
+ counts come from Claude Code's own session transcripts.
350
+
351
+ | paired, n=15/arm | ON (Peon) | OFF | delta |
352
+ |---|---|---|---|
353
+ | avg tokens (in+out) | **511** | 878 | **−42%** |
354
+ | median tokens | **225** | 1,048 | **−79%** |
355
+ | cache-read tokens | 81.6k | 110.5k | 1.35× less |
356
+ | cheaper arm | **ON wins 12/15** | | |
357
+
358
+ Answer quality, graded against repo ground truth: 8 ties, 1 clear Peon win, 5 baseline wins,
359
+ 1 where both were weak. The Peon win is the interesting one. A rule that was only ever stated
360
+ in a conversation (a professor's citation policy, from an email) got *"no such rule found"*
361
+ from the baseline; Peon recited it exactly. Conversation-borne knowledge has no file to grep.
362
+
363
+ Honest caveats: the test repo has unusually good docs (a maintained research log), which makes
364
+ the baseline strong, and most repos aren't like that. n=15 is small. Questions were picked to
365
+ have known answers, not sampled from real usage. The test also exposed two real bugs in Peon
366
+ itself: a stale superseded belief outranking the newer truth, and token rows getting lost when
367
+ consolidation outlived the hook timeout. Both are fixed (stale-shadow demotion at retrieval;
368
+ usage logged before consolidation).
369
+
279
370
  ## Honesty section
280
371
 
281
- Peon's development is eval-gated and keeps its negative results: an associative entity graph was
282
- built, measured (−2.9% Recall@10), and turned OFF by default. Consolidation is lossy by design —
283
- that's why the episodic layer exists and is regression-tested. The eval harness + committed
284
- ledger (`npm run eval`) let you verify retrieval changes on your own brain.
372
+ Peon's development is eval-gated and keeps its negative results. An associative entity graph
373
+ was built, measured (−2.9% Recall@10), and turned OFF by default. Consolidation is lossy by
374
+ design; that's why the episodic layer exists and is regression-tested. The eval harness and
375
+ committed ledger (`npm run eval`) let you verify retrieval changes on your own brain.
285
376
 
286
377
  ## Security
287
378
 
288
- - Daemon binds `127.0.0.1` only and rejects non-loopback `Host`/`Origin` (DNS-rebinding defense).
379
+ - The daemon binds `127.0.0.1` only and rejects non-loopback `Host`/`Origin` headers
380
+ (DNS-rebinding defense).
289
381
  - Secrets (API keys, tokens, JWTs) are redacted at the injection boundary.
290
- - Path-traversal guarded; per-project write locks; atomic tmp+rename writes; automatic backups
291
- before destructive-adjacent operations. Nothing is hard-deleted.
382
+ - Path traversal is guarded. Writes use per-project locks and atomic tmp+rename, with
383
+ automatic backups before anything destructive-adjacent. Nothing is hard-deleted.
292
384
 
293
385
  ## License
294
386
 
295
387
  MIT © Vineet Vora
296
388
 
297
- ## Contributing
389
+ ## Contributing — I'd love your help
390
+
391
+ Peon is built and maintained by one person, and I'm open to help making it better. Issues,
392
+ PRs, ideas, criticism of the architecture, or just telling me where it broke on your machine:
393
+ all of it is useful.
394
+
395
+ Where help would matter most right now:
396
+
397
+ - **Windows & Linux support.** The daemon install is macOS launchd today. The systemd and Task
398
+ Scheduler equivalents need real users to test them.
399
+ - **More agent integrations.** The wizard covers 11 apps, but hooks-level capture (like the
400
+ Claude Code integration) for Codex, Cursor, and others would make memory richer everywhere.
401
+ - **Retrieval quality.** The eval harness (`npm run eval`) makes experiments cheap: better
402
+ consolidation prompts, smarter staleness handling, local embedding models worth defaulting
403
+ to.
404
+ - **Benchmarks.** Run the token A/B on your repo and share the numbers, especially where Peon
405
+ loses. Negative results are first-class here.
406
+ - **Docs and onboarding.** If the README or the wizard confused you, that confusion is a bug
407
+ report.
408
+
409
+ Open an issue at [github.com/VineetV2/peon-mem/issues](https://github.com/VineetV2/peon-mem/issues)
410
+ or send a PR directly.
298
411
 
299
- PRs welcome. Rules of the house: every retrieval/quality change ships with a test and an
300
- eval-ledger run (`npm run eval`); negative results get documented, not deleted; nothing may
301
- hard-delete user memory. `npm test` must stay green (255 tests).
412
+ Rules of the house: every retrieval/quality change ships with a test and an eval-ledger run
413
+ (`npm run eval`); negative results get documented, not deleted; nothing may hard-delete user
414
+ memory. `npm test` must stay green.
package/bin/peon-mem.mjs CHANGED
@@ -8,7 +8,7 @@
8
8
  * peon-mem doctor health + config check
9
9
  */
10
10
  import { execFileSync, spawnSync } from "node:child_process";
11
- import { existsSync, mkdirSync, readFileSync, writeFileSync, copyFileSync } from "node:fs";
11
+ import { existsSync, mkdirSync, readFileSync, writeFileSync, copyFileSync, rmSync } from "node:fs";
12
12
  import { homedir, platform } from "node:os";
13
13
  import { dirname, join } from "node:path";
14
14
  import { fileURLToPath } from "node:url";
@@ -16,7 +16,8 @@ import { createInterface } from "node:readline/promises";
16
16
 
17
17
  const PKG = dirname(dirname(fileURLToPath(import.meta.url)));
18
18
  const HOME = homedir();
19
- const MAC = platform() === "darwin";
19
+ // PEON_FORCE_PLATFORM lets CI and maintainers exercise the other OS's install path.
20
+ const MAC = (process.env.PEON_FORCE_PLATFORM || platform()) === "darwin";
20
21
  const DEFAULT_HOME = MAC ? join(HOME, "Library", "Application Support", "Peon") : join(HOME, ".local", "share", "peon");
21
22
  const PLIST = join(HOME, "Library", "LaunchAgents", "com.peon.daemon.plist");
22
23
  const HOOK = join(PKG, "scripts", "claude-peon-hook.mjs");
@@ -156,8 +157,34 @@ function installService(memoryHome) {
156
157
  execFileSync("launchctl", ["load", PLIST]);
157
158
  });
158
159
  } else {
159
- log(" → Linux: systemd user unit:");
160
- log(` ExecStart=${NODE} ${DAEMON}\n WorkingDirectory=${memoryHome}\n Restart=always`);
160
+ // Linux: write a real systemd user unit instead of printing a recipe.
161
+ const unitDir = join(HOME, ".config", "systemd", "user");
162
+ const unitFile = join(unitDir, "peon-mem.service");
163
+ const unit = `[Unit]
164
+ Description=Peon memory daemon (local-first memory for AI coding agents)
165
+ After=network.target
166
+
167
+ [Service]
168
+ ExecStart=${NODE} ${DAEMON}
169
+ WorkingDirectory=${memoryHome}
170
+ Restart=always
171
+ RestartSec=2
172
+
173
+ [Install]
174
+ WantedBy=default.target
175
+ `;
176
+ act("daemon service (systemd user unit) → " + unitFile, () => {
177
+ mkdirSync(unitDir, { recursive: true });
178
+ writeFileSync(unitFile, unit);
179
+ // Enable + start when systemd is actually available; on failure fall back to instructions.
180
+ const r = spawnSync("systemctl", ["--user", "daemon-reload"], { stdio: "ignore" });
181
+ if (r.status === 0) {
182
+ spawnSync("systemctl", ["--user", "enable", "--now", "peon-mem.service"], { stdio: "ignore" });
183
+ }
184
+ });
185
+ log(" → if the daemon isn't running yet:");
186
+ log(" systemctl --user daemon-reload && systemctl --user enable --now peon-mem.service");
187
+ log(" loginctl enable-linger $USER # keeps it running after logout");
161
188
  }
162
189
  }
163
190
 
@@ -205,7 +232,12 @@ if (cmd === "install") {
205
232
  if (!up) log(" → Ollama not running. Install: https://ollama.com then: ollama pull llama3.2 && ollama pull nomic-embed-text");
206
233
  else log(" ✔ Ollama detected on :11434 — pull models if missing: ollama pull llama3.2 && ollama pull nomic-embed-text");
207
234
  } else {
208
- envLines.push("# no provider configured — Peon runs lexical-only. Re-run `peon-mem install` anytime.");
235
+ envLines.push(
236
+ "# No-AI mode: no provider, no model calls, no embeddings. Peon runs as a",
237
+ "# deterministic memory recorder with lexical retrieval. Re-run `peon-mem install` anytime.",
238
+ "PEON_AI_MODE=off",
239
+ "PEON_EMBEDDING_MODE=off"
240
+ );
209
241
  }
210
242
  if (existsSync(envFile)) log(" ✔ keeping existing " + envFile);
211
243
  else act("write " + envFile, () => writeFileSync(envFile, envLines.join("\n") + "\n"));
@@ -244,6 +276,11 @@ if (cmd === "install") {
244
276
  rl?.close();
245
277
  } else if (cmd === "uninstall") {
246
278
  if (MAC && existsSync(PLIST)) act("stop + remove daemon service", () => spawnSync("launchctl", ["unload", PLIST], { stdio: "ignore" }));
279
+ const UNIT = join(HOME, ".config", "systemd", "user", "peon-mem.service");
280
+ if (!MAC && existsSync(UNIT)) act("stop + remove daemon service", () => {
281
+ spawnSync("systemctl", ["--user", "disable", "--now", "peon-mem.service"], { stdio: "ignore" });
282
+ rmSync(UNIT, { force: true });
283
+ });
247
284
  const settings = join(HOME, ".claude", "settings.json");
248
285
  try {
249
286
  const s = JSON.parse(readFileSync(settings, "utf8"));
@@ -7,7 +7,7 @@ import { cosineSimilarity, createEmbeddingClient } from "./embeddings.js";
7
7
  import { applyDelete, applyMerge, applyPin, applyUpdate } from "./memory-mutations.js";
8
8
  import { runSleepCycle } from "./brain.js";
9
9
  import { readdir, rm } from "node:fs/promises";
10
- import { rankMemoryRecords as rankWithRetrieval, computeGraphActivation, diversifyByMMR } from "./retrieval.js";
10
+ import { rankMemoryRecords as rankWithRetrieval, computeGraphActivation, demoteStaleShadows, diversifyByMMR } from "./retrieval.js";
11
11
  import { currentAsOf, changesBetween } from "./temporal.js";
12
12
  import { inferCanonicalEntities, buildEntityRegistry, canonicalizeEntity } from "./entities.js";
13
13
  import { redactSecrets } from "./injection.js";
@@ -464,14 +464,14 @@ export class PeonMemoryStore {
464
464
  const limit = options.limit ?? 50;
465
465
  const direct = rankWithRetrieval(records, query, { limit, semantic });
466
466
  if (!options.expandGraph || direct.length === 0)
467
- return direct;
467
+ return demoteStaleShadows(direct, semantic?.vectorById);
468
468
  // FUSED associative recall: spread activation from the direct hits through the entity graph,
469
469
  // then RE-RANK with that activation as a (damped) signal — so a strongly-associated belief can
470
470
  // enter the top-K and displace a weak direct hit, instead of being appended out of the window.
471
471
  const graphActivation = computeGraphActivation(direct, records);
472
472
  if (graphActivation.size === 0)
473
- return direct;
474
- return rankWithRetrieval(records, query, { limit, semantic, graphActivation });
473
+ return demoteStaleShadows(direct, semantic?.vectorById);
474
+ return demoteStaleShadows(rankWithRetrieval(records, query, { limit, semantic, graphActivation }), semantic?.vectorById);
475
475
  }
476
476
  /**
477
477
  * Rank records WITHOUT mutating anything — uses only embeddings already on disk
@@ -497,7 +497,7 @@ export class PeonMemoryStore {
497
497
  // lexical-only on any embedding failure
498
498
  }
499
499
  }
500
- return rankWithRetrieval(records, query, { limit: options.limit ?? 50, semantic });
500
+ return demoteStaleShadows(rankWithRetrieval(records, query, { limit: options.limit ?? 50, semantic }), semantic?.vectorById);
501
501
  }
502
502
  async buildSemanticInput(query, records) {
503
503
  if (!query || !query.trim() || !this.embeddingClient || !this.embeddingStore || records.length === 0) {
package/dist/monitor.js CHANGED
@@ -92,6 +92,10 @@ const CLIENT_SCRIPT = String.raw `
92
92
  function renderRoute(force){
93
93
  var r=currentRoute();
94
94
  ROUTES.forEach(function(x){ var p=EL("page-"+x); if(p) p.hidden=(x!==r); });
95
+ // The project dropdown is CONTEXT, not chrome: it only applies inside a project's
96
+ // Insights/Memory pages. Global pages (Neural Core / Sectors / Systems) hide it —
97
+ // switching projects there happens by picking a galaxy or a sector card.
98
+ var sw=EL("switcher"); if(sw) sw.hidden=!(r==="overview"||r==="memory");
95
99
  // Nav highlight: overview/memory belong under "projects".
96
100
  ["brain","projects","ops"].forEach(function(x){ var t=EL("nav-"+x); if(t) t.classList.toggle("on", x===r || ((r==="overview"||r==="memory")&&x==="projects")); });
97
101
  if(r==="brain"){ loadDashboard(force); renderBrainHome(); }
@@ -184,6 +188,20 @@ const CLIENT_SCRIPT = String.raw `
184
188
  ' <span class="stl-dot" style="background:'+dotc+'" title="'+esc(stl)+'"></span>';
185
189
  }
186
190
 
191
+ var SPRITES={};
192
+ function nodeSprite(color, glow){
193
+ var key=color+(glow?"G":"");
194
+ if(SPRITES[key]) return SPRITES[key];
195
+ var pad=glow?10:2, R=6, size=(R+pad)*2;
196
+ var sc=document.createElement("canvas"); sc.width=size; sc.height=size;
197
+ var g=sc.getContext("2d");
198
+ if(glow){ g.shadowColor=color; g.shadowBlur=9; }
199
+ g.fillStyle=color; g.beginPath(); g.arc(size/2,size/2,R,0,6.29); g.fill();
200
+ if(glow){ g.fill(); }
201
+ SPRITES[key]={c:sc,R:R,half:size/2};
202
+ return SPRITES[key];
203
+ }
204
+
187
205
  function uniProject(n){ /* world position with slow galaxy rotation */
188
206
  var t=(Date.now()-UNI.t0);
189
207
  var th=n.th + t*n.cl.spin;
@@ -237,18 +255,22 @@ const CLIENT_SCRIPT = String.raw `
237
255
  });
238
256
  // nodes
239
257
  var hits=UNI.hits, dimOthers=!!(hits&&hits.size);
258
+ // viewport bounds in world coords (with margin) for culling
259
+ var vw=W/2/UNI.cam.z+30, vh=H/2/UNI.cam.z+30, vcx=UNI.cam.x, vcy=UNI.cam.y;
240
260
  UNI.nodes.forEach(function(n){
241
261
  uniProject(n);
262
+ if(n.x<vcx-vw||n.x>vcx+vw||n.y<vcy-vh||n.y>vcy+vh) return; // offscreen — skip draw
242
263
  var tw=0.78+0.22*Math.sin(t/900+n.tw);
243
264
  var a=n.a*tw, r=n.r;
244
265
  if(dimOthers){ if(hits.has(n.id)){ a=1; r=n.r*1.7; } else a*=0.08; }
245
266
  if(n.hot) r*=1.15;
246
- ctx.beginPath(); ctx.arc(n.x,n.y,r/Math.sqrt(UNI.cam.z),0,6.29);
267
+ var glow=(dimOthers&&hits.has(n.id))||n===UNI.hover||n===UNI.selected||n.rec.status==="conflicted";
268
+ var sp=nodeSprite(n.c, glow);
269
+ var scale=(r/Math.sqrt(UNI.cam.z))/sp.R;
247
270
  ctx.globalAlpha=Math.min(1,a);
248
- ctx.fillStyle=n.c;
249
- if((dimOthers&&hits.has(n.id))||n===UNI.hover||n===UNI.selected||n.rec.status==="conflicted"){ ctx.shadowColor=n.c; ctx.shadowBlur=12; } else ctx.shadowBlur=0;
250
- ctx.fill(); ctx.shadowBlur=0; ctx.globalAlpha=1;
271
+ ctx.drawImage(sp.c, n.x-sp.half*scale, n.y-sp.half*scale, sp.c.width*scale, sp.c.height*scale);
251
272
  });
273
+ ctx.globalAlpha=1;
252
274
  // hover crosshair
253
275
  if(UNI.hover){ var hN=UNI.hover; var hr=10/UNI.cam.z;
254
276
  ctx.strokeStyle="rgba(234,255,255,.85)"; ctx.lineWidth=1/UNI.cam.z;
@@ -257,14 +279,15 @@ const CLIENT_SCRIPT = String.raw `
257
279
  ctx.moveTo(hN.x,hN.y-hr*1.8); ctx.lineTo(hN.x,hN.y-hr); ctx.moveTo(hN.x,hN.y+hr); ctx.lineTo(hN.x,hN.y+hr*1.8); ctx.stroke();
258
280
  }
259
281
  ctx.restore();
260
- uniGridBuild();
282
+ if(!UNI.lastGrid||t-UNI.lastGrid>250){ uniGridBuild(); UNI.lastGrid=t; }
261
283
  // tooltip
262
284
  var tip=EL("uni-tip");
263
285
  if(tip){ if(UNI.hover){ var sp=uniToScreen(UNI.hover,W,H);
264
286
  tip.hidden=false; tip.style.left=Math.min(W-330,Math.max(8,sp.x+16))+"px"; tip.style.top=Math.max(8,sp.y-14)+"px";
265
287
  tip.innerHTML='<span class="tt-type" style="color:'+UNI.hover.c+'">'+esc(UNI.hover.rec.type)+'</span> '+esc(clip(UNI.hover.rec.content,140));
266
288
  } else tip.hidden=true; }
267
- UNI.raf=requestAnimationFrame(uniDraw);
289
+ // ~30fps is indistinguishable here and halves the draw cost on big brains
290
+ UNI.raf=requestAnimationFrame(function(){ setTimeout(function(){ UNI.raf=0; uniDraw(); }, 15); });
268
291
  }
269
292
  function uniToScreen(n,W,H){ return { x:(n.x-UNI.cam.x)*UNI.cam.z+W/2, y:(n.y-UNI.cam.y)*UNI.cam.z+H/2 }; }
270
293
  function uniToWorld(sx,sy){ var c=uniCanvas(); var W=c.clientWidth,H=c.clientHeight;
@@ -822,7 +845,7 @@ const DOCUMENT = String.raw `<!doctype html>
822
845
  .uniwrap::before{content:""; position:absolute; top:0; left:0; width:16px; height:16px; border-top:1.5px solid var(--line2); border-left:1.5px solid var(--line2); z-index:5; pointer-events:none;}
823
846
  .uniwrap::after{content:""; position:absolute; bottom:0; right:0; width:16px; height:16px; border-bottom:1.5px solid var(--line2); border-right:1.5px solid var(--line2); z-index:5; pointer-events:none;}
824
847
  #uni{position:absolute; inset:0; width:100%; height:100%; cursor:crosshair;}
825
- .uni-search{position:absolute; top:14px; left:50%; transform:translateX(-50%); z-index:6; display:flex; gap:8px; align-items:center; width:min(560px,80%);}
848
+ .uni-search{position:absolute; top:68px; left:50%; transform:translateX(-50%); z-index:6; display:flex; gap:8px; align-items:center; width:min(560px,72%);}
826
849
  .uni-search input{flex:1; padding:9px 16px; border:1px solid rgba(89,227,255,.5); clip-path:var(--cham); background:rgba(3,12,22,.88);
827
850
  font-family:var(--mono); font-size:12px; color:var(--cyan-ink); backdrop-filter:blur(8px);}
828
851
  .uni-search input:focus{border-color:var(--cyan); outline:none; box-shadow:0 0 22px -6px rgba(89,227,255,.9);}
@@ -831,7 +854,7 @@ const DOCUMENT = String.raw `<!doctype html>
831
854
  .uni-hud{position:absolute; z-index:6; font-family:var(--mono); font-size:10px; letter-spacing:.12em; color:var(--muted);
832
855
  background:rgba(3,12,22,.72); border:1px solid var(--line); padding:7px 12px; clip-path:var(--cham); backdrop-filter:blur(6px);}
833
856
  .uni-hud b{color:var(--cyan-ink); font-weight:700;} .uni-hud b.warn{color:var(--amber);}
834
- .uni-hud.tl{top:14px; left:14px;} .uni-hud.tr{top:60px; right:14px;}
857
+ .uni-hud.tl{top:14px; left:14px; max-width:58%;} .uni-hud.tr{top:14px; right:14px;}
835
858
  .stl-dot{display:inline-block; width:8px; height:8px; border-radius:50%; margin-left:6px; vertical-align:-1px; box-shadow:0 0 8px currentColor;}
836
859
  .uni-legend{position:absolute; left:14px; bottom:76px; z-index:6; display:flex; flex-wrap:wrap; gap:9px; max-width:70%;}
837
860
  .ul{display:inline-flex; align-items:center; gap:5px; font-family:var(--mono); font-size:9px; letter-spacing:.1em; text-transform:uppercase; color:var(--muted);}
@@ -104,3 +104,19 @@ export declare function computeGraphActivation(seeds: RankedMemoryRecord[], pool
104
104
  * rankMemoryRecords' graphActivation option, which lets associations compete inside the top-K.
105
105
  */
106
106
  export declare function expandByEntityGraph(seeds: RankedMemoryRecord[], pool: MemoryRecord[], options?: GraphExpandOptions): RankedMemoryRecord[];
107
+ /**
108
+ * Stale-shadow demotion. Measured failure mode (token A/B, question x1): recall answered with a
109
+ * SUPERSEDED architecture description because the old belief was still active, semantically strong,
110
+ * and outranked the newer truth. When two active beliefs in the ranked window describe the same fact
111
+ * — near-duplicate vectors, or moderately similar with a shared entity and the same type — but were
112
+ * written in different eras, the older one is treated as a stale shadow of the newer: its score is
113
+ * scaled down so the newer belief always outranks it. Nothing is deleted or re-statused here; real
114
+ * supersession stays the consolidator's job. Pinned records are never demoted.
115
+ */
116
+ export declare function demoteStaleShadows(ranked: RankedMemoryRecord[], vectorById: Map<string, EmbeddingVector> | undefined, options?: {
117
+ scan?: number;
118
+ hardSim?: number;
119
+ softSim?: number;
120
+ ageGapMs?: number;
121
+ penalty?: number;
122
+ }): RankedMemoryRecord[];
package/dist/retrieval.js CHANGED
@@ -390,3 +390,70 @@ function clamp(value) {
390
390
  function roundScore(value) {
391
391
  return Math.round(value * 1000) / 1000;
392
392
  }
393
+ /**
394
+ * Stale-shadow demotion. Measured failure mode (token A/B, question x1): recall answered with a
395
+ * SUPERSEDED architecture description because the old belief was still active, semantically strong,
396
+ * and outranked the newer truth. When two active beliefs in the ranked window describe the same fact
397
+ * — near-duplicate vectors, or moderately similar with a shared entity and the same type — but were
398
+ * written in different eras, the older one is treated as a stale shadow of the newer: its score is
399
+ * scaled down so the newer belief always outranks it. Nothing is deleted or re-statused here; real
400
+ * supersession stays the consolidator's job. Pinned records are never demoted.
401
+ */
402
+ export function demoteStaleShadows(ranked, vectorById, options = {}) {
403
+ if (!vectorById || vectorById.size === 0 || ranked.length < 2)
404
+ return ranked;
405
+ const scan = options.scan ?? 30;
406
+ const hardSim = options.hardSim ?? 0.8;
407
+ const softSim = options.softSim ?? 0.6;
408
+ const ageGapMs = options.ageGapMs ?? 3 * 24 * 60 * 60 * 1000;
409
+ const penalty = options.penalty ?? 0.35;
410
+ const window = ranked.slice(0, scan);
411
+ const recordTime = (item) => timestamp(item.record.updatedAt || item.record.createdAt);
412
+ const shadowOf = new Map(); // demoted id -> newer id it shadows
413
+ for (let i = 0; i < window.length; i += 1) {
414
+ for (let j = i + 1; j < window.length; j += 1) {
415
+ const a = window[i];
416
+ const b = window[j];
417
+ if (a.record.type !== b.record.type)
418
+ continue;
419
+ if (a.record.status !== "active" || b.record.status !== "active")
420
+ continue;
421
+ const va = vectorById.get(a.record.id);
422
+ const vb = vectorById.get(b.record.id);
423
+ if (!va || !vb)
424
+ continue;
425
+ const sim = cosineSimilarity(va, vb);
426
+ if (sim < softSim)
427
+ continue;
428
+ const sharedEntity = a.record.entities.some((e) => b.record.entities.includes(e));
429
+ if (sim < hardSim && !sharedEntity)
430
+ continue;
431
+ const [older, newer] = recordTime(a) <= recordTime(b) ? [a, b] : [b, a];
432
+ if (recordTime(newer) - recordTime(older) < ageGapMs)
433
+ continue;
434
+ if (older.record.pinned)
435
+ continue;
436
+ if (!shadowOf.has(older.record.id))
437
+ shadowOf.set(older.record.id, newer.record.id);
438
+ }
439
+ }
440
+ if (shadowOf.size === 0)
441
+ return ranked;
442
+ return ranked
443
+ .map((item) => {
444
+ const newerId = shadowOf.get(item.record.id);
445
+ if (!newerId)
446
+ return item;
447
+ return {
448
+ ...item,
449
+ score: roundScore(item.score * penalty),
450
+ explanation: `${item.explanation}; stale shadow of newer belief ${newerId}`,
451
+ reasons: [...item.reasons, { kind: "status", label: `stale shadow of ${newerId}`, score: 0 }]
452
+ };
453
+ })
454
+ .sort((left, right) => {
455
+ if (right.score !== left.score)
456
+ return right.score - left.score;
457
+ return timestamp(right.record.updatedAt) - timestamp(left.record.updatedAt);
458
+ });
459
+ }
Binary file
Binary file
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "peon-mem",
3
- "version": "1.0.0",
3
+ "version": "1.0.2",
4
4
  "type": "module",
5
5
  "main": "dist/index.js",
6
6
  "bin": {
@@ -21,7 +21,7 @@
21
21
  "dependencies": {
22
22
  "@modelcontextprotocol/sdk": "^1.23.0"
23
23
  },
24
- "description": "Local-first hierarchical memory brain for AI coding agents auto-capture, LLM consolidation into beliefs, hybrid retrieval, per-prompt injection, a living Neural Universe monitor, and a daily self-improvement loop. Works with Claude Code, Codex, and any MCP client.",
24
+ "description": "Local-first hierarchical memory brain for AI coding agents \u2014 auto-capture, LLM consolidation into beliefs, hybrid retrieval, per-prompt injection, a living Neural Universe monitor, and a daily self-improvement loop. Works with Claude Code, Codex, and any MCP client.",
25
25
  "license": "MIT",
26
26
  "keywords": [
27
27
  "memory",
@@ -36,7 +36,7 @@
36
36
  ],
37
37
  "repository": {
38
38
  "type": "git",
39
- "url": "https://github.com/VineetV2/peon-mem"
39
+ "url": "git+https://github.com/VineetV2/peon-mem.git"
40
40
  },
41
41
  "engines": {
42
42
  "node": ">=20"
@@ -53,5 +53,6 @@
53
53
  "docs/assets",
54
54
  "README.md",
55
55
  "LICENSE"
56
- ]
57
- }
56
+ ],
57
+ "mcpName": "io.github.VineetV2/peon-mem"
58
+ }
@@ -1,14 +1,15 @@
1
1
  #!/usr/bin/env node
2
- import { appendFile, mkdir, readFile, rm, writeFile } from "node:fs/promises";
2
+ import { appendFile, mkdir, readFile, rename, rm, writeFile } from "node:fs/promises";
3
3
  import { createReadStream as fsCreateReadStream, existsSync } from "node:fs";
4
4
  import { homedir } from "node:os";
5
5
  import { dirname, join } from "node:path";
6
6
  import { createInterface } from "node:readline";
7
7
 
8
- // Hard off-switch (for A/B testing). When PEON_DISABLED is set, the hook does
9
- // nothing at all — no recording, no context injection, no daemon calls — so the
10
- // session runs purely on the cloud model with zero Peon involvement.
11
- if (/^(1|true|yes|on)$/i.test(process.env.PEON_DISABLED || "")) process.exit(0);
8
+ // Hard off-switch (for A/B testing). When PEON_DISABLED is set, Peon does nothing —
9
+ // no injection, no recording, no daemon calls — EXCEPT counting the session's token
10
+ // usage on Stop/SessionEnd, because the OFF arm's whole purpose is to be the baseline
11
+ // in the token A/B comparison. (Handled inside the main block below.)
12
+ const PEON_OFF = /^(1|true|yes|on)$/i.test(process.env.PEON_DISABLED || "");
12
13
 
13
14
  const daemonUrl = (process.env.PEON_DAEMON_URL || "http://127.0.0.1:3737").replace(/\/$/, "");
14
15
  const stateDir =
@@ -39,6 +40,12 @@ const PEON_FIRST_DIRECTIVE =
39
40
  // so it is initialized before the top-level await block calls getProjectContext.
40
41
  const MAX_CONTEXT_QUERY_CHARS = 2000;
41
42
 
43
+ // Token A/B ledger paths — declared BEFORE the top-level await block (everything the main
44
+ // block touches must be initialized first, or the reference throws a silent TDZ error;
45
+ // that exact bug made token tracking a no-op for weeks).
46
+ const TOKEN_AB_LOG = join(homedir(), "Library", "Application Support", "Peon", "token-ab-log.jsonl");
47
+ const TOKEN_AB_LOGGED_SESSIONS = join(homedir(), "Library", "Application Support", "Peon", "token-ab-sessions.json");
48
+
42
49
  // Resolve any working directory to its ONE project brain, so memory never fragments:
43
50
  // 1. collapse git-worktree paths to the repo root (…/.claude/worktrees/x → repo)
44
51
  // 2. walk UP to the TOPMOST ancestor that already holds a Peon brain (.peon), bounded by
@@ -88,6 +95,11 @@ try {
88
95
  readText(input, ["session_id", "sessionId", "conversation_id", "conversationId", "thread_id", "threadId"]) ||
89
96
  externalSessionId;
90
97
 
98
+ if (PEON_OFF) {
99
+ if (eventName === "Stop" || eventName === "SessionEnd") await trackTokenUsage(input, projectPath, externalSessionId);
100
+ process.exit(0);
101
+ }
102
+
91
103
  if (eventName === "SessionStart") {
92
104
  await ensurePeonSession({ projectPath, externalSessionId, client: hookClient });
93
105
  const context = await getProjectContext(projectPath, "recent project context decisions artifacts current work");
@@ -159,14 +171,17 @@ try {
159
171
  // A response TURN finished — capture the summary and let Peon consolidate,
160
172
  // but keep the session ALIVE. Stop fires after every turn; the Claude session
161
173
  // (and the Peon session) continues across turns and only ends on SessionEnd.
174
+ // Token tracking runs FIRST: consolidation below can outlive the hook timeout
175
+ // ("Hook cancelled"), which used to silently drop the token ledger row.
176
+ await trackTokenUsage(input, projectPath, externalSessionId);
162
177
  const finalMessage = extractAssistantSummary(input);
163
178
  if (finalMessage.trim()) {
164
179
  await recordWithSession({ projectPath, externalSessionId, client: hookClient }, (sessionId) =>
165
180
  postJson("/events", { sessionId, type: "assistant_summary", content: finalMessage.slice(0, 2000) }));
166
181
  }
167
182
  await postJson("/process/auto", { projectPath, trigger: "turn_end" }).catch(() => undefined);
168
- await trackTokenUsage(input, projectPath, externalSessionId);
169
183
  } else if (eventName === "SessionEnd") {
184
+ await trackTokenUsage(input, projectPath, externalSessionId);
170
185
  const finalMessage = extractAssistantSummary(input);
171
186
  if (finalMessage.trim()) {
172
187
  await recordWithSession({ projectPath, externalSessionId, client: hookClient }, (sessionId) =>
@@ -180,7 +195,6 @@ try {
180
195
  // stale session id gets replayed forever and recording silently dies.
181
196
  await removeSession(externalSessionId);
182
197
  }
183
- await trackTokenUsage(input, projectPath, externalSessionId);
184
198
  }
185
199
  } catch (error) {
186
200
  await appendLocalError({
@@ -400,16 +414,11 @@ function safeName(value) {
400
414
  return String(value).replace(/[^a-zA-Z0-9_.-]/g, "_");
401
415
  }
402
416
 
403
- const TOKEN_AB_LOG = join(homedir(), "Library", "Application Support", "Peon", "token-ab-log.jsonl");
404
-
405
- const TOKEN_AB_LOGGED_SESSIONS = join(homedir(), "Library", "Application Support", "Peon", "token-ab-sessions.json");
406
-
407
417
  async function recordTokenUsage({ projectPath, externalSessionId, transcriptPath }) {
408
418
  try {
409
- // Only log once per session Stop fires on every response turn, not just session end.
410
- const logged = JSON.parse(await readFile(TOKEN_AB_LOGGED_SESSIONS, "utf8").catch(() => "[]"));
411
- if (logged.includes(externalSessionId)) return;
412
-
419
+ // Upsert per session: Stop fires after every turn, so each firing re-reads the full
420
+ // transcript and replaces this session's row — the last Stop leaves the final totals.
421
+ // (The old once-per-session gate froze turn-1 partials and dropped the rest.)
413
422
  const totals = { input: 0, output: 0, cacheRead: 0, cacheCreate: 0, model: "unknown" };
414
423
  const rl = createInterface({ input: fsCreateReadStream(transcriptPath), crlfDelay: Infinity });
415
424
  for await (const line of rl) {
@@ -425,6 +434,9 @@ async function recordTokenUsage({ projectPath, externalSessionId, transcriptPath
425
434
  if (obj?.message?.model) totals.model = obj.message.model;
426
435
  } catch { /* skip malformed lines */ }
427
436
  }
437
+ // A Stop can fire before any usage lines hit the transcript — never write an empty row.
438
+ if (totals.input + totals.output === 0) return;
439
+
428
440
  const peonEnabled = !/^(1|true|yes|on)$/i.test(process.env.PEON_DISABLED || "");
429
441
  const record = {
430
442
  ts: new Date().toISOString(),
@@ -438,13 +450,20 @@ async function recordTokenUsage({ projectPath, externalSessionId, transcriptPath
438
450
  cacheCreateTokens: totals.cacheCreate,
439
451
  totalTokens: totals.input + totals.output,
440
452
  };
453
+ if (process.env.PEON_AB_TAG) record.tag = process.env.PEON_AB_TAG;
441
454
  const dir = join(homedir(), "Library", "Application Support", "Peon");
442
455
  await mkdir(dir, { recursive: true });
443
- await appendFile(TOKEN_AB_LOG, JSON.stringify(record) + "\n", "utf8");
444
- // Mark session as logged (keep last 500 to avoid unbounded growth)
445
- logged.push(externalSessionId);
446
- await writeFile(TOKEN_AB_LOGGED_SESSIONS, JSON.stringify(logged.slice(-500)), "utf8");
447
- } catch { /* non-fatal */ }
456
+ // Rewrite the log with this session's previous rows dropped — file stays one row per session.
457
+ const existing = (await readFile(TOKEN_AB_LOG, "utf8").catch(() => ""))
458
+ .split("\n").filter(Boolean)
459
+ .filter((line) => {
460
+ try { return JSON.parse(line).sessionId !== externalSessionId; } catch { return true; }
461
+ });
462
+ existing.push(JSON.stringify(record));
463
+ const tmp = TOKEN_AB_LOG + ".tmp";
464
+ await writeFile(tmp, existing.slice(-2000).join("\n") + "\n", "utf8");
465
+ await rename(tmp, TOKEN_AB_LOG);
466
+ } catch (e) { if (process.env.PEON_AB_DEBUG) console.error("AB-ERR:", e && e.message); }
448
467
  }
449
468
 
450
469
  async function appendLocalError(error) {