brain-memory 0.1.0-beta.34 → 0.1.0-beta.35

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,43 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.1.0-beta.35] - 2026-08-01
10
+
11
+ ### Added
12
+
13
+ - **`/brain:import` — cold-start a brain from transcripts your agents already
14
+ wrote.** A new brain is empty, and an empty brain is worth nothing until
15
+ something fills it; meanwhile every coding agent has been keeping months of
16
+ local history. `brain import` harvests it: session titles, user prompts,
17
+ projects, branches and edited files, with harness wrappers
18
+ (`<system-reminder>`, slash-command tags, command output), subagent sidechain
19
+ traffic, and bare acknowledgements filtered out. The digest is bounded on
20
+ three axes (sessions, prompts per session, total characters) and
21
+ round-robins across projects, so one busy repo cannot consume the whole
22
+ budget — on a real 261-session history that lifted project coverage from 5
23
+ to 13 within the same ~15k tokens.
24
+
25
+ The CLI harvests; the **agent distills**. Extracting meaning is a semantic
26
+ judgement, and making it in the CLI would mean shipping an embedding model
27
+ or calling an LLM — the two things Brain exists to avoid. So the harvester
28
+ reports facts and never infers, and distilled memories go through the
29
+ ordinary `brain memorize` path as `agent-inferred`: an import inherits the
30
+ existing provenance ceilings for free and can never pin, entrench, or
31
+ outrank something the user said directly.
32
+
33
+ Import is incremental — a cursor at `~/.brain/.import/state.json` records
34
+ which sessions have been read so re-runs offer only new ones (`--all`
35
+ overrides). `--mark` validates ids against the source's real sessions and
36
+ exits non-zero on any that match nothing: silently accepting a truncated or
37
+ mistyped id would report a session as retired while the cursor never matches
38
+ it, re-offering that session forever. Scope with `--project`,
39
+ `--since 30d|6m|2026-01-01`, `--limit`.
40
+ `brain import --sources` lists detected history stores. `brain import` only
41
+ ever reads and prints; no memory is written without the agent.
42
+
43
+ Claude Code is the first source adapter; the registry in `src/harvest.js`
44
+ takes additional agents without changes downstream.
45
+
9
46
  ## [0.1.0-beta.34] - 2026-07-26
10
47
 
11
48
  ### Added
package/README.md CHANGED
@@ -242,10 +242,27 @@ The everyday loop is **ambient** — `remember` and `memorize` run automatically
242
242
  | `/brain:status` | Dashboard with brain health metrics and recommendations |
243
243
  | `/brain:pin [id\|query]` | Pin a memory to the always-present tier — loads every session, never decays. Toggles: also unpins (`--off`) |
244
244
  | `/brain:forget [target]` | Decay, archive, or remove memories. `--deep` performs forensic erasure — traces and removes every reference |
245
+ | `/brain:import [--project P] [--since 30d]` | Cold-start a new brain from transcripts your agents already wrote. Incremental — safe to re-run |
245
246
  | `/brain:sync [subcommand]` | Sync via Brain Cloud, Git remote, or export/import (auto-initializes the brain on first run) |
246
247
  | `/brain:skills [list\|show\|add\|use\|remove\|export]` | Manage procedural skills — reusable how-to workflows with progressive disclosure |
247
248
  | `/brain:sleep [scope]` | Full maintenance cycle — 9 neuroscience-inspired phases (replay, consolidation, review reinforcement, pruning, dreaming, …). Usually runs automatically/in the background |
248
249
 
250
+ ## Cold Start
251
+
252
+ A new brain is empty, and an empty brain is worth nothing until something fills it. But your agents have been keeping local transcripts for months — decisions, preferences, corrections, conventions. `/brain:import` reads them.
253
+
254
+ ```bash
255
+ brain import --sources # what history exists on this machine
256
+ brain import --project my-app # digest that project's past sessions
257
+ brain import --since 30d # or scope by time
258
+ ```
259
+
260
+ The split matters. The CLI **harvests** — deterministic, budget-bounded, no model call: it extracts session titles, user prompts, projects, branches, and edited files, filters out harness noise and subagent traffic, and interleaves across projects so one busy repo can't eat the budget. The agent then **distills** that digest into memories through the ordinary `brain memorize` path.
261
+
262
+ That split is deliberate. Extracting meaning is a semantic judgement, and doing it in the CLI would mean shipping an embedding model or calling an LLM — the two things Brain exists to avoid. So the harvester reports facts and never infers, and every imported memory lands as `agent-inferred` provenance: it inherits the existing ceilings for free, and **an import can never pin, entrench, or outrank something you said directly.**
263
+
264
+ Import is incremental. A cursor in `~/.brain/.import/state.json` records which sessions have been read, so re-running only offers new ones (`--all` overrides). Nothing is written without you — `brain import` only ever reads and prints.
265
+
249
266
  ## Session Lifecycle
250
267
 
251
268
  Brain Memory works automatically in the background — no commands needed for basic context awareness.
package/bin/brain.js CHANGED
@@ -33,6 +33,7 @@ const DELEGATED = {
33
33
  pin: 'pin.js',
34
34
  unpin: 'unpin.js',
35
35
  skill: 'skill.js',
36
+ import: 'import.js',
36
37
  };
37
38
 
38
39
  // install.js finds its positional subcommand from process.argv itself, so it is
@@ -59,6 +60,13 @@ Memory
59
60
  session-start [--project P] [--task T] [--top N]
60
61
  Budget-bounded startup payload (agent-invoked)
61
62
 
63
+ Cold start
64
+ import [--source S] [--project P] [--since 30d] [--limit N] [--all]
65
+ Digest past agent transcripts into import candidates
66
+ import --sources List detected agent history stores
67
+ import --mark <id> [<id>...]
68
+ Record sessions as imported (incremental cursor)
69
+
62
70
  Sync
63
71
  cloud <login|logout|push|pull|status>
64
72
 
package/bin/import.js ADDED
@@ -0,0 +1,132 @@
1
+ #!/usr/bin/env node
2
+
3
+ /**
4
+ * brain import — cold-start a brain from transcripts your agents already wrote.
5
+ *
6
+ * Emits a deterministic, budget-bounded digest of past sessions. The agent reads
7
+ * that digest and writes memories through the ordinary `brain memorize` path, so
8
+ * imported memories go through the same provenance gate, scoring, and index as
9
+ * anything else. Nothing here writes a memory.
10
+ *
11
+ * Usage:
12
+ * brain import [--source claude-code] [--project P] [--since ISO] [--limit N] [--all]
13
+ * brain import --sources List detected history stores
14
+ * brain import --mark <id> [<id>...] Record sessions as imported
15
+ *
16
+ * The `--mark` step is what makes import incremental: the agent calls it after a
17
+ * successful memorize so the next run offers only new sessions.
18
+ */
19
+
20
+ const fs = require('fs');
21
+ const path = require('path');
22
+
23
+ const { getBrainDir } = require('../src/index-manager');
24
+ const { harvest, availableSources, markImported } = require('../src/harvest');
25
+
26
+ function parseArgs(argv) {
27
+ const args = {
28
+ source: 'claude-code',
29
+ project: null,
30
+ since: null,
31
+ limit: null,
32
+ all: false,
33
+ sources: false,
34
+ mark: [],
35
+ };
36
+ for (let i = 0; i < argv.length; i++) {
37
+ switch (argv[i]) {
38
+ case '--source': args.source = argv[++i]; break;
39
+ case '--project': args.project = argv[++i]; break;
40
+ case '--since': args.since = argv[++i]; break;
41
+ case '--limit': args.limit = parseInt(argv[++i], 10) || null; break;
42
+ case '--all': args.all = true; break;
43
+ case '--sources': args.sources = true; break;
44
+ case '--mark':
45
+ // Consume every following non-flag token as a session id.
46
+ while (argv[i + 1] && !argv[i + 1].startsWith('--')) args.mark.push(argv[++i]);
47
+ break;
48
+ default: break;
49
+ }
50
+ }
51
+ return args;
52
+ }
53
+
54
+ /**
55
+ * Normalize `--since` into an ISO timestamp.
56
+ *
57
+ * Accepts a relative shorthand (`30d`, `6m`, `1y`) as well as any date string
58
+ * Date can parse, because "the last month of work" is the natural way to scope
59
+ * an import and computing that date by hand is friction.
60
+ */
61
+ function normalizeSince(value) {
62
+ if (!value) return null;
63
+ const relative = /^(\d+)([dmy])$/i.exec(value.trim());
64
+ if (relative) {
65
+ const amount = parseInt(relative[1], 10);
66
+ const date = new Date();
67
+ if (relative[2].toLowerCase() === 'd') date.setDate(date.getDate() - amount);
68
+ if (relative[2].toLowerCase() === 'm') date.setMonth(date.getMonth() - amount);
69
+ if (relative[2].toLowerCase() === 'y') date.setFullYear(date.getFullYear() - amount);
70
+ return date.toISOString();
71
+ }
72
+ const parsed = new Date(value);
73
+ if (Number.isNaN(parsed.getTime())) return null;
74
+ return parsed.toISOString();
75
+ }
76
+
77
+ function main(argv = process.argv.slice(2)) {
78
+ const args = parseArgs(argv);
79
+
80
+ if (args.sources) {
81
+ console.log(JSON.stringify({ sources: availableSources() }, null, 2));
82
+ return;
83
+ }
84
+
85
+ // A brain must exist before we offer to fill it — otherwise the agent gets a
86
+ // digest it has nowhere to write.
87
+ const brainDir = getBrainDir();
88
+ if (!fs.existsSync(path.join(brainDir, 'index.json'))) {
89
+ console.error(JSON.stringify({ error: 'Brain not initialized. Run `brain install` first.' }));
90
+ process.exit(1);
91
+ }
92
+
93
+ if (args.mark.length > 0) {
94
+ const result = markImported(args.source, args.mark);
95
+ console.log(JSON.stringify({
96
+ marked: args.mark.length - result.unknown.length,
97
+ ...result,
98
+ source: args.source,
99
+ ...(result.unknown.length
100
+ ? { warning: `${result.unknown.length} id(s) matched no ${args.source} session and were not recorded — check for a truncated or mistyped session id.` }
101
+ : {}),
102
+ }, null, 2));
103
+ // A partial mark is a real failure: the caller believes those sessions are
104
+ // done, and nothing else will ever tell them otherwise.
105
+ if (result.unknown.length > 0) process.exit(1);
106
+ return;
107
+ }
108
+
109
+ if (args.since && !normalizeSince(args.since)) {
110
+ console.error(JSON.stringify({ error: `Could not parse --since "${args.since}" (try 30d, 6m, or 2026-01-01)` }));
111
+ process.exit(1);
112
+ }
113
+
114
+ const digest = harvest({
115
+ source: args.source,
116
+ project: args.project,
117
+ since: normalizeSince(args.since),
118
+ limit: args.limit,
119
+ all: args.all,
120
+ });
121
+
122
+ if (digest.error) {
123
+ console.error(JSON.stringify({ error: digest.error }));
124
+ process.exit(1);
125
+ }
126
+
127
+ console.log(JSON.stringify(digest, null, 2));
128
+ }
129
+
130
+ if (require.main === module) main();
131
+
132
+ module.exports = { main, parseArgs, normalizeSince };
@@ -0,0 +1,125 @@
1
+ ---
2
+ description: Cold-start your brain from transcripts your agents already wrote
3
+ argument-hint: "[--project P] [--since 30d] [--limit N]"
4
+ ---
5
+
6
+ # /brain:import — Cold-Start From Past Sessions
7
+
8
+ A new brain is empty. But your agents have been keeping transcripts for months — decisions, preferences, corrections, conventions. This command reads those transcripts and turns them into real memories.
9
+
10
+ **User input:** $ARGUMENTS
11
+
12
+ **Flags** (passed straight through to the CLI):
13
+ - `--source <id>` — Which history store to read (default `claude-code`; `--sources` lists what's detected)
14
+ - `--project <name>` — Only sessions from one project
15
+ - `--since <30d|6m|2026-01-01>` — Only sessions since then
16
+ - `--limit <N>` — Max sessions to consider (default 40)
17
+ - `--all` — Re-offer sessions already imported
18
+
19
+ ## Your role
20
+
21
+ The CLI harvests. **You distill.** It hands you a digest of past sessions — titles, prompts, projects, files touched, dates. It does not decide what matters. That is the entire job here, and it is a judgement call the CLI deliberately does not make.
22
+
23
+ ## Steps
24
+
25
+ ### 1. Harvest
26
+
27
+ ```bash
28
+ brain import $ARGUMENTS
29
+ ```
30
+
31
+ Returns `{ source, scanned, budget, sessions[] }`. Each session has `session_id`, `title`, `project`, `git_branch`, `started`/`ended`, `turns`, `files_touched`, and a sample of `prompts`.
32
+
33
+ Read `scanned` honestly when you report — the fields mean different things:
34
+ - `skipped` — sessions that held nothing readable (too few real prompts)
35
+ - `filtered` — sessions excluded by the user's own `--project`/`--since` scoping, **not** discarded as junk
36
+ - `already_imported` — sessions a previous run already covered
37
+ - `remaining` — candidates that did not fit this pass; the number to quote when suggesting a re-run
38
+
39
+ If `scanned.files` is 0, tell the user no history was found and stop.
40
+
41
+ ### 2. Check what the brain already knows
42
+
43
+ **Do not skip this on a brain that is already populated.** Import is most often run cold, but it is also run against a brain with hundreds of memories, and there the dominant failure is duplication — re-writing facts that are already stored, which inflates the brain and splits recall across near-identical entries.
44
+
45
+ Before distilling, recall against the themes you can see in the digest:
46
+
47
+ ```bash
48
+ brain recall "<theme from the digest>" --project <project> --top 4
49
+ ```
50
+
51
+ If a theme already scores well against an existing memory, drop it. Only write what is genuinely absent, or what materially *updates* something already stored — and if it's an update, say so in the content rather than filing a near-duplicate.
52
+
53
+ ### 3. Read for durable facts, not events
54
+
55
+ You are looking at what someone *did*, and trying to recover what is still *true*. Those are different things.
56
+
57
+ **Worth remembering** — things that outlive the session:
58
+ - Stated preferences and conventions ("always use X", "I hate Y", "we commit straight to main")
59
+ - Architecture and tooling decisions, with the rationale if it's visible
60
+ - Recurring corrections — if the user pushed back on the same thing three times, that is a preference
61
+ - The shape of each project: what it is, its stack, where it deploys
62
+ - Constraints, deadlines, ongoing goals
63
+
64
+ **Not worth remembering** — things that were only true that afternoon:
65
+ - "fix the failing test", "what does this function do", "run the build"
66
+ - Anything already recorded in the repo, git history, or a CLAUDE.md
67
+ - One-off debugging that ended when the bug was fixed
68
+ - Anything superseded by later work in the digest
69
+
70
+ A good import from 40 sessions is roughly **5–20 memories**, not 40. One memory per session means you transcribed instead of distilled.
71
+
72
+ ### 4. Prefer synthesis across sessions
73
+
74
+ The digest's real value is the pattern across sessions, which no single session contains. Six sessions in the same repo tell you what that project *is* — that is one strong `semantic` memory, worth more than six thin episodic ones.
75
+
76
+ Merge aggressively. Write the memory the user would want to find in six months.
77
+
78
+ ### 5. Classify and write
79
+
80
+ Use the classification rules in `/brain:memorize` — same types, same cognitive types, same hierarchy under `professional/` etc.
81
+
82
+ Two rules specific to import:
83
+
84
+ **`origin` is `agent-inferred`.** You are inferring durable facts from a transcript, not being told them. Do not use `user` — the CLI's provenance gate will reject any attempt to pin or entrench from this path, which is the intended behavior. If a fact deserves entrenchment, the user can `/brain:pin` it deliberately afterwards.
85
+
86
+ **Confidence should reflect the evidence.** A convention stated once in passing is not a convention you are sure about. Something the user repeated across four sessions is. Set `confidence` accordingly rather than defaulting everything to 0.7.
87
+
88
+ Set `encoding_context.project` to the digest's `project` field, and `source` to `"import:<source>:<session_id>"` so the memory traces back to the transcript it came from.
89
+
90
+ Then write them:
91
+
92
+ ```bash
93
+ brain memorize <<'EOF'
94
+ { "memories": [ ... ] }
95
+ EOF
96
+ ```
97
+
98
+ ### 6. Mark what you used
99
+
100
+ ```bash
101
+ brain import --mark <session_id> <session_id> ...
102
+ ```
103
+
104
+ Mark **every session you read**, including the ones you decided held nothing worth keeping — otherwise the next run offers them again and you re-litigate the same judgement. This cursor is what makes import incremental.
105
+
106
+ Use the **full** `session_id` from the digest, never a shortened form you printed for display. The CLI rejects ids that match no real session and exits non-zero; check the `unknown` array if it does.
107
+
108
+ ### 7. Report
109
+
110
+ ```
111
+ ◉ Imported <N> memories from <M> sessions across <P> projects
112
+
113
+ <type> <title>
114
+ ...
115
+
116
+ <scanned.remaining> sessions remain — re-run to continue.
117
+ ```
118
+
119
+ Quote `scanned.remaining` for the re-run line, and mention `scanned.filtered` only if the user passed a scoping flag.
120
+
121
+ ## Notes
122
+
123
+ - **Safe to re-run.** The cursor means a second run offers only new sessions. `--all` overrides it.
124
+ - **Start scoped if the history is large.** `--project <name>` or `--since 30d` gives a better first pass than trying to swallow a year at once.
125
+ - **Nothing is written without you.** `brain import` only ever reads and prints; every memory goes through `brain memorize` and its provenance gate.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "brain-memory",
3
- "version": "0.1.0-beta.34",
3
+ "version": "0.1.0-beta.35",
4
4
  "description": "A brain-inspired hierarchical memory system for AI coding agents. Deep nested file structure that mimics human memory with time-decay and strength-based recall.",
5
5
  "keywords": [
6
6
  "claude-code",
@@ -44,7 +44,7 @@
44
44
  "url": "git+https://github.com/omelas-tech/brain.git"
45
45
  },
46
46
  "scripts": {
47
- "test": "node --test test/index-manager.test.js test/scorer.test.js test/tfidf.test.js test/receipt.test.js test/recall.test.js test/recall-cli.test.js test/forget.test.js test/functional.test.js test/erasure.test.js test/stress.test.js test/crypto.test.js test/git-sync.test.js test/cloud-sync.test.js test/export-import.test.js test/install.test.js test/prompts.test.js test/session-start.test.js test/memorize.test.js test/pinning.test.js test/skills.test.js test/contradiction.test.js",
47
+ "test": "node --test test/index-manager.test.js test/scorer.test.js test/tfidf.test.js test/receipt.test.js test/recall.test.js test/recall-cli.test.js test/forget.test.js test/functional.test.js test/erasure.test.js test/stress.test.js test/crypto.test.js test/git-sync.test.js test/cloud-sync.test.js test/export-import.test.js test/install.test.js test/prompts.test.js test/session-start.test.js test/memorize.test.js test/pinning.test.js test/skills.test.js test/contradiction.test.js test/harvest.test.js",
48
48
  "test:integrations": "npm run test:integrations:openclaw && npm run test:integrations:copilot && npm run test:integrations:kilo && npm run test:integrations:hermes",
49
49
  "test:integrations:openclaw": "node --test integrations/openclaw/test/*.test.mjs",
50
50
  "test:integrations:copilot": "node --test integrations/copilot/test/*.test.mjs",
@@ -58,6 +58,9 @@ Core:
58
58
  - `/brain-pin [id|query]` — Pin a memory to the always-present tier (toggle — also unpins, e.g. `--off`)
59
59
  - `/brain-forget [target]` — Decay or archive memories (`--deep` = forensic erasure of every reference)
60
60
 
61
+ Cold start:
62
+ - `/brain-import [--project P] [--since 30d]` — Seed a new brain from transcripts your agents already wrote
63
+
61
64
  Sync & skills:
62
65
  - `/brain-sync [subcommand]` — Sync via Brain Cloud, Git remote, or export/import (auto-initializes on first run)
63
66
  - `/brain-skills [list|show|add|use|remove|export]` — Manage procedural skills
package/prompts/claude.md CHANGED
@@ -57,6 +57,9 @@ The everyday loop is **ambient** — `remember` and `memorize` run automatically
57
57
  - `/brain:pin [id|query]` — Pin a memory to the always-present tier (toggle — also unpins, e.g. `--off`)
58
58
  - `/brain:forget [target]` — Decay or archive memories (`--deep` = forensic erasure of every reference)
59
59
 
60
+ **Cold start**
61
+ - `/brain:import [--project P] [--since 30d]` — Seed a new brain from transcripts your agents already wrote
62
+
60
63
  **Sync & skills**
61
64
  - `/brain:sync [subcommand]` — Sync via Brain Cloud, Git remote, or export/import (auto-initializes on first run)
62
65
  - `/brain:skills [list|show|add|use|remove|export]` — Manage procedural skills
@@ -60,6 +60,9 @@ Core:
60
60
  - `/brain-pin [id|query]` — Pin a memory to the always-present tier (toggle — also unpins, e.g. `--off`)
61
61
  - `/brain-forget [target]` — Decay or archive memories (`--deep` = forensic erasure of every reference)
62
62
 
63
+ Cold start:
64
+ - `/brain-import [--project P] [--since 30d]` — Seed a new brain from transcripts your agents already wrote
65
+
63
66
  Sync & skills:
64
67
  - `/brain-sync [subcommand]` — Sync via Brain Cloud, Git remote, or export/import (auto-initializes on first run)
65
68
  - `/brain-skills [list|show|add|use|remove|export]` — Manage procedural skills
package/prompts/kilo.md CHANGED
@@ -59,6 +59,9 @@ The everyday loop is **ambient** — `remember` and `memorize` run automatically
59
59
  - `/brain:pin [id|query]` — Pin a memory to the always-present tier (toggle — also unpins, e.g. `--off`)
60
60
  - `/brain:forget [target]` — Decay or archive memories (`--deep` = forensic erasure of every reference)
61
61
 
62
+ **Cold start**
63
+ - `/brain:import [--project P] [--since 30d]` — Seed a new brain from transcripts your agents already wrote
64
+
62
65
  **Sync & skills**
63
66
  - `/brain:sync [subcommand]` — Sync via Brain Cloud, Git remote, or export/import (auto-initializes on first run)
64
67
  - `/brain:skills [list|show|add|use|remove|export]` — Manage procedural skills
package/prompts/openai.md CHANGED
@@ -58,6 +58,9 @@ Core:
58
58
  - `/brain-pin [id|query]` — Pin a memory to the always-present tier (toggle — also unpins, e.g. `--off`)
59
59
  - `/brain-forget [target]` — Decay or archive memories (`--deep` = forensic erasure of every reference)
60
60
 
61
+ Cold start:
62
+ - `/brain-import [--project P] [--since 30d]` — Seed a new brain from transcripts your agents already wrote
63
+
61
64
  Sync & skills:
62
65
  - `/brain-sync [subcommand]` — Sync via Brain Cloud, Git remote, or export/import (auto-initializes on first run)
63
66
  - `/brain-skills [list|show|add|use|remove|export]` — Manage procedural skills
@@ -57,6 +57,9 @@ The everyday loop is **ambient** — `remember` and `memorize` run automatically
57
57
  - `/brain:pin [id|query]` — Pin a memory to the always-present tier (toggle — also unpins, e.g. `--off`)
58
58
  - `/brain:forget [target]` — Decay or archive memories (`--deep` = forensic erasure of every reference)
59
59
 
60
+ **Cold start**
61
+ - `/brain:import [--project P] [--since 30d]` — Seed a new brain from transcripts your agents already wrote
62
+
60
63
  **Sync & skills**
61
64
  - `/brain:sync [subcommand]` — Sync via Brain Cloud, Git remote, or export/import (auto-initializes on first run)
62
65
  - `/brain:skills [list|show|add|use|remove|export]` — Manage procedural skills
package/src/harvest.js ADDED
@@ -0,0 +1,562 @@
1
+ /**
2
+ * Harvest — cold-start memory from the transcripts your agents already wrote.
3
+ *
4
+ * A fresh brain is empty, and an empty brain is worth nothing until someone
5
+ * feeds it. But every coding agent already keeps a local record of months of
6
+ * work: prompts, decisions, corrections, preferences. This module reads those
7
+ * records and turns them into a *digest* — a deterministic, budget-bounded
8
+ * summary of candidate sessions.
9
+ *
10
+ * What this module deliberately does NOT do is decide what is worth
11
+ * remembering. That is a semantic judgement, and Brain's whole design puts
12
+ * semantic judgement in the agent (see CLAUDE.md, "Agent-driven intelligence").
13
+ * Doing extraction here would mean shipping an embedding model or calling an
14
+ * LLM — the two things Brain exists to avoid. So the split is:
15
+ *
16
+ * harvest (this file, deterministic) → digest
17
+ * digest → agent reads it → `brain memorize` (existing path)
18
+ *
19
+ * The agent's distillation is `agent-inferred` provenance by construction, so
20
+ * imported memories inherit the existing ASI06 ceilings for free: an import can
21
+ * never pin, never entrench, and never outrank something the user said directly.
22
+ */
23
+
24
+ const fs = require('fs');
25
+ const os = require('os');
26
+ const path = require('path');
27
+
28
+ const { getBrainDir, atomicWriteSync } = require('./index-manager');
29
+
30
+ // --- Budgets ---
31
+ //
32
+ // A 500-session history would blow any context window, so the digest is capped
33
+ // on three independent axes. Sessions are the coarse cap, prompts the medium,
34
+ // and total characters the backstop that holds even if a single session is
35
+ // pathological.
36
+ const DEFAULTS = {
37
+ maxSessions: 40,
38
+ maxPromptsPerSession: 6,
39
+ maxPromptChars: 400,
40
+ maxDigestChars: 60000, // ~15k tokens at chars/4
41
+ };
42
+
43
+ // Sessions below this many real user prompts are almost always a mis-start, a
44
+ // one-line question, or an aborted run. They cost digest budget and yield
45
+ // nothing memorable.
46
+ const MIN_PROMPTS = 2;
47
+
48
+ // --- Text cleaning ---
49
+
50
+ // Harness-injected wrappers that are not user speech. Left in, they dominate the
51
+ // digest and teach the agent to memorize plumbing.
52
+ const STRIP_PATTERNS = [
53
+ /<system-reminder>[\s\S]*?<\/system-reminder>/g,
54
+ /<command-name>[\s\S]*?<\/command-name>/g,
55
+ /<command-message>[\s\S]*?<\/command-message>/g,
56
+ /<command-args>[\s\S]*?<\/command-args>/g,
57
+ /<local-command-stdout>[\s\S]*?<\/local-command-stdout>/g,
58
+ /<local-command-stderr>[\s\S]*?<\/local-command-stderr>/g,
59
+ /<user-prompt-submit-hook>[\s\S]*?<\/user-prompt-submit-hook>/g,
60
+ /\[Image #\d+\]/g,
61
+ /\[Request interrupted[^\]]*\]/g,
62
+ ];
63
+
64
+ // Whole prompts that carry no intent worth remembering.
65
+ const NOISE_PROMPTS = [
66
+ /^caveat: the messages below were generated by the user while running local commands/i,
67
+ /^\/[a-z0-9:_-]+\s*$/i, // a bare slash-command invocation
68
+ /^(y|n|yes|no|ok|okay|k|sure|thanks|thank you|ty|continue|go|go ahead|proceed|yep|yeah|nope)\b[.!]*$/i,
69
+ ];
70
+
71
+ /** Strip harness wrappers and normalize whitespace. Returns '' for pure noise. */
72
+ function cleanPromptText(raw) {
73
+ if (typeof raw !== 'string') return '';
74
+ let text = raw;
75
+ for (const pattern of STRIP_PATTERNS) text = text.replace(pattern, ' ');
76
+ text = text.replace(/\s+/g, ' ').trim();
77
+ if (!text) return '';
78
+ if (NOISE_PROMPTS.some((p) => p.test(text))) return '';
79
+ return text;
80
+ }
81
+
82
+ /** Truncate on a word boundary so the digest never ends mid-token. */
83
+ function truncate(text, max) {
84
+ if (text.length <= max) return text;
85
+ const cut = text.slice(0, max);
86
+ const lastSpace = cut.lastIndexOf(' ');
87
+ return (lastSpace > max * 0.6 ? cut.slice(0, lastSpace) : cut).trimEnd() + '…';
88
+ }
89
+
90
+ /**
91
+ * Pull the user-authored text out of one transcript record.
92
+ *
93
+ * `message.content` is either a bare string (older transcripts) or an array of
94
+ * blocks. Only `text` blocks are user speech — `tool_result` blocks are replayed
95
+ * command output that merely happens to be recorded under the user role.
96
+ */
97
+ function userTextFrom(record) {
98
+ const content = record && record.message && record.message.content;
99
+ if (typeof content === 'string') return cleanPromptText(content);
100
+ if (!Array.isArray(content)) return '';
101
+ return cleanPromptText(
102
+ content
103
+ .filter((block) => block && block.type === 'text' && typeof block.text === 'string')
104
+ .map((block) => block.text)
105
+ .join('\n')
106
+ );
107
+ }
108
+
109
+ // --- JSONL reading ---
110
+
111
+ /**
112
+ * Parse a JSONL transcript, skipping unparseable lines.
113
+ *
114
+ * Transcripts are appended to live by a running agent, so the final line of an
115
+ * in-flight session is routinely a partial write. That is expected, not an
116
+ * error: skip the line and keep the rest of the session.
117
+ */
118
+ function readJsonl(file) {
119
+ let raw;
120
+ try {
121
+ raw = fs.readFileSync(file, 'utf-8');
122
+ } catch (_) {
123
+ return [];
124
+ }
125
+ const records = [];
126
+ for (const line of raw.split('\n')) {
127
+ if (!line.trim()) continue;
128
+ try {
129
+ records.push(JSON.parse(line));
130
+ } catch (_) {
131
+ // Truncated tail or corrupt line — skip.
132
+ }
133
+ }
134
+ return records;
135
+ }
136
+
137
+ // --- Source adapters ---
138
+ //
139
+ // Each adapter turns one agent's on-disk history into a common session shape.
140
+ // Adding a source means adding an entry here; nothing downstream changes.
141
+
142
+ const FILE_EDIT_TOOLS = new Set(['Edit', 'Write', 'NotebookEdit', 'MultiEdit']);
143
+
144
+ /**
145
+ * Claude Code — one JSONL file per session under ~/.claude/projects/<slug>/.
146
+ *
147
+ * The directory slug is a lossy encoding of the working directory (every path
148
+ * separator and literal dash both become '-'), so it cannot be decoded back
149
+ * reliably. Records carry the real `cwd`, so we read it from there and keep the
150
+ * slug only as a last resort.
151
+ */
152
+ const claudeCode = {
153
+ id: 'claude-code',
154
+ label: 'Claude Code',
155
+ root: () => path.join(os.homedir(), '.claude', 'projects'),
156
+
157
+ listFiles(root) {
158
+ let dirs;
159
+ try {
160
+ dirs = fs.readdirSync(root, { withFileTypes: true });
161
+ } catch (_) {
162
+ return [];
163
+ }
164
+ const files = [];
165
+ for (const dir of dirs) {
166
+ if (!dir.isDirectory()) continue;
167
+ const dirPath = path.join(root, dir.name);
168
+ let entries;
169
+ try {
170
+ entries = fs.readdirSync(dirPath);
171
+ } catch (_) {
172
+ continue;
173
+ }
174
+ // Deliberately one level deep. Nested `<session-id>/subagents/*.jsonl`
175
+ // files are subagent transcripts — delegated tool traffic containing no
176
+ // user speech, and on this machine they outnumber real sessions almost
177
+ // 1:1. Recursing would double the scan cost to harvest pure noise.
178
+ for (const name of entries) {
179
+ if (name.endsWith('.jsonl')) files.push(path.join(dirPath, name));
180
+ }
181
+ }
182
+ return files.sort();
183
+ },
184
+
185
+ parse(file) {
186
+ const records = readJsonl(file);
187
+ if (records.length === 0) return null;
188
+
189
+ const session = {
190
+ source: 'claude-code',
191
+ session_id: null,
192
+ title: null,
193
+ cwd: null,
194
+ git_branch: null,
195
+ started: null,
196
+ ended: null,
197
+ models: new Set(),
198
+ prompts: [],
199
+ files_touched: new Set(),
200
+ turns: 0,
201
+ };
202
+
203
+ for (const record of records) {
204
+ if (!record || typeof record !== 'object') continue;
205
+ if (record.sessionId && !session.session_id) session.session_id = record.sessionId;
206
+
207
+ // Claude Code names sessions itself. That title is a free, already-curated
208
+ // summary — far better than anything we could derive with regexes.
209
+ if (record.type === 'ai-title' && record.aiTitle) session.title = record.aiTitle;
210
+
211
+ if (record.cwd) session.cwd = record.cwd;
212
+ if (record.gitBranch) session.git_branch = record.gitBranch;
213
+ if (record.timestamp) {
214
+ if (!session.started || record.timestamp < session.started) session.started = record.timestamp;
215
+ if (!session.ended || record.timestamp > session.ended) session.ended = record.timestamp;
216
+ }
217
+
218
+ // Sidechain records are subagent traffic, not the user talking.
219
+ if (record.isSidechain || record.isMeta) continue;
220
+
221
+ if (record.type === 'user') {
222
+ const text = userTextFrom(record);
223
+ if (text) session.prompts.push(text);
224
+ }
225
+
226
+ if (record.type === 'assistant') {
227
+ session.turns++;
228
+ const model = record.message && record.message.model;
229
+ if (model) session.models.add(model);
230
+ const content = (record.message && record.message.content) || [];
231
+ if (Array.isArray(content)) {
232
+ for (const block of content) {
233
+ if (block && block.type === 'tool_use' && FILE_EDIT_TOOLS.has(block.name)) {
234
+ const target = block.input && (block.input.file_path || block.input.notebook_path);
235
+ if (target) session.files_touched.add(target);
236
+ }
237
+ }
238
+ }
239
+ }
240
+ }
241
+
242
+ if (!session.session_id) session.session_id = path.basename(file, '.jsonl');
243
+ if (!session.cwd) session.cwd = slugToPathGuess(path.basename(path.dirname(file)));
244
+ return session;
245
+ },
246
+ };
247
+
248
+ /**
249
+ * Best-effort recovery of a working directory from a Claude Code project slug.
250
+ *
251
+ * Lossy by nature — '-Users-me-code-my-app' could be '/Users/me/code/my-app' or
252
+ * '/Users/me/code/my/app'. Only used when no record carried a real `cwd`, and
253
+ * only to label the session, never to write anything.
254
+ */
255
+ function slugToPathGuess(slug) {
256
+ if (!slug || !slug.startsWith('-')) return slug || null;
257
+ return '/' + slug.slice(1).replace(/-/g, '/');
258
+ }
259
+
260
+ const SOURCES = { [claudeCode.id]: claudeCode };
261
+
262
+ /** Sources that actually have a history store on this machine. */
263
+ function availableSources() {
264
+ return Object.values(SOURCES)
265
+ .map((adapter) => {
266
+ const root = adapter.root();
267
+ const present = fs.existsSync(root);
268
+ return {
269
+ id: adapter.id,
270
+ label: adapter.label,
271
+ root,
272
+ available: present,
273
+ sessions: present ? adapter.listFiles(root).length : 0,
274
+ };
275
+ });
276
+ }
277
+
278
+ // --- Import state ---
279
+ //
280
+ // Import is incremental. Without a cursor, a second run re-offers every session
281
+ // and the agent writes the same memories again — the fastest way to poison a
282
+ // brain with duplicates.
283
+
284
+ function importStatePath(projectRoot) {
285
+ return path.join(getBrainDir(projectRoot), '.import', 'state.json');
286
+ }
287
+
288
+ function readImportState(projectRoot) {
289
+ try {
290
+ const parsed = JSON.parse(fs.readFileSync(importStatePath(projectRoot), 'utf-8'));
291
+ if (parsed && typeof parsed === 'object' && parsed.sources) return parsed;
292
+ } catch (_) {
293
+ // Missing or corrupt — a fresh cursor is always safe, it only means the
294
+ // next run re-offers sessions the agent can recognize as already-known.
295
+ }
296
+ return { version: 1, sources: {} };
297
+ }
298
+
299
+ function writeImportState(state, projectRoot) {
300
+ const file = importStatePath(projectRoot);
301
+ fs.mkdirSync(path.dirname(file), { recursive: true });
302
+ atomicWriteSync(file, JSON.stringify(state, null, 2) + '\n');
303
+ }
304
+
305
+ /**
306
+ * Record sessions as imported so later runs skip them.
307
+ *
308
+ * Ids are checked against the source's real sessions first. A silently accepted
309
+ * typo or truncated id is the worst possible failure here: the caller is told
310
+ * the session is done, the cursor never matches it, and the session is re-offered
311
+ * forever — so unknown ids are reported back rather than written.
312
+ *
313
+ * When the source's history is not enumerable — an unregistered source, or a
314
+ * store that isn't on this machine — there is nothing to check against, so ids
315
+ * are accepted as given rather than rejected on no evidence.
316
+ *
317
+ * @param {string} sourceId
318
+ * @param {string[]} sessionIds
319
+ * @param {string} [projectRoot] - Filesystem root whose .brain/ to use
320
+ * @param {Set<string>} [knownIds] - Override the enumerated id set (tests)
321
+ * @returns {{ added: number, total: number, unknown: string[] }}
322
+ */
323
+ function markImported(sourceId, sessionIds, projectRoot, knownIds) {
324
+ let known = knownIds || null;
325
+ if (!known) {
326
+ const adapter = SOURCES[sourceId];
327
+ if (adapter) {
328
+ const root = adapter.root();
329
+ if (fs.existsSync(root)) {
330
+ known = new Set(adapter.listFiles(root).map((f) => path.basename(f, '.jsonl')));
331
+ }
332
+ }
333
+ }
334
+
335
+ const state = readImportState(projectRoot);
336
+ if (!state.sources[sourceId]) state.sources[sourceId] = { imported: {} };
337
+ const now = new Date().toISOString();
338
+
339
+ const unknown = [];
340
+ let added = 0;
341
+ for (const id of sessionIds) {
342
+ if (known && !known.has(id)) { unknown.push(id); continue; }
343
+ if (!state.sources[sourceId].imported[id]) added++;
344
+ state.sources[sourceId].imported[id] = now;
345
+ }
346
+ writeImportState(state, projectRoot);
347
+ return {
348
+ added,
349
+ total: Object.keys(state.sources[sourceId].imported).length,
350
+ unknown,
351
+ };
352
+ }
353
+
354
+ // --- Digest ---
355
+
356
+ /** Deduplicate consecutive near-identical prompts (retries, resends). */
357
+ function dedupePrompts(prompts) {
358
+ const out = [];
359
+ const seen = new Set();
360
+ for (const prompt of prompts) {
361
+ const key = prompt.slice(0, 120).toLowerCase();
362
+ if (seen.has(key)) continue;
363
+ seen.add(key);
364
+ out.push(prompt);
365
+ }
366
+ return out;
367
+ }
368
+
369
+ /**
370
+ * Pick which prompts represent the session.
371
+ *
372
+ * The opening prompt states the goal and the closing ones state what it became,
373
+ * so both edges are kept and the middle is sampled. Taking the first N instead
374
+ * would systematically lose every outcome.
375
+ */
376
+ function selectPrompts(prompts, max) {
377
+ if (prompts.length <= max) return prompts;
378
+ const head = Math.ceil(max / 2);
379
+ const tail = max - head;
380
+ return [...prompts.slice(0, head), ...prompts.slice(prompts.length - tail)];
381
+ }
382
+
383
+ /**
384
+ * Round-robin candidates across projects, preserving recency within each.
385
+ *
386
+ * Sorting by recency alone lets one busy repo eat the entire budget — on a real
387
+ * 261-session history, one project took 32 of 38 slots and five others got a
388
+ * single session between them. Cold-start value comes from breadth (the agent
389
+ * learning that you work on N things, and what each one is) far more than from
390
+ * depth in whichever repo you happened to touch last week.
391
+ */
392
+ function interleaveByProject(candidates) {
393
+ const byProject = new Map();
394
+ for (const candidate of candidates) {
395
+ const key = candidate.project || '(unknown)';
396
+ if (!byProject.has(key)) byProject.set(key, []);
397
+ byProject.get(key).push(candidate);
398
+ }
399
+
400
+ // Projects with recent activity lead, so the first round is the work most
401
+ // likely to still be live.
402
+ const queues = [...byProject.values()];
403
+ const ordered = [];
404
+ for (let round = 0; ordered.length < candidates.length; round++) {
405
+ for (const queue of queues) {
406
+ if (round < queue.length) ordered.push(queue[round]);
407
+ }
408
+ }
409
+ return ordered;
410
+ }
411
+
412
+ /** Human-facing project label — the directory name is what users call a project. */
413
+ function projectNameOf(session) {
414
+ if (!session.cwd) return null;
415
+ return path.basename(session.cwd) || null;
416
+ }
417
+
418
+ /**
419
+ * Build a budget-bounded digest of candidate sessions.
420
+ *
421
+ * @param {Object} [opts]
422
+ * @param {string} [opts.source='claude-code'] - Source adapter id
423
+ * @param {string} [opts.project] - Only sessions whose project name matches
424
+ * @param {string} [opts.since] - ISO date; only sessions ended at/after it
425
+ * @param {number} [opts.limit] - Max sessions in the digest
426
+ * @param {boolean} [opts.all=false] - Include already-imported sessions
427
+ * @param {string} [opts.projectRoot] - Filesystem root whose .brain/ to use
428
+ * @returns {Object} digest payload
429
+ */
430
+ function harvest(opts = {}) {
431
+ const sourceId = opts.source || 'claude-code';
432
+ const adapter = SOURCES[sourceId];
433
+ if (!adapter) {
434
+ return { error: `Unknown source "${sourceId}" (available: ${Object.keys(SOURCES).join(', ')})` };
435
+ }
436
+
437
+ const limit = opts.limit || DEFAULTS.maxSessions;
438
+ const root = adapter.root();
439
+ if (!fs.existsSync(root)) {
440
+ return {
441
+ source: sourceId,
442
+ scanned: { files: 0, sessions: 0, skipped: 0, already_imported: 0 },
443
+ sessions: [],
444
+ note: `No ${adapter.label} history found at ${root}.`,
445
+ };
446
+ }
447
+
448
+ const state = readImportState(opts.projectRoot);
449
+ const done = (state.sources[sourceId] && state.sources[sourceId].imported) || {};
450
+
451
+ const files = adapter.listFiles(root);
452
+ const candidates = [];
453
+ // Kept apart on purpose: `skipped` means "held nothing worth reading",
454
+ // `filtered` means "excluded by your --project/--since scoping". Merging them
455
+ // would report a scoped run as though it had discarded hundreds of bad
456
+ // sessions, which is both alarming and false.
457
+ let skipped = 0;
458
+ let filtered = 0;
459
+ let alreadyImported = 0;
460
+
461
+ for (const file of files) {
462
+ const session = adapter.parse(file);
463
+ if (!session) { skipped++; continue; }
464
+
465
+ if (!opts.all && done[session.session_id]) { alreadyImported++; continue; }
466
+
467
+ const prompts = dedupePrompts(session.prompts);
468
+ if (prompts.length < MIN_PROMPTS) { skipped++; continue; }
469
+
470
+ const project = projectNameOf(session);
471
+ if (opts.project && project !== opts.project) { filtered++; continue; }
472
+ if (opts.since && session.ended && session.ended < opts.since) { filtered++; continue; }
473
+
474
+ candidates.push({ session, prompts, project });
475
+ }
476
+
477
+ // Most recent first: recent work is both likelier to matter and likelier to
478
+ // still be true. An import that runs out of budget should run out on the
479
+ // oldest sessions, not the newest.
480
+ candidates.sort((a, b) => String(b.session.ended || '').localeCompare(String(a.session.ended || '')));
481
+
482
+ const ordered = interleaveByProject(candidates);
483
+
484
+ const sessions = [];
485
+ let chars = 0;
486
+ let budgetTruncated = false;
487
+
488
+ for (const candidate of ordered) {
489
+ if (sessions.length >= limit) { budgetTruncated = true; break; }
490
+
491
+ const { session, prompts, project } = candidate;
492
+ const selected = selectPrompts(prompts, DEFAULTS.maxPromptsPerSession)
493
+ .map((p) => truncate(p, DEFAULTS.maxPromptChars));
494
+
495
+ const entry = {
496
+ session_id: session.session_id,
497
+ title: session.title || null,
498
+ project,
499
+ cwd: session.cwd,
500
+ git_branch: session.git_branch || null,
501
+ started: session.started,
502
+ ended: session.ended,
503
+ turns: session.turns,
504
+ prompt_count: prompts.length,
505
+ models: [...session.models],
506
+ files_touched: [...session.files_touched]
507
+ .map((f) => (session.cwd && f.startsWith(session.cwd) ? path.relative(session.cwd, f) : f))
508
+ .slice(0, 12),
509
+ prompts: selected,
510
+ };
511
+
512
+ const cost = JSON.stringify(entry).length;
513
+ if (chars + cost > DEFAULTS.maxDigestChars && sessions.length > 0) {
514
+ budgetTruncated = true;
515
+ break;
516
+ }
517
+ chars += cost;
518
+ sessions.push(entry);
519
+ }
520
+
521
+ return {
522
+ source: sourceId,
523
+ source_label: adapter.label,
524
+ scanned: {
525
+ files: files.length,
526
+ candidates: candidates.length,
527
+ returned: sessions.length,
528
+ skipped,
529
+ filtered,
530
+ already_imported: alreadyImported,
531
+ remaining: Math.max(0, candidates.length - sessions.length),
532
+ },
533
+ budget: {
534
+ max_sessions: limit,
535
+ max_digest_chars: DEFAULTS.maxDigestChars,
536
+ used_chars: chars,
537
+ token_estimate: Math.ceil(chars / 4),
538
+ truncated: budgetTruncated,
539
+ },
540
+ sessions,
541
+ };
542
+ }
543
+
544
+ module.exports = {
545
+ harvest,
546
+ availableSources,
547
+ markImported,
548
+ readImportState,
549
+ writeImportState,
550
+ importStatePath,
551
+ // Exported for tests
552
+ cleanPromptText,
553
+ userTextFrom,
554
+ selectPrompts,
555
+ dedupePrompts,
556
+ truncate,
557
+ slugToPathGuess,
558
+ readJsonl,
559
+ SOURCES,
560
+ DEFAULTS,
561
+ MIN_PROMPTS,
562
+ };