bigbrain-memory 1.1.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- bigbrain_memory-1.1.1/LICENSE +21 -0
- bigbrain_memory-1.1.1/PKG-INFO +420 -0
- bigbrain_memory-1.1.1/README.md +408 -0
- bigbrain_memory-1.1.1/bigbrain/__init__.py +3 -0
- bigbrain_memory-1.1.1/bigbrain/cli.py +250 -0
- bigbrain_memory-1.1.1/bigbrain/core.py +752 -0
- bigbrain_memory-1.1.1/bigbrain/install.py +271 -0
- bigbrain_memory-1.1.1/bigbrain/llm.py +93 -0
- bigbrain_memory-1.1.1/bigbrain/mcp_server.py +159 -0
- bigbrain_memory-1.1.1/bigbrain/schemas.py +191 -0
- bigbrain_memory-1.1.1/bigbrain/source.py +389 -0
- bigbrain_memory-1.1.1/bigbrain_memory.egg-info/PKG-INFO +420 -0
- bigbrain_memory-1.1.1/bigbrain_memory.egg-info/SOURCES.txt +19 -0
- bigbrain_memory-1.1.1/bigbrain_memory.egg-info/dependency_links.txt +1 -0
- bigbrain_memory-1.1.1/bigbrain_memory.egg-info/entry_points.txt +2 -0
- bigbrain_memory-1.1.1/bigbrain_memory.egg-info/requires.txt +1 -0
- bigbrain_memory-1.1.1/bigbrain_memory.egg-info/top_level.txt +1 -0
- bigbrain_memory-1.1.1/pyproject.toml +21 -0
- bigbrain_memory-1.1.1/setup.cfg +4 -0
- bigbrain_memory-1.1.1/tests/test_core.py +291 -0
- bigbrain_memory-1.1.1/tests/test_source.py +400 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 BigBrain Contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,420 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: bigbrain-memory
|
|
3
|
+
Version: 1.1.1
|
|
4
|
+
Summary: Deterministic, filesystem-first memory and context-management engine for LLM coding agents.
|
|
5
|
+
Author: BigBrain Contributors
|
|
6
|
+
License: MIT
|
|
7
|
+
Requires-Python: >=3.9
|
|
8
|
+
Description-Content-Type: text/markdown
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Requires-Dist: PyYAML>=6.0
|
|
11
|
+
Dynamic: license-file
|
|
12
|
+
|
|
13
|
+
# BigBrain
|
|
14
|
+
|
|
15
|
+
Deterministic, filesystem-first memory and context-management engine for LLM
|
|
16
|
+
coding agents.
|
|
17
|
+
|
|
18
|
+
Version 1.1 adds a local, rebuildable **source index** alongside the existing
|
|
19
|
+
Markdown memory. Agents can browse, search and read current source through
|
|
20
|
+
either the CLI or MCP. Existing memory commands and formats are unchanged.
|
|
21
|
+
|
|
22
|
+
Version 1.1.1 adds live CLI progress and indexed chunk cleanup for large
|
|
23
|
+
repositories. Existing 1.1 caches upgrade their derived metadata automatically;
|
|
24
|
+
no memory migration or manual cache deletion is required.
|
|
25
|
+
|
|
26
|
+
## New installation
|
|
27
|
+
|
|
28
|
+
Requirements: Python 3.9+, Git, and Python's SQLite with FTS5 support (included
|
|
29
|
+
in the standard Windows/macOS Python installers and common Linux packages).
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
python -m pip install "git+https://github.com/corbybender/bigbrain-memory.git"
|
|
33
|
+
cd /path/to/your/project
|
|
34
|
+
bigbrain install
|
|
35
|
+
bigbrain source-status
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Run these commands in the project you want BigBrain to understand. Memory
|
|
39
|
+
works without Git; source indexing requires a Git working tree. The first
|
|
40
|
+
source command creates the index automatically, and subsequent source commands
|
|
41
|
+
refresh it automatically. No enable flag, API key, scheduled task or watcher
|
|
42
|
+
setup is required. The status report should show `complete: true` and an
|
|
43
|
+
appropriate `indexed_files` count for the eligible files in your project.
|
|
44
|
+
|
|
45
|
+
## Upgrade an existing project to 1.1.1
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
python -m pip install --upgrade "git+https://github.com/corbybender/bigbrain-memory.git"
|
|
49
|
+
cd /path/to/your/project
|
|
50
|
+
bigbrain install --force
|
|
51
|
+
bigbrain source-status
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Use the Python environment that supplies your `bigbrain` command. Repeat
|
|
55
|
+
`bigbrain install --force` in each existing project to refresh its agent
|
|
56
|
+
instructions, MCP registration and tool schema. Existing memory and
|
|
57
|
+
configuration settings are preserved; no memory migration or manual source
|
|
58
|
+
index creation is needed. Restart/reconnect the MCP host afterward so it
|
|
59
|
+
discovers the five new source tools. If you move or clone a project, rerun
|
|
60
|
+
`bigbrain install --force` there to update its absolute MCP project root.
|
|
61
|
+
|
|
62
|
+
See [setup and troubleshooting](docs/setup.md) for Windows interpreter checks,
|
|
63
|
+
verification, automatic update behavior and recovery.
|
|
64
|
+
|
|
65
|
+
## Source context
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
bigbrain source-status
|
|
69
|
+
bigbrain source-files --prefix bigbrain/ --limit 20
|
|
70
|
+
bigbrain source-search "source cache recovery" --top 5
|
|
71
|
+
bigbrain source-read bigbrain/source.py --start-line 1 --max-lines 60
|
|
72
|
+
bigbrain source-refresh --rebuild
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Use `--path <project-root>` before or after the command to target another
|
|
76
|
+
project. Git must be installed and the project must be inside a Git working
|
|
77
|
+
tree; an explicitly selected subdirectory indexes only that subtree. No Git
|
|
78
|
+
commit, model, API key, vector service or background daemon is needed.
|
|
79
|
+
|
|
80
|
+
Source tools discover tracked and untracked files, honoring Git ignore rules
|
|
81
|
+
(including rules matching tracked files). They reconcile the working tree
|
|
82
|
+
before every query, so edits, renames, deletions, ignore changes and branch
|
|
83
|
+
switches are reflected without manual refresh. Eligible files are hashed on
|
|
84
|
+
every scan; unchanged files reuse their indexed chunks. This detects edits
|
|
85
|
+
even when file size and modification time are unchanged.
|
|
86
|
+
|
|
87
|
+
CLI source commands print discovery, scanning, indexing and commit progress
|
|
88
|
+
to stderr; stdout remains JSON for scripts. Large first builds can take time.
|
|
89
|
+
Ctrl+C cancels the operation and rolls back uncommitted source-cache changes
|
|
90
|
+
without changing memory. MCP calls do not print CLI progress into their
|
|
91
|
+
protocol stream; use the CLI for the initial build of a large repository.
|
|
92
|
+
|
|
93
|
+
The source index is disposable SQLite/FTS5 data under `.bigbrain/cache/`,
|
|
94
|
+
automatically excluded from Git by a cache-local ignore file. Markdown memory
|
|
95
|
+
remains authoritative for decisions and intent; source results provide current
|
|
96
|
+
implementation evidence. Source operations never rewrite memory. Rebuild a
|
|
97
|
+
damaged or incompatible cache with `source-refresh --rebuild`.
|
|
98
|
+
|
|
99
|
+
Results include a scope/freshness report, paths, SHA-256 hashes, and bounded
|
|
100
|
+
excerpts or line ranges. Search returns the best chunk per distinct file, with
|
|
101
|
+
deterministic lexical BM25 ranking and camelCase/snake_case token splitting.
|
|
102
|
+
Reads are limited to 200 lines and 16,000 characters; search excerpts to 1,600
|
|
103
|
+
characters per file and 50 files per request. Inventory is paginated up to 200
|
|
104
|
+
files per request. A hash identifies the bytes read during reconciliation;
|
|
105
|
+
files can change after that scan. Inventory pages can change between calls.
|
|
106
|
+
|
|
107
|
+
By default, indexing skips non-UTF-8/binary files, files larger than 512 KiB,
|
|
108
|
+
symlinks and Windows reparse points, dependency/cache directories, common
|
|
109
|
+
credential filenames/extensions, and `dist`, `build`, `generated`, `coverage`
|
|
110
|
+
and `vendor`. These filename exclusions are **not a secret scanner**; add
|
|
111
|
+
project-specific sensitive paths to Git ignores or source exclusions. Text
|
|
112
|
+
documentation and configuration are eligible alongside code.
|
|
113
|
+
|
|
114
|
+
Optional `.bigbrain/config.yml` settings:
|
|
115
|
+
|
|
116
|
+
```yaml
|
|
117
|
+
source:
|
|
118
|
+
max_file_bytes: 524288 # 1 through 10485760
|
|
119
|
+
exclude: # replaces the default generated-output exclusions
|
|
120
|
+
- dist
|
|
121
|
+
- build
|
|
122
|
+
- generated
|
|
123
|
+
- coverage
|
|
124
|
+
- vendor
|
|
125
|
+
- private-settings.json
|
|
126
|
+
- docs/internal/*
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Patterns match a full project-relative path or any path component, using
|
|
130
|
+
case-sensitive shell wildcards (`*` can span `/` in full-path matching).
|
|
131
|
+
Git ignores, credential and dependency/cache exclusions, and link restrictions
|
|
132
|
+
still apply when `exclude: []` is configured. Files skipped by policy are
|
|
133
|
+
counted by reason. Read failures are counted (with up to 50 details) and mark coverage
|
|
134
|
+
`complete: false`; CLI commands exit 1 for incomplete coverage. MCP clients
|
|
135
|
+
must inspect this report before treating an empty result as evidence of absence.
|
|
136
|
+
|
|
137
|
+
The MCP tools are `source_status`, `list_source_files`,
|
|
138
|
+
`refresh_source_index`, `search_source`, and `read_source`. Use memory tools
|
|
139
|
+
to retrieve rationale, then source tools to check its current implementation.
|
|
140
|
+
Run `bigbrain install --force` after upgrading to refresh tool schemas and
|
|
141
|
+
agent instructions; existing configuration settings are preserved. Restart/reconnect the MCP host
|
|
142
|
+
to discover new tools. New MCP registrations use an absolute project root,
|
|
143
|
+
independent of the host's working directory.
|
|
144
|
+
|
|
145
|
+
This release uses query-time reconciliation rather than a watcher. It hashes
|
|
146
|
+
all eligible files per request, so latency grows with repository size. Source
|
|
147
|
+
indexing requires SQLite FTS5 and Git. Symbol resolution, call graphs,
|
|
148
|
+
cross-language dependency analysis, embeddings and automatic memory/source
|
|
149
|
+
contradiction resolution are not implemented.
|
|
150
|
+
|
|
151
|
+
### Source validation
|
|
152
|
+
|
|
153
|
+
```bash
|
|
154
|
+
python tests/test_core.py
|
|
155
|
+
python -m unittest discover -s tests -p test_source.py -v
|
|
156
|
+
python benchmarks/source_retrieval.py
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
CI runs these checks on Windows and Linux with Python 3.9 and 3.13. The benchmark
|
|
160
|
+
uses six fixed keyword questions over an isolated copy of BigBrain's package
|
|
161
|
+
source and reports recall@5, indexing/query times, response sizes and cache size.
|
|
162
|
+
It is a reproducible smoke benchmark, not proof of improved agent task success
|
|
163
|
+
or performance on large repositories.
|
|
164
|
+
|
|
165
|
+
## TL;DR
|
|
166
|
+
|
|
167
|
+
- **What it is:** persistent project memory as plain Markdown files under
|
|
168
|
+
`.bigbrain/` — no vector DB, no embeddings, human-readable, git-friendly.
|
|
169
|
+
- **Why:** so an AI coding agent can carry decisions, architecture notes, and
|
|
170
|
+
task state across sessions instead of re-deriving them (or forgetting them)
|
|
171
|
+
every time.
|
|
172
|
+
- **Works with:** any AI coding tool — Claude Code, Cursor, Windsurf, Gemini
|
|
173
|
+
CLI, Copilot, Aider, whatever — via plain shell commands, with a bonus MCP
|
|
174
|
+
server for hosts that support native tool-calling.
|
|
175
|
+
|
|
176
|
+
### Setup
|
|
177
|
+
|
|
178
|
+
```bash
|
|
179
|
+
python -m pip install git+https://github.com/corbybender/bigbrain-memory.git
|
|
180
|
+
cd /path/to/your/project
|
|
181
|
+
bigbrain install
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
That's it. `bigbrain install` scaffolds `.bigbrain/`, wires instructions into
|
|
185
|
+
whatever AI-instructions files your tools use (`AGENTS.md`, `CLAUDE.md`,
|
|
186
|
+
`.cursor/rules/`, etc.), registers an MCP server for hosts that support it,
|
|
187
|
+
and sets up auto-compile hooks (git pre-commit + Claude Code session hooks)
|
|
188
|
+
so nobody has to remember to run anything. Re-running it later is safe —
|
|
189
|
+
it's idempotent.
|
|
190
|
+
|
|
191
|
+
Verify it worked:
|
|
192
|
+
|
|
193
|
+
```bash
|
|
194
|
+
bigbrain status # should report initialized: true
|
|
195
|
+
bigbrain index # should print a Markdown table of memory files
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
---
|
|
199
|
+
|
|
200
|
+
## Detailed guide
|
|
201
|
+
|
|
202
|
+
See [`plan.md`](plan.md) for the original design spec this implements.
|
|
203
|
+
|
|
204
|
+
### Core idea
|
|
205
|
+
|
|
206
|
+
Information lives in standard Markdown files with YAML frontmatter. An agent
|
|
207
|
+
reads a lightweight `INDEX.md` catalog at session start instead of ingesting
|
|
208
|
+
the whole memory directory, and pulls specific files on demand. New facts get
|
|
209
|
+
appended to an append-only transaction log (`log.md`) during a session, then
|
|
210
|
+
reconciled into the real memory files by a compile step — either a free
|
|
211
|
+
deterministic merge, or an optional LLM-assisted pass that actually dedupes
|
|
212
|
+
and resolves contradictions.
|
|
213
|
+
|
|
214
|
+
### Install into a repo
|
|
215
|
+
|
|
216
|
+
```bash
|
|
217
|
+
python -m pip install git+https://github.com/corbybender/bigbrain-memory.git
|
|
218
|
+
cd /path/to/your/project
|
|
219
|
+
bigbrain install
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
`bigbrain install`:
|
|
223
|
+
|
|
224
|
+
1. Scaffolds `.bigbrain/` (index, log, default memory files) at the repo root.
|
|
225
|
+
2. Writes `.bigbrain/tools.json` — the raw tool schema, for hosts that do
|
|
226
|
+
custom function-calling.
|
|
227
|
+
3. Appends a marked BigBrain instructions block to every well-known
|
|
228
|
+
agent-instructions file it finds or creates: `AGENTS.md`, `CLAUDE.md`,
|
|
229
|
+
`GEMINI.md`, `.windsurfrules`, `.github/copilot-instructions.md`, and a
|
|
230
|
+
Cursor rule at `.cursor/rules/bigbrain.mdc`. That block tells the agent
|
|
231
|
+
how to drive BigBrain through the `bigbrain` shell command — which needs
|
|
232
|
+
no protocol support at all, so it works with literally any agent that can
|
|
233
|
+
run a shell command.
|
|
234
|
+
4. Registers a BigBrain MCP server in `.mcp.json` and `.cursor/mcp.json` for
|
|
235
|
+
hosts with native MCP tool-calling (Claude Code, Cursor, ...), so those
|
|
236
|
+
hosts get real tool calls instead of shelling out.
|
|
237
|
+
5. Wires **auto-compile**, so nobody has to remember to run it:
|
|
238
|
+
- a `.git/hooks/pre-commit` hook that compiles before every commit (any
|
|
239
|
+
host, no-op in ~10ms if there's nothing pending), and
|
|
240
|
+
- for Claude Code specifically, `SessionStart` / `SessionEnd` /
|
|
241
|
+
`PreCompact` hooks in `.claude/settings.json` — `SessionStart` also
|
|
242
|
+
auto-injects `INDEX.md` into context, so the agent never has to
|
|
243
|
+
remember to read it either.
|
|
244
|
+
6. Drops `.bigbrain/config.yml` with commented defaults for the LLM CLI
|
|
245
|
+
override, archive rotation size and optional source settings. Existing
|
|
246
|
+
configuration is preserved, including with `--force`.
|
|
247
|
+
|
|
248
|
+
Source indexing initializes lazily on the first source command or MCP source
|
|
249
|
+
tool call; `install` does not crawl the repository. Every source query then
|
|
250
|
+
reconciles changes automatically. Memory compile hooks only compile memory;
|
|
251
|
+
they do not run source indexing in the background.
|
|
252
|
+
|
|
253
|
+
Re-running `bigbrain install` is idempotent (existing hooks/files aren't
|
|
254
|
+
duplicated). Pass `--force` to refresh the wiring blocks after upgrading
|
|
255
|
+
BigBrain. If a target file already has other content (an existing pre-commit
|
|
256
|
+
hook, an existing `AGENTS.md`), BigBrain appends a marked block rather than
|
|
257
|
+
overwriting it.
|
|
258
|
+
|
|
259
|
+
### Directory layout
|
|
260
|
+
|
|
261
|
+
```text
|
|
262
|
+
.bigbrain/
|
|
263
|
+
├── INDEX.md # compiled catalog, ingested at session start
|
|
264
|
+
├── log.md # append-only transaction log (pending entries)
|
|
265
|
+
├── log.archive.md # compiled/processed log entries (rotates when it gets big)
|
|
266
|
+
├── log.archive.<ts>.md # rotated-out archive segments, once log.archive.md exceeds archive_max_bytes
|
|
267
|
+
├── config.yml # llm_command override, archive_max_bytes
|
|
268
|
+
├── tools.json # raw tool schemas for function-calling hosts
|
|
269
|
+
├── cache/ # disposable, Git-ignored; created by first source query
|
|
270
|
+
│ ├── .gitignore # excludes this cache directory's contents
|
|
271
|
+
│ └── source.sqlite3 # source inventory and FTS5 search chunks
|
|
272
|
+
└── memory/
|
|
273
|
+
├── system_invariants.md
|
|
274
|
+
├── architecture.md
|
|
275
|
+
├── active_tasks.md
|
|
276
|
+
├── decisions.md
|
|
277
|
+
└── domain_knowledge.md
|
|
278
|
+
└── ... # compile can create new memory files on demand
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
### CLI
|
|
282
|
+
|
|
283
|
+
```bash
|
|
284
|
+
bigbrain init # scaffold .bigbrain/ only
|
|
285
|
+
bigbrain install # scaffold + wire into AI tooling (recommended)
|
|
286
|
+
bigbrain index # print INDEX.md
|
|
287
|
+
bigbrain read <file_id> # print one memory file
|
|
288
|
+
bigbrain grep <term> # exact-substring fallback search when you don't know the file_id
|
|
289
|
+
bigbrain search <query> # ranked fuzzy/word-overlap search when you don't know the exact wording either
|
|
290
|
+
bigbrain source-status # reconcile source and report scope/freshness/errors
|
|
291
|
+
bigbrain source-files # browse eligible source inventory (paginated)
|
|
292
|
+
bigbrain source-search "query" # reconcile and search current source/docs
|
|
293
|
+
bigbrain source-read <path> # reconcile and read bounded source lines
|
|
294
|
+
bigbrain source-refresh --rebuild # recover/rebuild disposable source cache only
|
|
295
|
+
bigbrain doctor [--fix] # validate .bigbrain/ (stale locks, bad frontmatter, stale index); repair with --fix
|
|
296
|
+
bigbrain log --type DECISION --target decisions \
|
|
297
|
+
--summary "Chose SQLite for log indexing" \
|
|
298
|
+
--details "ACID writes matter under concurrent CLI runs."
|
|
299
|
+
bigbrain compile --reason "session end" # reconcile log.md -> memory/*.md, rebuild INDEX.md
|
|
300
|
+
bigbrain compile --reason "session end" --llm # same, but LLM-assisted dedup/contradiction-resolution
|
|
301
|
+
bigbrain status # JSON snapshot of engine state
|
|
302
|
+
bigbrain serve-mcp # run the MCP stdio server
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
`--type` must be one of `DECISION`, `TASK_UPDATE`, `LEARNING`, `INVARIANT`.
|
|
306
|
+
|
|
307
|
+
### Deterministic vs. `--llm` compile
|
|
308
|
+
|
|
309
|
+
Plain `bigbrain compile` never calls a model: pending log entries are grouped
|
|
310
|
+
by target file and appended under a dated "Log Reconciliation" section. It's
|
|
311
|
+
free, instant, and can't fail in a surprising way — but memory files only
|
|
312
|
+
ever grow; nothing gets deduped or rewritten to resolve contradictions.
|
|
313
|
+
|
|
314
|
+
`bigbrain compile --llm` pipes each target file + its pending entries to a
|
|
315
|
+
configured LLM CLI and asks it to properly rewrite the file: merge related
|
|
316
|
+
facts, drop obsolete statements, and resolve contradictions in favor of the
|
|
317
|
+
newer entry — matching the reconciliation behavior in the original design
|
|
318
|
+
(`plan.md` §4, Phase 3). It needs a local LLM CLI:
|
|
319
|
+
|
|
320
|
+
- auto-detected from PATH (tries `claude -p`, then `llm`), or
|
|
321
|
+
- pinned explicitly in `.bigbrain/config.yml`:
|
|
322
|
+
```yaml
|
|
323
|
+
llm_command: "claude -p"
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
`--llm` never risks your data: if the CLI is missing, times out, errors, or
|
|
327
|
+
returns output that doesn't parse as a valid memory file (frontmatter with an
|
|
328
|
+
`id` + non-empty body), that *one file* silently falls back to the
|
|
329
|
+
deterministic merge and a warning is printed — compile always succeeds.
|
|
330
|
+
|
|
331
|
+
### Archive rotation
|
|
332
|
+
|
|
333
|
+
`log.archive.md` accumulates every compiled log entry. Once it exceeds
|
|
334
|
+
`archive_max_bytes` (default 2MB, configurable in `.bigbrain/config.yml`), the
|
|
335
|
+
next compile rotates it to `log.archive.<timestamp>.md` and starts fresh, so
|
|
336
|
+
it doesn't grow without bound.
|
|
337
|
+
|
|
338
|
+
### `grep` vs `search`
|
|
339
|
+
|
|
340
|
+
- `bigbrain grep <term>` — exact, case-insensitive substring match. Use when
|
|
341
|
+
you know the precise word/name/quote.
|
|
342
|
+
- `bigbrain search <query>` — deterministic word-overlap ranking across every
|
|
343
|
+
memory file's body/title/tags, with a phrase-match bonus. Use when you know
|
|
344
|
+
roughly what you're looking for but not the exact wording or which file it's
|
|
345
|
+
in. This is **not** embeddings/vector search — BigBrain's core principle is
|
|
346
|
+
no opaque embeddings — just explainable keyword scoring, good enough to
|
|
347
|
+
replace "guess the file_id" at the scale a flat-file memory store is meant
|
|
348
|
+
for. If your project outgrows this, that's a signal to split memory more,
|
|
349
|
+
not to bolt on a vector DB.
|
|
350
|
+
|
|
351
|
+
### Concurrency
|
|
352
|
+
|
|
353
|
+
Every write path (`log`, `compile`) takes the same advisory lock
|
|
354
|
+
(`.bigbrain/.lock`), so two agent sessions (or an agent and the pre-commit
|
|
355
|
+
hook) writing at the same time serialize instead of interleaving/corrupting
|
|
356
|
+
files. A lock older than 120s is treated as abandoned and reclaimed
|
|
357
|
+
automatically; `bigbrain status` shows the current lock's pid and age if one
|
|
358
|
+
is held, and `bigbrain doctor --fix` will clear a stuck one manually.
|
|
359
|
+
|
|
360
|
+
### `doctor` (validation, repair, and schema migration)
|
|
361
|
+
|
|
362
|
+
`bigbrain doctor` checks for: a stale lock file, memory files missing
|
|
363
|
+
required frontmatter (`id`, `domain`, `last_updated`), duplicate `id`s across
|
|
364
|
+
files, a `log.md` containing content that doesn't match the expected entry
|
|
365
|
+
format (e.g. from hand-editing), an invalid `config.yml`, and a stale
|
|
366
|
+
`INDEX.md`. Run it plain to see what's wrong (exits 1 if anything is); run
|
|
367
|
+
`--fix` to repair everything that's safely auto-fixable (exits 0 once clean —
|
|
368
|
+
a fixed issue is never also reported as still-broken). The one thing it never
|
|
369
|
+
auto-fixes is malformed `log.md` content, since guessing what a corrupted
|
|
370
|
+
entry meant risks losing data — that always needs a human look.
|
|
371
|
+
|
|
372
|
+
Each memory file also carries a `schema_version` field; `doctor` stamps files
|
|
373
|
+
onto the current version. There's only ever been one version so far, but this
|
|
374
|
+
is the hook a future BigBrain release would register a real migration
|
|
375
|
+
against — `doctor --fix` is the migration command.
|
|
376
|
+
|
|
377
|
+
### How an agent is expected to use it
|
|
378
|
+
|
|
379
|
+
1. At session start, read `.bigbrain/INDEX.md` (`bigbrain index`) instead of
|
|
380
|
+
guessing at project structure or past decisions.
|
|
381
|
+
2. Pull a specific memory file on demand (`bigbrain read architecture`) when
|
|
382
|
+
it needs deep context on a domain.
|
|
383
|
+
For current implementation, browse with `list_source_files`, search with
|
|
384
|
+
`search_source`, then inspect with `read_source` (or their `source-*` CLI
|
|
385
|
+
equivalents). Those tools create/refresh the source index automatically.
|
|
386
|
+
Check the coverage report and cite source paths/lines; use memory for intent
|
|
387
|
+
and decisions rather than assuming it describes unchanged code.
|
|
388
|
+
3. Log non-trivial decisions, task-state changes, learnings, and invariants
|
|
389
|
+
as they happen (`bigbrain log ...`), rather than trying to remember them.
|
|
390
|
+
4. Trigger compilation at milestones or session end (`bigbrain compile`) to
|
|
391
|
+
reconcile the log into `memory/*.md` and rebuild `INDEX.md`.
|
|
392
|
+
|
|
393
|
+
### MCP server
|
|
394
|
+
|
|
395
|
+
For hosts with native MCP tool-calling, `bigbrain serve-mcp` exposes six memory
|
|
396
|
+
tools and five source tools matching `bigbrain/schemas.py` / `.bigbrain/tools.json`:
|
|
397
|
+
`read_index`, `read_memory_file`, `append_transaction_log`, `compile_memory`
|
|
398
|
+
(with an optional `use_llm` argument), `search_memory` (ranked fuzzy
|
|
399
|
+
discovery), and `grep_memory` (exact substring). Source tools are
|
|
400
|
+
`source_status`, `list_source_files`, `refresh_source_index`, `search_source`
|
|
401
|
+
and `read_source`. It speaks newline-delimited
|
|
402
|
+
JSON-RPC 2.0 over stdio — no MCP SDK dependency.
|
|
403
|
+
|
|
404
|
+
### Library usage
|
|
405
|
+
|
|
406
|
+
```python
|
|
407
|
+
from pathlib import Path
|
|
408
|
+
from bigbrain import core
|
|
409
|
+
|
|
410
|
+
core.init_engine(Path("."))
|
|
411
|
+
core.append_log(Path("."), "LEARNING", "domain_knowledge", "Payments API v2", "Uses cursor pagination.")
|
|
412
|
+
core.compile_memory(Path("."), reason="milestone")
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
### Tests
|
|
416
|
+
|
|
417
|
+
```bash
|
|
418
|
+
python -m pip install -e .
|
|
419
|
+
python tests/test_core.py
|
|
420
|
+
```
|